Skip the message loop when an editor is opening

This is a bit more restrictive than the old approach that only skipped
when `effEditOpen()` got called after `effEditGetRect()`. Not sure why
the Melda plugins block indefinitely on the message loop without this
now.

src/wine-host/bridges/vst2.h

@@ -37,13 +37,6 @@
 #include "../editor.h"
 #include "../utils.h"
 
-/**
- * A marker struct to indicate that the editor is about to be opened.
- *
- * @see Vst2Bridge::editor
- */
-struct EditorOpening {};
-
 /**
  * This hosts a Windows VST2 plugin, forwards messages sent by the Linux VST
  * plugin and provides host callback function for the plugin to talk back.
@@ -79,6 +72,17 @@ class Vst2Bridge {
      */
     Vst2Bridge(std::string plugin_dll_path, std::string socket_endpoint_path);
 
+    /**
+     * Returns true if the message loop should be skipped. This happens when the
+     * editor is in the process of being opened. In VST hosts on Windows
+     * `effEditOpen()` and `effEditGetRect()` will always be called in sequence,
+     * but in our approach there will be an opportunity to handle events in
+     * between these two calls. Most plugins will handle this just fine, but
+     * some plugins end up blocking indefinitely while waiting for the other
+     * function to be called, hence why this function is needed.
+     */
+    bool should_skip_message_loop();
+
     /**
      * Handle events on the main thread until the plugin quits. This can't be
      * done on another thread since some plugins (e.g. Melda) expect certain
@@ -178,7 +182,9 @@ class Vst2Bridge {
 
     /**
      * Run the message loop for this plugin and potentially also for other
-     * plugins. This is called by both versions of `handle_dispatch()`.
+     * plugins. This is only used in `handle_dispatch_single()`, as this will be
+     * run on a timer when using plugin groups. The caller should first check
+     * whether the event loop can be run through `should_skip_message_loop()`.
      *
      * Because of the way the Win32 API works we have to process events on the
      * same thread as the one the window was created on, and that thread is the
@@ -189,7 +195,13 @@ class Vst2Bridge {
      * because of incorrect assumptions made by the plugin. See the dostring for
      * `Vst2Bridge::editor` for more information.
      */
-    void pump_message_loop();
+    void handle_win32_events();
+
+    /**
+     * Handle X11 events for the editor window if it is open. This can be run
+     * safely from any thread.
+     */
+    void handle_x11_events();
 
     /**
      * The shared library handle of the VST plugin. I sadly could not get
@@ -280,23 +292,17 @@ class Vst2Bridge {
      * Wine window, and embedding that Wine window into a window provided by the
      * host. Should be empty when the editor is not open.
      *
-     * This field can have three possible states:
+     * There is some special behavior with regards to message handling when the
+     * editor is in the process of being opened, see
+     * `should_postpone_message_loop()`.
      *
-     * - `std::nullopt` when the editor is closed.
-     * - An `Editor` object when the editor is open.
-     * - `EditorOpening` when the editor is not yet open, but the host has
-     *   already called `effEditGetRect()` and is about to call `effEditOpen()`.
-     *   This is needed because there is a race condition in some bugs that
-     *   cause them to crash or enter an infinite Win32 message loop when
-     *   `effEditGetRect()` gets dispatched and we then enter the message loop
-     *   loop before `effEditOpen()` gets called. Most plugins will handle this
-     *   just fine, but a select few plugins make the assumption that the editor
-     *   is already open once `effEditGetRect()` has been called, even if
-     *   `effEditOpen` has not yet been dispatched. VST hsots on Windows will
-     *   call these two events in sequence, so the bug would never occur there.
-     *   To work around this we'll use this third state to temporarily stop
-     *   processing Windows events in the one or two ticks between these two
-     *   events.
+     * @see should_postpone_message_loop
      */
-    std::variant<std::monostate, Editor, EditorOpening> editor;
+    std::optional<Editor> editor;
+
+    /**
+     * Keeps track of when the editor is being opened during the two-phase
+     * process of the host calling `effEditOpen()` and `effEditGetRect()`.
+     */
+    bool editor_is_opening = false;
 };