Add option to work around bugs in REAPER/Renoise

This is not ideal since it requires the user to know about this option
and to create a config file, but I think it's the best we can do without
compromising on yabridge's transparency and 'zero hacks' philosophy.

See #29 and #32.

CHANGELOG.md

@@ -8,6 +8,14 @@ Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+### Added
+
+- Added an option to work around implementation bugs in _REAPER_ and _Renoise_
+  where the hosts can freeze when a plugin calls a certain function while the
+  editor is refreshing, see
+  [#29](https://github.com/robbert-vdh/yabridge/issues/29) and
+  [#32](https://github.com/robbert-vdh/yabridge/issues/32).
+
 ### Changed
 
 - Don't print calls to `effIdle()` when `YABRIDGE_DEBUG_LEVEL` is set to 1.
@@ -37,8 +45,8 @@ Versioning](https://semver.org/spec/v2.0.0.html).
   embedding. Right now the only known plugins that may need this are
   _PSPaudioware_ plugins with expandable GUIs such as E27. The behaviour can be
   enabled on a per-plugin basis in the plugin configuration. See the
-  [readme](https://github.com/robbert-vdh/yabridge#editor-hosting-modes) for
+  [readme](https://github.com/robbert-vdh/yabridge#miscellaneous-fixes-and-workarounds)
-  more details.
+  for more details.
 
 ### Changed
 

README.md

@@ -23,7 +23,7 @@ compatibility while also staying easy to debug and maintain.
   - [Wine prefixes](#wine-prefixes)
   - [Configuration](#configuration)
     - [Plugin groups](#plugin-groups)
-    - [Editor hosting modes](#editor-hosting-modes)
+    - [Miscellaneous fixes and workarounds](#miscellaneous-fixes-and-workarounds)
     - [Example](#example)
 - [Troubleshooting common issues](#troubleshooting-common-issues)
 - [Performance tuning](#performance-tuning)
@@ -41,8 +41,8 @@ Yabridge has been tested under the following VST hosts using Wine Staging 5.9:
 - Carla 2.1
 - Ardour 6.2
 - Mixbus 6.0.702
-- REAPER 6.09
+- REAPER 6.09[\*](#runtime-dependencies-and-known-issues)
-- Renoise 3.2.1
+- Renoise 3.2.1[\*](#runtime-dependencies-and-known-issues)
 
 Please let me know if there are any issues with other VST hosts.
 
@@ -224,16 +224,34 @@ process. Of course, plugin groups with the same name but in different Wine
 prefixes and with different architectures will be run independently of each
 other. See below for an [example](#example) of how these groups can be set up.
 
-#### Editor hosting modes
+#### Miscellaneous fixes and workarounds
 
-The way yabridge embeds editor windows will work for most plugins. There is a
+Because Linux VST hosts are typically not tested using Windows VST plugins and
-second embedding mode available that adds yet another layer of embedding. This
+because some Windows VST plugins make incorrect assumptions about the host or
-can be enabled by setting the `editor_double_embed` option to `true`. At the
+the environment, you may run into implementation issues when combining certain
-moment the only known plugins that need this are _PSPaudioware_ plugins with
+hosts and VST plugins. This section contains a few options you can use to work
-expandable GUIs such as E27, as those plugins will otherwise draw in the wrong
+around these issues. The [known issues](#runtime-dependencies-and-known-issues)
-location after the GUI has been expanded. This setting may be replaced in the
+section contains more information on when these options might be necessary.
-future if we can come up with a better solution. See below for an
+Yabridge will show which of these options are active on startup as part of the
-[example](#example) of how to set this up.
+initialization message.
+
+- Both _REAPER_ and _Renoise_ can freeze when the plugin uses the
+  `audioMasterUpdateDisplay()` function while the host is updating the editor
+  window. As a temporary workaround until this is fixed you can set the
+  `hack_reaper_update_display` option to `true`. If you set this option for the
+  `["*"]` pattern like in the example below, then this will be applied to all
+  yabridge `.so` files in the directory of the `yabridge.toml` files and all
+  directories below it. If you have added any other patterns to the
+  `yabridge.toml` file you'll also have to add the setting there since yabridge
+  will only read settings from the first matching pattern. See the example below
+  for more clarification.
+- The way yabridge embeds editor windows will work for most plugins. There is a
+  second embedding mode available that adds yet another layer of embedding. This
+  can be enabled by setting the `editor_double_embed` option to `true`. At the
+  moment the only known plugins that need this are _PSPaudioware_ plugins with
+  expandable GUIs such as E27 as those plugins will otherwise draw in the wrong
+  location after the GUI has been expanded. This setting may be replaced in the
+  future if we can come up with a better solution.
 
 #### Example
 
@@ -254,6 +272,7 @@ editor_double_embed = true
 # Matches an entire directory and all files inside it, make sure to not include
 # a trailing slash
 ["ToneBoosters"]
+hack_reaper_update_display = true
 group = "toneboosters"
 
 # Simple glob patterns can be used to avoid a unneeded repitition
@@ -277,6 +296,14 @@ group = "This will be ignored!"
 # kinds of weird issues.
 # ["*"]
 # group = "all"
+
+# This will apply a workaround for an implementation issue in REAPER and Renoise
+# to all plugins in the current directory _that are not already matched by one
+# of the above patterns_. You will have to add this option to any other entries
+# if you are for instance using plugin groups. See the ToneBoosters entry above
+# for an example.
+["*"]
+hack_reaper_update_display = true
 ```
 
 ## Troubleshooting common issues
@@ -374,6 +401,10 @@ Any VST2 plugin should function out of the box, although some plugins will need
 some additional dependencies for their GUIs to work correctly. Notable examples
 include:
 
+- **REAPER** and **Renoise** can both freeze when using plugins that call the
+  `audioMasterUpdateDisplay()` function. Until REAPER and Renoise fix this issue
+  you can set an [option](#miscellaneous-fixes-and-workarounds) through
+  `yabridge.toml` to work around this.
 - **Native Instruments** plugins work, but Native Access is unable to finish
   installing the plugins. To work around this you can open the .iso file
   downloaded to your downloads directory and run the installer directly. When
@@ -388,7 +419,8 @@ include:
   when the window gets dragged offscreen on the top and left dies of the screen.
 - **PSPaudioware** plugins with expandable GUIs, such as E27, may have their GUI
   appear in the wrong location after the GUI has been expanded. You can enable
-  an alternative [editor hosting mode](#editor-hosting-modes) to fix this.
+  an alternative [editor hosting mode](#miscellaneous-fixes-and-workarounds) to
+  fix this.
 - Plugins like **FabFilter Pro-Q 3** that can share data between different
   instances of the same plugin plugins have to be hosted within a single process
   for that functionality to work. See the [plugin groups](#plugin-groups)

src/common/configuration.cpp

@@ -51,6 +51,9 @@ Configuration::Configuration(const fs::path& config_path,
         if (toml::table* config = value.as_table()) {
             editor_double_embed =
                 (*config)["editor_double_embed"].value<bool>().value_or(false);
+            hack_reaper_update_display =
+                (*config)["hack_reaper_update_display"].value<bool>().value_or(
+                    false);
             group = (*config)["group"].value<std::string>();
         }
 

src/common/configuration.h

@@ -87,6 +87,14 @@ class Configuration {
      */
     bool editor_double_embed = false;
 
+    /**
+     * If this is set to true, then any calls to `audioMasterUpdateDisplay()`
+     * will automatically return 0 without being sent to the host. This is a
+     * HACK to work around implementations issues in REAPER and Renoise, see #29
+     * and #32.
+     */
+    bool hack_reaper_update_display = false;
+
     /**
      * The name of the plugin group that should be used for the plugin this
      * configuration object was created for. If not set, then the plugin should
@@ -107,6 +115,7 @@ class Configuration {
     template <typename S>
     void serialize(S& s) {
         s.value1b(editor_double_embed);
+        s.value1b(hack_reaper_update_display);
         s.ext(group, bitsery::ext::StdOptional(),
               [](S& s, auto& v) { s.text1b(v, 4096); });
         s.ext(matched_file, bitsery::ext::StdOptional(),

src/plugin/plugin-bridge.cpp

@@ -625,6 +625,7 @@ void PluginBridge::log_init_message() {
     init_msg << "config from:   '"
              << config.matched_file.value_or("<defaults>").string() << "'"
              << std::endl;
+
     init_msg << "hosting mode:  '";
     if (config.group) {
         init_msg << "plugin group \"" << *config.group << "\"";
@@ -637,10 +638,18 @@ void PluginBridge::log_init_message() {
         init_msg << ", 64-bit";
     }
     init_msg << "'" << std::endl;
+
+    bool other_options_set = false;
     init_msg << "other options: '";
     if (config.editor_double_embed) {
         init_msg << "editor: double embed";
-    } else {
+        other_options_set = true;
+    }
+    if (config.hack_reaper_update_display) {
+        init_msg << "hack: REAPER 'audioMasterUpdateDisplay' workaround";
+        other_options_set = true;
+    }
+    if (!other_options_set) {
         init_msg << "<none>";
     }
     init_msg << "'" << std::endl;

src/wine-host/bridges/vst2.cpp

@@ -523,6 +523,16 @@ intptr_t Vst2Bridge::host_callback(AEffect* effect,
                                    intptr_t value,
                                    void* data,
                                    float option) {
+    // HACK: Sadly this is needed to work around a timing issue with REAPER and
+    //       Renoise. See #29 and #32.
+    // TODO: We don't have access to the verbosity level here, but it would be
+    //       nice to log that this is being skipped when `YABRIDGE_DEBUG_LEVEL
+    //       >= 2`.
+    if (config.hack_reaper_update_display &&
+        opcode == audioMasterUpdateDisplay) {
+        return 0;
+    }
+
     HostCallbackDataConverter converter(effect, time_info);
     return send_event(vst_host_callback, host_callback_mutex, converter,
                       std::nullopt, opcode, index, value, data, option);