vanja/yabridge
1
0
Fork 0
mirror of https://github.com/robbert-vdh/yabridge.git synced 2026-05-10 04:30:12 +02:00
Code Issues Packages Projects Releases Wiki Activity

Reuse VST2 audio processing buffers on Wine side

Browse Source 
Just like we made similar changes on the plugin side a few commits ago
to prevent allocations there.
This commit is contained in:
Robbert van der Helm
2021-05-23 16:36:41 +02:00
parent 206b528075
commit 29e0a0fd36
3 changed files with 130 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+3 -2
View File
@@ -31,8 +31,9 @@ Versioning](https://semver.org/spec/v2.0.0.html).
objects on both sides. This greatly reduces the overhead of our VST3 bridging objects on both sides. This greatly reduces the overhead of our VST3 bridging
by getting rid of all memory allocations during audio processing. by getting rid of all memory allocations during audio processing.
- VST2 audio processing also received the same optimizations. In a few places - VST2 audio processing also received the same optimizations. In a few places
yabridge would still reallocate heap data during audio processing. We now make yabridge would still reallocate heap data during every audio processing cycle.
sure to always reuse all buffers used in the audio processing process. We now make sure to always reuse all buffers and heap data used in the audio
processing process.
- Considerably optimized both VST2 and VST3 audio processing by preventing - Considerably optimized both VST2 and VST3 audio processing by preventing
unnecessary memory operations. As it turns out, the underlying binary unnecessary memory operations. As it turns out, the underlying binary
serialization library used by yabridge would always reinitialize the type-safe serialization library used by yabridge would always reinitialize the type-safe
src/wine-host/bridges/vst2.cpp
+120 -129
View File
@@ -206,144 +206,135 @@ Vst2Bridge::Vst2Bridge(MainContext& main_context,
// they start producing denormals // they start producing denormals
ScopedFlushToZero ftz_guard; ScopedFlushToZero ftz_guard;
// These are used as scratch buffers to prevent unnecessary allocations. sockets.host_vst_process_replacing.receive_multi<
// Since don't know in advance whether the host will call AudioBuffers>([&](AudioBuffers& process_request,
// `processReplacing` or `processDoubleReplacing` we'll just create SerializationBufferBase& buffer) {
// both. // Since the value cannot change during this processing cycle, we'll
std::vector<std::vector<float>> output_buffers_single_precision( // send the current transport information as part of the request so
plugin->numOutputs); // we prefetch it to avoid unnecessary callbacks from the audio
std::vector<std::vector<double>> output_buffers_double_precision( // thread
plugin->numOutputs); std::optional<decltype(time_info_cache)::Guard>
time_info_cache_guard =
process_request.current_time_info
? std::optional(time_info_cache.set(
*process_request.current_time_info))
: std::nullopt;
sockets.host_vst_process_replacing.receive_multi<AudioBuffers>( // We'll also prefetch the process level, since some plugins will
[&](AudioBuffers& request, SerializationBufferBase& buffer) { // ask for this during every processing cycle
// Since the value cannot change during this processing cycle, decltype(process_level_cache)::Guard process_level_cache_guard =
// we'll send the current transport information as part of the process_level_cache.set(process_request.current_process_level);
// request so we prefetch it to avoid unnecessary callbacks from
// the audio thread
std::optional<decltype(time_info_cache)::Guard>
time_info_cache_guard =
request.current_time_info
? std::optional(time_info_cache.set(
*request.current_time_info))
: std::nullopt;
// We'll also prefetch the process level, since some plugins // As suggested by Jack Winter, we'll synchronize this thread's
// will ask for this during every processing cycle // audio processing priority with that of the host's audio thread
decltype(process_level_cache)::Guard process_level_cache_guard = // every once in a while
process_level_cache.set(request.current_process_level); if (process_request.new_realtime_priority) {
set_realtime_priority(true,
*process_request.new_realtime_priority);
}
// As suggested by Jack Winter, we'll synchronize this thread's // Let the plugin process the MIDI events that were received
// audio processing priority with that of the host's audio // since the last buffer, and then clean up those events. This
// thread every once in a while // approach should not be needed but Kontakt only stores
if (request.new_realtime_priority) { // pointers to rather than copies of the events.
set_realtime_priority(true, *request.new_realtime_priority); std::lock_guard lock(next_buffer_midi_events_mutex);
}
// Let the plugin process the MIDI events that were received // Since the host should only be calling one of `process()`,
// since the last buffer, and then clean up those events. This // processReplacing()` or `processDoubleReplacing()`, we can all
// approach should not be needed but Kontakt only stores // handle them over the same socket. We pick which one to call
// pointers to rather than copies of the events. // depending on the type of data we got sent and the plugin's
std::lock_guard lock(next_buffer_midi_events_mutex); // reported support for these functions.
std::visit(
[&]<typename T>(
std::vector<std::vector<T>>& input_audio_buffers) {
// The process functions expect a `T**` for their inputs
thread_local std::vector<T*> input_pointers{};
if (input_pointers.size() != input_audio_buffers.size()) {
input_pointers.resize(input_audio_buffers.size());
for (size_t channel = 0;
channel < input_audio_buffers.size(); channel++) {
input_pointers[channel] =
input_audio_buffers[channel].data();
}
}
// Since the host should only be calling one of `process()`, // We also reuse the output buffers to avoid some
// processReplacing()` or `processDoubleReplacing()`, we can all // unnecessary heap allocations
// handle them over the same socket. We pick which one to call if (!std::holds_alternative<std::vector<std::vector<T>>>(
// depending on the type of data we got sent and the plugin's process_response.buffers)) {
// reported support for these functions. process_response.buffers
std::visit( .emplace<std::vector<std::vector<T>>>();
overload{ }
[&](std::vector<std::vector<float>>& input_buffers) {
// The process functions expect a `float**` for std::vector<std::vector<T>>& output_audio_buffers =
// their inputs and their outputs std::get<std::vector<std::vector<T>>>(
std::vector<float*> inputs; process_response.buffers);
for (auto& buffer : input_buffers) { output_audio_buffers.resize(plugin->numOutputs);
inputs.push_back(buffer.data()); for (size_t channel = 0;
channel < output_audio_buffers.size(); channel++) {
output_audio_buffers[channel].resize(
process_request.sample_frames);
}
// And the process functions also expect a `T**` for their
// outputs
thread_local std::vector<T*> output_pointers{};
if (output_pointers.size() != output_audio_buffers.size()) {
output_pointers.resize(output_audio_buffers.size());
for (size_t channel = 0;
channel < output_audio_buffers.size(); channel++) {
output_pointers[channel] =
output_audio_buffers[channel].data();
}
}
if constexpr (std::is_same_v<T, float>) {
// Any plugin made in the last fifteen years or so
// should support `processReplacing`. In the off chance
// it does not we can just emulate this behavior
// ourselves.
if (plugin->processReplacing) {
plugin->processReplacing(
plugin, input_pointers.data(),
output_pointers.data(),
process_request.sample_frames);
} else {
// If we zero out this buffer then the behavior is
// the same as `processReplacing`
for (std::vector<T>& buffer :
output_audio_buffers) {
std::fill(buffer.begin(), buffer.end(), (T)0.0);
} }
// We reuse the buffers to avoid some unnecessary plugin->process(plugin, input_pointers.data(),
// heap allocations, so we need to make sure the output_pointers.data(),
// buffers are large enough since plugins can change process_request.sample_frames);
// their output configuration. The type we're using }
// here (single precision floats vs double } else if (std::is_same_v<T, double>) {
// precisioon doubles) should be the same as the one plugin->processDoubleReplacing(
// we're sending in our response. plugin, input_pointers.data(),
std::vector<float*> outputs; output_pointers.data(),
output_buffers_single_precision.resize( process_request.sample_frames);
plugin->numOutputs); } else {
for (auto& buffer : static_assert(
output_buffers_single_precision) { std::is_same_v<T, float> ||
buffer.resize(request.sample_frames); std::is_same_v<T, double>,
outputs.push_back(buffer.data()); "Audio processing only works with single and "
} "double precision floating point numbers");
}
},
process_request.buffers);
// Any plugin made in the last fifteen years or so // We modified the buffers within the `process_response` object, so
// should support `processReplacing`. In the off // we can just send that object back. Like on the plugin side we
// chance it does not we can just emulate this // cannot reuse the request object because a plugin may have a
// behavior ourselves. // different number of input and output channels
if (plugin->processReplacing) { sockets.host_vst_process_replacing.send(process_response, buffer);
plugin->processReplacing(plugin, inputs.data(),
outputs.data(),
request.sample_frames);
} else {
// If we zero out this buffer then the behavior
// is the same as `processReplacing``
for (std::vector<float>& buffer :
output_buffers_single_precision) {
std::fill(buffer.begin(), buffer.end(),
0.0);
}
plugin->process(plugin, inputs.data(), // See the docstrong on `should_clear_midi_events` for why we
outputs.data(), // don't just clear `next_buffer_midi_events` here
request.sample_frames); should_clear_midi_events = true;
} });
AudioBuffers response{
.buffers = output_buffers_single_precision,
.sample_frames = request.sample_frames,
.current_time_info = std::nullopt,
.current_process_level = 0,
.new_realtime_priority = std::nullopt};
sockets.host_vst_process_replacing.send(response,
buffer);
},
[&](std::vector<std::vector<double>>& input_buffers) {
// Exactly the same as the above, but for double
// precision audio
std::vector<double*> inputs;
for (auto& buffer : input_buffers) {
inputs.push_back(buffer.data());
}
std::vector<double*> outputs;
output_buffers_double_precision.resize(
plugin->numOutputs);
for (auto& buffer :
output_buffers_double_precision) {
buffer.resize(request.sample_frames);
outputs.push_back(buffer.data());
}
plugin->processDoubleReplacing(
plugin, inputs.data(), outputs.data(),
request.sample_frames);
AudioBuffers response{
.buffers = output_buffers_double_precision,
.sample_frames = request.sample_frames,
.current_time_info = std::nullopt,
.current_process_level = 0,
.new_realtime_priority = std::nullopt};
sockets.host_vst_process_replacing.send(response,
buffer);
}},
request.buffers);
// See the docstrong on `should_clear_midi_events` for why we
// don't just clear `next_buffer_midi_events` here
should_clear_midi_events = true;
});
}); });
} }
src/wine-host/bridges/vst2.h
+7
View File
@@ -107,6 +107,13 @@ class Vst2Bridge : public HostBridge {
*/ */
Configuration config; Configuration config;
/**
* The object we'll serialize the response into after the plugin has
* finished processing audio. We reuse this object to avoid reallocations
* since it contains pointers to heap data.
*/
AudioBuffers process_response;
/** /**
* We'll store the last transport information obtained from the host as a * We'll store the last transport information obtained from the host as a
* result of `audioMasterGetTime()` here so we can return a pointer to it if * result of `audioMasterGetTime()` here so we can return a pointer to it if