- A new algorithm generates the portamento effect between any two audio signals in real-time.
- It seamlessly blends frequency segments from one audio signal into another, without increasing/decreasing the volume of signals.
Musicians use an effect called ‘portamento’ to transition between notes. It’s a technique of gliding from one note to another without actually defining the intermediate notes.
Musicians have been using portamento for more than 200 years. However, limited types of instruments can pull off this effect. These are the instruments that can continuously vary in pitch like the violin or the human voice.
Recently, a student at MIT — Trevor Henderson — developed an algorithm that generates the portamento effect between any two audio signals in real-time. He demonstrated the algorithm’s capability by merging different audio clips, such as one song blending into another, and a piano note gliding into a human voice.
How Does The Algorithm Work?
The algorithm is based on a centuries-old geometry-based framework known as ‘optimal transport’. The framework determines optimal ways to move data points between multiple origin and destination configurations. It has been applied to 3D modeling, image alignment, fluid dynamics, and computer graphics.
Henderson used this framework to interpolate audio signals. The algorithm works by splitting the audio signal into small segments (of about 50 milliseconds). Then, it determines the most efficient way to move the pitches in individual segments to pitches in the other audio signal.
In this way, the algorithm generates a smooth glide of the portamento effect. It also preserves the audio fidelity as the signal transitions.
Reference: arXiv:1906.06763 | MIT
Typically, fading effects are used to soften the transition between two audio signals. This is the traditional method in which the volume of one signal is reduced while the volume of other signal rises.
The new algorithm, on the other hand, seamlessly blends frequency segments from one audio signal into another, without increasing/decreasing the volume of signals.
This technique extends the portamento to various instruments, including polyphonic ones. In fact, it can be used to transition between different sets of instruments or any other pair of audio signals. It generates clear musical results in real-time.
The ‘interpolation parameter’ can be controlled by a slider component such as a crossfader on the DJ’s mixing board. As you slide the crossfader, the parameter changes, producing the portamento effect.
In order to eliminate distortions, Henderson used ‘frequency reassignment’, an application of a single-processing technique. It allows individual notes to easily transition between signals. He also developed a new method to synthesize new phases for each signal while merging the small segments of audios, so adjacent segments do not interfere with each other.
Read: Facebook AI Converts Music From One Style To Another
In the next study, Henderson will try to feed the portamento output back into its input. This might produce another effect called legato, which represents fluid, continuous motion between different notes.