This Software Can Separate Individual Audio Tracks From A Video

MIT has developed new artificial intelligence software that can isolate multiple musical performances from a video. Not only this, but it can make theses separated audio tracks louder or softer.

Trained on over 60 hours of videos, the “PixelPlayer” system can view a never-before-seen musical performance, identify specific instruments at pixel level, and extract the sounds that are associated with those instruments.

For example, it can take a video of a tuba and a trumpet playing the “Super Mario Brothers” theme song, and separate out the soundwaves associated with each instrument.

The researchers say that the ability to change the volume of individual instruments means that in the future, systems like this could potentially help engineers improve the audio quality of old concert footage. You could even imagine producers taking specific instrument parts and previewing what they would sound like with other instruments (i.e. an electric guitar swapped in for an acoustic one).

You can read the original blog post on the MIT website to learn more about this fascinating project.

Next: Synthesizer Performance Featuring Korg Volca FM + A Viola