The Virtual Mixer
The Future of
“David Gibson’s Groundbreaking work is a real eye-opener for all of us, music professionals and casual listeners alike. He has single-handedly rethought the whole metaphor for the visual representation of recorded music and conjured up a brand new way to interact with it. It’s high time we took a new look at the antique user interface employed by typical MIDI and sample editing tools”
– Thomas “Dolby” Roberson
THE VIRTUAL MIXER INTERFACE
The Virtual Mixer is a Graphical User Interface that displays the amplitude and frequency range of sounds on a multitrack, and then allows you to use these visuals to control various types of mixing consoles and effect units.
MIXING IN VIRTUAL REALITY
Sitting in the middle of the Surround Mix below wearing a Dataglove.
You could then reach out and grab a sound
and place it wherever you want!
You could even throw sounds and bounce them off the walls.
With a full Virtual Reality Head Tracking Helmet,
you could then turn around and see the sounds behind you.
You can also mix in normal stereo in using 3D images.
3D helps you to see around sounds in a mix in order to control them.
3D visualization allows you to see the sounds precisely where they are between the speakers
in the world of Imaging.
This is the ideal interface for Mixing in Surround Sound
and it will be so much fun!
CLICK HERE to see examples of Surround Sound Mixes from “The Art Of Mixing, 2nd Edition.
The next step is to place images and sounds inside the body
for more info
The mapping of audio parameters to visuals is based on our common perception of imaging between the speakers. The primary design consideration has been to show masking (where one sound hides another sound) visually. Of course, the second design consideration was to create an intuitive interface that is calibrated to the settings on a mixer so that a Professional Engineer will still have precise control.
Fader level (volume) is mapped out as a function of front to back, so that louder sounds appear closer (and therefore, a little larger), and softer sounds appear more distant. The grid on the floor is calibrated to the faders on the mixing console. Panning is naturally mapped out as left to right placement. The average pitch of the sound over the entire song is mapped out as a function of up and down. Higher pitch sounds appear higher; lower pitch sounds appear lower (just as they do in a mix). Also, hi frequency sounds are smaller because they take up less space in a mix. Low frequency sounds are larger because they mask other sounds more in a mix. You can also call up windows that show equalization and auxiliary send levels visually.
The next step is to put on 3D glasses and have the images floating in 3D right where the sounds seem to be between the speakers. In such a virtual environment the mixer uses a dataglove to move the sounds around in the mix. If you want a sound louder pull it toward you. If you want it in the left speaker put it over there. You can even toss sounds back and forth (and bounce them off the walls). Watch out, here comes an electric guitar sound. In 5.1 surround sound mixing, the visuals would be floating throughout the room.
FORMATS AND VERSIONS
The Interface is being developed for a wide range of mixing equipment — hardware and software products including:
Digital Audio Workstations: The program is being developed to work on PC and Macintosh computer formats as a plugin for TDM, RTAS, and VST systems. This includes Cakewalk, Cubase, Logic Audio, Ableton Live and ProTools. The Virtual Mixer will show up as an alternative to pictures of a mixing console and may be used to control the volume, panning and all of the automation parameters of the other plugins on each of the channels.
Digital Consoles – Yamaha, Roland, Tascam, Soundcraft, Mackie, Panasonic (Ramsa), and all future digital consoles. A software program will be developed to run on a separate computer. It will interface to the Digital Console by way of MIDI and Digital Audio. Sold for $400. Ultimately, it is assumed these companies will license the interface and incorporate it as an alternative to pictures of a mixing console on the computer screen.
Major 100% Automated Consoles: Euphonix, Sony, AMS NEVE, AMEC, and SSL. Price range will vary depending on the manufacturer.
3D DISPLAYS AND CONTROLLERS
– Touch Sensitive Computer Screen – Makes mixing as easy as dragging the sound to where you want it with your finger.
– 3D Glasses for the Computer Monitor
– Full Virtual Reality with Head Tracking Helmet – So you can turn around and see sounds floating behind you when mixing in Surround Sound or using a 3D Sound Processor.
– Big Screen Projection in full 3D (similar to IMAX 3D). Images float between the speakers right where you hear them.
The software will work in normal stereo with volume as front to back, or in Surround with volume as a function of size of the sound visual.
We will also put out versions that will control display various waveforms and audio synthesis parameters. These parameters will also be able to be controlled visually, and in full 3D Virtual Reality. Specific textures of spheres correspond to the waveform.
One of the most revolutionary aspects of the program is the way in which automation is visualized. Time is shown as the Z-axis (back to front).
The past is in the distance and time moves forward until you get to the the present moment in the middle of the screen.
The major advantage is that you can see the relationships of all of the settings at any one moment. Current “line” automation does not show you any usable relationships between the information. The other big advantage is that you use this window to create automation in real time by dragging spheres around while the mix is playing.
When mixing in Virtual Reality this timeline will appear as a tunnel of time. You will be able to go to the past or future to change any parameter of the mix.
More Engrossing – When you see the mix visually you are incorporating more of your senses in the musical process. The more senses you use (especially in a fleeting artistic endeavor such as mixing sound) the more your entire being is engulfed in the experience. And, the more you are engrossed in all of the details the easier it is to come up with a great mix.
More Intuitive – Pictures of sounds are one logical step closer to the music you are mixing than faders and knobs on a console. Manipulating visuals of the sounds themselves is much more intuitive than pictures of a console. Pictures of sounds are more like music than knobs.
More Conducive to the Creative Process – Studio equipment is notorious for getting in the way of the creative process. Faders and knobs on a console distract recording engineers from the music they are trying to mix. Images that are flashing to the music help the user to focus more clearly on the invisible sounds they are mixing.
More Information about the Mix – The visuals provide additional information helpful for the mix. The primary goal of the visual system has been to show “audio masking” visually. Engineers can use this information to discover hidden problems, and best of all, to be able to explain the problem to the band and producer. For example, a bad arrangement becomes clearly evident.
More Perspective – The visual framework provides the engineer with a perspective on all of the possibilities available to the engineer. The framework shows each parameter within each piece of equipment in the studio and how each parameter contributes to an overall mix. With this visual perspective of all that is possible in a mix, a humongous number of possibilities are displayed in a way that puts an array of creative ideas at an engineer’s fingertips.
More Relationships – The interface shows more than just all of the settings of each piece of equipment in the studio. The interface shows the relationships of all the settings and how they work together to create a mix. After all, it is the relationships of all the settings that really count.
Better Communication – The interface enhances and simplifies the communication process between recording engineer and client (band or producer). The interface is so intuitive that even inexperienced clients will be able to follow the development of the mix and communicate effectively with the engineer.
More Fun – And of course, we mustn’t forget — Flashing 3D visuals are a blast to watch and work with.
“The nature of the medium affects the art you create.”
• What’s it for and what does it do? The Virtual Mixer is an interface that replaces visuals of mixing consoles on the computer screen. Instead of using pictures of faders, we are using pictures of the sounds between the mix.
• Don’t visuals distract you from listening? Actually, they help you focus more on the music you are mixing. Pictures of a mixing board on a computer screen and physical faders, knobs and buttons on a console distract a listener way more. They look nothing like music. Since you have to have some type of interface, why not have one that looks more like the music that you are actually mixing. Also, when you have more different types of stimulus involving all of the senses, you are actually less distracted.
• What’s imaging? Imaging is the apparent placement of sounds between the speakers. It is where we imagine the sounds to be between the speakers.
• How does it aid in better communication? It makes it so that people who know very little about mixing can at least see what is being done every step of the way. It also helps you to focus better on the sounds in the mix — and often helps you to notice sounds that you might be missing or have forgotten.
• How’s it a teaching tool? The visuals are currently being used in the classes at Globe Institute of Recording and Production to explain and show all of the styles of mixes in the world — and to explain how to create all types of mixes with the available studio equipment.
• Why the spheres — why not pictures of instruments? Spheres best represent the reality of the dynamics that actually go on in mixing. They show the exact amount of space that each sound takes up in a mix between the speakers. This is important because of the problem of masking (where one sound hides another). Ultimately, however, you will be able to import any type of visual that you like — from instrument icons, to clouds, to crystals.