It’s the season of the wavetable – again. With Ableton Live 10.1 on the horizon and its free Wavetable device, we’ve got yet another free Max for Live device for making sound materials – and this time, you can make your wavetables from images.
Let’s catch you up first.
Ableton Live 10.1 will bring Wavetable as a new instrument to Standard and Suite editions – arguably one of the bigger native synth editions to Live in its history, ranking with the likes of Operator. And sure, as when Operator came out, you already have plug-ins that do the same; Ableton’s pitch is as always their unique approach to UI (love it or hate it), and integration with the host, and … having it right in the box:
Earlier this week, we saw one free device that makes wavetables for you, built as a Max for Live device. (Odds are anyone able to run this will have a copy of Live with Wavetable in it, since it targets 10.1, but it also exports to other tools). Wave Weld focuses on dialing in the sounds you need and spitting out precise, algorithmic results:
One thing Wave Weld cannot do, however, is make a wavetable out of a picture of a cat.
For that, you want Image2Wavetable. The name says it all: it generates wavetable samples from image data.
This means if you’re handy with graphics software, or graphics code like Processing, you can also make visual patterns that generate interesting wavetables. It reminds me of my happy hours and hours spent using U+I Software’s ground-breaking MetaSynth, which employs some similar concepts to build an entire sound laboratory around graphic tools. (It’s still worth a spin today if you’ve got a Mac; among other things, it is evidently responsible for those sweeping digital sounds in the original Matrix film, I’m told.)
Image2Wavetable is new, the creation of Dillon Bastan and Carlo Cattano – and there are some rough edges, so be patient and it sounds like they’re ready to hear some feedback on how it works.
But the workflow is really simple: drag and drop image, drag and drop resulting wavetable into the Wavetable instrument.
Okay, I suspect I know what I’m doing for the rest of the night.
Wavetables are capable of a vast array of sounds. But just dumping arbitrary audio content into a wavetable is unlikely to get the results you want. And that’s why Wave Weld looks invaluable: it makes it easy to generate useful wavetables, in an add-on that’s free for Max for Live.
Ableton Live users are going to want their own wavetable maker very soon. Live 10.1 will add Wavetable, a new synth based on the technique. See our previous preview:
Live 10.1 is in public beta now, and will be free to all Live 10 users soon.
So long as you have Max for Live to run it, Wave Weld will be useful to other synths, as well – including the developer’s own Wave Junction.
Because wavetables are periodic by their very nature, it’s more likely helpful to generate content algorithmically than just dump sample content of your own. (Nothing against the latter – it’s definitely fun – but you may soon find yourself limited by the results.)
Wave Wend handles generating those materials for you, as well as exporting them in the format you need.
1. Make the wavetable: use waveshaping controls to dial in the sound materials you want.
2. Build up a library: adapt existing content or collect your own custom creations.
3. Export in the format you need: adjusting the size les you support Live 10.1’s own device or other hardware and plug-ins.
The waveshaping features are really the cool part:
Unique waveshaping controls to generate custom wavetables
Sine waveshape phase shift and curve shape controls
Additive style synthesis via choice of twenty four sine waveshape harmonics for both positive and negative phase angles
Saw waveshape curve sharpen and partial controls
Pulse waveshape width, phase shift, curve smooth and curve sharpen controls
Triangle waveshape phase shift, curve smooth and curve sharpen controls
Random waveshape quantization, curve smooth and thinning controls
Wave Weld isn’t really intended as a synth, but one advantage of it being an M4L device is, you can easily preview sounds as you work.
The download is free with a sign-up for their mailing list.
They’ve got a bunch of walkthrough videos to get you started, too:
Major kudos to Phelan Kane of Meta Function for this release. (Phelan is an Ableton Certified Trainer as well as a specialist in Reaktor and Maschine on the Native Instruments side, as well as London chairman for AES.)
I’m also interested in other ways to go about this – SuperCollider code, anyone?
Reimagine pixels and color, melt your screen live into glitches and textures, and do it all for free on the Web – as you play with others. We talk to Olivia Jack about her invention, live coding visual environment Hydra.
Inspired by analog video synths and vintage image processors, Hydra is open, free, collaborative, and all runs as code in the browser. It’s the creation of US-born, Colombia-based artist Olivia Jack. Olivia joined our MusicMakers Hacklab at CTM Festival earlier this winter, where she presented her creation and its inspirations, and jumped in as a participant – spreading Hydra along the way.
Olivia’s Hydra performances are explosions of color and texture, where even the code becomes part of the aesthetic. And it’s helped take Olivia’s ideas across borders, both in the Americas and Europe. It’s part of a growing interest in the live coding scene, even as that scene enters its second or third decade (depending on how you count), but Hydra also represents an exploration of what visuals can mean and what it means for them to be shared between participants. Olivia has rooted those concepts in the legacy of cybernetic thought.
Following her CTM appearance in Berlin, I wanted to find out more about how Olivia’s tool has evolved and its relation to DIY culture and self-fashioned tools for expression.
CDM: Can you tell us a little about your background? Did you come from some experience in programming?
Olivia: I have been programming now for ten years. Since 2011, I’ve worked freelance — doing audiovisual installations and data visualization, interactive visuals for dance performances, teaching video games to kids, and teaching programming to art students at a university, and all of these things have involved programming.
Had you worked with any existing VJ tools before you started creating your own?
Alexandra Cárdenas and Olivia Jack @ ICLC 2019:
In your presentation in Berlin, you walked us through some of the origins of this project. Can you share a bit about how this germinated, what some of the precursors to Hydra were and why you made them?
It’s based on an ongoing Investigation of:
Collaboration in the creation of live visuals
Possibilities of peer-to-peer [P2P] technology on the web
Satellite Arts project by Kit Galloway and Sherrie Rabinowitz http://www.ecafe.com/getty/SA/ (1977)
I was also questioning some of the metaphors we use to understand and interact with the web. “Visiting” a website is exchanging a bunch of bytes with a faraway place and routed through other far away places. Rather than think about a webpage as a “page”, “site”, or “place” that you can “go” to, what if we think about it as a flow of information where you can configure connections in realtime? I like the browser as a place to share creative ideas – anyone can load it without having to go to a gallery or install something.
And I was interested in using the idea of a modular synthesizer as a way to understand the web. Each window can receive video streams from and send video to other windows, and you can configure them in real time suing WebRTC (realtime web streaming).
Here’s one of the early tests I did:
I really liked this philosophical idea you introduced of putting yourself in a feedback loop. What does that mean to you? Did you discover any new reflections of that during our hacklab, for that matter, or in other community environments?
It’s processes of creation, not having a specific idea of where it will end up – trying something, seeing what happens, and then trying something else.
Code tries to define the world using specific set of rules, but at the end of the day ends up chaotic. Maybe the world is chaotic. It’s important to be self-reflective.
How did you come to developing Hydra itself? I love that it has this analog synth model – and these multiple frame buffers. What was some of the inspiration?
I had no intention of creating a “tool”… I gave a workshop at the International Conference on Live Coding in December 2017 about collaborative visuals on the web, and made an editor to make the workshop easier. Then afterwards people kept using it.
I didn’t think too much about the name but [had in mind] something about multiplicity. Hydra organisms have no central nervous system; their nervous system is distributed. There’s no hierarchy of one thing controlling everything else, but rather interconnections between pieces.
Ed.: Okay, Olivia asked me to look this up and – wow, check out nerve nets. There’s nothing like a head, let alone a central brain. Instead the aquatic creatures in the genus hydra has sense and neuron essentially as one interconnected network, with cells that detect light and touch forming a distributed sensory awareness.
Most graphics abstractions are based on the idea of a 2d canvas or 3d rendering, but the computer graphics card actually knows nothing about this; it’s just concerned with pixel colors. I wanted to make it easy to play with the idea of routing and transforming a signal rather than drawing on a canvas or creating a 3d scene.
This also contrasts with directly programming a shader (one of the other common ways that people make visuals using live coding), where you generally only have access to one frame buffer for rendering things to. In Hydra, you have multiple frame buffers that you can dynamically route and feed into each other.
MusicMakers Hacklab in Berlin. Photo: Malitzin Cortes.
Livecoding is of course what a lot of people focus on in your work. But what’s the significance of code as the interface here? How important is it that it’s functional coding?
It’s inspired by [Alex McLean’s sound/music pattern environment] TidalCycles — the idea of taking a simple concept and working from there. In Tidal, the base element is a pattern in time, and everything is a transformation of that pattern. In Hydra, the base element is a transformation from coordinates to color. All of the other functions either transform coordinates or transform colors. This directly corresponds to how fragment shaders and low-level graphics programming work — the GPU runs a program simultaneously on each pixel, and that receives the coordinates of that pixel and outputs a single color.
What’s the experience you have of the code being onscreen? Are some people actually reading it / learning from it? I mean, in your work it also seems like a texture.
I am interested in it being somewhat understandable even if you don’t know what it is doing or that much about coding.
Code is often a visual element in a live coding performance, but I am not always sure how to integrate it in a way that feels intentional. I like using my screen itself as a video texture within the visuals, because then everything I do — like highlighting, scrolling, moving the mouse, or changing the size of the text — becomes part of the performance. It is really fun! Recently I learned about prepared desktop performances and related to the live-coding mantra of “show your screens,” I like the idea that everything I’m doing is a part of the performance. And that’s also why I directly mirror the screen from my laptop to the projector. You can contrast that to just seeing the output of an AV set, and having no idea how it was created or what the performer is doing. I don’t think it’s necessary all the time, but it feels like using the computer as an instrument and exploring different ways that it is an interface.
The algorave thing is now getting a lot of attention, but you’re taking this tool into other contexts. Can you talk about some of the other parties you’ve played in Colombia, or when you turned the live code display off?
Most of my inspiration and references for what I’ve been researching and creating have been outside of live coding — analog video synthesis, net art, graphics programming, peer-to-peer technology.
Having just said I like showing the screen, I think it can sometimes be distracting and isn’t always necessary. I did visuals for Putivuelta, a queer collective and party focused on diasporic Latin club music and wanted to just focus on the visuals. Also I am just getting started with this and I like to experiment each time; I usually develop a new function or try something new every time I do visuals.
Community is such an interesting element of this whole scene. So I know with Hydra so far there haven’t been a lot of outside contributions to the codebase – though this is a typical experience of open source projects. But how has it been significant to your work to both use this as an artist, and teach and spread the tool? And what does it mean to do that in this larger livecoding scene?
I’m interested in how technical details of Hydra foster community — as soon as you log in, you see something that someone has made. It’s easy to share via twitter bot, see and edit the code live of what someone has made, and make your own. It acts as a gallery of shareable things that people have made:
Although I’ve developed this tool, I’m still learning how to use it myself. Seeing how other people use it has also helped me learn how to use it.
I’m inspired by work that Alex McLean and Alexandra Cardenas and many others in live coding have done on this — just the idea that you’re showing your screen and sharing your code with other people to me opens a conversation about what is going on, that as a community we learn and share knowledge about what we are doing. Also I like online communities such as talk.lurk.org and streaming events where you can participate no matter where you are.
I’m also really amazed at how this is spreading through Latin America. Do you feel like there’s some reason the region has been so fertile with these tools?
It’s definitely influenced me rather than the other way around, getting to know Alexandra [Cardenas’] work, Esteban [Betancur, author of live coding visual environment Cine Vivo], rggtrn, and Mexican live coders.
Madrid performance. Photo: Tatiana Soshenina.
What has the scene been like there for you – especially now living in Bogota, having grown up in California?
I think people are more critical about technology and so that makes the art involving technology more interesting to me. (I grew up in San Francisco.) I’m impressed by the amount of interest in art and technology spaces such as Plataforma Bogota that provide funding and opportunities at the intersection of art, science, and technology.
The press lately has fixated on live coding or algorave but maybe not seen connections to other open source / DIY / shared music technologies. But – maybe now especially after the hacklab – do you see some potential there to make other connections?
To me it is all really related, about creating and hacking your own tools, learning, and sharing knowledge with other people.
Oh, and lastly – want to tell us a little about where Hydra itself is at now, and what comes next?
Right now, it’s improving documentation and making it easier for others to contribute.
Personally, I’m interested in performing more and developing my own performance process.
VCV Rack, the open source platform for software modular, keeps blossoming. If what you were waiting for was more maturity and stability and integration, the current pipeline looks promising. Here’s a breakdown.
Even with other software modulars on the scene, Rack stands out. Its model is unique – build a free, open source platform, and then build the business on adding commercial modules, supporting both the platform maker (VCV) and third parties (the module makers). That has opened up some new possibilities: a mixed module ecosystem of free and paid stuff, support for ports of open source hardware to software (Music Thing Modular, Mutable Instruments), robust Linux support (which other Eurorack-emulation tools currently lack), and a particular community ethos.
Of course, the trade-off with Rack 0.xx is that the software has been fairly experimental. Versions 1.0 and 2.0 are now in the pipeline, though, and they promise a more refined interface, greater performance, a more stable roadmap, and more integration with conventional DAWs.
New for end users
VCV founder and lead developer Andrew Belt has been teasing out what’s coming in 1.0 (and 2.0) online.
Here’s an overview:
Polyphony, polyphonic cables, polyphonic MIDI support and MPE
Multithreading and hardware acceleration
Tooltips, manual data entry, and right-click menus to more information on modules
Virtual CV to MIDI and direct MIDI mapping
2.0 version coming with fully-integrated DAW plug-in
More on that:
Polyphony and polyphonic cables. The big one – you can now use polyphonic modules and even polyphonic patching. Here’s an explanation:
Polyphonic MIDI and MPE. Yep, native MPE support. We’ve seen this in some competing platforms, so great to see here.
Multithreading. Rack will now use multiple cores on your CPU more efficiently. There’s also a new DSP framework that adds CPU acceleration (which helps efficiency for polyphony, for example). (See the developer section below.)
Oversampling for better audio quality. Users can set higher settings in the engine to reduce aliasing.
Tooltips and manual value entry. Get more feedback from the UI and precise control. You can also right-click to open other stuff – links to developer’s website, manual (yes!), source code (for those that have it readily available), or factory presets.
Core CV-MIDI. Send virtual CV to outboard gear as MIDI CC, gate, note data. This also integrates with the new polyphonic features. But even better –
Map MIDI directly. The MIDI map module lets you map parameters without having to patch through another module. A lot of software has been pretty literal with the modular metaphor, so this is a welcome change.
And that’s just what’s been announced. 1.0 is imminent, in the coming months, but 2.0 is coming, as well…
Rack 2.0 and VCV for DAWs. After 1.0, 2.0 isn’t far behind. “Shortly after” 2.0 is released, a DAW plug-in will be launched as a paid add-on, with support for “multiple instances, DAW automation with parameter labels, offline rendering, MIDI input, DAW transport, and multi-channel audio.”
These plans aren’t totally set yet, but a price around a hundred bucks and multiple ins and outs are also planned. (Multiple I/O also means some interesting integrations will be possible with Eurorack or other analog systems, for software/hardware hybrids.)
VCV Bridge is already deprecated, and will be removed from Rack 2.0. Bridge was effectively a stopgap for allowing crude audio and MIDI integration with DAWs. The planned plug-in sounds more like what users want.
Rack 2.0 itself will still be free and open source software, under the same license. The good thing about the plug-in is, it’s another way to support VCV’s work and pay the bills for the developer.
Rack v1 will bring a new, stabilized API – meaning you will need to do some work to port your modules. It’s not a difficult process, though – and I think part of Rack’s appeal is the friendly API and SDK from VCV.
You’ll also be able to take advantage of an SSE wrapper (simd.hpp) to take advantage of accelerated code on desktop CPUs, without hard coding manual calls to hardware that could break your plug-ins in the future. This also theoretically opens up future support for other platforms – like NEON or AVX acceleration. (It does seem like ARM platforms are the future, after all.)
While the Facebook group is still active and a place where a lot of people share work, there’s a new dedicated forum. That does things Facebook doesn’t allow, like efficient search, structured sections in chronological order so it’s easy to find answers, and generally not being part of a giant, evil, destructive platform.
Do you hate AI as a buzzword? Do you despise the millennial whoop? Do you cringe every time Valentine’s Day arrives? Well – get ready for all those things you hate in one place. But hang in there – there’s a moral to this story.
Now, really, the song is bad. Like laugh-out-loud bad. Here’s iOS app Amadeus Code “composing” a song for Valentine’s Day, which says love much in the way a half-melted milk chocolate heart does, but – well, I’ll let you listen, millennial pop cliches and all:
Fortunately this comes after yesterday’s quite stimulating ideas from a Google research team – proof that you might actually use machine learning for stuff you want, like improved groove quantization and rhythm humanization. In case you missed that:
Now, as a trained composer / musicologist, I do find this sort of exercise fascinating. And on reflection, I think the failure of this app tells us a lot – not just about machines, but about humans. Here’s what I mean.
Amadeus Code is an interesting idea – a “songwriting assistant” powered by machine learning, delivered as an app. And it seems machine learning could generate, for example, smarter auto accompaniment tools or harmonizers. Traditionally, those technologies have been driven by rigid heuristics that sound “off” to our ears, because they aren’t able to adequately follow harmonic changes in the way a human would. Machine learning could – well, theoretically, with the right dataset and interpretation – make those tools work more effectively. (I won’t re-hash an explanation of neural network machine learning, since I got into that in yesterday’s article on Magenta Studio.)
You might well find some usefulness from Amadeus, too.
This particular example does not sound useful, though. It sounds soulless and horrible.
Okay, so what happened here? Music theory at least cheers me up even when Valentine’s Day brings me down. Here’s what the developers sent CDM in a pre-packaged press release:
We wanted to create a song with a specific singer in mind, and for this demo, it was Taylor Swift. With that in mind, here are the parameters we set in the app.
Bpm set to slow to create a pop ballad
To give the verses a rhythmic feel, the note length settings were set to “short” and also since her vocals have great presence below C, the note range was also set from low~mid range.
For the chorus, to give contrast to the rhythmic verses, the note lengths were set longer and a wider note range was set to give a dynamic range overall.
After re-generating a few ideas in the app, the midi file was exported and handed to an arranger who made the track.
Wait – Taylor Swift is there just how, you say?
Taylor’s vocal range is somewhere in the range of C#3-G5. The key of the song created with Amadeus Code was raised a half step in order to accommodate this range making the song F3-D5.
From the exported midi, 90% of the topline was used. The rest of the 10% was edited by the human arranger/producer: The bass and harmony files are 100% from the AC midi files.
Now, first – these results are really impressive. I don’t think traditional melodic models – theoretical and mathematical in nature – are capable of generating anything like this. They’ll tend to fit melodic material into a continuous line, and as a result will come out fairly featureless.
No, what’s compelling here is not so much that this sounds like Taylor Swift, or that it sounds like a computer, as it sounds like one of those awful commercial music beds trying to be a faux Taylor Swift song. It’s gotten some of the repetition, some of the basic syncopation, and oh yeah, that awful overused millennial whoop. It sounds like a parody, perhaps because partly it is – the machine learning has repeated the most recognizable cliches from these melodic materials, strung together, and then that was further selected / arranged by humans who did the same. (If the machines had been left alone without as much human intervention, I suspect the results wouldn’t be as good.)
In fact, it picks up Swift’s ticks – some of the funny syncopations and repetitions – but without stringing them together, like watching someone do a bad impression. (That’s still impressive, though, as it does represent one element of learning – if a crude one.)
To understand why this matters, we’re going to have to listen to a real Taylor Swift song. Let’s take this one:i’
Okay, first, the fact that the real Taylor Swift song has words is not a trivial detail. Adding words means adding prosody – so elements like intonation, tone, stress, and rhythm. To the extent those elements have resurfaced as musical elements in the machine learning-generated example, they’ve done so in a way that no longer is attached to meaning.
No amount of analysis, machine or human, can be generative of lyrical prosody for the simple reason that analysis alone doesn’t give you intention and play. A lyricist will make decisions based on past experience and on the desired effect of the song, and because there’s no real right or wrong to how do do that, they can play around with our expectations.
Part of the reason we should stop using AI as a term is that artificial intelligence implies decision making, and these kinds of models can’t make decisions. (I did say “AI” again because it fits into the headline. Or, uh, oops, I did it again. AI lyricists can’t yet hammer “oops” as an interjection or learn the playful setting of that line – again, sorry.)
Now, you can hate the Taylor Swift song if you like. But it’s catchy not because of a predictable set of pop music rules so much as its unpredictability and irregularity – the very things machine learning models of melodic space are trying to remove in order to create smooth interpolations. In fact, most of the melody of “Blank Space” is a repeated tonic note over the chord progression. Repetition and rhythm are also combined into repeated motives – something else these simple melodic models can’t generate, by design. (Well, you’ll hear basic repetition, but making a relationship between repeated motives again will require a human.)
It may sound like I’m dismissing computer analysis. I’m actually saying something more (maybe) radical – I’m saying part of the mistake here is assuming an analytical model will work as a generative model. Not just a machine model – any model.
This mistake is familiar, because almost everyone who has ever studied music theory has made the same mistake. (Theory teachers then have to listen to the results, which are often about as much fun as these AI results.)
Music theory analysis can lead you to a deeper understanding of how music works, and how the mechanical elements of music interrelate. But it’s tough to turn an analytical model into a generative model, because the “generating” process involves decisions based on intention. If the machine learning models sometimes sound like a first year graduate composition student, that may be that the same student is steeped in the analysis but not in the experience of decision making. But that’s important. The machine learning model won’t get better, because while it can keep learning, it can’t really make decisions. It can’t learn from what it’s learned, as you can.
Yes, yes, app developers – I can hear you aren’t sold yet.
For a sense of why this can go deep, let’s turn back to this same Taylor Swift song. The band Imagine Dragons picked it up and did a cover, and, well, the chord progression will sound more familiar than before.
As it happens, in a different live take I heard the lead singer comment (unironically) that he really loves Swift’s melodic writing.
But, oh yeah, even though pop music recycles elements like chord progressions and even groove (there’s the analytic part), the results take on singular personalities (there’s the human-generative side).
“Stand by Me” dispenses with some of the ticks of our current pop age – millennial whoops, I’m looking at you – and at least as well as you can with the English language, hits some emotional meaning of the words in the way they’re set musically. It’s not a mathematical average of a bunch of tunes, either. It’s a reference to a particular song that meant something to its composer and singer, Ben E. King.
This is his voice, not just the emergent results of a model. It’s a singer recalling a spiritual that hit him with those same three words, which sets a particular psalm from the Bible. So yes, drum machines have no soul – at least until we give them one.
“Sure,” you say, “but couldn’t the machine learning eventually learn how to set the words ‘stand by me’ to music”? No, it can’t – because there are too many possibilities for exactly the same words in the same range in the same meter. Think about it: how many ways can you say these three words?
“Stand by me.”
Where do you put the emphasis, the pitch? There’s prosody. What melody do you use? Keep in mind just how different Taylor Swift and Ben E. King were, even with the same harmonic structure. “Stand,” the word, is repeated as a suspension – a dissonant note – above the tonic.
And even those observations still lie in the realm of analysis. The texture of this coming out of someone’s vocal cords, the nuances to their performance – that never happens the same way twice.
Analyzing this will not tell you how to write a song like this. But it will throw light on each decision, make you hear it that much more deeply – which is why we teach analysis, and why we don’t worry that it will rob music of its magic. It means you’ll really listen to this song and what it’s saying, listen to how mournful that song is.
And that’s what a love song really is:
If the sky that we look upon
Should tumble and fall
Or the mountain should crumble to the sea
I won’t cry, I won’t cry
No, I won’t shed a tear
Just as long as you stand
Stand by me
Stand by me.
Now that’s a love song.
So happy Valentine’s Day. And if you’re alone, well – make some music. People singing about hearbreak and longing have gotten us this far – and it seems if a machine does join in, it’ll happen when the machine’s heart can break, too.
Instead of just accepting all this machine learning hype, why not put it to the test? Magenta Studio lets you experiment with open source machine learning tools, standalone or inside Ableton Live.
Magenta provides a pretty graspable way to get started with an field of research that can get a bit murky. By giving you easy access to machine learning models for musical patterns, you can generate and modify rhythms and melodies. The team at Google AI first showed Magenta Studio at Ableton’s Loop conference in LA in November, but after some vigorous development, it’s a lot more ready for primetime now, both on Mac and Windows.
I got to sit down with the developers in LA, and also have been playing with the latest builds of Magenta Studio. But let’s back up and first talk about what this means.
Magenta Studio is out now, with more information on the Magenta project and other Google work on musical applications on machine learning:
Artificial Intelligence – well, apologies, I could have fit the letters “ML” into the headline above but no one would know what I was talking about.
Machine learning is a better term. What Magenta and TensorFlow are based on is applying algorithmic analysis to large volumes of data. “TensorFlow” may sound like some kind of stress exercise ball you keep at your desk. But it’s really about creating an engine that can very quickly process lots of tensors – geometric units that can be combined into, for example, artificial neural networks.
Seeing the results of this machine learning in action means having a different way of generating and modifying musical information. It takes the stuff you’ve been doing in music software with tools like grids, and lets you use a mathematical model that’s more sophisticated – and that gives you different results you can hear.
You may know Magenta from its involvement in the NSynth synthesizer —
NSynth uses models to map sounds to other sounds and interpolate between them – it actually applies the techniques we’ll see in this case (for notes/rhythms) to audio itself. Some of the grittier artifacts produced by this process even proved desirable to some users, as they’re something a bit unique – and you can again play around in Ableton Live.
But even if that particular application didn’t impress you – trying to find new instrument timbres – the note/rhythm-based ideas make this effort worth a new look.
Recurrent Neural Networks are a kind of mathematical model that algorithmically loops over and over. We say it’s “learning” in the sense that there are some parallels to very low-level understandings of how neurons work in biology, but this is on a more basic level – running the algorithm repeatedly means that you can predict sequences more and more effectively given a particular data set.
Magenta’s “musical” library applies a set of learning principles to musical note data. That means it needs a set of data to “train” on – and part of the results you get are based on that training set. Build a model based on a data set of bluegrass melodies, for instance, and you’ll have different outputs from the model than if you started with Gregorian plainchant or Indonesian gamelan.
One reason that it’s cool that Magenta and Magenta Studio are open source is, you’re totally free to dig in and train your own data sets. (That requires a little more knowledge and some time for your computer or a server to churn away, but it also means you shouldn’t judge Magenta Studio on these initial results alone.)
What’s in Magenta Studio
Magenta Studio has a few different tools. Many are based on MusicVAE – a recent research model that looked at how machine learning could be applied to how different melodies relate to one another. Music theorists have looked at melodic and rhythmic transformations for a long time, and very often use mathematical models to make more sophisticated descriptions of how these function. Machine learning lets you work from large sets of data, and then not only make a model, but morph between patterns and even generate new ones – which is why this gets interesting for music software.
Crucially, you don’t have to understand or even much care about the math and analysis going on here – expert mathematicians and amateur musicians alike can hear and judge the results. If you want to read a summary of that MusicVAE research, you can. But it’s a lot better to dive in and see what the results are like first. And now instead of just watching a YouTube demo video or song snippet example, you can play with the tools interactively.
Magenta Studio lets you work with MIDI data, right in your Ableton Live Session View. You’ll make new clips – sometimes starting from existing clips you input – and the device will spit out the results as MIDI you can use to control instruments and drum racks. There’s also a slide called “Temperature” which determines how the model is sampled mathematically. It’s not quite like adjusting randomness – hence they chose this new name – but it will give you some control over how predictable or unpredictable the results will be (if you also accept that the relationship may not be entirely linear). And you can choose number of variations, and length in bars.
The data these tools were trained on represents millions of melodies and rhythms. That is, they’ve chosen a dataset that will give you fairly generic, vanilla results – in the context of Western music, of course. (And Live’s interface is fairly set up with expectations about what a drum kit is, and with melodies around a 12-tone equal tempered piano, so this fits that interface… not to mention, arguably there’s some cultural affinity for that standardization itself and the whole idea of making this sort of machine learning model, but I digress.)
Here are your options:
Generate: This makes a new melody or rhythm with no input required – it’s the equivalent of rolling the dice (erm, machine learning style, so very much not random) and hearing what you get.
Continue: This is actually a bit closer to what Magenta Studio’s research was meant to do – punch in the beginning of a pattern, and it will fill in where it predicts that pattern could go next. It means you can take a single clip and finish it – or generate a bunch of variations/continuations of an idea quickly.
Interpolate: Instead of one clip, use two clips and merge/morph between them.
Groove: Adjust timing and velocity to “humanize” a clip to a particular feel. This is possibly the most interesting of the lot, because it’s a bit more focused – and immediately solves a problem that software hasn’t solved terribly well in the past. Since the data set is focused on 15 hours of real drummers, the results here sound more musically specific. And you get a “humanize” that’s (arguably) closer to what your ears would expect to hear than the crude percentage-based templates of the past. And yes, it makes quantized recordings sound more interesting.
Drumify: Same dataset as Groove, but this creates a new clip based on the groove of the input. It’s … sort of like if Band-in-a-Box rhythms weren’t awful, basically. (Apologies to the developers of Band-in-a-Box.) So it works well for percussion that ‘accompanies’ an input.
So, is it useful?
It may seem un-human or un-musical to use any kind of machine learning in software. But from the moment you pick up an instrument, or read notation, you’re working with a model of music. And that model will impact how you play and think.
More to the point with something like Magenta is, do you really get musically useful results?
Groove to me is really interesting. It effectively means you can make less rigid groove quantization, because instead of some fixed variations applied to a grid, you get a much more sophisticated model that adapts based on input. And with different training sets, you could get different grooves. Drumify is also compelling for the same reason.
Generate is also fun, though even in the case of Continue, the issue is that these tools don’t particularly solve a problem so much as they do give you a fun way of thwarting your own intentions. That is, much like using the I Ching (see John Cage, others) or a randomize function (see… all of us, with a plug-in or two), you can break out of your usual habits and create some surprise even if you’re alone in a studio or some other work environment.
One simple issue here is that a model of a sequence is not a complete model of music. Even monophonic music can deal with weight, expression, timbre. Yes, theoretically you can apply each of those elements as new dimensions and feed them into machine learning models, but – let’s take chant music, for example. Composers were working with less quantifiable elements as they worked, too, like the meaning and sound of the text, positions in the liturgy, multi-layered quotes and references to other compositions. And that’s the simplest case – music from punk to techno to piano sonatas will challenge these models in Magenta.
I bring this up not because I want to dismiss the Magenta project – on the contrary, if you’re aware of these things, having a musical game like this is even more fun.
The moment you begin using Magenta Studio, you’re already extending some of the statistical prowess of the machine learning engine with your own human input. You’re choosing which results you like. You’re adding instrumentation. You’re adjusting the Temperature slider using your ear – when in fact there’s often no real mathematical indication of where it “should” be set.
And that means that hackers digging into these models could also produce new results. People are still finding new applications for quantize functions, which haven’t changed since the 1980s. With tools like Magenta, we get a whole new slew of mathematical techniques to apply to music. Changing a dataset or making minor modifications to these plug-ins could yield very different results.
And for that matter, even if you play with Magenta Studio for a weekend, then get bored and return to practicing your own music, even that’s a benefit.
Where this could go next
There are lots of people out there selling you “AI” solutions and – yeah, of course, with this much buzz, a lot of it is snake oil. But that’s not the experience you have talking to the Magenta team, partly because they’re engaged in pure research. That puts them in line with the history of pure technical inquiries past, like the team at Bell Labs (with Max Mathews) that first created computer synthesis. Magenta is open, but also open-ended.
As Jesse Engel tells CDM:
We’re a research group (not a Google product group), which means that Magenta Studio is not static and has a lot more interesting models are probably on the way.
Our goal is just for someone to come away with an understanding that this is an avenue we’ve created to close the gap between our ongoing research and actual music making, because feedback is important to our research.
Plugin Boutique has announced an exclusive sale on the Midi Madness 3 world-class MIDI generator software for Windows and Mac. Midi Madness 3 can create an unlimited number of melodies using a simple set of probability weightings. Simply set some parameters, such as a chord sequence and some MIDI controllers, and let Midi Madness go […]
zenAud.io has released the previously announced Windows version of its critically acclaimed ALK looping software. The Mac version of ALK2 has been a favorite among songwriters and live performers and enjoys an enthusiastic following amongst loop artists all over the world, recognized as “an extremely easy to use looper station that overrides typical limitations, adds […]
There’s something about point releases – not the ones with any radical changes, but just the ones that give you a bunch of little stuff you want. That’s Live 10.1; here’s a tour.
Live 10.1 was announced today, but I sat down with the team at Ableton last month and have been working with pre-release software to try some stuff out. Words like “workflow” are always a bit funny to me. We’re talking, of course, mostly music making. The deal with Live 10.1 is, it gives you some new toys on the sound side, and makes mangling sounds more fun on the arrangement side.
Oh, and VST3 plug-ins work now, too. (MOTU’s DP10 also has that in an upcoming build, among others, so look forward to the Spring of VST3 Support.)
Let’s look at those two groups.
Sound tools and toys
User wavetables. Wavetable just got more fun – you can drag and drop samples onto Wavetable’s oscillator now, via the new User bank. You can get some very edgy, glitchy results this way, or if you’re careful with sample selection and sound design, more organic sounds.
This looks compelling.
Here’s how it works: Live splits up your audio snippet into 1024 sample chunks. It then smooths out the results – fading the edges of each table to avoid zero-crossing clicks and pops, and normalizing and minimizing phase differences. You can also tick a box called “Raw” that just slices up the wavetable, for samples that are exactly 1024 samples or a regular periodic multiple of that.
Give me some time and we can whip up some examples of this, but basically you can glitch out, mangle sounds you’ve recorded, carefully construct sounds, or just grab ready-to-use wavetables from other sources.
But it is a whole lot of fun and it suggests Wavetable is an instrument that will grow over time.
Here’s that feature in action:
Delay. Simple Delay and Ping Pong Delay have merged into a single lifeform called … Delay. That finally updates an effect that hasn’t seen love since the last decade. (The original ones will still work for backwards project compatibility, though you won’t see them in a device list when you create a new project – don’t panic.)
At first glance, you might think that’s all that’s here, but in typical Ableton fashion, there are some major updates hidden behind those vanilla, minimalist controls. So now you have Repitch, Fade, and Jump modes. And there’s a Modulation section with rate, filter, and time controls (as found on Echo). Oh, and look at that little infinity sign next to the Feedback control.
Yeah, all of those things are actually huge from a sound design perspective. So since Echo has turned out to be a bit too much for some tasks, I expect we’ll be using Delay a lot. (It’s a bit like that moment when you figure out you really want Simpler and Drum Racks way more than you do Sampler.)
The old delays. Ah, memories…
And the new Delay. Look closely – there are some major new additions in there.
Channel EQ. This is a new EQ with visual feedback and filter curves that adapt across the frequency range – that is, “Low,” “Mid,” and “High” each adjust their curves as you change their controls. Since it has just three controls, that means Channel EQ sits somewhere between the dumbed down EQ Three and the complexity of EQ Eight. But it also means this could be useful as a live performance EQ when you don’t necessarily want a big DJ-style sweep / cut.
Here it is in action:
The stuff above is fun, but you obviously don’t need it. Where Live 10.1 might help you actually finish music is in a slew of new arrangement features.
Live 10 felt like a work in progress as far as the Arrange view. I think it immediately made sense to some of us that Ableton were adjusting arrangement tools, and ironing out the difference between, say, moving chunks of audio around and editing automation (drawing all those lovely lines to fade things in and out, for instance).
But it felt like the story there wasn’t totally complete. In fact, the change may have been too subtle – different enough to disturb some existing users, but without a big enough payoff.
So here’s the payoff: Ableton have refined all those subtle Arrange tweaks with user feedback, and added some very cool shape drawing features that let you get creative in this view in a way that isn’t possible with other users.
Fixing “$#(*& augh undo I didn’t want to do that!” Okay, this problem isn’t unique to Live. In every traditional DAW, your mouse cursor does conflicting things in a small amount of space. Maybe you’re trying to move a chunk of audio. Maybe you want to resize it. Maybe you want to fade in and out the edges of the clip. Maybe it’s not the clip you’re trying to edit, but the automation curves around it.
In studio terms, this sounds like one of the following:
[silent, happy clicking, music production getting … erm … produced]
$#(*&*%#*% …. Noo errrrrrrrgggggg … GAACK! SDKJJufffff ahhh….
Live 10 added a toggle between automation editing and audio editing modes. For me, I was already doing less of the latter. But 10.1 is dramatically better, thanks to some nearly imperceptible adjustments to the way those clip handles work, because you can more quickly change modes, and because you can zoom more easily. (The zoom part may not immediately seem connected to this, but it’s actually the most important part – because navigating from your larger project length to the bit you’re actually trying to edit is usually where things break down.)
In technical terms, that means the following:
Quick zoom shortcuts. I’ll do a separate story on these, because they’re so vital, but you can now jump to the whole song, details, zoom various heights, and toggle between zoom states via keyboard shortcuts. There are even a couple of MIDI-mappable ones.
Clips in Arrangement have been adjusted. From the release notes: “The visualisation of Arrangement clips has been improved with adjusted clip borders and refinements to the way items are colored.” Honestly, you won’t notice, but ask the person next to you how much you’re grunting / swearing like someone is sticking something pointy into your ribs.
Pitch gestures! You can pitch-zoom Arrangement and MIDI editor with Option or Alt keys – that works well on Apple trackpads and newer PC trackpads. And yeah, this means you don’t have to use Apple Logic Pro just to pinch zoom. Ahem.
The Clip Detail View is clearer, too, with a toggle between automation and modulation clearly visible, and color-coded modulation for everything.
The Arrangement Overview was also adjusted with better color coding and new resizing.
In addition, Ableton have worked a lot with how automation editing functions. New in 10.1:
Enter numerical values. Finally.
Free-hand curves more easily. With grid off, your free-hand, wonky mouse curves now get smoothed into something more logical and with fewer breakpoints – as if you can draw better with the mouse/trackpad than you actually can.
Simplify automation. There’s also a command that simplifies existing recorded automation. Again – finally.
So that fixes a bunch of stuff, and while this is pretty close to what other DAWs do, I actually find Ableton’s implementation to be (at last) quicker and friendlier than most other DAWs. But Ableton kept going and added some more creative ideas.
Insert shapes. Now you have some predefined shapes that you can draw over automation lanes. It’s a bit like having an LFO / modulation, but you can work with it visually – so it’s nice for those who prefer that editing phase as a way do to their composition. Sadly, you can only access these via the mouse menu – I’d love some keyboard shortcuts, please – but it’s still reasonably quick to work with.
Modify curves. Hold down Option/Ctrl and you can change the shape of curves.
Stretch and skew. Reshape envelopes to stretch, skew, stretch time / ripple edit.
Insert Shapes promises loads of fun in the Arrangement – words that have never been uttered before.
Check out those curve drawing and skewing/scaling features in action:
You can freeze tracks with sidechains, instead of a stupid dialog box popping up to tell you you can’t, because it would break the space-time continuum or crash the warp core injectors or … no, there’s no earthly reason why you shouldn’t be able to freeze sidechains on a computer.
You can export return and master effects on the actual tracks. I know, I know. You really loved bouncing out stems from Ableton or getting stems to remix and having little bits of effects from all the tracks on separate stems that were just echos, like some weird ghost of whatever it was you were trying to do. And I’m a lazy kid, who for some reason thinks that’s completely illogical since, again, this is a computer and all this material is digital. But yes, for people are soft like me, this will be a welcome feature.
So there you have it. Plus you now get VST3s, which is great, because VST3 … is so much … actually, you know, even I don’t care all that much about that, so let’s just say now you don’t have to check if all your plug-ins will run or not.
Go get it
One final note – Max for Live. 10.0.6 synchronized with Max 8.0.2. See those release notes from Cycling ’74:
Live 10.1 is keeping pace, with the beta you download now including Max 8.0.3.
Ableton haven’t really “integrated” Max for Live; they’re still separate products. And so that means you probably don’t want perfect lockstep between Max and Live, because that could mean instability on the Live side. It’d be more accurate to say that what Ableton have done is to improve the relationship between Max and Live, so that you don’t have to wait as long for Max improvements to appear in Max for Live.
Live 10.1 is in beta now with a final release coming soon.
Ableton has announced the forthcoming Live 10.1 update, adding user wavetables, new devices, workflow upgrades and more to the music production software for Windows and Mac. Live 10.1 expands the possibilities for making and shaping sound, as well as improving key features for editing and finalizing music. The new features in Live 10.1 include: User […]