This is about as affordable and easy as gestural interaction with music can get. The powerful Geco music controller app pairs with the $80 Leap Motion hand tracking hardware – and now the app is free.
But it could be just the beginning.
For its part, the Leap Motion is now sort of yesterday’s news. But the small rectangular box is still a quick-and-easy way to get your computer tracking hand gestures – if you’re into that sort of thing. Geert Bevin’s Geco app provides the glue between the Leap’s sensing capabilities and your music software, allowing the computer to recognize gestures and then convey them as MIDI or OSC messages (among other tricks).
And if for some reason you had a Leap and waited to pick up the app – or if you needed an excuse to give this a play – now the app is free. (Since its release, it’s also had some major updates, so it’s worth another go even if you tried it before.)
I’ve played with Geert’s app before, and it’s fairly impressive. You’re always going to be a tough critic of any sort of gestural interaction, because the link between hands and perception is so finely tuned. But the Leap opens up some possibilities – even if you don’t really want to wave your hands around for a whole performance, it could add the ability to perform quick shortcuts or control a single parameter. And it’s a huge advance in comparison to things like Roland’s IR-tracking technology, for instance.
But it’s what’s coming round the bend that may be most interesting. The reason Geert had to make Geco free at this particular moment is that Leap is killing its app store. (See their blog post on the topic. It’s not the most elegant “sunsetting,” but then it seems the whole industry had to get over this idea that everyone should create an app store as Apple had.)
Leap are moving on to take the software and hardware smarts of the Leap Motion and start to build it into two new (overlapping) arenas – mobile and VR.
Right away, in fact, you can use the Leap Motion with Windows and Android VR headsets. (The, erm, sophisticated integration technology there is a “universal adapter” that involves just mounting the Leap Motion to the headset itself – plastic and 3M adhesive.)
The thing is, the Leap Motion is kind of cool when tethered to a computer, but way more interesting when it’s set loose. And that’s the next step, with something upcoming that Leap is calling the Leap Mobile Platform.
Think virtual reality and augmented reality – battery powered, untethered from a computer, and totally mobile.
For music, this is especially compelling as it opens up the possibility of new experimentation with interfaces. VR and AR have given us the visuals of what that could look like, but that’s meaningless without the ability to interact with those worlds.
Geert tells CDM he’s working in this direction: “I’ll be getting an early version in order to be able to take what I’ve learned from my GECO and GameWAVE Leap Motion apps and apply this to Mobile Leap Motion with VR and AR,” he says. “I’m really interested in the AR part for live performance.”
AR is augmented reality – that is, a visualization that you see atop the real world, instead of replacing your vision entirely. AR beats VR onstage, unless you want to shut yourself off from your audience with enormous goggles.
In the meantime, there’s no need to wait – you can use Geco right now, provided you can get your hands on a Leap Motion. And with Apple having unveiled its augmented reality solution last month, and a bunch of parties jumping on VR and AR on Windows, Android, and beyond for gaming and other experiences, we’ll be watching to see whether musicians find a way to use these technologies in coming months.
For up and coming cyber-pop talent, look no further than Chagall, the Amsterdam-born London-based cyborg diva.
Chagall van den Berg (full name) was an early adopter of the mi.mu gloves, a wearable interface that’s the latest generation of a tradition of interfaces that dates back to Amsterdam’s own STEIM research center and pioneering work by Michel Waisvisz. (Even if you have no interest in glove-based interfaces, Waisvisz can arguably be credited for producing the model of human/computer musical interaction as we now know it – it’s worth understanding.)
And Chagall herself is emblematic of the kind of brainpower-meets-virtuosic performance of that scene. mi.mu, now counting Chagall as one of its partners, is a who’s who of thinkers working in wearable technology and performance, including one other well-known icon, Imogen Heap. (We were lucky to have this crew join us for the first-ever CTM Festival Hacklab we hosted, which in turn was their first experiment in unleashing the gloves on new users.)
Accordingly, Chagall has become a spokeperson for innovative music technology as well as a musical star, gracing stages from TED to Futurefest to Ableton’s Loop.
Okay, so – tech, check. Brainy people, check.
How’s the music?
Oh, yeah – brilliant. See this gorgeous single music video, with visuals by Eduardo Fitch.
Chagall is the new all-in-one DIY pop everything: powerhouse songwriter with earworm-making skills, vocalist with easy, relaxed power, and producer. There’s a team behind the AV show, but the songs are all her. That marks a departure from the way pop is most conventionally fabricated.
And Chagall’s involvement with the gloves therefore represents something significant. Imogen Heap had already proven she could make use of the gloves, but they were designed around her with her direct input. Chagall has taken them in new directions.
The music itself seems to naturally extend itself into the gloves, rather than the other way round. This is silky, futuristic pop with the visuals to match. (Van den Berg works both with the medium of reactive visuals and music, while singing and writing and doing generally everything else, too.)
Expect to see more of this. “Calibration,” the touring show that incorporates this tech among others, is now set to burst onto the scene. See the behind the scenes video:
And yes, while Berlin has had a lot of the fun lately with its presumptive crown as music tech capital, here’s a British-based, British-backed brain trust in action:
Eduardo Fitch – Art Direction
Adam Stark – Visuals & Software Design
Leyla Rees – Movement Direction
Natalie Rowland – Lighting Design
Rob Donnelly-Jackson – Sound Engineer
Claire Eve – Producer
Georgina Harper – Executive Producer
Brandon Wade – Director, DoP, Post
Matthew McGuinees – Camera, Post, Sound Editor
Balasz Koszta – Camera Op
A year old, but for more detail on the gloves and how she works with the TC VoiceLive harmonizer, here’s a more intimate rehearsal video:
Yeah, it feels a bit like Imogen Heap: The Next Generation. But that’s a really good thing – a credit to Chagall and Imogen alike.
Here’s Chagall demonstrating at Ableton Loop:
Oh, the heck with it, she’s too fun to watch, here’s another video (any other singers a bit jealous?)
And dueling TED talks from Imogen and Chagall, because it’s the Internet and I don’t run out of page space:
German YouTuber “studentsmusic” has come across the Nintendo Power Glove mod used by none other than Eurorack originator Dieter Doepfer. And he had a hell of a client – Kraftwerk.
Now, whether you have any desire whatsoever to don some gloves and wave your hands around, the peculiar category of music glove has a long history intertwined with a lot of today’s thinking about music, controllers, and expression.
Michel Waisvisz gets a brief mention here, but it’s worth noting that his project “The Hands,” originating in 1984, was one of the first gestural controllers and likely shaped many other devices after. I would presume it’s possible someone at Nintendo was even aware of his work. The Power Glove, for its part, came later – 1989-1990. And certainly from that point on I suspect you can credit Waisvisz and the Amsterdam research center STEIM for suggesting that waving your hands around could be used for music.
In addition to custom-engineered solutions, since Nintendo’s debut musicians have been hacking gaming solutions, too. (Actually early in CDM’s life I was compiling these – as in this set of 2005 links to accompany something I wrote for Computer Music. Ooh, but that’s weird for me to read. Moving on.)
Anyway, Dieter’s creation is really quite clever – and it’s worth watching the video here.
Now, Nintendo is about to release another set of gestural controllers in the form of Switch. Even if you aren’t into gaming, the Nintendo launch game for that hardware is full of ideas. The Verge claim this is sophisticated technology, but my bet is the actual hardware is pretty simple and this is just smart, finely-tuned code.
Note the use of the mic for control, for instance – that’s something you can use.
So I don’t think any of these ideas are exhausted yet. But it does mean it’s time again to do this:
UK music startup ROLI are on one heck of a roll. And they’re slowly becoming a laboratory for trying out genuinely new ideas for music making – finally breaking the mold of an industry that often relishes nostalgia.
Now, ROLI’s got the engineering, acquiring plug-in maker FXpansion and cross-platform development architecture JUCE. They’ve got the flagship hardware, in the form of their Seaboard and Seaboard RISE expressive controllers – futuristic gadgets that look like the piano someone would play on Star Trek: The Next Generation more than what you’d find in your local Guitar Center. And they’ve got connectivity to bring it together, like the Noise app that puts touch-sensitive control on your iPhone, and Blend, an online collaboration platform.
Instead of waiting for the future of music making, in other words, ROLI will acquire and invent it.
The latest development is BLOCKS. They’re sleek, black boxes that work together wirelessly. And they’re different than what you usually see in three important ways.
First, because they’re modular, they aren’t the usual crowded combination of knobs and pads and buttons and faders that most often are the mark of music gear. Second, they’re arguably the first music gear that seems to look comfortable sitting next to something with the industrial refinement of an iPad Pro.
And lastly, most importantly, there’s nothing about this device that suggests it’s built for a musician. A unit called the Loop Block resembles the remote for an Apple TV. The Lightpad Block looks like a small light-up toy or piece of LED jewelry.
The idea is this: consumers buy the boxes they desire, then add them wirelessly in any combination to an iOS device that makes sound. Since they connect via Bluetooth, you skip the wires.
On their own, these gadgets do genuinely appear too simple. But the magic appears to be breathing intelligence into them with software. The Lightpad Block, true to ROLI’s core competency in touch interfaces, is sensitive to contact and gesture, evidently opening up various interactions. The Loop Block mainly serves as transport control, but seems to hold some potential when combined with other Blocks.
They’re small enough that they’d even make sense along a phone, not just a tablet. As a lot of the world’s markets shift to handheld platforms and not traditional computers, that could be a real sign of things to come.
Phone friendly. Darned cute devices.
Here you can watch the first two instruments in action. It’s the creative use of gestures that really sets this apart. We’ve seen ideas like that in apps or one-off experimental products, but here it’s part of a consumer-focused product – which is newsworthy.
It’s beautiful. I have just one caution. There’s one underlying assumption of our whole community that I sometimes wonder about. We assume that creating weird, new interfaces will appeal to non-expert musicians. To be slightly more pragmatic, I think it’s actually the creative music community who so often embrace that sort of experimentalism – and sometimes the general public are the tough sell. (There’s a reason they gravitate to stuff they know, like piano keyboards and guitars and drums – remember that a non-trivial portion of the population has actually learned on those instruments.)
On the other hand, the general public has embraced the multi-touch paradigm, so this could all be ripe for disruption.
Room to grow?
For now, you get only these two units. And they’re roughly the same price of some other entry-level music controller (even if they look nicer) – that’s US$$179 and $79 for Lightpad and Loop, respectively.
So I think the real verdict will be down to how the app and controllers evolve.
I’d also be hopeful that ROLI opens up this device to developers – that seems a no-brainer, given the company’s openness to third parties on Seaboard and its investment in development technology JUCE. In other words, even if you aren’t sold on ROLI’s own app, another app might change your mind. (I’d like to use the Lightpad Block with the Elastic Drums drum machine, for instance.) Also, since they’re Bluetooth enabled, I hope we’ll be able to use them on desktop computers, too. These ideas may all be independent of the core market and simplicity, but … looking at the history of tech like multitouch, the Wiimote, and Kinect, advanced experimental applications have often informed the mainstream, too.
But there’s another reason to look to ROLI to boldly go where no other musical instrument manufacturers have gone before.
The company’s business is different. So while they say the same things about democratizing music for everyone (blah blah anyone can make music easily blah blah), they’ve got a business to back it up. And think over $50 million from investors like Foundry Group, Balderton Capital, and Founders Fund, plus even Universal Music Group. (Hey, if selling music doesn’t pan out, we may all be into making things like this.)
And most interesting about the ROLI Blocks initiative is that the whole thing will be sold exclusively direct from the company and through Apple. That’s right – Apple Stores will have these gadgets, in retail locations and online.
I think that’s potentially a game changer. Music stores are unfriendly places for a lot of people. Heck, even some of us actual musicians avoid going to some of them like the plague, thanks to unfriendly staff and people playing Stairway to Heaven.
But seeing these alongside iPads could open up physical hardware to a whole new audience.
Having thoroughly trashed Apple for the last days for narrowing the appeal of their desktop Mac line, I have the reverse take on their mobile iOS stuff. I think they remain in just the right position to turn people on to making music on their mobile gadgets. I’m not being hypocritical somehow, either – you can contrast the absence of inspiring desktop hardware with the much more inspiring combination of, say, an iPad Pro or an iPhone 7 with something like Lightpad Blocks.
Something like it, anyway. I’m excited to try these out in person. CDM will hopefully be visiting the folks at ROLI before the end of the year to catch up on all they’ve been doing. If there’s something you’d like to see us cover, questions you’d like to ask, even something you want to criticize, let us know.
There’s a shift on in the worldwide community of visualists, of the growing field of people using electronic visuals as a medium for performance, art, and inquiry. As these media become more mature and more international, there’s a renewed sense of closeness among practitioners. While big media festivals focus on novelty and show, these maker-to-maker events emphasize something else: craft.
This summer seemed a particularly historic moment for not one but two tools – each of them built by small teams who make art themselves. We already covered the Berlin gathering for Isadora, the visual performance tool that has rich connections to the world of dance. Now, we get to look at TouchDesigner, which has made a name for itself as the leading go-to tool for interactive event visuals (among other things). And maybe it’s fitting that unique tools would leave a particular mark. For artists, that particular piece of software is their axe, their main instrument, something to know inside and out.
I asked Isabelle Rousset from Derivative, TouchDesigner’s developer, to help prepare a report on the gathering in Moscow.
And picking someone from the team here works, because these gatherings are family affairs. This is summer camp for visual nerds – a retreat for people of passion. And I was ready for an exhaustive “what did you do last summer” report. We got it, in the form of obsessive notes on what happened and endless leads to check out yourself.
What TouchFest was about
Moscow’s MARS Center is a hub for the city’s electronic media community – one of a handful of places everybody meets to see the latest tech and visiting artists from around the world. And in this case, that same community got their hands dirty organizing the event.
Curiosity Media Lab’s Yan Kalnberzin and Eugene Afonin spear-headed the four-day event in July. It came against the backdrop of the cancellation of Outline Festival (the last afternoon of TouchFest) – but as such, was a reminder of the possibility still latent in Moscow’s scene.
Here’s Derivative on the experience:
It was solidly packed with masterclasses, lectures, demos, audio visual performances and a ‘marathon of interactive madness’. There was zero fluff!
Derivative’s Greg Hermanovic, Markus Heckmann and Isabelle Rousset who were there to participate were blown away on many fronts: the scope, quality, range of TouchDesigner projects and applications, community engagement and support (100+ in attendance), festival programming and schedule where workshops, lectures and masterclasses ran in parallel for multiple days, the generosity and proficiency of the festival organizers for putting together such a BIG and exciting festival (while working on a major project for clients i.e. they didn’t sleep much if at all for days), the MARS Center who provided the fantastic facilities and staffing…. The volunteers and of course the performers and participants whose work and energies were very far out and intense. A ‘hotbed’ of TouchDesigner
I’m still surprised that around 250 people came))
This was a chance to learn if Moscow is ready for educational events like this. And we wanted to see how many people are really interested in the subject. I’m amazed how many people came from around Russia – three people from Krasnodar, several from Izhevsk, from Novosibirsk, and some other pretty far away cities. Ed.: Yeah – uh, Googling those myself!
We are super happy and thankful to MARS Center and their technical team. They totally made half of the event. And of course to all the speakers – mostly friends, but not all of them – that they agreed to perform for free.
Deep in technical education.
TouchFest’s organizers were themselves long-time TouchDesigner instructors. Yan Kalnberzin and Evgeniy Afonin have been teaching the tool since 2012 – even 3- and 6-month courses. These also culminated in presentations:
Here at Curiosity Media Lab we often get letters about education from scratch.
As we organize such an event, inviting Touch masters from Russia and abroad, we want to give you a chance to understand at least on a basic level what they are speaking about.
Our 8 hour masterclass will start from the very beginning – fields of application, nodes, logic, interface, contexts of the program.
Yan and Evgeniy were teaching again. Other highlights:
Markus Heckmann presented an eight-hour class entitled “Developing in TouchDesigner: Python Extensions and Custom Parameters.” “The master class examined how TouchDesigner’s new Custom Parameters and Extensions helps develop complex functionality within the environment of visual programming with networks of nodes,” says Isabelle.
“Probably the longest-standing” TouchDesigner user and teacher Andrew Quinn taught a course that incorporated audio and gesture, “Sound-Reactive Visuals and Gesture Tracking.”
Recorded or live sound could then animate movement, light, and color – in two and three dimensions. Gesture tracking transforms the VJ “into a puppeteer.” Andrew is applying the same concept to coursework with kids.
He played the closing audiovisual concert with composer Nikolai Popov and six musicians from the Russian Conservatory.
Conservatory musicians join the AndrewQuinn / NikolayPopov AV performance.
Total immersive interactive chaos intensive
Three days in the Vostochnaya Gallery turned into a circus of interactive hacking, thanks to Ildar Yakubov (someone I’ve also had the pleasure to know).
I like the chaos aspect. As Isabelle describes the event, “NEITHER SEW NOR FASTEN”:
[The marathon] showed the world what pure, boundaryless and unpredictable total interactivity looks like. (At one point, I’m told, a doctor in blue scrubs came down saying he was performing an operation upstairs and the floor was shaking!) There was also a flood!
But for three days the kids worked tirelessly, connecting all kinds of materials to TouchDesigner, including:
Microsoft Kinect 2
Intel RealSense F200
Intel RealSense SR300
Arduinos with a variety of peripherials
I/O devices / physical computing
Enttec Open DMX Ethernet
DMX controlled devices
Here’s the proposal-manifesto – slightly broken English here, sorry, but posted as-is (as maybe English can’t really describe everything they imagined)!
“Artists are really suppressed with the totality of technology and this way doomed to a both senseless and endless flirting with it” – some critics say “Until you are really familiar with the technology you are not able to reflect on it” – says the other “Turn off that weird shit or I call the police” – say the neighbours.
Armed with these tools, and using the unlimited potential Touchdesigner offers we will challenge an intuitive interface and the planned user experience.
After two days, we will create a real media art gezamkunstverk and put it to the mercy of the crowd.
Oh yes, about two days – the event will take place in a Hackathon mode: we will work tirelessly for two days. Some rumors say that on the second morning, participants will become familiar with the machine learning process and will learn to use this powerful tool in their practice!
Tools of the interactive madness.
Plotting the madness marathon.
For three days we have collected Krastinator – crazy device like a Rube Goldberg machine, the sole purpose of which was to stop the madness in and of itself. However, the rubber glove could not reach the big button, but successfully deceived Leap Motion posing as a real hand! The result of the analysis of the motion of non-existent bone defunct arms generate sound skeleton projected on screens, mixed with a picture of motorized cameras, controlled their own picture and includes a strobe on the floor rolled robo-ball, grazing contact microphones, vibration motor and the cooler is activated by a small Korg, smoke machine made his work, and all of this in an endless loop of interactions and relationships, secured by TouchDesigner.
Such collaboration with different people and skills backgrounds very useful in every sense of the activity, a huge thank you to all participants – you are super!
Lectures: praxis and philosophy
Thumbing through the notes from the lecture content, what strikes me is, you could navigate the full program without ever wanting to even use TouchDesigner user and still be really happy. There’s enough content dealing with theory and general technique. Or, on the other hand, you could come to TouchFest wanting to really hone up on skills – even from scratch – and have a ball, too.
To give these two areas physical space, the practical and technical were kept on downstairs and loftier topics literally above. (Nice.)
I was sent pages of notes, so let me summarize.
3D mapping technique with Andrew Flat, who has done everything from event design to VFX supervisor, and now is co-founder and technical director of AVEA company. https://www.facebook.com/FLaT4eRs
Roman Gavrilov of Curiosity Media Lab, who has spent years researching folk traditions in Russia and Ukraine and makes the leap from traditional craft to electronic media, covered LED control software GEOPIX. http://romanesco.org
SILA SVETA’s Dmitry Napolnov drew on an extensive background in events (from live motion capture to projector calibration) to present solutions to production tasks. https://www.facebook.com/cubicdisaster
A roundtable discussion looked at how to present LED lighting at low costs (from DMX controllers to DIY LED).
Dmitry Karpov covered the “battle” for VR, evaluating platforms and how the technology would shake up the landscape of designers.
Derivative’s Greg Hermanovic looked at TouchDesigner past, present, and (possible) future, in a talk called “A playground for design, and how it got that way”
Building audiovisual instruments: advanced ways to mix human impact with algorithms, TD+Ableton+Max4Live.
Anatomy of an installation
Coding a pixelshader(GLSL TOP) with DAT nodes, for “raymarching”
Dmitry Napolnov, lecture.
Upstairs philosophy topics
Procedural Functions as a New Canon – watch:
Anna Titovets (Intektra), of Russia’s Plums Fest, went deep into the question of live performance. I will paste the whole description here, as I think it matters:
Live video performance: from fractal tunnels to live cinema
Live video performance as a social and cultural phenomenon in contemporary media art.
Talk and talk about video performance as a format, which is a kind of marker for the changes occurring in today’s information society in the context of global changes taking place with the psychology of perception of the information society, with the communication methods with the audience and global technological development. How did the “vj” what “protovidzheing” Live Cinema differs from other genres, as changes in video performance linked with the development of technology and the changes taking place in society in general and in music in particular.
As part of the lecture will discuss the main current stylistic trends shaping and live video performances that exist at the moment (from 8-bit to glitch and Camp aesthetics of mash-up and political videointerventsy to generativa and nonlinear narrative Live Cinema).
And Isabelle’s own talk sounds fascinating:
A Timeline of TouchDesigner, or, How We Got From Touch001 to Russia
This talk proposes to untangle the co-joined history of TouchDesigner’s development in the context of
1. making music visuals for raves and
2. everything that followed. Isabelle will attempt to trace TouchDesigner’s development through time, technological advancements, historical events, historical achievements, grand projects, lightning bolt moments, community development and hard work.
Thinking out loud, or in response to your questions: Semantics and historical review – where the legs grow .. Art as a way of expression and how to make it? As a born and where to apply? Follow the trend and whether it is now fashionable?
Code as supreme and universal coauthor
Vadim Epstein is a top VJ – who happens to have a background in theoretical physics and 13 years consulting for HP, to boot. Now he makes generative work with code. http://eps.here.ru
The ZOOM ARQ AR-96 can be filed confidently under “wha?” in the annals of music tech. It’s a round, all-in-one groovebox with drum machine, loads of patterns and sounds, and synths. Oh yeah, and there’s a rechargeable, detachable doughnut/frisbee, uh, thing, which has velocity-sensitive touch sensors and responds to orientation so you can wave it around. Basically, it’s insane. But as at least one friend of mine suspected, it could also be insanely fun.
No English-language reviewers could really do this thing justice. No, for that we turn to musictrackjp – who do better demos, anyway. Sure, 97% of CDM’s readers likely don’t know what this guy is saying, but… no matter, as the demos are good enough that you can probably figure it out anyway. (Who needs talking? We’re musicians.)
Nice work, Katsunori UJIIE.
Speaking of lost in translation, I’m fairly certain DEEP HOUSE doesn’t mean what Zoom’s sound programmers think it does, but I digress.
You could also watch Zoom’s own English-language videos, too, for more detail.
I’m calling it the AR-96 (which sounds like a rare variant of assault rifle), but technically this is the ARQ Aero Rhythmtrak AR-96, a name that sort of seems to suggest Zoom is threatening to release a fleet of these UFO products.
Now, the actual specs are also, believe it or not, fairly powerful. There’s some engineering that allows you to grip the frisb– um, Ring Controller – without triggering the pads. And there are some 96 pads, with velocity, with scale mapping, with external MIDI output (meaning you aren’t limited to the internal sounds). It charges on the base station, then uses the three-axis accelerometer when detached for orientation.
If you do use the internal sounds, there are 79 kits, another 20 custom/user kit slots, a mixer, 33 part playback, a song mode that can chain up to 384 patterns, 468 sampled PCM waveforms, 70 synth sounds, plus filter, delay, reverb, envelopes, and modulation.
And it’s a sampler, too, with external audio input and 16-loop playback.
In fact, while its street price is hefty at first glance (around $500-600), right now the Zoom is weirdly more powerful than just about anything else in the category, until you shell out a lot more. You can load audio from SD card, external input, whatever, and constraints on number of simultaneous parts max out well after your ears would scream for someone to NOT PLAY 16 LOOPS AT THE SAME TIME.
In other words, behind this strange wireless doughnut concept and unusual circular housing, there’s a powerfully-spec’ed engine, just as a lot of rivals are comparatively simple. Now, whether you’d actually want that is another matter – but it is impressive. I have a feeling people are going to be lusting after this thing as a used device in no time. If you want to be the first on your block, though, you could go after now. And seriously, do read the specs – they’re surprising.
You can already connect your music software to MIDI devices. But why not Internet data, video, the weather, or physical worlds of Arduino and LEGO Mindstorms, too? With a new pack released today, making connections is a matter of adding some building blocks.
Arduino connected to Ableton Live. Photo courtesy Ableton.
The inclusion of Max inside Ableton Live means pretty much anything you can do in that open-ended patching environment you can do in Ableton Live. So in that sense, the free Max for Live Connection Kit actually doesn’t do anything you couldn’t do already. But what it does do is make a bunch of stuff ready to use out of the box. You can use these devices as-is, or take them as an example for your own patching if you choose.
The set looks like a boon for hackdays, education, or just trying something different in the studio. Even for experienced Max users, it’s nice having a set of idea-starters with that initial work done for you; it’s a huge motivator.
The biggest crowd pleaser is the LEGO MINDSTORMS EV3 brick module. Connect to the MINDSTORMS via Bluetooth, and you can receive sensor input and control motors, linking events to what’s happening in your Live set. Ableton were showing this functionality off in particular in preview days held at Berlin’s CTM Festival last month.
A Mindstorms play area, seen at CTM Festival last month (with some happy Abletons running motors and sensors)!
There are a number of devices dedicated to handling OSC (OpenSoundControl):
An OSC monitor
A device for receiving data from TouchOSC on the iPad (which also shows the active layout)
An example that sends MIDI data to OSC (with an accompanying Processing visualization example for receiving that data)
A Leap Motion example device for translating gestural data into Live
That’s pretty far from everything you’d want to do with OSC, but it’s a good starting point; because OSC is by definition open-ended, you might want to make your own device based on one of these.
There are two Arduino devices:
One device receives sensor data and sends parameters to LEDs or motors with an Arduino Uno
One is designed for use with the ins and outs of the Arduino module in littleBits
And you get three additional devices for data and video:
JSON Weather queries the weather over the Internet and then sonifies it – an example of how to fetch and parse data from the Web.
JSON Video is also an Internet example, but pulls #ableton-tagged videos from Vine.
Camera uses a webcam in Live and does some basic motion detection for webcam control of Live.
The Weather: now not just a reason to stay in and work on music, but also an Ableton Device!
All of these devices are available on GitHub, which means Ableton can keep them up to date, but Max users can make their own modifications, too.
When you install the Connection Kit, you’ll find all of these devices grouped in Packs. There’s a brief help summary of what they all do, with full documentation on GitHub (also meaning it can be kept up to date).
Open up those patches, and you can learn a bit about how to do this stuff in Max – or modify them for your own purposes.
I’m really curious to see what you’ll do with it. And this sort of functionality is a natural for Max for Live – there’s no logical way to build it natively into a host, but giving you some building blocks to play with your ideas fits perfectly. Let us know what you think – or if you have your own favorite Max creations for working with Ableton.
“Gesture” is a term that gets tossed about regularly in modern interaction design. But to me, the word is most deeply associated with classical music – and the gestures that first brought me to music, the piano. In this video for TED@BCG, I got to talk about that and why I think it can inform design through today’s newest interfaces.
In rapid-fire form, obviously more could be said about this.
The software involved: NI Mate, which I need to revisit – it’s gotten more sophisticated since what you see here
I am now deep into another TED project – TEDxESA at the European Space Agency – but while having a quick coffee, it seemed long overdue to share the last TED project, in front of the audience of TED’s TED@BCG (an official TED-curated event organized with Boston Consulting Group). I’ll be honest: I was a bit slow to share this partly because I would have loved to have had more time to prepare that talk and performance. Performing onstage at a conference is a unique challenge, on another order of magnitude when adding computer vision.
Anyway, thought I might share it (having put off doing so) to see what reactions were.
And I will say, it was a pleasure to work with the TED organization and see how they operate behind the scenes. It felt like having a personal trainer for talks, even in just a brief time. And now I look forward to an entirely different setting, with the folks at ESA. Stay tuned – suffice to say, in the midst of posting this, we’re exploring the world of space exploration and research through the medium of sound. And I think there will be a lot more to say very soon. So back to that – and greetings from south Holland.
We’ve seen apps made exclusively for touch devices like the iPad. And we’ve seen very basic touch support in desktop apps. But Bitwig Studio 1.3 is both.
Also, is Bitwig actually trolling Mac fans, or Apple? Because Bitwig is touting the fact that OS X will at least get its new “E-Cowbell device.” (I’m not making this up.)
For multi-touch devices on Windows and Linux (yes, Linux) – plus a specially-optimized profile for Microsoft Surface Pro and Surface Book – Bitwig has a lot of new touch features. They aren’t just responding to touch events; they’re going further.
Full multi-touch support. This is, of course, essential. It doesn’t work on OS X – there literally isn’t a model for processing the events – but it does open up some possibilities even on Linux.
Here’s what that looks like when mixing:
Radial menu and gestures. To try to make touch more useful, Bitwig are also adding a shortcut menu, for quick gestural access to settings for devices, drums, clips, arrangement, notes, and tracks. I really have no idea whether I’m convinced by this without having used it, but I’m intrigued. It also represents a different approach than Ableton’s, which has been to focus on moving control to physical hardware (Push). Clearly, there’s an argument for each approach – there’s something different about getting away from a display and using something tactile – but it’s nice to see something happening with the touch/display end of the equation.
Looking at this at first, it looked like a separate remote-control layer. In practice, though, that “radial menu” is maybe better thought of as a heads-up reference to what gestures do. The result can be really fast gestural editing, as seen here in arrangement:
I’m really keen to try this, especially as arranging with a mouse is painful. (It’s even worse when working with two people, as my studio colleague Nerk can attest.)
You can play right on the interface. Rather than go to a separate iPad remote (as Apple does with its own Logic and GarageBand), Bitwig are building a keyboard right into the tool so you can play directly. It’s like having a hardware controller or an iPad app built into your display.
There’s a built-in drum editor. There’s a pad layout for playing drum pads, as well, plus some touch editing options.
It’s hard to imagine what the evolution of the synthesizer would have been without Leon Theremin.
For one, it was Theremin’s invention that first captivated Robert Moog. Theremin kits were Dr. Moog’s first product and many would say, his first electronic instrumental love. That impact was significant, too, on a whole generation – actually, even my own father made building a kit Theremin one of his early experiences with electronics.
The fall of the Soviet Union still has ripples felt in the electronic music world today. And surely there’s no more poignant moment in the intertwining of post-Cold War history with musical invention as Leon Theremin’s 1991 visit to the USA – at 95 years of age.
Robert Moog wrote up that experience for Keyboard Magazine (USA), along with writer Olivia Mattis. Much of the history will be familiar, but it’s moving to read about the event.
The gathering with Lev Sergeyevich Termen may have been the single greatest convergence of the 20th century’s electronic inventors ever – John Chowning (CCRMA, FM synthesis), Don Buchla, Roger Linn, Bob Moog, Tom Oberheim, Max Mathews, and Dave Smith were all there. (It’s also remarkable to think how much Chowning, Linn, Oberheim, and Smith continue to contribute as teachers and inventors today, not to mention the ongoing contributions of Moog, Buchla, and Theremin instruments.)
And of course, because of history (hello, KGB), these inventors had never really had the opportunity to meet face to face. They had “met” through their instruments. Moog and Mattis also write eloquently of ghostly guests:
For the audience, the thread of continuity and tradition linking Theremins early instruments with the world of synthesizers and MIDI is clear and strong. If you looked hard, you could almost see the spirits of Maurice Martenot, Friedrich Trautwein (inventor of the Trautonium), and Laurens Hammond joining the audience in frenzied applause.
The Thereminists were notable, too – not only daughter Natasha Termen, but Clara Rockmore, reunited with Mr. Termen. Max played with Natasha, via his “Radio Drum” – a full decade before those sorts of gestural interfaces would enter popular consciousness (via Minority Report, the Wii, Kinect, and so on).
And we get Termen, the ‘cello player turned inventor turned KGB asset, in his own words. On the reason for the instrument:
The idea first came to me right after our Revolution, at the beginning of the Bolshevik state. I wanted to invent some kind of an instrument that would not operate mechanically, as does the piano, or the cello and the violin, whose bow movements can be compared to those of a saw. I conceived of an instrument that would create sound without using any mechanical energy, like the conductor of an orchestra.
I became interested in bringing about progress in music, so that there would be more musical resources, I was not satisfied with the mechanical instruments in existence, of which there were many. They were all built using elementary principles and were not physically well done, I was interested in making a different kind of instrument. And I wanted, of course, to make an apparatus that would be controlled in space, exploiting electrical fields, and that would use little energy. Therefore I used electronic technology to create a musical instrument that would provide greater resources.
And there’s more. There’s a Theremin lesson for Lenin, with whom Termen claimed kindred interests because the Soviet leader was “interested in how the whole world is created.” And there was Albert Einstein – yes, that Albert Einstein – taking up residence in the Termen studio in order to explore visual music and synesthesia:
Einstein was interested in the connection between music and geometrical figures: not only color, but mostly triangles, hexagons, heptagons, different kinds of geometrical figures. He wanted to combine these into drawings. He asked whether he could have a laboratory in a small room in my house, where he could draw.
There are electric cellos made for Stokowski and Varese, and the tale of imprisonment (along with Tupolev) and nightmare suspicion under Stalin, the removal of electronic instruments from the Conservatory in the late 60s because electricity is only “for electrocution.” Well worth reading the piece in its entirety:
But no reason to feel overly nostalgic or lost in the shadow of history. I think what Termen says about music from space and electrical fields is just as evocative today as it was a century ago – to say nothing of an Einsteinian flatland of geometric music. In a reversal of the Yogi Berra quote “the future ain’t what it used to be,” maybe it’s even more.