We’ve been waiting, but now the waiting is done. Propellerhead has added the VST performance boost it had promised to Reason users – meaning plug-ins now benefit from Reason 10’s patchable racks.
I actually flew to Stockholm, Sweden back in the dark days of December to talk to the engineers about this update (among some other topics). Short version of that story: yeah, it took them longer than they’d hoped to get VST plug-ins operating as efficiently as native devices in the rack.
If you want all those nitty-gritty details, here’s the full story:
But now, suffice to say that the main reason for the hold-up – Reason’s patchable, modular virtual rack of gear – just became an asset rather than a liability. Now that VSTs in Reason perform roughly as they will in other DAWs, what Reason adds is the ability to route those plug-ins however you like, in Reason’s unique interface.
Combine that with Reason’s existing native effects and instruments and third-party Rack Extensions, and I think Reason becomes more interesting as both a live performance rig and a DAW for recording and arranging than before. It could also be interesting to stick a modular inside the modular – as with VCV Rack or this week’s Blocks Base and Blocks Prime from Native Instruments.
Anyway, that’s really all there is to say about 10.3 – it’s what Propellerhead call a “vitamin injection” (which, seeing those dark Swedish winters, I’m guessing all of them need about now.
This also means the engineers have gotten over a very serious and time-consuming hurdle and can presumably get onto other things. It’s also a development for the company that they’ve been upfront in talking about a flaw both before, during, and concluding development – and that’s welcome from any music software maker. So props to the Props – now go get some sunshine; you’ve earned it. (and the rest of us can tote these rigs out into the park, too)
Max 8 – and by extension the latest Max for Live – offers some serious powers to build your own sonic and visual stuff. So let’s tune in some videos to learn more.
The major revolution in Max 8 – and a reason to look again at Max even if you’ve lapsed for some years – is really MC. It’s “multichannel,” so it has significance in things like multichannel speaker arrays and spatial audio. But even that doesn’t do it justice. By transforming the architecture of how Max treats multiple, well, things, you get a freedom in sketching new sonic and instrumental ideas that’s unprecedented in almost any environment. (SuperCollider’s bus and instance system is capable of some feats, for example, but it isn’t as broad or intuitive as this.)
The best way to have a look at that is via a video from Ableton Loop, where the creators of the tech talk through how it works and why it’s significant.
In this presentation, Cycling ’74’s CEO and founder David Zicarelli and Content Specialist Tom Hall introduce us to MC – a new multi-channel audio programming system in Max 8.
MC unlocks immense sonic complexity with simple patching. David and Tom demonstrate techniques for generating rich and interesting soundscapes that they discovered during MC’s development. The video presentation touches on the psychoacoustics behind our recognition of multiple sources in an audio stream, and demonstrates how to use these insights in both musical and sound design work.
The patches aren’t all ready for download (hmm, some cleanup work being done?), but watch this space.
If that’s got you in the learning mood, there are now a number of great video tutorials up for Max 8 to get you started. (That said, I also recommend the newly expanded documentation in Max 8 for more at-your-own-pace learning, though this is nice for some feature highlights.)
dude837 has an aptly-titled “delicious” tutorial series covering both musical and visual techniques – and the dude abides, skipping directly to the coolest sound stuff and best eye candy.
Yes to all of these:
There’s a more step-by-step set of tutorials by dearjohnreed (including the basics of installation, so really hand-holding from step one):
Suffice to say that also could mean some interesting creations running inside Ableton Live.
It’s not a tutorial, but on the visual side, Vizzie is also a major breakthrough in the software:
That’s a lot of looking at screens, so let’s close out with some musical inspiration – and a reminder of why doing this learning can pay off later. Here’s Second Woman, favorite of mine, at LA’s excellent Bl__K Noise series:
Arturia have followed up their hit “3 Filters…” and “3 Preamps You’ll Actually Use” with the inevitable trio of compressors – but as with the other bundles, there are some twists (and lower intro prices on now).
Before they expanded into doing their own MIDI controllers and synth hardware, Arturia rose to prominence on their modeling chops. And they have tended to spin those modeling engines and competency in recreating vintage gear into spin-off products. The trick with the “3 [things] You’ll Actually Use” series has been rising above the crowd of vintage remakes now available to music producers. So that’s been about doing two things: one, picking three blockbusters to reproduce, and two, adding some functionality extras that lets producers get creative with the results.
And that’s to me what has made the series interesting – while lots of vendors will sell you reproductions of classic studio equipment, these have been ones you might well use in the production process. It’s not only about perfecting a recording or mix, but also about integrating into your creative process as you’re developing ideas.
The compressors trio go that route, too – so in addition to using these routinely in mixing or mastering, you can also use them for some inventive sidechain or saturation.
The three compressors getting the Arturia treatment – and the circuitry inside:
The 1176 is pretty ubiquitously desired at this point – and of course among other recreations you can keep it in the family of creator Bill Putnam Sr. and try Universal Audio’s own creation. It’s something you can use for subtle tonal shifts even at lower levels, in addition to cranking up compression if you want. So why add another reproduction to the pile? Arturia has added a “link” button for automatic volume leveling if you want – giving you the 1176 sound but more modern behavior on demand.
And you can use the 1176 as a sidechain. Oh – wait, that’s really huge. And there’s a creative “Time Warp” feature with pre-delay. So thanks to the fact that Arturia aren’t being quite as precious with the historical design as some of their rivals, you can choose either an “authentic” 1176 recreation, or something that’s 1176-ish but does things that were impossible on the original analog hardware.
It’s surprising enough for the 1176 to be new again, but the other models here have some similar ideas.
Next, you’ve got the “Over Easy” 165A, an essential compressor in a lot of studios, which has both some nice dirty, gritty timbral character of its own and punchy processing plus Mid/Side processing. For this model, Arturia have introduced a whole new panel of additional controls that fold out when you want them in the UI.
Don’t be fooled by the skeumorphic knobs; the original DBX didn’t have these options. That also includes their “Time Warp” pre-delay, convenient side-chaining (here with an easy “manual mode” trigger so you can preview what it’ll sound like), and now an integrated EQ. That EQ is modeled on SSL-style channels, so it’s a bit like having a pre-configured mixer rig to use with your sidechaining.
The STA-Level is maybe the most interesting of the three as far as rarity. It also gets (optional) modernization, with an input-output link for automatic leveling, a parallel compression mode that’s integrated with the software (plus an easy “mix” knob for adjusting how much parallel compression you want to hear), and sidechaining.
All in all, it’s an intriguing approach. On one hand, you get panels that look and operate and sound more like the original than a lot of software models at the low end of the price range. (For instance, the compressors added recently to Logic Pro X, while free, are more loose impressions than authentic recreations.)
But on the other, and here’s where Arturia clearly has an edge, you get new sidechaining and auto-leveling and other features that make these more fun to use in modern contexts and easier to drop into your creative flow.
Sidechaining these kinds of compressor models alone I think is a win; the convenience of the UIs and the fact that these are native on any platform to me makes them invaluable – maybe even compared to the existing filter and preamp offerings.
I’ve been playing around with them a bit already; I’m especially curious if I can run a couple in a live context – will report back on that. But I’m already impressed on sound and functionality.
Everything is on sale, so if you own existing Arturia stuff, you could get these for as little as $/EUR 49 (or half off if you’re new to the series), or in discounted bundles. You can also buy the compressors individually, if there’s one that really catches your fancy.
Plus, there are some new tutorials to get you started:
pd knobs is a knob controller for MIDI. It’s built with Teensy with open source code – or you can get the pre-built version, with some pretty, apparently nice-feeling knobs. And here it is with the free software Pd + AUTOMATONISM – proof that you don’t need to buy Eurorack just to go modular.
And that’s relevant, actually. Laptops can be had for a few hundred bucks; this controller is reasonably inexpensive, or you could DIY it. Add Automatonism, and you have a virtually unlimited modular of your own making. I love that Eurorack is supporting builders, but I don’t think the barrier to entry for music should be a world where a single oscillator costs what a lot of people spend in a month on rent.
And, anyway, this sounds really cool. Check the demo:
From the creator, Sonoclast:
pd knobs is a 13 knob MIDI CC controller. It can control any software that recognizes MIDI CC messages, but it was obviously designed with Pure Data in mind. I created it because I wanted a knobby interface with nice feeling potentiometers that would preserve its state from session-to-session, like a hardware instrument would. MIDI output is over a USB cable.
For users of the free graphical modular Pd, there are some ready-to-use abstractions for MIDI or even audio-rate control. You can also easily remap the controllers with some simple code.
What’s to say a music idea can’t be both a tool and a tape, an instrument someone could play or an album they can get lost in? Puremagnetik are launching their new experimental label with two free tools that let you keep the drones and grains and ambient soundscapes flowing.
There’s a bunch of hype this week because Warner Music signed an algorithm. And in turn with everyone abusing the term “AI,” you might well think that a computer has automated a composer’s job. Except that’s not what happened – in the long tradition of algorithmic music, a group of composers applied their ideas to the development of software. One of the first apps launched for the iPhone, in fact, was the Brian Eno – Peter Chilvers “Bloom.” Endel has more in common with Bloom, I’d argue, than it does some dystopia where unseen, disembodied AI come to rob you of your lucrative ambient music recording contract. (At least, we’re not there yet. Endel is here in Berlin; I hope to talk to them soon – what they’ve done sounds very interesting, and maybe not quite what the press have reported.) Bloom in turn was a follow-up to Eno’s software-based generative music releases. Ableton co-founders Gerhard and Robert released software in the 90s, too.
So let’s talk about the role of musician as blurred with the role of instrument builder. Soundware and software shop Puremagnetik is made by musicians; founder Micah Frank was moonlighting in sound design for others as he worked on his own music. While this may come as shocking news to some, it turns out for many people, selling music tools is often a better day job than selling music or music performances. (I hope you were sitting down for that bombshell. Don’t tell my/your/anyone’s parents.)
But there are many ways to express something musically. Many of us who love tools as we do love playing live and recording and listening do so because all of these things embody sound and feeling.
It’s fitting, then, that Puremagnetik are launching their own record label to house some of the recorded experiments – Puremagnetik Tapes, which already has some beautiful music on cassette and as digital downloads.
And the perfect companion to those albums is these two free plug-ins. Like the label, they promise a trip for the mind.
The two first tapes (also available as digital)… gorgeous sound worlds to lose yourself in on loop.
The label announces it will focus on “experimental, ambient and acousmatic music.” That already yields two enchanting ambient forays. “Into a Bright Land” is in turns crystalline and delicate, warm and lush as a thick blanket. It’s Micah Frank himself, releasing under his Larum moniker. The musical craft is a digital-analog hybrid, part synths and tape machines – the kind the company has been known for sampling in its sound work – and partly Micah’s intricate custom coding work in the free environment Csound.
To accompany Into a Bright Land, there’s the plug-in “Expanse,” a “texture generator,” with a combination of “texture tone” filter, spectral blurring, adjustable pitch shift, and a healthy supply of noise generation and space.
Its drones and sonic landscapes draw from that same world.
Tyler Gilmore aka BlankFor.ms has crafted “Works for Tape and Piano,” pushing each instrument to its most vulnerable place, the tape itself becoming instrument, sounding almost as if at the point of a beautiful breakdown.
Since you can’t just borrow Tyler’s tape machines and such, Driftmaker is a digital equivalent – a “delay disintegration” device. Add your own audio, and the plug-in will model analog deterioration. The artist himself supplies the presets. Again, you have plenty of control – “parse” which sets the record buffer, “chop” which determines how much to recall, and then controls for delay, modulation, filtering, and wet/dry.
Both plug-ins are free with an email address or Gumroad login.
…and the plug-ins, each created to aesthetically accompany the albums.
There’s a pattern here, though. Far from a world where artists remove themselves from craft or automate the hard work, here, artists relish in getting close to everything that makes sound. They make music the hard way because each element of DIY is fun. And then they share that same fun. It might well be the opposite of the narrative we’re given about AI and automation (and I suspect that may also mean artists don’t approach machine learning for music in the way some people currently predict).
Or, well, even if you don’t believe that, I think you’ll easily lose whole evenings with these albums and plug-ins alike.
Native Instruments’ Massive synth defined a generation of soft synths and left a whole genre or two in its wake. But its sequel remains mysterious. Now the company is revealing some of what we can expect.
First, temper your expectations: NI aren’t giving us any sound samples or a release date. (It’s unclear whether the blog talking about “coming months” refers just to this blog series or … whether we’re waiting some months for the software, which seems possible.)
What you do get to see, though, is some of what I got a preview of last fall.
After a decade and a half, making a satisfying reboot of Massive is a tall order. What’s encouraging about Massive X is that it seems to return to some of the original vision of creator Mike Daliot. (Mike is still heavily involved in the new release, too, having crafted all 125 wavetables himself, among other things.)
So Massive X, like Massive before it, is all about making complex modulation accessible – about providing some of the depth of a modular in a fully designed semi-modular environment. Those are packaged into a UI that’s cleaner, clearer, prettier – and finally, scalable. And since this is not 2006, the sound engine beneath has been rewritten – another reason I’m eager to finally hear it in public form.
Massive X is still Massive. That means it incorporates features that are now so widely copied, you would be forgiven forgetting that Massive did them first. That includes drag-and drop modulation, the signature ‘saturn ring’ indicators of modulation around knobs, and even aspects of the approach to sections in the UI.
What’s promising is really the approach to sound and modulation. In short, revealed publicly in this blog piece for the first time:
Two dedicated phase modulation oscillators. Phase modulation was one of the deeper features of the original – and, if you could figure out Yamaha’s arcane approach to programming, instruments like the DX7. But now it’s more deeply integrated with the Massive architecture, and there’s more of it.
Lots of noise. In addition to those hundred-plus wavetables for the oscillators, you also get dozens of noise sources. (Rain! Birdies!) That rather makes Massive into an interesting noise synth, and should open up lots of sounds that aren’t, you know, angry EDM risers and basslines.
New filters. Comb filters, parallel and serial routing, and new sound. The filters are really what make a lot of NI’s latest generation stuff sound so good (as with a lot of newer software), so this is one to listen for.
New effects algorithms. Ditto.
Expanded Insert FX. This was another of the deeper features in Massive – and a case of the semi-modular offering some of the power of a full-blown modular, in a different (arguably, if you like, more useful) context. Since this can include both effects and oscillators, there are some major routing possibilities. Speaking of which:
Audio routing. Route an oscillator to itself (phase feedback), or to one another (yet more phase modulation), and make other connections you would normally expect of a modular synth, not necessarily even a semi-modular one.
Modulators route to the audio bus, too – so again like modular hardware, you can treat audio and modulation interchangeably.
More envelopes. Now you get up to nine of these, and unique new devices like a “switcher” LFO. New “Performers” can use time signature-specific rhythms for modulation, and you can trigger snapshots.
It’s a “tracker.” Four Trackers let you use MIDI as assignable modulation.
Maybe this is an oversimplification, but at the end of the day, it seems to me this is really about whether you want to get deep with this specific, semi-modular design, or go into a more open-ended modular environment. The tricky thing about Massive X is, it might have just enough goodies to draw in even the latter camp.
And, yeah, sure, it’s late. But … Reaktor has proven to us in the past that some of the stuff NI does slowest can also be the stuff the company does best. Blame some obsessive engineers who are totally uninterested in your calendar dates, or, like, the forward progression of time.
For a lot of us, Massive X will have to compete with the fact that on the one hand, the original Massive is easy and light on CPU, and on the other, there are so many new synths and modulars to play with in software. But let’s keep an eye on this one.
Hey, at least I can say – I think I was the first foreign press to see the original (maybe even the first press meeting, full stop), I’m sure because at the time, NI figured Massive would appeal only to CDM-ish synth nerds. (Then, oops, Skrillex happened.) So I look forward to Massive X accidentally creating the Hardstyle Bluegrass Laser Tag craze. Be ready.
Day in, day out, a lot of producers spend a lot of time editing in Ableton Live. Here’s a free tool that automates some common tasks so you can work more quickly – easing some FL Studio envy in the process.
This one comes to us from Madeleine Bloom’s terrific Sonic Bloom, the best destination for resources on learning and using Ableton Live. Live Enhancement Suite is Windows-only for the moment, but a Mac version is coming soon.
The basic idea is, LES adds shortcuts for producers, and some custom features (like sane drawing) you might expect from other tools:
Add devices (like your favorite plug-ins) using a customizable pop-up menu of your favorites (with a double right-click)
Draw notes easily with the ~ key in Piano Roll.
Pop up a shortcut menu with scales in Piano Roll
Add locators (right shift + L) at the cursor
Pan with your mouse, not just the keyboard (via the middle mouse button, so you’ll need a three-button mouse for this one)
Save multiple versions (a feature FL Studio users know well)
Ctrl-shift-Z to redo
Alt-E to view envelope mode in piano roll
And there’s more customizations and multi-monitor support, too.
Ableton are gradually addressing long-running user requests to make editing easier; Live 10.1 builds on the work of Live 10. Case in point: 10.1 finally lets you solo a selected track (mentioned in the video as previously requiring one of these shortcuts). But it’s likewise nice to see users add in what’s missing.
Oh, and… you’re totally allowed to call it “Ableton.” People regularly refer to cars by the make rather than the model. We know what you mean.
Here’s a video walking through these tools and the creator Dylan Tallchief’s approach:
K-Devices have brought alien interfaces and deep modulation to Max patches – now they’re doing plug-ins. And their approach to delay and tremolo isn’t quite like what you’ve seen before, a chance of break out of the usual patterns of how those work. Meet TTAP and WOV.
“Phoenix” is the new series of plug-ins from K-Devices, who previously had focused on Max for Live. Think equal parts glitchy IDM, part spacey analog retro – and the ability to mix the two.
TTAP is obviously both a play on multi-tap delay and tape, and there’s another multi-faceted experiment with analog and digital effects.
At its heart, there are two buffers with controls for delay time, speed, and feedback. You can sync time controls or set them free. But the basic idea here is you get smooth or glitchy buffers warping around based on modulation and time you can control. There are some really beautiful effects possible:
WOV is a tremolo that’s evolved into something new. So you can leave it as a plain vanilla tremolo (a regular rate amplitude shifter), but you can also adjust sensitivity to responding to an incoming signal. And there’s an eight-step sequencer. There are extensive controls for shaping waves for the effect, and a Depth section that’s well, deep – or that lets you turn this tremolo into a kind of gate.
These are the sorts of things you could do with a modular and a number of modules, but having it in a single, efficient, integrated plug-in where you get straight at the controls without having to do a bunch of patching – that’s something.
Right now, each plug-in is on sale (25% off) for 45EUR including VAT (about forty two bucks for the USA). 40% off if you buy both. Through March 17.
In the original modular synth era, your only way to capture ideas was to record to tape. But that same approach can be liberating even in the digital age – and it’s a perfect match for the open VCV Rack software modular platform.
Competing modular environments like Reaktor, Softube Modular, and Cherry Audio Voltage Modular all run well as plug-ins. That functionality is coming soon to a VCV Rack update, too – see my recent write-up on that. In the meanwhile, VCV Rack is already capable of routing audio into a DAW or multitrack recorder – via the existing (though soon-to-be-deprecated) VST Bridge, or via inter-app routing schemes on each OS, including JACK.
Those are all good solutions, so why would you bother with a module inside the rack?
Well, for one, there’s workflow. There’s something nice about being able to just keep this record module handy and grab a weird sound or nice groove at will, without having to shift to another tool.
Two, the big ongoing disadvantage of software modular is that it’s still pretty CPU intensive – sometimes unpredictably so. Running Rack standalone means you don’t have to worry about overhead from the host, or its audio driver settings, or anything like that.
Big thanks to chaircrusher for this tip and some other ones that informed this article – do go check his music.
Type “recorder” into the search box for modules, and you’ll see different options options from NYSTHI – current at least as of this writing.
2 Channel MasterRecorder is a simple stereo recorder.
2 Channel MasterReocorder 2 adds various features: monitoring outs, autosave, a compressor, and “stereo massaging.”
Multitrack Recorder is an multitrack recorder with 4- or 8-channel modes.
The multitrack is the one I use the most. It allows you to create stems you can then mix in another host, or turn into samples (or, say, load onto a drum machine or the like), making this a great sound design tool and sound starter.
This is creatively liberating for the same reason it’s actually fun to have a multitrack tape recorder in the same studio as a modular, speaking of vintage gear. You can muck about with knobs, find something magical, and record it – and then not worry about going on to do something else later.
The AS mixer, routed into NYSTHI’s multitrack recorder.
Set up your mix. The free included Fundamental modules in Rack will cover the basics, but I would also go download Alfredo Santamaria’s excellent selection , the AS modules, also in the Plugin Manager, and also free. Alfredo has created friendly, easy-to-use 2-, 4-, and 8-channel mixers that pair perfectly with the NYSTHI recorders.
Add the mixer, route your various parts, set level (maybe with some temporary panning), and route the output of the mixer to the Audio device for monitoring. Then use the ‘O’ row to get a post-fader output with the level.
(Alternatively, if you need extra features like sends, there’s the mscHack mixer, though it’s more complex and less attractive.)
Prep that signal. You might also consider a DC Offset and Compressor between your raw sources and the recording. (Thanks to Jim Aikin for that tip.)
Configure the recorder. Right-click on the recorder for an option to set 24-bit audio if you want more headroom, or to pre-select a destination. Set 4- or 8-track mode with the switch. Set CHOOSE FILE if you want to manually select where to record.
There are trigger ins and outs, too, so apart from just pressing the START and STOP buttons, you can either trigger a sequencer or clock directly from the recorder, or visa versa.
Record away! And go to town… when you’re done, you’ll get a stereo WAV file, or a 4- or 8-track WAV file. Yes, that’s one file with all the tracks. So about that…
Splitting up the multitrack file
This module produces a single, multichannel WAV file. Some software will know what to do with that. Reaper, for instance, has excellent multichannel support throughout, so you can just drag and drop into it. Adobe’s Audition CS also opens these files, but it can’t quickly export all the stems.
Software like Ableton Live, meanwhile, will just throw up an error if you try to open the file. (Bad Ableton! No!)
It’s useful to have individual stems anyway. ffmpeg is an insanely powerful cross-platform tool capable of doing all kinds of things with media. It’s completely free and open source, it runs on every platform, and it’s fast and deep. (It converts! It streams! It records!)
Installing is easier than it used to be, thanks to a cleaned-up site and pre-built binaries for Mac and Windows (plus of course the usual easy Linux installs):
That’s worth keeping around, too, since it can also mix and monitor your output. (No Linux version, though.)
Bonus tutorial here – the other thing apart from recording you’ll obviously want with VCV Rack is some hands-on control. Here’s a nice tutorial this week on working with BeatStep Pro from Arturia (also a favorite in the hardware modular world):
I really like this way of working, in that it lets you focus on the modular environment instead of juggling tools. I actually hope we’ll see a Fundamental module for the task in the future. Rack’s modular ecosystem changes fast, so if you find other useful recorders, let us know.
Gamers’ interest may come and go, but artists are always exploring the potential of computer vision for expression. Microsoft this month has resurrected the Kinect, albeit in pricey, limited form. Let’s fit it to the family tree.
Time flies: musicians and electronic artists have now had access to readily available computer vision since the turn of this century. That initially looked like webcams, paired with libraries like the free OpenCV (still a viable option), and later repurposed gaming devices from Sony and Microsoft platforms.
And then came Kinect. Kinect was a darling of live visual projects and art installations, because of its relatively sophisticated skeletal tracking and various artist-friendly developer tools.
A full ten years ago, I was writing about the Microsoft projectand interactions, in its first iteration as the pre-release Project Natal. Xbox 360 support followed in 2010, Windows support in 2012 – while digital artists quickly hacked in Mac (and rudimentary Linux) support. An artists in music an digital media quickly followed.
For those of you just joining us, Kinect shines infrared light at a scene, and takes an infrared image (so it can work irrespective of other lighting) which it converts into a 3D depth map of the scene. From that depth image, Microsoft’s software can also track the skeleton image of one or two people, which lets you respond to the movement of bodies. Microsoft and partner PrimeSense weren’t the only to try this scheme, but they were the ones to ship the most units and attract the most developers.
We’re now on the third major revision of the camera hardware.
2010: Original Kinect for Xbox 360. The original. Proprietary connector with breakout to USB and power. These devices are far more common, as they were cheaper and shipped more widely. Despite the name, they do work with both open drivers for respective desktop systems.
2012: Kinect for Windows. Looks and works almost identically to Kinect for 360, with some minor differences (near mode).
Raw use of depth maps and the like for the above yielded countless music videos, and the skeletal tracking even more numerous and typically awkward “wave your hands around to play the music” examples.
Here’s me with a quick demo for the TED organization, preceded by some discussion of why I think gesture matter. It’s… slightly embarrassing, only in that it was produced on an extremely tight schedule, and I think the creative exploration of what I was saying about gesture just wasn’t ready yet. (Not only had I not quite caught up, but camera tech like what Microsoft is shipping this year is far better suited to the task than the original Kinect camera was.) But the points I’m making here have some fresh meaning for me now.
2013: Kinect for Xbox One. Here’s where things got more interesting – because of a major hardware upgrade, these cameras are far more effective at tracking and yield greater performance.
Active IR tracking in the dark
Wider field of vision
6 skeletons (people) instead of two
More tracking features, with additional joints and creepier features like heart rate and facial expression
1080p color camera
Faster performance/throughput (which was key to more expressive results)
Kinect One, the second camera (confusing!), definitely allowed more expressive applications. One high point for me was the simple but utterly effective work of Chris Milk and team, “The Treachery of Sanctuary.”
And then it ended. Microsoft unbundled the camera from Xbox One, meaning developers couldn’t count on gamers owning the hardware, and quietly discontinued the last camera at the end of October 2017.
Everything old is new again
I have mixed feelings – as I’m sure you do – about these cameras, even with the later results on Kinect One. For gaming, the devices were abandoned – by gamers, by developers, and by Microsoft as the company ditched the Xbox strategy. (Parallel work at Sony didn’t fare much better.)
It’s hard to keep up with consumer expectations. By implying “computer vision,” any such technology has to compete with your own brain – and your own brain is really, really, really good. “Sensors” and “computation” are all merged in organic harmony, allowing you to rapidly detect the tiniest nuance. You can read a poker player’s tell in an instant, while Kinect will lose the ability to recognize that your leg is attached to your body. Microsoft launched Project Natal talking about seeing a ball and kicking a ball, but… you can do that with a real ball, and you really can’t do that with a camera, so they quite literally got off on the wrong foot.
It’s not just gaming, either. On the art side, the very potential of these cameras to make the same demos over and over again – yet another magic mirror – might well be their downfall.
So why am I even bothering to write this?
Simple: the existing, state-of-the-art Kinect One camera is now available on the used market for well under a hundred bucks – for less than the cost of a mid-range computer mouse. Microsoft’s gaming business whims are your budget buy. The computers to process that data are faster and cheaper. And the software is more mature.
So while digital art has long been driven by novelty … who cares? Actual music and art making requires practice and maturity of both tools and artist. It takes time. So oddly while creative specialists were ahead of the curve on these sorts of devices, the same communities might well innovate in the lagging cycle of the same technology.
And oh yeah – the next generation looks very powerful.
Kinect: The Next Generation
Let’s get the bad news out of the way first: the new Kinect is both more expensive ($400) and less available (launching only in the US and China… in June). Ugh. And that continues Microsoft’s trend here of starting with general purpose hardware for mass audiences and working up to … wait, working up to increasingly expensive hardware for smaller and smaller groups of developers.
That is definitely backwards from how this is normally meant to work.
But the good news here is unexpected. Kinect was lost, and now is found.
The safe bet was that Microsoft would just abandon Kinect after the gaming failure. But to the company’s credit, they’ve pressed on, with some clear interest in letting developers, researchers, and artists decide what this thing is really for. Smart move: those folks often come up with inspiration that doesn’t fit the demands of the gaming industry.
So now Kinect is back, dubbed Azure Kinect – Microsoft is also hell-bent on turning Azure “cloud services” into a catch-all solution for all things, everywhere.
And the hardware looks … well, kind of amazing. It might be described as a first post-smartphone device. Say what? Well, now that smartphones have largely finalized their sensing capabilities, they’ve oddly left the arena open to other tech defining new areas.
For a really good write-up, you’ll want to read this great run-down:
Here are the highlights, though. Azure Kinect is the child of Kinect and HoloLens. It’s a VR-era sensor, but standalone – which is perfect for performance and art.
Fundamentally, the formula is the same – depth camera, conventional RGB camera, some microphones, additional sensors. But now you get more sensing capabilities and substantially beefed-up image processing.
1MP depth camera (not 640×480) – straight off of HoloLens 2, Microsoft’s augmented reality plaform
Two modes: wide and narrow field of view
4K RGB camera (with standard USB camera operation)
7 microphone array
Gyroscope + accelerometer
And it connects both by USB-C (which can also be used for power) or as a standalone camera with “cloud connection.” (You know, I’m pretty sure that means it has a wifi radio, but oddly all the tech reporters who talked to Microsoft bought the “cloud” buzzword and no one says outright. I’ll double-check.)
Also, now Microsoft supports both Windows and Linux. (Ubuntu 18.04 + OpenGL v 4.4).
Downers: 30 fps operation, limited range.
Somehing something, hospitals or assembly lines Azure services, something that looks like an IBM / Cisco ad:
That in itself is interesting. Artists using the same thing as gamers sort of … didn’t work well. But artists using the same tool as an assembly line is something new.
And here’s the best part for live performance and interaction design – you can freely combine as many cameras as you want, and sync them without any weird tricks.
All in all, this looks like it might be the best networked camera, full stop, let alone best for tracking, depth sensing, and other applications. And Microsoft are planning special SDKs for the sensor, body tracking, vision, and speech.
Also, the fact that it doesn’t plug into an Xbox is a feature, not a bug to me – it means Microsoft are finally focusing on the more innovative, experimental uses of these cameras.
So don’t write off Kinect now. In fact, with Kinect One so cheap, it might be worth picking one up and trying Microsoft’s own SDK just for practice.