A Behind the Scenes Report on the Making of the Show Visuals and
Delivery Systems
by Isabelle Rousset

photo by Valerio Berdini - liveon35mm.com

Amon Tobin's show ISAM premiered at MUTEK June 1st to incredulity and awe from fans and the media alike. The
general consensus from all camps was that this show was something new, futuristic, not experienced before; it
exceeded all possible expectations and was very, very exciting. Suspension of disbelief had been achieved and
audiences transported with it.

Our friends at V Squared Labs and Leviathan responsible for the show visuals "killed it" - that's pretty much the
word.

The team made up of V Squared director Vello Virkhaus, designer and programmer Peter Sistrom and Leviathan
chief scientist Matt Daly along with the help of Bryant Place built a custom application that runs the visuals of the
entire Amon Tobin show end to end. The application is made inTouchDesigner, Derivative's visual programming
environment, and does the projection mapping, video playback, Kinect response, realtime effects and more.

If you've seen the live show or even video footage of the performance then you know what we're talking about. If
you've been living in a media blackout for the last few weeks here is a compilation of what people have been saying.

Gizmodo called it "The Concert of the Future, Today" and CHARTattack said that "it was a live show one might have
said could only be created and performed by a cybernetic organism." where "groundbreaking modeling and mapping
techniques were used to take the audience at the Metropolis on a trip to dozens of worlds. ... it felt like one was
transported into deep space or, alternately, inside the Tron grid, a game of Tetris, an M.C. Escher painting, the fiery
pits of Mordor, an acid-coloured kaleidoscope, a Splinter Cell video game..." Wired got straight to the point: "ISAM's
live show looks like a mindfuck of the highest order" with "little in the way of precedent."

As for the audience at the MUTEK premiere here is what some had to say: overwhelming, mind boggling, feeling
small, the most amazing visual performance i've ever seen in my life.. the amount of work, i can't even fathom the
technology behind it because it was just insane. The dynamics of everything, the range of imagery, i was really blown
away...

Also after the premiere, MUTEK founder and artistic director Alain Mongeau told Vello he was breathless and would
have to talk to Vello after he'd caught his breath.

Get the picture?

We had some questions for Peter, Matt, and Vello because even with what we at Derivative know about
TouchDesigner and building performative systems and interfaces for large shows and events, it was difficult to
construct a complete picture of what this team actually did and how to achieve the scale and complexity of Amon
Tobin show. With a single projector!

We reached the team at different times in different locations and various post-show states via Skype. What follows is
an amalgamation of their experiences and insights on the making of this show that includes as in-depth a description
as possible (without demystifying the whole thing) of the individual components that made up the superb system used
to run the show end to end.

ON VELLO, MATT, PETER & BRYANT

Vello Virkhaus, ISAM Live Creative Director.

Vello Virkhaus, Founder and CEO of V Squared Labs is an experienced Visual Director, Producer and VJ who takes
on hugely complex projects with brilliant results. He is also clearly not afraid of anything. Vello has enormous
technical proficiency and a long history of pioneering and innovative work for some of the industry's top talent. Vello
has produced visual shows for the likes of Paul Oakenfold, The Red Hot Chili Peppers, Korn, Ministry, Jay-Z,
50cent, and Beyonce. He has also directed and produced unique and technologically advanced audio visual
experience content packages for DJ's, nightclubs and architectural installations, most recently, for the Marquee
nightclub in Las Vegas.

Vello was introduced to TouchDesigner by his friend and frequent collaborator Matt Daly to create 'Dome 2', a
custom 360 visual performance system along with brand content for Heineken Inspire activations. They decided to
use TouchDesigner as their "creative tool", as Vello puts it, for its high quality realtime 3D and UI development
capabilities.

Vello: "Matt got me completely obsessed with TouchDesigner and we've been using it ever since." - on quite a few
engineering-intensive projects included Electric Daisy Carnival, Ultra Music Festival, Roger Sanchez, and last year
the Marquee in Vegas where Vello worked with Matt and Bryant to design and produce a pretty hot interactive,
multi-screen experience.

Matt Daly, ISAM Live Chief Scientist

Leviathan is celebrating their 1st anniversary the day after our conversation with Matt Daly. In a year the company
with a staff of 25 has a lot to show for itself. They're the largest 3D, visual effects shop in Chicago and they're only
just getting started.

Animation, broadcast, and traditional cinematic work forms the basis of the studio's practice and the "experiential
work" as Matt puts it, is another tangent that allows Leviathan to access different clients. Experiential projects
involve producing big systems with lots of projectors, computers and content. Leviathan creates the content as well as
the systems, interactive, video serving and so on and according to Matt "TouchDesigner is definitely giving us an
edge over everyone else." "Without your software we're up a river." he stresses.

So what you're saying is you use TouchDesigner where you wouldn't go without?

"Definitely - there's no other tool that can do the things we've done with it. I won't kid you, I have looked and there's
nothing else that can give you a really reliable system for running the kinds projects we produce. We've run some
very high-profile shows in front of a lot of people and TouchDesigner hasn't failed us. If there's any failure it's on our
part or the computer itself."

Bryant Place, ISAM Live Designer

Bryant Place a.k.a. CyberPatrolUnit is a busy guy and really good at what he takes on - which is considerable. He's an
accomplished VJ and visual programmer based in LA, though he spends a lot of time on the road. For the Skrillex
summer tour Bryant designed the entire show - the LED layout and stage design as well as the video system itself
built from scratch in 5 days at AG Lighting. The system uses TouchDesigner to speak DMX and move LED panels
around to animate them for the show's very cool visual effects. Prior to that Bryant toured with Plastikman Live 2010
tour driving the TouchDesigner based show visuals.

Bryant has work extensively with Vello Virkhaus' V Squared Labs team, performing for major musical acts and also
developing and installing a highly specialized audio-visualization system for Marquee Nightclub at
the Cosmopolitan Vegas. Bryant was an integral part of the V Squared team on the Tobin show assisting in many

ways, technically and with two of the pieces creatively.

Bryant was at V Squared when we reached Vello so we were lucky enough to bag 2 birds at once.

Peter Sistrom, ISAM Live Designer and Chief Programmer

Peter Sistrom is an incredibly capable young man schooled in architectural design who 3 years ago developed a
passion for motion graphics and visuals, particularly in a synesthetic relationship with music.

We met Peter last year when his impressive entry for the Plastikman Live Visual contest was selected as one of the
finalists. At that time he'd been using TouchDesigner for only a few months and prior to that had been working in
Processing and Pure Data. "I had no background in computer programming, motion graphics or film before
embarking on this journey, so I have been teaching myself everything by the seat of my pants." Peter told us then.

Very quickly Peter has become one of the software's prolific users and most engaged promoters. Earlier this year
Peter demoed TouchDesigner and its interoperability with Arduino at the LA Crash Space. Bryant Place who had
been working with Vello at V Squared brought Peter into the fold for the Amon Tobin project. It was effectively
Peter's first TouchDesigner job and as Vello and Matt have made clear, his role was significant in the execution of the
show.

Matt: Peter is the one responsible for putting together what we see as the show. We gave him some tools to use but
he definitely deserves the credit for assembling this whole thing and creating a performable show out of it.

Amon Tobin is Peter's favorite electronic music artist and in many ways this was an ideal experience. Peter's role
was, as he modestly explained, "to funnel it all in" and keep in touch with all the other artists and renderers
performing reality, quality and continuity checks, integrating content from various sources, integrating check and
lineup with the timeline and triggering it as well as designing visual content and working on the show's transitions.
Impressive!

Peter, is this your first mapping project?

Peter: Of a large scale yes. I've done it in my bedroom before!

ON THE MAKING OF ISAM

VELLO, you recently said about the show that you loved the emotional response the 4D mapping triggers in
people. What did you mean by '4D' exactly?

Vello: 4D in mathematics is a very abstract concept in which this additional dimension is indistinguishable, yet
acknowledged. This unknown to me relates to the pronounced visual effect the mapped structure of Tobin creates for
the viewer. What is fascinating is that we give viewers an idea of what it might be like to see beyond 3D space, to see
all points simultaneously for both the exterior and the virtual interior of the set. The combination of this mapped
effect and Amon's music produced some very intense emotional reactions from people after the MUTEK
performance.

Bryant: At times you do forget that there's a physical structure there, that's where the magic is.

It's a huge project how did it come about?

Vello: Alex Lazarus, production designer at Blasthaus had mistakenly emailed the wrong person at ninjatunes,
yielding an inquiry from Jeff Waye about video mapping for Amon Tobin. This humorous, yet powerful inquiry led
Alex to contact us, as he believed we could be perfect for the project with our mutual understanding of electronic art
and music. We then put together an initial presentation with a set of story boards and written treatments of ideas,
along with a reel of LED pixel mapping samples. Before this project, we had only done a few video mapping
projects, but nothing of this scale or complexity. We had been primarily working with LED, and mapping in planar
space.

Things escalated pretty quickly from the initial set of images with developments by Alex and Heather Shaw of
fantastic looking physical set designs. Once it seemed like we were actually in the running against our competition, I
got Matt Daly at Leviathan involved and collectively we began to storyboard out the whole show.

Storyboards supplied by VSL

Amon liked it, so I started expanding a written creative of everybody's design ideas and then added Amon's feedback.
Once we had the contract, I flew up to Amon's house in the Bay Area where we pounded out a complete written
treatment with final reference materials in a day. Two months later the show launched at MUTEK June 1st.

It sounds as if you were fully invested, more even...

Vello: Yes Matt Daly and I both put in a great amount of time and resources to get the project to this state.

Were you involved in the design of the structure in view of what you were going to be projecting on it? We've been
hearing a rumor you're using a single projector on this show...

Vello: I was very fortunate to have some technical input early on with the set design Heather and Alex were
developing. My involvement in this process was more from a technical video projection standpoint. I was looking at
the structure(s) and determining if they could be mapped, with how many projectors etc. Seeing their ideas come to
life was also an added inspiration in creative direction. I knew from the get-go that steering the show in an achievable
direction meant going with a single projector because there'd be no time for mixing and blending 2 projectors
together. So, I made technical evaluations regarding these matters.

What was the process, working with Amon? He appears to be a very considerate artist.

Vello: It's exciting to work with an artist who's interested in working with and building a team. They're active and
creative and interested in pushing the project into the fine art medium so there's a lot of territory to grow into with
this type of experience because it's transcended the commercial barrier.

ON VISUALS

There's a vast amount of content in the show and a lot of variety in that. Were the show visuals designed per track
or thematically?

Vello: The show visuals were designed per track with an overarching, loose narrative hemming it all together. Amon
was symbolically represented as a pilot and the structure itself the craft. We designed imagery that helped transform
the 2bit structure into these various narrative and abstract constructs. The vast variety of theme content was also
chosen to keep things moving and keep the audience interested.

Audience reaction to this show has been rather extreme. What did you guys do to get this kind of response?

Matt: What separates this project from others is the cinematic content, people really respond to that and then having
the generative material layered overtop. There was a story to be told, spaceship, asteroids, alien planet and other
things, we did storyboards, did extensive pre-production with Vello and helped get the project. Leviathan provided a
huge amount of HD content, our render farm was cooking for a couple of weeks.

photo by Valerio Berdini - liveon35mm.com

Did you have any references, overarching concepts?

Matt: The original references provided for this were Chris Cunningham's work where he took these very expensive
(at the time) renders and then completely messed them up, just ripped them to shreds without any regard for how
beautiful the rendering quality was. That's something we were trying to achieve.

We took these really high-end renderings - it took weeks of machine hours to render these scenes - and then
completely disregarded them and tore them up with our glitching and realtime dynamics. I think that was a welcome
sight to people because it didn't have that "Oh look how beautifully rendered and perfect this is". It had a bit of Atari
style video game appeal to it. Low resolution and very fluid, computers and digital and organic in a way I think a lot
of people haven't seen before.

Very much in line with Amon's musical composing style then? So much of his source material is organic which is
then treated as you're describing. As he told FastCompany, his work seeks to strip away the familiar, he"breaks
things down and reorders/transforms them to create hybrids that lead the imagination into other unsettling and
disorienting worlds."

Matt: Yes, very much so. Amon's music is very sculpted. When you break it down each song is like a sculpture in a
way because it's very carefully crafted and he's using these effects that maybe in other music would be a lot more
repetitive. He's a classical composer of the electronic age.

Peter, what was your experience working with Amon and designing visuals for this show?

Peter: He's my favorite electronic artist so it was a thrill. Amon pushes but respects and understands. He's got a
particular vision in terms of what it feels like but he's not a visual artist so he has to trust us. Great teamwork.

At MUTEK Amon said, "hey can we get something a little more happening here because it seems a little slow?". And
we could do that, we used these live effects - face mapping, procedural noise, broken scanning lines etc which are
used in the DJ set but also in transitions.

Bryant and Peter, your contributions to the show visuals were produced somewhat differently. Can you describe
that process please?

Bryant: Both Peter and I set out with the notion that there would be curtain songs, that is, songs that would run in
real-time; this was great because right away it set limits as to how one could approach the graphical design. Working
in realtime was a huge benefit because it really allows you to tune into the audio to create something tightly
coordinated. For the most part we worked with the Animation component.

Peter: Yeah, the Animation component came into play in a few different ways, either for animating the whole song
or little triggered bits as Bryant used it.

Once I found out we were not going to run in realtime, I actually really beefed things up and did things I wouldn't do
for a realtime piece. What was nice though was that it still rendered out the whole 3.5 minute song in 1080 hd in
about 10 minutes! Technically it could have run, just maybe not within the rest of the system at full bore.

This created a different workflow, making TouchDesigner into a sort of hybrid 3d animation and compositing system.
I pulled lots of textures from real life, like paper and rust for Lost and Found. All the geometry was procedurally
based on the model of the structure. Lots of depth rendering/compositing, 3d texture lookup animation for individual
instanced objects, lighting tricks, quite a bit to get the mood and depth, fun stuff!

ON BUILDING YOUR OWN SYSTEMS

Vello, you've been at the forefront show by show and project by project building your own applications vs. going
with turnkey solutions. How'd you go about building the system for the live show?

Vello: We'd never done such a huge mapping project and there was no precedent for the way this system had to
operate – it would need to be a unique, new system. I don't know what we would have done without building our own
solution in TouchDesigner to do this project. There existed a small number of previous TouchDesigner parts from
other projects that we could pull from, but pretty much everything was built from scratch. We wanted to make our
own proprietary engineering accomplishments for this show, to push ourselves further and prove that we could do it.

"It's a relatively simple system in the sense that it just runs" says Peter Sistrom but anybody who's pushing the
envelope is building their own application: a user interface with custom-made controls, compositing and rendering
for that show. Something completely unique that gets you to a level that's impossible to achieve with turnkey stuff.

This is indeed the case with the Amon Tobin show where TouchDesigner was used to design and build a Show
Previewing System, a Show Sequencing System, the Show Rendering Unit and Alignment System. We are not able at
this time to reveal much about these at request of the artist.

Show Sequencing System

Peter: The show's timeline or the Show Sequencing System is its own component and really the heart of the show.
It's essentially the 'central show unit' which was invaluable for streaming all of that content and making it all
manageable. Jarrett Smith from Derivative was instrumental in helping us with setting that up.

Show Previewing System

Peter: The show previewing system allows you to see what it looks like in 3D before the set is made. This enables
the team to see how the perspective illusion works and allows for tweaking, trading off depth illusion vs the left/right
position of the observer.

Show Rendering Unit

I guess it's a little nail-biting creating all that refined content before the cube set structure is built?

Peter: We'd been working off a digital model provided by Heather Shaw while the cube structure was being built and
doing all our renderings based on that model. I had tried to instill in everyone producing content - the big Leviathan
crew using Maya and Cinema 4D, to be somewhat procedural in their work... that adjustments might have to be made
to the model. But then it turned out that everything was built perfectly and matched up and we never adjusted our
virtual model - that was one of my big "Aha" moments which was more of a "Thank God" moment really!

ON KINECT

Matt, Kinect was used sparsely but very effectively on this show, what did you do?

Matt: One of the ways Amon appears during the show is projected on the entire set as a Kinect point cloud. What the
audience sees is a custom rendering of the point cloud using the Kinect image. We have some other ideas on how to
use the Kinect and it feels as though we've barely scratched the surface with this project. But we were also careful to
not over-do it and risk losing the interest factor here.

ON "AHA" MOMENTS

On a show of this scale and complexity and you're the creative director, were there any actually memorable
moments that were illuminating?

Vello: I was working with Peter in a hotel room in Vegas doing a Heineken event and working on the Roger Sanchez
system and my big "Aha" moment came when I asked Peter to make a box with a sphere inside it because I wanted to
prove that we could recreate a faces and edges (of the cubes) map look with a floating interior. A big part of what I'd
sold to Amon was that the show would have a floating cinematic feel combined with mapping effects. Peter was just
noodling around and I looked over his shoulder and he'd re-projected the box and the sphere on the box and the
sphere and as soon as I saw that I instantly knew that we had the technique to do ISAM. That was the big "Aha"
moment. That was "Oh shit, there it is!" and I felt fully confident.

Vello: The second "Aha" moment came when we actually saw the calibration tool project onto the 2bit set for the
first time at the rehearsal stage. This was one of the most powerful moments which provided me the most assurance
and relief. I had been really concerned about the overall technology and how things were coming together, how they
were going to work. I was actually super paranoid having heard a few video mapping calibration horror stories. The
projectors were literally dropped down off of the forklift and onto the truss stands, plugged in, powered on and
calibrated in less than 30 minutes - the first time.

Peter: A lot of "Aha" moments were centered around the illusion - the perspectival projection mapping illusion. We
had got it all set up in the warehouse we'd rented for a week and Matt did the first map, the first calibration and said
"Okay, throw something up so we can see." and literally the first thing I had was a rendering of the structure with just
a soft noisy gray - almost stone tone, and two lights. It popped up and we just kind of all looked at it and "Woah!"
Suddenly this thing looks like it's made out of stone and it's just sitting there. It was the most simple thing but it was
really remarkable. There were enough cues to make it believable that it was angled stone cubes.

Matt laughs "When we finally saw things light up for the first time going from the virtual to real, that was a really
good feeling like, "Wow! This thing is actually going to work!" Everything was playing back, we were getting good
frame rates and it all looked good. We're using SSD drives and the video's playing back flawlessly, I don't think we
ever really hit a bandwidth for that - you guys (Derivative) are really efficient with the resources of the system.

ON BIG SHOWS, SINGULAR SYSTEMS AND SPECTACLES.

There's been an apparent trend recently, at least, 3 of the artists using TouchDesigner to run their shows - Richie
Hawtin's Plastikman, DJ Shadow and now Amon Tobin have nested themselves right inside the performance set.
Amon said that the idea for ISAM was to integrate himself "quite literally, into an audio and visual presentation

of the album." What do you make of this?

Vello: The performance set is an extension of the individual musician experience. DJ'ing has been and is a very
individualistic thing, so it seems like a natural extension of the DJ as superstar to have them be inside a larger
structure and have visual representation of the sonic going on around them and be inserted into it.

Bryant: I think there's a common goal now where everyone is trying to achieve the effect of a singular system where
all devices are in communication working as a singular system. We're eliminating technical boundaries where
something might look as if it's on its own operational system and not quite doing the same things as another element.
So yeah, I think there's some common ground there between all these different projects to create the effect of a
singular system.

Amon Tobin sees tools as a starting and not an end point and has said that music technology can appeal to our
inherent laziness. You can now buy what you might lack through your own efforts and creativity. Shows are
definitely getting bigger and technologically more complex... where do you think that's going and what is it about?

Peter: Big shows have worked their way into the zeitgeist. Big electronic artists - Amon, Richie Hawtin, Deadmau5
et al. These days you've got to do a big 3D show. Amon came in at a good time after lots of experimentation in all of
this and brought his amazing depth and texture.

Bryant: I think also that the difference between Amon and a lot of previous shows, is that he's generating some of the
visuals on his own. There's a point in the show where he's interacting with the Kinect camera, painting waveforms
onto the structure by waving his hand around.

Matt: Everyone's always looking for the next big thing. Someone will make a tool and then everyone else will do it
and it becomes commonplace. Augmented reality for example. It's always going to be bigger and better.

Vello: Everyone wants a show and it's about expanding the immersive performance space. Everyone has a different
approach and no matter what they're using and how they're doing it the desire is to entertain an audience. Mediums
are converging in the entertainment sphere and they have been for a long time as far as what you do. The job
description's getting a little hazier these days.

Bryant: But it keeps it exciting. Every day it seems more and more possibilities are coming into fruition.

Vello: What makes this show so nice is that it has feeling. It's cinematic and a feast for your eyes. Projection mapping
is definitely a future direction - it has great transformative qualities. However there are also more complications
doing this in a festival environment.

ON HORIZONS

What's next on your radar Peter? It's been an action-packed few months. What do you have coming up and what
are your current sources of inspiration, things you'd like to do?

Peter: Right after the Roundhouse show in London there's the Electric Daisy Carnival in Vegas. It will be a big LED
extravaganza and 3 days of straight VJing. Then in July there's a lot of personal work I want to get back into - the
next version of my Name the Machine project with my good friend and collaborator Matthew Davis.

I personally love the expanse of big flat HD screen but there's so much out there. I like surfaces and spaces, maybe
it's my architecture background. I'd love to work on the integrated design of a space, a venue - not unlike
the Cinechamber but more of an architectural thing that integrates video and media as a venue space and then you can
either use those spaces or not as artists.

What's next for you guys? Now what?

Bryant: Well one of the things I'm going to take to the field very soon is doing things beyond video and video

mapping. I'm getting more into DMX and trying to control servos and send information to other devices rather than a
video signal and integrating those two. Like doing some sort of LED video mapping but then on top of that sending
information to lighting or other devices.

Matt: Definitely work on more experiential projects. We want to do bigger more integrated projects using everything
from projection system technology and LED wall technology to integrated systems large multi-screen venues, digital
signage integrated into building systems - that sort of range.

Vello: We plan on evolving the technology and system structure created for the Tobin experience as far as we
possibly can. I feel as if we are just scratching the surface on the potential of the immersive entertainment market. I
hope to gain more financial success and artistic recognition.

We've all had a lot of response from the Amon Tobin show, the phones are ringing and we'll be knocking on your
door for more software licenses really soon. ;-)

ON TOUCHDESIGNER: VISUAL THINKING IN AN AGE OF VISUALIAZTION

TouchDesigner is WYSIWID: what you see is what it's doing. Its open, broad and highly-visual procedural
architecture boosts and expands discovery, creativity and productivity making it the most complete authoring tool for
building interactive 3D art, visualizations, user interfaces and rapid prototypes.

No stranger to large, complex live events and mapping projects, TouchDesigner was recently used to map two icons
of modern architecture: the Sydney Opera House and The Guggenheim Museum. Both were wrapped inside and out,
the Opera House as visual accompaniment of a live symphonic performance and the Guggenheim for YouTube's first
Biennial awards. TouchDesigner was also used to build the audio-reactive visual show system for the Plastikman
Live Tour, voted RA's top live show of 2010.

The ambitious and complex Amon Tobin Show is an excellent example of not only what TouchDesigner can do but
also exemplifies how it can be used - the spirit of working with a tool if you like. The V Squared team used
TouchDesigner for prototyping and then building the show's UIs and components; as the glue that puts all these
systems together; as an interface through data bases, XML etc.; to produce show visuals; as a tool for rapid
visualization and to render and then run the show.

Performing high-profile complex live events involving tons of technology often in front of millions of people you
need a dependable system.

Matt: "When it comes to the glue that puts all these systems together and lets us interface through data bases, XML
etc. TouchDesigner has all those tools ready for us so we can prototype and get things up and running really fast."

Dedicated to advancing the way we make art and visualize information and ideas, Derivative has produced live
visuals and interactive art projects for a roster of international superstars that include Prada, Herzog & de Meuron,
Raster Noton and Rush. They have also developed major theme park attractions globally.

TouchDesigner FTE (Free Thinking Environment) and its counterpart TouchDesigner Pro puts a free development
environment with an extremely rich feature set into the hands of artists, animators, educators and "everyone else".
"TouchDesigner is our vision of what is possible in tomorrow's software tools for building interactive applications
and exploring data, imagery and sound", Derivative founder and CEO Greg Hermanovic states. "It exploits what is
possible in today's computing technology and is positioned to grow with the foreseeable advancements in computer