Preview of interface to drive a network of players

I am toying with the idea of a network of players synchronized to a central coordinator since a few months and I have already shared those movies:

New synch experiments at WWDC

Exploring an idea about distributing content on a network of players…

Our goal at ArKaos is to build a new range of products based on such architecture but we want to move forward step by steps and the first public step will be a simple interface to drive a limited pool of players connected to the same network.

Here you can preview what will look like the interface. This application is not even alpha code but I am happy to share it with those of you that are curious.

I used the opportunity when working on this project to experiment with the cocoa tools of Mac OS X. I am a big fan of cross platform programming and I worked with wxWidgets since a few years. Unfortunately because Apple did obsolete carbon we are now looking for new ways of creating our interfaces. I am happy that just within a few days I was able to create this already complex interface without writing too much code. In the end it’s true, interface builder rules and cocoa is a great idea!

Ok now to my prototype, the application is still useless but it demonstrate how to write a simple cue player. You can create list of events (cues) and assign them to computer keys. Then the idea is that when you will press those computer keys those players listening to the network events will start playing those video loops.

So at this time what can be done here is:
– importing static pictures to the cells on the left by drag and dropping from the finder.
– playing with the + / – buttons of the cue editor to add and remove steps of a cue.
– drag and drop visuals from the left cells to the cells of a cue.
– you can edit the layer position, start time and duration of a cue step. You need at this time to use the enter key to validate a new time or duration.

Here is a simple picture to show the interface in action:

Coordinator

Coordinator

Just drag and drop a few pictures on the left cells, create a few cues steps with the + and – buttons. The final prototype will be able to play movie loops across 3 zones of maximum 6 projectors. It will be possible to stack 4 layers of visuals on each zones. This is why by example the popup says z1l1, it means zone 1 layer 1.

I made a quick build if you want to play with this preview app, for Mas OS X only at the moment, download it here:

2009_08_CocoaCoordinator.dmg (208 KB)

The Optical Theremin

Matt Finke from LoopLight sent me this video using ArKaos software in a very creative way. In their toolkit you find NuVJ for standard VJ work and MediaMaster controlled from a GrandMA console.

The Optical Theremin

The Optical Theremin

Here is the video:

Get the Flash Player to see this content.

Philipp from LoopLight also sent me this information:

“I used Arkaos MediaMaster, Arkaos NuVJ and grandMA ultralight in one network session. The headlights were Jarag-5 in a matrix with par30 and 52 Movinglights with CMY. I made a pixelpatch in MediaMaster for Jarag and put the CMY of the movinglights in the same pixelpatch. A camera, the first input to MediaMaster, indicated towards a plexiglass disk, which i had divided in two areas. Left side for Jarag and the right side for the CMY. The NuVJ was the second input to MediaMaster for SD Clips. I set different fx like edge detect, greyscale and contrast in MediaMaster. Then I scaled my live input on pixelpatch, only the left area of plexiglass. Finally I moved my hand vertically and horizontal on the left area during playing SD Clips from NuVJ. For CMY I made four large Colourwheels in different bright colours. I moved one of the Colourwheels in the right area to change all CMY in the movinglights.”

As you can see it’s a cool example of creative processing with lighting!

ArKaos software used at Coachella 2009

Here is an article speaking about the famous Coachella festival.

Momo the monster was using a lot of tools but included was three NuVJ stations.

 

 

NuVJ

NuVJ

 

 

Read more here:

vj-at-coachella on createdigitalmotion.com

NuVJ is a very simple 2 layers left right video mixer, but because it’s packed with a lot of goodies like real time effects and music visualizers it’s very popular.

Creating a new video codec based on texture compression

At ArKaos we always fight to get the best out of current computer configuration regarding media playback.

When you are in the show industry and try to pick the best way to compress your content it’s still a little bit of black magic. While some codec do compress well video they are heavy to handle for the machines, even more, the codec that does the best job at compressing while keeping a good quality such as H264 are very bad when you need to scratch your media.

The best codec for artists that need to interact a lot with the content should allow to play forward and backward easily and should allow to jump into the content quickly.

Some have experimented with using a file format that is designed by the companies making the graphic chips of your computer. This file format is based on texture compression DXT1 DXT3 or DXT5.

I was wandering how hard it would be to add that file support to QuickTime inside a new codec. Being programing for the QuickTime API since it’s version 1.0 beta I was considering the challenge fun and interesting.

So I resuscitated an old sample code from 1999 on the apple web site and after a few hour I had a codec having my name and generating video files that could be played back by the QuickTime player.

I jumped then on the DXT texture compression problem and picked the squish library to handle texture compression. A few more hours and I had a working codec.

 

 

ArKaos codec in the QuickTime player

ArKaos codec in the QuickTime player

 

 

This codec when used by QuickTime is not efficient because the texture decompression is done on the CPU and so you don’t see the advantage of using the GPU to handle that texture format.

To demonstrate the speed up I used an experimental player I am working on and patched ffmpeg to handle that new codec I just created. Thanks to that I can leave the texture datas untouched and pass them to the GPU via the OpenGL extensions regarding texture compression.

When this was done I could enjoy the show and make some performance tests, file size is in MB and CPU load is taken from running top in the console:

name resolution data H264 Photo JPEG Dxt1 Dxt5 Dxt1 Compressed Dxt5 Compressed
Alternate Rotate DI 640*480 File Size 9,3 14,3 22 43,9 17,5 19,7
    CPU Load 31 30 7 9 15 18
                 
1920 HD 1920*1080 File Size 36,5 41,8 90 180 55,7 63,8
    CPU Load 99 95 13 22 42 56
                 
T25 Random Twirls 1280*720 File Size 35,9 73,2 65,9 131,8 51,5 58,4
    CPU Load 78 72 10 16 32 41
                 

If you are curious regarding frame quality here is a part of an original frame:

 

original frame Apple Intermediate

original frame Apple Intermediate

 

Here is the a part of the same frame using Dxt1

 

Part of an HD frame Dxt1 compressed

Part of an HD frame Dxt1 compressed

 

 

And here is a part of the same frame with Dxt5 compression:

 

Part of the same frame with Dxt5 compression

Part of the same frame with Dxt5 compression

 

 

The results of my test is that pure Dxt1 regarding CPU load is the best format, Dxt5 is better regarding quality but generate twice heavier files. Using Dxt1 I could play 7 HD loop at the same time with my laptop … if the data rate of my hard drive would allow it!

The problem with pure Dxt1 or Dxt5 files is that they are huge, 4 times bigger than photo jpeg or H264. To go around that I experimented with using data compression on the CPU because the CPU is now almost unused!

So using Dxt1 Compressed texture is the best solution regarding CPU usage and disk usage. The size is almost the same than photo jpeg and a little bit bigger than H 264. When using the CPU to decompress the texture data from disk the codec is still twice faster than any other codec I tested.

On my MacBook pro I can’t play a full HD loop and even at 1280*720 my machine is struggling. Using Dxt1 compressed I can play 2 HD layers and 3 720p layers.

I made available the test player I used and a set of video loops. This dmg file is for Mac OS X x86 only (no Power PC), so if you are curious here are the files:

CompressedTextureDemo.dmg (435 MB)

We will bundle that codec experiment in the next update of GrandVJ and MediaMaster, that’s for sure!