An Ndisplay like media play system based on TouchDesigner

Why not unreal?

For visual production project, some works require playing multiple high-res media plate at sametime on the LED wall. Although Unreal has it’s own media play system, it have difficulty when playing multiple high-res media at same time, which result of occationly glitching, skip frame and stuttering.

Unreal has Bink but it’s not realistic to convert multiple quite long high-res video to bink format, also the color space is limited. Unreal has exr image sequence support, which solved the color space issue, but file size is huge, put a great stress on machine IO and data management. Unreal doesn’t support Notch yet, a good codec which balance the quality and file size.

Why TouchDesigner?

TouchDesigner supports Notch encoding and has been widely used in large live events, which really proves its reliability. And it is fun to create an Ndisplay like system and bunch of virtrual production tool there.

How it works?

Preview client and Wall client It’s the same TouchDesigner project, which contain all the Ndisplay like projection core, virtual production tools and media management system, depends on the IP it will act like a Wall or Preview client.

Sync Server This is a TouchDesigner project, sole purpose is to general and sync data like, frame, camera tracking data… which the data need to be sync perfectly through all the wall nodes.

Control Server This is a TouchDesigner project, which manage syncing control data like media plate transform, color grading, light card, green screen, preset management… Which the data don’t need to by sync perfectly across the wall nodes.

TouchOSC I choose to use TouchOSC as our remote control interface, because it’s easy to make UI and features live, and cross platform by default. There is no need to go through build and deploy process, just simply push the TouchOSC project to PC and iPad devices.

How good is this system?

This system has been used in multiple film project for playing high-res car plates. The limite is on individual machine’s IO and GPU, for the setup we use It can handle max 5 x 16K 16bit video playing at same time, with no stuttering or glitching during play or looping. It can switch to different plates set in one click one sec, no restarting or resetting process required.

Highlight features

Ndisplay like projection

Not like some system which provide the simple UV like mapping, the system I made inspired by the Unreal Ndisplay plugin, which will project the content on the wall mesh from a viewpoint position, which provide the correct perspectives inside camera.

TouchOSC control console – File and preset management

TouchOSC control console – Plate global control

ouchOSC control console – Plate Transform

TouchOSC control console – Color Grading

TouchOSC control console – Lightcard control

TouchOSC control console – Green screen control

My Role

1. R&D the whole system, include TouchDesigner project and TouchOSC project dev.
2. Intergrate new features for VP Sup on the fly during shot
3. Debugging and maintain the system.