ART 7030

OVERVIEW

As part of my independent study, I focused on parametric modeling, electronics, fabrication, and computer programming.  The driving force behind this study was to unlock the potential of creating a computer that could receive video from my main Touchdesigner output via HDMI input while also providing a touch control surface to manipulate and control an audiovisual performance on a separate computer via OSC (Open Sound Control) messaging. 

The final version of the device will have a few hardware controls such as encoders, xy pad, pads, faders, a leap motion controller, and a camera.

Hardware

Raspberry Pi 5

10” Touchscreen

HDMI to USB capture card

USB Hub

USB Keyboard

Arduino* - for future connectivity to mechanical devices and sensor input.

Software

Ubuntu – Linux

TouchOSC – sending OSC messages to TouchDesigner

VLC – Video Input

Kodelife – GLSL/Shader – would allow for standalone or installation usage.

Shaper3D – Parametric Modeling app – for device build

PROGRAMMING OPTIONS

MacBook Pro

TouchDesigner, with its user-friendly interface, is remarkably easy to program. When connected to the network ports, the OSC IN/OUT CHOPs channel operator instantly recognizes the message. The CHOPs can then be routed via Python or node programming to control values, program changes, etc., in TouchDesigner. 

Ableton is also easy and has plugin options through TouchDesigner and within the software, such as Max, to send and receive UDP/TCP messages.

I then sent a video output via HDMI capture card to a USB port on the RPi, enabling me to get a live feed via VLC.

Raspberry Pi 5

Various OSs can run on a Raspberry Pi. I chose Ubuntu because of the vast software available on that platform. Other options included Raspberry OS and a non-GUI version of Raspberry OS that connects directly to Max and RNBO for instrument creation, web sockets, Arduino, etc.

Running Ubuntu also makes the Raspberry Pi capable of accepting class-compliant USB audio devices. Hardware improvements such as 8GB of memory and an improved GPU enable the RPi to run various programs, such as VLC and TouchOSC, stably and simultaneously.

As these computers' capabilities improve and AI chips become more available to help with processing and computing, the RPis can handle standalone live audiovisual manipulation at an acceptable frame rate.

PARAMETRIC MODELING

Shapr3D

I spent the first half of the term learning how to model in 3D. I had no prior experience, but I quickly grasped how parametric modeling works and its uses in my research and continued professional development. Learning this tool has been of incredible value to me. If there is any major takeaway from this independent study, I can now bring my sketches to life via 3D printing, CNCing, or outsourcing them for development.

Further interests within Shapr3D include working in the app on an Apple Vision Pro. There is a version that allows you to model and sketch in a headset and spatially create, which would also be a tool and skillset I would like to continue developing.