me
maker

← back to portfolio

Interactive Modular Zoetrope

PhD thesis · UIST 2022 demo · Audiotrope (light-reactive audio) · with Paul H. Dietz and Alec Jacobson

Arduino · Raspberry Pi · OpenCV · sensors · 3D printing · modularity · interactive storytelling

Read with caution: this page contains strobing videos.

the full zoetrope installation behind curtains
The Audiotrope: a modular bike-wheel zoetrope with user-controlled strobe light and light-reactive audio.

The play experience

Step inside a dark curtain, pick up a strobe flashlight, and peer into a spinning 3D diorama. You control where the light lands on each revolution — and when you shine it on specific objects, sound plays: a pancake sizzling, a character yelling as she falls, cars honking on a distant highway. The same visuals can tell different stories depending on where you look and which audio clips are loaded.

This evolved from my UIST 2022 demo (viewer-controlled strobe) into the Audiotrope: a light-reactive 3D zoetrope that tracks which frame you're seeing and where in the scene you're pointing the flashlight, then triggers synchronized audio in real time.

Design challenge

sketch of a spiral zoetrope
Spiral zoetrope vision. Sketch by Alec Jacobson.

Zoetropes are usually small, periodic, and passive — a single looping motion with no viewer agency. I wanted to expand what stories could be told in the format by building bigger, making scenes modular, putting the strobe in the viewer's hands, and eventually using light as an input for sound.

Long-term, I'd like to produce full short 3D films in the form of a giant spiral zoetrope — a large undertaking with extreme space, material, and engineering constraints. The bike-wheel Audiotrope is a first step in that direction.

Evolution of prototypes

The bike-wheel Audiotrope grew out of many earlier builds:

First proof of concept

backflip zoetrope with strobe
Mechanical strobe triggered by spinning spokes.

A laser-cut wooden plate on a lazy susan bearing, 16 3D-printed backflip frames, and a photointerrupter IR sensor triggering an LED strobe via Arduino. The wooden spokes mechanically broke the IR beam once per frame, flashing the light in sync with the spin.

Record player zoetrope

record player zoetrope
Timed strobe with a record player.

Constant-speed spinning from a record player with timed strobe code, plus a bouncing ball animation using squash and stretch. Because the record player handled rotation, I could no longer use mechanical spokes — instead the Arduino timed the strobing to the turntable's fastest setting.

Cardboard zoetrope

cardboard zoetrope stop motion
Stop-motion capture with hidden supports.

A larger backflip zoetrope using invisible support structures so the character appears to float mid-air. Without time to build a motor drive, I captured it stop-motion style — rotating the wheel 1/16 turn by hand between exposures.

Modular bike-wheel zoetrope

clipping a scene box to the wheel
Clipping a scene box to the wheel.

Each of the 16 frames is a full 3D scene in a box, clipped to the bike wheel with magnets. Scenes swap in and out for rapid prototyping — the same principle I explored with my LEGO zoetrope. Laser-cut boxes with pressure-fit tabs replaced heavy 3D-printed frustums for faster iteration.

Lightweight laser-cut scene holders.
full kitchen scene in a scene box
An entire kitchen environment fits into one scene box.

Hardware

The Audiotrope sits in an aluminum extrusion cage. A Nema 23 stepper motor drives the bike wheel through a chain and sprocket. An inductive proximity sensor on each frame bolt triggers the strobe flashlight and camera. Spoke weights counterbalance the heavy scene boxes.

labeled diagram of zoetrope hardware
Labeled hardware: start button, magnetic scene boxes, chain tensioner, camera, proximity sensor, spoke weights.
modular zoetrope wheel with electronics
Electronics and modular wheel design.
stepper motor and pulley
Stepper motor and drive system.
spoke weights for wheel balancing
Spoke weights with 3D-printed inserts.

The stepper motor spins at microstepped precision so the wheel holds a steady 8 fps. Motorcycle spoke weights slide along each spoke for fine-grained balancing — the wheel with all scene boxes clipped in can be very heavy and unevenly loaded. A spring-loaded chain tensioner keeps rotation smooth.

bike chain tensioner
Chain tensioner for smooth rotation.
flashlight controller schematic and PCB
Custom flashlight controller PCB.

To strobe the light, I designed a small PCB that replaces the driver board inside a store-bought Pocketman flashlight. The flash lasts only microseconds to reduce motion blur — short enough that the Arduino's digital write was too slow, so I used a faster GPIO library. The viewer holds this flashlight and chooses where to shine it each revolution.

flashlight controller installed
Controller soldered inside the modified flashlight.
viewer holding strobe flashlight
Viewer holding the strobe flashlight.
zoetrope beneath curtains
Black curtains give each viewer a personal experience.

Black fabric surrounds the structure so ambient light doesn't wash out the strobe. The entire mechanism can be exposed by lifting the curtain — a chance to show how the animation device actually works.

View from inside the zoetrope with strobe lights.

The Audiotrope: light-reactive audio

The Audiotrope extends the interactive zoetrope by detecting where in the scene the viewer is shining the flashlight and playing audio tied to those objects — what I call Action Objects. Shine the light on the pancake and hear it sizzle; shine it on the falling girl and hear her yell.

How it works

The system needs two pieces of information: which of the 16 frames is currently visible, and where in that frame the flashlight beam lands. An Arduino Uno tracks frame number via the proximity sensor; a USB camera captures each strobe-synced image and sends it to a Raspberry Pi running OpenCV.

data flow through the Audiotrope system
Data flow through the Audiotrope: motor, sensor, strobe, camera, Raspberry Pi, speaker.

Frame tracking

frame tracking with proximity sensor
Extra nut on the last frame resets the counter.

Each scene holder has a metal bolt detected by the proximity sensor. Only the last frame has an extra nut — a shorter elapsed time between detections resets the frame counter from 16 back to 1. The current frame number is sent serially to the Raspberry Pi over USB.

Light tracking & image processing

When the sensor detects a bolt, the strobe flashes for ~500 microseconds and the camera captures an image. On the Raspberry Pi, Gaussian blur, thresholding, erosion, and dilation isolate the flashlight beam. If the beam falls inside an Action Object's hitbox for the current frame, audio plays.

control loop diagram
Control loop: wheel spin triggers light and camera; image processing checks for Action Objects; audio plays if matched.

Calibration

file structure for images and audio
Directory layout for Action Objects and audio.

Before playing an animation, the artist calibrates Action Objects using OpenCV template matching. For each object in each frame, a cropped template image is matched against the full scene image to find hitbox coordinates. Audio files are associated with each object through a structured directory layout and saved as JSON for the Raspberry Pi to load at runtime.

calibration process diagram
Template matching locates the falling girl in frame 9 of the Pancake Flip animation.
template matching result on frame 9
Template matching result: the girl-falling template identified within the full scene.

Playing the animation

Turn off the room lights, press the big red start button, and launch the Python script on the Raspberry Pi. The viewer holds the flashlight and explores the scene — audio fades in and out as they shift focus between objects, using PyGame mixer channels so clips can pause and resume without abrupt cuts.

Making animations for the Audiotrope

boolean operations clipping geometry to scene box
Boolean difference clips geometry to the scene box.

Animations are authored in Blender at 30 fps, then regularly sampled down to 16 frames (~8 fps on the wheel). Scene boxes are 9 × 16 cm; geometry outside the box is clipped with boolean operations before printing. Models from online libraries often need Blender modifiers — Solidify, Decimate, Remesh — to become printable.

frame number carved into printed base
Frame numbers carved into each print's base.

A Python script in Blender carves frame numbers into the bottom of each print so assembly stays organized across dozens of parts. Each scene is split into smaller prints to speed up fabrication, then magnets are glued into the back of each box (mind the polarity!) before clipping to the wheel.

16 frames split into smaller prints
Each scene is split into smaller prints for faster fabrication.
3D-printed scene parts
3D-printed parts assembled into scenes.
painted 3D-printed scenes
Labmates helped paint scenes for contrast.

Painted scenes improve contrast under the strobe — important when the viewer is searching for small Action Objects with the flashlight beam.

Stories

Pancake Flip

hidden supports in pancake zoetrope
Invisible supports mid-air.

A girl flips a pancake too high and has to jump after it. She misses and lands on the floor. Three Action Objects drive the audio: the pancake (sizzling), the girl while falling (``Noooo, my pancake''), and the girl on landing (thud). Hidden support rods hold the girl and pancake invisibly mid-air — generated using the method from my Eurographics hidden supports paper.

pancake flip scene box
The Pancake Flip kitchen scene.
pancake animation in Blender
Animation authored in Blender.
pancake flip animation frames
She flips a pancake but miscalculates.
Painted kitchen scene on the wheel.

City Girl

City Girl in Blender and printed
Blender (left) and printed (right).

A girl sits on a rooftop bobbing to music, phone ringing beside her. Buildings in the distance hold different people — parents arguing, someone watching sports, animals playing poker. The visuals stay the same; different audio tracks tell happy or sad versions of the story. Focus on the girl to hear her music or inner thoughts; focus on her phone to hear it ring; focus on buildings to overhear conversations inside.

unpainted city frames
City frames before painting.
printed figure before PVA bath
Before PVA bath.
printed figure after PVA bath
After PVA bath — ready for painting.

Detailed city buildings required dissolving PVA support material in a bath before the figures could be painted and assembled into scene boxes.

Monster Under the Bed (in progress)

A boy sits in his bedroom — seemingly ordinary. But when the viewer narrows the flashlight beam and looks closely, hidden animals and a monster under the bed are revealed. This uses frame-set switching: odd frames show the empty room; even frames contain the hidden creatures. A wide beam strobes odd frames only; a focused beam switches to even frames, revealing what was hidden in the dark.

Future directions

Capturing a zoetrope on camera

camera synced to zoetrope
Camera synced to the spinning wheel.

Zoetropes are an in-person experience, but conferences want proof they work. I synced a programmable uEye camera's shutter to the strobe flash via an 8-pin connector on the Arduino, capturing one clean frame per revolution — much easier than extracting frames from high-speed video after the fact.

Technical stack

Outcomes

UIST 2022 Extended Abstract · UIST demo video · ACM DOI · PhD thesis (Ch. 2 — Zoetropes)

Thank you to everyone who helped: Esther Lin, Varun Parumal, Paul H. Dietz