Some of my favourite childhood memories are of playing EyeToy with friends. A camera plugged into the PlayStation, a game on the TV, and suddenly our bodies were the controllers.
There’s a particular feeling you get from that kind of play. You reach for something and the game responds. You move a little too much, look a little ridiculous, and your friends laugh. The fun spills out of the screen and into the room. Even watching someone else play becomes part of it.
EyeToy made that feel possible with a little camera. What stayed with me was how much fun it was to move around and play together.
That feeling came back
Years later, playing Party Fowl brought it back. It uses a device’s camera to turn body movements into controls for wonderfully silly games. There was that familiar feeling again: moving, reacting, and laughing at what the game was asking us to do.
It reminded me how much I like games that give you a reason to get up. The movement is part of the pleasure. So is being in the same room as the people you’re playing with.
That’s the feeling behind Playfield, an experiment I’m building with an iPhone, a projector, and games you play with your hands. The game goes on the wall. Your hands do the rest.
Let the room join in
Spatial computing has interested me for a long time, especially mixed reality, augmented reality, and tangible interfaces. I like the possibility of reaching into a digital experience through the space and objects around us.
Folk Computer was a big inspiration. It uses cameras and projectors to bring computation onto physical surfaces and objects. Paper can become an interface. Moving something on a table can change what a program does. The computer gets a place in the room, alongside the people using it.
What drew me in was how much room that leaves for experimentation. An interface can be something you touch, rearrange, or share with another person. Seeing Folk made me want to explore that direction myself.
Playfield starts with a small piece of it: a wall that responds to your hands. Something you can walk up to, understand by trying, and play together.
A smaller setup
Interactive walls and floors have existed for years. Systems like LUMOplay already turn projected surfaces into places to play. Their recommended setups combine a computer, a projector, and a dedicated 3D camera.
That made me curious about what a phone could take over. A modern iPhone brings a camera, the compute to run hand-tracking models, and, on supported models, LiDAR depth sensing into one device. AI vision models can locate hands in the camera image; depth can help with interactions that need to know how close a hand is to a surface.
For this experiment, the phone handles tracking and runs the games. The projector makes them big enough to play on a wall. LiDAR is used in the wall-touch mode; the other games use camera-based hand tracking.
There’s still a projector to connect and a phone to mount. But being able to try this without a separate computer and tracking camera makes it much easier to keep experimenting.
A wall with a few new rules
The phone first looks for markers projected at the corners of the play area. That lets it map what the camera sees to where things appear on the wall. Once calibration is ready, the markers disappear and the game takes over.
In Meteor Keeper, your hands steer paddles to bounce meteors away from a city. In Counter Crew, you grab tomatoes, chop them with your hand, cook imaginary soup, and serve it. There’s a drawing mode for leaving trails across the projection, and a physics playground for grabbing and dropping a ball.
The games are deliberately small. They’re ways to find out which interactions feel good: reaching, grabbing, opening your hand to let go. When the hands leave view, the games can pause. Even an imaginary kitchen should let you take a break.
What I want to build toward
Playfield is still an experiment. A lot of the work is making the tracking, calibration, and gestures feel natural enough that you can pay attention to playing. A hand movement should do what you expect, without making you think about the camera watching it.
The goal is a setup that’s easy to bring out, with games that invite people to join in. There’s plenty to explore beyond these first few games, but this feels like a good place to start.
EyeToy gave me some of my favourite memories with friends. Party Fowl reminded me how good that kind of play can feel. Folk opened up more possibilities for what the space around us could do.
Now there’s a little city on my wall, and it needs saving.