Whew!
It’s been a massively intense few weeks getting my PixelPhones project ready in time for my presentation at FOTB. I’ve been coding day and night for months, and I was up until 4am the night before making sure that I’d thought of everything.
The worst part was that I had no idea if it would work – it’s kinda difficult to get hundreds of phones together to test it. But it did work – and the results were better than I could have hoped for!
thanks to Momo for the amazing video
PixelPhones is this crazy project where I turn each phone in the audience into pixels in a large display. It runs in your phone browser so you don’t need to install an app.
Built with openFrameworks in C++, the app sets up its own WebSocket server and phones connect straight into it over the local wifi network (although in future I hope to get this working over 3G).
Of course socket connections are two way so I couldn’t help but take this chance to involve the audience too – I made a game where Nyan Cat runs from screen to screen, and whoever catches him fastest wins! (partly inspired by Rob Davis’ Cat on Yer Head game)
The tricky bit was figuring out where the phones are – GPS just isn’t accurate enough. I was inspired by the Junkyard Jumbotron – it puts markers on each phone, then you take a picture of them all. But in my app a marker would be way too small so instead I use a unique sequence of flashes.
It uses OpenCV to find the flashing objects, read the pixels, and decode the sequence.
Latency was also an issue, even on the fastest networks the packets arrive at slightly different times. And when you want super fast smooth animation, unpredictable timing is unacceptable. I solved it thanks to Jobe Makar’s excellent method of syncronisation outlined in his ActionScript multi-player gaming book.
I’m so happy and relieved that it worked so well and the timing was split-second perfect. The next day I put myself through the whole ordeal again, this time with a larger audience and only 10 minutes to demo (at the FOTB Jam session). Over 220 devices connected and I could have handled more but we ran out of time.
This is just the start, and I have many many ideas that I have yet to try. I’ll also be open sourcing this code when it’s finished – I can’t wait to see what the rest of you do with it! Both sessions were recorded in full, I’ll let you know when they’re uploaded.
No related posts.
