There are an infinite number of ways to start coding a fresh website, but the HTML5 Boilerplate is one of the best. It starts you out with all the HTML, CSS, and JavaScript, that you’ll most likely need to start any website. Exactly one year since the initial release, version 2.0 is now available with a ton of great updates!
Head over to http://html5boilerplate.com/ to read release notes and download the latest and greatest version.
Pick up a pen and draw a sketch. There, that was easy – however crude, you can get out an idea. Sketching with paper is still the fastest way for most of us to imagine something. But between that immediacy and the end result, you need prototypes.
The Processing language has long been one of the easiest ways to sketch an idea in code – best after you’ve first put pen to paper, but as an immediate next step (and for ideas you just can’t draw). Built in Java, the creation of Ben Fry and Casey Reas and a broad community of free software makers, it runs on Mac, Windows, and Linux. Via Processing.js, the same API works in a browser via JavaScript. It has inspired the OpenFrameworks project, which uses nearly the same API. Those tools let you intermingle Java, JavaScript, and C++, as they’re written natively in those languages.
Processing now runs just as easily on a mobile platform with Android. You can try it with the free SDK and emulator, but it’s most fun with a device. Using all free software, you can sketch easily on mobile and desktop from one environment, and with only minor modifications, run the same code on a desktop, a browser, and a mobile device.
Translation: with one, elegant API, you can “sketch” visual ideas on screens from an Android phone to a browser to a projection or installation. As a prototyping tool, or for finished projects, that makes it an exceptional, expressive tool for making interactive visuals for screens.
This is a first-draft tutorial, as I make the same presentation in Stockholm at the info-rich Android Only conference. I’m eager to put it out there and find out that people have problems, as I can then improve the documentation. So do give it a try – especially if you’ve got (or can borrow) an Android phone.
Processing on Android performs all of the core functions of Processing on desktop – 2D and 3D visuals, data manipulation, images, and type – and you can mix in Android code using the standard Android APIs, right in the same project. Processing controls the screen on which it’s drawing, for now, but you can mix and match everything else, to support multi-touch, sensors, and even NDK code. (I’m using it with Pd for Android, with Pd doing sound and Processing doing visuals.)
I’m assuming basic familiarity with Processing, so if you haven’t tried it out yet, check out the excellent tutorials available online to get rolling.
Definitely read the latest official instructions:
http://wiki.processing.org/w/Android
1. Download the latest pre-release version of Processing. Eventually, the standard Processing download will support Android automatically out of the box – very cool. The current stable version doesn’t yet have Android support, however, so for now, you’ll need the latest pre-release build, which includes all the code for the testing release of Processing for desktop and Android. I tested with 0190.
2. Download the Android SDK. For now, you do need to download the SDK for Android separately, which is an extra step – but at least it’s completely free, and runs on any OS. See:
developer.android.com (This step will eventually go away.)
You can put the SDK anywhere you want; just make a note of where you’ve installed it, as you’ll need to point Processing to that location later. Follow the instructions carefully on Google’s site. On Windows, you’ll need to download the USB driver. On Linux, you’ll want to read the complete Developing on a Device instructions, as there are a few short commands you have to enter at the command line because of the way Linux talks to USB devices.
Lastly, even though that SDK is a big download, you’re not done until you download additional components from Google. This allows a single recent version of the SDK – like the 2.2 SDK I tested – to support a variety of older versions for backwards compatibility.
I have Eclipse installed, because it’s a handy tool for developing Processing for desktop and Processing for Android on any OS. Once you’ve installed the Eclipse plug-in for the Android SDK, you can add components from inside Eclipse. Inside that IDE, choose Window > Android SDK and AVD Manager Manager and you’ll get a graphical interface for adding these components (pictured).
You can now also access the same interface from within Processing. Choose Android > Android SDK and AVD Manager.
For Processing 0190, you need the SDK Platform 2.1, API 7 and Google APIs by Google, Android API 7. (If you’re an Android developer, you’ll probably also select some other components here, as well.) Components are available under the Available Packages.
If for some reason you prefer to use the command line, head to the Windows, Mac, or Linux command line, and follow the commands in the Quick Start under the Android SDK download instructions.
3. Run Processing, and switch to Android Mode. Load the Processing IDE you just installed, and switch to Android Mode.
The first time you do this, you’ll need to direct Processing to your Android SDK. Choose the root directory of the SDK – the directory that contains “add-ons,” “docs,” “platforms,” “tools,” and the like.
4. Try running a sketch on the emulator. I’m compiling a set of simple sketches you can use with Android, but even many of the basic examples will work out of the box. For fun, you can try sketches like those found in the Examples > Basics folder and some of the other included Processing Examples. They don’t take into account varying screen resolutions, which is a cool feature of Processing for Android, but it’s still fun to watch them run.
It’s simple to run a sketch. Open the sketch you want, go to Android > Android Mode to enable Android Mode for the sketch, and then hit Run. Instead of opening in a window as desktop Processing sketches do, you’ll see an Android emulator appear.
Be prepared for the emulator to take … some … time. You’ll spend a long time looking at the load screen. When it does finally load and install, there may still be a delay as you stare at the main screen. You may have to hit run again. You may see error messages saying an application has become unresponsive – don’t worry about that; Android has a low threshold for applications timing out, so while the emulator is loading, just choose “wait.” And because the emulator defaults to a large screen size, it’ll also take up some room on-screen.
It’s just slow, painful, slow, and unreliable. Fortunately, running on the device is near-instantaneous and speedy, so beg or borrow access to one and try the next step.
5. Try running a sketch on a device (if you’ve got one). Running on the emulator is a pain, but running on a device is very quick – even quicker than you might imagine if you’ve come from a background on iOS. (Remember, you’re working with Java, not native code.)
In a clever design feature, to run on the device instead of the emulator, simply choose Present instead of Run. (The quickest way is simply to shift-click the play button.)
So long as you have a device connected with USB, and USB debugging is switched on, and you’ve done the setup correctly above, you’ll see your sketch appear on the device.
Now, that’s more like it.
“But, wait,” you say. “Android devices have all different screen sizes. Augh, fragmentation, fragmentation!”
Fret not: computers have different screen sizes. Projectors have different screen sizes. And Processing on Android can easily adapt to the size of the screen. We won’t need much Android-specific code, but let’s get started with our own Processing sketch and see how it works when you’re preparing for mobile.
Okay, so what does a Processing sketch for Android look like? Here’s a really simple example, just to show off basic drawing, managing different screen sizes, and simple (single) touch. (I’ll leave density for another day; multitouch and other touch events are also possible.)
float sw, sh, touchX, touchY;
void setup() {
size(screenWidth, screenHeight, A2D);
println(screenWidth);
println(screenHeight);
sw = screenWidth;
sh = screenHeight;
}
void draw() {
background(0);
smooth();
fill(255);
noStroke();
ellipse(sw/2, sh/2, sw/4, sw/4);
noFill();
stroke(255,0,0);
strokeWeight(2);
ellipse(touchX, touchY, sw/4, sw/4);
}
void mouseDragged() {
touchX = mouseX;
touchY = mouseY;
}Two things to note, in particular:
size(screenWidth, screenHeight, A2D);
(Warning: don’t put variables in that size() command.)
Processing for Android will map the desktop renderer names to either A2D (for 2D graphics) or A3D (for 3D). (I’m using the Android names here, but the other names will work just fine.)
Also, since static pixel sizes don’t make sense on mobile platforms with different screens, you can substitute “screenWidth” or “screenHeight” and get dimensions from the device on which you’re running. (I just used “sw” and “sh” because they’e shorter, and then I can make everything else relative.)
void mouseDragged() {
touchX = mouseX;
touchY = mouseY;
}Here’s the really cool part — touch input works exactly the same way as it does on Processing on a desktop, for relevant mouse commands (nothing involving these strange “buttons” of which you speak). mousePressed, mouseDragged, and mouseX and mouseY all work as expected, so long as you’re satisfied with simple, single touch and don’t need gestures, multitouch, and the like. Don’t be confused by my touchX and touchY variables — those are just to demonstrate that my circle only moves when it’s “dragged” — or your finger slides across the screen. What we have is the beginnings of a drawing app, which I’ll be expanding over coming tutorials.
Here’s the other cool thing: modifying only the references to screenWidth and screenHeight, and the renderer, I can run exactly the same code in Processing on the desktop, or even in Processing.js on a browser.
This is not a replacement for the official wiki documentation, so do read that, please.
In coming installments, I’ll share:
1. How to get Processing for Android working inside Eclipse. In short – it works. That makes adding Processing to an existing Android app easy. (Hoping to follow that up with gedit and Ant)
2. More advanced tricks for dealing with screen dimensions and touch.
3. Android-native snippets for connecting sensors.
4. How you can combine sound and visuals with Pd and Processing, both running on Pd.
Other things that are possible to do with Processing for Android (which I’ll cover):
I also hope to take some notes on what happens here at Android Only.
Processing for Android is still in development, and this is pre-release software. So be prepared to encounter issues. That said, I want to refine this documentation, so please let me know if you encounter issues with my instructions.
Here’s my presentation from Stockholm:
Noisepages registration is open again as we finish the site. Share what you’re doing, devices you’re testing, or other thoughts:
http://noisepages.com/groups/processing/forum/
You’ll hear odd cynicism about people working on free software / open source projects. Something like, “well, harumph, it’s not as though a bunch of people will make this stuff in their free time.”
Not only are these folks wrong, but you can actually visualize the contributions to source trees – and the results look spectacularly hypnotic. It’s free software – the music video.
Okay, now, granted, I may get so mesmerized by the results that I’ll just spend time staring at that instead of getting actual work done, but – working too hard isn’t good for you, anyway. It’s an organic high, audiovisuals.
At top, Ryadh Amar sends in a visualization of the excellent, lightweight LXDE windowing environment for Linux. (Actually, I’m inspired to give LXDE a fresh install.) At bottom, a collage of various projects showing that these data visualizations can take on various identities. Gource can support just about any project repository, too: Git, Bazaar (popular on Ubuntu), and Mercurial (recently added to Google Code, incidentally) are available native, and CVS and SVN are available as third-party extensions for those of you kickin’ it oldskool and non-distributed. (Though, really, come join the 21st Century – it’s awesome.)
And Gource can visualize itself. Freaky. It’s all thanks to the ongoing awesomeness of OpenGL.
http://code.google.com/p/gource/
I’d love to see this added to project management so you’d have a sort of live, superb visual to inspire you to keep the code moving forward.
Who knew source code would turn out to be so visually inspiring? (Now I just need a new way of visualizing me writing bad code and then correcting and cleaning it up. I think it could be best represented as a set of stick figures getting stuck in quicksand and hitting each other over the head. Then there could be a big Smoke Monster that represented the Evil Force of Procrastination.)
But wait! There’s more! You can visualize web logs, too. (It works with Apache; I have to see if I can make it work with our nginx logs, as visualizers could actually be very beneficial with the kind of complex data you get in something like a web log):
Boards Interactive Magazine – Walkthrough from Theo Watson on Vimeo.
Augmented reality has inspired plenty of experiments, magazines included – some successful, some failing to get far beyond the gimmick. What’s nice about this work is that it introduces the concept of motion to the typically-static pages of print, and inspires readers to imagine a world beyond the bounds of the page. There’s also an elegant expression of the theme.
It’s all made with OpenFrameworks (again – yes, OF is one of the easiest ways to hook into augmented reality). And none other than Theo Watson was involved.
For the March 2010 issue of Boards Magazine, Emily Gobeille and I worked with Nexus Productions to develop an interactive cover experience called Rise and Fall. Here is a little preview of the experience.
You can download the software and the cover from: boardsmag.com/RiseAndFall
Update: Found out you can buy a copy of the magzine for $7 by emailing – BoardsCustomerCare@boardsmag.com . You can also download the cover as a pdf from the link above.
The project uses the Ferns library for tracking ( cvlab.epfl.ch/software/ferns/index.php ) and the whole project is open source released under the GPL v2.0 . Grab the source code here: boardsmag.com/RiseAndFall
Credits:
Digital Directors:
Emily Gobeille – zanyparade.com
Theo Watson – theowatson.comProduced by:
Nexus Productions – nexusproductions.comSound Design:
MOST Original Soundtracks – m-ost.nlSoftware:
Made with openFrameworks – openframeworks.cc
Using the Ferns library for tracking – cvlab.epfl.ch/software/ferns/index.php
Thanks to @wetterberg via Twitter for sending this our way.
Google’s open-source Android mobile platform runs on Java. Processing, the elegant coding language for visualization, art, and media, is built in Java. The marriage of the two, therefore, is one we’ve long been anticipating. Processing’s ability to focus on lightweight, portable implementation is a perfect match for the demands of mobile development. For artists and visualists wanting to make handheld devices and tablets more expressive, I have good news: it’s coming. While not fully implemented or ready for widespread testing yet, Processing is in active development on the platform.
Ben Fry, co-creator of Processing, has been hard at work making Processing run on Android. Interestingly, in some ways Android itself vindicates the direction Processing charted years ago. Google made their platform work based on open-source development tools. They removed parts of the Java platform that would have overwhelmed the limited processing power of handhelds – even as a handful of purist Java developers cried foul. Graphics focuses on lightweight 2D and OpenGL drawing output, much in the same way that Processing strips Java to its bare essentials.
Best of all, doing Processing development for Android promises the same streamlined, sketchbook-like production flow that Processing does on desktop. To create Android apps, you simply set Processing to “Android mode.” (Right now, you have to point to Google’s SDK, but that happens only once and may be removed in future versions.) Hit run, and your sketch launches in the Emulator. Plug in an Android device via USB, and it’ll run on your device. Soon, you should have one-click-export of apps in the way that you do Processing desktop apps.
This also suggests some great possibilities for cheap, handheld devices, installations, and reliable portable devices for visual performance. Sure, right now a lot of Android-powered devices require a phone contract and don’t output video, but a lot of devices slated for later this year are media devices with video output and tablets, too.
Processing for Android is an early work-in-progress. If you’re thinking about playing with it and you don’t know what you’re doing, or you have little patience for bugs and wrinkles, you should wait until the waters grow safer. Right now, the team working on testing the port is intended to be limited, although as someone testing it myself, I can say it’s already a lot of fun and holds a lot of promise for the future. OpenGL support is currently not available, but it’s coming – and I’ll say again, if you want to know what Processing’s 3D future looks like, check out the superb GLGraphics library. Anyone who thinks Processing can’t be fast, or do intensive GPU work, or mix HD video (okay, not on Android, but on desktop at least), check out this library. OpenCL should also be possible soon.
http://android.processing.org/
BIG, BIG disclaimers! Please don’t go testing Processing on Android assuming it’ll work out of the box – for the brave ONLY, at least for now. (That should change very, very soon, but I couldn’t keep the news under my hat any longer.) As the disclaimer says (hilariously):
Do not use this code while operating heavy equipment. Do not rely on this code for thesis or diploma work, as you will not graduate. Do not use this code if you’re prone to whining about incomplete software that you download for free.
A number of us have already begun talking about the possibility of adding libraries to connect Processing’s capabilities to Google’s own APIs for the phone, SMS, sensors, and so on. Sound and synthesis via external libraries should also be possible.
I think Processing for Android is the perfect complement to openFrameworks for native code on mobile platforms. Right now, OF has already been used for terrific work on the iPhone. It’s not quite as user-friendly as Processing for Android for a number of reasons, but it has Processing-derived syntax, aesthetics, and philosophy, and it’s also free and open. For a superb guide to developing on the iPhone with OF, plus some links:
http://www.openframeworks.cc/setup/iphone
OF does not appear to be a practical solution for Android development, not least because Android requires apps to be distributed as Java, and uses Java to talk to all the native APIs. But it’s a good option for some of these other devices, and you can run the two alongside one another in Eclipse if you like, more advanced users.
With OF on the iPhone (and presumably other native platforms in future – Nokia N900, anyone?), and Processing for Android, plus tools like Pd and SuperCollider for synthesis (more on that soon), there’s no reason not to go completely open with portable interactive art on mobile devices.
I have finally found a venue to show a decent working version of bitPong, a piece I created some time ago when I was still working on the close relationship with phsyical implementations of data and their aesthetic consequences.
The idea is simple: a two-player game, based on the uber-referenced Pong, here played with 8-bit controllers. When I say « 8-bit controller », I mean literally 8-bit, i.e. 8 buttons, each representing 1-bit of data. Collected together, this byte represents a 256 value variable which is used to control a visual paddle representation within the game. To aid players in the conversion of 1-bit discrete switches into their collective base-two 8-bit value, each button has been labelled: 2^n, i.e. two to the power of zero, two to the power of one, two to the power of two, and so on. This is otherwise known to mere mortals as the values 1, 2, 4, 8, 16, 32, 64, and 128. To move your paddle, you must add each of these values together in order to position it somewhere between position 0 and position 255.
For those who know little about how the computer works internally, this is how the computer moves from the well-known binary 0 | 1 value, to complex values such as the letters you are reading right now: by associating a different value to each bit (the 1 values of « 0 1 0 1 0 1 » get converted to « 0+2+0+4+0+8+0+32 », otherwise known as the value 42) the computer can use a physically limited scheme (0 or 1, on or off, yes or no, true or false) in order to represent a far greater sum of possibilities (here a number from 0 to 255). bitPong plays off of this configuration and brings its dynamics to the surface of the playing field. In order to take control of your paddle, you will have to quickly master binary encoding.
In this Victor Vasarely inspired version of bitPong, hexagons populate the playing field and create an added diversion. Therefore, bitPong has now turned into something like a two-player bitBreakout. I was actually inspired by the following sign which is posted on the wall just next to my installation, indicating the escape routes out of the museum.
I have to admit, even considering the current legal battle of the Fondation, and the related embezzlement of it’s holdings by its president / family members, all leading to the current dilapidated state of this curious monument, it’s still a pretty cool place to show work.
AIR is very popular for creating twitter clients, Sönke Rohde just made it much easier to make AIR apps for twitter with an AS3 library for Twitter. This library is built on top of core oauth as3 library by iotashan. Core OAuth as3 library is a standard OAuth library this can be used for your own OAuth backends or connecting to other OAuth services as well.
A very nice feature of this library, in addition to being coded cleanly and as3 style, is the ability to have the Twitter OAuth page render inside of Flash.
Instead of opening the Twitter authorization page in the browser the library also contains OAuthLoader which is a wrapper around HTMLLoader which enables to directly show the authorization page within an AIR window:
// use this in the requestTokenHandler instead of navigateToURL var loader:OAuthLoader = new OAuthLoader(); loader.load(request); loader.percentWidth = 100; loader.percentHeight = 100; var w:Window = new Window(); w.width = 800; w.height = 400; w.title = req.url; w.addChild(loader); w.open();
Interface 27 from CyberPatrolUnit on Vimeo.
There has been a long tradition in live visuals and motion graphics, inherited from many other media, of maintaining a “secret sauce,” or the guarded formula of eleven herbs and spices. Ironically, for all you hear today “DIY” and “open source” in the same sentence, a lot of the motivation for doing something yourself has historically been doing something no one else can. Keep your secrets, and raise your value.
As our friend Bryant Place / CyberPatrolUnit sends over this latest set of live clips from a recent gig, and I browse through the comments, and reflect on the conversations I had last week at OFFF and during and following my own talk there, though, I’m struck.
The world has changed. First off, the Internet isn’t really about secrets. Your value is almost in direct proportion to how much you can share. Connections are forged through links of mutual exchange and good will. It’s not just about sharing your output or getting fans (the MySpace model), but sharing with a network of enthusiasts, and fellow artists. Those are the people from whom you often get real support (artistic, technical, and personal), gigs – and inspiration. (Even if you hate 8-bit music, that community is a really amazing model: their work to support each other and advocate for the whole subgenre has been I think the single biggest ingredient in their viral success.)
The visualist community increasingly itches not only to improve the quality of their own individual work, but everyone around them. A lot of us are in a battle for the future of this whole medium. Some parts of the world are devoid of live visuals, while others have mass-produced club visuals filling the nightlife.
Before I get carried away, the video itself is just the latest from the ongoing Interface 27 series. It employs a touch interface to control abstract visual pictures formed from streams of particles.
The reason I’m pulling back into the larger question is that these visuals are enabled by a library for Processing, a library we’ve seen here previously, developed by Memo Atken:
MSAFluid for processing (and Java)
If you’d rather use openFrameworks, there’s that version, too, as pictured below running blazingly fast:
ofxMSAFluid for openFrameworks
There’s even an ActionScript 3 port, in case you want to code Flash on the beach.
ofxMSAFluid for openFrameworks from Memo Akten on Vimeo.
So, why do I bring this up? Well, the work done on Processing (Ben Fry, Casey Reas, contributors like Karsten Schmidt, and others), on openFrameworks (Zachary Lieberman, Theo Watson, and their own team), and Memo’s own library, based in turn on many other libraries and implementations, was all a big risk.
It’s not an easy thing to put blood, sweat, and tears into open source. None of those people has exactly gotten rich in the process – not even via the ways you’re supposed to profit from open source, doing the lecture circuit and such. But on the other hand, we’re seeing things that would have been otherwise impossible.
And there’s artistic merit, too. Bryant’s work looks different than Memo’s. The library actually takes on a new life as it gets in someone else’s hands. Bryant actually just wrote me:
As for the Interface video - mention how cool it is that people like Memo post code for other VJ’s to tweak and use. Mention "FaderTouch" - a 100buk touchscreen off ebay that "vjFader" programmed - using a rear projection onto a translucent screen/ touch sensor we were able to use processing in a very intuitive way.
I got the “mention” part down, I guess. ;)
The responsibility is partly ours to make all of this work: file bug reports, fix bugs if you can, document your work, properly credit the people making it, write documentation for projects, and so on. But it’s not hard to see an ideal start to happen:
1. Person x makes a library / framework.
2. Person y build on that library to make their own tool – and contributes it.
3. Artist uses the tool, gives back to the project, goes in a new direction.
4. More and better work spreads, the project grows, the medium grows, and the audience grows.
None of this happens automatically. We all have a lot more work to do. But having stood onstage in front of a few thousand people calling for just this, it’s nice to keep opening my inbox and seeing it happening. We’re seeing the first seeds planted for what could ultimately be a larger ecosystem. Now, I know there’s also a big gap left – Processing doesn’t have nearly enough contributors, bug squashers, or documenters, and it’s one of the biggest projects, so you can imagine what happens when you get upstream to libraries and the like.
Over the coming months, I think we’ll continue to look for opportunities to help structure some of that involvement and to explaining how you can contribute, too. Stay tuned.
In the meantime, go play with some particles.
For more on Bryant, here he is on his current activities:
- I just did Coachella with [Friend of CDM and contributor] Momo, and in the near future, will be heading to Detroit for http://www.myspace.com/detroitmusicfest
I’m not on the website, however, Kero.fm and Derek Michael - two people who essentially helped build the festival from the ground up 10 years ago - are booking me to play with various acts including CLP, Richard Devine, Drumcell, Busy P (which I did a solo VJ set with at Coachella) so I am super excited to be a part for the first time this year.
Here is a cool video from previous Interface 26:
After Detroit - Mutek.
There are also some killer podcasts from past Mutek - http://www.mutek.org/podcast
I am going to meet artists, see the latest AV performances, attend workshops.
I’ll be at Mutek, too, so see you there.
Last week, Stéphane Cousot wrote a basic library for communicating between Processing and Pinguino. While you’ve probably heard of Processing before (since you’re reading this blog) you’ve almost certainly never heard of Pinguino. It’s one of the roll-your-own projects that we are developing here at the Aix-en-Provence School of Art in order to fill our own needs. It is loosely based on the Arduino platform, most of all its philosophy, but redesigned around PIC microcontrollers with integrated USB. In other words, it’s our in-house Arduino.
The project is in its first year of development and experimentation, and a lot remains to be ameliorated, most of all the editor and the installer. But as in the past we have found that the best way to mature our technologies is to use them in our own work as if they already work (or at least as if they will work), we figured that we would just jump in head first and start developing all the various pieces, even while we’re in the middle of building crazy festivals like the next Eniarof. So while Stéphane is here for Eniarof, I decided that it would be best to put him to work on developing a Processing Library to connect up directly with the Pinguino circuits various projects are building for their attractions.
Jean-Pierre developed the original solution using the libusbJava wrapper to libusb. This is not a virtual serial port, but direct communication with the USB bus. Stéphane then took this solution and built the basic library that we will use to further develop the communication protocol between Processing and Pinguino. One of Jean-Pierre’s original ideas for his platform was to build a default mode that would allow Processing, Pure Data, Python or one of the many other languages and environments we use, to directly command the Pinguino. There is something similar to this idea in the current Firmata protocol built in to the Processing Core Libraries. But we’re still a long way from getting there. For the moment we just have a simple protocol to get pin states, change pin states and send/receive more generic, or non-pin related messages.
Here is a video of Stéphane running his demo:
And some photos:
