something in the way

For people coding for visuals, Processing just keeps getting better. And for people who aren’t … well, you might just want to give it a second look, as a growing global army of people who never fancied themselves coders suddenly start typing up new creations. A new release makes mobile development easier and corrects lots of bugs. But specifically of interest to readers here, powerful libraries for 3D help make Processing an intensive tool for creating visuals. With the aid of running of your GPU, they can also deliver eye-popping real-time performance not normally associated with Java.

Processing 1.5

In a surprise release, much of what’s coming in Processing 2.0 this summer is now available in a stable, general release of Processing 1.5. It lacks the new built-in renderer – OPENGL2 – but does incorporate new features for developing for Android and, via the third-party library GLGraphics, you get all sorts of new OpenGL goodies. (Note: Processing 1.5 = 0196. The previous pre-release version, 0195, is worth downloading if you want to play with the new OPENGL2 renderer and its examples. Both are on the download page, and they can be installed side-by-side.)

Most importantly, for anyone publishing to the Web, you need to download this version now and re-export sketches. Applets were seeming to run very slow in Chrome and Firefox 4. (While I vastly prefer the Java version for things like performance tools or installations, I also appreciate the Processing.js JavaScript fork for Web delivery – but this fix does make applets work pretty well.)

New to Processing 1.5:

• Tons of editor fixes in the PDE development environment
• An essential bug fix that corrects slow performance when exporting for the Web.
• Android development support – in preview builds, but now in a stable build. (I think this is still considered a pre-release feature, but it means you can run the stable build to try it out.) Now go make stuff for phones and tablets easily.

Processing 1.5 is a good start, but for me, the live visual workflow isn’t complete without two additional libraries, toxiclibs and GLGraphics. A new GLGraphics update improves integration with both toxiclibs and the new Processing release.

toxiclibs

toxiclibs represents over 270 classes in 7 libraries, some 25,000 lines of code, written by digital artist Karsten “toxi” Schmidt. There’s gobs of stuff in there. The most useful is a basic set of classes for things like geometries, meshes, and 3D vectors which you’d otherwise have to build from scratch, and which are generally built in fairly standard ways. With elegant math, Verlet physics, and color libraries, and wonderfully-usable API design, Karsten gives you all the essentials in a way that will inspire you to use them and make something truly original. (A full tour is probably a good subject for another post. Just ignore the toxi.audio packages; with the libpd crew, I’m working on something with Pd I think you’ll find more useful and stable.)

I had a discussion with one colleague who felt that, indeed, toxi’s libraries are so powerful that people are simply using it as a crutch. That may be true to an extent: people should prominently credit toxi’s work, and to do otherwise is plagiarism. But with proper credit, I feel that standing on the shoulders of someone else’s work can be a good thing. Digging through toxi’s libraries is like going to school for the sorts of math and geometry that you need to learn to understand 3D generation. For many of these classes, involving essential mathematics operations and 3D modeling, I’d have no idea where to start, would spend weeks or months writing something inefficient, and would come out with something that reinvented the wheel. A lot of the techniques themselves in those 27,000 lines of code weren’t developed by Karsten, either: it’s more like seeing the wisdom of a master teacher, assembled from a wide variety of sources and passed on.

In fact, this is a rant that I should probably invoke elsewhere, but to me received knowledge is the essence of any craft. Composers don’t invent new rules of harmony (well, at least, not tonal harmony). Engineers don’t work out the laws of physics from scratch each time they build something.

And most importantly, because all of this exists in code, you can read and modify anything you find. It’s a black box if you want it to be, but I very often dig directly into the code to understand how things work.

There’s a bunch of documentation and a showcase for great work:
http://toxiclibs.org/

GLGraphics

All of this, though, brings me to GLGraphics. As I’ve been saying for – gee, years now – GLGraphics is the future of Processing. Now, that’s come to pass: gifted developer Andres Colubri authored the new OPENGL2 renderer that is similarly based on native OpenGL calls, and now runs the Processing Android port and upcoming Processing 2.0.

GLGraphics is separate from OPENGL2, but it’s your best bet for work on the stable Processing 1.5 build. Version 0.95 adds compatibility for that latest release, and adds two essential other features:

• An example of how to integrate with toxiclibs
• GLSL shader support, which can in turn be used for complex mesh generation.

See yesterday’s blog post announcing the update:
Processing 1.5 / GLGraphics 0.95

And there should be still more coolness to talk about soon, at least for Mac users, with the availability of Syphon for Processing. Stay tuned on that.

I think I have to hide away in a hole and do nothing but code this weekend. Anyone want to hop on IRC or PiratePad and pass code snippets back and forth?

eCLOUD, conceived by Aaron Koblin, Nik Hafermaas, and Dan Goods, displays weather data from the National Oceanic and Atmospheric Administration (NOAA) via specialized liquid crystal displays, suspended from the ceilings of the San Jose International Airport.

The eCLOUD is a dynamic sculpture inspired by the volume and behavior of an idealized cloud. Made from unique polycarbonate tiles that can fade between transparent and opaque states, its patterns are transformed periodically by real time weather from around the world.

When it's raining, the eCLOUD appears to be dropping, while in a city that has hazy skies, the display feel slow and lazy. Stormy weather shows a busy cloud, bouncing around wildly.

Below is some footage of the piece in action.

There is also a display that accompanies the piece, which shows you the city that is currently up and its weather conditions. A smaller animation plays what's going on above.

How is this done? Again, as the site explains, it comes down to the liquid crystal tech, which some manufacturers have used for windows and dressing rooms. Opaque in its resting state, but then clear when you add some electricity, demonstrated below.

Data feeds into the main machine, and then delivers directions via Processing (the software) and a hundred circuit boards (the hardware).

Very cool stuff. If anything, it sure beats the nappy carpet in Las Vegas' airport.

[eCLOUD | Thanks, Dan]

• What’s the Weather Like In Your City?
• European Economic Weather Map – Sudden Change in Outlook
• Grabbing Weather Underground Data with BeautifulSoup

The awesomely talented UCLA duo of Casey Reas (1/2 of Processing and come on, he has a 4 letter .com url. This guy is on the ball!), Chandler McWilliams (of The Barbarian Group and general art/philosophy brilliance), and the design/interactive shop LUST have teamed up to create a beautiful and expansive compendium of computational aesthetics. The book, entitled ‘Form + Code’, published by the Princeton Architectural Press, features an impressive collection of work. It is a must have for anyone finding themselves at the crossroads of art and technology. Not only is the book beautifully designed, but it is equally beautifully curated.

The book begins with a brief history of code and then showcases specific examples through the categories of repetition, transformation, parameterization, visualization and stimulation. You will likely see many examples that you recognize and hopefully, just as many which are new to you. I was happy to see ‘The Visual Thesaurus’ because it was the first website which made me see the artistic potential of code despite it being more of a reference site than an art site.

The Form + Code website is a good place to start. You can see some spreads from the book, list of contributors, and a bunch of code examples (written in Processing)… pretty much everything you need to get inspired. Also, check out the more thorough review by Rhizome.org.

The book is available from Amazon.com.

Pick up a pen and draw a sketch. There, that was easy – however crude, you can get out an idea. Sketching with paper is still the fastest way for most of us to imagine something. But between that immediacy and the end result, you need prototypes.

The Processing language has long been one of the easiest ways to sketch an idea in code – best after you’ve first put pen to paper, but as an immediate next step (and for ideas you just can’t draw). Built in Java, the creation of Ben Fry and Casey Reas and a broad community of free software makers, it runs on Mac, Windows, and Linux. Via Processing.js, the same API works in a browser via JavaScript. It has inspired the OpenFrameworks project, which uses nearly the same API. Those tools let you intermingle Java, JavaScript, and C++, as they’re written natively in those languages.

Processing now runs just as easily on a mobile platform with Android. You can try it with the free SDK and emulator, but it’s most fun with a device. Using all free software, you can sketch easily on mobile and desktop from one environment, and with only minor modifications, run the same code on a desktop, a browser, and a mobile device.

Translation: with one, elegant API, you can “sketch” visual ideas on screens from an Android phone to a browser to a projection or installation. As a prototyping tool, or for finished projects, that makes it an exceptional, expressive tool for making interactive visuals for screens.

This is a first-draft tutorial, as I make the same presentation in Stockholm at the info-rich Android Only conference. I’m eager to put it out there and find out that people have problems, as I can then improve the documentation. So do give it a try – especially if you’ve got (or can borrow) an Android phone.

Processing on Android performs all of the core functions of Processing on desktop – 2D and 3D visuals, data manipulation, images, and type – and you can mix in Android code using the standard Android APIs, right in the same project. Processing controls the screen on which it’s drawing, for now, but you can mix and match everything else, to support multi-touch, sensors, and even NDK code. (I’m using it with Pd for Android, with Pd doing sound and Processing doing visuals.)

I’m assuming basic familiarity with Processing, so if you haven’t tried it out yet, check out the excellent tutorials available online to get rolling.

Install Processing and the Android SDK

Definitely read the latest official instructions:
http://wiki.processing.org/w/Android

1. Download the latest pre-release version of Processing. Eventually, the standard Processing download will support Android automatically out of the box – very cool. The current stable version doesn’t yet have Android support, however, so for now, you’ll need the latest pre-release build, which includes all the code for the testing release of Processing for desktop and Android. I tested with 0190.

2. Download the Android SDK. For now, you do need to download the SDK for Android separately, which is an extra step – but at least it’s completely free, and runs on any OS. See:
developer.android.com (This step will eventually go away.)

You can put the SDK anywhere you want; just make a note of where you’ve installed it, as you’ll need to point Processing to that location later. Follow the instructions carefully on Google’s site. On Windows, you’ll need to download the USB driver. On Linux, you’ll want to read the complete Developing on a Device instructions, as there are a few short commands you have to enter at the command line because of the way Linux talks to USB devices.

Lastly, even though that SDK is a big download, you’re not done until you download additional components from Google. This allows a single recent version of the SDK – like the 2.2 SDK I tested – to support a variety of older versions for backwards compatibility.

I have Eclipse installed, because it’s a handy tool for developing Processing for desktop and Processing for Android on any OS. Once you’ve installed the Eclipse plug-in for the Android SDK, you can add components from inside Eclipse. Inside that IDE, choose Window > Android SDK and AVD Manager Manager and you’ll get a graphical interface for adding these components (pictured).

You can now also access the same interface from within Processing. Choose Android > Android SDK and AVD Manager.

For Processing 0190, you need the SDK Platform 2.1, API 7 and Google APIs by Google, Android API 7. (If you’re an Android developer, you’ll probably also select some other components here, as well.) Components are available under the Available Packages.

If for some reason you prefer to use the command line, head to the Windows, Mac, or Linux command line, and follow the commands in the Quick Start under the Android SDK download instructions.

3. Run Processing, and switch to Android Mode. Load the Processing IDE you just installed, and switch to Android Mode.

The first time you do this, you’ll need to direct Processing to your Android SDK. Choose the root directory of the SDK – the directory that contains “add-ons,” “docs,” “platforms,” “tools,” and the like.

4. Try running a sketch on the emulator. I’m compiling a set of simple sketches you can use with Android, but even many of the basic examples will work out of the box. For fun, you can try sketches like those found in the Examples > Basics folder and some of the other included Processing Examples. They don’t take into account varying screen resolutions, which is a cool feature of Processing for Android, but it’s still fun to watch them run.

It’s simple to run a sketch. Open the sketch you want, go to Android > Android Mode to enable Android Mode for the sketch, and then hit Run. Instead of opening in a window as desktop Processing sketches do, you’ll see an Android emulator appear.

Be prepared for the emulator to take … some … time. You’ll spend a long time looking at the load screen. When it does finally load and install, there may still be a delay as you stare at the main screen. You may have to hit run again. You may see error messages saying an application has become unresponsive – don’t worry about that; Android has a low threshold for applications timing out, so while the emulator is loading, just choose “wait.” And because the emulator defaults to a large screen size, it’ll also take up some room on-screen.

It’s just slow, painful, slow, and unreliable. Fortunately, running on the device is near-instantaneous and speedy, so beg or borrow access to one and try the next step.

5. Try running a sketch on a device (if you’ve got one). Running on the emulator is a pain, but running on a device is very quick – even quicker than you might imagine if you’ve come from a background on iOS. (Remember, you’re working with Java, not native code.)

In a clever design feature, to run on the device instead of the emulator, simply choose Present instead of Run. (The quickest way is simply to shift-click the play button.)

So long as you have a device connected with USB, and USB debugging is switched on, and you’ve done the setup correctly above, you’ll see your sketch appear on the device.

Now, that’s more like it.

“But, wait,” you say. “Android devices have all different screen sizes. Augh, fragmentation, fragmentation!”

Fret not: computers have different screen sizes. Projectors have different screen sizes. And Processing on Android can easily adapt to the size of the screen. We won’t need much Android-specific code, but let’s get started with our own Processing sketch and see how it works when you’re preparing for mobile.

Try Writing Some Code

Okay, so what does a Processing sketch for Android look like? Here’s a really simple example, just to show off basic drawing, managing different screen sizes, and simple (single) touch. (I’ll leave density for another day; multitouch and other touch events are also possible.)

float sw, sh, touchX, touchY;

void setup() {
  size(screenWidth, screenHeight, A2D);
  println(screenWidth);
  println(screenHeight);
  sw = screenWidth;
  sh = screenHeight;
}

void draw() {
  background(0);
  smooth();
  fill(255);
  noStroke();
  ellipse(sw/2, sh/2, sw/4, sw/4);
  noFill();
  stroke(255,0,0);
  strokeWeight(2);
  ellipse(touchX, touchY, sw/4, sw/4);
}

void mouseDragged() {
  touchX = mouseX;
  touchY = mouseY;
}

Two things to note, in particular:

size(screenWidth, screenHeight, A2D);

(Warning: don’t put variables in that size() command.)

Processing for Android will map the desktop renderer names to either A2D (for 2D graphics) or A3D (for 3D). (I’m using the Android names here, but the other names will work just fine.)

Also, since static pixel sizes don’t make sense on mobile platforms with different screens, you can substitute “screenWidth” or “screenHeight” and get dimensions from the device on which you’re running. (I just used “sw” and “sh” because they’e shorter, and then I can make everything else relative.)

void mouseDragged() {
  touchX = mouseX;
  touchY = mouseY;
}

Here’s the really cool part — touch input works exactly the same way as it does on Processing on a desktop, for relevant mouse commands (nothing involving these strange “buttons” of which you speak). mousePressed, mouseDragged, and mouseX and mouseY all work as expected, so long as you’re satisfied with simple, single touch and don’t need gestures, multitouch, and the like. Don’t be confused by my touchX and touchY variables — those are just to demonstrate that my circle only moves when it’s “dragged” — or your finger slides across the screen. What we have is the beginnings of a drawing app, which I’ll be expanding over coming tutorials.

Here’s the other cool thing: modifying only the references to screenWidth and screenHeight, and the renderer, I can run exactly the same code in Processing on the desktop, or even in Processing.js on a browser.

Tune in Next Time…

This is not a replacement for the official wiki documentation, so do read that, please.

In coming installments, I’ll share:
1. How to get Processing for Android working inside Eclipse. In short – it works. That makes adding Processing to an existing Android app easy. (Hoping to follow that up with gedit and Ant)
2. More advanced tricks for dealing with screen dimensions and touch.
3. Android-native snippets for connecting sensors.
4. How you can combine sound and visuals with Pd and Processing, both running on Pd.

Other things that are possible to do with Processing for Android (which I’ll cover):

• Use install custom or installed fonts
• Data visualization (just as on the desktop version – making this much easier on mobile than in the past)
• Touch events, relative motion, and key events
• Force a specific orientation
• Add security requirements to the manifest (for things like sensors), right from the PDE app – no need to edit a file or go to Eclipse

I also hope to take some notes on what happens here at Android Only.

Processing for Android is still in development, and this is pre-release software. So be prepared to encounter issues. That said, I want to refine this documentation, so please let me know if you encounter issues with my instructions.

Presentation slides

Here’s my presentation from Stockholm:

Share what you’re doing…

Noisepages registration is open again as we finish the site. Share what you’re doing, devices you’re testing, or other thoughts:
http://noisepages.com/groups/processing/forum/

I built this demo by porting one of my old Processing sketches to the fantastic Processing.js. The demo draws particle trails using Perlin noise to direct the particle motion. Move the mouse to change the noise function and click to randomize the particle drawing parameters.

Processing.js is a JavaScript port of Processing that renders to the HTML5 Canvas tag. It was originally built by the same guy who built jQuery. It’s nice to have a framework for handling common drawing and math functions. As a bonus, if you are familiar with Processing you will be able to jump right in.

Yay! Snakes! Wrote up a post about the process I used to create these lovely reptiles with code. Read it over at
the Barbarian Group blog.

Here it is in action…

Snake Study from flight404 on Vimeo.

No, not that ARToolKit stuff. You wont be manhandling fiducial markers with this one. Read all about it on the Barbarian Group blog.

Here is a sample video.

Augmented webcam study from flight404 on Vimeo.

Eye Magazine’s Winter 2008 edition has finally arrived in my grubby hands. Took me a while to track down a copy (I guess the $30 price tag and the British origin keep it rare in these parts) but I finally found a copy of issue #70 at Fog City News (also the best place to buy designer chocolate in these parts).

I was asked to provide some high res images from the (unofficial) Goldfrapp video I made last year. Since that video was rendered out at 1280×720, I didn’t have anything larger to offer. I ended up running the code at a small resolution (500×500) and implemented Marius Watz TileSaver class. I coded in a few time-code triggers and went to get coffee. While I was gone, the program ran at a fast clip (nearly 30fps) and once it reached a time trigger, It kicked out a high-res still (6000×6000+). Looks great printed. I passed a few dozen of these images along to Eye and let them decide their fate.

I was pleased to see they decided to choose my work to be shown on the cover, framed by a ’silent character’ from the font ‘Replica’ which is also profiled within the pages of this issue. Read more about the collaboration on the Eye Magazine blog.

I am even happier with the full spread image they used as a lead-in to an article about digital craft. Like I said, it looks great printed!