Mozilla tech evangelist Paul Rouget has a go at Internet Explorer 9 in a series of simple graphs, comparing it to Firefox 4. I think Rouget doesn't like IE9. Not sure though.
The obvious progression of this series is to compare Firefox 4 to Chrome and Safari. I await the results.
“The hills are alive /
with the sound of browsers”
Ever thought you’d make sounds in a browser, or have new ways of visualizing music playback? It’s happening, with builds of Firefox anyone can download.
Work to make browsers rich with sound synthesis and visualization continues. “Compatibility” isn’t really an advantage yet, because Firefox is the only browser with support, and only in the next version, though that could change in the future. And yes, Flash is capable of some of this, too (though not real 3D), with 90-95% saturation, conservatively, of computers. But if not compatibility, what these experiments do represent is what happens when someone working on a tool (Firefox, in this case) really commits to making sound a priority, and supporting free standards and developer tools (an emerging standard API, WebGL, Processing.js, etc.).
In fact, it’d be great if this occurred everywhere: if you’re making a platform, make sound a priority, and people will do mind-blowing stuff with your platform.
Among the latest fruits:
1. 3D eye candy. Charles Cliffe has a psychedelic visualization of sound playback. The JavaScript nuts are also proceeding to do more things with their language than most would deem possible, even moving DSP calculations to JavaScript code. I remain a bit skeptical there: the question to me isn’t whether JavaScript is “fast enough,” but whether native code is faster or simply the better tool for some jobs. Details below.
2. Patching in a browser – with a Pd clone. Chris McCormick is porting a subset of basic Pd objects to the browser. Now, one side of me wonders whether Pd is the best choice; it’s a somewhat idiosyncratic, if powerful, language for describing sound patching. But on the other hand, I could see this being fantastic in teaching and sharing: put basic patches up in a browser, let people play with them live, then build more advanced tools (with greater hardware access and external support than is possible in a browse) in the traditional Pd tool. As I keep saying, I think there’s far too much partisanship in the discussion (“Browsers for everything!” / “Browsers are useless!”), far too little thinking about how the browser and the desktop tool are more powerful together.
Web Audio Data API – Pure Data and Processing.js from David Humphrey on Vimeo.
Check out:
mccormick.cx/dev/webpd/
wiki.mozilla.org/Audio_Data_API
Also — heck, I may try this out in workshops as soon as next week. The browser could build a basic language for music and visuals in Processing and Pd, then robust performance tools could be built in the native tools, with quite a lot of compatibility between the two.
3. Actual standards. The W3C, the standards body behind HTML, has added this discussion to an Audio Incubator group. (It’s been incubating for some time, but maybe this will help something actually hatch.) Now I’d just like to see these things in Chrome/Chromium, too – I wonder if anyone’s up to a test build, as the standards adoption discussion continues. A number of readers have pointed out that MPEG4 had a specification that included, wholesale evidently, Csound. But this process seems more organic to me – you need actual tools and real-world experiments to evaluate the validity of something, not just standards on paper.
A side rant, though: why do Web geeks only care about what happens in the browser? It’s funny to me it seems that outlets like Slashdot jump on stories like browser-based tools, but ignore exactly the same ideas if they’re in a separate app. That’s not a criticism of the Mozilla crew or these brilliant hackers – this is what development is all about, pushing your tools to the limits. But if there isn’t a broader recognition of the value of what you’re doing or why you’re doing it in the first place, there’s a danger that unsustainable tool fetish will miss the point. That is, synthesis in the browser is excellent, but if people don’t understand the value of the synthesis itself, we have a lot more work to do.
Even the tools themselves need a context. It also JavaScript is amazing, but so are tools in Python, Java, Scala, and so on… and some of the enduring power of C still shows here. Browser powers are cool, but the OS is just as important – performance of Firefox would be heavily dependent on support for OS-native, low-latency audio outputs, like JACK on Linux. (Yes, it’s open source, so you can go do it yourself. No, I have no idea how to build Firefox for JACK – maybe a reader does?)
I’ve still yet to see a compelling explanation of what the browser really is, and what’s possible with its interface paradigm. That should be a fascinating discussion, actually, especially with the radical transformation of the browser, particularly as players like Google make it the central aspect of TV-watching or tablet experiences. But the discussion is only really interesting if you don’t start out with the value as a given. For instance, if browsers become a bigger part of what we do, is its simplistic tab metaphor really sufficient? If browsers simply bundle a set of native tools, are there ways “standalone” apps might adopt similar, standards-based approaches?
David Humphrey argues that part of the value here is the view source concept, but the Web has had the same empowering influence on sharing, collaboration, and reuse with platforms other than just JavaScript. The browser itself is a largely misunderstood piece of technology, partly because users (understandably) focus on their experience, and doesn’t pay attention to which aspects are delivered by the browser, the OS, or some other piece of code.
Oh, side note: this isn’t about “the cloud.” The cool stuff here is happening on your local hardware, period. That’s what makes it fast, and that’s what makes it work for audio, and your local machine is getting cheaper, cooler, and less power-hungry all the time. New DSP and floating-point capabilities in devices like tablets could make sound more powerful and flexible than ever before – provided people work out how to maximize, not squander, those capabilities.
So, here’s what I’d like to ask: what form will the standards discussion take? And how can these larger discussions – many of which transcend the discussion of any one tool or standard – find a forum?
While you ponder that (and I’m open to suggestions), here’s more reading for you:
Experiments with audio, part X [Dave Humphrey's increasingly-awesome blog]
Previously:
Real Sound Synthesis, Now in the Browser; Possible New Standard?
More details on the first example, and how it was built (Minefield is Firefox 3.7):
All runs in real-time with Javascript, WebGL and HTML5 only (uses Minefield Audio build) — no browser plugins are used.
This demo combines the CubicVR 3D engine on WebGL (www.cubicvr.org) with the Mozilla HTML5 Audio API (hacks.mozilla.org), Processing.js (www.processingjs.org) and BeatDetektor.js (www.beatdetektor.com)
Mozilla Audio API is used to sample the HTML5 audio tag on the page, this information is processed by BeatDetektor.js which produces timing information for the Processing.js real-time canvas textures and the CubicVR.js procedurally generated WebGL scene using them.
The camera is set to free roam a simple chase pattern with a probability to follow a nearby cube (fully automated).
Available online at:
http://cubicvr.org/CubicVR.js/bd3/BeatDetektor3HD.html
or if you have a Float32Array enabled Minefield build:
http://cubicvr.org/CubicVR.js/bd3/Bea…
you can find more info about audio api-enabled Minefield builds at:
https://wiki.mozilla.org/Audio_Data_API
You can also feel free to chat with us about the Audio API via the #audio channel on irc.mozilla.org
Enjoy! And yes, I’ll have to work out a more beginner-friendly, here’s how to do this post.
WebGL is pretty exciting offering browser based OpenGL and hardware rendered graphics. When this becomes mainstream this will change up gaming and interactive on the web immensely. Unity 3D and Flash 3d engines add lots of immersive environments and WebGL will be just as exciting, if all browsers adopt it (canvas/webgl).
At Google, we’re deeply committed to implementing and advancing standards, so as of today, the O3D project is changing direction, evolving from its current plug-in implementation into a JavaScript library that runs on top of WebGL. Users and developers will still be able to download the O3D plug-in and source code for at least one year, but other than a maintenance release, we plan to stop developing O3D as a plug-in and focus on improving WebGL and O3D as a JavaScript library.
About WebGL
WebGL is a cross-platform, royalty-free web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the Canvas element as interfaces. Developers familiar with OpenGL ES 2.0 will recognize WebGL as a Shader-based API using GLSL, with constructs that are semantically similar to those of the underlying OpenGL ES 2.0 API. It stays very close to the OpenGL ES 2.0 specification, with some concessions made for what developers expect out of memory-managed languages such as JavaScript.
WebGL brings plugin-free 3D to the web, implemented right into the browser. Major browser vendors Apple (Safari), Google (Chrome), Mozilla (Firefox), and Opera (Opera) are members of the WebGL Working Group. “It feels like, someone’s missin-ing”
Wish granted!
Think 3D in the browser will never catch on? Think again. The folks at Google Labs have built an incredible-looking 3D API called O3D. It does just about everything you want, and then some:
There are already some complaints about “another standard,” but to me, putting together a whole package here and employing other, lower-level standards (JavaScript, COLLADA, OpenGL, DirectX) makes a lot of sense.
http://code.google.com/apis/o3d/
I expect the folks working on Java and JavaFX are busy thinking about the fact that Sun just got bought by Oracle – something I’m hopeful, at least, ensures Sun’s future and is ultimately a good thing. But I hope someone on those teams is starting to get the message: 3D isn’t just something that’d be “nice to have.” It’s essential. And while even most developers likely don’t have a clue about things like custom shaders, having access to customize the graphics pipeline is likewise something ultimately benefits all developers – even if they just wind up relying on someone else’s code. I really do hope this is a priority with the coming development of Java and JavaFX, which could have the power to do these sorts of things. (Heck, Java could even benefit from the code Google just posted.)
On the proprietary side, this to me is a big blow to Microsoft’s WPF and Silverlight and Adobe’s Director. Unlike those products, O3D looks simple, powerful, flexible, open source, and directly programmable with JavaScript.
That’s not to say there aren’t some questions here – and the Java/JavaFX comparison is especially relevant:
(I’ll try to get answers to those questions.)
Oh yeah, and then there are details like the necessity to write your own custom shaders just to add more than one light to a scene – I think this will initially appeal only to folks with some real 3D experience.
But am I excited? Ohhhhh, yes, indeed. O3D itself looks fantastic, and I think this is a sign that 3D times ahead are going to be really fantastic.
And as long as you have the plug-in working and a browser in full-screen mode, you could literally set up an O3D project as a performance / installation tool. O3D visualists? Absolutely. Enjoy.
