something in the way

“I don’t have to know anything about technology because I know that whatever I think up there’s always someone who can build it.”

I’ve heard this statement, or a variation thereof, proffered by copywriters, art directors and creative directors. And it’s true. Chances are that no matter what you conceive – experience, installation, interactive film, video game, touch-screen, augmented reality – there is a team of developers somewhere who can bring it to life.

There’s just one problem. Without some knowledge of technology, social media, API’s, HTML5, LBS, etc. you probably won’t think up the very coolest of ideas.

The last time someone shared this sentiment with me I agreed, as I always do, but then asked a series of simple questions. Holding up as an example Google’s Arcade Fire video, Wilderness Downtown, I asked if he could have thought that up. I showed him Breakfast’s Instaprint and asked the same question. Ditto as I reminded him of Wieden and Kennedy’s Old Spice Twitter campaign, Garmin’s Garmin Connect and Mr. Youth’s spinoff action platform Crowdtap.

I don’t have to tell you what the answers were.

But if we believe that storytelling has changed, that agencies need to build things and create utility, yet that it still takes creativity to distinguish the best ideas from the also rans, then all writers, art directors and creative directors need at least a cursory knowledge of today’s digital technology and all that it enables.

Shit, they might even have to learn something about data, at least the personal kind that inspires the likes of Garmin Connect.

You don’t need to take a lesson in writing code. But you may want to make friends with the nerds and learn a little bit about what they can do before you bring them your next ad like object and ask them to make something digital out of it.

 

Cocos2D has been ported to run in javascript and called cocos2d-javascript by Ryan Williams.

Cocos2D is a game engine similar to Flash in that it is a 2d engine but it natively renders to OpenGL ES.  Cocos2D-iphone was originally a port of Cocos2D, a python game engine.  But the similarities to Flash and DisplayObjects = Nodes, Sprites, Scenes, Layers etc. This helps to port games over fairly quickly or start in productive in cocos2D.

Having this in javascript is a great thing! It runs on html5/canvas/javascript.

What is cocos2d-javascript?

Cocos2d-javascript is a 2D game/graphics engine based on cocos2d-iphone but designed to run in the web browser. It uses the latest features available in HTML 5 allowing real-time rendering of 2D graphics without the need for plug-ins such as Adobe Flash.

While HTML 5 is still new and not fully supported across all browsers it won’t be long before the vast majority of web users are able to enjoy all that it offers. When this time comes cocos2d-javascript will be an excellent way to develop games and applications.

To see a small sample of what is on offer, please check out the demo section.

cocos2d engines are now available on almost all platforms, so if you are building a 2d game and need a 2d engine typically with Box2D physics, cocos2d offers lots of ways to get the game out there with some porting work.

• cocos2d-javascript (web)
  • cocos2D-javascript site
  • cocos2d-javascript at github
  • cocos2d-javascript docs
  • demos
    • Sprites
    • Tile Maps
    • Jump ‘n’ Run
    • Mario demo
    • Breakout demo
• cocos2d-iphone (mobile)
  • cocos2d-iphone site (objective-c)
  • cocos2d-android-1 (java)
  • cocos2d-android (java)
• cocos2d – original python (desktop)
  • cocos2d site
  • cocos2d python download
• cocos2d other
  • cocos2d-x c++ (C++ port for win, iphone, android-ndk)
  • Similar to cocos2d – andengine (android java)

Firefox 4 in beta to be released soon, also joins the WebGL ranks with Chrome 9.  Safari has it in nightlies and IE hasn’t even mentioned it.

There really is too much to list as this release is feature packed! Of course the most exciting being WebGL and hardware acceleration from our perspective.

Firefox 4 now has WebGL enabled by default. Based on the original 3-D Canvas work by Vladimir Vukićević, this is being

widely implemented in browsers. The WebGL spec is on the short path to a 1.0 release and we’re very excited to see this be used in the wild.

Hardware acceleration has finally arrived even though it should have been in nearly all platforms for web last decade, but we’ll take it.

Firefox 4 supports full hardware acceleration on Windows 7 and Windows Vista via a combination of D2D, DX9 and DX10. This allows us to accelerate everything from Canvas drawing to video rendering. Windows XP users will also enjoy hardware acceleration for many operations because we’re using our new Layers infrastructure along with DX9. And, of course, OSX users have excellent OpenGL support, so we’ve got that covered as well.

The javascript engine JaegerMonkey is comparably fast to SunSpider and V-8 javascript benchmarks and has support for EC5 javascript.

And you might have noticed that it’s really fast. This is the world’s first third-generation JavaScript engine, using Baseline JIT technology similar to engines found in other browsers and kicked up a level with the Tracing engine found in Firefox 3.6. As such, we’re competitive on benchmarks such as Sunspider and V8, but we’re also fast at a whole mess of things that we expect to see in the set of next-generation web applications, hence Kraken.

WebConsole looks like they are joining Chrome and Safari with built in inspection tools similar to Firebug, however Firebug still available.

Firefox 4 will include the Web Console. This is a new tool that will let you inspect a web page as it’s running, see network activity, see messages logged with console.log, see warnings for a page’s CSS, and a number of other things.

Note this that is something that we’ll be including with Firefox 4 directly. It’s not an add-on.

(Also Firebug will be ready for the final Firefox 4 release.)

Firefox 4 has other improvements like layering (in-memory retained layers), caching/scheduling improvements and lots of other performance enhancements.

2011 is looking like the year all this is coming together, at least for Chrome, Firefox, possibly Safari (need WebGL in main release) and IE is still the biggest problem to getting WebGL. At this point WebGL looks like it is still over a year out as it may not come to IE until IE10 or possibly never, the WebGL 1.0 spec is on the fast track though (don’t we all love Khronos? They have been amazing with OpenGL since they took over).  html5 is looking like it will be close to mainstream by the end of this year depending on the install rate of IE9 when released.

The world is waiting to see if Microsoft implements WebGL or tries the old DirectX/D2D only ways.  Nevertheless, getting a push for hardware acceleration and fast renders in 2d/3d is a very sweet direction.

Good news for the beginning of hardware accelerating the web, WebGL will now be part of the main Chrome releases not just a compile option for Chromium nightlies.

Google Chrome 9 enables WebGL support by default. “WebGL is a new web technology that brings hardware-accelerated 3D graphics to the browser without installing additional software” and it can be used to create cool applications like Google Body Browser, Field, Aquarium and more.

The update for Chrome 9 also sandboxes Flash, WebGL and plugins like extensions and tabs so that using them will be more secure and not crash the browser or the tab. Hopefully Safari has this soon, and then a few years from now IE may get it. Or they will put out their own DirectX web plugin so everyone has to write it twice like currently in game development. /sarcasm

Pretty promising project over at PhobosLab of the Impact Game Engine running on the iOS platform using the JavaScriptCore Framework. The best part is it has all Canvas calls passed into run on OpenGL ES thus the speedy demo.

Biolab Disaster on the iPhone 3GS from Dominic Szablewski on Vimeo.

The game is running in its own process and is not using the iPhone’s browser at all. Instead, it’s just using the JavaScriptCore Framework to run the game. All the necessary calls to the Canvas API have been reimplemented with OpenGL-ES and the touch input is passed over to JavaScript to be evaluated by the engine. I of course had to make some changes to the engine, but the game source code is exactly the same as for the web version.

This would probably never fly on the App Store because it executes code or interprets it but is a very nice experiment. More discussion at Hacker News..

In his Web vs. App vs. Hybrid presentation at Design for Mobile in Chicago IL, Jason Grigsby showed how the question of native vs. mobile Web experiences is really a conversation about how to do both well for your product. Here's my notes from his talk:

• At the WWDC in 2007, Steve Jobs touted a sweet solution to use build apps for the iPhone using Safari and Web apps. Web advocates were thrilled. Native Mac developers were disappointed.
• In October 2007, Apple announced the App Store. The extent of its success was a big surprise. It went against the trend of Web-based apps, Cloud computing, software as services, etc. iPhone apps were the opposite: few APIs between apps, local storage.
• But from a mobile paradigm the app store was much better. It was much more open and potentially profitable than shipping apps on mobile carrier’s terms. From this perspective it made a lot of sense that is was successful.
• Native Mobile Apps vs. Mobile Web apps. They are both apps. Mobile is bigger than just apps. Native Apps will always have better access to platform, resources, etc. Because they are not within a Web browser. But this doesn’t mean that native apps are always the way to go.

Mobile Web

• Small businesses –it might never make sense for them to have native applications. But it does make sense for them to have a mobile Web presence.
• Even companies that make their money solely on building native apps need to have a good mobile Web presence. So people can learn about and share URLs to get information about their products.
• URLS don’t open applications. Use case: people email urls to each other –people should able to go these urls while on the go.
• The mobile Web does not censor content and allows you to publish when you want.
• 85% of the smartphones that ship use WebKit as their core engine. While not all are created equal, there’s a lot more consistency then differences in WebKit builds.
• Find a single report that doesn’t show exponential growth for the mobile Web. All data shows extreme growth in mobile Web.
• Mobile Web is more open, no entry costs, you get to keep 100% of revenue, you can do instantaneous releases, but there is no micropayments.
• Biggest mobile Web experience challenges = no position: fixed, no accelerated scrolling. You want position fixed relative to the view port not to the viewable area.

Native Apps

• Do native apps really create platform lock-in? Most people use less then 6 apps per day –that’s about $20. Your switching costs between platforms are not that high.
• Android is outselling the iPhone but has an inferior app store. So is the app platform creating lock-in? Doesn’t seem to be.
• In the video game market, the leader changes as platforms change. The mobile market may be more like the video game market than the PC market.
• In a model where people stop using your apps after a few times, where is the recurring the revenue for app developers? Can you really build a sustainable business on someone else’s app store?
• Some apps will be rejected from App stores (based on their terms) –so how much should you bank your businesses future on a platform you can’t control.
• In order to have a viable business opportunity, you need to move beyond a single platform’s app store. When you do, there’s a lot of fragmentation you hit between all the development requirements. It makes the cost of development high.
• Six main reasons for apps: performance, offline mode, findability, device capabilities, monetization.
• Performance: All Javascript rendering engines are getting faster and are good enough for most Web applications. The size of Web pages has tripled; the number of objects on Web pages has doubled. We need to do better on how we serve mobile Web pages to devices.
• Offline mode: HTML5 can do offline mode. This isn’t really a concern.
• Findability: there are issues with being found in the app store just liek there are on the Web. Need a marketing plan in both cases. People will begin to search for tasks they need answers on apps stores and on the Web.
• Monetization: this is a gap on the mobile Web and perhaps the biggest unresolved one.

Why Choose?

• So what should you do in the meantime to avoid the gaps between native and Web apps?
• We should hedge our bets and go for the long term = mobile web.
• Start with content, detect devices, use reference designs, then add functionality using PhoneGap and Titanium –to develop cross platform apps with Web technologies.
• Native vs. Web is not an either or question. The answer is both! Websites should work no matter what device you are on.

Tags: mobile, technology, Web applications, design4mobile

A sweet engine for getting started with Android game development is the andengine 2D OpenGL ES engine. This is very simple and compares with cocos2d-iphone for iOS development in 2D with OpenGL ES.  They both support a wide range of 2d techniques with an OpenGL renderer.  Some great videos are posted on the andengine google code page showing a box2D example, multiplayer example and more.

Mobile games are on slower hardware, similar to later 90′s computers so native is a great way to go for 3d and 2d game development because of this limitation at the current time and well into the next few years.  Take this time to learn you some native gamedev. andengine isn’t native directly as it is Java based but compiled with the Dalvik JIT virtual machine. Another way to go native on Android is the Android NDK which allows C and C++.

• andengine download at google code
• andengine examples at google code
• andengine blog
• Mercurial repository at google code
• cocos2d-android port of cocos2d-iphone with BSD license
• rokon android is another BSD licensed android game engine

The engine also has extensions that can be easily added and some great ones exist already.

Google has decided to put weight behind WebGL and stop actively developing O3D as a plugin, rather they will make O3D a Javascript library on top of WebGL. This will focus the 3D plugin development efforts from Google into just WebGL on top of the OpenGL ES 2 spec, which in turn runs in the html5 <canvas> tag.

WebGL is pretty exciting offering browser based OpenGL and hardware rendered graphics. When this becomes mainstream this will change up gaming and interactive on the web immensely. Unity 3D and Flash 3d engines add lots of immersive environments and WebGL will be just as exciting, if all browsers adopt it (canvas/webgl).

At Google, we’re deeply committed to implementing and advancing standards, so as of today, the O3D project is changing direction, evolving from its current plug-in implementation into a JavaScript library that runs on top of WebGL. Users and developers will still be able to download the O3D plug-in and source code for at least one year, but other than a maintenance release, we plan to stop developing O3D as a plug-in and focus on improving WebGL and O3D as a JavaScript library.

About WebGL

WebGL is a cross-platform, royalty-free web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the  Canvas element as  interfaces. Developers familiar with OpenGL ES 2.0 will recognize WebGL as a Shader-based API using GLSL, with constructs that are semantically similar to those of the underlying OpenGL ES 2.0 API. It stays very close to the OpenGL ES 2.0 specification, with some concessions made for what developers expect out of memory-managed languages such as JavaScript.

WebGL brings plugin-free 3D to the web, implemented right into the browser. Major browser vendors Apple (Safari), Google (Chrome), Mozilla (Firefox), and Opera (Opera) are members of the WebGL Working Group. “It feels like, someone’s missin-ing”

Andrew Bell, Robert Hodgin, and The Barbarian Group just released LibCinder, a creative coding framework in C++.

It’s a cross platform, open-source project, very similar to Processing or OpenFrameworks, but with better memory management and OpenGL support. Features include standalone applications and screensaver creation, Cocoa touch support (iPhone, iPad), OpenGL texture classes, webcam capture support and full Quicktime support. Besides the tech specs, what can it really do?

The most famous example is probably the Augmented Reality cover on Esquire but there’s lot of video goodness by Robert Hodgin (aka flight404) below.

For starters, the usual “Hello, Cinder” tutorial.

tip by Peter Kirn

Content under a Creative Commons License. (Digital Fingerprint: bfff8c3002d3e0f3f95495bddf32fef0)

Also of Interest

• Logitech, hi honey
• Best Of 2006: Music
• Tips to improve your portfolio on the web

Amazing, so many things have happened in the Flash Player engineering team over the past year. Lots I would love to talk about. But the purpose of this post is to deep dive into a subject Kevin Lynch touched upon recently, specifically Mac performance and his comment about Core Animation. Whenever performance is mentioned in the context of Flash it gathers a lot of the attention and some of the technical background is lost in the PR.

So what's the deal with Core Animation in Flash Player 10.1? Let's look at how Apple's documentation summarizes what Core Animation does:

Core Animation is an Objective-C framework that combines a high-performance compositing engine with a simple to use animation programming interface.

Sounds like perfect match for Flash does it not? So yes, Flash Player 10.1 is attempting to leverage this framework to work around a few specific technical issues we've had in Safari and all other browsers on OS X.

The drawing model jungle on OS X

Before going into more specifics of why we are going towards Core Animation lets get an overview about how plugins on OS X draw into the browser window. There 4 possible ways (compared to one on Windows):

1. QuickDraw. Default mode used by Opera, older Firefox and Safari versions.
2. Quartz 2D (a.k.a. Core Graphics). Supported by newer versions of Firefox and Safari.
3. OpenGL. No browser I know of supports this properly today.
4. Core Animation. Only available in Safari 4 + OS X 10.6 right now, with caveats in the current version.

In addition to these drawing models designers can embed Flash content in 3 different ways by specifying wmode:

1. Normal
2. Opaque
3. Transparent

Normal means that you can't have overlapping HTML sitting on top of your SWF, Opaque allows it and Transparent means that the SWF is transparent and underlying HTML content will show through. Taking all these variables into account we come up with these tables which shows when a particular drawing model is used (and subject for change before we release Flash Player 10.1):

Flash Player 10.0:

	Safari 4	Firefox 3	Opera 10
Normal	Quartz 2D	QuickDraw	QuickDraw
Opaque	Quartz 2D	QuickDraw	QuickDraw
Transparent	Quartz 2D	QuickDraw	QuickDraw

Flash Player 10.1:

	Safari 4 (*)	Firefox 3	Opera 10
Normal	Core Animation	Quartz 2D	QuickDraw
Opaque	Quartz 2D(**)	Quartz 2D	QuickDraw
Transparent	Quartz 2D(**)	Quartz 2D	QuickDraw

(*) Actually using nightly builds of WebKit because support for Core Animation is work in progress.
(**) Core Animation is used when the SWF is the front most object on the HTML page.

What are the issues with Quartz 2D?

The basic premise of Quartz 2D as Apple describes it:

Quartz 2D is an advanced, two-dimensional drawing engine available for iPhone application development and to all Mac OS X application environments outside of the kernel. Quartz 2D provides low-level, lightweight 2D rendering with unmatched output fidelity regardless of display or printing device.

Quartz 2D is not designed for multimedia applications, like animation or video playback. That's where OpenGL, Core Video, Core Animation shine. Safari's use of Quartz 2D to draw HTML content makes perfect sense as its content is static in most cases. Everything works well until Flash comes into the picture. For instance when the Flash Player plays a SWF using the Quartz 2D drawing model is has to do so with the full involvement of the browser. The sequence of events looks like this (you can follow the stack traces in Shark):

1. Whenever the Flash Player is ready to display a new frame, the Flash Player requests a refresh of its region using NPN_InvalidateRect.
2. The browser adds the the rectangle provided by the Flash Player to its dirty region.
3. The browser traverses its own display list (the HTML DOM) and paints every node which is part of the dirty region.
4. When the browser finds a node with a Flash Player instance it first draws the HTML background and then posts an event to the Flash Player to tell it that it has to paint over the requested region now.
5. The Flash Player then finally draws its frame.

So far so good, makes sense I hope. So what's the technical issue? Think of a fairly complex HTML page, for instance a page with a CSS gradient in the background. Add to add a SWF which runs at 30 frames/sec. You will see that a lot of time is spent in the browser, not in the Flash Player. This is where Core Animation kicks in: step 3 and 4 pretty much go away (as long as the SWF is the top most object).

Core Animation in the Flash Player

Flash Player 10.1 implements the Core Animation drawing model to fix this technical issue, among others. Instead of using a CGImageRef + CGContextDrawImage to get the bits to screen we pass a CAOpenGLLayer to Safari and use an OpenGL texture of type GL_TEXTURE_RECTANGLE_ARB to get our bits to the screen.

The support for the Core Animation drawing model was originally driven by Apple and we have worked feverishly to finish the engineering work on both sides. Yes that's right: This was and is a joint effort between Apple and Adobe engineers. Given the now almost perfect integration of Core Animation plugins into Safari I hope that future versions of the Flash Player will take advantage of more capabilities of OpenGL. And that without the requirement of setting any special wmode. I am pretty stoked about it.

As of today (2/10/2010) we are getting closer to having it stable enough for public consumption. That means though: You will need Flash Player 10.1, OS X 10.6 and updated version of Safari (or the nightly WebKit build), otherwise you will not see anything.

What difference does it really make?

This is by no means panacea for all performance issues in the Flash Player. Far from it. But it is a small step to a larger goal which is to improve the experience in the browser with the ever more complex web content out there. That said here is a comparison between Flash Player 10.0 and Flash 10.1 using this test case (this only works in Safari). Keep in mind that that is an extreme test case which has little to do with real world web content.

Flash Player 10.0 + nightly WebKit + OS X 10.6

Flash Player 10.1 + nightly WebKit + OS X 10.6

PS: You might have noticed that Core Animation is a Cocoa API. Yes, Flash Player 10.1 is a true Cocoa app now (with a Carbon fallback to support Firefox and Opera which are not Cocoa yet).