something in the way

Past couple of weeks I was working on FlashSURF project updates. More likely cosmetic updates: refactoring, adding methods, etc. I’ve screen-casted several videos showing up some usage possibilities.
Project SVN repo was divided in to 2 separate projects: one for FlashSURF SWC Lib development only and another for example projects. Now FlashSURF become really easy to use as far you don’t have to compile and process it with TDSI every time you testing your project. Provided SWC lib already processed, all you need is to include it in your project.

As I’ve already mentioned I’ve screen-casted some videos. All this features are presented in Examples project and can be easily reproduced.

This one demonstrates multiple image tracking/searching in input video stream. I’ve precalculated several images and store them as references for future tracking. Then we just loops through references to search each on the screen.

Here I’ve implemented selection tool and with the help of it I can easily select regions on the screen to store it as references. I was experimenting with playing cards and as you see it works pretty well.

You will find more examples in the repository with demonstration of panoramic stitching, finding Homography matrix between reference and tracked object on the screen, saving and loading references as local binary files etc.

I do realize there is a lot to add and improve in the project. I kindly ask everyone who is interested in using it to take a part by submitting features/methods requests and sharing improvements that can be done to existing methods. Just submit an issue in Code repo or simply drop me a line.

It’s a fact that mobile is one of the big trends for technology and lifestyle for 2010 – specially due to the rise of applications via iPhone and the subsequent follow up done by competitors to engage their audiences on their mobile devices. The Mobile Phone is becoming less a phone and more an application desk that connects the users with the world around them. This means that the whole choice process for the purchase of the new mobile phone has new key variant – application viability and potential. True, every year there has been some buzz regarding the mobile technology and how it will impact society sooner or later. So it is natural that the demand for professional mobile developers is rising – and ad agencies all over are up for the run.

With this in mind, we have come to the attention of the launch an interesting platform – TheyMakeApps – in which you can find a portfolio of the best App makers you can find close to you (or not!).

While being a simple idea, TheyMakeApps is quite ingenious. Most agencies have been up to their hairs in finding ways to answer to the client’s demands regarding mobile applications for their brands (and as part of our Big Bet for 2010, the demand is only sure to rise) so the agencies can have one of two solutions – either develop the know-how inside the company by hiring professional App Developers or by hiring App Developers per job. In any case, where do you find someone with such specific skills for something as important nowadays as mobile applications? TheyMakeApps solves the problem by providing a directory of App Developers by Geographic localization and even price range. You search for the App Maker of your liking, check his portfolio and contact him – all in the same website.

For now the big focus of TheyMakeApps is the IPhone – which is the granddaddy of Mobile Applications, thus has the biggest Programmer Base. But it’s looking for the opportunity to spread to other Platforms like the Blackberry, Android or Palm that, while a little more difficult to enter at this moment, are very relevant on the global mobile business. But one thing is for sure – this is another strong indicator on the strength of mobile applications today and how the App Making skill’s importance is growing. So if you are an App Maker or know someone who is, make sure you direct them to TheyMakeApps – I’m sure they’ll thank you.

Our global society is unsustainable. That’s not in dispute. What we need to be discussing is why. The dominant thought is that over-consumption is just what happens when a lot of people seek to be happy and prosperous. And so becoming sustainable is about sacrifice. I think this is all wrong. I believe that our over-consuming ways are the result of a specific design solution that once filled an important need and now is in need of a refresh. What we really have to sacrifice is the old way of thinking.

Here’s the very simplified story of how over-consumption was designed. I’ll begin in the year 1950. Americans had built growing demand for manufactured products in the immediate recovery after World War II, but once most of those needs had been met, demand threatened to level off. After all, the American people had made it through a depression and war by valuing thrift and durability. Why would they now begin buying far more than they perceived they needed? But the ramped-up war economy could produce astounding quantities of consumer goods. It needed markets for those goods soon or economists were predicting a massive depression.

At this moment, Madison Avenue shifted gears quickly and designed a solution to push the idea of the “consumer” as the primary identity of the American citizen. Instead of selling products, they began to sell stories. They stopped promising features, they started promising a refreshed sense of self. The Marlboro Man appeared and equated freedom with the kind of cigarette you smoked. Chevrolet hawked the American landscape. The VW Bug even began an ad campaign that offered to allow you to buy your way into the counterculture.

The strategy has been incredibly successful. It replaced centuries-old pride in American thrift with an entirely new ethic of citizenship through consumption. It was incredibly quick too, changing lifeways in a single generation, which by the way should give us hope that we can make opposite shift in a similarly brief amount of time.

Today two-year olds can name and even show preference for hundreds of brands. Some not too-uncool people walk around proudly proclaiming “I’m a Mac.” Even concern about our overtaxing of the planet can be expressed and eased by Toyota, BP and even Fiji Water!



In America, the center of consumption (we consume twice the resources of our European counterparts)we are what we buy. But we are not happy. Many studies say that the US is near the bottom of the heap when it comes to wringing happiness out of resources consumed. And that’s where the hope lies for a redesign. This old design solution is undeniably attractive but somehow the client, we the people, is still uneasy about it. In fact, although we haven’t put the pieces together to realize it, we actually hate it. We work too much. We’re deeply in debt. Our communities are fractured. We’re unhealthy. And shopping just isn’t doing it any more for us. For an excellent discussion of this, I recommend grabbing a copy of Annie Leonard’s new book The Story of Stuff (out in March and designed by Free Range). We are ready for a redesign.



The success of a few of our collaborations with forward thinkers on this topic convinces me the time is right. Millions of people watched, discussed, critiqued and studied the Internet movie version of The Story of Stuff which pulls back the curtain of the flawed consumption solution. A new site we helped launch, Shareable.net is aggregating the best of the new open source everything lifestyle in which people are learning to share resources as a way of saving the Earth, increasing their wealth and rediscovering joy. The site has huge momentum. I believe this is partly because the internet has changed the whole idea of sharing from something kindergartners are forced to do to a major form of social currency. On the web the more something is “shared” the more value it has. Talk about a paradigm shift away from the consumer mindset. And then there’s No Impact Man, Colin Beaven who decisively demonstrated that happiness can be inversely proportionally to how much we shop and spend.



As a communications designer, I am well aware that it is my discipline that helped to build and sell the deeply flawed model that threatens our very existence. And so we have the responsibility and opportunity to redesign it. By studying how designers before us changed everything in a single generation, I think we can do it again and save civilization in the process.

Alchemist, by InRuntime Ltd.

As thousands of mobile companies and developers converge on Barcelona for this year's Mobile World Congress, the AIR team is delighted to announce a preview of Adobe AIR on Android. We are excited to publicly show a sample of AIR applications running on an Android phone. Using Adobe AIR, developers and designers will be able to build standalone applications to target devices running the Android operating system. These very same applications can also be deployed as desktop AIR applications on Windows, Mac, Linux, and also as applications on the iPhone using the Packager for iPhone . If you are in Barcelona, come by our booth to check out the exciting apps that we have running on an Android device. For the rest of you, we have some video demos of AIR on Android @ http://www.adobe.com/go/airmobile

Developers using AIR on Android will be able to leverage mobile-specific features--such as multi-touch, gestures, accelerometer, GPS, and screen orientation--to deliver richer and more immersive user experiences across multiple operating systems.

Prior to the availability of AIR on Android, developers will be able to create mobile applications targeting the iPhone using the Packager for iPhone solution that we announced at Adobe MAX 2009. They share the same APIs to make the applications portable. We've worked with several developers on the applications below to get them running on AIR Android. Most of these applications are currently available in the Apple App Store today!

Alchemist by InRuntime Ltd.	FickleBlox by BlueskyNorth Ltd.
Gridshock by Bowler Hat Games	Chroma Circuit by Bowler Hat Games
Red Hood by DifferenceGames	South Park Avatar Creator by MTV Networks
Maverie by DifferenceGames	Su by Theodore Patrick

AIR on Android will be available in the second half of 2010. Please check back for additional announcements this year. Our team is tremendously excited about helping Flash and AIR developers bring innovative applications to mobile devices using our tools and frameworks.

On Perfectionism, "stopping at 'good enough' is an easy way to ensure you’ll never accomplish anything remarkable."

Amazing, so many things have happened in the Flash Player engineering team over the past year. Lots I would love to talk about. But the purpose of this post is to deep dive into a subject Kevin Lynch touched upon recently, specifically Mac performance and his comment about Core Animation. Whenever performance is mentioned in the context of Flash it gathers a lot of the attention and some of the technical background is lost in the PR.

So what's the deal with Core Animation in Flash Player 10.1? Let's look at how Apple's documentation summarizes what Core Animation does:

Core Animation is an Objective-C framework that combines a high-performance compositing engine with a simple to use animation programming interface.

Sounds like perfect match for Flash does it not? So yes, Flash Player 10.1 is attempting to leverage this framework to work around a few specific technical issues we've had in Safari and all other browsers on OS X.

The drawing model jungle on OS X

Before going into more specifics of why we are going towards Core Animation lets get an overview about how plugins on OS X draw into the browser window. There 4 possible ways (compared to one on Windows):

1. QuickDraw. Default mode used by Opera, older Firefox and Safari versions.
2. Quartz 2D (a.k.a. Core Graphics). Supported by newer versions of Firefox and Safari.
3. OpenGL. No browser I know of supports this properly today.
4. Core Animation. Only available in Safari 4 + OS X 10.6 right now, with caveats in the current version.

In addition to these drawing models designers can embed Flash content in 3 different ways by specifying wmode:

1. Normal
2. Opaque
3. Transparent

Normal means that you can't have overlapping HTML sitting on top of your SWF, Opaque allows it and Transparent means that the SWF is transparent and underlying HTML content will show through. Taking all these variables into account we come up with these tables which shows when a particular drawing model is used (and subject for change before we release Flash Player 10.1):

Flash Player 10.0:

	Safari 4	Firefox 3	Opera 10
Normal	Quartz 2D	QuickDraw	QuickDraw
Opaque	Quartz 2D	QuickDraw	QuickDraw
Transparent	Quartz 2D	QuickDraw	QuickDraw

Flash Player 10.1:

	Safari 4 (*)	Firefox 3	Opera 10
Normal	Core Animation	Quartz 2D	QuickDraw
Opaque	Quartz 2D(**)	Quartz 2D	QuickDraw
Transparent	Quartz 2D(**)	Quartz 2D	QuickDraw

(*) Actually using nightly builds of WebKit because support for Core Animation is work in progress.
(**) Core Animation is used when the SWF is the front most object on the HTML page.

What are the issues with Quartz 2D?

The basic premise of Quartz 2D as Apple describes it:

Quartz 2D is an advanced, two-dimensional drawing engine available for iPhone application development and to all Mac OS X application environments outside of the kernel. Quartz 2D provides low-level, lightweight 2D rendering with unmatched output fidelity regardless of display or printing device.

Quartz 2D is not designed for multimedia applications, like animation or video playback. That's where OpenGL, Core Video, Core Animation shine. Safari's use of Quartz 2D to draw HTML content makes perfect sense as its content is static in most cases. Everything works well until Flash comes into the picture. For instance when the Flash Player plays a SWF using the Quartz 2D drawing model is has to do so with the full involvement of the browser. The sequence of events looks like this (you can follow the stack traces in Shark):

1. Whenever the Flash Player is ready to display a new frame, the Flash Player requests a refresh of its region using NPN_InvalidateRect.
2. The browser adds the the rectangle provided by the Flash Player to its dirty region.
3. The browser traverses its own display list (the HTML DOM) and paints every node which is part of the dirty region.
4. When the browser finds a node with a Flash Player instance it first draws the HTML background and then posts an event to the Flash Player to tell it that it has to paint over the requested region now.
5. The Flash Player then finally draws its frame.

So far so good, makes sense I hope. So what's the technical issue? Think of a fairly complex HTML page, for instance a page with a CSS gradient in the background. Add to add a SWF which runs at 30 frames/sec. You will see that a lot of time is spent in the browser, not in the Flash Player. This is where Core Animation kicks in: step 3 and 4 pretty much go away (as long as the SWF is the top most object).

Core Animation in the Flash Player

Flash Player 10.1 implements the Core Animation drawing model to fix this technical issue, among others. Instead of using a CGImageRef + CGContextDrawImage to get the bits to screen we pass a CAOpenGLLayer to Safari and use an OpenGL texture of type GL_TEXTURE_RECTANGLE_ARB to get our bits to the screen.

The support for the Core Animation drawing model was originally driven by Apple and we have worked feverishly to finish the engineering work on both sides. Yes that's right: This was and is a joint effort between Apple and Adobe engineers. Given the now almost perfect integration of Core Animation plugins into Safari I hope that future versions of the Flash Player will take advantage of more capabilities of OpenGL. And that without the requirement of setting any special wmode. I am pretty stoked about it.

As of today (2/10/2010) we are getting closer to having it stable enough for public consumption. That means though: You will need Flash Player 10.1, OS X 10.6 and updated version of Safari (or the nightly WebKit build), otherwise you will not see anything.

What difference does it really make?

This is by no means panacea for all performance issues in the Flash Player. Far from it. But it is a small step to a larger goal which is to improve the experience in the browser with the ever more complex web content out there. That said here is a comparison between Flash Player 10.0 and Flash 10.1 using this test case (this only works in Safari). Keep in mind that that is an extreme test case which has little to do with real world web content.

Flash Player 10.0 + nightly WebKit + OS X 10.6


Flash Player 10.1 + nightly WebKit + OS X 10.6


PS: You might have noticed that Core Animation is a Cocoa API. Yes, Flash Player 10.1 is a true Cocoa app now (with a Carbon fallback to support Firefox and Opera which are not Cocoa yet).