something in the way

a tumblog about design + code
May 6

Molehill on mobile devices

Last week at FITC, Lee Brimelow demoed Molehill running in AIR on a mobile platform (Android) for the first time. It seems like you guys liked it!

Media_httpwwwbytearra_yhidz
As we stated earlier, Molehill has been designed from the ground up with mobile in mind, on mobile platforms Molehill (Stage3D) is using OpenGL ES2.

You guys will be able to leverage the Molehill APIs on mobile platforms for 2D and 3D rendering, for you guys who did not attend FITC, here is a little video demoing Molehill on Android through AIR. The same code is reused for the desktop version and then pushed to mobile :

Molehill - Tablet demo from Thibault Imbert on Vimeo.

This demo illustrates a common thing in 3D like cube map textures for reflections and is one of the demo from our test suite, so we have many little demos like this. I will be posting more complex examples soon!

Media_httpwwwbytearra_yhidz

Apr 11

Sneak Peek of Future of the Flash Runtime!

Media_httpwwwbytearra_ioshm
For those who have missed this, Arno Gourdol, leading the Flash Runtime engineering team presented a session at Flash Camp Brasil last week entitled, Future of the Flash Runtime. Arno unveiled some very cool features we are working on right now, and I wanted to make sure you guys know about those in case you did not luckily attend Arno's session.

If you are curious about the slides, you can download them here.

Here are some of the features we are working on :

- Faster GC : GC hint API and more.
- New numeric types : float and float4 (very useful for Molehill in the future).
- Concurrency : Worker threads (shared nothing model) to leverage multicore CPU's. No more UI 's blocked when doing expensive operations.
- Stage3D : The API used for Molehill (that you know through the Incubator builds).
- StageVideo : Allowing full GPU acceleration (decoding + blitting) when used with H.264. Part of Flash Player 10.2 and coming to AIR soon.
- Threaded video pipeline : Will decode non H.264 streams on another thread (H.264 being decoded by the GPU), Net I/O will also be moved to another thread, all this bringing smoother playback.

I will be covering some of those in more details later on, stay tuned!

Media_httpwwwbytearra_ztjbg

Dec 1

Introducing Flash Player 10.2 Beta!

Media_httpwwwbytearra_saeue
I told you, a lot of stuff is happening lately for Flash developers. I am happy to announce that we just posted a new Flash Player 10.2 Beta on labs. I will not cover here all the questions that are covered already in the FAQ and release notes but I would like to focus on the new features introduced in this new beta:

  • Internet Explorer 9 hardware accelerated rendering support – Flash Player takes advantage of hardware accelerated graphics in Internet Explorer 9, utilizing hardware rendering surfaces to improve graphics performance and enable seamless composition.
  • Stage Video hardware acceleration – A new method for video playback in Flash Player allows developers to leverage complete hardware acceleration of the video rendering pipeline, enabling best-in-class playback performance. Stage Video can decrease processor usage by up to 85% and enables higher frame rates, reduced memory usage, and greater pixel fidelity and quality.
  • Native custom mouse cursors – Developers can define custom native mouse cursors, enabling user experience enhancements and improving performance.
  • Support for full screen mode with multiple monitors – Full screen content will remain in full-screen on secondary monitors, allowing users to watch full-screen content while working on another display.

Obviously, there is no much code I can show you regarding IE9 GPU compositing, the Flash Player does this automatically by leveraging the new GPU compositing APIs on IE9 to actually render the Flash Player frames as surfaces and push them on screen through Direct3D9 whatever wmode value you use. Note that graphics computing is still done by the CPU but we use GPU APIs to composite and blit the Flash Player frames on screen. It is similar to what we did in the past with CoreAnimation on MacOS with Safari. This will improve rendering performance on existent content without any modifications.

Concerning Stage Video, this feature has been sneaked at FOTB earlier this year and at Max last month by Tinic. We are very happy to announce that YouTube is already leveraging Stage Video. This will dramatically change the way video performs in Flash and bring native video performance on MacOS, Windows and Linux. For more infos about the YouTube work, check the Flash Player team's blog post.

To get an idea of how StageVideo performs, you can try the StageVideo enabled players here.
Make sure you are running it with Flash Player 10.2 beta.

I wrote a complete article on Devnet which goes into the details of StageVideo and wrote a very simple video player to illustrate how the StageVideo APIs work.

Here is a snaphost of the demo using Stage Video, playing the Big Buck Bunny movie in 1080p on my MacBook Pro :

Media_httpwwwbytearra_cfbbs

For comparaison, here is a 720p video playing on YouTube, using about 40% of my MacBook Pro CPUs with Flash Player 10.1 :

Media_httpwwwbytearra_zibhd

Now, let's browse to same video with Flash Player 10.2 with Stage Video, CPUs usage drops to 9% :

Media_httpwwwbytearra_idhtb

Pretty cool, hu ? If you make the test on YouTube, make sure you wait a few seconds, some ActionScript logic in the YouTube player makes the CPU usage go wild for a few seconds before reaching its normal state.

For info, the performance improvements are way bigger if you overlay content on top of video. In some cases, CPU usage can drop from 200% to 10% using Stage Video, check my first post about Stage Video for more infos.

While listening to my track and browsing the web through another tab, the pause and resume feature introduced in Flash Player 10.1 stops rendering the frames and only decodes sound. Flash Player CPU usage then drops to 5% :

Media_httpwwwbytearra_iipgj

But that is not all, we added another nice feature related to cursors. A highly requested feature. The idea is simple, no more Mouse.hide() or onMouseMove handler or startDrag to have a custom cursor. You can now directly work at the OS level and assign your own custom native bitmap-based cursors. Here is a little code sample which illustrates the idea:

// we create a MouseCursorData object
var cursorData:MouseCursorData = new MouseCursorData();
// we specify the hotspot
cursorData.hotSpot = new Point(15,15);
// we pass the cursor bitmap to a BitmapData Vector
var bitmapDatas:Vector.<BitmapData> = new Vector.<BitmapData>(1, true);
// we create the bitmap cursor (bitmaps should not be bigger than 32x32 pixels, this is an OS limitation)
var bitmap:Bitmap = new zoomCursor();
// we pass it to the bitmapDatas vector
bitmapDatas[0] = bitmap.bitmapData;
// we assign the bitmap to the MouseCursor object
cursorData.data = bitmapDatas;
// we register the MouseCursorData to the Mouse with an alias
Mouse.registerCursor("myCursor", cursorData);
// whenever we neeed to show it, we pass the alias to the existing cursor property
Mouse.cursor = "myCursor";

Pretty cool? But wait, there is even better, you can also pass a series of BitmapData objects, then pass a specific frame rate and get automatic native animated cursor:

// we create a MouseCursorData object
var cursorData:MouseCursorData = new MouseCursorData();
// we specify the hotspot
cursorData.hotSpot = new Point(15,15);
// we pass the cursors bitmap to a BitmapData Vector
var bitmapDatas:Vector.<BitmapData> = new Vector.<BitmapData>(3, true);
// we create the bitmap cursor frames (bitmaps should not be bigger than 32x32 pixels, this is an OS limitation)
var frame1Bitmap:Bitmap = new frame1();
var frame2Bitmap:Bitmap = new frame2();
var frame3Bitmap:Bitmap = new frame3();
// we pass it to the bitmapDatas vector
bitmapDatas[0] = frame1Bitmap.bitmapData;
bitmapDatas[1] = frame2Bitmap.bitmapData;
bitmapDatas[2] = frame3Bitmap.bitmapData;
// we assign the bitmap to the MouseCursor object
cursorData.data = bitmapDatas;
// we register the MouseCursorData to the Mouse
Mouse.registerCursor("myAnimatedCursor", cursorData);
// we just pas a frame rate
cursorData.frameRate = 1;
// whenever we neeed to show it, we pass the alias to the existing cursor property
Mouse.cursor = "myAnimatedCursor";

Very simple, but so cool. Game and RIA developers will love this feature.

Another addition, we enhanced the full screen behavior by allowing one screen to stay full screen while working on the other screen. This feature was also highly requested by the community and video industry. Very nice feature for Hulu, YouTube and other video websites.

Media_httpwwwbytearra_mmmfp

A lot of people expressed the need to actually detect if the current context allowed the SWF to go full screen. This would allow you to detect this at runtime and react appropriately. Departing from Flash Player 10.2, you will be able to use the allowFullscreen property on stage:

if ( !stage.allowFullScreen )
throw new Error ("Please use the allowFullScreen HTML tag.");

Finally, a little improvement regarding Flash Player version info in the context menu, Here is what you have today when right click, no way to know which exact version you have running :

Media_httpwwwbytearra_eixdh

Here is what you will get starting from 10.2 :

Media_httpwwwbytearra_xqejb

I agree, it was about time. But yes, way easier now

Media_httpwwwbytearra_mmmfp

To use those new APIs you need to use the special build 18623 of the Flex SDK 4.5, from the Hero Stable Builds table here.
You also need to use a specific compiler argument –swf-version=11 to target SWF11.

I hope you guys will enjoy those features. Give it a try and let us know about bugs or any feedback you may have, thanks!

Oct 26

Introducing the Molehill 3D APIs

I am sure you guys have heard about what we just announced at the Max keynote this morning.

Media_httpwwwbytearra_lvjfu

I am happy to share this with you guys

Media_httpwwwbytearra_ysstg

Molehill Introduction :

Another video from MaxRacer demoed at Max Keynote Day 1 built on top of Alternativa 8, leveraging the upcoming 3D Molehill APIs :

Tom demoes the Peer 2 Peer feature he developed for MaxRacer :

MAX Racer - 3D Flash Game with P2P Multiplayer from Tom Krcha on Vimeo.

Another one, with those beautiful islands, still with Molehill :

Another one from Frima Studio, who ported their engine used on the PSP for Zombie Tycoon to Molehill :

Another one from Away3D and EvoFlash using Molehill :

And again from Away3D :

For more details about implementation and how it works, check the Adobe labs Molehill page.

Update : For info, Molehill is also available in the browser, this is not limited to AIR or standalone player

Media_httpwwwbytearra_jbmgk

Oct 14

Is that HTML in your AIR on Android App? Yes it is!

AIR on Android doesn’t have an HTML component like its desktop counterpart but there is a way to get HTML in your application. The StageWebView class can accomplish this for you. I have added this to my AIRonAndroid Browser application (version 0.0.4) so you can see it in action if you have an Android device. [...]
Media_httpfeedsfeedbu_nsuoh
Media_httpfeedsfeedbu_cpmwh
Media_httpfeedsfeedbu_pcnyw
Media_httpfeedsfeedbu_qaiej
Media_httpfeedsfeedbu_ekael
Media_httpfeedsfeedbu_bjghu
Media_httpfeedsfeedbu_thhlj
Media_httpfeedsfeedbu_hezky
Media_httpfeedsfeedbu_gaibf
Oct 2

Video Improvements, a sneak peek from “Flash on the Beach” keynote

Last Monday at "Flash on the Beach" keynote, I showcased new improvements we have been doing lately with video playback in the Flash Player.

So what happened ?

As you guys may know, we introduced earlier this year Flash Player 10.1 with H.264 GPU decoding on MacOS, Windows and Linux, which already highly improved the CPU usage when viewing H.264 video. But we are going further with dramatic CPU usage reduction in an upcoming version of the Flash Player.

Below is a snapshot of the Activity Monitor using Flash Player 10.1, playing a 1080p video with H.264 GPU decoding enabled :

Media_httpwwwbytearra_hczpg

You see that for such a video, CPU is still being used for about 50%, cause the CPU is not totally offloaded, still displaying and scaling the video frames.
Now in the following picture I am using the new feature we will be introducing soon, as you can see the CPU is highly offloaded and playing a 1080p video with around 8% CPU usage.

Note how the memory usage drops down too :

Media_httpwwwbytearra_uiyba

It gets even better when you start overlaying content on top of the video. Most of Flash websites use a lot of overlayed content on top of the video. In the snapshot below, the CPU usage goes pretty high, cause complex content is on top of my video, the CPU is working on compositing the video frames and the vector content in the display list which is an intensive task :

Media_httpwwwbytearra_gasgq

By leveraging the upcoming feature, take a look at my CPU usage and memory now, the exact same content with a few lines of code changed :

Media_httpwwwbytearra_daaie

Again, this will dramatically improve the video performance in the Flash Player. Note that this feature will be available on Windows, MacOS and Linux, and this new improvement is not restricted to H.264 video or specific video cards vendors, any video frame whatever the codec used will benefit from this.

If you are interested into learning more about this, do not hesitate to meet us at Max.
If you want to try this as soon as possible, just drop me an email.

Hope you guys like it!

Jul 7

Flash Player 3D Future session at Max 2010 [ by Thibault Imbert ]

Media_httpwwwbytearra_myiex

If you are into 3d development for games, augmented reality or just interactive stuff like websites, you just can't miss the session entitled Flash Player 3D future scheduled for Max 2010 scheduled on October 27 at 11:00AM in room 503. Sebastian Marketsmueller (Flash Player engineer) will deep dive into the next generation 3D API coming in a future version of the Flash Player.

Now you may wonder, what does this means, what kind of 3D are we talking about ?

What kind of API ? True textured z-buffered triangles ? GPU acceleration ? Even better ? What I can say is forget what you have seen before, it is going to be big

Media_httpwwwbytearra_pxqta

When this will be available ?

We will share plans with you at Max during this session, I tell you, some serious stuff is coming for 3D developers.

If you are also curious about the inner details of the Flash Player renderer, Lee Thomason (Flash Player architect) will delve into the details of the Flash Player renderer, and show how to optimize the rendering performance of your applications. Lee will cover mechanisms like the display list, text rendering, shaders, GPU hardware acceleration, and exclusive features coming in a future version of Flash Player (hehe).

After this session, Flash Player rendering will no longer be a mystery for you. This track called Deep Dive into Flash Player Rendering is scheduled on October 27 at 09:30AM in room 511A.

I will be happy to meet you there at Max in Los Angeles and talk about our future plans and also get your feedbacks about the player around a fresh beer. I will post further details about all this in the following weeks.

If you haven't checked all the sessions available for Max this year check the online scheduler.

Dec 16

Voice Gesture [ by Didier Brun ]

Do you remember Mouse Gesture ?

I am currently working on its good friend : Voice Gesture.

Well, the name is not very explicit, but it's a voice recognition library based on the Flash Player 10.1 Microphone new feature.

I just want to show you a very early demonstration :

Voice Gesture from didier.brun on Vimeo.

As you can see, it works pretty fine ( > 95% accuracy ) but I have to admit that, for now, the algorithm require these 2 points :

  • The user is the trainer (I have recorded my own voice models into the library)
  • A silent place

I have some more work to optimize the algorithm and build an AIR application to record and organize the sound library.

So keep in touch, I will publish this library soon...

Tweet friend ? please share :
http://twitter.com/home?status=RT @didierbyte: I have just published a voice recognition demo using FP10.1 http://bit.ly/6b5LM0

PS : Voice Gesture works in a simple web-based Flash Player 10.1 (I have recorded this demo using FireFox), it is not specific to AIR 2.0.

Oct 6

Flash CS5 Will Compile Native iPhone and Touch Games and Applications Coded in AS3

Well good news for Flash developers, Flash CS5 will finally compile to native iPhone and Touch Applications. This is great news for many developers out there who have stuck with the Flash platform.  I am sure there will still be limitations to what you can do with Flash on the iPhone and it will probably be mostly 2D games and apps but this is a great start to getting the Flash platform truly mobile and up to the rest of the industry.

Flash Professional CS5 will enable you to build applications for iPhone and iPod touch using ActionScript 3. These applications can be delivered to iPhone and iPod touch users through the Apple App Store.*

A public beta of Flash Professional CS5 with prerelease support for building applications for iPhone is planned for later this year. Sign up to be notified when the beta starts.

I have been questioning why they have not moved to this model before when others are doing so such as haXe, Unity3D and MonoTouch.  Getting Flash on the web browsers on a mobile is hard because Flash is pretty CPU intensive on embedded devices which is really where computers were in the late 90’s and close to 400-600 MHz processors.  Today these machines wouldn’t be able to run Flash very well and that is the same effect you get on a mobile phone.  But cross-compiling to native, similar to how Unity 3D does it or other solutions like MonoTouch and XNATouch, this is the best solution until mobile/embedded devices have 1GHz processors and more than 500MB of memory. Adobe is using LLVM, much like the Alchemy model, to achieve getting AS3 content onto an iPhone/Touch with AOT or Ahead of Time compilation rather than JIT compilation.

So how do you build an application for the iPhone? It’s simple, really. The forthcoming beta of Adobe Flash Professional CS5 incorporates the ability to create an iPhone application. You have access to nearly all the AIR 2.0 and Flash Player 10.1 APIs. For example, you can use APIs such as RTMP, Remote Shared Objects, and AMF as well as AIR APIs like SQLite and filesystem access. For more information see the developer FAQ on Adobe Labs.

I am glad to see Adobe finally moving on mobile platforms beyond Flashlite.  Flashlite is a poor solution in most cases on embedded devices because they really need native apps to perform, again due to the hardware limitations and it is a whole new platform to learn. Adobe is doing the hard work to make it easy to get developers content on the new embedded devices that are storming the world such as the iPhone and Touch.

Media_httpfeedsfeedbu_xmcep
Jul 8

AS3 Library for OAuth with Twitter for AIR Apps

AIR is very popular for creating twitter clients, Sönke Rohde just made it much easier to make AIR apps for twitter with an AS3 library for Twitter. This library is built on top of core oauth as3 library by iotashan.  Core OAuth as3 library is a standard OAuth library this can be used for your own OAuth backends or connecting to other OAuth services as well.

A very nice feature of this library, in addition to being coded cleanly and as3 style, is the ability to have the Twitter OAuth page render inside of Flash.

Instead of opening the Twitter authorization page in the browser the library also contains OAuthLoader which is a wrapper around HTMLLoader which enables to directly show the authorization page within an AIR window:

// use this in the requestTokenHandler instead of navigateToURL
var loader:OAuthLoader = new OAuthLoader();
loader.load(request);
loader.percentWidth = 100;
loader.percentHeight = 100;
var w:Window = new Window();
w.width = 800;
w.height = 400;
w.title = req.url;
w.addChild(loader);
w.open();
Media_httpfeedsfeedbu_ofppb

Get Updates

Tags

Archive

2012 (1)
2011 (11)