something in the way

a tumblog about design + code
Apr 11

Sneak Peek of Future of the Flash Runtime!

Media_httpwwwbytearra_ioshm
For those who have missed this, Arno Gourdol, leading the Flash Runtime engineering team presented a session at Flash Camp Brasil last week entitled, Future of the Flash Runtime. Arno unveiled some very cool features we are working on right now, and I wanted to make sure you guys know about those in case you did not luckily attend Arno's session.

If you are curious about the slides, you can download them here.

Here are some of the features we are working on :

- Faster GC : GC hint API and more.
- New numeric types : float and float4 (very useful for Molehill in the future).
- Concurrency : Worker threads (shared nothing model) to leverage multicore CPU's. No more UI 's blocked when doing expensive operations.
- Stage3D : The API used for Molehill (that you know through the Incubator builds).
- StageVideo : Allowing full GPU acceleration (decoding + blitting) when used with H.264. Part of Flash Player 10.2 and coming to AIR soon.
- Threaded video pipeline : Will decode non H.264 streams on another thread (H.264 being decoded by the GPU), Net I/O will also be moved to another thread, all this bringing smoother playback.

I will be covering some of those in more details later on, stay tuned!

Media_httpwwwbytearra_ztjbg

Mar 8

Introducing Flash Player 10.3 beta!

Media_httpwwwbytearra_gzmox
We recently announced the Incubator program with a preview of the next generation 3D GPU accelerated APIs (Molehill) coming in the Adobe Flash runtimes. As you know, features or builds from the Incubator program include features under development or things we are considering adding. But what about features coming in the next Flash Player release ? Well, we are happy to announce the availability of Flash Player 10.3 beta for Windows, MacOS and Linux.

Note : Note that the Incubator builds available in the Incubator program contain the features available in Flash Player 10.3.

Flash Player 10.3 will also be available in the future on mobile devices, we invite you to test the following features and let us know what you think through the Adobe Flash Player 10.3 beta forums:

Media Measurement – Measuring video just got easier. With Flash Player 10.3 and Adobe® SiteCatalyst®, developers can implement video analytics with as little as two lines of code for the first time.  Analytics solutions can use a new set of open APIs to easily implement consistent video analytics irrespective of implementation or delivery protocol.  Media Measurement for Flash allows companies to get real-time, aggregated reporting of how their video content is distributed, what the audience reach is, and how much video is played. For instance, using a very few lines of code you will be able to retrieve all the NetStream instances created in your applications, access any meta data (XMP) associated to media being played, and more through the NetDataEvent event. Make sure to check the NetMonitor API and the additions done to the NetStreamInfo API. We also improved the consistency over events dispatched by theNetStream API when dealing with progressive, HTTP or RTMP streaming. We also added helper APIs like pageDomain property on the Security object to expose domain of the current URL.

Acoustic Echo Cancellation – Flash Player 10.3 enables developers to create real-time online collaboration experiences with high-quality audio, such as telephony, in-game voice chat, and group conferencing applications. Developers can take advantage of acoustic echo cancellation, noise suppression, voice activity detection, and automatic compensation for various microphone input levels. End users will be able to experience higher quality audio facilitating smoother conversation flow, without using a headset. Make sure you check the newMicrophoneEnhancedOptions and Microphone.getEnhancedMicrophone() APIs.

In addition to acoustic echo removal, the following functionality will also be available through the MicrophoneEnhancedOptions API:

  • Noise suppression - Remove ambient noise from microphone input.
  • Automatic gain control - Compensate for level variation of microphone input (due to distance of speaker, etc.).
  • Voice activity detection - Provides information whether speech/voice is detected.

Integration with browser privacy controls for managing local storage – Users will have a simpler way to clear local storage from the browser settings interface – similar to how users clear their browser cookies today. Flash Player 10.3 integrates control of local storage with the browser’s privacy settings in Mozilla Firefox 4, Microsoft Internet Explorer 8 and higher, and future releases of Apple Safari and Google Chrome. See this related post we published in January.

Native Control Panel – The Global Settings Manager is currently a SWF residing on macromedia.com, Flash Player 10.3 provides users with streamlined controls for managing their Flash Player privacy, security and storage settings. Windows, Mac, and Linux users can access the Flash Player Settings Manager directly from the Control Panels or System Preferences on their computers. See this related post we published in January.

Auto-Update Notification for Mac OS – Flash Player 10.3 supports automatic notification of software updates on Mac OS, making it easier for Mac users to stay current with new capabilities in the latest version of Flash Player.

Now, you may be thinking, what! No 64-bit version! As we announced during the Flash Player "Square" program, 64-bit is coming for the next major version of the Flash Player, so please wait a little more time, I know it is painful, but this is for the good! Next major version will be killer

Media_httpwwwbytearra_vbcjf

To download the beta release of Flash Player 10.3, visit Adobe Labs. We look forward to hearing your feedback, which will help us make Flash Player better for you. To log bugs about issues found in Flash Player 10.3 builds, please use the Adobe bugbase at bugs.adobe.com/flashplayer and select Flash Player 10.3 for the version.

Dec 1

Introducing Flash Player 10.2 Beta!

Media_httpwwwbytearra_saeue
I told you, a lot of stuff is happening lately for Flash developers. I am happy to announce that we just posted a new Flash Player 10.2 Beta on labs. I will not cover here all the questions that are covered already in the FAQ and release notes but I would like to focus on the new features introduced in this new beta:

  • Internet Explorer 9 hardware accelerated rendering support – Flash Player takes advantage of hardware accelerated graphics in Internet Explorer 9, utilizing hardware rendering surfaces to improve graphics performance and enable seamless composition.
  • Stage Video hardware acceleration – A new method for video playback in Flash Player allows developers to leverage complete hardware acceleration of the video rendering pipeline, enabling best-in-class playback performance. Stage Video can decrease processor usage by up to 85% and enables higher frame rates, reduced memory usage, and greater pixel fidelity and quality.
  • Native custom mouse cursors – Developers can define custom native mouse cursors, enabling user experience enhancements and improving performance.
  • Support for full screen mode with multiple monitors – Full screen content will remain in full-screen on secondary monitors, allowing users to watch full-screen content while working on another display.

Obviously, there is no much code I can show you regarding IE9 GPU compositing, the Flash Player does this automatically by leveraging the new GPU compositing APIs on IE9 to actually render the Flash Player frames as surfaces and push them on screen through Direct3D9 whatever wmode value you use. Note that graphics computing is still done by the CPU but we use GPU APIs to composite and blit the Flash Player frames on screen. It is similar to what we did in the past with CoreAnimation on MacOS with Safari. This will improve rendering performance on existent content without any modifications.

Concerning Stage Video, this feature has been sneaked at FOTB earlier this year and at Max last month by Tinic. We are very happy to announce that YouTube is already leveraging Stage Video. This will dramatically change the way video performs in Flash and bring native video performance on MacOS, Windows and Linux. For more infos about the YouTube work, check the Flash Player team's blog post.

To get an idea of how StageVideo performs, you can try the StageVideo enabled players here.
Make sure you are running it with Flash Player 10.2 beta.

I wrote a complete article on Devnet which goes into the details of StageVideo and wrote a very simple video player to illustrate how the StageVideo APIs work.

Here is a snaphost of the demo using Stage Video, playing the Big Buck Bunny movie in 1080p on my MacBook Pro :

Media_httpwwwbytearra_cfbbs

For comparaison, here is a 720p video playing on YouTube, using about 40% of my MacBook Pro CPUs with Flash Player 10.1 :

Media_httpwwwbytearra_zibhd

Now, let's browse to same video with Flash Player 10.2 with Stage Video, CPUs usage drops to 9% :

Media_httpwwwbytearra_idhtb

Pretty cool, hu ? If you make the test on YouTube, make sure you wait a few seconds, some ActionScript logic in the YouTube player makes the CPU usage go wild for a few seconds before reaching its normal state.

For info, the performance improvements are way bigger if you overlay content on top of video. In some cases, CPU usage can drop from 200% to 10% using Stage Video, check my first post about Stage Video for more infos.

While listening to my track and browsing the web through another tab, the pause and resume feature introduced in Flash Player 10.1 stops rendering the frames and only decodes sound. Flash Player CPU usage then drops to 5% :

Media_httpwwwbytearra_iipgj

But that is not all, we added another nice feature related to cursors. A highly requested feature. The idea is simple, no more Mouse.hide() or onMouseMove handler or startDrag to have a custom cursor. You can now directly work at the OS level and assign your own custom native bitmap-based cursors. Here is a little code sample which illustrates the idea:

// we create a MouseCursorData object
var cursorData:MouseCursorData = new MouseCursorData();
// we specify the hotspot
cursorData.hotSpot = new Point(15,15);
// we pass the cursor bitmap to a BitmapData Vector
var bitmapDatas:Vector.<BitmapData> = new Vector.<BitmapData>(1, true);
// we create the bitmap cursor (bitmaps should not be bigger than 32x32 pixels, this is an OS limitation)
var bitmap:Bitmap = new zoomCursor();
// we pass it to the bitmapDatas vector
bitmapDatas[0] = bitmap.bitmapData;
// we assign the bitmap to the MouseCursor object
cursorData.data = bitmapDatas;
// we register the MouseCursorData to the Mouse with an alias
Mouse.registerCursor("myCursor", cursorData);
// whenever we neeed to show it, we pass the alias to the existing cursor property
Mouse.cursor = "myCursor";

Pretty cool? But wait, there is even better, you can also pass a series of BitmapData objects, then pass a specific frame rate and get automatic native animated cursor:

// we create a MouseCursorData object
var cursorData:MouseCursorData = new MouseCursorData();
// we specify the hotspot
cursorData.hotSpot = new Point(15,15);
// we pass the cursors bitmap to a BitmapData Vector
var bitmapDatas:Vector.<BitmapData> = new Vector.<BitmapData>(3, true);
// we create the bitmap cursor frames (bitmaps should not be bigger than 32x32 pixels, this is an OS limitation)
var frame1Bitmap:Bitmap = new frame1();
var frame2Bitmap:Bitmap = new frame2();
var frame3Bitmap:Bitmap = new frame3();
// we pass it to the bitmapDatas vector
bitmapDatas[0] = frame1Bitmap.bitmapData;
bitmapDatas[1] = frame2Bitmap.bitmapData;
bitmapDatas[2] = frame3Bitmap.bitmapData;
// we assign the bitmap to the MouseCursor object
cursorData.data = bitmapDatas;
// we register the MouseCursorData to the Mouse
Mouse.registerCursor("myAnimatedCursor", cursorData);
// we just pas a frame rate
cursorData.frameRate = 1;
// whenever we neeed to show it, we pass the alias to the existing cursor property
Mouse.cursor = "myAnimatedCursor";

Very simple, but so cool. Game and RIA developers will love this feature.

Another addition, we enhanced the full screen behavior by allowing one screen to stay full screen while working on the other screen. This feature was also highly requested by the community and video industry. Very nice feature for Hulu, YouTube and other video websites.

Media_httpwwwbytearra_mmmfp

A lot of people expressed the need to actually detect if the current context allowed the SWF to go full screen. This would allow you to detect this at runtime and react appropriately. Departing from Flash Player 10.2, you will be able to use the allowFullscreen property on stage:

if ( !stage.allowFullScreen )
throw new Error ("Please use the allowFullScreen HTML tag.");

Finally, a little improvement regarding Flash Player version info in the context menu, Here is what you have today when right click, no way to know which exact version you have running :

Media_httpwwwbytearra_eixdh

Here is what you will get starting from 10.2 :

Media_httpwwwbytearra_xqejb

I agree, it was about time. But yes, way easier now

Media_httpwwwbytearra_mmmfp

To use those new APIs you need to use the special build 18623 of the Flex SDK 4.5, from the Hero Stable Builds table here.
You also need to use a specific compiler argument –swf-version=11 to target SWF11.

I hope you guys will enjoy those features. Give it a try and let us know about bugs or any feedback you may have, thanks!

Oct 26

Introducing the Molehill 3D APIs

I am sure you guys have heard about what we just announced at the Max keynote this morning.

Media_httpwwwbytearra_lvjfu

I am happy to share this with you guys

Media_httpwwwbytearra_ysstg

Molehill Introduction :

Another video from MaxRacer demoed at Max Keynote Day 1 built on top of Alternativa 8, leveraging the upcoming 3D Molehill APIs :

Tom demoes the Peer 2 Peer feature he developed for MaxRacer :

MAX Racer - 3D Flash Game with P2P Multiplayer from Tom Krcha on Vimeo.

Another one, with those beautiful islands, still with Molehill :

Another one from Frima Studio, who ported their engine used on the PSP for Zombie Tycoon to Molehill :

Another one from Away3D and EvoFlash using Molehill :

And again from Away3D :

For more details about implementation and how it works, check the Adobe labs Molehill page.

Update : For info, Molehill is also available in the browser, this is not limited to AIR or standalone player

Media_httpwwwbytearra_jbmgk

Oct 2

Video Improvements, a sneak peek from “Flash on the Beach” keynote

Last Monday at "Flash on the Beach" keynote, I showcased new improvements we have been doing lately with video playback in the Flash Player.

So what happened ?

As you guys may know, we introduced earlier this year Flash Player 10.1 with H.264 GPU decoding on MacOS, Windows and Linux, which already highly improved the CPU usage when viewing H.264 video. But we are going further with dramatic CPU usage reduction in an upcoming version of the Flash Player.

Below is a snapshot of the Activity Monitor using Flash Player 10.1, playing a 1080p video with H.264 GPU decoding enabled :

Media_httpwwwbytearra_hczpg

You see that for such a video, CPU is still being used for about 50%, cause the CPU is not totally offloaded, still displaying and scaling the video frames.
Now in the following picture I am using the new feature we will be introducing soon, as you can see the CPU is highly offloaded and playing a 1080p video with around 8% CPU usage.

Note how the memory usage drops down too :

Media_httpwwwbytearra_uiyba

It gets even better when you start overlaying content on top of the video. Most of Flash websites use a lot of overlayed content on top of the video. In the snapshot below, the CPU usage goes pretty high, cause complex content is on top of my video, the CPU is working on compositing the video frames and the vector content in the display list which is an intensive task :

Media_httpwwwbytearra_gasgq

By leveraging the upcoming feature, take a look at my CPU usage and memory now, the exact same content with a few lines of code changed :

Media_httpwwwbytearra_daaie

Again, this will dramatically improve the video performance in the Flash Player. Note that this feature will be available on Windows, MacOS and Linux, and this new improvement is not restricted to H.264 video or specific video cards vendors, any video frame whatever the codec used will benefit from this.

If you are interested into learning more about this, do not hesitate to meet us at Max.
If you want to try this as soon as possible, just drop me an email.

Hope you guys like it!

Jul 7

Flash Player 3D Future session at Max 2010 [ by Thibault Imbert ]

Media_httpwwwbytearra_myiex

If you are into 3d development for games, augmented reality or just interactive stuff like websites, you just can't miss the session entitled Flash Player 3D future scheduled for Max 2010 scheduled on October 27 at 11:00AM in room 503. Sebastian Marketsmueller (Flash Player engineer) will deep dive into the next generation 3D API coming in a future version of the Flash Player.

Now you may wonder, what does this means, what kind of 3D are we talking about ?

What kind of API ? True textured z-buffered triangles ? GPU acceleration ? Even better ? What I can say is forget what you have seen before, it is going to be big

Media_httpwwwbytearra_pxqta

When this will be available ?

We will share plans with you at Max during this session, I tell you, some serious stuff is coming for 3D developers.

If you are also curious about the inner details of the Flash Player renderer, Lee Thomason (Flash Player architect) will delve into the details of the Flash Player renderer, and show how to optimize the rendering performance of your applications. Lee will cover mechanisms like the display list, text rendering, shaders, GPU hardware acceleration, and exclusive features coming in a future version of Flash Player (hehe).

After this session, Flash Player rendering will no longer be a mystery for you. This track called Deep Dive into Flash Player Rendering is scheduled on October 27 at 09:30AM in room 511A.

I will be happy to meet you there at Max in Los Angeles and talk about our future plans and also get your feedbacks about the player around a fresh beer. I will post further details about all this in the following weeks.

If you haven't checked all the sessions available for Max this year check the online scheduler.

Get Updates

Tags

Archive

2012 (6)
2011 (11)