Nexus One running Flash Player 10.1 from Ralph Hauwert on Vimeo.

This is a video of the Nexus One with a pre-release version of Adobe Flash Player 10.1. It does not represent the final quality of the Flash Player on this specific device. But it's pretty good as is!

This is the Nexus One running 2 demos. First a Nexus one specific version of the "Milkyball", which is adjusted in resolution and screensize.

Second a non-adjusted version of FlashModPlug, an experimental project I'm currently working on. It's a port and wrapper of LibModPlug, running inside the Flash Player through Adobe Alchemy.

Conclusion 10.1 on the Nexus is really, really fast!

(go to vimeo for the full resolution version). (link to nexus demo)

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...
2. Goodbye, Papervision. Last week at Flash on the Beach I had...
3. Help me test! Flash VSync and Screen Tearing. So, here's a post where I am asking you,...

Long time no see! I've been busy, sorry. First off all, I want to congratulate you on the great show today. Wow, that iPad is truly one of those things everyone is going to want to have, like your previous successes with the iPod and iPhone. I take my hat off for that! And wow, what a lead up to the show. Probably the first hype of 2010. And do you guys know how to keep a secret and then give a great show unveiling it.

That being said, I felt as an avid Apple products user as well as your friend, (you know I own a total of 3 Macbooks, a plethora of Airports and have converted nearly my entire family to buy one of those nice machines of yours) (actually it was Grant who got me into buying one) it's my duty to inform you that I think something went wrong during your presentation. When you were showing the web capabilities of the iPad, something was missing in it's browser (see screen-shot above). As your keynote and product presentations are normally flawless (ah, well, maybe not always), I think you might have missed this one.

You touted the iPad's (great product name, btw) web capabilities as being amazing, perfect, you know, the regular Apple thing. But during the presentation I couldn't help but notice that little "missing plugin" logo we all know from the iPhone. Now, I was thinking that this might have been one of those very exotic plugins of back in the day, like Director or Realplayer. But, as it turned out, the missing plugin was the Flash Player.

As you've started the introduction to the iPad with a reference to Netbooks, and we can safely say that most netbooks support Flash, I think we are both safe to assume that to be on par with those netbooks, the minimum you should have is a fully featured browser?
I mean, this is not some kind of small screen device with limited capabilities in terms of performance and graphics. It's a fully featured machine ? Be it with that amazing iPhone touch interface. And be it in a slightly new package intended for true casual and business use. (Honestly, you kind of confused me, games and reading books ? I thought the gaming generation didn't read and vice-versa, ah we'll talk about that after you write me back on this letter).

Anyway, I think it must have been a demonstration error, because it seemed like the iPad didn't support Flash?

Mythhackers.

Ok, now, who am I kidding. I know you're a perfectionist. And you're a man of subtle remarks too. Between you and me, you can admit it. We both know that little missing plugin logo wasn't an accident. I think we both know why you did it. It must have been your kind of humor, getting back at that little prank  Kevin and Johnny of Adobe pulled on you at Adobe MAX. (see picture above too). You have to admit, that was funny. Man, we all laughed about that one. But hey, I always think that if they're teasing you, they must want your attention. All in all, they didn't mean any harm.

(I do agree with you that they should've spent some more cash on the explosion effect in the video, being the maker of some of the most premiere video effect software out there, but hey, it was a good prank video, executed very well).

Now, given some of the easter eggs in Mac OS X, which essentially is your baby, through that other thing you did, with that next-cube thing you were so enthousiastic about back in the day, I take it you have a fair amount of subtle humor in you. It always reflects in your letters and I always have to laugh when I think about how you made that joke about a mouse only needing one button. But, as always, there's a time for fun and a time for seriousness. And you seriously can't mean that the full featured web browser doesn't support the biggest plugin out there?

If you do mean it, I can understand why. I've never really bought that, "Flash is too slow for the iPhone" story you told everyone at that party. I also know for a fact that RIM, Nokia, Samsung, NVidia and all those guys also didn't believe it.

And yeah, I know that little stunt Adobe pulled with compiling directly from Flash to iPhone isn't the same as running it in the browser, but hey, it is the output of Flash running on an iPhone. And surely there's a lot of content which can now be readily available in the App store. And if that puny little ARM processor in an iPhone can, then surely that incredible piece of custom silicon you so proudly called the Apple A4 can. (it's ARM too, right?).

Your nightmare ?

Ahh, going off track, I was saying that I understood why you would not want Flash. I know having rich, web enabled apps which don't go through the App store can be a scary thought, food for nightmares. I know you've had a thing for nightmares, especially after Monster's inc. But Steve, I have to tell you. A device which is intended for casual and easy use with all the slickness Apple always brings, but doesn't support so much web content? Come on, what will the people think? Those netbooks you called slow and PC-ish, can run Flash, but Apple's state of the art technology can't?

Yeah, I know it's hard to have that piece of proprietary software running so much content essentially being a separate platform, while having no control over it. But that's why Kevin and Johhny where throwing those jokes at you. They want to help you, really. And again between you and me, isn't it true that you're always selling computers when those guys at Adobe come up with a new version of their creativity software?

I would like to also come with a more serious note, as your friend, I know I can remind you of this one. Do you remember that when I was still using a PC, we were always talking sh*t about those guys at Microsoft for dominating the market? About how they pulled that monopoly thing on Netscape with that "lovely" (I still remember the word you had for it) browser we all still despise, Internet Explorer. And what they did to Real by bundling that Mediaplayer? Well, as it turns out, they kind of ran into some anti-trust issues with the European Union. I hear it was quite costly for Bill. While I agree the iPhone was a completely different thing, you can't bring out a computer and expect everyone to think it's just a big iPhone and then pull all that stuff off that our friend at Microsoft once did, right?

Yeah, yeah, I know what you're going to say. Flash is proprietary too and there should be an open alternative to it. But honestly, in terms of open initiatives competing with it, it seems those HTML5 guys are still not ready for it, and although having H.264 video support native to the browser is nice, it hardly replaces what Flash can do right now. Not to mention all that content already being offered by Flash, people can't see on the magnificent iPad. If pure HTML5 content is even 1% of that, we both know that would be a lot. I do feel you should be open to any plugin, honestly, so the guys at MS/Silverlight and Unity3D, for instance, also get a shot. All in all, it's your product's users who are going to suffer for it otherwise. Now, as I've made my living building Flash apps, of course I am biased, but doesn't every developer deserve a fair shot?

You're the man!

You were always this idealist, but this thing is making you look like you are just plain greedy, at the cost of your end users, not being able to access all web content. Give me back the idealist Steve, I love that guy! (I've included an old picture of you to remind you of that man).

Now, you know I do admire that you're not like all those others and you're doing things your way (and it seems to work), I think you're going down the wrong path on this one, or this might have been just an error. If it was, just fix it before release, and we'll keep it between you and me.

Sincerest Regards, your friend,
Ralph.

(quick note, can you make Johnathan stop saying the word Magical, every time he's talking about one of the products he designs ? Frankly, it's a bit annoying. And he's not designing unicorns or leprechauns.)

(quick note 2 : if you do manage to talk to Kevin or Shantanu at the next party, ask them if their engineering people can talk to your engineering people about that video hardware support and performance issues, I think it would make many of our friends happy).

No related posts.

(people who want to know more would refresh after seeing the pieces in full)

No related posts.

So, here's a post where I am asking you, the Flash community, for a hand. Any of it is truly appreciated and will hopefully get some attention and maybe even a solution to a problem which has been in around in Flash since as long as I can remember; screen tearing.

First, what does tearing mean in this context ? Read this wikipedia article. In this case, it's a visual split along a vertical line in a Flash movie. It's been a known issue with Flash since quite some time (always as far as I know), with improvements over several Flash player versions, but I haven't had to deal with it much lately....until now. I'm in a project where this visual artefact is a major distraction from the content and the experience.

All in all, this seems to be the cause (at least, my analysis of it) of tearing in this case; Flash internally rasterizes a screen, and pushes it to the screen buffer. Internally, this should be synced to your display refresh (60hz or 72hz, for instance), so that the buffer doesn't get updated halfway through the videocard building up the frame. But on some browsers and platforms it does update during the video card frame refresh, resulting in two rendered frames being displayed at once (or better; 1 frame being updated to the next while drawing to screen), with an optically visible vertical split visible between the two frames. This problem has been around for a long time. It's more prone to show up in content with "tall" items in it, fast frame rates with large portions of the screen moving, and during horizontal scrolling of displayobjects.

The picture above shows the problem in action. It's a picture taken using a regular camera pointed at the screen, as this problem doesn't show up in screencapture software. It shows two different frames of the testing movie; both a black and white square, updated halfway through the screen. While the position of where the slicing happens changes, in many single frames, 2 frames will be displayed at the same time.

I'm seeing problems with the latest 10 Player on Windows XP (Firefox), Mac OS X (Firefox, safari doesn't but "stutters), and Windows 7. All in all, this is a graphical issue which has been around for a long time and should be solved. It's definitely a Flash Player bug and with FlashPlayer 10.1 around the corner, I would really love to see this fixed. As well as being hopeful for finding a solution for current player/browser configurations. But testing results are varying and that's where I am asking you for help, by running 2 flash movies and posting the results to my comments.

Over the last days I've been frantically testing several configurations and options, I would hope force vsync with the flash screen buffer, amongst which the following :

• Adding a video at 30fps to the stage (no result)
• Adding system text to the display list (seems to work on windows xp)
• Several configurations of stage.invalidate() and Event.RENDER (no result)
• Different drawing / caching methods.
• Wmode="direct" for flash player embed options. (result across the board!)

The last option is the only thing which seems to work on all testing configurations, but comes at a cost. Switching to wmode="direct" seems to force vsyncing and this would adhere to what has been said about direct wmode:

"This mode tries to use the fastest path to screen, or direct path if you will. In most cases it will ignore whatever the browser would want to do to have things like overlapping HTML menus or such work. A typical use case for this mode is video playback. On Windows this mode is using DirectDraw or Direct3D on Vista, on OSX and Linux we are using OpenGL. Fidelity should not be affected when you use this mode." [Link to Tinics blog]

"Hardware offscreen buffer
Software renderer, perfect fidelity" [link to Jim Corbett's presentation (ppt)]

As opposed to the wmode="gpu" flag, which moves rasterization to the videocard (at a hight cost in most content cases, also altering the appearance of your content depending on the videocard), this option still uses the Flash Software Rasterizer to build up content for the screen. But on both my Mac and PC testing this comes at a performance cost of several frames per second, up to about 15fps on larger screens (1280x768). Also, I'm not sure about the results of this on machines with no gpu. (mostly, cheap on board graphic chipsets, which share system memory). While most of the time, your content type might vary in terms of results with either GPU enabled wmodes, in this test that shouldn't be an issue.

Obviously, abusing wmode="direct" to force vsyncing on the Flash Player isn't ideal, but it seems to work. In an effort to survey the results of this, I've built a very simple test swf. What I'd like you to do is run both swfs and copy the text field of both and paste them in my comments. Although I'm already in contact with Tinic about this, there's currently no JIRA bug describing the problem. With the results you have, I'd like to put up an official bug there, and use your results to describe what systems are impacted.

Firefox seems to be the main culprit. Since FF 3 there have been a number of big issues with Flash and Firefox; for instance, in some cases a right click on the flash player will cause the updated areas in the Flash movie to render as an opaque rectangle with the background color. In another case, Flash doesn't respond to rollover events and doesn't even render without a mouse click occuring.

Please help me and hopefully Adobe identify and fix this problem, by running these tests. Here's what you should do :

• Run both movies
• Look for tearing. Typically, in these test, instead of steady flickering, you'll see a moving vertical tear between the black and white area. It is essentiall you identify tearing correctly, so make sure you know what it looks like (picture above is a still). The effect of tearing shows even better on my mac, when you move the firefox window so that the flash movie intersects your screenborder, halfway splitting vertically through the flash movie).
• Copy and paste the textfield contents when the test is done, on both tests to my comments. Fill in the required 2 fields, and if you can, what type of graphics hardware your machine has.
• Send this link to other people who are willing to test (and be part of the solution, making Flash a better place!).

Test with wmode opaque

Test with wmode direct

[WARNING : if you are prone to visually induced epileptic fits, this might induce one. Don't run it then]

(don't forget to post to the comments).

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...

Over the last days I've spent some time twittering back and forth with Eugene. His supershape demo is very cool and we spent some time 'tweeting' about shaders. This lead me to upload and twitter one of my experiments which I used as part of my FOTB / MAX presentation. Judging by the huge number of retweets it seems people enjoyed it a lot, so I figured that it was worth a blog post. I started working on this experiment a while back after toying around with OpenGL, OpenFrameworks and audio reactive shapes and that inspired me to do something similar in Flash. This is an unfinished demo as it lead me to pursue something else, at which point I stopped working on it.

In the session where this came from, specifically this part, I shared information about the path of yet another demo, where this experiment led me too. Some of you who attended Flash on the Beach, might recall it from the Adobe keynote, where it was aptly named "milky ball".  I'll present on it one more time on the 3rd of December at the AUG XL event, in Amsterdam. I'll post more of the session's demos with explanation, when I have the time.

I do want to clarify that the code base I've created for this, which I currently call Triangle3D, is not to be a new Papervision3D, Away3D, Alternativa3D, Infinity3D, Sandy3D, Null3D or Yogurt3D (I'm probably missing a couple, sorry, have your way in the comments). It's my experimental playground of classes to play around, with no care for architecture restrictions or a generalized API. It's all about me playing and enjoying. Pure experimentation of what can be achieved with Flash 3D and hoping to find something beautiful in it.

Orignal unfinished demo

Demo with adjusted lightmaps from my new mirrorball.

(here's the shot of the mirrorball, on flickr).

The music track is Sixtyten by Boards of Canada.

Related posts:

1. Flash Player 10.1 pre-release on Nexus One Nexus One running Flash Player 10.1 from Ralph Hauwert...
2. Goodbye, Papervision. Last week at Flash on the Beach I had...

Last week at Flash on the Beach I had an announcement regarding me and Papervision3D. While some people took that as a hint to the release of PapervisionX, the actual news was a bit more sad. After pondering on this for a while, I made the rather big and tough decision to leave the Papervision3D team. As many of you couldn't be in the session, since the room was packed and as I heard later, many, many people not being able to come in from the line outside. Next to that many of you not able to come out to FOTB, this post is to ellaborate on my decision and explain it to all of you.

A great experience

I've been on the Papervision3D team for around 3 years in total. Only weeks after it's first closed alpha release, I joined the effort. All in all it was a great ride and I had an enormous amount of fun working on the project. Mind you, contrary to what some people believe, Papervision3D is not a company, there is no steady revenue and there is no direct income from it. It's an opensource project, with all the team members contributing out of fun, love and a feel for our great community. With the team, we have been able to deliver a solid, fun and easy to use 3D API for the Flash Platform. While Papervision3D certainly wasn't the first Flash 3D engine, many of it's features where and it's ease of use propelled the name to the top-ranking name in Flash 3D engines.  Along with that, Papervision3D helped putting Flash 3D solidly on the map for companies and individuals, making Flash 3D a tool accepted and available within the creative palette. I'm very proud to have been part of that and very grateful for all the valuable and nice feedback from the community. When I started on the project, it was only me, John Grden and founder Carlos Ulloa working on it. With time the team grew, changed composition and we also got more and more community input. These days, the team has grown to a rather large size, we got many blogs and sites dedicated to the engine and a vibrant community working with and on the engine. There's nothing to describe how proud I am of achieving that together with my fellow team members.

Building engines

Over time, I've spent more and more time in creating and architecting the core of 3D engines. With Papervision3D, this was the 1.7, 1.9 and 2.0 core of the engine. For PapervisionX I've taken that even further, creating the entire core architecture and implementing the many things I've learned over the years. But next to Papervision3D, I've also built multiple engines in my commercial effort as a freelancer, and have greatly enjoyed doing so. During my FOTB presentation, I showed a video of the Anne Frank Virtual House, for which I was commissioned to build the 3D engine by LBi / Lost Boys and the Anne Frank Foundation. For the project I worked closely together with Floris Drupsteen of Biqini, the 3D modeller on this project, who did an awesome job of building the actual 3D models. It was an all in all great experience to build an engine on that scale and I'm proud to have been part of that effort. Considering the goal of the effort, it was a very meaningful implementation of a 3D engine to me, and an honor to be part of that effort.

With me stopping to work on the Papervision3D project, this is certainly not the end of me building 3D engines. Rather a rehash on how and why I do that. First and foremost, I will keep in chipping in the community effort, through this blog and upcoming projects to be released in the opensource domain. Also, commercially, I see myself building graphical / 3D engines, when possible, for a while to come too. All in all, I just love building the graphical engines for projects, hence also my self-given titel; "visual programmer". With no noticable big difference in what I do, there is a question remaining.

Why ?

There's a couple of reasons to the exact why of me leaving. First and foremost, this is a personal choice on my side, has nothing to do with my feelings towards the engine or the other team members. An important factor for me, things have to do with momentum. The engine has had an incredible amount of momentum over the years, and that pushes me to work on such a project even harder. Currently the project is a bit lower on momentum, which doesn't help me either. I think it's important to know I've been working on the engine many, many hours over the last years, and at some point you get to some fatigue of working on the same thing for a while. With the success of the engine and the personal success of my fellow team members, all of us have gotten more busy on the commercial side of things. This means less time goes into the project, which in turns into less momentum for the entire project, which in turn leaves me less "pushed" to do more. Considering the enormous amount of pressure from people on the outside, this wasn't good for me either. On the other side, this change in team composition, with me leaving, gives room for other people to full-fill that position and get new ideas, new work and new momentum going. All in all, I think it will do the engine good.

Next to momentum, another important reason for me was the goal of building an opensource 3D engine. With many people doing awesome experiments and examples / implementations of the engine, it really gives me a natural high to see how it's beeing used. But next to that I'm also looking to create beautiful things myself. Now, I'm not anything near a real visual designer, I like to see myself as one who builds the tools and technology to help enable these experiences. But I want to be part of something which will lead to beauty in visuals, story telling and emotion of the experience. Without a doubt many Papervision3D powered experiences can be classified as beautiful experiences, but most of the time, just building the engine means that they use the technology, but you are not directly involved in that process of creation. I really feel I need to work on projects which will give me a more direct link to the artists, or even more closely enabling artists. One of the projects I have recently joined is the Rhonda Forever project. Certainly very different as a tool to people, my job within the team is very similar to what I did for Papervision3D. For Rhonda I'm building the online drawing viewer for the project. The change for me there is that I will be working directly with it's creative community and seeing results of that effort in a more tangible/graphical way.

Wanting to create visual beauty also means that for me, there was a decisive direction to take. The work you do is the work that defines you within the industry and that in turn gets you the next big job. With me wanting to do more directly involved work on creating things with beauty, but my work mostly being hidden in technical solutions like for instance Papervision3D, I wanted to expand my working area to more visual beauty and become more part of that side of the process. The only way to do that is by just start doing it, finding people to work with and changing that direction. This one of my first steps towards that. With a renewed sense of freedom, I'm hoping to work on projects more closely and be part of the actual implementation more, rather then being part of a tool that enables people to do that, but not being part of the creation of the end result.

Community wise; I'm still there. I am pondering on some of the directions. Along with doing more work with and on OpenFrameworks and Rhonda, I will still build 3D things for the Flash Platform. What I can release, will be released through this blog. With the 3D Flash community having considerably grown from near to nothing to an almost distinct part of the industry, I will be part of it. Also, I'm looking forward to build technology which spans and extends all the available 3D engines, rather then just focusing on one engine.

PapervisionX

One thing I'm not entirely sure of as of yet, is what to do with the PapervisionX code base. Built entirely from scratch by me and Tim Knip, this project really has part of my soul in it. It's been exclusively me and Tim working on it, with plans to have the rest of the team start on it a while back. Currently, that hasn't happened yet and the engine is not ready for release and officially not part of the Papervision3D code base. In other words; it's mine and Tim's. While I'd love to release it at some point in time, currently it's not ready for that and with me leaving the team, it's most likely not going to be released under that name, if released at all. I might decide together with Tim to donate it to the Papervision3D project, but currently I'm not sure on that decision yet; as there would have to be a considerable effort to make it stable for release. As it is unfinished now and publishing source code is somewhat a personal thing when most of it is yours, I'd think this is not what should be done right now. As I am going to pursue my own goals for the time being, this decision might take a while.

I will discuss with the team and Tim closely to see what can be done in the future, but for now, I don't think it should be released. It might also be better for the team to work on it's own Flash 10 compatible version of the engine.

Triangle3D

Another engine I showed at FOTB was my personal experimental 3D engine Triangle3D. Not at all meant to compete with any of the 3D engine, rather to be my own basis for experimenting with Flash and 3D, this engine might be released through this blog in the nearer future. Again, this is not me splitting off from the team and doing my own thing, this is more or less just giving the community my personal playground for Flash 3D. The engine is far from feature complete, and honestly, I don't want to focus on that. Rather just build it as I go and need something. This engine will most likely form the basis for my commercial 3D efforts too, as a custom based engine is almost always going to be more focussed on the specific implementation then a more generalized 3D engine API.

Thanks

All in all, in regards of the Papervision3D project, I would like to thank my fellow team members for all the good times. I would like to thank the community for their beautiful implementations and experiments as well as feedback. You make it all happen! Lastly, as a special member of the team, I would like to thank founder Carlos Ulloa for having me on board in the first place. It's been a great ride, and I'm sure we'll be riding our race cars for quite some time, even if we have to build them ourselves.

Related posts:

1. Flash Player 10.1 pre-release on Nexus One Nexus One running Flash Player 10.1 from Ralph Hauwert...

After the overwhelming response of posting the sources of 300.000 pixels with Alchemy and Pixelbender, I thought it would be nice to post some more from my session at FITC Amsterdam.

First let's take a look at what people have done in response of the previous sources.

Yonatan Offek - Sierpinski Particles

Peter Nitsch - Alchemy Particles

David Lenaerts - Smoke, Milk and Ink

Dennis Ippel - Incredibly fast Plasma

Joa Ebert - Massive amounts of 3D particles without Alchemy and Pixelbender

All the work in that list is incredible, but for this post, I'm most interested in the last 2. Joa's work on getting AS3 up to speed with my Alchemy demo (Joa's demo is actually faster then the Alchemy version on my machine) is incredible. And it also proves the one thing I tried to convey in my blog post Adobe Alchemy, is it ActionScript Heresy ?.

Joa's optimization relies mostly on a linked list implementation to iterate over the particles, and using the Flash 10 Vector type to allow for faster access to write to screen. Another speed improvement over the Alchemy version is not using Pixelbender, but having both rotation and projection done in the same loop as drawing to screen. All in all, his result on this is pretty awesome.

Again, writing highly optimized ActionScript should enable you to do the same speedwise as Alchemy compiled code. There's at least one definitive advantage which Alchemy has in that case. The first advantage in Alchemy lies is the fast ByteArray access (which currently only Alchemy and Haxe can use).

The second advantage is why I think we should have a better AS3 compiler. It is the use of a more advanced compiler. This reduces the need to *highly* optimize your code, although I'd always recommend doing that in any case. My demo's here are meant of examples on how to use Alchemy, rather then to get the last bit of performance out. That, we'll leave to PapervisionX work.

Now, as I said, for this post we would be looking at two of the responses on my earlier Alchemy demo, the second one being Dennis Ippel's Fast Plasma effects. Plasma effects and Lookup Table efffects aren't too different. Ricardo Cabello did some experiments with them earlier, using Pixelbender. I worked on taking this type of effect and implementing it using Alchemy. Since that I had already done some experiments with them earlier, I thought it would be nice to release the sources, and see what you guys do with them. Maybe Joa feels like another shot of doing the same with ActionScript ?

Example

Source

Play around, let me know what you do with it, and make sure you check out Mr. Quilez his webpage, specifically, the article on Plane Deformations, where part of this code is derived from. If you want to optimize more, you *could* read Kebby's article F***ing learn to code again.

One more observation  : the difference between running this on Firefox on a Mac and just running it in the FlashPlayer is huge, even bigger then normal. Could someone from Adobe finally explain this ?

No related posts.

Pushing around +300.000 3D particles, realtime, on screen, using Flash ? No problem, if you are using Adobe Alchemy & PixelBender to compile and run your code!

During my session "professionally pushing pixels" at FITC Amsterdam this year, amongst other things, I talked about how to best utilize parts of the Flash Player to get top performance. This is one of the examples I showed. What you are seeing in this example, is +300.000 particles being 3D transformed, projected and draw to 2D. And it does so at quite a good framerate (well, it'll depend on your machine too).

So, how do we achieve this ? The answer is a combination of PixelBender and Alchemy.

First, let's look at the 3D transformation and projection.

Flash 10 has a number of native features to allow for 3D transformation and projection. You'll find that this is a combination of using the Vector, Vector3D, Matrix3D, PerspectiveProjection, etc. Although these features are great, we can't use them in combination with Alchemy easily. I'll explain why later, for now, let's look at an alternative method to do the projection.

Where oh where in the Flash Player do we have a method of doing very fast math ? The answer is; pixelbender! Although Pixelbender is normally used for image based manipulation, you can make it do any type of number-crunching which is able to be executed in parallel and without loops.

To calculate rotations and projecting our 3D data, we use Pixelbender in "ShaderJob" mode. When using pixelbender in image based mode, it operates in 8 bits per channel. Thankfully, when using it with a ShaderJob, it allows 32 bits precision per channel for the data processing. Since 8 bit precision wouldn't be enough for this example, we use a shaderjob.

The VertexProjector pixelbender kernel, included with the source is a simple way of transforming and projecting vertices (representing particles, in this case) in 3D space. We feed this kernel a bytearray of x,y,z paired data, and execute the shaderjob. It then returns the data as a bytearray, in px, py, pz format.

Drawing things to screen.

Now we have all the 2D projected 3D data, we need to draw things to screen, and we have to do so as quickly as possible. This step is traditionally called rasterization. In AS3, you're most likely to use getPixel when drawing on a per pixel basis. Doing so in a loop for 300.000 pixels turns out to be very slow. The solution for this would be to optimize that loop as much as possible. Either by writing your own bytecode, or maybe writing your own post-processor for you code, before you compile. But we don't have too, since Adobe Alchemy exists.

As you can read in my earlier post about Adobe Alchemy, I openly questioned why it was so speedy, as compared to regularly compiled ActionScript 3 code. Although the answer is rather complex, the combination of C Based code, the LLVM compiler and "Alchemy Virtual Memory" are the base of this. The large difference between Alchemy compiled actionscript and regular compiled Actionscript can be further explained by the regular AS3 compiler not doing any optimisation. This example shows off those performance increases.

One thing to worry about when using Alchemy in your ActionScript projects is marshalling. You can read Branden Hall's post on Alchemy for more info on that. Since we wouldn't be able to marshal 300.000 vertices from a Vector.<Number> in AS3 to our alchemy code, we need to find a better solution. This is exactly why we are using Pixelbender and more-over, the bytearray data.

It is possible to manipulate the memory Alchemy uses in the runtime. This memory is represented as an AS3 ByteArray object. If we directly write and get our data from this memory block, no marshalling is needed. Although this means not all things can be done this way, for some things, this can be very useful. For instance, getting large blocks of data, like images and bytearrays of coordinates.

Getting all these 3D particles to screen is simply 1 inner loop. While we would normally call setPixel for that, in Alchemy code, we don't have that luxury. Instead of that, we write directly to our screenbuffer memory, which is represented as a set of int's. Here, one more problem comes into play. Endianess, defines the byte ordering for a set of data. Alchemy uses little-endianess for it's internal memory representation. Specificall, it uses a small class called LEByteArray. This class extends ByteArray and ensures no changes are made to the endianess of the memory. Makes sense, since otherwise your code would blow up.

Writing to the screen is then a piece of cake. We take the alchemy processed data from it's memory, and write it to a bitmapdata using the formerly much less usable setPixels() command. It's amazing to see how fast this is.

Look at the example here, and download the full sourcecode here. As you can see from the example, the difference between doing this with regular ActionScript versus Alchemy nears a 5 fold speed increase.

Thanks to Keith Peters, for providing me with the 3D Strange Attractor code! And additional thanks to Mr.Doob for the stats object.

In future I'll be posting more demos of the technology. Amongst which there will be one appliance for the future version of Papervision3D, PapervisionX.

No related posts.

It's been silent here. Next to tripping to Tokyo to speak at Adobe MAX in Januari and doing some small jobs, I'm doing a very, very cool Flash 10 Project utilizing the upcoming Papervision for Flash 10. For some optimization parts I've been looking into using Alchemy as an easy way to write optimized bytecode. As opposed to what some people took from my previous post on Alchemy, I don't hate Alchemy. My point there was that if Alchemy can perform that well, so should ActionScript be able to, with the standard compiler.

Now, that being said, all work and no play makes jack a dull boy. With no time to spare during the morning, daytime and evening, I took the laptop with me to bed last night, and decided to do something I didn't do before, port something with Adobe Alchemy. I decided upon a long time thing on my wish list; SID Playback in Flash. What is SID ? Well, SID stands for Sound Interface Device, referring to the MOS6581 and it's a programmable audiochip as used in the Commodore 64 and Commodore 128. I grew up with the Commodore 64, and the audio it produces is still something I hold dearly. You are quite likely to have heard a SID chip before, if you listen to the radio, as it is quite a possible you've heard one of these baby's scream. Good recent examples are :

Bastian - You've got my love. (the beat in this song comes from Jeroen Tel's Rubicon).

Nelly Furtado - Do It (controversy all around, Timbaland blatantly stole Glenn Rune Gallefoss's version of Acid Jazzed Evening).

De Jeugd van Tegenwoordig - Hollereer (Produced by Bastian, C64 lover himself).

My interest was more playing back the old C64 tunes from games and demos. Keep in mind, an average SID file is about 2kilobyte to 10kilobyte; a perfect Small Web Format for all your music during preloading needs . Now, this is something I have been working on before in Flash 9, using the FC64 project, the amazing Flash C64 emulation effort by Claus Wahlers and Darron Schall. Last year, I even made a start with porting FC64 to Flash 10, but that needed to be dropped in favor of actual client work. SID Playback is something different from playing back .mod .xm or .mid files, as the SID chip doesn't do much on it's own. Essentially, a .SID file is an executable meant to be ran on the 6510, the Commodore 64's CPU. So to playback the file, you not only need to emulate the SID chip, but also the 6510 and the needed registers / memory.

Software emulation of the SID chip has been around for a while, and all modern platforms have a way of playback SID files. Winamp plugins, command line utilities for conversion, it's all there. As is there opensource code to do this. There's libraries for most major languages. So, after a bit of research, I decided upon the Linux version of TinySid. This library is small enough and well optimized. It looked simple to port, so I went at it. All in all, 4 hours later, I had this baby up and running! (And an unhappy girlfriend who isn't into the nerd thing, nor is she into listening to chiptunes whilst trying to sleep). Considering my very rusty C and a little hack around I had to do for the audio output, Alchemy shows off it's feathers here. Many kudo's to the Alchemy and Flash Player team for this effort. Just now I spent half an hour building a tiny interface using Keith Peter's Minimal Components, and collected some songs; and here it is.

TinySidFlash (open link, requires Flash 10).

My favorites : Politik and Science (A C64 version of Coldplay's the Scientist), Morphing and Turbo. Yes, I love Dane's tunes!

There's no code yet, as this is still very rough and unfinished work. I'm pretty sure that when I get time again, I'll have a go at LibSidPlay instead, since the overall emulation in that library is a lot better.

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...
2. Help me test! Flash VSync and Screen Tearing. So, here's a post where I am asking you,...
3. Flash Player 10.1 pre-release on Nexus One Nexus One running Flash Player 10.1 from Ralph Hauwert...

Take your time, this is a lengthy post. Before I jump in the matter, I want to go back a bit.

The magic of C64 Assembler

One of my first experiences with computer programming was on the C64. My dad bought my uncles C64...... Of course we played games at first, but at some point I started to look at all the books that my uncle had gaven with it. One of the first books I opened up and started playing with was called "Basic for Beginners". I remember thinking, wow, so this is how you make games ? I started playing with existing programs and later on began writing my own, such brilliant programs as "My cassette collection" and "Adding Numbers". But what I really wanted was to write my own games.

How come my C64 basic programs where always so slow ? Surely, there must be something I was doing wrong. With nobody around to teach me, or tell me anything about this machine, I dived in the pile of books that came with the computer once again. Two books took my interest. "C64 Programmers Reference Guide" and a home printed & binded book that had written on it "C64 Kernel listing". I had two problems with these, being 1, my ability to read english and understanding the magic listings this book had. It took me a long, long time to grasp what these books where saying, but these 2 books, together with a dutch book on C64 programming, I slowly learned how to do 6510 assembly language for the C64.

This was the realm of pure magic. All of a sudden things went a lot faster. Moving a sprite along the screen could be done fast and smooth. But things where a lot more complicated. 6510 (or actually 6502, for the purists) asm, although especially now a days, quite simple, isn't exactly humanly readable, nor was it clear to me how to structure full programs or games. I learned how to do a lot of tricks, like getting more then the 8 hardware supported sprites on screen at the same time (something then called multiplexing), playing sound while still having being able to do other things at the same time, getting sine waving color bars on screen and screen border, but I never managed to build a full game. What I learned about that was that interpreted basic was a lot slower then assembler and if you wanted to get real speed you'd have to target the platforms native code. Assembler wasn't magic, it was just a better way to target the platform.

Back to current day.
I've been working with Flash for almost 9 years now. I dare to say I know quite a lot of Flash as a platform, and have been doing many projects and experimentation with it. Last year at MAX, when FlaCC or now, Alchemy was shown it looked like pure magic! Sure, I understood the basic principles, but running Quake in the Flash VM was nothing short of magic. Running Quake within the Flash VM using a port of the C code was quite an achievement, but the speed was more incredible. This was nothing short of pure magic, like turning lead into gold!

It's an ironic name Adobe chose for FlaCC. Let's look at the word and dabble into the occult. There's a clear distinction between alchemy and science. Alchemy is most known for trying to create noble metals from otherwise less-valuable materials. Even Sir. Isaac Newton spent a lot of time in the field, surprisingly, even more time then he spent working on subjects such as optics and physics, for which he is most known. In a way, Alchemy is the forerunner for what now is chemistry. From the occult to exact sciences. So, with that name, is Adobe suggesting Alchemy is pure magic ? Turning lead into gold ?

Is it witchcraft ?
After the initial amazement I had with seeing Quake run within Flash, I started to question some things and started asking around with people who might know more of this witchcraft. Was this really Flash Player 9 running this ? From multiple sources I heard it was a slightly modified version of the Flash Player 9. Something to do with reserving a block of memory. Later on, after finding Scott Petersen his LLVM session on this subject, this was confirmed. In this session, Scott also shows an emulator running on top of the Flash VM. Having some experience in the FC64 (a commodore 64 emulator for flash) project, again, the speed was pure magic.

What I ended up finding most interesting about Alchemy, wasn't the ability to take C code and compile it to Flash (although that's an incredible achievement on itself), but the pure raw speed it seemed to harness. And with that, I also got a bit disappointed in Flash. It boils down to this.

The first magic bullet in Flash speed came in the form of the ActionScript Virtual Machine 2. With JIT compilation of ActionScriptByte code the platform just became a whole lot faster. With the coming of the AVM2, lot's of things have changed. For me personally, one of the displays of the game changer was Papervision3D, in which I got involved quite early on. John Grden's early demo using Papervision3D running on AS3 was an incredible testament to the speed of the AVM2. Now, that demo was built using ActionScript 3 as the language to be compiled down to ABC. ActionScript 3 is the language which was intended to run on top of this platform and one might thus derive that the language and it's compiler is most optimized for the AVM2. So logically, one might asses that the best way to write code for the AVM2 is using ActionScript.

Heresy I say! Heresy! Burn the witch!
Alchemy is proof that this is not the case. Here's a bit of pure heresy in the Adobe church : Alchemy is a testament to the underachievement of the ActionScript compiler. Ouch. Surely, this can't be the case, right ? Well, yes and no.

Let's start with the "no" part. That reference of a modified player running Quake was partially true. This was an early demo of Flash 10, in a way. As Nicolas Canasse points out in this blog post about Alchemy virtual memory opcodes, it seems like Alchemy is cheating a bit. Cheating ? While, yes. This part of the API is not available from within ActionScript itself. So, the platform (AVM2) now has features which are not supported by the language which is native to it ? Wow. I find this a very significant move by Adobe. Moving to a more flexible compiler makes total sense, but is this the first symptom caused by the EcmaScript4 draft debacle ?

To achieve the speed as displayed by Alchemy, I have thus have to move to using C code, or HaXe to compile for the AVM ? This is odd, to say the least. Nicolas his work on HaXe is nothing less then brilliant. The features this compiler has are those which would greatly help the development of a 3D engine, such as Papervision3D. I've always had a great deal of interest of porting Papervision3D to HaXe, purely to be able to implement things like inlining easily, which will safe us function calls and thus speed up the process of rendering 3D to the screen. The downside to that is, that the code base would have to change to HaXe, and thus become incompatible with the compilers of Flash and Flex. We (Papervision3D) could provide with a SWC built with HaXe, but in a way, I'm still a bit weary of that.

Now we come to the yes part, of my earlier "yes and no". Next to the virtual memory opcodes that are currently unsupported from within the AS3 language, Branden Hall states something very disturbing on his blog about Alchemy, "Understanding Adobe Alchemy".

"In fact, because LLVM does a lot of optimizations, whereas Flash and Flex have no optimization step built into their compilers, Alchemy code manages to overcome a lot of it is overhead just on its own."

Ok, that is down right disturbing. The AVM2 was built with ActionScript in mind and visa versa. But compiling C using LLVM with ActionScript as a target produces better faster ActionScript (that's actionscript, not bytecode) then the actual compiler built for the platform does ? Ouch. Now, moving to C development for the ActionScript platform to utilize this "magically" acquired speed seems somewhat illogical to me ? Joa Ebert already showed us some work he had done on the compiler, allowing for inline bytecode to be used. Nicolas shows us he can harnass the new Virtual Memory Opcodes, and his work on HaXe also shows us that inlining is very, very nice.

I'd love someone from adobe shedding some light on this issue. I love the Flash Platform, I love the work all the people at Adobe do, but this leaves me somewhat confused on how to develop for the platform I hold so dearly. If we have to let go of ActionScript to harness the pure speed Alchemy proofs is possible, then tell us. What is the future of our beloved language and our compilers for it ?

Related posts:

1. Dear Steve, Long time no see! I've been busy, sorry. First...

A smooth shaded elephant.

Behind the scenes I've been insanely busy for the last months. But just to keep you posted on some progess on my side, here's some demos of me playing around with Flash Player 10's 3D API's.

I'll keep this short, here's some demos.

Some tests with shading : 1, 2, 3, 4, 5, 6

You'll notice that the speed slows down when you go up in the number of examples. The amount of triangles increase per example. Since shading is still one of the most heavy appliances of any 3D engine, I thought it would be good to start out testing that, and see how it would work in a realistic 3D engine environment.

Next, this is not adviced to look at if you get car-sick easily.....a pure speed test.

A bit slower then it could run using any of the new wmodes, but for speeds sake, we're getting there.

Source code not available yet. First I'm looking at how to optimize. And yes, I'm working hard on a new version of Papervision3D, with the rest of the team. Soon I'll post some better examples, including a BSP example.

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...
2. Help me test! Flash VSync and Screen Tearing. So, here's a post where I am asking you,...
3. Goodbye, Papervision. Last week at Flash on the Beach I had...

I just returned from teaching the Papervision3D workshop in Sweden; it was an awesome group and good fun! For all those who attended : Thank you very much for the good and fun workshop, you where an awesome group! Also many thanks for Johan and Marcus from the local usergroup for helping out with the event!

This week Saturday I'll be heading off to Chicago to be part of the FITC Chicago event. I'm looking forward to doing my Flash 2D & 3D effects session as well as a 1 day workshop on Papervision3D. If you haven't registered yet, make sure you do so as there are still some tickets left, but they are going fast!

Register here for the RMI Papervision3D 2.0 workshop Chicago.

Sorry for the silence on this blog lately, personal matters take priority and as such it will be more quiet here over the coming months. But expect a nice blog on some exiting news here tomorrow. Looking forward to see any of you attending FITC Chicago.

And then some more exiting news; FITC Seoul is happening and I am lucky enough to be invited to come over and speak amongst these great names ; Joshua Davis, Erik Natzke, Kyle Cooper, Richard Galvan, Marcos Weskamp, and Euna Seoul. With such an awesome line up, I'm sure this will surely be a success and a opportunity for us to reach out to the korean / asian community! Looking forward to see any of you there!

No related posts.

Intro

As promised in my previous post, here's a small opensource project from my side. It's tiny, really, but I hadn't seen an ActionScript implementation of any form of Image dithering before.

Dithering....what ?

Just a brief explanation what dithering in this context means. From wikipedia :

"Dither is an intentionally applied form of noise, used to randomize quantization error, thereby preventing large-scale patterns such as contouring that are more objectionable than uncorrelated noise."

Words to take note of in that sentence are quantization, and "intentionally applied form of noise". In our case quantization is the removal or swapping of colors. The form of noise applied differs over the several implementations and in so called "ordered dithering" it's hardly noise, rather a carefully chosen threshold matrix.

Dithering, why ?

Dithering imagery has been around for some time. Put a close eye to any black and white printed newspaper's pictures and you'll see the effect.

It's basically been around a lot longer then I have in my 28 years of life...that's for sure. I first learned terms like "ordered dithering" and "Floyd-Steinberg Error Diffusion" in my young and early days on the Amiga hardware. See, in those days computer hardware wasn't capable of displaying the huge arrays of colours like now-a-days. Since you then had few colours to spare (a typical amiga workbench ran at 16 to 128 colours or so), you needed to be creative to get yourself a nicely pimped desktop image, whilst still sparing colours for your icons.

A better example of a dithering implementation for our industry is probably that checkbox when you want to save for web in Photoshop, using GIF. Or when you print something on a pure black and white printer.

Conclusion; dithering is normally used to create the illusion of tones on a device which is otherwise not capable of displaying it. So why port this to our ARGB/32Bit enabled Flash Player ? Part of the reason why I worked on this is because I just like the aesthetic of the effect. It just brings back wonderful memories of pimping my Amiga desktop.

Dithering in AS3

So, the algorithms for dithering are really quite simple. I was playing around with hydra Pixel Bender and had some ideas on converting some old algorithms to have them run in realtime. Then I thought I could make this run in realtime on Flash 9. So, first I did a version of the Algo in pure AS3. I played around with converting it to something fast enough to run at at least 20 frames per second on an average machine...and painfully failed. The remains are the AS3 version (hydra Pixel Bender version still in progress)....and I decided to clean that up a bit, and have it released as an OS project for anyone to use.

Currently it only contains so-called error-diffusion based ditherers. They make the most sense anyway, since the ordered ditherers really have nasty visual sideeffects, like Bayer's crosshatches. Check the variants out in this little demo application using the class.

The Demo : try and use "No Dithering" first to see the effects of regular palette conversion.

The Source

Usage :

ImageDithering.dither(bitmapData, type, levels, grayscale);

Where :

BitmapData is the image to be manipulated.

Type is the form of dithering, currently supported :

1. ImageDitheringType.FLOYD_STEINBERG
2. ImageDitheringType.FALSE_FLOYD_STEINBERG
3. ImageDitheringType.STUCK
4. ImageDitheringType.NO_DITHER

Levels is the amount of colour levels to quantize to per channel.

Grayscale is a boolean indicating whether to convert the image to grayscale before the process is ran.

Right click and viewsource on the example to see how it's implemented in Flex.

Quick update : Mario Klingemann, working on Aviary's Peacock did a quick test with it. My effort had some use anyway!

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...

Yes! April is coming up quickly and that means amongst others 1 awesome event, FITC Toronto. I'm excited to be speaking at this event alongside other speakers like GMUNK, Joa Ebert, Andre Michelle, Erik Natzke, Patrick Matte, Ivan Todorov, James Paterson, Grant Skinner, Joshua Davis, Thijs Thiemstra, Keith Peters, Evan Roth, Niko Stumpo, Mario Klingemann and many more!

I'm really looking forward to do my 3D and 2D effects session and hope to see you there!
Also, I will be giving a one day workshop on Papervision3D, in front of the event; as one of the dev's of the API I can assure this is will be one day of great fun, knowledge and learnings! Make sure you register in time, these things tend to sell out fast.

FITC was kind enought to pass me a special 10% discount code for the event in Toronto; using the code unitzeroone you will be given that discount.

You're not sure yet ? Well, check out these videos on FITC Amsterdam and be convinced!

Related posts:

1. Goodbye, Papervision. Last week at Flash on the Beach I had...

You always have projects lying around that are to be finished...but probably never have the time for...

Today Exey Panteleev posted some creative use of Papervision3D and the Shaders in 2.0. Accompanying to his post he asked if it would be possible to run water simulation with Papervision3D....

Actually, yes...
Water Ball Experiment 1 Water Ball 2(cpu burner alert!)

As one of those things I never finished and probably don't have time to finish within the next month or so, I have two examples of Papervision3D running interactive water on top of a sphere.

Disclaimer : these things are hacked together experiments from a couple of months ago...as a result performance isn't to good. I'm sure that given some time they can run smooth, without burning your cpu to the ground. Hope you enjoy anyway

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...
2. Goodbye, Papervision. Last week at Flash on the Beach I had...
3. Help me test! Flash VSync and Screen Tearing. So, here's a post where I am asking you,...