(people who want to know more would refresh after seeing the pieces in full)

No related posts.

Over the last days I've spent some time twittering back and forth with Eugene. His supershape demo is very cool and we spent some time 'tweeting' about shaders. This lead me to upload and twitter one of my experiments which I used as part of my FOTB / MAX presentation. Judging by the huge number of retweets it seems people enjoyed it a lot, so I figured that it was worth a blog post. I started working on this experiment a while back after toying around with OpenGL, OpenFrameworks and audio reactive shapes and that inspired me to do something similar in Flash. This is an unfinished demo as it lead me to pursue something else, at which point I stopped working on it.

In the session where this came from, specifically this part, I shared information about the path of yet another demo, where this experiment led me too. Some of you who attended Flash on the Beach, might recall it from the Adobe keynote, where it was aptly named "milky ball".  I'll present on it one more time on the 3rd of December at the AUG XL event, in Amsterdam. I'll post more of the session's demos with explanation, when I have the time.

I do want to clarify that the code base I've created for this, which I currently call Triangle3D, is not to be a new Papervision3D, Away3D, Alternativa3D, Infinity3D, Sandy3D, Null3D or Yogurt3D (I'm probably missing a couple, sorry, have your way in the comments). It's my experimental playground of classes to play around, with no care for architecture restrictions or a generalized API. It's all about me playing and enjoying. Pure experimentation of what can be achieved with Flash 3D and hoping to find something beautiful in it.

Orignal unfinished demo

Demo with adjusted lightmaps from my new mirrorball.

(here's the shot of the mirrorball, on flickr).

The music track is Sixtyten by Boards of Canada.

Related posts:

1. Flash Player 10.1 pre-release on Nexus One Nexus One running Flash Player 10.1 from Ralph Hauwert...
2. Goodbye, Papervision. Last week at Flash on the Beach I had...

My favorite Adobe User Group does it again! And how! At the threshold of end of the year, on the 3rd December, the dutch AUG in co-operation with Adobe presents to you a full XL event. With the popularity of the normal meetings rising and rising and these event being of an exceptional level for any type of usergroup, this time, the lid is fully blown off!

The XL event will bring some great national and international speakers to the stage; Dave August of OOOii will talk about his use of Flash in JJ Abram's Star Trek, as well as his creative input to movies such as Fast & Furious. Tobias Mannewitz of Karakter will be there and with work on projects for Spellbound, EA Games and SCEE this promises to be a session not to miss! Anita and Geoff from Champagne Valentine will amongst other great things talk about an interactive music video they created for Placebo.  Adobe will give you insights in their products and tooling with such international renowned speakers / evangelists as Serge Jespers, Jason Levine and Paul Burnett. If that's not enough, Niko Stumpo of Hanazuki will be talking about the studio's new work together with Mark Barzinski of Barzinski and Jean Jean. Dirk Groten of Layar fame will be talking about the Layar platform, something I'm looking forward to find out more about. Last but hopefully not least, I was honored to get an invitation to speak, and I will be talking about my latest forays into realtime flash.

This lineup is already stellar, with more names to come and be added to the list! Combine that with the friendly atmosphere and a great place to extend your network, learn and enjoy, this is an event not to be missed. Even if this is run by the dutch usergroup, I wouldn't be surprised if people come in from all over, as this XL event truly is XL. Be there! And if you do, make sure you say hi!

Get more information at the XL webpage, follow the AUG on twitter, and order the insanely low priced tickets here.

No related posts.

With the addition of the 2.5d API in Flash 10, lot's of developers now have even easier access to basic 2.5D environments. But with those new capabilities and API's new challenges for developers arise. Famously, Flash 10 doesn't do the Z-Sorting natively. When Lee Brimelow asked me if I could send him an easy solution for that, I sent him the SimpleZSorter. Which he in turn used to paste my profile picture on as much as possible planes, but more significantly, use for this tutorial.

Althought the name Simple would imply something a bit away from a PerfectZSorter (which, technically, has some challenges considering the 2.5D API and the player performance, don't ask), the thing which astounded me was the amount of feedback. Sure, more people had asked me for such a solution and this simple little tool did it for them most of the time.

But considering the simplicity of the SimpleZSorter and the amount of feedback I got on it (even if I didn't publish on it myself) made clear that although the 2.5D API of Flash 10 opened up loads of opportunities for developers and designers a like, some of the simpler issues aren't addressed by it, whist there is clearly a need for it. That need can be addressed and with this post I'm hoping to do a another small part.

The ArcBall

One of the questions which people have been asking me about for in relation to using the 2.5D API, is (loosely combined) "How do I rotate things in 3D correctly with the mouse".As this question came more and more, I've also realized how fundamental this is to our community. With Flash 10 we have native realtime (2.5D) 3D, but with realtime comes interactivity. And what use is it for users to have a 3D interface, while the controls are not intuitive ? At that point 3D becomes a worse experience, rather then a better. While this solution won't address every issue with that, it most certainly addresses one, 3D rotation dragging.

Native 3D interaction might be new to Flash, but it certainly isn't new to Flash 3D, or 3D in general for that matter. And the problem of having a user rotate an object with a mouse has been addressed before in numerous different ways. So luckily the wheel doesn't need any inventing. The wheel I chose for to address this problem is called an ArcBall, first proposed by Ken Shoemake.  Now if you want to read that paper go ahead, but let me define this in flashers layman terms.

The Problem

Let's first have a look at the problem as I've seen examples of code of people and derived it to this "problem" example. Drag the cube to rotate it.

Although the mouse dragging does offer some feeling of control, you'll find it difficult to rotate it in a natural intuitive way, to let's say to the front of one of those colored planes. Here's what's happening here in that code, and what I've been seeing most.

if(isDragging){
boxSprite.rotationX = mouseY;
boxSprite.rotationY = mouseX;
}

This is example is the most common and is responsible for the example above. Now, the problem here is that it doesn't take into account the local object transformation, as well as the dragging coordinates. As the object rotates around it's axis, so do it's axis change. Also, as the dragging goes on, we don't take into account the 2D coordinate versus it's 3D transformation dragging start and final output. All in all it ends up being a bit messy.

The Solution

Here's the solution using the ArcBall class. Drag to rotate.

The solution, the complex part

As you can see / feel, object rotations now feel much more direct and natural. But what is this magic ? First off all, the class is fully Flash 10, and uses it's native methods and features, where available for this solution. To explain it as simple as possible, as you click and start dragging on the cube, a sphere is established. This virtual sphere, around the object is then used to establish a point in 3D space on the sphere. As you are dragging, your actually rotating other points on the sphere. From these angular differences on the sphere, we can derive a new transform matrix for the object.

The hard part is in the establishing the rotational angles from those 2 coordinates on a sphere. A well proven way to do that (and the only form I have seen in ArcBall implementations, so far), is using Quaternions (go ahead, read it now, all of it, I dare you). Quaternions are complex things (pun intended). Now Quaternions are a bit daunting to start at and at this point I would be lying if I said I fully understand them. I'm not the only one who finds them scary, quoting Keith Peters / Bit-101 ; "Oh no. Vector3D’s w is the second most scary w in the world." (I wonder what the first is, Keith ?).

Quaternions and Flash 10

If you are looking for the clearcut solution and implementation of my class for it, skip this part, otherwise, read on, enjoy.

The reference to the w in that quote is also a reference to the complex component real component of the quaternion, where the xyz are complex numbers. Yes, complex numbers (I dare you again) and their appliance is where stuff does get scary for most of us. The great thing about the Flash 10 3D and geom API is that there is a partial native support for quaternions, next to having a vector with x,y,z and w components. For instance, this:

matrix3D.decompose(OrientationStyle.QUATERNION);

Which will return a Vector of 3 Vector3D's, which are in order translation, rotation, and scale. When using the quaternion orientation style, the rotation will be filled with a Vector3D with it's x,y,z,w values nicely filled; a quaternion. While this is something we could have handled ourselves (decomposing a matrix to it's components using quaternions), it's great Flash 10 has it. Less code, less size and it's faster then doing it with your own defined functions.

This ArcBall implementation relies heavily on that piece of code and some added code to do some additional stuff with the quaternions (multiply doesn't seem to be available native). Now while I'm not enough of an authority to try and explain quaternions, the applications for them in 3D rotation are numerous, most famously, the first thing which will get mentioned when talking about the topic of Gimbal lock, which is a problem most encountered by astronauts and people developing 3D applications (see! us developers do have something in common with astronauts, we're now doing rocket science!).

That being said, on a wider look on Quaternions, for example, Both Andy Zupko and John Lindquist have done some nice articles on the subject and it's appliances in Papervision3D. Papervision3D, running from Flash 9 and higher, has it's own Quaternion implementation, as Flash 9 doesn't support it natively.

Deriving an easy solution for all

While the solution to ArcBalls using Quaternions isn't even that hard, I still felt the problem should be abstracted a bit more, with an easy one stop solution for all. After all, the reason I got that much feedback on the SimpleZSorter, is clearly because people actually use it. Why do people use it ? My estimate is, it addresses a real problem and it's simple to implement. And while that might cause a little overhead, and make it less flexible, it works for people looking for a simple solution. Here's how that was implemented.

SimpleZSorter.sortClips(boxSprite,true);

Simple enough right ? So for the abstraction of the ArcBall something similar made sense.
Using the ArcBall class (and getting your correct dragging rotations up and running in no time) works like this.

arcBall = new ArcBall(boxSprite);

Now you can drag this object, and get nice, intuive rotations from it. There are a bunch of options to use;

arcBall = new ArcBall(boxSprite, useRadius, radius);

If you set the useRadius argument to true, and set radius to a specific value for that object (think about a sphere which completely encapsulates the object you have, it's bounding sphere, see my beautiful artistic rendering at the top of this post), the bounding sphere used per dragging action is the same size. When that option is left set to false, the point on the sphere is derived from the click in 3D space on that object and thus speed might vary, depending where the user start the drag.

Because the ArcBall class automatically adds listeners to the object, and automatically enables dragging on it, you need a way to switch it off when it's not wanted.

arcBall.enabled = false;
//To enable it again
arcBall.enabled = true;

So, that's it. I hope it helps you in getting a better experience for the end user. Since this is an abstraction for ease of use and much more can be done, the entire code base for both examples and the ArcBall under the MIT License; essentially, do with it what you want, if it breaks anything, it's not my fault. Let me know what you think.

Lastly a mention of sources which helped me with getting this right; rainwarrior, NeHe, processing. Sharing works!

Sources

ArcBall (just the sources for use).

RotationWrong (sources for the first example, incorrect drag rotation)

RotationArcBall (sources for the second example, correct dragging using the ArcBall class)

[Update 8/9/2009]

Little update on the quaternion explanation.

No related posts.

I normally don't delve into book reviews or endorsements; and I won't go there just yet with this post. But I do want to get your attention for the first released Papervision3D book; Papervision3D Essentials.

Written by Jeff Winder and Paul Tondeur, this elaborate, 13 chapter book looks great for anyone wanting to learn or deepen their knowledge of the Papervision3D API and Engine. Congratulations to both authors for getting it released! Great job!

Although I haven't recieved my ([total disclosure] free, thank you guys), copy for review yet, the book looks promising, if you can judge it by these two free online chapters, about Lines3D and VectorVision.

As one of the core members of Papervision3D and having architectured and written rather large portions of the Papervision 2 Engine, for me personally it's amazing to see a project like this. It's documenting code that the team has been writing over the years and getting it on paper, along with an explanation and examples is something huge to me, as well as being huge for people wanting to start or already experienced with the engine! Having recieved multiple mails full of questions and additional bug fixes (as have other members of the team) from Paul and Jeff, I can assure they have been thorough in reviewing & examplifying the Papervision code base.

In a way, that's a very odd feeling; an obscure function which might have been thought up and written at 2pm at night (which during the initial dev of Papervision3D 1.7, 1.9 and 2.0 was rule, rather then exception), might now have gotten an explanation in the book.

One thing the project has been lacking the most though, was a paper resource to look things up; and that has now arrived!

I will blog a more thorough review after having recieved and read the book. Can't wait to see this book as a piece of tangible material and actually hold it!

[UPDATE]

Another free chapter has emerged as PDF. Chapter 8; modelling and importing to Papervision3D.

No related posts.

Microsoft released Silverlight 3. So it's time for Silverlight on my Flash blog ? Heresy ?  No, not really.

I'm not going to do an intro to Silverlight here, I guess you all know what it is, or what it is proposed to be. As a developer, I'm naturally interested in platforms available to me or in a similar area to what I work in. Therefore, Silverlight has my attention. Not because I believe it's a Flash killer, not because I think a bag of Microsofts cash will come my way, not because I want to destroy Silverlight, but because it's a technology which plays in the same realm as my professional environment. Surely being Flash Platform developer, I'm biased. The platform that has allowed me to earn my professional living and have so much fun is clearly my favorite any day. But let's have a look anyway.

Silverlight vs Flash

I've always found the developer oriented marketing message for Silverlight somewhat confusion. One moment it has nothing to do with the realm of Flash, the other it's the designated Flash killer, and I find myself hosting heated discussions from both camps. It's been confusing. Personally, I've had a look at SL1, 1.1 and left it there. During those tests I found it was still very young technology and a far cry from the capabilities from my personal favorite platform, Adobe Flash.

I remember the first release site for Silverlight, and it left me confused if I was watching a commercial for a snowboard brand or a new web technology. Things seem clearer now, Silverlight clearly has an ambition towards what Flash is, but currently still isn't. Neither towards end users or developers. What has happened with the release of SL3 ?

Features

Now in it's 3rd iteration, the Silverlight team seems to be playing catch up and there's an akward similarity between Silverlight 3 and Flash 8/9/10. Let's look at a selection (these are the biggest though) of the features added, and how they translate to the world of Flash.

I'd like to get one thing out of the way as an older feature; Silverlight 1.1 - .NET support / Translates losely into ActionScript AS2 to AS3. A far better runtime for code in both cases.

- Silverlight Perspective3D - Flash 10 Perspective 3D
Well, with both players now doing 2.5D "postcards in space" natively, it's up to third party developers to build the 3D frameworks. Good thing I like that kind of stuff a lot. Before, it didn't seem possible to build anything like Papervision3D on a usable level for Silverlight, but I think that with 3.0 we might finally see some competition coming from the Silverlight camp, now that they can actually control bitmap content ?

- Silverlight PixelShader API's - Flash 10 PixelBender & Flash 8/9 Bitmap Effects
I'm not entirely sure if Silverlight PixelShader API's are GPU accelerated or not. Flash 10 Pixelbender surely isn't. So far I haven't seen a running version of Silverlight PixelShader API yet, but hoping someone will fill me in. The ability to write 'real' HLSL for it seems enticing. Currently I don't think there's enough demo's online on the Silverlight side, but it would be interested to see where that goes.

- Silverlight Cached Compositing - Flash 8/9 gpu assist (fullscreen, mac) and Flash 10 GPU / Direct WMODE.
Somewhat similar to each other, Silverlight Cached compositing seems to be a combo of using the GPU for rendering as well as good old, cacheAsBitmap. I'd be interested to hear from both camps how far integration goes. From a developer perspective to both platform, both seem to just be a "switch" to be turned on, no control.

- Silverlight Bitmap API & BitmapCaching- Flash 8 Bitmaps and .cacheAsBitmap
Silverlight gets my favorite Flash 8 feature. Bitmaps. WriteAble bitmap is similar to BitmapData and allows for richer effects and graphics to be created by the developer. BitmapCaching is surprisingly similar in even name to .cacheAsBitmap, which allows the developer to reduce the overhead of rasterizing components of which the visual state has not changed.

- Silverlight LocalConnection API - Flash 6 LocalConnection API
What's there too say ? Player instance talking to Player instance. I actually assumed this was already inside Silverlight, but hey, now they do. Seems to be related with Silverlight endeavours on the Desktop (think AIR, but a little less well featured).

- Silverlight Application Model Enhancements - Flash 9/10 FileReference API
Again, this seems to be related to MS preparing Silverlight to jump to the desktop, although this feature is available in both Flash and it's desktop brother AIR.

- Silverlight Assembly Caching - Flex Framework / Flash 9 RSL Caching.
Storing chunks of frameworks on the client so they don't have to be loaded in everytime for every site / platform. Not sure how cross platforms silverlight solution is. Neither do I know how to erase that cache with SL3.

- Silverlight Network Monitoring API - AIR ServiceMonitor, SocketMonitor and URLMonitor
Again, a feature for SL jumping to Desktop, this allows you to see if you're connected to the cloud or not. AIR is a little more extensive with it, Flash has no implicit features.

- Silverlight Application Frameworks - Seems to be a code base like the Flex SDK
Application Frameworks loosely relates to what we call the Flex framework, and the enhancements on that for SL3 seems be very minor. Flex2 - Flex 3 - Flex 4 (gumbo) steps seem much bigger.

- Silverlight UIFrameworks Enhancements - Flex SDK / Flash Components
Well, not much to say here either, UIFrameworks seems to be playing catch up it with Flash / Flex UI Components (the latter to be most precise).

- Silverlight Text Improvements - Flash 8/9/10 text improvements
Although the new Text Rendering engine for Flash 10 is a huge step, the Silvlerlight of things looks more like the new control over text rendering like the one we got in Flash 8-9. Flash text layouting seems to be far ahead on this one.

- Silverlight Animation Easing Effects - Flash - Robert Penner, anyone ? Combined with some mx.easing.
Again, this was a feature I thought to be native Silverlight already, but it is just introduced now. As with the early days with Flash easing, I hope people see how to leverage this for a better UI, and not an experience of windows and things shooting by, rendering the UI useless.

A strange coincidence....

A lot of similarity between new SL3 features and Flash 8-9-10 features, right ?
Now, none of this is a coincidence. All these features are surprisingly similar to some of the new features we've seen appearing in Flash over the last 4 years. In a way, you can say SL is still playing catch-up. In the new feature set I see no real innovative RIA / Plugin features that really distinct Silverlight. That being said, SL plugin is still not as feature rich as Flash 10 is currently. Seeing what Microsoft has done in 3 versions, one could asses that they are moving very fast on the plugin feature front. But there are some things consider here. First of all, Silverlight obviously is learning it's lessons and features from Flash. I'm yet to be amazed by a novel Silverlight feature. Another thing to consider is that Silverlight 1, 1.1 and 2 compared to Flash had virtually no installed base, and backwards compatibility of all features and API's doesn't seem to be retained. Compare that with a Flash Player with more then 10 years of history and still be entirely backwards compatible.

Plugin stability

Stability wise, my experience with the SL plugin have been less then good. Browser crashes, followed by content not running (I'm still looking to see working demos of most of the new features online, which run on my mac too, anyone ?). The experience of years of Flash Player seems to be owning Silverlight here. With content behaving differently between my both Macbook Pro's, ubiquity doesn't seem to be the strong point of Silverlight; I'd say this would be me only, but my twitter feed tells me differently today.

Video

One thing which is worth mentioning in Silverlight's favor is the accelerated video rendering. Flash 9 / Moviestar (9.0.115) introduced fullscreen acceleration to allow for smoother feedback, but the acceleration mentioned for Silverlight seems to be decoding based. VC-1 is supported by graphic card manufacturers, so this would mean the actual decoding is done on the GPU, allowing the plugin to offload that work from the CPU. In that same area, DRM for VC-1 seems to be pretty sturdy too.

Tools

On the tools front, I can't say that much, since the Microsoft tools for building Silverlight content still don't run native on Mac OS. From back in the day, I can still say the Visual Studio IDE is one of the best Microsoft products, but beyond that, I have no experience with their new tools. I hear integration with Photoshop is pretty good. In a way tools and development language will also greatly influence the people using them, in turn influencing the output created for the end users. Flash is still a very nice melee of designers, actionscript developers coming from a design backend, and hardcode java developers turned Flexer (just to name a few sub categories). At any conference we always get a nice melee of peope all with their own angle and usage of the platform. This really reflects in all the different content being produced for the platform, where I'm still waiting to see that diversity in the Silverlight universe.

Some personal ideas

Now, although SL is making some progress on their own feature front, they are still behind. But in the end it's about the end user experience, and this is where Microsoft clearly failed in this context. Their see the light website is less then optimal, if I put it mildly. Seemingly infite loading time, followed by a horrendous design and user experience, leaving me wondering where all those megabytes of preloaded content went.I wonder if this is a trend amongst all SL content, as I've yet to not see one of those dead ugly SL preloader circles. This is the site which is supposed to showcase Silverlight, but it is it own worst advertising. Maybe they can get a Flasher to redo the seethelight.com website.

[UPDATE 11th July 2009]

Apparently, I'm not the only one who didn't find seethelight.com up to par, after seeing the light, microsoft removed it, linking directly to the html version (in the Silverlight version, this was linked as "reduced experience"). Now, we get the "reduced experience" by default. Let's hope they come with something to replace it, and be a real showcase for Silverlight.

[/UPDATE]

Silverlight is good for Flashers too.

All in all, Silverlight is good for everyone. It means competition in the field of rich internet plugins. Yes, there are more. But these are the 2 big companies playing and competing in that field. (I really don't think Sun is a real contender in this area yet). And general logic would say that with competition, both products will be more rapidly developed and improved. Which, for us developers and end users is only a good thing. With SL3 being just out, we'll just have to see how many developers adopt it and how many content providers are willing to take the jump on a non-ubiqitous plugin and how steep the adoption slope will be. In terms of catching up,in my opinion it's still Microsoft's move (even with SL3), but the incremental step they did for this version looks promising for the future. At which point we can all look forward to Flash 11, Flash on devices and mobile to compete with that.

No related posts.

I just stumbled upon this very short TED talk by Chris Hughes. In his presentation Chris Hughes shows of an "Augmented Reality demo using Adobe Flash!". In an exited tone he talks about how he wrote a piece of software (!) to allow the replacement of a "2D Barcode" with something "really, really cool" using nothing but Adobe Flash!

In reality he hardly wrote any software, rather did he do a pretty standard and in my opinion a very ugly implementation of Papervision3D and FLARToolkit, for which tutorials can be found online quite easily.

Why is this worth blogging ? Well, as a Papervision3D core developer, personally I'm offended for someone taking credit for both the Papervision3D and the libspark project's FLARToolkit libaries, none of which he has had any hand in. He's essentially taking credit for 100's of hours of work by both the Papervision3D and the Libspark team. With the TED conference headline being "Ideas worth spreading", which implies the sharing of knowledge and ideas. Both Papervision3D and FLARToolkit are opensource projects, with exactly that in mind. And both of them get ripped off by Chris Hughes, who not only fails to cite his sources, but takes credit for it by saying the following things on the TED stage :

"......I wrote a piece of software ...... "

"Oh, no, it does not quite stop there.......[]......another bit of the technology we managed to introduce here......"

With both statement not only failing to mention the actual technology and projects, Chris Hughes but actually taking credit for it. Disgusting. But hold on, to cite Chris himself, "it does not quite stop there"....in the followup interview on TED.cm Chris Hughes states the following :

"So what you've seen today is a Flash port of an existing project. There's a project out there known as ARToolKit. Unbelievably awesome. It's a university research project. Folks ported it from C, which is what it's written on, onto Java. And then we took the Java port and made it work in Flash."

That last piece of that sentence is very interesting "we took the java port and made it work in Flash". Let's make it easy. No you didn't. You took a port named FLARToolkit, and used it in your project. (here's the sources to his demos, which show Papervision3D and FLARToolkit).

So what did Chris Hughes do for him to take credit on a TED Stage ? Most likely, he followed a tutorial like this one from the FlashBlog, then gathered all his courage and energy to work with 2 opensource projects and take credit for it.

One more quote from Chris Hughes here :

"It's open source. Everything I do is about being very open."

Openness at a conference like TED would be giving urls, credit and a clear explanation of what you did. Right now you're just taking full credit for work done by 2 software team's hundreds (if not thousands of hours) of work, for your own credit.

I hope the TED conference takes the proper steps to avoid people like Chris Hughes standing upon on a stage, presenting other people's work as their own, purely for their own glory. I'm disgusted by this.

[EDIT]

To avoid any confusion, looking back at this hastly and angrily typed post, I want to clarify that most of the work he's demoing which makes this so awesome is the FLARToolkit project. Papervision3D is used for visualization, but I'm guessing most of the awe is for the FLARToolkit project, which is done mainly by saqoosha.

[UPDATE]

The TED conference has removed the video from the Talks page, and is in the process of removing subsequent posts, as a response to the contreversy around Chris his talk. I do feel a rectification of the statement "And then we took the Java port and made it work in Flash." is still missing, since this is just clearly not true.

I have a lot of feeling towards this kind of presentation of other peoples work without proper accreditation and honestly, I do feel the real victim here is Saqoosha / LibSpark. Saqoosha has posted a blog around this, unfortunately only in japanese.

I'm sorry for the uproar and feeling some people have of this being a witch hunt, or the feeling this was instigating "get your pitchforks and torches out". That was not the intent of my blogpost. I feel that injustice was done, and it wasn't publicly rectified. As this blogpost was part of the reasoning behind the removal of the video of the TED pages, I do feel it served the proper purpose and in that specific result hit it's goal.

It wasn't my intent to be insulting, or publicly binding Chris Hughes to a stake. Rather this was a way to rectify the injustice which was done to the original developers of the source code he displayed, and took credit for. I apologize for any of my wording which is that strong above and as such removing it would be wrong. Rather then doing that, I'll leave it up and apologize for too strong wording or instigation of a witch hunt. What I won't apologize for is pointing out this issue and subsequently asking attention for it.

[UPDATE2]

I have just been in an enlightning IM discussion with Keith Peters / Bit-101.  In this discussion Keith pointed something out I feel I am at fault at. I've could have contacted Chris personally to ask him for a public correction of both his talk and the interview. Instead, I first wrote this post and then mailed Chris later, after any reply from his side stayed out.

I've got no excuse for this and I am at fault for not contacting him directly first. I apologize for that towards Chris Hughes.

[UPDATE3]

Upon further investigation, TED.com have decided to completely retract the entire speech, naming it "misleading".

[UPDATE4 25/06/09]

With TED having researched the grounds of these allegations and subsequently removing the video and retracting the entire thing, Chris Hughes still wants to pass this off as a fluke. I have yet to see an updated blogpost, or a reply to my emails. As such, let me make it perfectly clear for the last time, for future reference.

Chris Hughes took Mikko Haapoja's tutorial on using FLARToolkit and Papervision3D, two open source projects he was not involved in at all. Then he did some slight modification of the tutorials, not adding, or extending anything and recompiled the examples. He then went on stage saying "He programmed" etc...

In reality, he passed someone else's work off as his own, without giving attribution or even mentioning the projects, rather just claiming credit for it in an interview, and obfuscating what he did both online and in his presentation.

End of story.

No related posts.

Before I start, I want to explain the timeline which led to this blog post. This was supposed to become a post of being able to get much faster access to bytearray's through ActionScript, by (ab)using the Alchemy memory opcodes. I spent quite some time trying to build a swc library which would allow you to do just that. Only to learn that you can't build a generalized API for this in ActionScript. It just can't work out with the current ActionScript compiler technology; here's the story of why.

Leveling the field between Alchemy, Haxe and Actionscript

The reason I set out to do this, is because I found it somewhat strange that the only language in the row above that is not able to access the faster memory, is the platform's native language, ActionScript. I could use many strong terms for this, but it is what it is. So why not change this myself ? Surely adding a piece of custom built (byte) code, should be able to allow access to this hidden gem ? I had discussed this option with Joa before and it seemed to be a viable path. So why not do it ? A good thing to note in the mention above is that Haxe has already implemented this. Read Nicolas his post about haxe flash.Memory.

Compiling inline ABC with Alchemy's asc.jar, a bit different.

As I posted earlier about Alchemy, one of it's advantages is the fact that Flash Player 10 has an implementation for actionscript byte code opcodes that are currently not supported within the ActionScript API. These opcodes allow for faster access to 1 bytearray inside the player, which you can set by accessing ApplicationDomain.currentDomain.domainMemory.

Strangely enough, that is actually opened up to AS API, presumably to allow for "regular" ActionScript access to the Alchemy memory. The actual opcodes have no ActionScript equivalent though.

Since Alchemy boils C code down to ActionScript, how does it obtain access to these opcodes ? The answer lies in the compiler which comes with Alchemy. The asc.jar in the /bin package allows for so called "inline asm".  This type of code looks something like the following.

__asm(op(0xD2),op(0xD1),op(0x3A));

This code can be put anywhere, and the opcodes will be added to the abc bytestream. Great huh ? So, ideally, we would just switch out our asc.jar in our Flex install directory, and we could just start doing this. Well. NO.
Compc, which is used for actionscript project compilation isn't compatible with this special asc.jar. Not too worry, we can get around this, more on that later.

Discovering special memory opcodes
Although at the first release of Alchemy, these special memory opcodes where somewhat ellusive, by now there's loads of information on that. Most specific info comes from Adobe Alchemy's creator, Scott Petersen, in his presentation FlashCCompiler (pdf).

Now, armed with these special opcodes, writting an API around it shouldn't be to hard, so let's go at it.

public function setFastMem(byteArray : ByteArray):void
{
 currentDomain.domainMemory = byteArray;
}
 
public function writeByte(position:uint, value:int):void
{
__asm(op(0xD2),op(0xD1),op(0x3A));
}

Ha, yes, combined with the Alchemy asc.jar, this code works, and translates out into working bytecode! So, now by calling writeByte, we are using one of the opcodes to write an opcode into the FastMem. Great! When I had all the opcodes to work, I forgot to do one thing, which I should have known and could have saved me a lot of time. Benchmark it. Instead, I first wanted to figure out a way to distribute this, so developers could easily do this.

The long path from .abc to .swc
Now, when using the asc.jar, we can not compile a swc directly, since the asc.jar is incompatible with the compc compiler.
It's only able to output a swf, or the abc bytecode for that class.
What I went and did was examine the gcc script in the alchemy/achacks folder. Interestingly, the gcc script is a replacement for the normal gnu gcc compiler, written in c. It handles the steps for alchemy to go from c to llvm to actionscript to swf/swc distibutable. Even more interesting is HOW alchemy creates a swc, if you use the -swc compiler option. You'd think there would be some magic going on there, but instead it is not so magic at all. When compiling to a swc file, the compiler extracts the abc data from the previously compiled swf using a script called GetABC2.pl, and then replaces the code in an existing .swf file which is in the achacks directory, using PutABC2.pl.

A thing to note about that .swf file it uses, is that the file is an uncompressed .swf, marked in the header by FWS. These are getting more and more rare, since most swf files are compressed these days, and start with CWS instead. This is mostly common knowledge. Now, the problem here is that I wanted to replace that template.swf by my own....only to find out there's no documented way of compiling a swf file without compression. To make matters worse, the perl helper scripts seem to have a bug, where they do not work with compressed files. Although these scripts can be patched, that's not what I wanted to spend my time on, so I went my own route.

Injecting bytecode
Instead, I wrote a little air app, which scanned a swf for the DoABC tag, (id 82), and replaced it with my own ABC bytecode. This involved changing the size tags of both the swf and the tag, and saving it out again. It took little time, but in the end it worked like a charm, and I could now generate my own swf's with custom inject ABC bytecode, great! Now I needed to get this swf into a .swc file.

SWC and zip, a little tip
As most of you would know, as SWC file is nothing more then a .zip file, normally containing a library.swf and a catalog file. In my naive thought I would just unzip the swc, replace the library with my abc injected library and things would work. Not so. I'll save you the hours of pondering on this, but as it turns out, when using the Mac OS X finder, and unzipping it through that, leaves you with something crippled; what I don't know. So here's a good tip, if you want to unzip and rezip a SWC, do so using the terminal / unzip & zip. Really, I wish someone had blogged about this, because it drove me crazy, ending up with Flex Builder saying "can't load swc", even if I didn't change anything in the .swc, just uncompressed and compressed it again. I caught on to that quite fast, but the fact it was the uncompressing the swc which was the problem, took me a little while to realize.

It works! Now let's benchmark
As I said before, with some logical reason and benchmarking before I went on a spent a lot of time on bytecode injectors, build scripts and swc compilation, I could have saved myself a lot of time. The reason why I wanted to have this distributable was the entire reason to do so, so I didn't want to go on, without knowing how to get from a special __asm .as file to a working .swc. In the end it was all in vain. When I started benchmarking this, the result was disappointing.

The FastMem bytearray, and the special functions I had built for it, where nearly 2x SLOWER then just using ByteArray.writeByte(). And then I figured it out. I have forget to think about the overhead of calling a function in ActionScript. Since a built in function is much faster then an actionscript function, the overhead of calling that function far outweighed the faster access speed to the data.

So it ended up in a failed experiment. That's ok. Failing is a way of learning, and during this process I learned a lot of new things. But in this case, the biggest lesson wasn't how to inject bytecode, use the asc.jar, write your own swc compilation pipeline. It was that with the current compiler for ActionScript, it's just not possible to use the FastMem opcodes in a easy and generalized way.

The difference between the compilers

Now, the saddening conclusion is that I wasn't able to build a FastMemory.swc that all Flash developers could use. The more alarming conclusion I take from this, that the problem doesn't just resides in the compiler not having any way to do so, but the fact that it's due to missing features other compilers already have, leaving ActionScript developers behind.

Haxe is able to pull the exact same thing off. With Nicolas his work on the flash.Memory api, you can select a bytearray, and get very fast read and write access. So why isn't this possible with ActionScript ? The Haxe compiler supports inlining. For those unfamiliar with this, this allows you to keep portable and reusable functions, while the compiler inserts these functions directly in the place they are called. So you're not actually calling function, you are essentially inserting a piece of code inside your code. This eliminates the overhead of the function call during the compilation phase.

The reason this does work within Alchemy is that it not only supports inlining (less important in this case), but the Alchemy code actually is processed before hand by the llvm compiler with ActionScript target. Even further, within Alchemy, your code lives inside the reserved fast memory, and access to for instance malloc'ed pieces of memory are inside this memory.

Conclusion and final shout out to Adobe

This leaves ActionScript behind. Not because of the language, just because of the compilers. ActionScript doesn't support inlining, nor does the compiler do any optimizing steps. We could start modifying our own asc.jar and doing this, but with the Flex SDK being opensource, it's still not legal to modify the asc.jar and release, essentially making that step useless to do.

All in all, ActionScript isn't able to target it's own platform as well as both Haxe and Alchemy. Sure, you can insert optimized hand written bytecode in performance critical places, but at that point, why shouldn't I just switch to using Haxe or Alchemy ?  In my opinion, both have their downsides.

Alchemy is great, but as it's currently in beta, it's not even an industry standard. Sure, C as a language is, but Alchemy just isn't. Also, when using Alchemy, even the smallest piece of code results in a huge .swc because of all the standard C library code. All in all, it's amazing Alchemy runs this fast considering all the steps the code takes before it actually runs on your machine.

Haxe is a work of genius. It's a great compiler, I like the language and I specifically love the .abc format it contains, allowing me to write bytecode inside the Flash runtime and executing it. In my book, that's awesome. As of today, Haxe can even target the iPhone ! The language is great and has many features ActionScript lacks. It baffles me how one man army nicolas can single handedly write a language and a multi target compiler, and in many respects stay on top of the ActionScript compiler.

My first problem with Haxe is just that. The one man army. Should nicolas decide he doesn't feel like it anymore, the platform is dead. Sure, there's more people working on it, but in it's core it seems to be that nicolas is the driving force. My second problem is that although there is a huge pool of ActionScript developers, which can work together within the industry standard called ActionScript. There's not even near that many people who've even touched haxe, let alone the number of developers who are able to call themselves "Haxe Professionals".

Instead, I'll stick with ActionScript for now, and hand roll my optimized code using pure abc, haxe or Alchemy, which ever suits the purpose best. But the necessity to do so leaves me wondering what the state of ActionScript and it's compilers is. Should we start building our own ActionScript compiler ? Add our own features to the language ?

[EDIT]

What I failed to mention is that due to the overhead the asc.jar added to the bytecode (some return's and at every function a getlocal0 + pushscope I didn't need) I eventually switched from using that to moving to Haxe with it's Haxe format library. One of the supported formats is ABC, and I ended up using that to create my own .abc bytecode, injecting that instead of the asc.jar output. Again, I'm not saying either Haxe or Alchemy are bad tools, I'm just saying currently ActionScript is lagging behind.

No related posts.

It's been silent here. Next to tripping to Tokyo to speak at Adobe MAX in Januari and doing some small jobs, I'm doing a very, very cool Flash 10 Project utilizing the upcoming Papervision for Flash 10. For some optimization parts I've been looking into using Alchemy as an easy way to write optimized bytecode. As opposed to what some people took from my previous post on Alchemy, I don't hate Alchemy. My point there was that if Alchemy can perform that well, so should ActionScript be able to, with the standard compiler.

Now, that being said, all work and no play makes jack a dull boy. With no time to spare during the morning, daytime and evening, I took the laptop with me to bed last night, and decided to do something I didn't do before, port something with Adobe Alchemy. I decided upon a long time thing on my wish list; SID Playback in Flash. What is SID ? Well, SID stands for Sound Interface Device, referring to the MOS6581 and it's a programmable audiochip as used in the Commodore 64 and Commodore 128. I grew up with the Commodore 64, and the audio it produces is still something I hold dearly. You are quite likely to have heard a SID chip before, if you listen to the radio, as it is quite a possible you've heard one of these baby's scream. Good recent examples are :

Bastian - You've got my love. (the beat in this song comes from Jeroen Tel's Rubicon).

Nelly Furtado - Do It (controversy all around, Timbaland blatantly stole Glenn Rune Gallefoss's version of Acid Jazzed Evening).

De Jeugd van Tegenwoordig - Hollereer (Produced by Bastian, C64 lover himself).

My interest was more playing back the old C64 tunes from games and demos. Keep in mind, an average SID file is about 2kilobyte to 10kilobyte; a perfect Small Web Format for all your music during preloading needs . Now, this is something I have been working on before in Flash 9, using the FC64 project, the amazing Flash C64 emulation effort by Claus Wahlers and Darron Schall. Last year, I even made a start with porting FC64 to Flash 10, but that needed to be dropped in favor of actual client work. SID Playback is something different from playing back .mod .xm or .mid files, as the SID chip doesn't do much on it's own. Essentially, a .SID file is an executable meant to be ran on the 6510, the Commodore 64's CPU. So to playback the file, you not only need to emulate the SID chip, but also the 6510 and the needed registers / memory.

Software emulation of the SID chip has been around for a while, and all modern platforms have a way of playback SID files. Winamp plugins, command line utilities for conversion, it's all there. As is there opensource code to do this. There's libraries for most major languages. So, after a bit of research, I decided upon the Linux version of TinySid. This library is small enough and well optimized. It looked simple to port, so I went at it. All in all, 4 hours later, I had this baby up and running! (And an unhappy girlfriend who isn't into the nerd thing, nor is she into listening to chiptunes whilst trying to sleep). Considering my very rusty C and a little hack around I had to do for the audio output, Alchemy shows off it's feathers here. Many kudo's to the Alchemy and Flash Player team for this effort. Just now I spent half an hour building a tiny interface using Keith Peter's Minimal Components, and collected some songs; and here it is.

TinySidFlash (open link, requires Flash 10).

My favorites : Politik and Science (A C64 version of Coldplay's the Scientist), Morphing and Turbo. Yes, I love Dane's tunes!

There's no code yet, as this is still very rough and unfinished work. I'm pretty sure that when I get time again, I'll have a go at LibSidPlay instead, since the overall emulation in that library is a lot better.

Related posts:

1. Triangle3D : Small Flash 10 3D demo. Over the last days I've spent some time twittering...
2. Help me test! Flash VSync and Screen Tearing. So, here's a post where I am asking you,...
3. Flash Player 10.1 pre-release on Nexus One Nexus One running Flash Player 10.1 from Ralph Hauwert...

The Adobe User Group Netherlands has organized a bustrip from the Netherlands to belgium...yes, an oldschool bustrip ! The FREE Multimania event will be held on the 23rd of this month, and will feature speakers like Aral Balkan, Mario Klingemann, Jason Levine, Serge Jespers, Dave Schroeder and incidentally, I, Ralph Hauwert will be speaking there too.As such I'm looking forward to hopefully see you at my session and/or have a beer and a chat afterwards! It promises to be a lot of fun; for those of you in the Netherlands, wanting to join the organized trip, here's the invite from the Adobe Usergroup Netherlands (dutch, sorry) :

Ga mee met AUG naar MultiMania 2008!
De Adobe User Group Nederland organiseert een reis naar MultiMania, dat vrijdag 23 meivoor de 8e achtereenvolgende keer in Kortrijk, Belgie plaatsvindt. MultiMania is het grootste gratis multimedia event van Europa, en met maar liefst 30 internationale sprekers en presentaties op het gebied van design, development, audio, video, 3D en game development is er voor iedereen wat wils. Aan het einde van de dag is er een exclusief feest, gesponsord door Boulevart. Ga mee met de AUG in de touringcar naar MultiMania!

Sprekers op de conferentie zijn onder andere Group94, Rob Chiu, Aral Balkan, Mario Klingemann, Jason Levine, Sander Lipmann, Ralph Hauwert, Serge Jespers, Capt. Luke en nog veel meer. Kijk hier voor het volledige programma.

Er is door de AUG een touringcar geregeld die vanuit Amsterdam, via Utrecht en Breda naar Kortrijk in Belgie rijdt. In de avond wordt iedereen door dezelfde bus weer netjes naar Nederland gebracht. Geen gezeur met gemiste treinen, vertragingen of ‘de bob zijn’ maar met vakidioten en studenten een gezellig uitje met de Adobe User Group! De kosten van deze bustrip bedragen 39,50 euro en als je student bent betaal je maar 24,50 euro. Om jezelf aan te melden voor deze trip, ga je naar de inschrijvings pagina. Hopelijk tot ziens op de 23e!

No related posts.