Bringing Flash Video to HTML5 Canvas
After I was working with video again I got curious about bringing a flash video stream to HTML5 canvas, for whatever reason, so I did some experimenting and here are my findings.
I used the AS3 ExternalInterface provided by flash player to access video data in flash from javascript. I tried using calling javascript from flash first, pushing the data with JSInterface but as it turns out it is faster the other way around. Overall the performance of the js/as interface is pretty sh*t. A call of a flash function from javascript costs about 200ms, if the data transferred reaches a certain size. If the data is just a short string the calls are pretty fast. I didn’t take times here. Although I thought that maybe I could speed things up by splitting up the data transferred and tranferring many little chunks, but that went really bad.
So I was left with that ridiculously slow interface, which doesn’t allow a fluent video stream at all but at least I could render a Video Image to an HTML5 canvas. (Click here to watch the experiment)
There is not much to it to get this done. Everytime javascript requests an image from flash, the current video frame is drawn to a BitmapData Object from which we can get a ByteArray with ARGB pixels. Unfortunately we cannot directly return the ByteArray to javascript, but have to reduce it to a plain flash array. In the javascript part we match the pixels from the flash ByteArray onto a canvas ImageData. As the canvas ImageData is RGBA we have to switch the A part. Here is how it is done:
1 2 3 4 5 6 7 8 9 10 11 12 | function jsDrawPixels(pixels) { for(i = 0; i < pixels.length; i += 4) { imageData.data[i] = pixels[i+1]; imageData.data[i+1] = pixels[i+2]; imageData.data[i+2] = pixels[i+3]; imageData.data[i+3] = pixels[i]; } context.putImageData(imageData, 0, 0); }; |
The full javascript part can be found in the source of the experimental HTML page.
Here is the AS3 part.
In a second experiment I tried speeding things up by reducing the data transferred. I encoded the BitmapData in flash to JPEG with JPEGEncoder and then to a Base64 string what reduced the data enormously, but that would mean to decode it in javascript again to render it to the canvas, which would just be stupid, so I just used an img tag to show it. At least in that case I got 4 fps in average and each Image rendered that way can at least be saved by “right-click-save-as”. Maybe this can be usefull.
(Click here to watch the second experiment)
Summary: No (real) video, but at least image data.
Thanks for sharing your findings! I was able to get pretty good frame rates by compressing the webcam images as PNG then Base64 encoding the PNG ByteArray in AS3, then using context.drawImage(image, 0, 0) to put the image in canvas. I have some more testing to do, but here is my cobbled-together experiment: http://forresto.github.com/camcanvas/
My bad, I have you the wrong link. I’m now using a fast JPEG encoder that uses Alchemy, and getting 20fpsish.
Demo: http://forresto.github.com/meemoo-camcanvas
Source: https://github.com/forresto/meemoo-camcanvas
Nice improvement! Thanks for getting back to me on this. I also like this one: http://www.pixine.fr/820/utiliser-la-webcam-flash-dans-une-page-html/