Drawing using Flash - Live Long and Prosper

Thanks for those kind words, Mr. Spock, and here's your portrait, as I drew using Flash CS3. It was my first attempt at drawing with Flash, and it took me a little over an hour (click to enlarge):

Now that I'm unemployed, I decided to learn some Flash. I downloaded the trial from Adobe, and started reading around. I was more interested in writing programs using ActionScript 3, but while writing those programs, I also wanted to have decent images/animations to be shown.

I could find very little information about how to draw things in Flash. This looks like a very important aspect of Flash, but even the Adobe CS3 Video Workshop neglects this issue (or at least, I couldn't find it). All the tutorials I could find were fine with animating a circle or a rectangle.

Then, a friend showed me a tutorial called Drawing Line Art with Flash. It was so simple and smart, that I had to try it right away. Don't know why, but I decided that Spock would be a good candidate, and you saw the results above. I must say I have a keen eye for what looks pleasant, but I'm definitely an amateur graphics artist (and even less than that). If I can do this with Flash, it's a great tool (at least for this purpose), and almost anyone can do it.

For reference, here's the original image, if you search the web for Spock images, you'll see this image repeated many times:

I'm Unemployed

All the excellent bits and bytes I put (with others in the RND team) into the Transparency Software Vision(tm) product were not enough. The startup company I worked for since January 2005 (almost 3 years) is shutting down - I suspect the previous link to the company site would be broken pretty soon.

The investors lost faith in the company management and decided to cut their losses and take whatever's left in the company's bank account.

So I was officially fired yesterday - Sunday, September 16th 2007, and I'm now unemployed. I was actually one of the founders of the company, but being a technical guy, and not a businessman, I was a mini-founder - getting just a few percents of the original company. Oh, in my many years in the the hi-tech industry I've learned and re-learned that the developers in a startup company (no matter how good or bad they are) have little influence on the company's chances to succeed. It's much more in the hands of all the non-technical guys. It's a shame, since I'm a very good developer, and I wish I could attribute more to the success of the places I work in.

Well, wish me luck for next time... ;-)

Azureus Status Command Line Invoker

I've written a plug-in for Azureus that allows running an arbitrary command line once a download completes. You could, for example, send yourself an e-mail once this happens, but there's actually already a plug-in dedicated for this.

I'm using this plug-in to check if AVIs I download use packed bitstream, and if they do, unpack them. This way, there's one less thing to do before I can play the movie on my standalone DVD/DivX player.

For more information, see my plug-in page.

Why Frame Rate Matters

I saw the following in MythTV's forum (MythTvTalk):

"Once the video is captured in the file (...), you are past any PAL/NTSC differences..."

While this is true in some aspects, it's very wrong with regards to one important aspect: frame rate. Why is frame rate important? Because there's no way to do interpolations on time.

Let's think about it this way - if your monitor is set to 1024 x 768 and the movie you're watching is using a different resolution, it's relatively easy to scale the movie to fit the entire screen - scaling using various methods of interpolation has been done a lot. In most cases, it's the video card that does the scaling, so it doesn't even affect your computer playback performance (scaling doesn't work well on interlaced video, which is one of the reasons I think interlaced video sucks, but that's the topic for another post).

If your monitor is set to 60Hz (it's practically showing 60 fps), and you're watching a PAL movie that runs at 25 frames per second, there's no way to interpolate 60 frames out of the existing 25 frames. Instead, during playback frames are repeated so that the overall speed of the movie is maintained.

Here's an easy example, and let's assume that NTSC runs at exactly 30 fps for simplicity: If your monitor is set to 60Hz, and you play a 30 fps NTSC movie, each frame is shown for two progressive scans of the screen - the frame rate is maintained exactly, and the quality and smoothness of the video is excellent.

But now try to watch a 25 fps PAL movie using the same monitor. Now you have to fit 25 original frames into 60 display frames. That's 5 original frames into 12 display frames (in 1/5 of a second). There's no way to spread the frames evenly. Let's assume an original frame switch is done as soon as it should be shown, but not within a display frame (otherwise you'll see a discontinuity at a certain vertical position, whenever there's horizontal motion between frames). That's the way DirectShow works on Windows. In this case, here's how 1/5 of a second would look like:



After this conversion, some of the frames would be shown for 1/30 of a seconds, and some would be shown for 1/20 of a second, there's actual a rhythm here: 3,2,3,2,2. This rhythm will repeat itself 5 times per second throughout the video you're seeing. The visual result of this is called judder, and is especially common when converting films to NTSC.

How annoying is this jitter? Many people wouldn't be able to spot the problem, but the video would looks a bit jumpy. It's especially visible in scenes of consistent (and slow) horizontal movement of the camera (panning).

Connecting your PC to a TV

What if you have a TV-Out option on your display card, and you hook your PC to a TV? Will this solve the problem? The TV shows 25 fps, and the movie also has 25 fps, but there's no way to skip the 60 fps of the display adapter (actually, some display adapters lock the frame rate to something sensible when the TV-Out is turned on, but let's assume you can feed a PAL TV from a display card showing 60 fps).

In this case the result is even worse. The display adapter has to convert the 60 fps back to 25 fps without knowing the original material was in 25 fps. This would probably look like this:



What happened here? The first original frame (red) is shown without a problem, but by the time we need to show the second frame (on the TV), the second video frame (yellow) is not yet shown, so the red frame will be shown once more. The outcome is that out of every 5 frames, one will be repeated, and one will be skipped. This is a pretty noticeable judder.


What's the Solution?

In some cases it's quite simple - if you're using your PC to watch video, make sure the display refresh rate matches your video's frame rate. If you watch PAL movies, set your display to 75Hz or 100Hz. If you watch NTSC movies, it sounds like you should set your display to 60Hz (or 90Hz or 120Hz), but most NTSC shows are actually filmed at 24 fps (film rate), so the best frame rate is a multiplication of that, probably 72Hz.

Is that all?

No, of course not. Many people in Europe want to use their computers to watch both shows they recorded using a capture card (which will be recorded at 25 fps) and movies or US TV shows (which are recorded at 24 fps). How do you set your monitor's refresh rate in this case? I'll leave that to the next post.

DWR problem: InboundVariable error when combined with prototype.js

General
I'm using DWR at work and it's an excellent tool for AJAX (easily doing RPC from the client browser in JavaScript back to the Java Server). It also has good enough integration with Spring, which is a plus.

But for quite some time, we've had errors popping up in our logs every time we sent a JavaScript object to over DWR. The errors would look something like this:

2007-03-07 03:56:14,783 [ERROR] InboundVariable - Found reference to variable named 'c0-e6', but no variable of that name could be found.

To keep a long story short, if you happen to use the prototype.js library, it adds a function called 'extend' to the prototype of Object, so all objects have it. For some reason, DWR tries to marshal that 'extend' function as a property, but fails (it's a function), the the POST request that DWR sends, the 'extend' property is specified, but doesn't have a corresponding value.

In our case, we didn't acutally use prototype at that point, so we removed the
<script src="prototype-x.y.x.js"> from our JSP pages and the error was gone. If you need to fix this problem, but want to keep using prototype.js, you'll probably have to remove the 'extend' property from objects you send using DWR (or set the property to null).

BTW, if you ask me - it's a bug in DWR, since functions should not be marshaled. I'm using DWR 1.1.3, so this might have been fixed in 1.1.4, but I didn't see too much information about this using the almighty search engine.


How the problem was found
I used the LiveHttpHeaders extension for FireFox, and captured a DWR call. It seems very straightforward:

POST /Console/dwr/exec/XXXSearch.handleAddOrUpdate.dwr HTTP/1.1
Host: localhost:8085
User-Agent: Mozilla/5.0 (...) Gecko/20070216 Firefox/1.5.0.10
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9...
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: text/plain
Content-Length: 393
Cookie: JSESSIONID={don't tell}; Country=US; Language=en
Pragma: no-cache
Cache-Control: no-cache
callCount=1
c0-scriptName=XXXSearch
c0-methodName=handleAddOrUpdate
c0-id=8297_1173264435303
c0-param0=boolean:false
c0-e1=number:1
c0-e2=string:session.5
c0-e3=boolean:false
c0-e4=string:1
c0-e5=string:orcl
c0-param1=Object:{currRow:reference:c0-e1, factory:reference:c0-e2,
              orCondition:reference:c0-e3, cond:reference:c0-e4,
              values_0:reference:c0-e5, extend:reference:c0-e6}
xml=true

The request reminded me that DWR calls can be batched (hence the callCount field), and all the c0-{name} arguments describe call number 0, I assume I would have seen c1-{name} if I had more calls in this request.
I've also sent two parameters. The first (co-param0) is a boolean. The second (c0-param1) is complex, so it references other entries in the request, called c0-e{num}.
It's easy to see that 'extend' references 'c0-e6', that doesn't exist in the request.
I didn't know why 'extend' was there - I didn't send it, so I tried a short HTML with a DWR request, and indeed, it didn't send an 'extend' as part of the object I sent. Now, it was obvious this is done by one of the many JS files included in our real application page, and a quick search revealed the offender.

Why I want an HD based DVD/Xvid player

Why I need an HD based DVD player
The reason is quite simple. I have two kids. The older is 3.5 years old and the younger is 2.5 years old. The both watch a fair amount of kids' DVD and like other kids their age, they love seeing the same movies over and over again. Like other kids their age, they've become experts at operating the DVD player and scratching the DVD discs quite hard.

There's no use, DVDs are much better than the old VHS videocasettes (I can't even watch VHS content anymore), but for sure, they're less kids-proof. What's can I say? They're less durable. When you rent a DVD that's not new it's more likely it will pause and skip and stutter than an old VHS tape. The tape deteriorates over time, and the initial quality is lower, but a DVD can become unusable much faster.

Why I need an HD based DVD player? So I can rip those kids DVDs and have them on a hard drive for repeated playback. I would then store the kids' DVDs up with my grown-up DVD collection. If you carefully protect your DVDs like I do, I hope we'll be able to watch them for many more years.

Why I need an HD based Xvid player
I'm downloading videos over the internet, usually using Azureus. I'd like to watch those videos on my 32" TV in the living room, not in the spare room where my computer is. These days, whenever I download something, I burn it on a CD-RW and take it to my DVD/Divx player that's connected to my TV. This wastes a lot of time. I still don't understand why it takes over 10 seconds for a DVD player to recognize a media once it's inserted, but that nothing compared to the few minutes it takes to erase the CD-RW and then rewrite it with the video I downloaded. In addition, CD-RW discs are generally not very reliable. Every once in a while the burn fails, or succeeds, but the player doesn't read the disc or freezes in the middle etc. I need something more reliable like an HD, where I'm sure the information is stored properly.


What I want from an HD based DVD/Xvid Player
I want to be able to play videos without being afraid that my media will be scratched (by my kids, not the player) or that I'll need to burn CD-RWs all the time. Wait, that's not enough. I'm picky and I have more demands:

• I won't settle for image quality that's less than what I get from a DVD player. I have a Compaq/HP nw8000 laptop with TV out (S-Video), but the video out quality is not on par with DVD players. The image is washed out and I can't get overscan to work properly.
• I won't settle for poor video decoding performance which introduces frame skips and other visual distractions.
• I won't settle for something that outputs only NTSC or PAL. I have a multisystem TV, and I want to see PAL movies in PAL and NTSC movies in NTSC. A conversion is doable, but changing frame rates introduces some jitter to the video that really annoys the eye.
• I'd be happy to be able to attach thumbnails to files, so that it's easier for my wife and kids to find the videos they want to play.

That's basically it. I don't think that's too much to ask...

What is Packed Bitstream?

Many Xvid encoded movies are encoded using Packed Bitstream. What is it? First of all, for me it's important since I have a stand-alone DVD player that can play Xvid encoded avi files, but only if they don't use packed bitstream. Whenever I obtain a movie, and before I burn it on a CD-RW, I have to check if it uses packed bitstream, and if it does, use a utility to convert it to a non-packed bitstream movie (more on that later).

Let me just say up front what packed bitstream is not - it's not saving any space. Actually, when "unpacking" a packed bitstream video the result is a bit smaller (but just a few KBs smaller).

Frame (or VOP) Types
Before we can talk about packed bitstream, we'll need to talk about frame types. In MPEG4 talk, each encoded image of the movie is called a VOP (Video Object Plane). A VOP is usually equivalent to a frame: here's the explanation from Wikipedia: "In MPEG-4, VOP refers to a "Video Object Plane" which is effectively a Video frame." (We'll later see that packed bitstream is exactly where a VOP is not a frame). Here are the VOP types (I'll intermix frames and VOPs, since at this point we can consider them the same):

• I-VOP (Intra): A frame encoded like a still image (much like a JPEG image). Why aren't all frames encoded like that? It's not efficient. Here's why:
• P-VOP (Previous): A frame that encodes only the changes from the previous frame (we'll be more accurate shortly). In many (actually most) cases, a video frame is very similar to the its preceeding frame. It makes sense to encode a frame by saying "this frame is exactly like the previous frame, except for the following changes". P-VOPs are much more efficient than I-VOPs.
• B-VOP (Bi-directional) - we'll discuss these in a moment.

When would you use I-VOPs and when would you use P-VOPs? Theretically, you'd use an I-VOP for the first frame of the movie, and all the rest would be P-VOPs. Of course, whenever there's a scene cut, it makes sense to use an I-VOP, descibing the changes between unrelated frames is more complex that describing a frame.

This is theoretically, mainly because of the following reason: If you want to jump to a specific frame, you have to decode all the frames from the previous I-VOP to the current frame. The frame itself doesn't contain enough information to be seen. Decoding many frames is a long process. Therefore, it's important to have I-VOPs not too far apart. Two comments on this: First: while DVD is MPEG2 and not MPEG4, it also has I-Frames and P-Frames (and also B-Frames), and each chapter mark always points to an I-Frame. Makes sense, right? Second, in many Xvid encoded movies, there are sometimes long scenes where there are I-VOPs every 300 frames! This is very good for overall compression, but seeking takes quite some time.

What about B-VOPs? B-VOPs are encoded based on the previous I-VOP or P-VOP, and also the next I-VOP or P-VOP. The MPEG group found that in many cases, describing a frame as the sum of changes from previous and next frames is more efficient than just the changes from a previous frame. Why? I don't know but the 'E' in MPEG stands for 'Experts', so they know what they say. To complete the picture, I'll just say that P-VOPs are encoded from the previous I-VOP or P-VOP, but never from a previous B-VOP. This means that you B-VOPs never affect the quality of any other frames but themselves, which means you can compress them as much as you want, and it will affect only the current frame. That's not true for I-VOPs or P-VOPs.

Frame Type Example
Let's say we want to encode a sequence of 10 frames. We might end up with the following sequence of frames (or VOPs):



The first frame (frame 0) uses an I-VOP. Frame 3 is encoded based on frame 0, frame 6 is encoded based on frame 3, etc. Frames 1 and 2 are encoded based on frames 0 and 3. Frames 4 and 5 are encoded based on frames 3 and 6, etc. Makes sense, right?

Playback Issues
This is all fine, but how can the player decode a B-VOP if the P-VOP it's based on is later in the stream? What if we'll have much more than 2 consecutive B-VOPs before the P-VOB they're based on? The answer is simple, we need to rearrange the order of the frames as the appear in the stream (or in the file). Here's how this is done in non-packed bitstream files:



The P-VOP preceeds the B-VOPs that depend on it. Of course, it's marked in a special way so that it will not be shown in the stream order. There's still a problem. If the player starts shows frame 0 as soon as it reads it, it will have nothing to show when it reads the next frame. It has to read two frames before encountering the B-VOP it should show next (frame 2). The solution is to insert a playback delay:

• When the player reads and decodes frame 0 (I), it displays nothing.
• When the player reads and decodes frame 1 (P), it displays frame 0.
• When the player reads and decodes frame 2 (B), it displays frame 2.
• When the player reads and decodes frame 3 (B), it displays frame 3.
• When the player reads and decodes frame 4 (P), it displays frame 1.

It seems that the player needs a single frame delay to be able to play the stream properly. I read somewhere that the player actually uses a 3 frame delay.

Packed Bitstream
A video stream that uses packed bitstream tackles this issue using a different method:



The change here is that a single frame in the stream includes 2 VOPs. Whenever a future P-VOP or I-VOP is needed, it's included within the frame that needs it. Certain frames need no VOP data, since it was already included in a previous frame. In this case an N-VOP (nop? null?) is used. This is an empty VOP that takes up very little space.
There's no need to delay the playback, but on the other hand, the player has to be capable of decoding two VOPs during the time of a single frame.

Unpacking Packed Bitstream Files
I don't know if packed bitsream is better or not for encoders, players, developers, etc. I only know that my standalone DVD player can't handle packed bitstream, so I have to "unpack" such files. If you need this as well, I know of two options you may use:

• Use Moitah's Mpeg4 Modifier, either in its GUI or command line version.
• Use UnpackMP4, a Java conversion of Moitah's command line version. I created the Java port since I wanted to use it on Linux and didn't want to install Mono. Also, I was missing basic Linux behavior like being able to convert multiple files in a single call. When I ported the application, I also optimized it so that it runs faster than Moitah's version, but I have to addmit that the newest Mpeg4 Modifier runs a bit faster than my port (tested only on Windows).

Happy unpacking.