Why Frame Rate Matters

I saw the following in MythTV's forum (MythTvTalk):

"Once the video is captured in the file (...), you are past any PAL/NTSC differences..."

While this is true in some aspects, it's very wrong with regards to one important aspect: frame rate. Why is frame rate important? Because there's no way to do interpolations on time.

Let's think about it this way - if your monitor is set to 1024 x 768 and the movie you're watching is using a different resolution, it's relatively easy to scale the movie to fit the entire screen - scaling using various methods of interpolation has been done a lot. In most cases, it's the video card that does the scaling, so it doesn't even affect your computer playback performance (scaling doesn't work well on interlaced video, which is one of the reasons I think interlaced video sucks, but that's the topic for another post).

If your monitor is set to 60Hz (it's practically showing 60 fps), and you're watching a PAL movie that runs at 25 frames per second, there's no way to interpolate 60 frames out of the existing 25 frames. Instead, during playback frames are repeated so that the overall speed of the movie is maintained.

Here's an easy example, and let's assume that NTSC runs at exactly 30 fps for simplicity: If your monitor is set to 60Hz, and you play a 30 fps NTSC movie, each frame is shown for two progressive scans of the screen - the frame rate is maintained exactly, and the quality and smoothness of the video is excellent.

But now try to watch a 25 fps PAL movie using the same monitor. Now you have to fit 25 original frames into 60 display frames. That's 5 original frames into 12 display frames (in 1/5 of a second). There's no way to spread the frames evenly. Let's assume an original frame switch is done as soon as it should be shown, but not within a display frame (otherwise you'll see a discontinuity at a certain vertical position, whenever there's horizontal motion between frames). That's the way DirectShow works on Windows. In this case, here's how 1/5 of a second would look like:



After this conversion, some of the frames would be shown for 1/30 of a seconds, and some would be shown for 1/20 of a second, there's actual a rhythm here: 3,2,3,2,2. This rhythm will repeat itself 5 times per second throughout the video you're seeing. The visual result of this is called judder, and is especially common when converting films to NTSC.

How annoying is this jitter? Many people wouldn't be able to spot the problem, but the video would looks a bit jumpy. It's especially visible in scenes of consistent (and slow) horizontal movement of the camera (panning).

Connecting your PC to a TV

What if you have a TV-Out option on your display card, and you hook your PC to a TV? Will this solve the problem? The TV shows 25 fps, and the movie also has 25 fps, but there's no way to skip the 60 fps of the display adapter (actually, some display adapters lock the frame rate to something sensible when the TV-Out is turned on, but let's assume you can feed a PAL TV from a display card showing 60 fps).

In this case the result is even worse. The display adapter has to convert the 60 fps back to 25 fps without knowing the original material was in 25 fps. This would probably look like this:



What happened here? The first original frame (red) is shown without a problem, but by the time we need to show the second frame (on the TV), the second video frame (yellow) is not yet shown, so the red frame will be shown once more. The outcome is that out of every 5 frames, one will be repeated, and one will be skipped. This is a pretty noticeable judder.


What's the Solution?

In some cases it's quite simple - if you're using your PC to watch video, make sure the display refresh rate matches your video's frame rate. If you watch PAL movies, set your display to 75Hz or 100Hz. If you watch NTSC movies, it sounds like you should set your display to 60Hz (or 90Hz or 120Hz), but most NTSC shows are actually filmed at 24 fps (film rate), so the best frame rate is a multiplication of that, probably 72Hz.

Is that all?

No, of course not. Many people in Europe want to use their computers to watch both shows they recorded using a capture card (which will be recorded at 25 fps) and movies or US TV shows (which are recorded at 24 fps). How do you set your monitor's refresh rate in this case? I'll leave that to the next post.

DWR problem: InboundVariable error when combined with prototype.js

General
I'm using DWR at work and it's an excellent tool for AJAX (easily doing RPC from the client browser in JavaScript back to the Java Server). It also has good enough integration with Spring, which is a plus.

But for quite some time, we've had errors popping up in our logs every time we sent a JavaScript object to over DWR. The errors would look something like this:

2007-03-07 03:56:14,783 [ERROR] InboundVariable - Found reference to variable named 'c0-e6', but no variable of that name could be found.

To keep a long story short, if you happen to use the prototype.js library, it adds a function called 'extend' to the prototype of Object, so all objects have it. For some reason, DWR tries to marshal that 'extend' function as a property, but fails (it's a function), the the POST request that DWR sends, the 'extend' property is specified, but doesn't have a corresponding value.

In our case, we didn't acutally use prototype at that point, so we removed the
<script src="prototype-x.y.x.js"> from our JSP pages and the error was gone. If you need to fix this problem, but want to keep using prototype.js, you'll probably have to remove the 'extend' property from objects you send using DWR (or set the property to null).

BTW, if you ask me - it's a bug in DWR, since functions should not be marshaled. I'm using DWR 1.1.3, so this might have been fixed in 1.1.4, but I didn't see too much information about this using the almighty search engine.


How the problem was found
I used the LiveHttpHeaders extension for FireFox, and captured a DWR call. It seems very straightforward:

POST /Console/dwr/exec/XXXSearch.handleAddOrUpdate.dwr HTTP/1.1
Host: localhost:8085
User-Agent: Mozilla/5.0 (...) Gecko/20070216 Firefox/1.5.0.10
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9...
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: text/plain
Content-Length: 393
Cookie: JSESSIONID={don't tell}; Country=US; Language=en
Pragma: no-cache
Cache-Control: no-cache
callCount=1
c0-scriptName=XXXSearch
c0-methodName=handleAddOrUpdate
c0-id=8297_1173264435303
c0-param0=boolean:false
c0-e1=number:1
c0-e2=string:session.5
c0-e3=boolean:false
c0-e4=string:1
c0-e5=string:orcl
c0-param1=Object:{currRow:reference:c0-e1, factory:reference:c0-e2,
              orCondition:reference:c0-e3, cond:reference:c0-e4,
              values_0:reference:c0-e5, extend:reference:c0-e6}
xml=true

The request reminded me that DWR calls can be batched (hence the callCount field), and all the c0-{name} arguments describe call number 0, I assume I would have seen c1-{name} if I had more calls in this request.
I've also sent two parameters. The first (co-param0) is a boolean. The second (c0-param1) is complex, so it references other entries in the request, called c0-e{num}.
It's easy to see that 'extend' references 'c0-e6', that doesn't exist in the request.
I didn't know why 'extend' was there - I didn't send it, so I tried a short HTML with a DWR request, and indeed, it didn't send an 'extend' as part of the object I sent. Now, it was obvious this is done by one of the many JS files included in our real application page, and a quick search revealed the offender.