Sunday, December 27, 2009

Combing vs Aliasing: One Man's Eternal Struggle

WEAVE Deinterlacing:
Note the combing artifacts on the mouth.

BOB Deinterlacing:
Note the aliasing on... well, everything.

This, right here, is why nobody does progressive DVDs on SD animation: There is no regular 3:2 cadence, just a mix of random fields that, if you're lucky, you can manipulate into something that looks like progressive frames with just occasional interlacing artifacts left over. Experiments have proven time and time again that it's physically impossible to remove every instance of combing in Hininden Gausu (or even *most* of them, what with the constant temporal chroma blending), so I'm sure as fuck not going to bother trying to do this manually in TMPGEnc. I may not have a life, but that doesn't mean I don't still have my pride.

This is one of those rare times where the source is such a train-wreck I am going to have artifacts... the question is, which artifacts are less irritating? TIVTC has done a mostly fantastic job of getting rid of combed frames, and only dodgy scenes with excessive video editing seem to have problems like you see above. Using TDecimate afterwards sounded like a good idea... but that turned out to be less of a perfect plan than I'd hoped. You see, the final edits were done on interlaced video hardware, forever discarding and manipulating the frames outside of regular 3:2 cadence. So I considered BOB deinterlacing fucking everything and then using a Decimate filter to restore the original 24fps framerate, but quickly found that several special effects shots were actually animated at 30fps, so dropping frames would make those whole scenes look choppy.

MKV files are able to combine 24fps and 30fps, making these mixed frame-rates no problem. DVD has the ability to mix 24fps and 30fps, but only at the encoder stage. I can't make a VFR (Variable Framerate) file, feed that to CCE SP and then get a VFR DVD back - oh no, instead I have to feed an interlaced source to the MPEG encoder and then let its' god-awful 3:2 algorithms trip over the occasional progressive frame like a blasted blind pig in a shit house.

So could I do that? Use a decomb filter, crop/upscale the transfer, and then re-interlace it? Having tried scripts that promise to BOB deinterlace "perfectly" by doubling the frame-rate and then allow you to recreate the interlacing patterns, they simply don't work.

This is the atrocity I was treated to for my sin of thinking I was an encoding God:

IVTC on the original 480i source.
Damn, that looks out of phase doesn't it...

IVTC on the upscaled "Interlaced" signal.

I also realized that I was ruining the 30fps material by "re-interlacing" the material, since I'm manually applying 3:2 pulldown in a way that doesn't allow for variable frame-rates. And so it's back to TIVTC I crawl!

The only realistic solution on this "hybrid" material is to run a field-matching filter, but keep the signal at 30fps (doubled frames and all), and finally encode that as 30fps despite the material already having been field-matched. The downside is that when you try to deinterlace this crap bad things happen, because the two fields no longer "mate" the way that they used to - not just because you're using the wrong field, but also because the upscaling has warped the size of the scan lines themselves.

Honestly, how they upscale mixed cadence 480i to 1080i is nothing short of mind-boggling... and makes me wonder if stuff like Full Metal Panic and Kiddy Grade and other JP only upscaled Blu-ray do, in fact, have completely fucked up cadence errors.

Still, random bugs like this are making me throw everything in my digital tool box, to little gain...

100% Interlaced Frames.

Out of Phase Field Matching.

Those above artifacts are - at least some of the time - theoretically fixable with manual tweaking. The trouble is every scene has dozens - sometimes hundreds! - of these little errant fuckers just waiting to pop out and ruin my day. Since they're random - essentially based on noise and a constantly flipping TFF order - I would literally have to click through all 49,635 frames, manually fixing the cadence on anything that looks off as I go. The whole point was to NOT do a manual IVTC, wasn't it?

So these relatively infrequent interlacing errors are staying. I don't like that fact one bit, but the only other reliable option left is to deinterlace the entire OAV using some complex BOB filter, which at best would soften the shit out of an already limited resolution transfer, and at worse would cause even worse artifacts than what I'm already trying to fix.

Just so nobody thinks that there are some
perfect frames in Gausu. There aren't. EVER.

See all that combing in the color that doesn't affix itself to the grayscale? The ENTIRE FUCKING FEATURE has that. God, another week of this hideous shit and I'd gladly take a hacksaw to my own face before manually fixing frames that already have combing in them...