Guest post written by Ernie Sanchez, VP & COO, Cinnafilm
Content that originated at filmic rates of 23.976/24 FPS has been seen in broadcast for decades. Consequently, taking four frames at a filmic frame rate and generating five frames from them (29.97i or 30i) has been a regular occurrence in broadcast media for decades. Unfortunately, broken cadence can result during the process of preparing content for broadcast television.
Directors/producers typically want the “story telling effect” of a 180 degree shutter angle (shutter speed 2x the frame rate) because it best mimics how our eyes would naturally see the images. The problem is this filmic looking project will need to air on broadcast television – which of course is 29.97i (some people call this 59.94i, but this is field rate, not frame rate – more on that later).
That is when the idea of creating a pattern of five frames from four filmic frames came to fruition. The idea is to use a pattern to make those four frames turn into five so it can fit into a broadcast stream, and then the end device removes the pattern during the deinterlace process. The pattern is simple, but it is often broken during the editing process. However, NTSC broadcast playback is quite resilient and even though the pattern can be badly broken, it will still play back just fine. So, why should we care about broken patterned content?
OTT and conversion to PAL-based playback systems are why we should care. OTT is strictly a progressive format, and shows that currently exist in interlaced containers must be “progressified” in order to be seen without interlacing on phones, tablets, computers, you know, those things everyone has these days. PAL is 25fps based, and for any frame rate conversion to be accurate, duplicate frames/fields must be removed to return back to the original video essence frame rate. Everyone has seen the result of not properly removing these patterns. The two biggest clues are stuttered playback and interlacing artifacts (combing) in areas of motion.
To understand how to correct this situation, we need to first understand the patterns.
Interlaced Patterns
For every four frames at 23.976, we will need to take those four progressive images, and place them in a container that has two fields per frame at a frame rate of 29.97. There are two patterns we can apply to achieve an “interlaced looking” output:
In both of the above examples, there are two fields in every resulting frame. Different systems typically call these upper/lower fields (most common), field 1/2, or even/odd fields. The more popular of the two is the “2:3 pulldown” and it is sometimes referred to as “telecine.” With either pattern, there are two images inside each interlaced frame to adhere to the 29.97i broadcast standard.
- If the original filmic frame is repeated in both upper and lower fields, the image will look progressive.
- If the output frame has a mix of two original filmic frames, that frame will look interlaced (combing will be present).
Important note: In case you were wondering where the “2” and the “3” in 2:3 came from, it refers to how many times the original filmic image will be placed into a field position in the new stream. This is often called “the cadence” and “broken cadence” is when this pattern is disturbed due to editing.
Progressive Patterns
Sony invented a way of applying a pattern that keeps the file in a purely progressive state. What is unique is this pattern can be applied to both interlaced and progressive file containers, which is great, but each will have its own unique challenge when wanting to extract the original filmic video essence.
This pattern is quite simple, just repeat every fourth frame and keep everything looking progressive:
- Each original filmic frame will be placed into the upper and lower field of the resulting 29.97 video stream
- Every fourth original, filmic frame will be duplicated across two of the 29.97 frames
Important note: This pattern is known as the 2:2:2:4 where the “4” refers to how many times the original filmic “D” frame will be placed into a field.
It is important to note that this pattern can be mistaken for simple 29.97p, just placed in an interlaced container (known as 29.97i PsF). It is important that any time you have an interlaced file, and there are progressive images, that you step through a few frames to ensure there isn’t a repeating progressive pattern.
Keep in mind that 2:2:2:4 patterned 23.976 will also be edited, causing the cadence to be broken around scene cuts, so be sure to always step through at least a full second of video to ensure you don’t miss anything.
Removing Patterns – Even with Broken Cadence
Tachyon was created with a very powerful pattern analysis and removal engine to identify what type of cadence is present, if it is broken, and what the scene boundaries are for each scene. This enables Tachyon to extract the original video essence frame rate from just about every patterned file you can possibly throw at it – broken cadence and all! To invoke this powerful tool, simply check the “Allow Telecine/Pulldown Removal” button and it will remove patterns every time they are identified during analysis.
Note: Tachyon only removes fields/frames that are positively identified as pattern repeats. This is not a tool that once it starts, it assumes the cadence is not broken and removes according to where the pattern should be in the cadence. Tachyon verifies every repeat before it is ejected.
There are situations where Tachyon will not be able to extract the video essence, and that is if the file was malformed. The first two patterns we discussed are designed for interlaced (two fields per frame) containers ONLY. If you happen to see these patterns and the metadata says the file is progressive, IT MIGHT NOT BE FIXABLE! Let me explain:
- If the file just has incorrect metadata, and the upper/lower field separation is still intact, then this is just an issue of incorrect metadata. Typically this happens during an incorrect transwrap of the file. Tachyon can easily overcome this situation by invoking the Metadata Override setting using either the “Upper” or “Lower Field First” setting in the Tachyon profile. If the file is standard definition, set it to Lower Field First. If the file is HD, set it to Upper Field First and Tachyon will ignore the metadata and process the file correctly.
- If the 29.97i patterned file has been transcoded or rendered to 29.97p, then there will no longer be any separation of the upper/lower fields and the interlacing lines created when stacking the BC and CD frames will be “burned in” to the asset. The file has no hope of ever recovering to the original progressive video essence when this happens. Combing artifacts will just be a part of every downstream conversion going forward.
Since patterns can exist in progressive content, Tachyon’s Allow Telecine/Pulldown Removal setting will automatically adjust based on the scan type and remove progressive patterns just as well as interlaced patterns.
There are three options for Telecine/Pulldown removal, and they have to do with special use cases:
- Excessive fade up/out scene transitions. In these scenarios, turning on the “Increased Detection Sensitivity” option will reduce the window required to identify duplicate fields/frames. There is a very slight chance if this is always on that a false positive could be encountered.
- Composites/overlays where one of the images is either a different scan type or the pattern was not synced up prior to composite. In these situations, the matte would be progressive while the composite is interlaced or vice-versa. There will be some frames where both scan types line up, but the end result is after telecine/pattern removal, there are combing artifacts that would need a rotoscope to remove. Tachyon’s “Residual Combing Removal” algorithm is an automated roto-scoper that identifies combing caused by a field difference and removes it. Note: if the pattern has been “burned in” due to an improper render or transcode, this setting will not remove the combs.
- Anime presents special problems to footage. With the frame having just a very small section of movement (typically eyes or mouth) the frame can be mistaken as progressive or a duplicate. To combat this, we have the “Cartoon/Anime” checkbox that enables Tachyon to look for subtleties that it would normally not be looking for.
While these three algorithms can slow down processing a bit (1-2 FPS per feature), it is sometimes the only way to achieve a proper inverse pulldown to achieve the original video essence frame rate.
End Results – No More Broken Cadence
After using the Telecine/Pattern Removal feature in Tachyon, the end results will be the original video essence, ready to be viewed in OTT environments, or to be converted to another broadcast standard. In the case of a different broadcast standard, Tachyon will immediately convert via motion compensation to the new frame rate designated as it pulls the video essence out of the patterned source file – multiple renders are not needed.
Telecine and patterns are part of life when it comes to standards conversion and conversions for OTT. Compounding the situation would be when editors mix 59.94 fields, 29.97p, and patterned material in a single project. Look for a blog soon with the topic name of “Mixed Mode Nightmares” and we’ll dive into that topic. Until then, happy transcoding!
More Information
For more information about the integration of Cinnafilm Tachyon and Dark Energy with Telestream Vantage, you can read the solution brief posted on the Telestream website.
For more information about Cinnafilm Tachyon, you can go to the Cinnafilm website.