WO2001013647A9 - Method and apparatus for telecine detection - Google Patents
Method and apparatus for telecine detectionInfo
- Publication number
- WO2001013647A9 WO2001013647A9 PCT/US2000/040598 US0040598W WO0113647A9 WO 2001013647 A9 WO2001013647 A9 WO 2001013647A9 US 0040598 W US0040598 W US 0040598W WO 0113647 A9 WO0113647 A9 WO 0113647A9
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- field
- telecine
- video
- pixels
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0112—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level one of the standards corresponding to a cinematograph film standard
Definitions
- FIGS. 1A and IB show the telecine pattern of field duplication for interlaced formats and progressive formats, respectively. As shown, both types of formats create 5 fields for every 2 frames of the input film based material. However, the manner in which the fields are created is fundamentally different.
- FIG. 1 A In the interlaced format shown in FIG. 1 A, four film frames (FA, FB, FC, FD) are upconverted to ten video fields (top field 1, bottom field 1, top field.2, bottom field 2, top field 2, bottom field 3, top field 3, bottom field 4, top field 4, bottom field 4). This conversion is referred to as 3:2 pulldown. Each field contains only one half of the total information needed to represent the entire picture. Therefore, without frame rate conversion, two input film frames could be expected to create 4 fields, for a total unconverted field rate of 48 fields per second. To upconvert to 60 fields per second means that out of every ten fields, two are redundant. In this case, the redundant fields are top field 2 and bottom field 4. This gives a 20% field redundancy for interlaced formats.
- FIG. 1 A four film frames (FA, FB, FC, FD) are upconverted to ten video fields (top field 1, bottom field 1, top field.2, bottom field 2, top field 2, bottom field 3, top field 3, bottom field 4, top field 4, bottom field 4). This conversion is
- the labels U, S, and R stand for unique, source, and repeat fields, respectively, and are used to indicate that for correct field polarity, the duplicated field is actually a repeat of the field before last, rather than the immediately prior field.
- the S field is also unique, but is labeled differently from the other unique fields to show that it is the source of the R field. Fields must be duplicated in this way or else two bottom or top fields in a row would be produced, which would alter the field polarity of the incoming video stream.
- FIG. IB there is no splitting of the incoming film-based frames into separate fields, so that each incoming frame becomes an outgoing field.
- MPEG encoders include the capability to detect telecine mode for SDTN (standard definition television). This detection is fairly straightforward for an encoder, since it already has access to external field buffers for performing a pixel- by-pixel comparison of current and prior fields.
- encoding for higher video data rates such as HDTN (high-definition television) may entail using multiple encoders to encode the entire picture.
- Such a multiple encoder approach is disclosed in U.S. Patent Application No. 09/054,427 which is incorporated herein by reference. In that approach, the video images are divided into overlapping regions with each region being assigned a dedicated encoder.
- the present approach attempts to identify whether the incoming video is in telecine mode, and if so, to lock to the telecine pattern, so that redundant fields can be identified. These identified fields can then be dropped from the video bit stream.
- a method for detecting telecine mode in a sequence of video fields includes comparing a first video field and a second video field of the video sequence to determine if one of the first and second fields is a repeat field and declaring telecine mode if a sequence of repeat fields corresponds to a telecine pattern.
- the comparing includes comparing the first field and the second field to generate a difference indication and declaring a repeat field if the difference indication is less than a threshold.
- the declaration of telecine mode is monitored and the threshold is increased unless stable telecine mode detection occurs or a threshold ceiling is reached.
- An apparatus for detecting telecine mode in a sequence of video fields includes repeated field detection logic for comparing a first video field and a second video field of the video sequence to determine if one of the first and second fields is a repeat field and telecine pattern detection logic for declaring telecine mode if a sequence of repeat fields corresponds to a telecine pattern.
- the repeated field detection logic includes an accumulator for summing the pixels of the first field to generate a first pixel sum and for summing the pixels of the second field to generate a second pixel sum.
- the repeated field detection logic further includes a comparator for comparing the first and second pixel sums to generate a pixel sum difference indication and for declaring a repeat field if the pixel sum difference indication is less than a pixel sum threshold.
- the video fields are divided into subfields and a subfield difference indication is generated for each subfield. A repeat field is declared if all subfield difference indications are less than a corresponding subfield threshold.
- each pixel includes a luma value and a chroma value.
- First and second luma pixel values are summed to generate respective first and second luma pixel sums and first and second chroma pixel values are summed to generate respective first and second chroma pixel sums.
- the first and second luma pixel sums are compared to generate a luma difference indication and the first and second chroma pixel sums are compared to generate a chroma difference indication.
- a repeat field is declared if the luma difference indication is less than a luma threshold and the chroma difference indication is less than a chroma threshold.
- a first pixel grid is applied to the first video field to provide a selection of first pixels; the pixel grid is applied to the second video field to provide a selection of second pixels; each pixel of the selection of first pixels is compared with the corresponding pixel of the selection of second pixels to generate a corresponding first pixel difference indication; and a repeat field is declared if all of the first pixel difference indications are less than a pixel threshold.
- a second pixel grid that is centered about a field and is smaller than the first pixel grid is applied in parallel to the video fields along with the first pixel grid to provide corresponding second pixel difference indications.
- the telecine pattern detection logic includes a first state machine for tracking occurrence of repeat fields to generate a first signal indicative of acquisition of telecine mode and a second state machine for tracking occurrence of repeat fields to generate a second signal indicative of loss of telecine mode.
- the telecine pattern detection logic further includes a switch responsive to the first and second signals to set and reset, respectively, a telecine detection signal indicative of telecine mode status.
- the system looks at a full HDTN image and instructs multiple encoders simultaneously when to drop fields. This helps ensure a seamless transition into and out of telecine mode.
- FIG. 1 A illustrates telecine field duplication for interlaced scan video format.
- FIG. IB illustrates telecine field duplication for progressive scan video format.
- FIG. 2 is a schematic block diagram of a high-definition television encoding system which includes a telecine detection circuit.
- FIG. 3 is a schematic block diagram of the telecine detection circuit of FIG. 2 which includes repeated field detection and telecine pattern detection logic.
- FIG. 4 illustrates an example of a repeated field pattern in a progressive scan video format.
- FIGs. 5 A and 5B are schematic block diagrams of an embodiment of the repeated field detection logic of FIG. 3.
- FIG. 6 is a schematic block diagram of an embodiment of the telecine pattern detection logic of FIG. 3.
- FIGs. 7 A and 7B show processing of input luma and chroma streams in accumulators provided in the repeated field detection logic of FIG. 5 A.
- FIG. 8 shows a main pixel grid and a middle pixel grid provided in the repeated field detection logic of FIG. 5 A.
- FIG. 9 shows a window comparison block provided in the repeated field detection logic of FIG 5 A.
- FIG. 10 is a state diagram for an acquisition state machine for progressive scan video format.
- FIG. 11 is a state diagram for a tracking state machine for progressive scan video format.
- FIG. 12 is a state diagram for an acquisition state machine for interlaced scan video format.
- FIG. 13 is a state diagram for a tracking state machine for interlaced scan video format.
- FIG. 14 is a flow diagram of an automatic windowing approach.
- FIG. 2 An exemplary HDTV encoding system embodying the principles of the invention is shown in FIG. 2.
- the system includes a telecine detection circuit 10, video splitter 11, microprocessor 12, and encoders 14-1 to 14-N.
- the telecine detection circuit has inputs for external digital video signal 22 and a window signal 16 from microprocessor 12.
- the telecine detection circuit attempts to identify whether the incoming video is in telecine mode, and if so, to lock to the telecine pattern, so that redundant fields can be identified. These identified fields can then be dropped from the video bit stream.
- the telecine detection is performed on the input video stream 22 prior to encoding. That is, the input video stream as seen by the telecine detection circuit represents the source video before compression, and before it is split by video splitter 11 into independent video streams which are sent to the individual encoders.
- a field_drop signal 18 and telecine_history signal 20 are output from the telecine detection circuit 10 and fed into microprocessor 12.
- the telecinejhistory signal 20 and the window_value signal 16 are used to enable automatic telecine detection in varying quality video sequences. This aspect is discussed in more detail further herein.
- the field_drop signal 18 can be used to create an interrupt or the microprocessor can poll this signal periodically, e.g., at the start of every field.
- An interrupt is preferred, since the microprocessor only needs to sample this signal once every five fields (interlaced) or three times out of five fields (progressive) if the video is truly in telecine mode. Seeing this signal high, the microprocessor can then write to registers within the MPEG encoders 14-1 to 14-N which instruct the encoders to perform the task of dropping and/or reordering the incoming fields.
- the field_drop signal 18 is output only when the telecine detection circuit 10 has determined that the incoming video is film-based, i.e., the video is in telecine mode. Therefore, only repeat fields from a valid telecine sequence which have the proper phase will cause a field to be dropped.
- the telecine detection circuit 10 for detecting incoming telecine video includes two major functional blocks: repeated field detection logic 30 and telecine pattern detection logic 32.
- the repeated field detection logic 30 receives window_value signal 16 and the digital video input 22 (FIG. 2) as composite video signals (luma signal 22Y and chroma signal 22C) and synchronization signals (vertical sync 22N and horizontal sync 22H).
- the detection logic 30 provides a field_match signal 31 to the pattern detection logic 32 which generates the field_drop signal 18.
- the pattern detection logic also provides the telecine_history signal 20.
- An optimal method for repeated field detection uses one or more field buffers to store all of the pixels within given fields.
- the contents of the current field are then compared to the contents of the previous or next to last field (depending on whether the target format is progressive or interlaced) on a pixel-by-pixel basis.
- a sum of pixel differences or low pass filtering is used to reduce inter-field noise.
- This noise can be produced from a variety of sources, one of which is prior multiple encode-decode passes on the input video as noted herein.
- the differences should be small, since large pixel differences would actually become visible in the video as it is displayed.
- a less accurate method that is realizable in programmable logic, e.g., a field programmable gate array (FPGA), and does not require external field buffers comprises adding together all pixels in the current field, storing the sum in an accumulator, and then comparing it with the sum from the last field. This is referred to herein as the pixel accumulation method. Fields which are duplicates should have somewhat similar sums. However, a problem with this method is that small pixel differences can add up very quickly (e.g., there are over 2 million luma (Y) and chroma (C) values in a 1080i field), thereby producing large variations between fields, even those which are supposed to be repeats.
- Y luma
- C chroma
- Another method realizable in an FPGA is to use the limited RAM available in the FPGA to perform a partial pixel comparison that in effect is a scaled down version of the field buffer approach.
- This partial pixel comparison method attempts to strategically pick the pixels which are most likely to be in the path of motion, and compares those pixels with the associated pixels from the previous field. Since most motion will occur in the center of an image, the pixels to be compared might typically be in a grid which spans a large percentage of the image. An additional grid can be used that is smaller and located in the middle of the image where most of the motion will occur.
- ⁇ is the number of pixels in a field
- x and y are pixels from the current and prior fields. While this is an effective method for statistically comparing all pixels within a field, it is not the optimum method when only a limited pixel grid is available.
- a video sequence shows a man staring at a computer screen intently, slowly tapping his nose with his index finger.
- the differences between fields in progressive mode or alternate fields in interlaced mode are minimal, consisting of only the slow motion of a finger.
- a sum of differences approach may indicate all fields as matched, since the amount of motion is small. This small accumulated pixel difference may fall well within the sum of differences threshold value, especially if there is little inter-field noise in the rest of the image.
- a more accurate method in this case is to assign each pixel a threshold, and if any of the pixels exceeds its assigned threshold, the field match test fails.
- This approach can detect very small amounts of motion as in the case cited above, because with almost any type of motion, there will be at least one pixel which is significantly altered from its value in the previous field. In the case cited above, a pixel in the path of the moving finger would indicate a large change from its value in the prior field. Thus, the field match test fails even if every other pixel in the grid indicates a perfect match.
- a particular embodiment of the repeated field detection logic 30 uses both the pixel accumulation and partial pixel comparison methods to increase the likelihood of an accurate field match comparison.
- the pixel accumulation includes breaking the field into four subgroups or sections and storing four different sets of accumulator values in order to increase the accuracy of the comparison.
- the partial pixel comparison uses two pixel grids, one large and widely spaced pixel grid which attempts to cover a large percentage of the field, and a separate, smaller grid centered about the middle of the field to increase motion detecting capability in the center of the image. Taken together, the two methods can realize over 99.9% accuracy in detecting repeat fields within telecine sequences.
- the telecine pattern detection logic 32 determines whether the input video is derived from film by searching for the proper telecine pattern among the repeated fields detected by the repeated field detection logic 30.
- the pattern detection logic tests to see first that several valid telecine patterns are received before allowing fields to be dropped.
- the field_drop signal 18 is enabled only for fields which fall into the proper phase of the established telecine pattern, so that fields from stationary video sequences are not also dropped inadvertently unless they have the proper telecine phase.
- the telecine pattern is different for interlaced and progressive formats, since it is governed by the difference between the 24 field per second input rate and the output field rate.
- the two patterns are now described further.
- the telecine pattern to track in the incoming sequence of video fields is as follows: repeat unique repeat unique repeat repeat unique repeat unique repeat unique repeat....
- the pattern can be represented more concisely as follows:
- the telecine pattern detection logic 32 (FIG. 3) must be able to handle both stationary video (the most common case), and occasional mis-identified fields.
- Stationary video is handled by maintaining the current setting during stationary sequences, i.e., if telecine mode has not yet been detected, then do not enter it, and if telecine mode has been detected and entered, do not exit it. In other words, stationary video has no effect on the current state of the system, since it is an indeterminate state which does not provide enough information about the input video to determine whether it is a telecine sequence.
- the occasional misidentified field is handled by assuming it is real when not in telecine mode, and allowing for one such field every N valid sequences when in telecine mode. In other words, if the system is not yet in telecine mode but is searching for the pattern, a 0 where a 1 should be located (i.e,. a unique field where a repeat was expected) causes the logic to fall out of the current pattern, and the sequence is declared invalid. If a 0 is received where a 1 was expected but the system is in telecine mode, then this error is disregarded if several more valid telecine sequences are subsequently received.
- the telecine detection 5 logic is made immune to small errors, but a continual pattern of invalid sequences causes the system to fall out of telecine mode.
- fields will not be dropped as long as the field_match signal 31 (FIG. 3) detects that the fields are different. Therefore, for video with normal amounts of motion, a several 10 sequence delay in exiting telecine mode will not cause video artifacts, since even while telecine mode is on, fields which are detected to be not matched will not be dropped.
- some non-redundant fields may be dropped when transitioning from telecine to normal video. However, because the video is stationary, these dropped fields do not cause any noticeable change in the image.
- a duplicate field is a repeat of the field before last, as opposed to the immediately previous field in progressive formats.
- the repeat field pattern can be represented by the following:
- U represents a unique field
- S represents the source field
- R represents the repeat field, which is a duplicate of the source field.
- the source field is also a unique field, but it is designated differently here in order to show that the repeat field is a repeat of the field before last. As mentioned previously, this is done to ensure that the proper field polarity is retained, since if the duplicate field was a repeat of the immediately previous field, there would be two top or bottom fields in a row.
- a standard way to detect the telecine pattern and account for the one field skip between repeat fields is to maintain two field buffers, thereby incurring a two field delay. This approach allows downstream logic to compare the contents of the current field with that of two fields previous.
- a first option is to use two sets of accumulator registers, and to use only half the area covered by the pixel grids, in order to be able to store the accumulator values and pixel grid values from two fields simultaneously. This option essentially provides a two field delay for detection purposes, although the fields are not delayed as they move through the system.
- a second option is to operate the detection logic such that latching occurs only on alternate fields, thereby ignoring intermediate fields. According to this option, a full telecine pattern would take 10 field periods to manifest itself to the detection logic, since if only alternate fields are latched, it takes 10 fields to repeat the telecine pattern.
- a drawback with the first option is that it requires that the area covered by the pixel grids be cut in half, in order to make room for the results from two fields simultaneously.
- the pixel grids may akeady be widely spaced due to limited RAM associated with the detection logic and are optimized to detect motion given the amount of RAM available. Cutting such pixel grids in half may degrade the accuracy of the field match logic.
- the second option would maintain the accuracy of the field match logic, but would take twice as long to lock to the telecine pattern, or to detect loss of lock.
- a preferred approach relates to the first option.
- the pixel grids used in an embodiment for progressive formats store both the luma and chroma values at all grid points, essentially requiring two bytes of memory for each pixel (4:2:2 input video).
- This allows two fields worth of pixel grids having the same original grid dimensions to be entirely stored in the RAM of the detection logic, so that same polarity fields can be compared.
- storing only the luma components in progressive mode allows for a doubling of the original grid dimensions.
- doubling the grid does not seem to affect the accuracy of the pixel match function in a noticeable manner.
- the detection logic is also simplified if the grid dimensions for all formats are kept identical, changing only the number of bytes per pixel stored.
- two sets of accumulator registers are kept in logic, in order to be able to store the accumulator values of all fields, and two sets of pixel grid points are also kept.
- Top and bottom fields from two preceding field periods can then be compared against the current field.
- Each polarity field has its own control logic, so that there is a separate state machine for both top and bottom fields, which simplifies the state diagrams and other logic.
- the telecine detection on both top and bottom fields contributes equally to the acquisition and tracking mode operation.
- the acquisition and tracking logic in this embodiment searches for the following telecine pattern when the input format is interlaced:
- FIGs. 5 A and 5B show schematic block diagrams of a particular embodiment of the repeated field detection logic 30 (FIG. 3) which can be implemented in an FPGA.
- the repeated field detection logic includes accumulator control 120, pixel accumulation section 122, partial pixel comparison section 124 and field match section 126. Each of these sections is now described in detail.
- the accumulator control 120 provides a latching signal denoted latch natch 121 which is a timing signal derived from the horizontal and vertical sync signals 22H, 22N.
- the latchjnatch signal 121 provides a signal transition corresponding to each field period for latching results of the windowing blocks 110- A to 110-D and 118 described further herein.
- the pixel accumulation section 122 includes accumulators A to D (102-A to 102-D) which provide pixel accumulation for repeated field detection; registers 104, 106; multiplexers 108 and windowing blocks 110-A to 110-D.
- accumulator A 102-A takes every other pair of luma pixels starting at luma pixels 1 and 2
- accumulator B 102-B takes the alternate pairs of luma pixels, starting at pixels 3 and 4.
- Accumulators C and D 102-C, 102-D perform the identical function for the chroma pixels.
- Providing multiple accumulators and splitting the pixels between them serves two purposes: 1) the accuracy of the field match logic is enhanced by splitting the field into subgroups; and 2) system timing is enhanced by ping-ponging pixels between accumulators, thereby allowing two clock cycles for the accumulator output to become valid.
- the accumulator results are stored in a two stage pipeline comprising registers 104 and 106, in order to hold the pixel sums from the two previous fields.
- interlaced formats e.g., 1080i
- the accumulated output from field N-l in register 106 and the current field 103 are used as inputs to the windowing block (110- A to 110-D), while for progressive formats (e.g., 720p) the output from field N in register 104 and the current field 103 are used as inputs.
- the selection between formats is made in multiplexer 108 based on an interlacedprogressive selection signal 101.
- This selection signal 101 allows the appropriate field match detect signal described further herein to be formed by comparing the current field with the immediately prior field or the field before that, depending on whether the input format is interlaced or progressive.
- the windowing block 110-A to 110-D detects whether the pixel sum of the current field falls within +/- Win/2 of the pixel sum of the previous or next to previous field, and if it does, a field match for that particular subgroup is declared.
- the value Win/2 is provided to the repeated field detection logic 30 (FIG. 3) on window_value signal 16.
- There are four field match signals denoted as Amatch, Bmatch, Cmatch, and Dmatch which represent the progressive luma even and odd and chroma even and odd field match indicators, respectively.
- Similar signals are used to signal top and bottom field matches, respectively, when the input format is interlaced. All the accumulator match signals are formed in the same manner, the only difference is that the interlaced field match signals are formed by comparing the results from two field times in the past, and are broken up into top and bottom field match signals, so they can be fed into their respective state machines described further herein.
- the partial pixel comparison section 124 includes main grid and middle grid pixel compare blocks 126-1 and 126-2, respectively.
- Each pixel compare block 126- 1, 126-2 includes a pixel grid control 112, round block 114, respective grid AMs 116-1, 116-2 and windowing block 118.
- Each pixel in the respective grid is compared against the associated pixel from the previous field. If the pixel of the current field falls within +/- Pix_Win/2 of the associated pixel from the previous field, then a pixel match for that pixel is declared.
- Field match signals denoted FldPixMch and FldMidMch are used to indicate field match for the main and middle grid pixel compare blocks 126-1, 126-2, respectively for progressive formats.
- field match signals TopFldPixMch, BotFldPixMch, TopFldMidMch, BotFldMidMch are used to signal top and bottom field matches, respectively, for the corresponding main and middle grid compare blocks.
- the field match signals are brought high at the beginning of every field period. If any of the pixels in the current field do not match that from the previous field, then the field match signal for the associated grid is brought low (no match) until the begimiing of the next field period. Therefore, even one non-matched pixel (after thresholding) will cause the field match test to fail.
- a chroma_enable signal is output from the pixel grid control logic 112 to enable the chroma portion of each grid pixel to be written only for progressive formats.
- the pixel grid control 112 also provides read and write control of the respective RAMs 116-1, 116-2. Both the main and middle grid blocks 126-1, 126-2 work identically, but use different positioning constants and different RAM banks 116-1 and 116-2, respectively.
- FIG. 5B shows the field match section 126 which includes AND gates 128-1, 128-2 and 128-3.
- the final field jnatch signals denoted FMjprog 130-1, Top_FM 130-2 and Bot_FM 130-3 correspond to the field jnatch signal 31 (FIG. 3).
- Each signal is the logical AND of six individual field match signals corresponding to the pixel accumulator section 122 and the partial pixel comparison section 124.
- the progressive format signals are coupled to gate 128-1 while the top and bottom field signals for interlaced format are coupled to gates 128-2 and 128-2, respectively.
- the field match signal FM_prog 130-1 is the logical AND of signals
- Amatch, Bmatch, Cmatch, Dmatch, FldPixMch and FldMidMch The match signals FMjprog, Top_FM and Bot_FM are used as inputs to the progressive and interlaced acquisition/tracking state machines described further herein.
- FIG. 6 shows a block diagram of the acquisition and tracking logic.
- the logic includes acquisition section 202 and tracking section 204.
- the acquisition section 202 includes interlaced acquisition state machine 206 and progressive acquisition state machine 208.
- the tracking section 204 includes interlaced tracking state machine 218 and progressive tracking state machine 226.
- the acquisition and tracking state machines are functions of both the top and bottom field logic, which both contribute equally to the decision making process. They are split up in this way in order to simplify the state logic, which then has to look only for the pattern 0 0 0 0 1 to detect telecine mode, where 1 is a field match.
- the interlaced acquisition state machine 206 includes top and bottom field blocks 210 and 212, respectively, which receive corresponding input field match signals Top_FM 130-2 and Bot_FM 130-3.
- the output of each field block 210, 212 is input to a counter 214.
- the output of counter 214 provides an input to multiplexer 216.
- the progressive acquisition state machine 208 receives input field match signal FMjprog 130-1 and provides an output to multiplexer 216.
- the multiplexer 216 selects between the respective outputs of the interlaced and progressive acquisition state machines 206, 208 responsive to frit/Prog select signal 101 to provide pattern_det signal 217.
- the interlaced tracking state machine 218 includes top and bottom field blocks 220 and 222, respectively, which receive corresponding input field match signals Top_FM 130-2 and Bot_FM 130-3.
- the output of each field block 220, 222 is input to a counter 224.
- the output of counter 224 provides an input to multiplexer 228.
- the progressive tracking state machine 226 receives input field match signal FMjprog 130-1 and provides an output to multiplexer 228.
- the multiplexer 228 selects between the respective outputs of the interlaced and progressive tracking state machines 218, 226 responsive to Int/Prog select signal 101 to provide pattern oss signal 229.
- the pattern_det signal 217 from the acquisition section 202 and the pattemjloss signal 229 from the tracking section 204 are input to the respective S and R inputs of SR flip-flop device 230.
- the Q output of SR flip-flop device 230 provides a telecine iet signal 231.
- the field lrop signal 18 is the logical AND of telecine ⁇ et 231 and pattern_sync signal 233.
- the telecine let signal 231 when high indicates establishment of telecine mode, i.e., an indication that the input video is film-based, and is set by the acquisition state machine.
- the pattern_sync signal 233 is only output at the proper phase in the telecine pattern, so that the combination of the telecine iet and pattern_sync signals ensures that only repeat fields of the proper phase in film-based video can be dropped from the stream.
- FIGs. 5 A, 5B and 6 are now described in more detail.
- FIGs. 7 A and 7B show how the pixel streams 22 Y, 22C are split among accumulators A to D (102-A to 102-D) in an embodiment.
- the first pair of luma (Ya, Yb) and chroma (Ca, Cb) pixels is input to accumulators A and C, respectively, whereas the second pair (luma Yc, Yd and chroma Cc, Cd) is input to accumulators B and D, respectively.
- subsequent pairs alternate between accumulators.
- Pixels are grouped into pairs because upstream interface logic (not shown) moves luma and chroma pixels tlirough the pipeline in groups of two in order to reduce the 74MHz input clock rate to a more manageable 37 MHZ.
- Each pixel is 10 bits in an embodiment. Each pair of 10 bit pixels is first added together to form an 11 bit result (not shown), and the 11 bit sum is then passed to the accumulators. It should be noted that other ways of forming and processing subgroups of pixels can used.
- FIG. 8 shows the pixel grids used in the pixel comparison portion of the repeated field detection logic 30 (FIG. 3).
- a main grid 302 which occupies a vertical length of 500 lines and a horizontal width of 1000 pixels
- a middle grid 304 which occupies a vertical length of 250 lines and a horizontal width of 1000 pixels.
- Both grids have a spacing of 20 pixels in the horizontal direction 318 and 25 lines in the vertical direction 320. In other words, both grids contain points every 20 pixels horizontally and every 25 lines vertically.
- the middle grid 304 is no more dense than the main grid 302; however, the middle grid is offset from the main grid both horizontally and vertically.
- the middle grid starts 10 pixels to the right of the main grid start pixel 306, thereby filling in the horizontal spaces.
- the middle grid still has a vertical spacing of 25 lines, it is offset from the main grid by 12 lines.
- the end result is that the middle 250 lines of the field denoted at 322 are more densely covered due to the overlapping of the main and middle grids. This is desirable, since most motion will occur at or near the center of the image.
- the offsets must be different (i.e., the horizontal and vertical start and stop points for the grids), in order to fully center them.
- the main grid start and end pixels are denoted at 306, 308, respectively, and the start and end lines are denoted at 310, 316.
- the start and end pixels are offset by 12 pixels from the main grid as noted above.
- the start and end lines for the middle grid are denoted at 312, 314.
- pixels are stored into their associated RAMs 116-1, 116-2 (FIG. 5A), they are rounded from 10 to 8 bits in blocks 114 to minimize RAM utilization and to allow for the largest number of pixels possible to be compared. Note that in order to declare a pixel field match, all the pixels from both grids must match the associated pixels from the previous field plus or minus a user-defined pixel window.
- FIG. 9 An embodiment of the windowing blocks 110-A to 110-D and 118 (FIG. 5 A) is shown in FIG. 9.
- the condition being tested for by the windowing block is the following:
- test is whether the accumulator or pixel value of the current field lies within a certain negative or positive distance from the accumulator or pixel value of the previous field. Since this type of equation is cumbersome to implement in an FPGA, the actual (exactly equivalent) expression implemented is:
- ABS Previous - Current
- the accumulator or pixel value of the current field is subtracted from that of the previous field in block 402.
- the absolute value of the difference is taken in block 404 and then this value is compared to the user defined window in block 406.
- the user variable written to the FPGA actually represents Window / 2.
- window values for the accumulator and pixels window functions since the former must be 32 bits wide, and the latter is only 8 bits wide (the pixels are rounded to 8 bits before being stored in RAM 116-1, 116-2 in FIG. 5A).
- the telecine pattern detection logic 32 (FIG. 3) is broken up into two separate functions; acquisition and tracking. This is done because even though both functions are looking for the same telecine sequence, they take different actions if the pattern is not seen. Since the telecine pattern is different for progressive and interlaced formats, the acquisition and tracking logic is further split into interlaced and progressive logic modules. The following describes acquisition and tracking for both types of formats.
- FIG. 10 shows a state diagram for the acquisition state machine 208 (FIG. 6) used for progressive formats.
- the acquisition state machine determines whether the incoming video is in telecine mode and increments a counter (not shown) each time a valid telecine sequence is seen. If the counter reaches a user-defined value, then the pattern let signal 217 (FIG. 6) is brought high which in turn sets the telecine let signal 231. The notation for transitions between states is shown as
- states SO through S4 represent the case in which a valid telecine sequence is seen. That is, to get from state SO to SI, the field match signal FMjprog 130-1 (FIG. 5B) has to be 1, to get from SI to S2 it has to be 0, to get from S2 to S3 it has to be 1, to get from S3 to S4 it has to be 0, and to increment the valid sequence counter in S4, it has to be a 1. This represents an entire telecine pattern of 10101. The other states in FIG. 10 represent deviations from the ideal telecine pattern.
- States S5 through S7 represent the case where one of the fields which was supposed to be unique (i.e., non-repeat) was detected to be a repeat field, in which case there may be stationary video. In that case, there is not enough information to determine if this group of fields belongs to a telecine sequence. Therefore, the valid counter is not cleared if it has already been incremented, nor is it incremented. From this operation it can be seen that telecine mode will not be entered into until enough motion occurs in the video to produce several valid sequences.
- FIG. 11 shows the state diagram for the progressive telecine tracking state machine 226 (FIG. 6). This machine is only started once the system is in telecine mode, and has the task of verifying that valid telecine sequences continue to be received.
- the field_sync signal indicated above is used to synchronize the tracking state machine with the start of the telecine pattern, and will only go high at the beginning of the first telecine sequence to be received in telecine mode (i.e., the beginning of the first telecine sequence when the telecine detect bit is high).
- the notation for transitions between states is shown as Telecine, FieldSync, FieldMatch/IncInvalid, IncNalid.
- a counter is incremented. If four valid telecine sequences in a row are subsequently received, the counter is cleared, on the assumption that it was an aberration. Otherwise, if another invalid sequence is seen before having seen four consecutive valid ones, the telecine signal is cleared and acquisition mode is re-entered. Note that if the input stream has truly gone back to normal video, the delay in transitioning out of telecine mode will not cause video artifacts. This is because even while the system is still in telecine mode, a field match indication of 0 (not matched) will cause the field not to be dropped. However, once the system has transitioned out of telecine mode, no fields will be dropped regardless of the status of the field match signal FMjprog.
- the state of the telecine detect signal is only sampled in states S5, S8, or SI 2, which represent the end of a valid, uncertain, or invalid sequence, respectively.
- states S5, S8, or SI 2 represent the end of a valid, uncertain, or invalid sequence, respectively.
- the tracking state machine will increment the invalid sequence counter one last time, which several clock cycles later has the effect of dropping the telecine detect signal low.
- the tracking state machine has akeady transitioned back to state 1 to start the next sequence. It is only when the tracking state machine again reaches state S5, S8, or S12 with the telecine detect signal low that it will return to state 0 and re-enter acquisition mode. Although this has the effect of adding a five field delay for the tracking state machine, the telecine detect signal still transitions immediately (without the five field delay).
- the pattern_sync signal is not shown in FIG. 11. This signal takes on the value of field match when the tracking state machine is in one of the tenninal states S5 or S8. It is used externally to form the field_drop signal 18 (FIG. 6) which is only brought high at the proper point in the telecine pattern.
- FIG. 12 shows the state diagram for the acquisition mode state machine 202
- FIG. 6 used for interlaced formats.
- acquisition state machines There are actually two identical acquisition state machines, one for top fields 210, and one for bottom fields 212, both of which contribute equally to the acquisition function. Only one state machine diagram is shown in FIG. 12. The notation for transitions between states is shown as Fieldjnatch Inc_cnrr, Clr_cntr.
- the acquisition state logic for interlaced formats is somewhat less complicated than for progressive formats, due to the fact that locking to the wrong phase of the sequence is not possible. This is true because there is only one repeated field in every five field sequence, so if a repeat is seen and the incoming sequence is not stationary video (i.e., only one field out of five is a repeat), the pattern must be at the last field in the five field sequence.
- the states in FIG. 12 correspond to top or bottom fields only, so that it actually takes ten field periods to cycle through states SI through S5.
- a transition from state SO to SI occurs only when a repeat field is detected.
- the state machine looks for pattern of no repeats for the next four top fields, followed by a repeat (states SI through S5). If this pattern is seen, a counter 214 (FIG. 6) is incremented, and a transition is made back to state SI to look for the next pattern. If field match is 0 in state S5, this is not a valid telecine pattern, so the state logic transitions back to state SO to await the next repeat field.
- stationary video has its own separate path, as represented by states S6 - S9. Stationary video sequences do not increment the valid pattern counter, since there is not enough information to determine if this is a telecine pattern. However, if in state S9 a field match value of 0 is received, the valid pattern counter is cleared, since it is determined that the current sequence is not a valid telecine pattern.
- FIG. 13 shows the state diagram for the interlaced tracking state machine 204 (FIG. 6).
- the state machine diagram shown is actually duplicated for both top and bottom fields; however, only one state diagram is shown in FIG. 13.
- each state transition actually represents two field periods, since each state machine will only handle either top or bottom fields. If either state machine detects a non-repeat field where a repeat field is expected, a counter increment signal is output. There are two counter increment signals, one each from the top and bottom field tracking state machines. Either signal will increment the pattern loss counter, and it is only cleared after four valid telecine patterns have been detected from both the top and bottom tracking state machines.
- the tracking state machine remains in state SO as long as the telecine signal is 0, i.e., during acquisition mode.
- the telecine_det signal goes high and the field sync signal goes high (indicating that the system is in telecine mode and this field is the first one in the pattern), then there is a transition to state SI to start the pattern.
- States SI through S5 represent the valid telecine pattern of 0 0 0 0 1. If the field match signal To ⁇ _FM or Bot_FM 130-2, 130-3 (FIG. 5B) is 0 when state S5 is reached, the invalid counter is incremented; otherwise, the valid counter is incremented.
- the invalid pattern counter is only cleared when the valid counter reaches 4, i.e., when four valid consecutive sequences are then seen. If the invalid counter reaches 2, then the telecine detect signal is cleared, and the next time the tracking state machine hits state S5, it will transition back to state SO, which represents acquisition mode.
- the field match signal Top_FM or BotJFM (FIG. 5B) is 1, then stationary video is assumed, as represented by states S6 through S9. If in state S9 the field match signal is 1, then the logic transitions back to state SI without incrementing the counter, since the sequence is uncertain. If in state S9 the field match signal is 0, then the invalid counter is incremented, just as it is in state S5. If the telecine detect signal is low in state S9, the sequence transitions back to state SO to begin acquisition mode. To reduce the complexity of the diagram, the pattern_sync signal is not shown in FIG. 13. This signal takes on the value of field match when the tracking state machine is in one of the terminal states S5 or S9. It is used externally to form the field kop signal 18 (FIG. 6), so that field drop is only brought high at the proper point in the telecine pattern. Automatic Windowing Using Feedback
- an automatic system includes a feedback from the telecine detection circuit 10 to the microprocessor so that the microprocessor can increase the window value 16 until a stable telecine pattern is detected.
- the automatic windowing approach samples transitions on the telecine detect signal every ten fields. This ten field sample period is derived from the fact that if the system is akeady in telecine mode, it takes two invalid sequences to fall out of telecine mode, which at 5 fields per sequence, yields a total often fields.
- the number of valid sequences necessary to declare telecine mode is a user-programmable number; however, it is assumed that this number should not be less than 2. Therefore, the telecine bit should not change more often than every ten fields.
- the microprocessor 12 at block 502 writes a nominal window value to the telecine detection circuit 10, which begins to increment an internal high counter (not shown).
- the high count represents the number often-field increments for which the telecine detect signal was high (telecine detected).
- the telecine detect circuit 10 latches the counter value into a holding register, and interrupts the microprocessor (while clearing the counter in preparation for the next 10 second interval).
- the microprocessor 12 then reads the counter value.
- the microprocessor increases the window value by some amount at block 512 and waits for the next interrupt. This continues until one of two events happens: 1) the system detects a stable telecine pattern (i.e., the telecine high counter reads 60) or 2) a predetermined window ceiling is reached at block 508 in which case, the microprocessor "rolls" the window value back to the bottom at block 510, and starts over.
- the microprocessor will end up continually cycling through a range of window values without seeing the telecine detect bit high, since no window value will result in a telecine pattern being detected if the video is truly not in telecine mode. Note that it is very important that the "rollover" window value is chosen carefully, and should not be too high. If a very high window value is written while the telecine detect bit is high, then most subsequent fields will appear to be repeated. If the video then transitions to non-telecine mode (such as during a commercial), the telecine detection circuit may not detect the transition quickly, since most fields will falsely appear to be repeated. This will result in unsatisfactory video. The proper "rollover" window value can be determined by careful testing.
- the window value requked is the nominal value (as determined from experimentation with various video sources)
- a steady telecine mode will be entered into very quickly.
- the video is especially noisy, it may take a few seconds for the software to settle on the conect window value.
- the embodiments described herein are applicable to both HDTV and SDTV formats, with modification of system parameters for pixel comparison functions which keep pixel grids centered about the middle of the field.
- a primary design consideration is that the telecine detection be provided entirely within programmable logic using no external memory. While this requires some significant optimizations and tradeoffs from the traditional multiple field buffer approach, it nevertheless is quite accurate, having a better than 99.9% accuracy in detecting matched fields within a telecine sequence (given the proper threshold window, as determined from software feedback).
- the cost savings incurred by using only a single programmable logic device to perform the telecine detection are especially apparent with HDTV systems, where two 4:2:2 1080i field buffers require over 4 Mbytes of memory. Performing statistical analysis across multiple fields requires even more memory, which may become prohibitive in HDTV systems.
- using field buffers as delay elements has the effect of adding some amount of latency to the system.
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU77583/00A AU7758300A (en) | 1999-08-17 | 2000-08-08 | Method and apparatus for telecine detection |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US37597699A | 1999-08-17 | 1999-08-17 | |
US09/375,976 | 1999-08-17 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2001013647A1 WO2001013647A1 (en) | 2001-02-22 |
WO2001013647A9 true WO2001013647A9 (en) | 2002-08-08 |
Family
ID=23483167
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/040598 WO2001013647A1 (en) | 1999-08-17 | 2000-08-08 | Method and apparatus for telecine detection |
Country Status (2)
Country | Link |
---|---|
AU (1) | AU7758300A (en) |
WO (1) | WO2001013647A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040090554A1 (en) * | 2002-10-23 | 2004-05-13 | Takahiro Nishi | Picture coding method |
KR100530223B1 (en) | 2003-05-13 | 2005-11-22 | 삼성전자주식회사 | Frame interpolation method and apparatus at frame rate conversion |
WO2008081386A1 (en) * | 2007-01-03 | 2008-07-10 | Koninklijke Philips Electronics N.V. | Film cadence detection |
GB2468358A (en) | 2009-03-06 | 2010-09-08 | Snell & Wilcox Ltd | Regional film cadence detection |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB9001079D0 (en) * | 1990-01-17 | 1990-03-14 | Avesco Plc | Standards conversion |
US5317398A (en) * | 1992-08-17 | 1994-05-31 | Rca Thomson Licensing Corporation | Video/film-mode (3:2 pulldown) detector using patterns of two-field differences |
JP3443880B2 (en) * | 1992-09-18 | 2003-09-08 | ソニー株式会社 | Video signal encoding method and decoding method |
US5828786A (en) * | 1993-12-02 | 1998-10-27 | General Instrument Corporation | Analyzer and methods for detecting and processing video data types in a video data stream |
JP3094903B2 (en) * | 1995-06-08 | 2000-10-03 | 松下電器産業株式会社 | Television signal conversion device and image coding device |
-
2000
- 2000-08-08 WO PCT/US2000/040598 patent/WO2001013647A1/en active Application Filing
- 2000-08-08 AU AU77583/00A patent/AU7758300A/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
WO2001013647A1 (en) | 2001-02-22 |
AU7758300A (en) | 2001-03-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7916784B2 (en) | Method and system for inverse telecine and field pairing | |
CN1106108C (en) | Method for generating variable frequency synchronous signal, and apparatus thereof | |
US6236806B1 (en) | Field detection apparatus and method, image coding apparatus and method, recording medium, recording method and transmission method | |
US6891571B2 (en) | Method and apparatus for improving video quality | |
KR950014343B1 (en) | Method of and apparatus for motion estimation of video data | |
US6144410A (en) | Telecine signal conversion method and an up-converter | |
JPH02199970A (en) | System of determinating segnentral position of video field which is obtained from film | |
JPH0654346A (en) | Video memory device | |
EP0772353A2 (en) | An apparatus for displaying a plurality of compressed images | |
WO2001013647A9 (en) | Method and apparatus for telecine detection | |
USRE38720E1 (en) | Image motion compensating address generator | |
US20030048853A1 (en) | Multi-channel image encoding method and system | |
US4951143A (en) | Memory configuration for unsynchronized input and output data streams | |
JP2550856B2 (en) | Video coding preprocessor | |
EP0392618A1 (en) | Apparatus for generating a horizontal reset signal synchronous with a subcarrier locked clock | |
EP1606954B1 (en) | Arrangement for generating a 3d video signal | |
US6791621B2 (en) | Image processing apparatus and method and signal switching output device | |
US4907072A (en) | Mosaic picture generation circuit | |
US4903127A (en) | Field generator with incomplete line correction | |
US4825288A (en) | Method and apparatus for processing video signals | |
KR20010083459A (en) | Multi-channel image encoding system and method for operating memory blocks therein | |
KR20010083460A (en) | Multi-chennel image encoding system using synch priority detection | |
JPH0865639A (en) | Image processor | |
JPS63245084A (en) | Interlace picture data conversion system | |
JPH01256880A (en) | Signal processor for video format signal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
AK | Designated states |
Kind code of ref document: C2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: C2 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
COP | Corrected version of pamphlet |
Free format text: PAGES 1/9-9/9, DRAWINGS, REPLACED BY NEW PAGES 1/15-15/15; DUE TO LATE TRANSMITTAL BY THE RECEIVINGOFFICE |
|
122 | Ep: pct app. not ent. europ. phase | ||
NENP | Non-entry into the national phase in: |
Ref country code: JP |