WO1993015586A1

WO1993015586A1 - A partial interpolation method and apparatus for frame rate conversion

Info

Publication number: WO1993015586A1
Application number: PCT/US1993/000808
Authority: WO
Inventors: Jennifer Clare Loveridge
Original assignee: Eastman Kodak Company
Priority date: 1992-02-03
Filing date: 1993-01-29
Publication date: 1993-08-05

Abstract

A frame rate converter is provided for use in film scanning apparatus of the type wherein pixels relating to input images derived from consecutive frames of film are sampled at a predetermined rate related to a film scanning speed, and wherein output pixel information is converted to output fields of a standard television signal. A frame store (18) holds input image pixel data corresponding to at least two consecutive frames of film (1, 2). Every fifth output field (3) is obtained by accessing the consecutive frames (1, 2) in the frame store (18) and interpolating between image pixel data, specifically by calculating an unweighted interframe average of the corresponding lines in the consecutive frames. Thereafter, a standard television signal is generated from the interpolated fifth field (3) and the intervening uninperpolated fields (1, 2, 4, 5).

Description

A PARTIAL INTERPOLATION METHOD AND APPARATUS FOR FRAME RATE CONVERSION

Field of the Invention

This invention relates to signal processing apparatus for converting the repetition rate of an image signal from a motion picture frame scanning rate to a television field scanning rate.

Background Art

In an electronic imaging system which is concerned with a time-varying sequence of images, it may be necessary to have different input and output frame rates. A typical film- to-video frame rate conversion is from a 24 film frames per second input signal to a 30 video frames per second(or 60 fields per second, interlaced) output signal. This is a requirement, for example, when the input medium is motion picture film and the output signal is displayed on a television monitor operating according to a ΝTSC television standard, that is, 525 lines/frame and 30 frames(60 fields)/second.

The following three known methods of frame-rate conversion may be used to obtain the necessary conversion: (1) The so-called "3,2" method is the most commonly used method in current telecine film scanners to obtain a 60 field per second interlaced television output from a 24 frame per second film input. This method is illustrated in Figure 1, which shows the replication of one in every four output fields. More particularly, a five field sequence is generated in which the first two fields are generated from the odd lines and the even lines of the first film frame, the third field repeats the odd lines (or the even lines) of the first film frame, and the remaining two fields are generated from the odd lines and the even lines of the second film frame. In other words, two or three output fields are generated alternately from each input frame. This produces the required frame-rate conversion but also leads to objectionable temporal artifacts, notably motion-judder. U.S. Patent No . 4,205,337, "Television Film Scanner", generally discloses a telecine scanner utilizing paired field stores to convert a line sequential scan into interlaced fields in the aforementioned(3:2) manner.

(2) Full interpolation between fields may be used to generate five output fields for every two input film frames. This has the advantage of overcoming the motion-judder inherent in the "3,2" method by creating sequential fields which are placed correctly in time. However, motion blur or image ghosting artifacts are produced in areas of image motion. Furthermore, in order to minimize the trade-off between loss of sharpness and temporal aliasing artifacts it is required that the interpolation filter should have a large number of taps, and, due to the resulting fra estore requirements, this method is expensive to implement in hardware. An example of this type of interpolation circuit is disclosed in U.S. Patent No. 4,607,282, "Television Circuit for the Reduction of Flicker on a Display" .

(3) Frame rate conversion methods employing motion- compensation implement temporal interpolation in a direction determined by a motion vector field. The motion vectors may be generated by several methods including block-matching and phase correlation. Effective motion-compensated algorithms are computationally complex and difficult to implement at high speed in hardware. In addition, they may suffer from noticeable failure in cases where motion vectors are incorrectly determined, for example at occlusion boundaries. Such methods are described in "The development of an HDTV to PAL standards converter and its picture quality", by T. Ohmura, Y. Tanaka, and T. Kurita, IBC Conference Proceedings, 1986, and in "European approaches and prospects on HDTV standards conversion", by D. Nasse, Symposium Record of 16th International TV Symposium, Montreux, Switzerland, 17-22 June 1989, pp. 472-483.

Clearly, it would be desireable to find a design approach that balances computational complexity and implementation speed, on the one hand, while minimizing image artifacts such as motion judder, loss of sharpness, temporal aliasing, and motion failures, on the other.

Summary of the Invention

The invention strikes a reasonable compromise between complexity and artifacts by using linear interpolation to — generate every fifth field in the aforementioned five field sequence(i.e. , every five consecutive fields are derived from every two consecutive film frames) . Accordingly, frame rate conversion apparatus is provided for use in film scanning apparatus of the type wherein pixels relating to input images derived from consecutive frames of film are sampled at a predetermined rate related to a film scanning speed, and wherein output pixel information is converted to output fields of a standard television signal. A frame store holds input image pixel data corresponding to at least two consecutive frames of film. Periodically, an output television field is obtained by accessing image pixel data stored in the frame store and interpolating between corresponding odd (or even) lines in selected input frames. Thereafter, a standard television signal is generated from the interpolated field and the intervening uninterpolated fields. Generally, in terms of a method,the conversion is from an even number of input frames to an odd number of output fields, comprising the steps of generating a first even number of output fields from a first input frame, interpolating an intermediate field from first input frame and a second input frame,and generating a second even number of output fields from the second input frame. More specifically the conversion is from two input frames to five output fields, wherein the intermediate field is an interfra e average of corresponding lines of the two input frames.

Brief description of the Drawings

The prior art and the invention will be described in relation to the drawings, wherein Figure 1 is a time sequence diagram of a known technique for frame rate conversion;

Figure 2 is a block diagram of a frame rate conversion apparatus incorporating partial interpolation in accordance with the invention; and Figure 3 is a time sequence diagram of the partial interpolation performed by the apparatus of Figure 2.

Description of the Preferred Embodiment

A signal processing channel for use in a telecine film scanner for frame rate conversion in accordance with the invention is shown in block form in Figure 2. A film scanner 10 develops image pixel data corresponding to image pixel areas on a motion picture film. A preferred film scanner 10 is described in U.S. Patent No . 5,045,932, "Method and

Apparatus for Generating A High Definition Electronic Image Signal from a Line Scan of a Color Original, " which is incorporated herein by reference. The signal processing channel comprises an analog-to-digital(A/D) converter 12, an input image store 14, a spatial interpolator 16, an output image store 18, and a frame rate interpolator 20. A controller 22 provides system timing for pipelined operation according to the scanning speed of the film scanner 10 and the television standard in use. Film pixel data from the film scanner 10 is converted to digital pixel data by the A/D converter 12 and applied to the input image store 14 at a real-time, synchronous rate determined by the controller 22.

The store 14 is capable of storing all of the pixels of at least one complete film frame. Since a preferred application, as set forth in the afore-mentioned '932 patent, is for generation of effects in connection with a HDTV signal, about 1600 lines of film information are stored when scanning a 35mm Academy format motion picture film. This is more than the 1035 active lines which are required for the proposed 1125 line HDTV standard because of the aspect ratio difference of the film and HDTV systems. While not an integral part of this invention, the spatial interpolator 16 can be used in television standards conversion, in particular, to decimate an input sequence of higher definition signals into an output sequence of lower definition signals. A suitable decimation filter is described in U.S. Patent No. 4,991,010, entitled "Dual-mode Image Interpolation Filter Operable in a First Mode for Storing Interpolation Coefficients and in a Second Mode for Effecting Television Standards Conversion at a Pixel Rate", which is incorporated herein by reference. If this capability is provided in the spatial interpolator 16, then it is necessary to provide a valid pixel signal from the controller 22 to the output image store 18 in order to identify the wanted output pixels. If decimation is not employed, basically all of the output pixels are valid.

The high definition output signal is applied to the image store 18, which includes a first framestore 18a and a second framestore 18b for storing two consecutive frames. Recalling now that the high definition signal to this point is a sequential signal, an interlace/interpolate controller 24 loads a video frame sequentially into one framestore while extracting video fields (of a previously loaded frame) in interlace format from the other framestore. A digital red, green, and blue high definition field signal is thus provided to the frame rate converter 20, which is controlled by the interlace/interpolate controller 24 to interpolate every fifth field and pass the other fields without modification. The output of the frame rate converter 20 is then available for further use, which may include immediate broadcast transmission or recording, e.g., on video tape(after suitable standards conversion or encoding, as necessary) . In any event, such further use is not to be part of the present invention.

The frame rate converter 20 performs a linear interpolation between fields as generally shown in the time sequence diagram of Figure 3. An output sequence consists of five television fields for every two input film frames (that is, for a conversion from 24 frames/second to 60 fields/second) . The first two fields of the output sequence are generated from the first input(film) frame by the usual interlace method; that is, the first field comprises the odd lines scanned from the first input frame, and the second field the even lines from the first input frame. Hence, fields 1 and 2 are displaced spatially by one line, but are sampled from substantially the same point in time. Output field three, which could be considered the first of two fields of the second output(video) frame, is generated by interpolation. Specifically, each line of the interpolated output field consists of an unweighted interframe average of the corresponding odd lines scanned from input(film) frames 1 and 2. The fourth and fifth output fields consist of the even and odd lines, respectively, from the second(film) frame. This process is then repeated for pairs of input(film) frames over the remainder of the input frames. Since the conversion is from an even number of input frames to an odd number of output fields, the lines used in the respective conversions, i.e., whether odd or even, will alternate for each five field sequence. Specifically, this means that the lines comprising every fifth interpolated field will alternate field-by-field between interframe averages of the odd lines and the even lines of two consecutive input frames.

The disclosed method is believed to be of particular advantage, for the following reasons: (1) The level of motion judder artifacts should be considerably lower than when the "3,2" method is employed. There are two sources of motion judder artifacts: first, due to temporal field replication, and secondly due to the use of input frames for unequal time intervals in the output sequence (as is the case in the "3,2" method, where the interval alternates between 1/20th and 1/30th second, respectively) . The present invention resolves the latter problem by employing each input frame for an equal interval, l/24th second, in the output sequence; (2) Only every fifth output field suffers from artifacts due to the interpolation process, as opposed to every field in the case of full interpolation. Hence the interpolation algorithm can be very simple, e.g., linear interpolation between two frames, and the resultant artifacts should not be perceptible; and

(3) The cost of the implementation of this algorithm in hardware should be comparable with that of the "3,2" method, and significantly cheaper than either the full interpolation or motion-compensated methods. The invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.

Claims

What is claimed is:

1. Frame rate conversion apparatus for use in film scanning apparatus of the type wherein pixels relating to input images derived from consecutive frames (1,2) of film are sampled at a predetermined rate related to a film scanning speed, and wherein output pixel information is converted to output fields (1,2,3,4,5) of a standard television signal, said frame rate conversion apparatus comprising: means (18) for storing input image pixel data corresponding to at least two frames (1,2) of film; means (20,24) responsive to said storing means (18) for interpolating between image pixel data corresponding to selected input frames (1,2) to periodically generate an output television field (3); and means (20) responsive to said interpolating means for generating a standard television signal from the interpolated field (3) and the intervening uninterpolated fields (1,2,4,5). 2. Apparatus as claimed in Claim 1 wherein said interpolating means (20,24) performs a linear interpolation between image pixel data corresponding to two consecutive image frames (1,

2) .

3. Apparatus as claimed in Claim 2 wherein said linear interpolation comprises an unweighted interframe average of corresponding lines from the two consecutive frames (1,2) .

4. Apparatus as claimed in Claim 3 wherein the frame rate conversion involves converting from an even number of input frames (1,2) to an odd number of output fields

(1,2,3,4,5) and wherein corresponding lines being averaged for the interpolated fields (3) alternately comprise odd lines and even lines from two consecutive frames (1,2).

5. Apparatus as claimed in Claim 1 wherein the interpolated field (3) is every fifth output field.

6. A method for converting between an input frame rate derived from scanning a motion picture film to an output field rate corresponding to a standard television signal, said conversion being from an.even number of input frames (1,2) to an odd number of output fields (1,2,3,4,5), said method comprising the steps of: generating a first even number of output fields (1,2) from a first input frame (1) ; interpolating an intermediate field (3) from the first input frame (1) and a second input frame (2); and generating a second even number of output fields (4,5) from the lines of a second input frame (2).

7. A method as claimed in Claim 6 wherein said conversion is from two input frames (1,2) to five output fields (1,2,3,4,5), and wherein said intermediate field (3) comprises every fifth output field.

8. A method as claimed in Claim 7 wherein said output fields alternate between two five field sequences, and wherein in the first sequence said first even number of output fields comprises a first field (1) and a second field (2) generated from the odd lines and the even lines of the first input frame (1) , said second even number of output fields comprises a fourth field (4) and a fifth field (5) ^■ generated from the even lines and the odd lines of the second input frame (2), and said intervening field (3) comprises an interpolation of the odd lines of both first and second input frames (1,2), and wherein in said second sequence said first even number of output fields comprises a first field (6) and a second field (7) generated from the even lines and the odd lines of a third input frame (3), said second even number of output fields comprises a fourth field (9) and a fifth field (10) generated from the odd lines and the even lines of a fourth input frame (4) , and said intervening field (8) comprises an interpolation of the even lines of both third and fourth input frames (3,4).

9. A method as claimed in Claim 6 wherein the interpolation is an interframe average of corresponding lines from the input frames.