JP2005094286A - Method for converting frame rate utilizing three-dimensional image interpolation technology - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an algorithm for converting a frame rate utilizing a three-dimensional image (moving image) interpolation technology which regulates a screen resolution and the frame rate of received input sound/video image data based on the broadcasting standard of a video image broadcasting unit. <P>SOLUTION: A method for converting the frame rate utilizing the three-dimensional image interpolation technology includes a step of regulating the screen resolution and the frame rate of the inputted video image data based on the broadcasting standard of the video image broadcasting unit. The method further includes a step of extracting two screens continued according to a broadcasting sequence (100). The method also includes a step of deciding an intermediate position according to the frame rate (200). The method also includes a step of forming a plurality of interpolating pixels at the intermediate position from a corresponding pixel value in the two continued screens, and forming the interpolation screen with the interpolated pixel value (300). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a frame rate conversion method, and more particularly, to a frame rate conversion method using a three dimensions image interpolation algorithm.

  With the advancement of science and technology, the quality of audio and video broadcasting facilities has also evolved with the times. From the earliest video tapes to recent VCDs, DVDs, etc., recording media tend to be smaller in volume and larger in capacity. As a result, it has become possible to provide users and consumers with high-quality audio and video with a portable recording media unit. However, the computing power of the data processing unit installed in the audio / video broadcasting unit cannot be overlooked.

  Enormous audio / video broadcast data can be combined with a powerful arithmetic processing unit of an audio / video broadcast unit (for example, a television) at the same time to realize a high-resolution display and smooth image broadcasting.

  In general, the problem of differences in broadcasting standards is often encountered from the time of audio / video data capture to video broadcasting. For example, the problem is resolution or frame rate. Currently, the pixel value of the screen (frame) is adjusted mainly by adopting a two-dimensional (still image) image interpolation method, and a plurality of screens close to each other on the time axis are duplicated.

  For example, US Pat. No. 5,574,572 discloses a technique for changing a broadcast reference screen using space and time factors.

In FIG. 3, F1 and F2 are screens present in the audio / video broadcast content, that is, drawings in which pixels after interpolation calculation are expanded, and F1 ′ is a screen in which interpolation pixel points are formed on the time axis. The principle of interpolation is that if the screen interpolation position obtained from the execution rate of the frame rate is close to F1 between F1 and F2, the screen formed at that interpolation position is at the corresponding position on F1. By duplicating a certain pixel, interpolation is performed on the time axis to form an interpolation screen F1 ′. The other screens are derived in the same way.
US Pat. No. 5,574,572 US Pat. No. 5,621,870 US Pat. No. 5,089,893 US Pat. No. 5,283,651 US Pat. No. 5,384,904

  The above method can reduce the calculation time relatively and does not place an excessive burden on the system. The powerful arithmetic unit of the video broadcasting unit cannot be fully utilized, resulting in substantial waste of hardware in terms of audio / video broadcasting.

  Other inventions described in other U.S. Pat. No. 5,621,870, U.S. Pat. No. 5,089,893, U.S. Pat. No. 5,283,651 and U.S. Pat. No. 5,384,904 are also described above. Since the method is used, there is a problem that the viewer's appreciation is adversely affected by the deterioration of the broadcast quality.

  In view of this point, the present invention uses a three-dimensional (moving image) image interpolation technique that adjusts the screen resolution and frame rate of received input audio / video data based on the broadcast standard of the video broadcast unit. An algorithm for converting the frame rate is provided.

  That is, a step of extracting two consecutive screens according to the broadcast order, a step of determining an intermediate position according to the frame rate, and a plurality of interpolated pixel values at the intermediate position from corresponding pixel values in the two consecutive screens. The gist of the present invention is a frame rate conversion method using a three-dimensional image interpolation technique, which includes a step of creating an interpolation screen with the interpolation pixel value and a step of storing the interpolation screen.

The interpolated pixel value is calculated from the corresponding pixel values in two consecutive screens by the following formula: P _int = P _first * j / (i + j) + P _next * i / (i + j)
Where P _int is the interpolated pixel value, P _first and P _next are the pixel values for interpolation, and i and j are the ratios obtained by adjusting the frame rate. The variables are: i = (frame rate of input audio / video data) / (frame rate of audio / video player), j = 1−i, and the intermediate position is the frame rate of input audio / video data. And a frame rate of an audio / video player, and a frame rate conversion method using a three-dimensional image interpolation technique with an additional condition.

  Next, the best embodiment of the method for converting the frame rate using the 3D image interpolation technique of the present invention will be described.

  First, in FIG. 4, if the multimedia format of the video broadcast data storage destination of the video broadcast unit (TV or liquid crystal projector) is generally set to 600 (pixels) × 400 (pixels), the frame rate is set. Is 40fps (frame per second). However, if the broadcasting standard on the hardware side is 1200 (pixels) x 800 (pixels) and the output is required, and the frame rate is 80 fps, the audio / video broadcast unit will interpolate. The purpose of broadcasting must be achieved by a data processing method through an algorithm.

  Next, FIG. 5 is a position sketch in a method for obtaining an interpolation pixel using a pixel group in the screen. In the figure, a, b, c, and d are proportional values for obtaining P point from each of A, B, C, and D points.

  Take FIG. 4 as an example. When it is desired to display an image of a pixel amount with the length and width increased on the same size screen, the interval between the pixels must be increased or decreased according to the pixel amount. That is, when the image is enlarged by interpolation from 600 pixels to 1200 pixels in the horizontal direction, a represents a unit interval between two pixels on one side of the newly formed screen, that is, 600/1200. b is 1- (600/1200). For this reason, a and b show a first-order proportional relationship with the pixels on the original screen. Further, the pixel value of the virtual pixel point X can be determined from the pixel values of A and B.

That is: X = A * (b / a + b) + B * (a / a + b).
Similarly, the pixel value of the virtual pixel point Y on the other horizontal line is: Y = C * (b / a + b) + D * (a / a + b).

  Further, by using these virtual pixel points X and Y again, the interpolated pixel value of the point P is obtained based on the conversion of the pixel ratio on the screen (enlarged by interpolation from 400 pixels to 800 pixels in the vertical direction).

That is:
P = X * (d / c + d) + Y * (c / c + d) = (A * (b / a + b) + B * (a / a + b)) * (d / c + d ) + (C * (b / a + b) + D * (a / a + b)) * (c / c + d).

  This is the same logic as the calculation of two-dimensional image interpolation. The important point is that a new pixel point can be calculated by interpolation using surrounding pixel points, which meets the needs for the resolution of the audio / video broadcasting equipment. Each broadcast screen has pixel points interpolated through such a procedure, and then adjusts the frame rate again.

  FIG. 2 is a conceptual diagram showing an interpolation screen for frame rate conversion using the 3D image interpolation technique of the present invention.

  P1 to P10 are pixel points in the actual video broadcast data. P11 to P15 are interpolated pixel points interpolated through the method shown by the present invention. I and II are a section (block) of the screen composed of pixel points that actually exist in the video broadcast data. III is a section (block) in the screen consisting of interpolated pixel points interpolated by the method according to the present invention.

Variables i and j are ratios obtained by changing the frame rate on the time axis.
That is: i = (frame rate of the input video data) / (frame rate of the video broadcast unit)
j = 1−i
Here, a, b, c, and d are pixel span ratios obtained by modifying the resolution. According to the method of the present invention, the pixel span ratio of block III is obtained based on block I and block II. Can do.

  Reference is now made to FIGS. Here, a description will be given using blocks of the broadcast screen. The calculation formula for finding the interpolated pixels P5 and P10 is as follows:

P5 = (P1 * (b / a + b) + P2 * (a / a + b)) * (d / c + d) + (P3 * (b / a + b) + P4 * (a / a + b)) * (c / c + d),
P10 = (P6 * (b / a + b) + P7 * (a / a + b)) * (d / c + d) + (P8 * (b / a + b) + P9 * (a / a + b)) * (c / c + d)

  Thereafter, the screen after pixel interpolation is stored in the buffer memory of the video broadcast unit. Subsequently, two consecutive screens I and II are extracted in accordance with the broadcast order (step 100). An intermediate position is determined based on the frame rate (step 200). That is, if the frame rate on the screen of the original video broadcast data is 40 fps and it is desired to change it to 70 fps, the playing time between the two screens is reduced to 40/70 = 4/7. That is, in the video broadcast data, the interpolation screen III is inserted between the first display screen I and the second display screen II at a position slightly away from the first display screen I. The distance between screen III and screen I is 4/7 of the distance between screen II and screen I.

  Other interpolation screen positions are determined by the same method. As a result, the interpolation position on the other screen is analogized, and the interpolation pixels on the two consecutive screens are used to generate a plurality of interpolation pixel values at the intermediate position. These interpolated pixels constitute an interpolated screen (step 300).

These pixel values are calculated as follows:
P15 = P5 * j / (i + j) + P10 * i (i + j)
P15 is an interpolated pixel value, and this type of interpolated pixel value constitutes an interpolated screen III between two consecutive screens I and II. Finally, the interpolation screen III is stored in the memory of the video broadcast unit (step 400).
What has been described above is only a relatively excellent example of the present invention, and does not limit the scope of the present invention.

  Improving image quality in multimedia devices such as DVD players.

The flowchart which shows the frame rate conversion method using the three-dimensional image interpolation technique by this invention. The conceptual diagram which shows the interpolation screen of the frame rate conversion using the three-dimensional image interpolation technique by this invention. Explanatory drawing which shows the screen interpolation method by the conventional method. The conceptual diagram explaining the two-dimensional image interpolation method by the conventional method. Explanatory drawing which shows an interpolation pixel position using the pixel group in a screen by the conventional method.

Explanation of symbols

a, b, c, d: Space ratio of each pixel obtained by resolution conversion
A, B, C, D, P1, P2, P3, P4, P6, P7, P8, P9: Pixels present in video broadcast data
P5, P10: Interpolated pixels present in video broadcast data
P, P11, P12, P13, P14, P15: Interpolated pixels that make up the interpolation screen
X, Y: Virtual pixel
F1, F2: Screens present in audio / video broadcast content
F1 ': Screen formed by interpolated pixel points on the time axis

Claims (4)

Adjusts the screen resolution and frame rate of the input audio / image data based on the broadcast standard of the video broadcasting unit, and sets multiple interpolated pixel values to match the screen resolution from the multiple pixel values included in the screen And converting the frame rate using a 3D image interpolation technique to save the newly generated screen,
Extracting two consecutive screens according to the broadcast sequence;
Determining an intermediate position according to a frame rate;
Creating a plurality of interpolated pixel values at the intermediate position from corresponding pixel values in the two consecutive screens, forming an interpolated screen with the interpolated pixel values, and storing the interpolated screen A frame rate conversion method using a 3D image interpolation technique characterized by: The interpolated pixel value is calculated from the corresponding pixel value in two consecutive screens as follows:
P _int = P _first * j / (i + j) + P _next * i / (i + j)
(Here, P _int is an interpolated pixel value, P _first and P _next are pixel values for interpolation, and i and j are ratios obtained by adjusting the frame rate.)
The frame rate conversion method using the three-dimensional image interpolation technique according to claim 1, wherein: The variables in the above formula are:
i = (Frame rate of input audio / video data) / (Frame rate of audio / video player)
j = 1−i
The frame rate conversion method using the three-dimensional image interpolation technique according to claim 2.   2. The frame rate conversion using the three-dimensional image interpolation technique according to claim 1, wherein the intermediate position is determined based on a frame rate of input audio / video data and a frame rate of an audio / video player. Method.

Images