JPH0478284A - Frame rate converter - Google Patents

Abstract

PURPOSE:To simplify the frame rate conversion by weighting a moving vector with a coefficient depending on a frame number before and after the conversion and averaging the result of weighting. CONSTITUTION:A coefficient is multiplied with a data read from a frame mem ory 23 at a multiplier 35 and a coefficient is multiplied with a data read from a frame memory 24 at a multiplier 36. Then after outputs of the multipliers 35,36 are added by an adder 37 and a coefficient 1/5 is multiplied with the sum by a multiplier 38. The picture data obtained in this way is written in a frame memory 39. A frame number counter 40 counts inputted frame signals and circulates coefficients for the multipliers 28, 29, 35, 36 depending on the count and writes picture data FD(A)-FD(E) to a frame memory 39.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention converts the number of frames in the NTSC system to the number of frames in the PAL system, and converts the number of frames of a movie film into the number of frames in the NTSC system.
The present invention relates to a frame rate conversion device suitable for use in converting the number of frames of a system.

[Conventional technology]

FIG. 7 shows the configuration of an example of a conventional image data processing device.

This device basically consists of an encoder 1 and a decoder 2. The encoder 1 includes a frame memory 3.4, a motion detection circuit 5, a motion compensation circuit 6. It is composed of a motion vector selection circuit 7, a multiplexer (MPX) 8, and an encoding circuit 9. Also.

The decoder 2 includes a decoding circuit 11. Day multiplex f (
DMPX) 12, frame memories 13 to 15, a motion compensation circuit 16, and a multiplexer (MPX) 17.

Image data supplied from a circuit (not shown) is written into frame memories 3 and 4 frame by frame. The motion detection circuit 5 extracts image data in a predetermined range (for example, nXn pixels) from the image data written in the frame memories 3 and 4, and detects the direction and amount of movement (motion vector). The motion compensation circuit 6 compensates the image data according to the motion vector and outputs the compensated image data to the multiplexer 8. Also, the mode in which motion compensation was performed at this time is also the multiplexer 8.
supplied to Furthermore, motion vectors necessary for motion compensation are selected by the motion vector selection circuit 7 and supplied to the multiplexer 8.

The multiplexer 8 compiles the input image data, mode data, and motion vector (MV) data into a predetermined format, and outputs it to the encoding circuit 9. The encoding circuit 9 encodes the input data according to a predetermined rule,
It is transmitted to the decoding circuit 11.

Next, two motion compensation operations will be explained with reference to FIG.

Now, for example, if the interframe distance m is 3, frame 0.3,
6. ... are core frames, the data of these frames is encoded by intra-frame processing. On the other hand, inter frame 1.2 (4, 5) is encoded by motion compensating core frame 0.3 (3, 6).

For example, when frame 1 is most similar to core frame O or core frame 3, motion compensation is performed (image data is generated) from the motion vectors of these frames. If it does not approximate either core frames O or 3, intra-frame processing is performed. Furthermore, when core frames O and 3 are added and approximated to the averaged value, image data is generated corresponding to the motion vector from the frame of the same friend. Which of these motion compensation has been performed is encoded as mode data.

The decoding circuit 11 decodes the data input from the encoding circuit 9 and outputs it to the demultiplexer 12 . The day multiplexer 12 day formats the input data and separates it into image data, mode data, and MV data. Image data is sequentially written into frame memories 13,14. The mode data and MV data are transferred to the motion compensation circuit 16.
is input. The motion compensation circuit 16 uses mode data and MV
The image data of the block corresponding to the data is read out from the frame memory 13, 14 and motion compensated.

The image data obtained by this motion compensation is written into the frame memory 15. The multiplexer 17 selects the image data read out from the frame memory 14 or 15,
Output.

[Problem to be solved by the invention]

However, this conventional device can be used for example at 24
If you try to apply it to a frame rate conversion device that converts a film image of frames to an NTSC image of 30 frames (60 fields) per second, the transmitted MV data will only be that necessary for encoding. Therefore, it is necessary to newly detect a motion vector from the image data output from the multiplexer 17, which makes the configuration complicated and disadvantageous.

The present invention has been made in view of this situation, and it is an object of the present invention to enable frame rate conversion with a simple configuration.

[Means for Solving the Problem] The frame rate conversion device of the present invention detects a motion vector between core frames at a distance m (m≧2) from input image data, and detects a motion vector between core frames with a distance of m (m≧2), In a frame rate conversion device that includes an encoder that outputs the image data together with the image data, and a decoder that converts the image data input from the encoder from the number of frames Q to the number of frames P using a motion vector, the encoder converts the core frame The decoder has a motion vector memory that stores the motion vectors transmitted from the decoder, and a block address generator that generates an address corresponding to a block of motion vectors. a multiplier that multiplies a motion vector by a coefficient determined by the number of frames Q to P, a frame memory that stores image data of at least two consecutive frames, and an image corresponding to the motion vector corrected by the multiplier. data,
It is characterized by having an averaging circuit that performs weighting using a coefficient determined by the number of frames Q and P, and performs averaging.

[Effect]

In the frame rate conversion device configured as described above, all motion vectors between core frames are transmitted to the decoder side.

The motion vector is then weighted with a coefficient determined according to the number of frames before and after conversion.

averaged. Therefore, frame rate conversion is possible with a simple configuration.

〔Example〕

FIG. 1 is a block diagram showing the configuration of an embodiment of a frame rate conversion device of the present invention.

Encoder 21 and decoder 22 basically correspond to encoder 1 and decoder 2 shown in FIG. However, the encoder 21 is not provided with the motion vector selection circuit 7 shown in FIG. 7, and all motion vector (MV) data is transmitted to the decoder 22, not just the motion vectors necessary for encoding. It looks like this.

The decoder 22 motion-compensates the image data also input from the encoder 21 using the motion vector and mode data input from the encoder 21, and sequentially outputs the motion-compensated image data to frame memories 23 and 24. and have it written. That is, the frame memories 23 and 24 store image data of two consecutive still image frames.

On the other hand, motion vector data output from the decoder 22 is input to the motion vector memory 25 and stored therein. The block address generator 34 generates an address corresponding to a block of motion vectors of n×n pixels and supplies it to the motion vector memory 25 and also to the frame memory 23.24 via an adder 32.33.

The motion vector memory 25 includes frame memories 23 and 24.
The motion vectors between the core frames immediately before and after the frame stored in are output to the multiplier 26. Multiplier 26
multiplies the input motion vector by a coefficient 1/m (m is the core interframe distance n), and outputs the multiplication result to the subsequent multiplier 28. The multiplier 28 outputs the count value output by the frame number counter 4o The coefficient K corresponding to . . . KP-x is cycled and one of them is selected. Then, the input motion vector data is multiplied by the selected coefficient.

The output of multiplier 28 is further input to multiplier 30.

Multiplied by a factor of 1/P. This value P corresponds to the number of frames of the output image (the number of frames after conversion).

The output of the multiplier 30 is input to the adder 32, added to the block address input from the block address generator 34, and output to the frame memory 23.

Similar to the multipliers 26, 28, 30, multipliers 27, 29, 31 are provided to process frame data stored in the frame memory 24.

The image data read from the frame memory 23 is input to a multiplier 35, and multiplied by one of predetermined coefficients Kp-o to Ko. These coefficients are.

As in the case of the multiplier 28, it cycles in accordance with the count value of the frame number counter 40. Multiply! The output of 35 is input to an adder 37.

A multiplier 36 is provided corresponding to the multiplier 35, and outputs a predetermined coefficient from 0 to Kp-, to the output of the frame memory 24.
After being multiplied by one of the following, the output is output to the adder 37. The adder 37 adds the outputs of the multipliers 35 and 36 and outputs the result to the 1 multiplier 38. The multiplier 38 multiplies the input data by a coefficient 1/P and outputs the result to the frame memory 39. Frame memory 39 and frame number counter 4
A frame signal is input to O.

Note that the multipliers 35, 36, and 38 and the adder 37 constitute an averaging circuit.

The distance between core frames m is 3, and the number of frames before conversion Q is 6.
, when the number of frames P after conversion is 5.

The main parts of the circuit shown in FIG. 1 are shown in FIG. That is, the coefficients L/m in the multipliers 26 and 27 are each ±1/3, and the coefficient 1/P of the multiplier 30°31.38 is 115. Also.

Coefficient K of multipliers 28 and 36. 〜KP-0 is 0〜4
, the coefficients Kp-□ to 0 of multipliers 29 and 35 are from 5 to 1.
becomes.

Next, the operation of the embodiment shown in FIG. 2 will be explained with reference to FIG.

Since the distance m between core frames is now 3, frames 0 and 3.6 of the input image are taken as core frames. For example, if the image data of frame 1 is stored in the frame memory 23,
Assuming that the image data of frame 2 is stored in the frame 24, at that timing, the motion vector memory 25 stores 5 frames O and 3, which are the core frames immediately before and after frames 1 and 2.
The motion vector MVO between is stored. This motion vector MVO is input to a multiplier 26.

Multiplyed by a coefficient 1/m (=1/3). This results in
The motion vector between frame 1 and frame 2 is determined. Assuming that the timings of frame ○ and frame A match, the frames of input image data (frames 1 to 6) are compared to the frames of octagonal image data (frames A to F).

The distance between each frame is shifted by 175. This amount of deviation is 0.115. ．． 215.315°415.0
．． 115. ...and changes sequentially. Frame B is 115 and 9 in the direction of motion vector MVO with respect to frame 1. Therefore, the coefficient 1 is selected in the 1 multiplier 28. As a result, together with the coefficient 115 of the multiplier 30,
The motion vector (1/3) MVO is multiplied by the coefficient 115 to obtain (1/3)XMVOX (115). Also, at this time, the block address generator 34 generates motion vectors (1/3) M, V
Address F(1) of block of frame 1 corresponding to O
Since the following calculation is performed in the adder 32, the following calculation is performed.

F(L)+(MVOXl)/(3X5) Since the address shown in the above formula is input from the adder 32, the image data specified by this address is sent from the frame memory 23.

IMD rF(1)+(MVOxl)/(3X5)) is read.

On the other hand, as shown in FIG.
Since there is a shift of 415, which is the interframe distance, in the opposite direction to the motion vector (1/3) MVO, the amount of shift can be expressed by the following equation.

F (2)-(MVOX4)/(3X5) As in the above case, multipliers 27, 29.31 and adder 33
executes the operation in the above equation. As a result, the data specified by this address from the frame memory 24, IMD I:F(2)-(MVOX4)/(3X5))
is read out.

The image of frame B is obtained by combining the images of frame 1 and frame 2, but the influence on the distance between frames 1 and 2 is the distance between frame 1 and frame B, and the distance between frame 2 and frame B. It is thought that it is inversely proportional to . That is, image data FD of frame B (B)
is obtained by calculating a linear equation.

FD(B)=(415)IMD[F(1)+(MVOx
l)/(3X 5))+(115)IMDrF(2)
-(MvOX 4)/(3 is multiplied by a coefficient of 1. Then, the outputs of the multipliers 35 and 36 are added together in an adder 37, and then multiplied by a coefficient 115 in a multiplier 38. In this way, the obtained image data is written into the frame memory 39.

The frame number counter 40 counts the input frame signals, and the multipliers 28, 29, 3 correspond to the count value.
Cycle the coefficient of 5.36. In this way, as shown by the linear equation, image data FD (A) of frames A to E
FD (E) to FD (E) are written into the frame memory 39.

FD (A)= (515) IMD [F (0)+
(MVOX O)/ (315))+(015)IM
D[F(1)-(MVOx5)/(3x5))FD(B
)”(415)IMD[F(1)+(MVOX 1)/
(3X5))+(115)IMD[F(2)-(MVO
X4)/(3X 5))FD(C)”(315)IMD
[F(2)+(MVOx2)/(3X5))+(215
)IMD(F(3)-(MVOX 3)/(3x 5)
)FD(D)=(215)IMD[F(3)+(MVI
X 3)/(3X 5))+ (315)IMD(F
(4)-(MVI x 2)/(3x 5)]FD(
E)=(115)IMD(F(4)+(MVI X4)
/(3X5))+(415)IMD(F(5)-(MV
I x 1)/(3x 5)) FIG. 4 shows the configuration of another embodiment. In this embodiment, FIG.
The motion vector memory 25 in FIG. It is composed of an interpolation processing circuit 52. In the embodiment shown in FIG. 1 (FIG. 2), the block address generated by the block address generator 34 is a block address of nXn pixels as a unit of motion vector processing; The block address generated by the unit 54 is rXr(r<
n) It is a block address of a pixel. The other configurations are the same as those in FIG. 1 (FIG. 2).

The relationship between blocks A to d of nXn pixels and blocks a to d of rXr pixels is shown in FIG. 5, for example.

That is, in this case, r=n/2.

Then, when a block B exists on the right side of a block A of nXn pixels, and a block D exists on the lower right of a block C1 below, a block a of rXr pixels is arranged in the center of the block A.

As a result, the left half of block b on the right side of block a belongs to block A, and the right half belongs to block B.

Similarly, the upper half of the block C adjacent to the block a belongs to the block A, and the lower half belongs to the block C. Furthermore, block d adjacent to the lower right of block a
1/4 of each belongs to blocks A to A.

Therefore, in the case of the embodiment shown in FIG. 4, the motion vector output from the decoder 22 is input to the motion vector memory 51 and stored therein. Then, each motion vector from block A to block A is MVA.

MVB, MVC, MVD, the motion vectors of blocks a to d are respectively M V a , M V b , M
When V c and M V d, the interpolation processing circuit 52
reads the motion vectors MVA to MVD from the motion vector memory 51 and calculates the motion vectors MVa to MV from the linear equation.
Calculate d.

MV a = MV A MVb = (MVA + MVB) / 2 MVc = (MVA + MVC) / 2 MVd = (MVA + MVB + MVC + MVD) / 4 The motion vector of the rXr pixel obtained in this way is shown in Fig. 1 (Fig. The process is the same as in Figure 2).

In this way, by reducing the size of the block to be interpolated, it becomes possible to deal with even minute movements.

Note that the area around 1 (for example, rx (r
The motion vector of the area 2) can be the same as that of the block inside it (block a).

FIG. 6 shows yet another embodiment.

In this embodiment, the outputs of multipliers 35, 36 are fed to adder 37 via multipliers 62, 63, respectively. The coefficients of the multipliers 62 and 63 are controlled in accordance with the output of the mode information memory 61. The other configurations are the same as those in FIG. 1 (FIG. 2).

The mode information output by the decoder 22 is stored in the mode information memory 6.
1 is stored. As explained earlier with reference to FIG. 8, there are four types of modes. For example, in FIG. 3, when frame 1 is motion compensated from frames 0 and 3, in the first mode, frame 1 is independently intra-frame processed. This is the case when frame 1 is a completely different image from frames O and 3. In the second mode, frame 1 is motion compensated from frame 0, and in the third mode, frame 1 is motion compensated from frame 3. This is the case where one image is completely switched between frame 1 and frame 3 or between frame O and frame 1. Furthermore, in a fourth mode, frame 1 is motion compensated from both frame O and frame 3. This is the case where one image from frame O to frame 3 is related.

Corresponding to these four modes, the coefficients of the 1 multipliers 62 and 63 are set as shown in Table 1.

In Table 1 mode 2, the coefficient of multiplier 63 is set to 0 and the coefficient of multiplier 62 is set to P, so only motion compensation from the previous frame is performed. Also, in the case of mote 3, the coefficient of the multiplier 62 is set to 0, and the coefficient of the multiplier 63 is set to P.
, so only motion compensation from later frames is performed.

In modes 1 and 4, the coefficients of both multipliers 62 and 63 are set to 1, so that the same processing as in FIG. 1 (FIG. 2) is performed.

〔Effect of the invention〕

As described above, according to the frame rate conversion device of the present invention,
All motion vectors between core frames are transmitted to the decoder side, and the motion vectors are weighted and averaged with coefficients determined according to the number of frames before and after conversion, so frame rates can be converted with a simple configuration. becomes possible.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of an embodiment of the frame rate conversion device of the present invention, FIG. 2 is a block diagram when predetermined conditions are set in the embodiment of FIG. A frame conversion diagram explaining the operation of the embodiment shown in the figure, FIGS. 4 and 6 are block diagrams showing the configuration of other embodiments of the frame rate conversion device of the present invention, and FIG. 5 is an embodiment of the embodiment shown in FIG. FIG. 7 is a block diagram showing the configuration of an example of a conventional image processing apparatus, and FIG. 8 is a diagram explaining the operation of the example shown in FIG. 1.21...Encoder, 2,22...Decoder, 23°24.39...Frame memory, 25,51,53...
・・Motion vector memory, 2671J to 31, 35, 3
6.38... Multiplier, 34... Block address counter, 40... Frame number counter. Patent applicant: Victor Japan Co., Ltd.

Claims (1)

[Scope of Claims] An encoder that detects a motion vector between core frames at a distance m (m≧2) from input image data, and outputs mode data representing a mode of the motion vector together with the image data; A frame rate conversion device comprising: a decoder that converts the image data inputted from the encoder from a number Q of frames to a number P of frames using the motion vector, wherein the encoder converts the motion vector between the core frames. All are encoded and transmitted to the decoder, and the decoder includes: a motion vector memory that stores the motion vectors transmitted from the encoder; and a block address generator that generates addresses corresponding to blocks of the motion vectors. the number of frames Q and P
a multiplier that multiplies the motion vector by a coefficient determined by the above; a frame memory that stores image data of at least two consecutive frames; and a frame memory that stores image data of at least two consecutive frames; 1. A frame rate conversion device comprising: an averaging circuit that performs weighting using coefficients determined by numbers Q and P for averaging.