JP3179475B2 - Signal processing device and signal processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention is suitably applied to, for example, a motion blow generation device that adds a shake (motion blow) effect to an image by computer graphics or an image by a video camera. And a signal processing method.

[Summary of the Invention] The present invention relates to a signal processing device and a signal processing method suitable for application to, for example, a motion blow generation device that adds a shake (motion blow) effect to an image by computer graphics or an image by a video camera. ,
The input image signal is divided into a plurality of blocks in which one block is composed of one or more pixels, and motion data indicating the direction and magnitude of motion of the image between the blocks set by the division is formed. A weighting coefficient is set for each block as a set of blocks in a predetermined area according to the motion data, and a weighting coefficient in a state where there is a motion is set by the coefficient setting. Only, the image data of the input image signal is averaged by weighted addition using a weighting coefficient, and an output image signal that is subjected to a partial averaging process in accordance with the motion data and becomes an image to which a motion blow effect is added is obtained. Thus, a natural moving image with little flicker can be obtained.

2. Description of the Related Art Advances in computer graphics have made it possible to produce animations that are closer to actual images. In such animations, it is required to realize images that are closer to natural moving images. ing.

In addition, it has become possible to easily record moving images with a video camera equipped with a charge-coupled imaging device (CCD). However, since the shutter speed is relatively slow in the past, the integration effect of the CCD and the like makes it relatively natural. Video was obtained.

[Problems to be Solved by the Invention] However, in order to realize an image closer to a natural moving image in the animation, for example, simply increasing the number of different scenes per second causes a flickering image of a moving character. And the like.

In recent years, for video cameras, for example, 1/4000
Although a high-speed shutter of about seconds can be used, when a moving object is photographed with such a high-speed shutter, a reproduced image may appear to flicker unnaturally. What is common to the original image of the computer graphics and the image captured by the high-speed shutter is that the original image corresponding to the video signal of each one frame or one field is almost complete even if it is a moving image as a whole. This is a still image and almost no blur is included.

In view of the above, the present invention is applied to a case where an image corresponding to an input video signal is a moving image as a whole, but an image of each individual frame or one field is close to a complete still image. It is another object of the present invention to provide a signal processing device and a signal processing method in which an unnatural flicker does not occur in an image to be reproduced.

[Means for Solving the Problems] A signal processing apparatus according to the present invention divides an input image signal into a plurality of blocks each including one or a plurality of pixels as shown in FIG. Motion detection means (4) for forming motion data indicating the direction and magnitude of the motion of the image between the blocks set by the division, and weighting for each block as a set of blocks in a predetermined area according to the motion data Coefficient setting means (5) for setting a coefficient,
The image data of the input image signal is averaged by weighting addition using the weighting coefficient only for the block group in the predetermined area including the block in which the weighting coefficient in the state where there is a motion is set by the coefficient setting means, and according to the motion data. And a two-dimensional filter means (6) for obtaining an output image signal which is an image to which the effect of the motion blow is added by performing a partial averaging process.

In the signal processing method according to the present invention, the input image signal is
A motion detection step of dividing one block into a plurality of blocks each including one or a plurality of pixels and forming motion data indicating a direction and a magnitude of a motion of an image between the blocks set by the division; A coefficient setting step of setting a weighting coefficient for each block as a set of blocks in a predetermined area according to data, and a block group of the predetermined area including a block in which a weighting coefficient in a state of motion is set in the coefficient setting step The output image is obtained by averaging the image data of the input image signal by weighted addition using a weighting coefficient only, and performing a partial averaging process according to the motion data to add the motion blow effect. And a weighted addition step of obtaining a signal.

[Operation] According to the present invention, when the input image signal VI corresponds to a moving image, the motion detecting means (4) forms motion data indicating the direction and magnitude of the motion of the moving image. . Then, a predetermined portion of the input image signal VI (for example, each image signal of a predetermined number of pixels in the vicinity of a pixel to be processed) is weighted by a weighting coefficient set in accordance with the motion data and added. As an output image signal of a pixel to be processed, a signal obtained by averaging a predetermined portion of the original input image signal in the direction of motion of the pixel is assigned.

Therefore, an unnatural flicker or the like does not occur as a whole in an image obtained by reproducing the output image signal.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this example, the present invention is applied to a motion blow generation device.

FIG. 1 shows a configuration of a motion blow generating apparatus according to the present embodiment. In FIG. 1, (1) denotes a frame memory for an input image sequence, and a video camera in which the contents of the frame memory (1) are not shown. Or analog from VTR etc.
Rewrite periodically with the video signal supplied via the digital converter. This frame memory (1) can be omitted. (2) and (3) denote delay circuits each having a delay time of one frame period. The video signal VI of the current frame read from the frame memory (1) is passed through the delay circuit (2) to the video signal of the previous frame. VJ is obtained, and the video signal VI of the current frame is supplied to the motion vector detection circuit (4) directly and via the delay circuit (3).

This motion vector detection circuit (4)
By applying, for example, a block difference method to two video signals VI and VJ separated by a frame period, a motion vector <MV> of a block composed of one or more pixels to be processed in the previous frame is obtained. In the block difference method,
The difference between the video signal of the block to be processed in the previous frame and the video signal of a plurality of blocks in a predetermined range of the current frame is calculated, and the block of the current frame in which the absolute value of the difference is minimum (this is referred to as the ") Is specified. However, the vector from the center of the block to be processed in the previous frame toward the center of the target block is the motion vector <MV>.

The motion vector <MV> is used as a parameter generation circuit (5).
The parameter generation circuit (5) generates a set of parameters whose number and value change in accordance with the magnitude and direction of the motion vector <MV>. (6) shows a shake filter formed by combining a plurality of multiplication circuits and a plurality of addition circuits, and supplies a set of parameters and the video signal VJ of the previous frame to the shake filter (6). The shake filter (6) outputs the video signal VJ of the previous frame.
Are added to a signal obtained by weighting the video signal corresponding to a set of blocks including the block to be processed by the set of parameters, thereby forming an output video signal VP of the block to be processed.

The motion vector detection circuit (4) of the present example detects a motion vector <MV> for each block of the video signal of the previous frame, and the parameter generation circuit (5) detects each motion vector <MV> for each block. A set of parameters is generated and supplied to a shake filter (6). In response, the shake filter (6) performs a filter process on all blocks of the video signal VJ of the previous frame using a corresponding set of parameters. The output video signal VP output from the shake filter (6) is supplied to a television receiver or the like (not shown) via an output image sequence frame memory (7).

The operation of the parameter generation circuit (5) of the present example will be described in detail with reference to FIGS.

FIG. 2 shows an example of a motion vector supplied from the motion vector detection circuit (4). In FIG. 2, a hatched block (for example, a block (8) in FIG. 2A) corresponds to the previous frame. This is a block composed of one or more pixels to be processed. Vector starting from the center of this block (for example, block diagram (8) in FIG. 2A)
A vector (9) from the block to the block (10)) indicates the motion vector <MV> of the block.

FIG. 3 shows an example of a set of parameters generated by the parameter generation circuit (5) corresponding to the motion vector of FIG. 2. For example, when the motion vector has a component for one block in the horizontal direction (see FIG. 2A), as a set of parameters, as shown in FIG. 3A, the start point block (8) and the end point block (9) of the motion vector are respectively 1 /
2 is assigned, and 0 is assigned to other blocks. When the motion vector has a component of one block in the horizontal direction and one block in the vertical direction (FIG. 2B), the start point and the end point of the motion vector as shown in FIG. 3B as one set of parameters. (Two blocks on the diagonal line) are each assigned 1/2, and the other blocks are assigned 0.

Similarly, when the motion vector has a component for two blocks in the horizontal direction (FIG. 2C) or when the motion vector has a component for two blocks in both the horizontal direction and the vertical direction (FIG. 2E), the corresponding condition is satisfied. As one set of parameters, 1/3 is assigned to only three blocks through which the motion vector passes (FIGS. 3C and 3E). And the motion vector is 2 in the horizontal direction.
When one block has a component for one block in the vertical direction (FIG. 2D), as shown in FIG. 3D, a set of parameters corresponding to the start point and the end point of the motion vector are respectively 1 / 3 is assigned, and the middle two blocks are assigned 1/6 each.

To summarize the above rules, the motion vector is generally composed of (N-1) blocks in the horizontal direction (N = 2,3, ‥‥) and M blocks in the vertical direction (0≤ | M | ≤N-1). When there is a component, 1 / N is assigned to each of the blocks at the start and end points of the motion vector. In addition, since the motion vector always crosses one or two blocks in the vertical direction between the start point and the end point, a parameter which is 1 / N in total in the one or two blocks in the vertical direction, respectively. Assign. When the motion vector crosses the two blocks in the vertical direction, a parameter having a magnitude inversely proportional to the distance between the center of each of the two blocks and the motion vector is assigned to the two blocks.

In addition, a parameter having a value of 0 is assigned to the other blocks so that the total value of the assigned parameters becomes 1.

Specifically, the case of N = 4 will be described. First, when the motion vector has components for three blocks in the horizontal direction (FIG. 2F) or when the motion vector has three blocks in the horizontal and vertical directions, respectively. With components (Fig. 2 I)
, A parameter having a value of 1/4 is assigned to each of the four blocks through which the motion vector passes, as shown in FIGS.

On the other hand, as shown in FIG. 2H, when the motion vector (11) has components for three blocks in the horizontal direction and two blocks in the vertical direction, the motion vector (8) at the starting point of the motion vector (11) is The end block (12) is shown in FIG.
As shown in (1), parameters each having a value of 1/4 are assigned. The vertical distance between the center of two blocks (13) and (14) adjacent to the starting block (8) and the motion vector (11) is L1 and L2, respectively.
Assuming that the values of the parameters assigned to the two blocks (13) and (14) are PA1 and PA2, respectively, these parameters have the following relationship.

PA1 + PA2 = 1/4 PA1: PA2 = L2: L1 = 1: 2 ‥‥ (1) From this equation (1), the two parameters are PA1 = 1/12,
PA2 = 1/6. Similarly, assuming that the distance between the center of the two blocks adjacent to the end point block (12) and the motion vector is L3 and L4, respectively, since L3: L4 = 1: 2, they are assigned to these two blocks. Parameters are calculated. The motion vector is the vector (15) in FIG.
, The corresponding parameter assignments are as shown in FIG. 3G.

For blocks other than the block through which the motion vector passes, 0 is assigned as a parameter. When parameters are assigned in this manner, the sum of the values of the assigned parameters becomes 1.

Further, when the vertical component of the motion vector <MV> is larger than the horizontal component, for example, the motion vector (16) is the transposition of the motion vector (15) as shown in FIG. 2G. In this case, the parameter corresponding to the motion vector (16) is a transposition of the parameter in the array shown in FIG. 3G.

Next, the operation of the shake filter (6) in the example of FIG. 1 will be specifically described. In this example, the motion vector supplied from the motion vector detection circuit (4) is the vector shown in FIG. It is assumed that the parameter array shown in FIG. 3H is supplied from the parameter generating circuit (5) to the shake filter (6). FIG. 4A shows the arrangement of FIG. 3H repeatedly, and FIG. 4B shows a block of 4 rows × 4 columns having the block (8) to be processed as the lower left vertex.

In this example, since a set of supplied parameters has a value other than 0 only in a block of 4 rows × 4 columns, the shake filter (6) has four rows starting from the block (8) to be processed. Only the data of the video signal of the block of × 4 columns need to be the target of the calculation. If the video signal of the i-th row and the j-th column in the block of 4 rows × 4 columns is represented by Gij, the video signal of the block to be processed is G41.

In this case, the shake filter (6) adds the values obtained by weighting the video signals of the 16 blocks starting from the block (8) with the corresponding parameters, and outputs the output video signal corresponding to the block (8). Since VP is obtained, the output video signal VP is as follows.

VP = G41 / 4 + G32 / 6 + G42 / 12 + G23 / 12 + G33 / 6 + G24 / 4 (2) Also, a set of commonly supplied parameters is N rows × N
When the block (N = 1, 2,...) Has a value other than 0 only inside the block, the shake filter (6) uses N rows ×
The video signal of the block of N rows is set as a calculation target.

As described above, in this example, the motion vector of the block to be processed and a set of parameters corresponding to the vector are obtained, and the set of blocks arranged in the direction of the motion vector starting from the block is determined. The result of adding the value obtained by weighting the video signal with the set of parameters is the output video signal VP of the block to be processed. Therefore, since the output video signal VP is a signal obtained by substantially averaging the input video signal VJ in the direction of the motion vector, that is, a signal in a moving image state, an image corresponding to the output video signal VP is a natural moving image without flicker. There are benefits.

In the above-described embodiment, the moving image processing is performed on the image of the previous frame, that is, the image before moving, using the motion vector. May be performed using a vector whose direction is inverted.

As described above, the present invention is not limited to the above-described embodiment, but can take various configurations without departing from the gist of the present invention.

[Effects of the Invention] According to the present invention, since processing is performed such that an input image signal is averaged in the direction of motion, there is an advantage that an unnatural flicker does not occur in a reproduced image of an output image signal. is there.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of a motion blow generator according to one embodiment of the present invention, FIG. 2 is a diagram showing an example of a motion vector in the embodiment, and FIG. 3 is a parameter corresponding to the motion vector. FIG. 4 is a diagram for explaining the operation of the shake filter of the embodiment. (1) is a frame memory, (2) and (3) are delay circuits, (4) is a motion vector detection circuit, (5) is a parameter generation circuit, and (6) is a shake filter.

Claims (2)

(57) [Claims] An input image signal is divided into a plurality of blocks in which one block is composed of one or more pixels, and a motion indicating a direction and a magnitude of a motion of an image between the blocks set by the division. Motion detecting means for forming data; coefficient setting means for setting a weighting coefficient for each block as a set of blocks in a predetermined area according to the motion data; The image data of the input image signal is averaged by weighting addition using the weighting coefficient only for the block group of the predetermined area including the set block, and a partial averaging process according to the motion data is performed. And a two-dimensional filter means for obtaining an output image signal which becomes an image to which a motion blow effect is added. 2. An input image signal is divided into a plurality of blocks in which one block is composed of one or more pixels, and a motion indicating a direction and a magnitude of a motion of the image between the blocks set by the division. A motion detection step of forming data; a coefficient setting step of setting a weighting coefficient for each block as a set of blocks in a predetermined area according to the motion data; The image data of the input image signal is averaged by weighting addition using the weighting coefficient only for the block group of the predetermined area including the set block, and a partial averaging process according to the motion data is performed. A weighted addition step of obtaining an output image signal which becomes an image to which the effect of motion blow has been added.