JPH0595544A - Method and device for moving amount detection for picture data - Google Patents

Abstract

PURPOSE:To improve the accuracy of detecting the moving amount in the detection method for moving amount between two picture data. CONSTITUTION:The picture data of the current frame and the picture data of the former frame are inputted to neighboring processing circuits 11 and 12. The neighboring processing circuits 11 and 12 performs the processing averaging the input data in a block unit for a plurality of neighboring blocks, outputting, for example, sample data taking the collection of average values as many as the number of picture element data of a block as a unit. The sample data is provided with the collected picture element data of blocks. In short, it includes the information of the neighboring picture element data. Thus, the correlated blocks can be obtained accurately with the S/N of data improved even when the blocks or representative points to be compared are separated in performing the correlation operation in the circuit of the poststage.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting a motion amount between two image data in a multi-stage search format and a motion amount detecting time using the method, more specifically, a signal-to-noise ratio (SN ratio). The present invention relates to a technique for detecting the amount of motion of image data capable of increasing the calculation speed, and is effectively applied to a device for processing a moving image signal such as a videophone system.

[0002]

2. Description of the Related Art When an image signal or a television (also referred to as TV) signal is subjected to analog / digital conversion for image processing, a large amount of data must be processed at high speed. There is a problem that a dedicated large-scale circuit is required to calculate the correlation between the two image data, and a huge amount of calculation time is required when the calculation is performed by software. In this regard, for example, as a device for highly efficiently encoding and recording or transmitting a TV signal, there is a "motion-compensated interframe predictive coding device" described in Japanese Patent Laid-Open No. 1-243790. This is (1) sampling the TV signal into pixels,
(2) A plurality of pixels (for example, 8 pixels in each of the vertical and horizontal directions) is set as one block, and (3) a block most similar to the above block is searched from the data of the transmitted frame while shifting the position little by little (hereinafter , Called the motion amount detection method),
(4) Each of the processes of transmitting the "positional deviation amount" and the "difference between two blocks after motion compensation" is performed. By this "motion-compensated inter-frame prediction" technique, it is possible to reduce the transmission speed when converting a television signal having a wide frequency band into a digital signal and transmitting the digital signal, and to increase the calculation speed. become.

In the motion amount (or motion vector) detection method described in (3) above, it is necessary to calculate the correlation between two blocks. This detection method is roughly classified into a full search method and
Two types of multistage search methods are known. In the full search method, the reference block is compared with blocks at all displaced positions within the range to be searched, and the position of the block having the smallest difference is used as the motion amount. In the multi-stage search method, blocks (representative points) at a predetermined number of different positions are compared with a reference block, and it is estimated that there is an optimum block around the most similar block among them.
Furthermore, the surrounding area is searched in detail and the final result is used as the motion amount.

[0004]

The motion amount detection methods of the full search method and the multi-step search method have the following problems, respectively. That is, in the full search method, the most similar block can be selected within the range, but the number of blocks to be compared is very large, the processing time becomes long, and the operation speed becomes a problem. Therefore, there has been a problem that the operating speed of the circuit used must be increased or the circuit scale must be increased to perform parallel processing. Also, the signal-to-noise ratio (SN ratio)
If the value is bad, there is a drawback that the detection accuracy becomes poor.

On the other hand, the multi-stage search method has an advantage that the processing time can be greatly shortened because the number of blocks to be calculated is small, but the accuracy of motion amount detection is deteriorated as compared with the full search method. There was a problem.

An object of the present invention is to provide a technique capable of detecting a motion of image data by improving a signal-to-noise ratio for the image data. Another object of the present invention is to provide a technique for detecting the amount of motion of image data, which can increase the calculation speed for detecting the motion of image data.

The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

[0008]

The outline of a typical one of the inventions disclosed in the present application will be briefly described as follows.

That is, in the motion amount detection of the multi-stage search method, when comparing with blocks at a predetermined number of types of shifted positions, correlation is checked after calculation such as averaging with neighboring pixels. The calculation is performed. More specifically, the image data of the current frame and the image data of the previous frame are averaged in block units, and the averaging process is performed on a plurality of neighboring blocks. Generate sample data in units of a set of average values. The sample data is provided by aggregating a plurality of blocks of pixel data, and inherently contains information about peripheral pixel data. For blocks of data representative points, correlation calculation is performed by averaging processing while preserving the properties of the image around the representative points to determine the most similar representative block. Further, it is assumed that there is an optimum block in the surroundings, and the process of finely searching the surroundings is repeated.
In the final stage, averaging is not performed, and it can be performed in pixel units. When an attempt is made to improve the calculation speed, a calculation including a process of thinning out pixel data in the data area and sampling can be adopted.

[0010]

According to the above-mentioned means, the data to be subjected to the correlation calculation is the signal obtained by averaging the image signals, and even if the representative points are greatly separated, the averaging processing used for the correlation determination is performed. The signal contains the peripheral information of the block. This works so that correlated blocks can be accurately obtained. Further, in comparison with the conventional multi-stage search method, when the number of pixels to be averaged is n, the correlated signal is n times, but the noise is √n times, so the signal-to-noise ratio is √n times. The most correlated block can be accurately detected from the representative points in an improved form.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 4 shows an example of a motion compensation interframe predictive coding apparatus for TV signals to which the present invention is applied.

The TV signal picked up by the TV camera 101 is scanned in the scanning line direction, and becomes a pixel signal converted into a digital signal by the analog / digital converter 102. Further, the scanning line / block scanning conversion circuit 103 collects a plurality of pixels to form a block, which is stored in the temporary storage memory 104. Digital data as reference data read from the temporary storage memory 104,
The reference data read from the frame memory 105 that stores the transmitted signal of the previous TV frame is supplied to the motion compensation circuit 106. The motion compensation circuit 106 searches the reference data for an area of data that is correlated or similar to the standard data. Then, the motion amount (motion vector) is calculated from the position having the highest correlation. The calculated amount of movement is given to the control circuit 107. The control circuit 107
The previous TV frame pixel data at the position where the motion amount is compensated is read from the frame memory 105 and given to the subtractor 108. The read data is positioned as data that predicts a pixel to be encoded. The subtraction circuit 108 calculates the inter-frame prediction error after motion compensation.

The prediction error is quantized by a quantizer 110 as it is or after being orthogonally transformed by an orthogonal transformer 109, and a variable length coding circuit 111 assigns a codeword having a length corresponding to the occurrence frequency. .. It is generally known that the codeword length is short when the prediction error is small, and the codeword length is long when the prediction error is large. The signal coded by the variable length coding circuit 111 is a first-in first-out (FIF) for smoothing the transmission rate together with the above-mentioned motion amount.
O) Written in the memory 112. FIFO memory 11
The coded signal is read out at a constant speed from
13 is sent. An addition circuit 115 adds the quantized signal output from the quantizer 110, or the signal obtained by inverse orthogonal transforming the quantized signal by the inverse orthogonal transform circuit 114, and the motion compensation prediction signal read from the frame memory 105. After that, it is written in the frame memory 105 to predict the next TV screen.

The motion compensation circuit 106 provides the control circuit 107 with a shift amount (motion amount) that shifts the position read from the previous TV frame, and the front TV at the position where the motion amount is compensated.
The pixel signal of the frame is read from the frame memory 105.

Here, in order to facilitate the subsequent description, first, the operation principle of the motion amount detection of the image data of the general multistage search method will be briefly described with reference to FIG.

FIG. 3 shows the reference block (A) and the search area (B). For example, the reference block (A) is a part of data of the current frame, and the search area (B) is a part of data of one frame before. Here, as an example, the reference block (A) is 4 × 4 pixels, and the search area is 18 × 18 pixels. Therefore, the search for the search area is 4 × 4.
It is performed in reference data units in pixel units. The search range is x
-7 to +7 in the direction and -7 to +7 in the y direction.

In the first step of the multi-stage search method, as shown in FIG. 3C, nine representative points (111 to 119) are taken at positions displaced by ± 4 pixels in both the x and y directions. Correlation calculation is performed with the data of the standard block (A) using the surrounding data of 4 × 4 pixels as reference data. As the correlation calculation, the data A (i, j) (1 ≦ i, j ≦ 4) of the pixel of the corresponding reference block (A) and the search area B are used.
4 × selected from B (m, n) (1 ≦ m, n ≦ 18)
The squared error (or absolute value) from the 4-pixel reference data B (i, j) (1 ≦ i, j ≦ 4) may be calculated to calculate the integrated value in the block. The results are compared and the block having the minimum value is obtained. From this, it is estimated that there is the most correlated block in the vicinity of that position. In this description, it is assumed that there is a block having the highest correlation in the vicinity of the representative point 111 of FIG.

In the second step of the multi-stage search method, as shown in FIG. 3D, in the vicinity (111) of the representative point obtained in the first step, the position shifted by ± 2 pixels in both the x and y directions. The first of nine representative points (121-129)
The same calculation as in step is performed, and it is estimated that there is a most correlated block in the vicinity of the position where the squared error is minimum.
The calculation for the center position 125 has already been performed and may be omitted. For example, in this second step, it is assumed that there is a block having the highest correlation in the vicinity of the representative point 121.

In the third step of the multistage search method, as shown in FIG. 3E, in the vicinity (121) of the representative point obtained in the second step, the position is shifted ± 1 pixel in both the x and y directions. Nine representative points (numbers not shown) are taken, the same calculation as in the first step is performed, and the position with the smallest error is found. FIG. 3 is an enlarged view of the movement of these three searches.
As in (F), the search range gradually decreases at each step, and the block having the highest correlation can be finally determined.

Next, one embodiment of the image data motion amount detecting circuit according to the present invention will be described in detail with reference to FIG.

The motion amount detection circuit 106 has an input terminal 10 for receiving image data from the primary storage memory 104, an input terminal for receiving image data from the frame memory 105, and a data output terminal 20 to the control circuit 107, and the proximity processing. Circuits 11, 12, subtraction circuit 15, squared error calculation circuit 1
6, an integration circuit 17, a minimum value determination circuit 18, and a comparison memory 19 are incorporated. In the motion amount detecting circuit 106, the circuit blocks except the proximity processing circuits 11 and 12 are the same as those in FIG.
In this embodiment, the neighborhood processing circuits 11 and 12 are circuit blocks corresponding to the characteristic parts of the present invention.

Before describing the proximity processing circuits 11 and 12, the circuit blocks of the subsequent stages will be described. Data A (i, j) (1 ≦
i, j ≦ 4) is input to the subtraction circuit 15. In response to this, the image signal BB of the necessary search area is selected from the image signals of one frame before stored in the frame memory 13.
Data B (i, j) output from the proximity processing circuit 12 corresponding to data for 4 × 4 pixels around the representative point selected from (m, n) (1 ≦ m, n ≦ 18) (1 ≦ i, j
≦ 4) is input to the subtraction circuit 15. The subtraction circuit 15 is 2
Calculate the difference between two input data. Next, the square error calculation circuit (or absolute value calculation circuit) 16 calculates the square error or absolute value, for example, the square of the difference between the two data. This data is given to the integrating circuit 17, the sum (integrated value) of the square error for one block is calculated, and the minimum value judging circuit 1
8 is input. In the first step of the multi-stage search method, 9
Calculations are performed on the blocks at the offset positions. First, for example, the block of (C) 111 in FIG. 3 is calculated, and the value and the block number are stored in the comparison memory 19. Next, the block 112 in FIG. 3C is calculated, and the result is compared with the value of the block previously stored in the comparison memory 19, and the value with the smaller square error and the block number are compared. Are stored in the comparison memory 19. This is repeated 9 times, and it is determined that the block having the highest correlation exists near the block obtained as the final result.

In the second and third steps of the multistage search method, the block number stored in the comparison memory 19 is read out, and the same operation as above is performed for the nine representative points around the block. After the end of the third step, the block number stored in the comparison memory 19 is read out, the amount of movement is determined from this number, and output to the output terminal 20.

Image signal AX input to the input terminal 10
(I, j) is input to the proximity processing circuit 11. Examples of the neighborhood processing include simple averaging processing and weighted averaging processing. That is, as the image signal AX (k, l), not the 4 × 4 pixels but a wider area, for example, 8 × 8 pixels are taken in the first step, and the neighborhood processing circuit 11 takes the average value of 4 × 4 pixels as 1 pixel ( As sample),
A total of 4 × 4 sample data A (i, j) (1 ≦ i, j
≦ 4) is created and input to the subtraction circuit 15. In particular, when the sampling data is generated in this way, the pixel information around the block of interest is reflected in the average value included in the sample data, which makes it possible to obtain a highly correlated block with higher accuracy. To enable. From a different point of view, this processing means that the information of 8 × 8 pixels is collected in 4 × 4 sample data. Similarly, the image signal BX (i, j) of 8 × 8 pixels in the required search area is read out from the image signal BX of one frame before stored in the frame memory 13, and the neighborhood processing circuit 12 is read.
To enter. Similarly, this circuit also creates a total of 4 × 4 sample data B (i, j) (1 ≦ i, j ≦ 4) and inputs it to the subtraction circuit 15. The subtraction circuit 15 calculates the difference between the two sample data, and then the square calculation circuit (or absolute value calculation circuit) 16 calculates the squared error (or absolute value). Further, with respect to the output of the integrating circuit 17, the minimum value determination circuit 18 and the comparison memory 19 calculate the minimum value with respect to the nine blocks at the shifted positions. This is repeated up to the third step, and the block obtained as the final result is determined to be the block having the highest correlation. In the second step, the average value of 2 × 2 pixels may be processed instead of the average value of 4 × 4 pixels. Further, in the third step, the correlation calculation may be directly performed in the unit of 4 × 4 pixels without performing the average value processing.

FIG. 2 shows an embodiment of the proximity processing circuit 11. The proximity processing circuit 12 can also be configured by a circuit similar to that in FIG. The image signal input to the input terminal 10 is input to the first integrating circuit 21 and the second integrating circuit 22. In these circuits, addition is performed for the designated data, and the result is output to the switching circuit 23. For example, the integrating circuit 21 adds data for 4 × 4 = 16 pixels, then performs necessary scaling (1/16) and outputs the result. The integrating circuit 22 adds data for 2 × 2 = 4 pixels, then performs necessary scaling (1/4) and outputs the result. Switching circuit 23 between these two integrating circuits and the original signal
Select with. For example, in the first step of motion amount detection, the output of the integrating circuit 21 is selected, and in the second step, the integrating circuit 2 is selected.
In the third step, the output of 2 is selected and the original signal is output to the subtraction circuit 15. The proximity processing circuit 12 may also perform the same operation on the signal of the frame memory 105.

By performing such a neighborhood process, even if the representative points are far apart, the signal for performing the correlation calculation includes the information of the surroundings, which is a problem in the conventional multistage search method. It is possible to solve the problem of erroneous estimation of the motion amount in the first stage. Further, even when the signal-to-noise ratio of the image signal is poor, the amount of motion can be detected in a state in which the influence of noise is reduced by averaging with the surrounding signals. For example, if the number of pixels to be averaged is n, the correlated signal is n times, but the noise is √n times. Therefore, the most correlated block is detected with the SN ratio improved by √n times. it can. Eg 4
When averaging × 4 pixels, S for uncorrelated noise
The N ratio can be improved four times (about 12 dB). In this way, by adding the neighborhood calculation, it is possible to realize accurate motion amount detection.

Although the invention made by the present inventor has been specifically described based on the embodiments, the present invention is not limited thereto, and needless to say, various modifications can be made without departing from the scope of the invention. Yes.

For example, in the description of the above embodiment, the amount of motion detection is described with the block size of 4 × 4 pixels.
Similar processing can be realized with x8 pixels or 16x16 pixels.

In addition, as a calculation in the neighborhood processing circuit, 4
Although an example of averaging of × 4 pixels is shown, M × N pixels may be used. For example, 2 × 4, 4 × 2, 4 × 1 pixels or the like may be used. As the averaging method, not only a simple average value,
Weighting may be performed according to the position for calculation.

The arithmetic processing can be switched in each step of the multistage search method. For example, in the first step, the average of M × N pixels, in the second step, the average of M ′ × N ′ pixels, and in the third step, 1 × 1 pixel, that is, averaging is not performed. May be switched. The switching of this processing may be determined by an external condition such as a signal-to-noise ratio of the image. This makes it possible to streamline the motion amount detection process that handles an image signal having a large amount of data, and contributes to a reduction in the overall calculation processing time.

The target of the neighborhood calculation may be the same as the original block size. In this case, averaging may be performed within the block. In addition, a process of sampling and averaging data in each data area may be adopted as the neighborhood process.

In the above description, the case where the invention made by the present inventor was mainly applied to the motion compensation interframe predictive coding technique of a television signal which is the field of application which is the background of the invention has been generally explained. INDUSTRIAL APPLICABILITY The present invention can be widely applied to image processing in videophone systems, workstations, personal computer systems, and the like, and further to a technique for preventing camera shake of a TV camera.

[0033]

The effects obtained by the typical ones of the inventions disclosed in the present application will be briefly described as follows.

That is, by detecting the motion amount of the image data by using the value obtained by averaging the pixels in block units, the sample data as the average value or a set of the average values is obtained by using the pixel information in the vicinity of the region of interest. It is inherent. Even if the representative point or the region of interest for which the correlation is to be determined are greatly separated from each other, the data to be subjected to the correlation calculation includes the surrounding pixel information, so that the estimation of the motion amount at the first stage in the multistage search method Is effective.

Further, even when the signal-to-noise ratio of the image signal is bad, the data used for correlation determination or motion detection is averaged with the surrounding pixel data, so that the influence of noise is reduced. The amount of movement can be detected with. For example, if the number of pixels to be averaged is n, the number of correlated signals is n times, but the noise is √n times.
The most correlated block can be detected with the ratio improved by √n times. For example, averaging 4 × 4 pixels can improve the SN ratio four times (about 12 dB) with respect to noise having no correlation.

With these, highly accurate motion amount detection can be realized.

When this method is applied to a motion amount detecting circuit, an arithmetic circuit for performing averaging processing and the like may be arranged in a preceding stage such as a subtracting circuit, and only a few circuits are added to the conventional circuit. There is an effect that the detection accuracy of the motion amount can be easily improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a motion amount detection circuit for image data according to the present invention.

FIG. 2 is a block diagram showing an embodiment of a proximity processing circuit included in a motion amount detection circuit.

FIG. 3 is an explanatory diagram showing an example of a processing procedure of a multistage search method together with a reference block and a search area for calculating a motion amount of image data.

4 is a block diagram of an embodiment of a motion compensation interframe predictive coding apparatus for a television signal to which the motion amount detecting circuit of FIG. 1 is applied.

[Explanation of Codes] 10 Input Terminal 11 Proximity Processing Circuit 12 Proximity Processing Circuit 13 Input Terminal 15 Subtraction Circuit 16 Square Error Calculation Circuit 17 Accumulation Circuit 18 Minimum Value Judgment Circuit 19 Comparison Memory 20 Output Terminals 21 and 22 Accumulation Circuit 105 Frame Memory 106 motion amount detection circuit

Claims (5)

[Claims] 1. A method for detecting the amount of movement of image data between pixel data in a standard data area and pixel data in a reference data area, in which data in each data area is averaged in block units of pixel data. A method for detecting the amount of motion of image data, characterized in that a position having a high correlation with the pixel data of the standard data region in the reference data region is calculated as the amount of motion of the image data based on both of the calculated data. .. 2. The averaging operation averages all or partial pixel data included in a block corresponding to a plurality of pixel data areas and calculates a value thereof, 2. The sample data corresponding to a data area of a plurality of pixels is obtained by performing an operation on a plurality of blocks located in a plurality of blocks and averaging values obtained by an averaging process for each block. Method for detecting the amount of motion of image data in the above. 3. A method of detecting a motion amount of image data between pixel data in a standard data area and pixel data in a reference data area, wherein data in each data area is thinned out and sampled, and the calculation is performed. A method for detecting a motion amount of image data, characterized in that a position having a high correlation with pixel data in a standard data region in a reference data region is calculated as a motion amount of image data based on both of the performed data. 4. A method of detecting a motion amount of image data between pixel data in a standard data area and pixel data in a reference data area, wherein data in each data area is thinned out and sampled. Each data obtained by averaging is further averaged, and based on both averaging processed data, the position of high correlation with the pixel data of the standard data area in the reference data area A method for detecting the amount of movement of image data, which is characterized in that it is calculated as an amount. 5. An image data motion amount detection circuit for reconstructing moving image data by comparing image data to be encoded with image data that has already been encoded. An arithmetic circuit for performing the arithmetic operation according to any one of paragraphs, and means for detecting a motion amount of the image data based on a difference between the arithmetic result for the image data in the standard data area and the arithmetic result for the image data in the reference data area. A motion amount detection circuit comprising: