JPH07222157A - Device and method for detecting moving amount - Google Patents

Abstract

PURPOSE:To eliminate moving amount erroneous detection by forming the picture data of plural hierarchies with different resolutions by using input pictures and forming the activity data of the plural hierarchies indicating the high frequency components of the picture data of the respective hierarchies based on the data. CONSTITUTION:In this moving amount detector 1, the picture data of inputted original pictures are inputted to a blocking circuit 2 and successively blocked by a prescribed size. The blocked picture data of the original pictures are respectively inputted to the frame memory 3 and evaluation value calculation circuit 4 of a hierarchy (1), the average value hierarchizing circuit 5 and activity hierarchizing circuit 6 of the hierarchy (2) and the activity hierarchizing circuit 7 of the hierarchy (3). In the average value hierarchizing circuit 5 of the hierarchy (2), an average value hierarchizing processing is conducted by an arithmetic operation and input to the blocking circuit 8, frame memory 9 and activity hierarchizing circuit 6 of the hierarchy (2) and the average value hierarchizing circuit 10 of the hierarchy (3) is respectively performed. In the circuit 10, the inputted picture data of the hierarchy (2) are used, the average value hierarchizing processing by the arithmetic operation is conducted and the picture data of the hierarchy (3) are defined.

Description

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Field of Industrial Application Conventional Technology Problems to be Solved by the Invention Means for Solving the Problems (FIGS. 1 to 5) Actions (FIGS. 1 to 5) Embodiments (FIGS. 1 to 14) (1) First Motion amount detection method and device of the first embodiment (FIGS. 1 to 7) (2) Motion amount detection method and device of the second embodiment (FIGS. 3 and 5) (3) Motion amount detection method of the third embodiment And device (FIGS. 5, 8, and 9) (4) Motion amount detection method and device of fourth embodiment (FIG. 5) (5) Motion amount detection method and device of fifth embodiment (FIG. 10) (6) ) Method and apparatus for detecting motion amount of sixth embodiment (FIG. 11,
(FIG. 12) (7) Motion amount detection method and device of seventh embodiment (FIG. 13,
FIG. 14) (8) Other embodiment (FIG. 10) Effect of the invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion amount detecting device and a motion amount detecting method, and in particular, when detecting a motion of an image, two motion image data which are temporally different from each other are layered and then the motion amount is detected. Can be applied to.

[0003]

2. Description of the Related Art Conventionally, as a processing of a moving image, there is one that uses a motion amount (motion vector), that is, a moving direction and a size (or speed) of an object in an image which is temporally different. For example, the motion amount is used for motion-compensated interframe coding in high-efficiency image coding, parameter control by motion in a television noise reduction device using an interframe time domain filter, and the like. As a motion amount detecting method for obtaining the motion amount of this image, the block matching method is used (Japanese Patent Publication No. 54-124927).

In this block matching method, first, one screen is divided into blocks each having an appropriate number of pixels. Subsequently, a predetermined evaluation function is set between the image data blocked in this way and the search area in which the image data of the screen that is temporally different in order to search the area where this image data has moved is blocked. Is used to evaluate in pixel units, and the optimum value that minimizes this evaluation value is obtained to detect the amount of motion between the two block image data.
As a result, the amount of movement of the image can be detected with high accuracy.

[0005]

However, in the block matching method, it is necessary to search all the search areas, which are the detection range, for all the pixels of the block to be detected, and obtain the difference between them. . For this reason, the amount of calculation for detecting the amount of motion becomes large, and the size of the apparatus itself becomes large and the calculation time becomes long. In order to solve such a problem, a motion amount detection method has been proposed in which a hierarchical image is hierarchized at a plurality of resolutions and the motion amount is detected by the block matching method using the hierarchical images (special feature. Wishhei 5-2448814).

In this motion amount detection method, first, original image data (hereinafter referred to as layer 1) is hierarchically averaged by averaging or low-pass filter processing to reduce the number of pixels (hereinafter referred to as layer 1). 2). Next, a rough motion amount is detected from the created layer 2 image data, and fine motion amount detection is performed on the layer 1 image data based on the motion amount so that the motion amount can be detected with a small amount of calculation. Has been done. Note that the number of layers here is not limited to two layers, and by repeating the average value layering sequentially, layers 3, 4, ...
It is also possible to create image data of ...

If such a motion amount detecting method is used, the motion amount can be calculated with a smaller calculation amount as the number of layers increases. That is, since the block size and the search area of the image data in the higher hierarchy are smaller, the amount of calculation by the evaluation function is inevitably smaller. The block size itself is the same as the normal block matching method for finally obtaining the motion amount, but motion compensation is performed according to the motion amount obtained from the image data of the upper layer, and the search area is calculated. Can be reduced, so that the amount of calculation can be reduced.

However, in this motion amount detecting method,
There is a problem that the accuracy of motion amount detection deteriorates as the number of layers increases. In practice, in this motion amount detection method, the motion amount for each block is first detected in the upper layer where the image is coarse, and the motion amount is detected in the lower layer based on this detection result. It has a great influence on the motion amount detection in. That is, since the block size of the image data in the upper layer is reduced by the average value layering process, the feature amount of the image differs from that of the original image in the lower layer. In particular, since the edge component is lost due to the average value layering, the correspondence between the motion amount in the average value layered image data and the motion amount due to the original image may shift.

When the motion amount is actually obtained in a layer lower than the current layer, the result of the motion amount in the upper layer is reflected. Therefore, if the shift amount of the correspondence is large and it cannot be covered within the search area, a malfunction occurs. Becomes Therefore, in the motion amount detection method using the image data in which the average value is layered,
There is a problem that malfunction may increase due to lack of information amount due to average value hierarchy. In addition, since the block size becomes smaller as the hierarchy gets higher, the motion amount detection on the average value layered image data also has a problem that the resolution of the motion amount detection with respect to the original image decreases and malfunction occurs. .

The present invention has been made in consideration of the above points, and when the motion amount is detected by the block matching method using layered image data, it is possible to improve the motion amount detection accuracy. An object of the present invention is to propose an amount detection device and a movement amount detection method.

[0011]

In order to solve the above problems, the present invention is based on an image layering step for forming image data of a plurality of layers having different resolutions from an input image and image data of a plurality of layers. An activity layering step that forms activity data of a plurality of layers representing high-frequency components of image data for each layer, and image data layered by the image layering step and activity layering corresponding to input images at different times. About the activity data hierarchized in steps,
The evaluation value calculation step for obtaining an evaluation value by block matching in a predetermined block unit for each layer, and the evaluation values obtained by the evaluation value calculation step for the image data and the activity data are comprehensively determined to be different for each layer. A motion amount detecting step for detecting the motion amount between the input images at the time point is provided.

Further, according to the present invention, the image layering means 2, 5 and 10 for forming image data of a plurality of layers different in resolution from the input image and the image data of each layer based on the image data of the plurality of layers. The activity layering means 6 and 7 that form a plurality of layers of activity data representing high frequency components, and the image data and activity layered by the image layering means 2, 5 and 10 corresponding to the input images at different times. With respect to the activity data hierarchized by the hierarchizing means 6 and 7, evaluation value calculating means 19, 20, 26, 27 and 4 for obtaining an evaluation value by block matching in predetermined block units for each hierarchy, image data and With respect to the activity data, each evaluation value obtained by the evaluation value calculation means 19, 20, 26, 27, 4 is comprehensively determined, and the hierarchy is obtained. And to provide a motion amount detecting means 22,29,32 for detecting the motion amount between the input images of different times.

[0013]

According to the present invention, a plurality of layers of image data having different resolutions are formed from the input image, and a plurality of layers of activity data representing high frequency components of the image data of each layer are formed based on the plurality of layers of image data, and the plurality of layers of activity data are different. For the image data and activity data corresponding to the input image at the time point, an evaluation value is obtained by block matching in predetermined block units for each layer, and each evaluation value is comprehensively judged, and between the input images at different time points for each layer. By detecting the amount of motion of each layer, it is possible to prevent erroneous detection of the amount of motion in each layer, and improve the accuracy of detecting the amount of motion of the final input image with a simple configuration and a short calculation time. You can

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Motion amount detecting method and apparatus of the first embodiment In the motion amount detecting method of the first embodiment, image data of a plurality of layers is formed for an original image by a method of average value layering. Data that represents high-frequency components (hereinafter referred to as “activity”) of each layer is also layered (hereinafter referred to as activity layering) to form activity data, and the amount of motion is detected using these for each layer. Is.

For the image data and the activity data which are actually hierarchized, the evaluation value is obtained by the block matching method in order from the upper hierarchy, and the amount of movement of the hierarchy is obtained by making a comprehensive judgment. The motion amount of the original image is finally detected by referring to the motion amount by motion compensation when sequentially obtaining the evaluation value of the lower layer.

Here, the motion amount detection means a block B on the current field (or frame) F1 as shown in FIG.
Which block B2 in the search area SA on the past field (or frame) F2 is matched with 1 is evaluated by an evaluation function, and the position giving the minimum evaluation value is detected as the amount of movement. The motion amount detecting method using the layered image data requires the layered image data in addition to the original images of both the current and past fields.

An example of generating hierarchical image data is as follows:
As shown in FIG. FIG. 2A shows a case where image data layered into three layers is generated for an original image, which is layer 1
Is the original image. Letting M _n (x, y) be image data in a layer higher than a block (for example, 16 × 16) on the original image, and image data in layer n,

[Equation 1] Thus, the block size is halved in the horizontal and vertical directions, respectively. Further, when the image data of the layer 2 is generated from the image data of the layer 1 in which the average value is layered as described above, the image data of the layer 2 can be similarly obtained by the equation (1).

In the conventional motion amount detecting method using a hierarchical image formed by layering images, only the average value layering shown in FIG. 2A is performed, but in this embodiment, the average value layering is performed by the equation (1). 2B at the same time that the generated image data is generated.
As shown in (1), the activity data is generated by hierarchically activating the average value hierarchical image data. Akutei bi Tay data delta ₂ of the hierarchy 2 (x, y)
Then, this activity data Δ ₂ (x, y)
Is the sum of absolute values of the differences of the corresponding pixels of the original image from the averaged hierarchical image data.

[Equation 2] Similarly, the activity data Δ of layer 3 is obtained.
₃ (x, y) is the following formula

[Equation 3] Ask in. By obtaining all the activity data based on the original image, high frequency components faithful to the original image can be extracted.

When the activity data is obtained in this way, its hierarchical structure is as shown in FIG.
It will have a layer plane other than the lowest layer. This activity data reflects the feature amount that is missing in the image data when the average value is hierarchized.

Here, for example, 4 × 4 pixels x1, x2, ... Of the image data in the original image as shown in FIG.
When a block consisting of x15 and x16 is layered as layer 1, pixel y1 in the image data of layer 2 of average value layering,
y2, y3, and y4 are the following equations based on the equation (1).

[Equation 4] The pixel z1 in the image data of the layer 3 which is average value layered by using the 2 × 2 pixels of the layer 2
Based on equation (1),

[Equation 5] Ask by.

Similarly, the activity data of the layer 2 formed by layering the activities is the pixels y1, y2, y3, y of the image data of the layer 2 having the average value layered.
Pixels x1, x2, ..., x1 of image data of 4 and layer 1
Based on equation (2), using 5 and x16, the following equation

[Equation 6] As the activity data of the layer 3 obtained by the above, the pixel z1 of the image data of the layer 3 and the pixels x1, x2, ... Then, based on equation (3),

[Equation 7] Ask by.

By using the average-value-layered image data and the activity-layered activity data obtained as described above, the motion amount is detected by the block matching method in each layer. That is, the evaluation function of the block matching is expressed by the following equation, where t is the current field.

[Equation 8] It is represented by. However (u _n, v _n) indicates the amount of motion of the hierarchy n. V ′ giving the minimum of this evaluation function E (Y) _{n
Let n} = (u _n , v _n ) be the amount of motion. The motion amount V _n in the current layer is calculated by the following equation.

[Equation 9] Therefore, the final amount of movement can be obtained.

The evaluation function in the case of this embodiment is shown in FIG.
As for the activity data of (B), as in the case of (8),

[Equation 10] Then, the new evaluation function E (G) _n is

[Equation 11] And However, w ₁ and w ₂ are weighting factors. Then, the amount of motion that gives the smallest evaluation function E (G) _n is obtained. Since there is no activity data in the lowest layer, evaluation is performed only with the evaluation function E (Y) _{n of the} expression (8).

By using both the average-value-layered image data and the activity-layered activity data for evaluation, even if the optimum evaluation value of one layer is erroneous detection, the other is evaluated. Correct evaluation can be performed with the evaluation value, and the accuracy of motion amount detection in each layer is improved. When the motion amount detection accuracy in each layer is improved as described above, the detection accuracy of the final motion amount obtained by the expression (9) can be improved.

The motion amount detection processing procedure SP0 in this embodiment is shown in FIG. That is, first, step SP1
In (2), the original images of the two screens to be compared are blocked, and the original image blocked in the next step SP2 is averaged into a hierarchical value according to the equation (1).
The activity hierarchization is performed according to the equations (2) and (3), and, for example, the image data of the hierarchy 1, the hierarchy 2 and the hierarchy 3 and the activity data of the hierarchy 2 and the hierarchy 3 are generated.

At the subsequent step SP3, it is determined whether or not the currently processed layer is the lowest layer, and if the result is negative, it is determined at step SP4 whether or not the search has been completed for all search areas, and if a negative result is obtained here. In step SP5, the evaluation data E (Y) _n and E (D) _n described above are used to evaluate the equations (8) and (10) using the current and past data of the image data and the activity data, and step SP4 Return to.

When a positive result is obtained in step SP4, the evaluation function E (Y) obtained for the image data and activity data of a predetermined layer is moved to step SP6.
_{Using n} and E (D) _n , the new evaluation function E (G) _n obtained from the equation (11) is evaluated to obtain the optimum evaluation value, and the process proceeds to step SP7. In step SP7, the motion amount is determined from the optimum evaluation value, the motion amount is added in the next step SP8, and the sum is applied to the lower layers.
Return to. When a negative result is obtained in step SP3, the process proceeds to step SP9 and the processing procedure SP0 of the motion amount detecting method is ended.

In this way, the search in the search area is started from the highest layer, and the optimum value is calculated while comprehensively determining the evaluation values of the average value layered image data and the activity layered activity data by the equation (11). Find the quantity,
While applying the motion amount calculated by the equation (9) to the lower layer, the process is repeated up to the lowest layer to obtain the final motion amount. As a result, the average value is calculated by comparing the motion amount obtained from only the averaged layered image data with the evaluation value obtained from the activity data consisting of high-frequency components of each layer taken into consideration. It is possible to effectively prevent the erroneous determination due to the conversion and detect the motion with high accuracy.

Here, the motion amount detecting apparatus using the motion amount detecting method of this embodiment is constructed as shown in FIG. In the motion amount detecting device 1, the image data of the input original image is input to the block circuit 2,
Blocks are sequentially formed in a predetermined size (for example, a size including a search area for a block size of 16 × 16). As a result, the image data of the original block imaged is the frame memory 3 and the evaluation value calculating circuit 4 of the layer 1, the average value layering circuit 5 of the layer 2, the activity layering circuit 6, and the activity layering circuit of the layer 3. Input to 7.

The average value layering circuit 5 of the layer 2 executes the average value layering process by the calculation of the equation (1) for a predetermined block of the image data of the input original image to obtain the image data of the layer 2. Ask. The image data of the layer 2 obtained as a result are the block circuits 8 of the layer 2,
It is input to the frame memory 9, the activity hierarchization circuit 6, and the average value hierarchization circuit 10 of the hierarchy 3.

The hierarchy 2 activity hierarchy circuit 6 executes the activity hierarchy process by the equation (2) by using the input original image data and the hierarchy 2 image data to perform the hierarchy 2 activity. Ask for data. The resulting activity data of layer 2 is input to the block circuit 11 and frame memory 12 of layer 2, respectively.

The average value layering circuit 10 of layer 3 uses the input image data of layer 2 to perform average value layering processing by the operation of equation (1) to obtain image data of layer 3. The resulting image data of the layer 3 is input to the block circuit 13, the frame memory 14, and the activity layering circuit 7 of the layer 3, respectively.

The hierarchy 3 activity hierarchy circuit 7 uses the input original image data and the hierarchy 3 image data to execute the activity hierarchy process by the operation of equation (3) to execute the hierarchy 3 activity. Ask for data. The resulting activity data of layer 3 is input to the block circuit 11 and frame memory 16 of layer 3, respectively.

As described above with reference to FIG. 2, the image data blocked in this way is averaged into hierarchical values as the image data of the respective layers 1, 2 and 3, and the layers 2 and 3 are also included. Is hierarchized as activity data of.

In the layer 3 which is the uppermost layer, the actual amount of motion is detected. First, the image data and the activity data of the past (that is, one frame before) set in the frame memories 14 and 16 are respectively set according to the search area. The data is read out to the search block circuits 17 and 18. Next, in the evaluation value calculation circuits 19 and 20, using the image data and the activity data of the block circuits 13 and 15 and the search block circuits 17 and 18, respectively (8)
An evaluation value is obtained based on the evaluation functions E (Y) ₃ and E (D) ₃ of the expressions and (10). In the adder circuit 21, the evaluation value is weighted with a predetermined weighting according to the weighting factors w ₁ and w ₂ and added in the addition circuit 21, and a new evaluation function E (G) ₃ obtained as a result is obtained. The evaluation value based on this is input to the motion amount detection circuit 22.

Here, in the case of this embodiment, the evaluation value calculation circuits 19 and 20 are composed of a circuit 40 as shown in FIG. That is, the evaluation value calculation circuit 40 has a reference block memory 41 corresponding to the block circuits 13 and 15, and a candidate block memory 42 corresponding to the search block circuits 17 and 18, respectively.
The contents of 1 and the candidate block memory 42 are read in the order of the addresses designated by the memory control 43, and are subtracted by the subtraction circuit 46 through the registers 44 and 45, respectively. The difference data obtained as a result is converted into an absolute value by the absolute value conversion circuit 47, and cumulatively added by the addition circuit 48 and the register 49.

The cumulative addition result is stored in the evaluation value memory 50,
Inputs are made in the order of addresses designated by the evaluation value memory control 51. In this way, the calculation of the expressions (8) and (10) is executed by the evaluation value calculation circuit 40, and the evaluation value obtained as a result is input to the evaluation value memory 50.
In the actual hierarchy 2 and hierarchy 3, the evaluation value for the average value hierarchical image data and the evaluation value for the activity hierarchical activity data are weighted, added, and stored in the evaluation value memory 50. .

In the case of this embodiment, the motion amount detecting circuit 2
2 is composed of a circuit 60 as shown in FIG. That is, in the motion amount detection circuit 60, the evaluation value memory 50 in which the evaluation value is stored as described above with reference to FIG. 6 is sequentially read according to the address designated by the evaluation value memory control 51, and the comparator 61 and the register are registered. 62 is input.
The comparator 61 sequentially compares the other input with the evaluation value read from the evaluation value memory 50, and when the input evaluation value is smaller, outputs a signal for updating the contents of the registers 62 and 63.

The register 63 has an evaluation value memory 50.
The addresses for reading are sequentially set. In this way, the evaluation values stored in the evaluation value memory 50 are sequentially evaluated,
The address that gives the smallest evaluation value is register 6
3 is output, and this is output as the output of the motion amount detection circuit 60, that is, the motion amount MV composed of a motion vector.

In the case of the motion amount detecting device 1, the motion amount obtained by the motion amount detecting circuit 22 of the layer 3 is given to the frame memory 9 of the image data of the layer 2 and the frame memory 12 of the activity data. The search area is motion-compensated according to the motion amount. That is, when detecting the motion amount of the layer 2, the past (that is, one frame before) image data and activity data set in the frame memories 9 and 12 are respectively searched according to the motion-compensated search area. Read to 24 and 25.

Next, the evaluation value calculation circuits 26 and 27 include the block circuits 8 and 11 and the search block circuits 24 and 25.
Using the past and present image data and activity data of and, evaluation values are obtained based on the evaluation functions E (Y) ₂ and E (D) ₂ of equations (8) and (10), respectively.
In the adder circuit 28, the evaluation values are weighted with a predetermined weighting according to the weighting factors w ₁ and w ₂ and added, as shown in equation (11), and a new evaluation function E (G) obtained as a result is obtained.
_The evaluation value based on ₂ is input to the motion amount detection circuit 29. As a result, the motion amount obtained by the motion amount detection circuit 29 is added to the motion amount of the layer 3 by the addition circuit 23 and is sent as the motion amount of the layer 2 as shown in Expression (9). The evaluation value calculation circuits 26 and 27 and the motion amount detection circuit 29 are also
The configuration is similar to that of the evaluation value calculation circuit 40 and the motion amount detection circuit 60 of FIGS. 6 and 7.

The motion amount obtained in the layer 2 in this manner is given to the frame memory 3 of the image data of the layer 1, and the search area is motion-compensated by this motion amount.
That is, in the motion amount detection of the layer 1, the past (that is, one frame before) image data set in the frame memory 3 is read to the search block circuit 31 in accordance with the motion-compensated search area. Next, the evaluation value calculation circuit 4
Using the image data of the block circuit 2 and the search block circuit 31, the evaluation value is obtained based on the evaluation function E (Y) ₁ of the equation (8).

This evaluation value is input to the motion amount detection circuit 32, and the resulting motion amount of layer 1 is added to the addition circuit 30.
Then, as shown in the equation (9), the motion amount of the layer 2 is added, and the motion amount of the original image is detected in this way, and is output as the output of the motion amount detecting device 1. The evaluation value calculation circuit 4 and the motion amount detection circuit 32 are also configured in the same manner as the evaluation value calculation circuit 40 and the motion amount detection circuit 60 of FIGS. 6 and 7.

In this way, the images are hierarchized at a plurality of resolutions, and when the motion amount is detected by the block matching method using this hierarchal image, the evaluation value obtained for the activity hierarchized activity data is evaluated for each hierarchy. By making it the target of (1), the motion amount can be detected with high accuracy with a small calculation amount. According to the experiment, it was found that the same SN can be obtained with a normalized hardware amount of about 1/10 as compared with the case where the full search is performed for all the original images by block matching.

According to the above construction, the original image is hierarchized by the average value and the activity hierarchies are obtained by summing the absolute values of the differences, and the resulting plural hierarchies of image data and activity data are hierarchically hierarchized. Evaluate more sequentially by block matching,
By comprehensively judging these on the same layer, it is possible to prevent erroneous detection of the motion amount in each layer in advance, and thus the motion amount of the final original image can be calculated with a simple configuration and a short calculation time. The detection accuracy can be significantly improved.

(2) Method and Apparatus for Detecting Motion Amount of Second Embodiment In the first embodiment described above, the average value of the absolute sums of the differences is used when the activity hierarchization is performed. When the activity is hierarchized, the activity data is created using the standard deviation or the variance value. That is, the average value layering is created by the same process as described above. As shown in FIG. 2, the activity layering of layer 2 is performed by using the following equation from the average value layered image data.

[Equation 12] It is calculated by calculating the standard deviation based on.

For the activity hierarchy of the layer 3, the average value layered image data of the layer 3 to the average value layered image data of the layer 2

[Equation 13] It is calculated by calculating the standard deviation based on. Or, from the averaged layered image data of layer 3, see the original image of layer 1

[Equation 14] You may make it calculate a standard deviation like this.

Here, for example, 4 × 4 pixels x1, x2, ... Of the image data in the original image as described above with reference to FIG.
.., x15, x16, when hierarchically layering as a layer 1, first, as the average value layering, the pixels y1, y2, y3, y4 in the image data of the layer 2 are expressed by the equation (4) based on the equation (1). Then, using the 2 × 2 pixels in the image data of layer 2, the pixel z1 in the image data of layer 3 is also calculated by the equation (5) based on the equation (1).

Next, as the activity hierarchization, the activity data of the hierarchy 2 is the pixels y1, y2, y3, y4 in the image data of the hierarchy 2 in which the average value is hierarchized, and the hierarchy 1.
Using the pixels x1, x2, ..., x15, x16 in the image data of, the following equation based on equation (12)

[Equation 15] Level 3 as the activity hierarchy
Image is a pixel z1 in the image data of the layer 3 and the pixels x1, x2, ...
..., x15, x16, based on equation (14),

[Equation 16] Ask by. The motion amount detecting device of the second embodiment is similar to the motion amount detecting device 1 of FIG. 5, and in the activity hierarchy circuits 6 and 7, the activity data based on the standard deviation is obtained as described above.

According to the above structure, the average value hierarchical image data and the activity deviation hierarchical activity data with the standard deviation are sequentially evaluated for each layer from the top layer by block matching and the same. By comprehensively determining these on the layer, it is possible to prevent the erroneous detection of the amount of motion in each layer in advance, thus finally improving the accuracy of detecting the amount of motion in the original image. .

Further, according to the above configuration, the standard deviation is used instead of the average value of the sums of the absolute values of the differences for the activity hierarchization, so that the feature amount for the high frequency component of the original image is more used as the activity data. The stored data can be obtained, and the false detection of the motion amount in each layer can be prevented more effectively.

(3) Method and Apparatus for Detecting Motion Amount of Third Embodiment In the first and second embodiments described above, the average value of the sum of absolute values of differences is used when hierarchizing the activity.
Although the case where the standard deviation is used has been described, in the third embodiment, the activity data is generated by hierarchizing the activity after extracting the high frequency component of the image by using the Laplacian filter for the preprocessing.

That is, as shown in FIG. 8, from the image data (FIG. 8 (A)) in which the average value is hierarchized in the first embodiment,
First, Laplacian data (FIG. 8B) is created by Laplacian filter processing for each layer. This Laplacian data is divided into, for example, 2 × 2 small blocks in the same manner as the image data is created by averaging the average values, and the average value of the absolute values of the Laplacian data in the small blocks is calculated to make the activity data hierarchical. (FIG. 8C) is created.

Here, as the Laplacian filter, as shown in FIG. 9 (A), as shown in FIG. 9 (A), a 3 × 3 coefficient filter in which values are set in consideration of the vertical and horizontal differentiation of the screen, and FIG. 9 (B) is used. As shown, a 3 × 3 coefficient filter in which a value considering the diagonal direction is set is used. The motion amount detecting apparatus of the third embodiment is similar to the motion amount detecting apparatus 1 of FIG. 5, and in the activity hierarchy circuits 6 and 7, as described above, the Laplacian filter processing is pre-processed to make the activity hierarchy. Request activity data.

According to the above-mentioned structure, the average value hierarchical image data and the activity data preprocessed by the Laplacian filter processing and activity hierarchicalized are sequentially subjected to the block matching from the topmost layer for each layer. By making an evaluation and comprehensively determining these on the same layer, it is possible to prevent erroneous detection of the amount of motion in each layer, thus ultimately increasing the accuracy of detecting the amount of motion in the original image. Can be improved.

Further, according to the above-mentioned configuration, by performing the pre-processing by the Laplacian filter processing before the activity hierarchization, it is possible to obtain the data in which the characteristic amount of the high frequency component of the original image is stored as the activity data. Therefore, it is possible to more effectively prevent the erroneous detection of the motion amount in each layer.

(4) Method and Apparatus for Detecting Motion Amount of Fourth Embodiment In the first to third embodiments described above, when hierarchizing the activities, the average value of the sum of absolute differences is used or the standard deviation is used. Although it has been described that the image is used or the Laplacian process is performed as the pre-process on the original image, in the fourth embodiment, the activity is determined by using the dynamic range defined by the difference between the maximum value and the minimum value of the original image. Hierarchical activity data is generated.

In the case of this activity hierarchization, hierarchy 2
When the Akutei bi Tay data Δ ₂ (x, y) that, this Akutei bi Tay data Δ ₂ (x, y) is the average value hierarchical image data, the difference between the maximum value and the minimum value of the corresponding pixel of the original image Using a dynamic range consisting of

[Equation 17] Similarly, the activity data Δ of layer 3 is obtained.
₃ (x, y) is the following formula

[Equation 18] Ask in. By obtaining all the activity data based on the original image, high frequency components faithful to the original image can be extracted.

As described above, the activity hierarchization is performed by using the dynamic range, so that the resulting activity data reflects the missing feature amount when the average value hierarchization is performed. The motion amount detecting device of the fourth embodiment is also the motion amount detecting device 1 of FIG.
Similarly to the above, in the activity hierarchy circuits 6 and 7,
As described above, the activity data is obtained by hierarchizing the activity by the dynamic range.

According to the above-mentioned structure, the average value hierarchical image data and the activity hierarchical activity data by the dynamic range are evaluated for each hierarchical layer sequentially from the top layer by block matching. By comprehensively judging these on the same layer, it is possible to prevent erroneous detection of the amount of motion in each layer, thus ultimately improving the accuracy of detecting the amount of motion in the original image. obtain.

Further, according to the above configuration, the activity hierarchy is stored in the dynamic range reflecting the feature quantity that is missing when the average value is hierarchized, so that the feature quantity is stored as the activity data for the high frequency component of the original image. The obtained data can be obtained, and the false detection of the motion amount in each layer can be prevented more effectively.

(5) Method and Apparatus for Detecting Motion Amount of Fifth Embodiment In the above-described first to fourth embodiments, the block of the original image is blocked when the image data of the upper layer is created by the average value layering. Although the calculation is completed up to the upper rank, in the fifth embodiment, as shown in FIG.
An image value is created by calculating an average value of 16 pixels in a range wider than a block of × 2, for example, a block of 4 × 4 and hierarchically dividing the average value. In order not to change the structure of the upper layer from the conventional one, the blocks of the lower layer are overlapped, and the layers can be layered while maintaining the relationship with the adjacent blocks, so that the amount of movement caused by the block completion can be reduced. False detection can be prevented.

That is, by doing so, the relationship with the adjacent blocks up to the upper hierarchy is not completely broken, but the image data in which the average value is hierarchized with some mutual relation is constructed. It is possible to reduce the probability of false detection in the block matching method in the hierarchy.

For the activity data with hierarchical hierarchies, the average value of the sum of absolute differences as in the first to fourth embodiments is calculated from the image data of the average value hierarchies created by overlapping as described above. , The standard deviation or the variance value, and further the dynamic range can be calculated.

According to the above construction, when the images are layered, the blocks are overlapped to form a block, and the weighted average of the pixels in the block is taken to generate the image data of a plurality of layers in which the average value is layered. At the same time, the activity data is created by hierarchizing the activity according to the image data of multiple layers, and each layer is evaluated sequentially from the top layer by block matching, and these are comprehensively judged on the same layer. By doing so, erroneous detection of the motion amount in each layer can be prevented in advance, and thus the motion amount detection accuracy of the original image can be significantly improved.

Further, according to the above configuration, in the average value hierarchy and the activity hierarchy, the blocks are overlapped to form the image data and the activity data. Therefore, the erroneous detection of the motion amount can be prevented in advance.

(6) Method and Apparatus for Detecting Motion Amount of Sixth Embodiment In the above-described first embodiment, block matching is performed for each layer of image data and average activity hierarchical activity data. A new evaluation function E is obtained by weighting and adding the evaluation functions E (Y) _n and E (D) _n with weights w ₁ and w ₂ as shown in equation (11).
(G) _n was calculated. In this case, the weights w ₁ and w ₂ are calculated as the value 1 by equal weighting.
In the embodiment, the weight is adaptively changed so as to reduce the error in the motion amount detection.

(6-1) Adaptation Process 1 In the adaptation process 1, first, the standard deviations σ _Y and σ _D of the blocks of the average value hierarchical image data and the activity hierarchical activity data are calculated. Think It is considered that the larger the standard deviations σ _Y and σ _D , the steeper the mortar shape of the evaluation table and the greater the credibility of its optimum value. Therefore, in practice, using the standard deviation values σ _Y and σ _D ,

[Formula 19] Then, a new evaluation function E (G) _n is obtained by normalization and adaptation processing by

(6-2) Adaptation process 2 In the adaptation process 2, first, it is evaluated that the absolute levels of the optimum evaluation values are different between the average value hierarchical image data and the activity hierarchical activity data. Since the weight of changes, the normalization is performed so that the levels of the optimum evaluation values are the same. That is, the respective optimum evaluation values are defined as e _Y and e _D , and

[Equation 20] Then, a new evaluation function E (G) _n is obtained by normalization and adaptation processing by

(6-3) Adaptation process 3 In the simple addition value of the evaluation values of the average value hierarchical image data and the activity hierarchical activity data, the relationship between the shape of the evaluation value and the weighting ratio is correlated. It was discovered through experiments that there was a relationship. According to this, when there are many blocks with sharp evaluation values, it is better to increase the weight of the average value with respect to the evaluation value. This adaptation processing 3
Then, the relations are made into a table by using various sources in advance, and the weight ratio is output to the shape of a certain evaluation value and re-evaluated.

As a method for quantifying the shape of the evaluation value, the evaluation value around the optimum evaluation value is used. For example, in FIG.
As shown in FIG. 8, for the central pixel x, eight surrounding points e1 to e
Using an evaluation value of 8, the shape measure S is

[Equation 21] Can be found at. However, N is the number of data existing around the optimum evaluation value. The value of the scale S is used to construct a look-up table (LUT), an optimum weighting coefficient is output from this table, and the evaluation by the equation (11) is performed again.

(6-4) Adaptation processing 4 It is considered that the steeper the shape of the evaluation value, the higher the possibility that the optimum evaluation value is correct. Also, if the shape of the evaluation value is almost flat, especially if one shape is flat,
There is little adverse effect on equation (11). Therefore, in the adaptation process 4, weighting is applied to each evaluation value shape of the evaluation values of the image data in which the average value is hierarchized and the activity data in which the activity is hierarchized.

The quantitative scale of each evaluation value is considered in the same manner as the equation (21), and if the image data having the average value layered is S _Y and the activity data having the activity layered is S _D , the following equation is obtained.

[Equation 22] And the following equation

[Equation 23] And the evaluation function is

[Equation 24] Is evaluated, and a new evaluation function E (G) _n is obtained.

(6-5) Adaptation processing 5 The contents of the evaluation value memory have different values depending on the characteristics (flat, edge, etc.) of the image in the block. Generally, a flat image has a small change in the evaluation value, and an image including an edge or the like has a large change in the evaluation value. Therefore, if a new evaluation function is obtained by equal addition, the position of the true minimum value may shift, resulting in erroneous detection as motion amount detection.

Therefore, in this adaptation processing 5, first, the evaluation function E (Y) _n for the image data in which the average value is hierarchized.
Of the maximum value E (Y) _nmax and the minimum value E (M) _nmin of
The following formula

[Equation 25] And the evaluation function E (D) _n for the activity data in which activity hierarchization is performed in the same manner.
Of the maximum value E (D) _nmax and the minimum value E (D) _nmin of
The following formula

[Equation 26] Normalize using.

Using the evaluation value table normalized in this way, a new evaluation function E (G) _n

[Equation 27] I will ask for it.

(6-6) Adaptation processing circuit In each of the above-described adaptation processing 1 to 5, in the motion amount detection device 1 described above with reference to FIG. 5, the average value hierarchical image data and activity hierarchy of each hierarchical layer. Evaluation value calculation circuits 26, 19 and 2 for the converted activity data
The contents of the evaluation value memory obtained in steps 7 and 20 are added to the adder circuit 2
It is executed when adding 8 and 21. Incidentally, FIG. 12 shows the configuration of the adaptation processing circuit 70 for realizing the adaptation processing 5.

That is, in the adaptation processing circuit 70, the evaluation value calculation circuit 2 for the average value hierarchical image data and the activity hierarchical activity data is calculated.
6, 19 and 27, the evaluation value memory 71 obtained by
The evaluation values E (Y) _n and E (D) _{n of} 72 are the maximum / minimum value detection circuits 73 and 74 and the delay circuit 75, respectively.
It is input to 76.

The maximum value / minimum value detection circuits 73 and 74 have the maximum value E for the respective evaluation values E (Y) _n and E (D) _n.
(Y) _nmax , E (D) _nmax and minimum values E (Y) _nmin , E
(D) Obtain _nmin . Maximum value E (Y) obtained as a result
_nmax and E (D) _nmax are the respective first subtraction circuits 77,
It is input to 78 and the minimum value E (Y) _nmin , E (D)
_nmin is input to the first subtraction circuits 77 and 78 and the second subtraction circuits 79 and 80, respectively. Second subtraction circuit 7
The evaluation values E (Y) _n and E (D) are shown at 9 and 80, respectively.
_n is input after being delayed by the processing time of the maximum / minimum value detection circuits 73 and 74.

The subtraction outputs of the first subtraction circuits 77 and 78 and the second subtraction circuits 79 and 80 are divided by the division circuits 81 and 81, respectively.
It is input to 82 and is divided, and the output is added by the adder circuit 83. As a result, the operations of the above equations (25) to (27) are executed. The new evaluation function E (G) _n subjected to the adaptation processing 5 in this manner is stored in the evaluation value memory 84 and sent to the motion amount detection circuit. Note that the evaluation value E (Y) for the lowest layer is the original image.
Calculate ₁ to detect the minimum value.

(6-6) Effects of Adaptation Processing According to the above configuration, the original image is hierarchized in the average value and activity hierarchies, and the resulting image data and activity data of plural hierarchies are
Each layer is evaluated sequentially from the top layer by block matching, and a new evaluation function obtained by optimal weighting is used when comprehensively judging these on the same layer. The erroneous detection of the motion amount in each layer can be prevented, and thus the motion amount detection accuracy of the final original image can be further improved.

(7) Method and Apparatus for Detecting Motion Amount of Seventh Embodiment In the first to fourth embodiments described above, when the image data of the upper layer is created by the average value layering, the block of the original image is blocked. Although the calculation was completed up to the upper level, the operation such as average value stratification and activity stratification was completed in block units, so there was a risk that the motion amount would be erroneously detected. For this reason, in the fifth embodiment, when the average value hierarchy is used, the blocks of the lower hierarchy are overlapped so that the hierarchy is maintained while maintaining the relationship with the adjacent blocks. However, in the seventh embodiment, the average value hierarchy is used. When obtaining the evaluation values of the converted image data and the activity hierarchized activity data, a plurality of, for example, two types of evaluation values are obtained and motion compensation is performed in the lower hierarchy.

That is, in this motion amount detecting method, similarly to the above-described first embodiment, the evaluation value in the search area of the average value hierarchical image data is E (Y) _n , and the activity is hierarchical. E (D) _{n is used} as the evaluation value of the activity data within the search area, and an evaluation value table is created in each evaluation value memory. This evaluation value table has considerably different values depending on the characteristics (flat, edge, etc.) of the image in the block. Generally, a flat image has a small change in the evaluation value table, and an image including an edge or the like has a large change in the evaluation value. Normally, the values in the evaluation value table have a mortar shape such that the evaluation value at the position corresponding to the amount of movement is based on the minimum point, but depending on the image, it is fairly flat, or there are multiple minimum points. There are also things to do. In particular, this tendency becomes strong in the upper layer in the image data having the average value layered.

Therefore, in this motion amount detecting method, a plurality of local minimum points in the evaluation value table in the uppermost hierarchy are detected and used as the motion amount candidates in the first stage. Specifically, there are two types of evaluation value tables in the highest hierarchy, one having an average value hierarchy and one having an activity hierarchy, and a plurality of minimum points are detected from the smallest one in each. Then, the positions of the minimum points in the average value hierarchy and the positions of the minimum points in the activity hierarchy are logically ORed to eliminate the overlap, and the positions of the plurality of minimum points in hierarchy 3 are used as candidates for the amount of movement in the first stage. To do.

Next, the following operation is performed for each motion amount candidate in the layer 3, that is, the highest layer. First, in Tier 2 in the average value stratification, select one of the motion amounts,
A search range is set with this position as the center. Within the search range, level 2 block matching is performed to calculate the evaluation value. Similarly, in the hierarchy 2 of the activity hierarchy, the search range is set around the position of the selected motion amount, and the block matching of the hierarchy 2 is performed within the search range to calculate the evaluation value.

Subsequently, the evaluation value in the average value hierarchy in the hierarchy 2 and the evaluation value in the activity hierarchy are weighted and added to detect the minimum point. The position of this minimum point is set as the movement amount in the layer 2 corresponding to one of the movement amounts in the first stage. Next, the search range in the layer 1 is determined centered on one of the plurality of initial stage motion amounts determined as described above and the position determined by the corresponding layer 2 motion amount.

Layer 1 is the layer of the original image, only one type of block matching is performed, and the minimum evaluation value calculated by the sum of absolute differences is detected. The position of this minimum value becomes the movement amount of the final stage corresponding to one of the movement amounts of the first stage. The above operation is performed for each determined amount of movement in the first stage, and a plurality of positions of the minimum evaluation values in layer 1 are obtained. From among these, the one having the smallest evaluation value is detected, and the final movement is detected. Send as quantity.

FIG. 13 shows the motion amount detection processing procedure SP10 in the seventh embodiment. That is, first, in step SP11, the original images of the two screens to be compared are blocked, and the original image that is blocked in the next step SP12 is averaged and activity hierarchized. Image data and layer 2 and layer 3 activity data are generated.

At the following step SP13, it is judged whether or not the currently searched hierarchy is the highest hierarchy, that is, hierarchy 3, and if the result is negative, at step SP14 the average value hierarchical image data and the activity hierarchical activity. The search area is searched by using the current and past data, the evaluation value by the evaluation function is calculated, and the process returns to step SP13. When an affirmative result is obtained in step SP13, the process proceeds to step SP15, n minimum points are selected from the evaluation value, and the process proceeds to step SP16.

At step SP16, it is judged whether or not the processing at the layer 2 and the layer 3 has been completed for the n minimum points obtained at the highest layer, and if a negative result is obtained here, the process proceeds to step S17. Layer is the lowest layer, that is, layer 1
It is determined whether or not, and if a negative result is obtained here, step SP1
8, the search area is searched using the current and past data of the layer 2 image data and activity data, and the evaluation is performed by the evaluation function.
Return to.

When a positive result is obtained in step SP18, the evaluation function obtained for the image data and the activity data of the layer 2 is used to perform an evaluation with a new evaluation function to obtain an optimum evaluation value, and the process proceeds to step SP21. In step SP21, the motion amount is determined from the optimum evaluation value, applied to the lower layer, and the process returns to step SP17.

When a negative result is obtained in step SP17, the process returns to step SP16, and steps SP17-S are executed for the n minimum values of the evaluation values obtained in the highest hierarchy.
In P18-SP19-SP20-SP21, it is determined whether or not the motion amounts of the layers 2 and 1 have been detected. If a positive result is obtained, the process moves to step SP22 and the minimum motion amount of the motion amounts obtained in the layer 1 is obtained. As the final motion amount, and in step SP23 the motion amount detection processing procedure SP10
To finish.

Here, the motion amount detecting apparatus using the motion amount detecting method of this embodiment is constructed as shown in FIG. In this motion amount detecting device 90, the image data of the input original image is input to the block circuit 91 and the frame memory 92 of the layer 1. The block circuit 91 converts the input image data into a predetermined size (for example, 16 × 16).
Block size), and the resulting image data of the original block image is
It is inputted to the motion amount detection circuit 93 of the layer 1 and the layering circuits 94 and 95 of the layer 2 and the layer 3, respectively. The hierarchization circuits 94 and 95 are configured to include average value hierarchization and activity hierarchization, respectively.

The image data and the activity data obtained by layering in the layer 2 layering circuit 94 are input to the frame memory 97 and the first and second motion amount detecting circuits 98 and 99. Further, the image data and the activity data obtained by being layered by the layering circuit 95 of the layer 3 are the frame memory 100 and the motion amount detecting circuit 101.
Entered in. The image data blocked in this way is averaged into hierarchical values as the image data of the respective layers 1, 2, and 3, and is also activity hierarchical as the activity data of the layers 2 and 3.

In the layer 3 which is the uppermost layer, the actual amount of motion is detected by first searching the past (that is, one frame before) image data and the activity data set in the frame memory 100 according to the search area. Read out to the circuit 102. Next, the motion amount detection circuit 1
At 010, the image data and the activity data of the hierarchical circuit 95 and the search block circuit 100 are used to obtain evaluation values based on the respective evaluation functions, and these are weighted according to the weighting coefficient and added, and the result is obtained. Two minimum points having the minimum evaluation value based on the new evaluation function are selected, and these are transmitted as two movement amounts of the layer 3, respectively.

In the case of this embodiment, the two motion amounts obtained by the motion amount detecting circuit 101 of the layer 3 are input to the search block circuits 103 and 104 and the adding circuits 105 and 106 of the layer 2, respectively. In the layer 2, when the image data and the activity data stored in the frame memory 97 are read to the search block circuits 103 and 104, the search areas are motion-compensated according to different motion amounts.

As a result, when the motion amount is detected in the layer 2, the motion amount detecting circuits 98 and 99 set the past (that is, 1 in the search block circuits 103 and 104).
Using the image data and activity data (before the frame) and the current image data and activity data input from the layering circuit 95, the evaluation value is obtained based on the evaluation function. Next, these evaluation values are weighted and added according to the weighting coefficient, and the one that minimizes the evaluation value based on the new evaluation function obtained as a result is detected and transmitted as each motion amount.

The motion amounts obtained by the motion amount detection circuits 98 and 99 of the layer 2 are added to the corresponding motion amounts of the layer 3 in the adder circuits 105 and 106, respectively, and the motion amount of the layer 1 is added as the motion amount of the layer 2. Search block circuit 107, 1
08 and the addition circuits 109 and 110, respectively.
In the layer 1, when the image data and the activity data stored in the frame memory 92 are read to the search block circuits 107 and 108, the search areas are motion-compensated according to different motion amounts.

As a result, when the motion amount is detected in the layer 1, the motion amount detection circuits 93 and 94 set the past (that is, 1) in the search block circuits 107 and 108.
Using the image data and activity data (before the frame) and the current image data and activity data input from the block circuit 91, the evaluation value is obtained based on the evaluation function. Next, these evaluation values are weighted according to the weighting coefficient and added, and the one that minimizes the evaluation value based on the new evaluation function obtained as a result is detected and transmitted as each motion amount.

In this way, the motion amount detecting circuits 93, 94
The amount of movement obtained by
It is sent to 0 and input to the judgment circuit 111.
The motion amounts input to the adder circuits 109 and 110 are added to the corresponding motion amounts of the layer 2 and are input to the selection circuit 112 as the motion amount of the layer 1. Judgment circuit 111
Determines which of the input movements gives the minimum,
The selection circuit 112 selects the motion amount of the layer 1 based on the motion amount that gives the minimum, and sends it as the final motion amount.

According to the above configuration, the original image is hierarchized by average value to create image data of a plurality of hierarchies, and the activity hierarchization is performed to create activity data representing high frequency components of the image data for each hierarchies. Calculate the evaluation value of block matching in the highest layer, select multiple minimum point positions of the new evaluation value of the highest layer obtained by comprehensively judging this evaluation value, and select it as a candidate for the amount of movement. By performing block matching in the lower hierarchy sequentially for each amount of motion, the amount of motion is calculated from the minimum evaluation value, and the final amount of motion is obtained with the minimum evaluation value of the lowest hierarchy. Thus, it is possible to comprehensively evaluate a plurality of motion amount candidates in the highest layer from the candidates to the lowest layer, and finally, the motion amount detection accuracy can be significantly improved.

(8) Other Embodiments (8-1) In the above embodiment, the case where the image data of three layers is created from the original image by the average value layering is described, but the number of layers is not limited to this. Alternatively, even if the number of layers is two or four or more, the same effect as that of the above-described embodiment can be realized. Also, when the average value is hierarchized, the average value is calculated in the block range of 2 × 2 as in the formula (1). You can change it. For example, in the case of filtering a wider block range of 4 × 4, if the weighting factors are expressed by horizontal and vertical matrices, the following equation is obtained.

[Equation 28] Can be expressed as

(8-2) In the above-described first embodiment, when the activity hierarchization is performed, the hierarchy 1 of the original image is formed.
Although the activity data is obtained by the sum of the absolute values of the differences between the image data of No. 1 and the image data of the hierarchy to be obtained, the activity of the hierarchy n is replaced by Δ _n (x,
y)

[Equation 29] You may ask in. In this case, since the image data in which the average value one layer below is used is used, the activity data is obtained from the image data in which the feature amount is missing from the original image. Then the hardware scale becomes smaller. In addition to this, Tier 2
The activity data of is obtained by the equation (2), and once the activity data is obtained, the following equation is obtained.

[Equation 30] Alternatively, the activity data of the upper hierarchy may be sequentially obtained.

(8-3) In the above embodiment, in order to obtain the evaluation value of the block matching, the absolute value of the difference between the two blocks is calculated by the evaluation function of the expressions (8) and (10). Although the sum is used, in addition to this, since the evaluation function of the block matching is obtained, the same effect as that of the above-described embodiment can be realized by applying the sum of squared difference or the correlation coefficient.

(8-4) In the second embodiment described above,
By calculating the standard deviation based on the equation (12) from the image data obtained by layering the average values, the activity is layered and the activity data is used, but instead of this, the variance value is used. Also, the same effect as that of the above-described embodiment can be realized. Incidentally, in this case, the value before calculating the square root may be used in the above-mentioned expression (12).

(8-5) In the above third embodiment,
The case where the Laplacian processing is performed on the image data of the layer 1 and the layer 2 has been described, but instead of this, only the original image in the lowest layer is processed by the Laplacian filter, for example, in a block of 2 × 2 of Laplacian data. It is also possible to calculate the average value of the absolute values of the above to create the activity data of the layer 2, and the activity data of the higher level may be created by layering the average value of the activity data of the lower level.

Alternatively, the maximum absolute value of the small blocks in the Laplacian data of the lowest hierarchy is used as activity data to make the hierarchy 2 an activity hierarchy, and the activity hierarchy of a higher hierarchy is a small block of the lower activity data. The maximum value within may be used as the activity data.

Furthermore, instead of this, regarding the difference between the maximum value and the minimum value of the small blocks in the Laplacian data of the lowest hierarchy, that is, the dynamic range as the activity data, the hierarchy 2 is made into the activity hierarchy and the activity hierarchy of the higher hierarchy is concerned. May be taken as the activity data by taking the maximum value of the lower order activity data in a small block.

(8-6) In the above-mentioned fourth embodiment,
When making the activity hierarchy, the activity data is obtained by using the dynamic range which is the difference between the maximum value and the minimum value of the image data of the layer 1 which is the original image and the image data of the layer to be obtained. , The activity data of layer n is Δ _n (x, y),
The following formula

[Equation 31] You may ask in. In this case, since the image data that is one layer lower than the original image is used, the activity data is obtained from the image data in which the feature amount is missing from the original image. Compared with the cases of equations (17) and (18), The scale of hardware becomes smaller.

In addition to this, the activity data of layer 2 is obtained by the equation (17), and once the activity data is obtained, the following equation is obtained.

[Equation 32] There is also a method of sequentially obtaining the activity data of the upper hierarchy by. Further, in the case of activity hierarchization in the dynamic range, the original image may be hierarchized using the maximum value or the minimum value instead of the average value hierarchization. In this way, the amount of calculation in the activity hierarchy can be reduced, so that the circuit configuration can be simplified and the calculation time can be shortened.

(8-7) In the above fifth embodiment,
As shown in FIG. 10, the average value layered image data of layer 3 is created from layer 2, but the present invention is not limited to this.
In the layer 1, for example, 6 × 6 blocks may be overlapped to form an average value layer.

(8-8) In the above fifth embodiment,
Although the average value is calculated by overlapping, the weighted average may be calculated by giving more weight to the central pixel of the block when calculating the average value. By weakening the influence of the adjacent block on the hierarchical image in this way, the sensitivity of the motion amount detection can be adjusted. Similarly, the original image of layer 1 may be subjected to a two-dimensional low-pass filter process and pre-processed so that the influence of adjacent pixels can be added to some extent, and then the average value may be used for layering. Also, the hierarchical image is formed by average value layering without overlapping, and when the activity layering is performed, the average value of the absolute value sums of the differences is calculated by the overlapping blocks in the lower layered average value layered image data. You may create it by doing so.

(8-9) In the seventh embodiment described above,
Regarding each of the evaluation value in the average value hierarchy and the evaluation value in the activity hierarchy, the case where a plurality of local minimum values are used as candidates for the amount of movement in the first stage is described, but instead of this, in the average value hierarchy After weighting and adding the evaluation value and the evaluation value in the activity hierarchy, a plurality of local minimum points may be detected and their positions may be used as candidates for the amount of movement in the first stage.

(8-10) In the seventh embodiment described above,
Although the minimum position is selected as a candidate for the amount of movement in the hierarchy 2, a plurality of minimum points are selected for each evaluation value in the average value hierarchy and the evaluation value in the activity hierarchy, as in the top hierarchy, and further lower ranks are selected. It is also possible to increase the number of branches in the detection of the motion amount in the hierarchy, detect the smallest evaluation value of all the evaluation values in the lowest hierarchy, and send it as the final motion amount.

[0116]

As described above, according to the present invention, as described above, according to the present invention, image data of a plurality of layers having different resolutions than an input image is formed, and based on the image data of a plurality of layers, Forming multiple layers of activity data that represent the high-frequency components of the image data for each layer, and evaluating the image data and activity data corresponding to the input images at different times by block matching in predetermined block units for each layer. By calculating the value and making a comprehensive judgment of each evaluation value to detect the motion amount between the input images at different times for each layer, it is possible to prevent the motion amount in each layer from being erroneously detected. It is possible to realize a motion amount detection method and a motion amount detection device capable of improving the motion amount detection accuracy of a final input image with a simple configuration and a short calculation time. .

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining a method of detecting a movement amount by block matching.

FIG. 2 is a schematic diagram for explaining a hierarchical image according to an embodiment.

FIG. 3 is a schematic diagram for explaining a hierarchical image according to an example.

FIG. 4 is a flowchart showing a motion amount detection processing procedure according to the embodiment.

FIG. 5 is a block diagram showing a configuration of a motion amount detecting device according to an embodiment,

6 is a block diagram showing a configuration of an evaluation value calculation circuit in the motion amount detection device of FIG.

7 is a block diagram showing a configuration of a motion amount detection circuit in the motion amount detection device of FIG.

FIG. 8 is a schematic diagram showing preprocessing by a Laplacian filter as an explanation of a motion amount detection method according to a fourth embodiment.

FIG. 9 is a schematic diagram showing an example of coefficients of a Laplacian filter as an explanation of a motion amount detecting device according to a fourth embodiment.

FIG. 10 is a schematic diagram for explaining the mean value hierarchization of the fifth embodiment.

FIG. 11 is a schematic diagram showing a relationship between an optimum evaluation value and a neighborhood evaluation value as an explanation of a motion amount detecting method according to a sixth embodiment.

FIG. 12 is a block diagram showing a configuration of an adaptation processing circuit as a motion amount detecting device according to a sixth embodiment.

FIG. 13 is a flow chart showing a motion amount detection processing procedure as an explanation of the motion amount detecting method according to the seventh embodiment.

FIG. 14 is a block diagram showing the configuration of a motion amount detecting device according to a seventh embodiment.

[Explanation of symbols]

1, 90 ... Motion amount detecting device 2, 8, 11, 13, 1
5, 91 ... Block circuit, 3, 9, 12, 14, 1
6, 92, 97, 100 ... Frame memory, 4, 1
9, 20, 26, 27, 40 ... Evaluation value calculation circuit 5,
10 ... Average value hierarchy circuit, 6, 7 ... Activity hierarchy circuit, 17, 18, 24, 25, 31, 102,
103, 104, 107, 108 ... Search block circuit 21, 23, 28, 30, 48, 105, 106,
109, 110 ... Addition circuit, 22, 29, 32, 6
0, 93, 94, 98, 99, 101 ... Motion amount detection circuit, 41 ... Reference block memory, 42 ... Candidate block memory, 43 ... Memory control, 44, 45,
49, 62, 63 ... Register, 46 ... Subtraction circuit, 4
7 ... Absolute value conversion circuit, 50 ... Evaluation value memory, 51 ...
Evaluation value memory control, 61 ... Comparator, 95, 9
6 ... Hierarchical circuit, 111 ... Judgment circuit, 112 ... Selection circuit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Ken Hori, 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Toshiya Ishizaka 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation

Claims (32)

[Claims] 1. An image layering step for forming image data of a plurality of layers having different resolutions from an input image, and an activity of a plurality of layers representing a high frequency component of the image data of each layer based on the image data of the plurality of layers. The activity hierarchy step forming data, and the image data layered by the image hierarchy step and the activity data layered by the activity hierarchy step corresponding to the input images at different time points are described above. An evaluation value calculation step for obtaining an evaluation value by block matching in a predetermined block unit for each layer, and for the image data and the activity data, comprehensively judge each evaluation value obtained by the evaluation value calculation step, The amount of motion between the input images at the different times for each layer is Motion estimation method characterized by comprising a motion estimation step for output. 2. The evaluation value calculation step and the motion amount detection step are executed on the image data and the activity data of the upper layer, and the lower layer is calculated based on the motion amount of the upper layer which is the execution result. The evaluation value calculation step and the motion amount detection step are performed by motion-compensating the block of the block matching, and finally between the input images according to the motion amount of the upper layer and the motion amount of the lower layer. The motion amount detecting method according to claim 1, wherein the motion amount is detected. 3. The image layering step is characterized in that an average value of the input image is obtained in a predetermined block unit, and the image data of an upper layer is formed according to the average value. The motion amount detection method according to claim 1 or claim 2. 4. The image layering step is characterized in that the input image is subjected to a filter process in units of a predetermined block, and the image data of an upper layer is formed according to a result of the filtering. The motion amount detection method according to claim 1 or claim 2. 5. In the image layering step, the maximum value or the minimum value of the input image is obtained in a predetermined block unit, and the image data of the upper layer is formed according to the maximum value or the minimum value. Claim 1 characterized by the above-mentioned.
Alternatively, the motion amount detecting method according to claim 2. 6. In the activity hierarchization step,
3. The motion amount according to claim 1 or 2, wherein the activity data is formed by using a sum of absolute values of differences between the hierarchical upper layer image data and the lower layer image data. Detection method. 7. The activity hierarchy step comprises:
3. The motion amount detecting method according to claim 1, wherein the activity data is formed by using the standard deviation of the hierarchical image data. 8. In the activity hierarchization step,
3. The motion amount detecting method according to claim 1, wherein the activity data is formed by using the variance value of the hierarchical image data. 9. In the upper activity hierarchy step,
The activity data is formed based on an average of absolute values of Laplacian data obtained by subjecting the layered image data to a Laplacian filter process. Quantity detection method. 10. The upper activity hierarchy step is characterized in that the activity data is formed based on a dynamic range which is a difference between the maximum value and the minimum value of the hierarchical image data. The motion amount detection method according to claim 1 or 2. 11. The upper activity hierarchy step is characterized in that the activity data is formed on the basis of a dynamic range defined by a difference between the maximum value and the minimum value of the hierarchical image data. The motion amount detection method according to claim 1 or 2. 12. The movement according to claim 1, wherein in the upper activity hierarchy step, the activity data is formed based on the maximum value of the hierarchized image data. Quantity detection method. 13. The movement according to claim 1 or 2, wherein in the upper activity hierarchization step, the activity data is formed based on the minimum value of the hierarchized image data. Quantity detection method. 14. The evaluation value calculation step obtains the evaluation value obtained by blocking the image data layered by the image layering step and the activity data layered by the activity layering step. The motion amount detecting method according to claim 1 or 2, wherein the evaluation value to be evaluated is weighted and added. 15. The image hierarchization step and / or the activity hierarchization step comprises forming the hierarchized image data and / or the activity data using an input image. , Claim 2, claim 3, claim 4, claim 5, claim 6, claim 7, claim 8, claim 10, claim 11, claim 12, or claim 13. Motion detection method. 16. The image hierarchization step and / or the activity hierarchization step is characterized in that the hierarchized image data and / or the activity data is formed by using image data one level lower. Claim 1, claim 2, claim 3, claim 4, claim 5, claim 6, claim 7, claim 8, claim 9, claim 10, claim 11, claim 12 or The motion amount detection method according to claim 13. 17. An image layering means for forming image data of a plurality of layers different in resolution from an input image, and an activity of a plurality of layers representing a high frequency component of the image data for each layer, based on the image data of the plurality of layers. With respect to the activity layering means for forming data, the image data layered by the image layering means and the activity data layered by the activity layering means corresponding to the input images at different time points, respectively. Evaluation value calculating means for obtaining an evaluation value by block matching in a predetermined block unit for each layer, and for the image data and the activity data, comprehensively judge each evaluation value obtained by the evaluation value calculating means, Motion amount detecting means for detecting a motion amount between the input images at the different times for each of the layers Motion estimation apparatus characterized by comprising a. 18. With respect to the image data and the activity data of the upper layer, the evaluation value calculating means and the motion amount detecting means detect the motion amount of the upper layer, and the lower layer is detected based on the motion amount. The evaluation value calculation means and the movement amount detection means perform motion compensation on the block of the block matching, and finally the movement amount between the input images is calculated according to the movement amount of the upper layer and the movement amount of the lower layer. The detection is performed according to claim 1.
7. The motion amount detecting device described in 7. 19. The image layering means obtains an average value of the input image in units of a predetermined block, and forms the image data of an upper layer according to the average value. The motion amount detecting device according to claim 17 or claim 18. 20. The image layering means filters the input image in units of a predetermined block, and forms the image data of a higher layer in accordance with the filtering result. The motion amount detecting device according to claim 17 or claim 18. 21. The image layering means obtains a maximum value or a minimum value in units of a predetermined block for the input image,
The motion amount detecting device according to claim 17 or 18, wherein the image data of an upper layer is formed according to the maximum value or the minimum value. 22. The activity layering means forms the activity data by using the sum of absolute values of the differences between the layered upper layer image data and the layered lower layer image data. The motion amount detection device according to claim 17 or 18. 23. The movement according to claim 17, wherein the activity hierarchizing means forms the activity data by using the standard deviation of the hierarchized image data. Quantity detection device. 24. The movement according to claim 17, wherein the activity hierarchizing means forms the activity data by using a variance value of the hierarchized image data. Quantity detection device. 25. The upper activity layering means forms the activity data based on an average of the absolute values of the Laplacian data obtained by subjecting the layered image data to the Laplacian filter process. 19. The motion amount detecting device according to claim 17 or 18. 26. The upper activity hierarchizing means forms the activity data based on a dynamic range defined by a difference between the maximum value and the minimum value of the hierarchized image data. Item 17
Alternatively, the motion amount detecting device according to claim 18. 27. The upper activity hierarchizing means forms the activity data on the basis of a dynamic range defined by a difference between the maximum value and the minimum value of the hierarchized image data. Item 17
Alternatively, the motion amount detecting device according to claim 18. 28. The movement according to claim 17 or 18, wherein the upper activity hierarchizing means forms the activity data based on the maximum value of the hierarchized image data. Quantity detection device. 29. The movement according to claim 17 or 18, wherein the upper activity layering means forms the activity data based on the minimum value of the layered image data. Quantity detection device. 30. The evaluation value calculation means obtains the evaluation value obtained by blocking the image data layered by the image layering means and the activity data layered by the activity layering means. The motion amount detecting device according to claim 17 or 18, wherein the evaluation value to be evaluated is weighted and added. 31. The image hierarchizing means and / or the activity hierarchizing means are adapted to form the hierarchized image data and / or the activity data by using an input image. , Claim 18, claim 19, claim 20, claim 21, claim 22, claim 23, claim 24, claim 25, claim 2.
The motion amount detection device according to claim 6, claim 27, claim 28, or claim 29. 32. The image hierarchizing means and / or the activity hierarchizing means is configured to form the hierarchized image data and / or the activity data by using image data of one hierarchy lower. Claim 17, claim 18, claim 19, claim 20, claim 21, claim 22, claim 23, claim 24, claim 25, claim 26, claim 27, claim 28 or The motion amount detecting device according to claim 29.