JP2000236554A - Image processing unit and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a motion vector with high accuracy between images with different density. SOLUTION: A motion vector is detected between a reference image Ps and an image Ph with a density at a multiple of 4. The 4 multiple density image Ph is separated into foru images Ph1-Ph4 whose line phase differs. A block of the same position is formed spatially between the reference image Ps and the image Ph1, an absolute value of the difference of the pixel between blocks is calculated and the absolute values are collected in the block. Similarly the absolute sum of the differences of the images Ph2-Ph4 is calculated. Then the absolute sum obtained from the 4 images is synthesized. A table storing the absolute sums after the synthesis is obtained by precision of the 4 multiple density image. A motion vector is detected by detecting a minimum value in the final absolute value sum table.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus and method applied to detection of a motion vector.

[0002]

2. Description of the Related Art A motion vector represents a temporal motion of an image, and is used, for example, in motion compensation inter-frame predictive coding. As one of the motion vector detection methods, block matching is known. This subdivides two temporally different images into blocks,
The position of a block cut out from one image is sequentially shifted in x (horizontal direction) or y (vertical direction) in units of one pixel,
In the shifted state, the absolute value of the difference between the pixels at the same spatial position in the block is calculated, the absolute values are totaled for each block, and the sum of the absolute differences is calculated. This is a block motion vector. In general, the density of pixels of an image for which a motion vector is to be detected is equal.

[0003] However, there are cases where motion vectors are detected between images of different densities. A high-density image (for example, an HD (High Definition) equivalent) is a low-density image (for example, an SD (Standard Definition) equivalent).
It has twice the density. An example of a possible method for detecting a motion vector between images having different densities will be described.

As an example, as shown in FIG. 5, a description will be given of the detection of a motion vector between one single-density image and a quadruple-density image having a density four times higher in the vertical direction than the single-density image. I do. In order to detect a motion vector, the density of the low-density image Ps is adjusted by thinning out the quad-density image Ph. In FIG. 6, three lines are thinned out from four consecutive lines of the high-density image Ph (such thinning is referred to as 間 thinning) to create a low-density image Ph ′. Then, a motion vector is detected between the low-density image Ph 'and the reference image.

That is, as shown in FIG. 7, the reference block Bs and the reference block Bs having the same size and the same shape are used.
Consider h ′ (5 × 5 blocks are shown in FIG. 7) at corresponding positions in images Ps and Ph ′. First,
First, the absolute value of the difference between the pixel values of the pixels existing at the same position as the reference block Bs is obtained, and the absolute values are totaled over one block to obtain the absolute value sum. Next, the reference block Bh ′ is moved to various positions on a pixel-by-pixel basis, and the sum of absolute values is obtained at the position after the movement. The moving range is defined as a search range. Then, of all the absolute value sums in the search range, a vector from the position of the reference block giving the minimum value to the processing target block is set as a motion vector.

FIG. 8 shows an example of the configuration of a detecting device for realizing such a motion vector. In FIG. 8, reference numeral 21 denotes a blocking circuit for decomposing a reference image into blocks.
Is a block circuit for decomposing the quadruple-density image into blocks. The blocking circuit 22 also performs a 1/4 thinning-out process in order to match the vertical density with the reference image.
The error detection circuit 23 calculates the absolute value sum of the difference between the pixel values, and creates a table for storing the absolute value sum for each position of the reference block. The motion vector detection circuit 24 detects the minimum value of the sum of absolute values and outputs a motion vector. FIG.
The lower part of the block diagram schematically shows the spatial relationship between the pixels of the two images and the sum of absolute values of the differences in the respective processes of the error detection circuit 23 and the motion vector detection circuit 24.

[0007]

As described above, a method of thinning out a high-density image and adjusting the density to a low-density image, and thereafter detecting a motion vector by block matching, is based on the pixel accuracy of a low-density image. Can only be obtained.

It is therefore an object of the present invention to provide an image processing apparatus and method capable of detecting a motion vector with high pixel accuracy of a high-density image.

[0009]

In order to solve the above-mentioned problems, the present invention is directed to a method for detecting a motion between one first image and a second image having a density N times as high as that of the first image. The first pixel included in the first image,
Means for determining the absolute value of the difference between the first pixel and the spatially coincident second pixel included in the second image; and shifting the second pixel horizontally and / or vertically. And N
An image processing apparatus comprising: means for repeating a process of obtaining an absolute value by twice the density; and means for obtaining a minimum value among the obtained absolute values.

According to a third aspect of the present invention, in the image processing method for performing motion detection between a single first image and a second image having a density N times that of the first image, the first image is included in the first image. Determining the absolute value of the difference between the first pixel to be detected and a second pixel included in the second image and spatially coincident with the first pixel; And / or a step of repeating a process of obtaining an absolute value for N times the density by shifting in the vertical direction, and a step of obtaining a minimum value of the obtained absolute values. .

In the present invention, an N-fold dense image is treated as a combination of N single-density images having different phases in the horizontal and / or vertical directions. Accordingly, the absolute value of the difference between spatially corresponding pixels between the 1-density image and the N single-density images is calculated. The obtained absolute difference values are those obtained from the 1-density images and have N times the density. Therefore, by detecting the motion vector from the minimum value among the absolute values of the differences, the motion vector can be detected with the pixel accuracy of the N-fold dense image.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an embodiment of an image processing method according to the present invention. As an example, as shown in FIG. 1A, one image (reference image) Ps
The following describes the detection of a motion vector between the image Ps and the image Ph having a density four times as high as the image Ps in the vertical direction. FIG.
In A, the broken line indicates the position of the line, and is actually a line where no pixel exists. 4 times denser image P
h is a total of four single-density images Ph, which are spatially coincident with the reference image Ps and whose line positions coincide with the reference image Ps, and three images whose line positions are shifted by one line at a time.
1, Ph2, Ph3, and Ph4.

That is, in FIG. 1A, Ph1 is obtained by selecting the uppermost line in a set of four consecutive lines of the high-density image Ph. Ph2
Is obtained by selecting the line located second from the top of this set of four lines. Ph3 is obtained by selecting the line located third from the top of this set of four lines. Ph4 is obtained by selecting the lowest line of the set of four lines. By combining these images Ph1 to Ph4, a quadruple-density image is formed.

First, as shown in FIG. 1B, a reference block Bs and a reference block Bs having the same size and the same shape.
h1 (FIG. 1B shows 5 × 5 blocks)
Is set at a position spatially corresponding to one image of the images Ps and Ph1 to Ph4, for example, Ph1. And
As shown in FIG. 2, the absolute value of the difference between the pixel values of the pixels existing at the same position as the reference block Bs is obtained, and the absolute values are totaled over one block to obtain the sum of absolute values. Next, as shown by a broken line in FIG. 2, the reference block Bh1 is moved to various positions in units of pixels of the 1-density image, and sums of absolute values are obtained at the positions after the movement. The obtained absolute value sum is stored in the absolute value sum table.

The moving range is defined as a search range.
For example, the sum of absolute values is calculated for a total of 5 × 3 reference blocks including five reference blocks shifted by one pixel in the horizontal direction and three reference blocks shifted by one pixel in the vertical direction. In that case, as shown in FIG. 3A, 5 × 3 absolute value sum tables T1 are obtained. The center position in the 5 × 3 range is the origin O. The origin O coincides with the spatial center of the reference block Bs and the reference block Bh1. If the position of the reference block giving the minimum value is at the origin in the final absolute value sum table T0 obtained as described later, the motion vector is 0.

Next, an image Ph in which a reference block Bh2 having the same size and the same shape as the reference block Bh1 has been thinned out.
2 are set at the same spatial position. Reference block Bh
1, the reference block Bs and the reference block B
The absolute value sum of the difference between h2 is obtained, and a table T2 of the absolute value sum is obtained. This table T2 is spatially located at a position one line below the four-times denser image than the table T1. Further, for the reference blocks Bh3 and Bh4,
Like the reference blocks Bh1 and Bh2, the reference block Bs
Is performed, and tables T3 and T4 of the calculated absolute value sums are obtained. The table T3 is spatially at a position one line lower than the table T2 four times denser image, and the table T4 is spatially at a position one line lower than the table T3 four times denser image. is there.

Then, as shown in FIG. 3B, the four tables are combined in the reverse relationship to the case where four 1-density images are obtained from the 4-density image, and the final absolute value sum table T0 is obtained.
Create Table T0 shows the distribution of the sum of absolute values of 5 × 3 × 4. The minimum value is detected in the table T0. A vector from the origin O to the position of the reference block giving the minimum value is detected as a motion vector.
In this manner, motion vector detection can be performed with the accuracy of a quadruple density image.

The above-described embodiment is an example in which a motion vector is detected between a reference image and a quadruple-density image in the vertical direction. However, the present invention is applicable not only to the vertical direction but also to the case where a motion vector is detected between the N-fold dense image and the reference image in the horizontal direction, or in the vertical and horizontal directions. Then, a motion vector can be detected with N times higher precision. N is preferably an integer of 2 or more.

FIG. 4 shows a configuration of a motion vector detecting device according to an embodiment of the present invention. In FIG. 4, reference numeral 1 denotes a blocking circuit for decomposing a reference image (image Ps in FIG. 1A) into blocks. 2, N images having the same density as the reference image and spatially the same relationship as the reference image, for example, four images (images Ph1, Ph2, Ph3, and P in FIG.
h4). Phase separation circuit 2
Indicates that the reference image Ps is input, and from the data of the first line to the data of the fourth line of the set of four lines,
Outputs data separated for each line.

The output of the image Ph1 of the phase separation circuit 2 is supplied to a block circuit 3, where it is decomposed into blocks, and
The output of h2 is supplied to the blocking circuit 4 and decomposed into blocks. The output of the image Ph3 is supplied to the blocking circuit 5 and decomposed into blocks. The output of the image Ph1 is supplied to the blocking circuit 6 and Decomposed. The blocking circuits 3 to 6 are blocks having the same shape and the same size as the blocks formed by the blocking circuit 1 (for example, 5 × 5 pixels).
The images Ph1 to Ph4 are respectively subdivided.

The output of the blocking circuit 1 is supplied to the error detection circuits 7, 8, 9, and 10 in common. Error detection circuit 7
As the other input, the output of the blocking circuit 3 is supplied. The error detection circuit 7 determines whether the block Bs of the reference image and the block of the image Ph1 from the blocking
The sum of absolute differences of the pixel values of the pixels at the corresponding positions in the block is totaled. Then, a table T1 for storing the sum of absolute values for each position of the reference block is created.

As the other input of the error detection circuit 8, the output of the blocking circuit 4 is supplied. The error detection circuit 8
Reference Image Block and Image Ph from Blocking Circuit 4
The sum of absolute differences between the pixel values of the pixels at the corresponding positions in the block between the two blocks is counted. Then, a table T2 for storing the sum of absolute values for each position of the reference block
Create Similarly, the error detection circuit 9 creates a table T3 of the sum of absolute values of the differences between the block of the reference image and the block of the image Ph3, and the error detection circuit 10 generates the table T3 between the block of the reference image and the block of the image Ph4. A table T4 of the sum of absolute values of the differences is created.

The tables T1 to T4 created by the error detection circuits 7 to 10 are supplied to the phase synthesis circuit 11. The phase synthesizing circuit 11 synthesizes the sum of absolute values contrary to the phase separation in the phase separating circuit 2, and the phase synthesizing circuit 11 creates a final table T0. The motion vector detection circuit 12 detects a motion vector with reference to the table T0 created by the phase synthesis circuit 11. That is, the motion vector detection circuit 12 detects the minimum value in the sum of absolute values and outputs a motion vector. In the lower part of the block diagram in FIG.
0, the spatial relationship between the pixels of the two images and the sum of absolute values of the differences in each process of the motion vector detection circuit 12 is schematically shown.

The present invention can be applied to the case where the reference image is an N-fold dense image. The present invention is not limited to detecting a motion vector in units of blocks, but also detecting a motion of the entire image. Can also be applied.

[0025]

According to the present invention, unlike the method of detecting a motion vector after thinning out pixels to convert to a one-density image, the present invention provides higher accuracy, that is, pixel accuracy of an N-times denser image. A motion vector can be detected.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating an embodiment of a motion vector detection method according to the present invention.

FIG. 2 is a schematic diagram for explaining an embodiment of a motion vector detection method according to the present invention.

FIG. 3 is a schematic diagram for explaining an embodiment of a motion vector detection method according to the present invention.

FIG. 4 is a block diagram of an embodiment of a motion vector detecting device according to the present invention.

FIG. 5 is a schematic diagram illustrating an example of a motion vector detection method.

FIG. 6 is a schematic diagram for explaining an example of a motion vector detection method.

FIG. 7 is a schematic diagram illustrating an example of a motion vector detection method.

FIG. 8 is a schematic diagram illustrating an example of a motion vector detection device.

[Explanation of symbols]

Ps: Reference image, Ph: 4-fold dense image, T0 to T
4 ... Absolute value sum table, 2 ... Phase separation circuit, 1
1 ... Phase synthesis circuit, 12 ... Motion vector detection circuit

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Takayoshi Fujiwara F-term (reference) 5C059 KK19 KK27 LA06 MA05 NN03 NN21 NN28 SS05 UA06 5L096 GA08 GA19 GA53 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo HA04

Claims (3)

[Claims] An image processing apparatus for performing motion detection between a single first image and a second image having a density N times that of the first image, comprising: a first pixel included in the first image; Means for determining the absolute value of the difference between the first pixel and the spatially coincident second pixel included in the second image; and determining the absolute value of the second pixel in the horizontal and / or vertical directions. An image processing apparatus comprising: means for repeating the process of obtaining the absolute value by N times the density, and means for obtaining the minimum value of the obtained absolute values. 2. The method according to claim 1, wherein the first image and the second image are decomposed into first and second blocks having the same shape, and the absolute value is obtained in block units. The sum of absolute values is obtained by summing up in block units, and the first and second absolute values are calculated.
An image processing apparatus characterized in that one of the blocks is moved in units of pixels to obtain the absolute value sums, and a minimum value is detected from the plurality of obtained absolute value sums. 3. An image processing method for performing motion detection between one first image and a second image having N times the density of the first image, comprising: a first pixel included in the first image; Determining the absolute value of the difference between the first pixel and a spatially consistent second pixel included in the second image; and determining the absolute value of the second pixel in the horizontal and / or vertical directions. An image processing method, comprising: a step of repeating the process of obtaining the absolute value for N times the density by shifting; and obtaining a minimum value of the obtained absolute values.