JPH0846968A - Method and device for detecting hierarchical motion vector - Google Patents

Abstract

PURPOSE:To improve encoding efficiency by detecting a motion vector where an actual motion is reflected even when the plural motions are added in an input picture. CONSTITUTION:A current reduced picture frame 26 stored in a frame memory 12 is read with the value of an address signal 25 consisting of the two coordinates of (x, y) as an address value and inputted to a small area dividing part 37. The small area dividing part 37 reads the kXj number of block data from the present frame reduced picture 26, divides read data into (k/(m1Xm2))X(j/(n1Xn2)) number of blocks and, then, calculates the characteristic quantity of the respective blocks to integrate the blocks by comparing them. An integrated present frame block signal 43, the reading start address (the address corresponding to the extreme left-upper picture element in the block) of outputted block data and an integration block information 55 indicating the mutually integrated blocks are outputted. The small area dividing part 38 executes a similar processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical motion vector detecting method and apparatus.

[0002]

2. Description of the Related Art In the conventional hierarchical motion vector detection method,
When detecting a motion vector, a block with the same size as the original image was used for motion vector search in a reduced image. In this case, one block in the reduced image corresponds to a plurality of blocks in the upper layer image (the image that is the source of the reduced image). For example, one block in an image reduced in size to 1 / m ₁ and 1 / n _{1 in} the horizontal and vertical directions with respect to the upper layer image corresponds to the m _i × n _i block in the upper layer image. In the hierarchical motion vector detection, the motion vector found in the reduced image is m in the horizontal and vertical directions.
_i, is n _i times, used as an initial value in the motion vector detection in the upper-layer image, the upper layer to explore around this initial value. That is, the search result in one block of the image reduced to the size of 1 / m _i and 1 / n _{i in} the horizontal and vertical directions with respect to the upper layer image is the same as the search result of m _i × n _i blocks in the upper layer image. Used as an initial value.

FIG. 3 shows the states of blocks for a reduced image and an original image when the reduction coefficient m ₁ = n ₁ = m ₂ = n ₂ = 2. The blocks A "to P'in the third layer correspond to A'to P'in the second layer and A to P in the original image respectively. In the conventional method, 16 blocks of blocks A" to P "are Matching is performed as one block collectively, and the obtained vector is divided into four blocks A ′ + B ′ +
E '+ F', C '+ D' + G '+ H', I '+ J' +
Using as initial values of M '+ N', K '+ L' + O '+ P', a search in the second hierarchy is performed, and blocks A '+ B'
The search result of + E '+ F' is the block A in the original image,
In B, E, and F, the search results of block C '+ D' + G '+ H' are shown in blocks C, D, G, and H, and block I '+ J'.
Search results of + M '+ N' in blocks I, J, M, N,
The search result of the block K '+ L' + O '+ P' is given to the blocks K, L, O, and P as initial values, respectively.

[0004]

In the hierarchical motion vector detection method, when a block of the same size as the original image is used for motion vector search in a reduced image as conventionally used, even if the image is reduced, Since the block size is fixed, the ratio of the block to the image increases as the reduction ratio increases. As a result, the probability that an object with multiple different motions is included in one block increases, but since only one motion vector is given to one block, it is necessary to detect the correct motion for each object. Is impossible.

As a method for solving this, on each reduced image
For the size of the block that finally obtains the motion vector
And use blocks of size proportional to the image size
A method of performing block matching can be considered. This place
The total block size is a reduction coefficient m for obtaining a reduced image.₁ , M
₂ , ..., m_i Product of₁ And n₁ , N₂ , ・ ・ ・, N _i 
Product N_i Using k / M_i Pixel x j / N_i Line service
It becomes Iz. In the example of FIG. 3, blocks A "to P" match
And the result is the map from A'to P '.
Matching with A'to P'to the initial value during ching
Are the initial values at the time of matching in AP respectively.
It However, with this method, the size is reduced in proportion to the pixel size.
Sample points at the time of matching by using the block
The question is that the probability of false positives increases because the number is reduced.
There was a subject.

It is an object of the present invention to provide a hierarchical motion vector detection method and apparatus which detects a motion vector reflecting an actual motion even when an input image includes a plurality of motions and improves coding efficiency. It is in.

[0007]

A hierarchical motion vector detection method according to the present invention divides an original image into blocks of k pixels × j lines (k and j are arbitrary natural numbers), and the motion vector is calculated for each block. In order to detect the error, the original image V ₀ is reduced to 1 / m using the reduction factors m ₁ and n ₁ in the horizontal and vertical directions, respectively.
_{1, 1 / n 1 (m} 1, n 1 is an arbitrary natural number) to create an image V ₁ which is reduced to the reduced image V ₁ horizontal, using the reduced, respectively in a vertical direction coefficient m _2, n ₂ 1 / M ₂ , 1 /
Image V ₂ reduced to n ₂ (m ₂ and n ₂ are arbitrary natural numbers)
Create and following similar shrink operation coefficients m _c, until n _c c
After repeating (m _c and n _c are arbitrary natural numbers and c is an arbitrary natural number), a motion vector is detected in the reduced image V _C , and the detected motion vector is horizontally and vertically respectively m _c and n _c.
With a vector doubled (m _c and n _c are reduction factors of the reduced image c) as an initial value, motion vector detection is performed around the initial value on the reduced image V _c-1 , and detected on the reduced image V _C-1. An operation of detecting a motion vector around the initial value on the reduced image V _C-2 is performed using the vector obtained by multiplying the generated motion vector by m _c-1 and n _c-1 in the horizontal and vertical directions as an initial value. In the hierarchical motion vector detection method that repeats the motion vector detection on V ₀ to detect the final motion vector, the reduced image V
_{On i} (i is an integer satisfying 1 ≦ i ≦ c), the shape of the block for performing the motion vector detection is adaptively switched to perform the motion vector detection, and the initial stage for detecting the motion vector in the reduced image V _i−1 is set. It is characterized by setting it as a value.

The hierarchical motion vector detecting apparatus of the present invention
Is the original image of the current frame that stores the original image of the current frame
Frame memory and the current frame Original image Frame memo
The original image of the current frame stored in
(K and j are natural numbers) and the current frame block
Signal and block read start address signal
The original image small area dividing unit and the first, second, ..., Cth (c
Is an arbitrary natural number) current frame reduced image frame memo
And the current frame original image stored in the frame memory
The original image of the current frame that is being displayed horizontally and vertically
Reduction factor m₁ , N₁ Using 1 / m₁ , 1 / n₁,
(M₁, N₁ Is an arbitrary natural number)
Create a reduced frame image, and reduce the first current frame reduced image
The first current frame reduction unit stored in the memory and the first current frame reduction unit.
Reduces the current frame reduction image horizontally and vertically
A few meters₂ , N₂ Using 1 / m₂ , 1 / n₂ (M₂, N₂ 
Is the second current frame reduced image reduced to an arbitrary natural number)
Create an image and store it in the second current frame reduced image memory
A second current frame reduction unit that is ...
The reduced image of 1) is reduced by m in the horizontal and vertical directions.
_c, N_cUsing 1 / m_c, 1 / n_c(M_c , N_cIs any
Create a c-th reduced image reduced to a natural number), and
Current frame's reduced image c-th current stored in frame memory
The frame reduction unit and the first current frame reduction image frame
K × j blocks from the reduced images stored in memory
Read the block data and add the block data
(K / m₁) × (j / n₁) After dividing into blocks, each
The feature quantity of the block is calculated and the feature quantity is compared.
The block is integrated with and the integrated current frame
Start reading the block signal and output block data
Integrated block showing addresses and blocks integrated with each other
First current frame small area division unit that outputs lock information
And store it in the second current frame reduced image frame memory
Read k × j block data from the reduced image
Then, the block data is further extracted by {k / (m₁ × m
₂)} × {j / (n₁× n₂)} Block
After that, the feature amount of each block is calculated and the feature amount is compared.
By combining the blocks, the integrated current
Read out ram block signal and output block data
The starting address and the blocks integrated with each other
The second current frame small area for outputting integrated block information
Dividing part, ..., c-th current frame reduced image frame
K × j blocks from the reduced images stored in memory
Read the block data and add the block data
{K / (m₁× m₂ × ・ ・ ・ × m_c)} × {j / (n₁×
n₂ × ・ ・ ・ × n_c)} Blocks, and then
Compute lock feature and compare the features
Block integration with the current frame
Lock signal, start reading the output block data
Integrated block showing dresses and blocks integrated with each other
First current frame small area dividing unit for outputting the check information
And the reference frame original image that stores the reference frame original image
Frame memory and 1st, 2nd, ...
Reference frame reduced image frame memory
And stored in the reference frame original image frame memory
The original image of the reference frame is horizontally and vertically contracted.
Small coefficient m₁, N₁ Using 1 / m₁ , 1 / n₁ (M₁, N
₁ Is the first reference frame reduction reduced to an arbitrary natural number)
Create an image and store it in the first reference frame reduced image memory
First reference frame reducing unit, and first reference frame
The reduced image is scaled down horizontally and vertically by m₂, N₂
 Using 1 / m₂, 1 / n₂ (M₂, N₂ Is any nature
Create a second reference frame reduced image reduced to
Second reference frame Second reference stored in reduced image memory
Frame reduction unit, ..., (c-1) th reduced image
Reduction factor m in the horizontal and vertical directions_c, N_cUsing
1 / m_c, 1 / n_c(M_c, N_cIs an arbitrary natural number)
Created the c-th reduced image and the c-th reference frame reduced image
The c-th reference frame reduction unit stored in the image frame memory
And the vector data of the layer currently being searched is
Current hierarchy vector written according to integrated block information
After finishing searching the memory and all blocks in the current hierarchy,
The vector data of the current hierarchy vector memory is copied.
Vector memory, including the previous layer vector memory,
The integrated block output from the current frame small area division unit 1
According to the check information, the reduction that was in the form of the integrated block
Reading a small image from the first reference image frame memory,
The read signal is used as the reference frame block signal.
At that time, the previous layer vector is set to the read start address.
Stored at the address corresponding to the current block of
M in the horizontal and vertical directions₂, N₂Doubled
Block read start address signal
And a first reference frame small area dividing unit for outputting as
The integrated block output from the current frame small area division unit 2
According to the check information, the reduction that was in the form of the integrated block
Reading a small image from the second reference image frame memory,
The read signal is used as the reference frame block signal.
At that time, the previous layer vector is set to the read start address.
Stored at the address corresponding to the current block of
M in the horizontal and vertical directions ₃, N₃ Doubled
Block read start address signal
A second reference frame small area dividing unit that outputs as
.., output from the c-th current frame small area dividing unit
With the integrated block information, in the form of integrated block
Read the reduced image from the c-th reference image frame memory.
The signal read out and read is used as the reference frame block signal.
Block the read start address at that time.
The c-th reference frame output as the read start address signal
Lame small area division part and the current frame original image
The current frame image stored in the memory is k × j (k,
j is a natural number), and the current frame block signal and
And block offset address signal output reference frame
Original image small area dividing unit and the i-th current frame small area
The current output from the division unit (i = 1, 2, ..., C)
Frame block signal and i-th reference frame small area division
Input the reference frame block signal output from the section,
A matching error calculation unit that calculates the difference value of pixel values, and
It has a small value memory, and the difference from the matching error calculator
Input the value and the error value is stored in the minimum value memory.
Is smaller than the value stored in the minimum value memory,
Update with the difference value, and then from the i-th reference frame small area dividing unit
Output reference frame block read address
The signal is output from the i-th current frame area division unit
Current frame block read start address signal is subtracted
The above address as a vector address
It has a comparator for storing it in memory.

[0009]

According to the present invention, similarly to the conventional hierarchical motion vector detection method, a reduced image is created from the luminance signal of the input image, and the shape of the block used for obtaining the motion vector on each reduced image is adaptively switched. Block matching is performed.

That is, in the hierarchical motion vector search, in order to reduce the possibility of detecting a motion different from the actual one due to the influence of a plurality of different motions in the block, the block in the reduced image is reduced to the same reduction ratio as the image. It is divided into blocks of a size, and the blocks included in the same object are integrated and used for matching in a reduced image.

Block size in detection in reduced image
Is larger, it is more likely to include the boundaries of the object,
The detection accuracy may be reduced, but it is not the same object.
Raba, motion detection is performed collectively as one block.
However, the number of sample points during matching increases, so
The output accuracy increases. Moving image with reduced image i in the conventional method
Block k pixel xj line, which is the unit of cuttle detection, is reduced
Horizontal reduction factor m to obtain a small image₁ , M₂ ・ ・ ・ ・ ・ ・
m_i The product of M_i , Vertical reduction factor n₁ , N ₂・ ・ ・
., N_i The product of N_i Then, M in the original image_i × N_i Individual
Corresponding to each block of k / M in the reduced image i._i 
Pixel x j / N_i It becomes a block of lines. k / M_i Pixel
× j / N_i For the same object with the line block as a unit
Integrates blocks that are considered to be included, each
Motion vector detection is performed using the area as the unit of matching.
U

For example, in FIG. 3 (reduction coefficient m ₁ = n ₁ = m ₂ =
n ₂ = 2), the block A ″ in the third hierarchy
When the integration result in P "is divided into three as shown in Fig. 4,
Matching is performed with blocks A "+ E" + F ", and the detected motion vector is m ₂ and n in the horizontal and vertical directions, respectively.
_The doubled value is used as the initial value in the search in the blocks A ′, E ′, and F ′, B ″ + C ″ + D ″ + G ″ + H ″ + K ″ +
The motion vector detected by L "+ M" + N "is multiplied by m ₂ and n _{2 in the} horizontal and vertical directions, respectively, to obtain a block B ',
To the initial value in the search at C ', D', G ', H', K ', L', M ', N', and I "+ J" + M "+ N"
The motion vector detected in
₂ and n ₂ times the blocks I ', J', M ', N'
Matching in the second layer is performed as an initial value in the search in.

According to this method, even if the input image includes a plurality of different motions, a motion vector reflecting the actual motion is detected.

Next, a block integration determination method will be described with reference to FIG.
The case of (reduction coefficient m ₁ = n ₁ = m ₂ = n ₂ = 2) will be described. First, the feature amount of each block of A "to P" is calculated. As the feature amount, any value such as variance or average of brightness values can be used. A threshold Th is set, it is determined that blocks whose difference between the feature amount of a certain block and the feature amount of another certain block is within Th belong to the same object, and matching is performed as one block. When the feature quantities of the blocks A ″ and B ″ are F _{A ″} and F _{B ″} , respectively, when the formula (1) is satisfied, the blocks A ″ and B ″ are regarded as one block for matching.

| F _{A ″} −F _{B ″} | <Th (1) This determination is performed for all combinations of blocks, and blocks determined to be included in the same object are regarded as one block. Integrate and match. By the integration processing, for example, when the block in the third layer is divided into three as shown in FIG. 4, matching is performed with the block A ″ + E ″ + F ″, and the detected motion vector is set in the horizontal and vertical directions. To m ₂ and n ₂ times the block A ′,
The initial values in the search with E'and F'are B "+ C" + D "+ G" + H "+ K" + L "in the horizontal and vertical directions, respectively.
The motion vectors detected by + O "+ P" are multiplied by m ₂ and n _{2 to} obtain blocks B ', C', D ', G', H ',
The initial value in the search with K ', L', M ', N', and the motion vector detected by I "+ J" + M "+ N" multiplied by m ₂ and n _{2 in the} horizontal and vertical directions, respectively. Matching in the second layer is performed as an initial value in the search in the blocks I ′, J ′, M ′, N ′. Also when performing the matching in the second layer, the integration process is performed by the same operation as in the third layer, the matching is performed in the integrated block, and the initial value of the corresponding block in the original image is set. The block A ′,
When B ', E', F'are integrated into one, in block A ', E', F ', block A "+ E" + F "
In the block B ', the matching result in block B "+ C" + D "+ G" + H "+ K" + L "+ O" +
The matching result in P ″ becomes the initial value, and two initial values are given in one integrated block. This state is the initial value in the blocks A ′, E ′, F ′ and the block B. This can be avoided by taking the average of the initial values in ', or by not allowing integration that spans blocks integrated in images with a high reduction rate.

[0016]

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram of a hierarchical motion vector detecting apparatus according to an embodiment of the present invention.

The image 1 of the current frame is the frame memory 2
Stored in. The reduced image creation unit 5 and the address calculation unit
The image that is composed of the calculation unit and stored in the frame memory 2
1 / m horizontally and vertically from image 4₁ , 1 / n₁ 
(M₁, N₁Is an image reduced to the size of any natural number 6
Is created and stored in the frame memory 7. Reduction is m ₁
× n₁ This is an operation to convert each pixel to 1 pixel,
In the image creating unit 5, first, m corresponding to one pixel of the reduced image
₁× n₁The number of pixels, frame the corresponding address signal 3
It is read by giving it to the memory 2. Create reduced pixels
It is necessary to apply a filter to
If a filter is used, the read m₁× n₁ Pixels
Add values, m₁ × n₁ Divide by and
The result is stored in the frame memory 7 as the reduced image signal 6.
It Note that the filter used is not limited to the average value filter.
It is possible to use various filters such as the median value
Is. The structure and operation of the reduced image creation unit 10 are also reduced.
Similar to the image creation unit 5, the reduced image creation unit 10 addresses
Read out the reduced image 9 with
1 / m horizontally and vertically₂ , 1 / n₂ (M
₂ , N₂ Is a natural number of 2 or more)
The image 11 is stored in the frame memory 12. Reference frame
The image 13 of the frame is the same as the image 1 of the current frame.
After being stored in the memory 14, the reduced image creation unit 17
1 / m₁ , 1 / n₁ Image reduced to the size of
Reduced after being stored in the frame memory 19 as 18
1 / m in image creation unit 22₂ , 1 / n₂ Reduced to the size of
Then, the data is stored in the frame memory 24. Reduced image
In the creating unit 22, the reduced image 2 is generated by the address signal 20.
1 is read and this is further horizontally and vertically respectively.
1 / m₂ , 1 / n₂ Reduced to the size of
And store it in the frame memory 24.

The reduced image 26 of the current frame stored in the frame memory 12 is read as an address value of the address signal 25 consisting of two coordinates (x, y), and is input to the small area dividing unit 37. The small area dividing unit 37 reads k × j block data from the current frame reduced image 26 by the address signal 25, and further reads the read data by {k / (m ₁ × m ₂ )} × {j / (n ₁ Xn ₂ )} blocks, then the feature amount of each block is calculated, and the blocks are integrated by performing the comparison shown in Expression (1), and the integrated current frame block signal 43, output The read start address (address corresponding to the upper leftmost pixel of the block) 49 of the block data and the integrated block information 55 indicating the blocks integrated with each other are output.

FIG. 2 is a block diagram of the small area dividing section 37. First, the image signal at the position corresponding to the upper left of the block signal to be read is read by giving the stored read start address to the frame memory 12 as the address signal 67. Thereafter, while changing the value of the address signal so as to scan all the coordinates in the block to be read, the reading is performed k × j times and stored in the block memory 69. Next, the feature amount calculation unit 72 performs the same operation as the reading from the block memory 69 to convert the k × j block signals 71 stored in the block memory 69 into an address signal composed of two coordinates (x, y). According to the value of 70, {k / (m ₁ × m ₂ )} × {j / (n
₁ × n ₂ )} is divided and read out, and the feature amount 74 is calculated for each read block and stored in the memory 73. The block integration determination unit 76 reads out the feature value 74 from the address signal 75, determines which block is integrated according to the equation (1), and outputs integrated block information 77, 78 (55) indicating the blocks combined with each other. . The block memory 69 uses the integrated block information 77 to read the combined block signal 79 (43) and the read start address 80 (49) of the output block signal.
Is output. The configuration and operation of the small area dividing unit 38 are similar.

Next, the reduced image 36 of the reference frame stored in the frame memory 24 is read with the value of the address signal 35 consisting of two components of (x, y) as an address and input to the small area dividing unit 42. To be done. Small area dividing unit 42
Reads the image signal 36 suitable for the shape of the integrated block from the frame memory 24 according to the integrated block information 57 indicating the integrated state of the block, and at the same time as the reference frame block signal 48, at the same time, the read start address (x, y) is output as the block read start address signal 54. The operation of the small area dividing unit 41 is similar.

The current frame block signal 43 and the reference frame block signal 48 are matched with each other by the matching error calculator 58.
Is input to The matching error calculation unit 58 is composed of a calculation unit, calculates the difference square value of the pixel values output from the two small area division units 37 and 42, and outputs the sum of them to the comparator 60 as the difference value 59. To do. In addition, the matching error calculation unit 59 performs the above calculation with a plurality of reference frame block signals 48 within the search range corresponding to one current frame block signal 43. The difference value calculation is not limited to the difference squared, and it is possible to use the difference absolute value or the like.

The comparator 60 has a minimum memory 61, and if the input difference value 59 is smaller than the value stored in the memory 61, the value in the memory 61 is updated and each of the two components (x, y) is updated. An address 62, which is a value obtained by subtracting the current frame block read start address signal 49 from the reference frame block read start address signal 54, is stored in the vector memory 63.

The vector memory 63 has a previous layer vector memory 64 and a current layer vector memory 65,
The address (motion vector) 62 output from the comparator 60 is transferred to the current hierarchy vector memory 6 according to the integrated block information 55.
Write to an appropriate address of 5. For example, in the case of the integrated state as shown in FIG. 4, the motion vector calculated in the block A ′ + C ′ + D ′ is the blocks A ′, C ′, D ′.
Are written to three addresses corresponding to.

After the above operation is completed within the search range of the motion vector in the reduced image of the reference frame, the motion vector from the current frame to the reference frame on the reduced image is stored in the current hierarchy vector memory 65. After the search is completed in all blocks in the current layer, the current layer vector memory 65 is used as an initial value in the search in the next layer.
Vector data of the above is copied to the previous layer vector memory 64.

Next, the data is stored in the frame memory 7.
The reduced image 28 of the current frame is generated by the address signal 27.
The data is read and input to the small area dividing unit 38. Small area
The dividing unit 38 operates in the same manner as the small area dividing unit 37.
From the reduced image 28 of the current frame, k × j blocks
Read out the block and (k / m₁) × (j /
n₁) Integrated judgment and processing after dividing into blocks
And the integrated current frame block signal 44,
The read start address 50 of the output block signal and
And integrated block information 56 indicating the integrated state. same
Similarly, the reduced image 34 of the reference frame is the address signal 33.
Is read from the frame memory 19 and divided into small areas.
It is input to the section 41, but the reduced image with the smallest size is
Exclude motion vector detection is a reduced image created from that image.
Since the motion vector detected in the image is the initial value,
The read start address to read out
It is necessary to shift by the amount of torr. Address signal 33
And the block read start address signal 53 are in the previous layer.
The address corresponding to the current block of the vector memory 64
Address (motion vector) 66 stored in
M in each direction₂ , N ₂ It ’s supposed to be the sum of doubled values.
It The small area division unit 41 integrates using the address signal 33.
Integrated block shape shown in block information 57
The reference frame reduced image 34 that matches
The read signal is used as the reference frame block signal 47.
Output. At the same time, the block read start address
The signal 53 is output. And similar to the case of reduced image 43
Then, the difference value is calculated by the matching error calculation unit 58.
And the difference value input in the comparator 60 is stored in the memory 61.
If it is smaller than the value stored in, the value in memory 61 is updated.
New reference frame consisting of two components (x, y)
The current frame from the block read start address signal 53
This is the value obtained by subtracting the
Address (motion vector) 62 for the current hierarchy vector memory
Write to the corresponding address of 65.

Next, the original image 30 of the current frame stored in the frame memory 2 is read by the address signal 29 and input to the small area dividing unit 39. Small area division unit 3
In 9, the original image 30 of the current frame is divided into k × j blocks, and the current frame block signal 45 and the block read start address signal 51 are output. Similarly, the original image 32 of the reference frame is input to the small area dividing unit 40, but the address signal 31 and the block read start address signal 52 are the (x, y) output values 66 from the previous layer vector memory 64. Vertically m ₁ , respectively
It is the value obtained by adding the value multiplied by n ₁ . The reference frame image 32 read according to the address signal 31 is input to the small area dividing unit 40, divided into k × j blocks, and output as a reference frame block signal 46.
At the same time, the block read start address signal 52 is output. Then, as in the case of the reduced image, the matching error calculation unit 58 calculates the difference value, and the comparator 60
If the difference value input at is smaller than the value stored in the memory 61, the value in the memory 61 is updated, and the current frame block read start is started from the reference frame block read start address signal 52 each consisting of two components (x, y). An address (motion vector) 62, which is a value obtained by subtracting the address signal 51, is stored in the current hierarchy vector memory 65.

When the above operation for the search range is completed, the detected motion vector is stored in the current hierarchy vector memory 65.

Although the present embodiment is an example in which the number of layers is 3, the current frame reduction unit in the dotted frame in the figure is provided between the output of the frame memory 7 and the input of the reduced image creation unit 10 and the reference frame reduction unit. Output of frame memory 19 and reduced image creation unit 2
It is possible to handle an arbitrary number of hierarchies of 4 or more by connecting (2) (3) to 2 each. Further, in the present embodiment, the image is reduced in the horizontal and vertical directions, but only in the horizontal direction,
Alternatively, the size may be reduced only in the vertical direction. Also,
When the number of layers is 2, the current frame reduction unit in the dotted line frame in the figure is deleted, the output 4 of the frame memory 2 is input to the reduced image creation unit 10, and the reference frame reduction unit in the dotted line frame in the figure is also deleted. Similarly, it can be realized by deleting and outputting the output 16 of the frame memory 14 to the reduced image creating unit 22.

[0030]

As described above, the present invention adaptively switches the shape of the block for detecting a motion vector on the reduced image of the previous layer to detect the motion vector, and detects the motion vector in the reduced image of the current layer. By using the initial value for detection, when using the hierarchical motion vector search in motion compensation interframe coding, it is possible to accurately estimate the motion vector even for images including a plurality of different motions.
This has the effect of enabling motion-compensated inter-frame prediction and improving coding efficiency.

[Brief description of drawings]

FIG. 1 is a block diagram of a hierarchical motion vector detection device according to an embodiment of the present invention.

FIG. 2 is a block diagram of a small area dividing unit 37.

FIG. 3 is a diagram showing an example of a block relationship between layers.

FIG. 4 is a diagram showing an example of block integration.

[Explanation of symbols]

1 image signal of current frame 2 frame memory 3 address signal for reading original image of current frame 4 current frame original image 5 reduced image creation unit 6 reduced image of current frame 7 frame memory 8 address signal for reading reduced image of current frame 9 of current frame Reduced image 10 Reduced image creation unit 11 Reduced image of current frame 12 Frame memory 13 Reference frame image signal 14 Frame memory 15 Reference frame original image read address signal 16 Reference frame original image 17 Reduced image creation unit 18 Reduced image of reference frame 19 frame memory 20 address signal for reading reference frame reduced image 21 reduced image for reference frame 22 reduced image creation unit 23 reduced image for reference frame 24 frame memory 25 address for reading reduced image of current frame Signal 26 reduced image of current frame 27 address signal for reading reduced image of current frame 28 reduced image of current frame 29 address signal for reading current frame image 30 original image of current frame 31 reference frame address signal for reading original image 32 reference frame Original image 33 Reference frame reduced image read address signal 34 Reference frame reduced image read 35 Reference frame reduced image read address signal 36 Reference frame reduced image 37 Small area dividing unit (current frame reduced image 26) 38 Small area dividing unit ( Current frame reduced image 28) 39 Small area dividing unit (current frame original image 30) 40 Small area dividing unit (reference frame original image 32) 41 Small area dividing unit (reference frame reduced image 34) 42 Small area dividing unit (reference frame) Reduced image 36) 3 Block signal of the current frame reduced image 26 44 Block signal of the current frame reduced image 28 45 Block signal of the current frame original image 30 46 Block signal of the reference frame original image 32 47 Block signal of the reference frame reduced image 34 48 Reference frame reduced image 36 block signal 49 current frame reduced image block read start address signal 50 current frame reduced image 28 block read start address signal 51 current frame original image block read start address signal 52 reference frame original image block read start address signal 53 reference frame reduced image Block read start address signal 54 Reference frame reduced image block read start address signal 55 Current frame reduced image integrated block information 56 Current frame reduced image series Combined block information 57 Current frame reduced image integrated block information 58 Matching error calculation unit 59 Difference value 60 Comparator 61 Memory (block difference minimum value) 62 Address data (motion vector) 63 Vector memory 64 Previous layer vector memory 65 Current layer vector memory 66 address data (moving vector search initial value) 67 block signal 68 block read address signal 69 block memory 70 block read address signal 71 block signal 72 feature quantity calculation unit 73 memory 74 block feature quantity 75 feature quantity read signal 76 block Integration determination unit 77, 78 Integrated block information 79 Combined block signal 80 Block read start address signal

Claims (3)

[Claims] 1. An original image V ₀ is divided horizontally and vertically in order to divide the original image into blocks of k pixels × j lines (k and j are arbitrary natural numbers) and detect a motion vector for each of these blocks. Reduction factors m ₁ and
n _{1 1} / m _{1 with, 1 / n 1 (m 1} , n 1 is an arbitrary natural number) to create an image V ₁ which is reduced to, reduced image V ₁
Is reduced to 1 / m ₂ and 1 / n ₂ (m ₂ and n ₂ are arbitrary natural numbers) using the reduction factors m ₂ and n ₂ in the horizontal and vertical directions, respectively, and an image V ₂ is created. The operation is performed c times in total up to the reduction factors m _c and n _c (m _C and n _C are arbitrary natural numbers,
(c is an arbitrary natural number), a motion vector is detected in the reduced image V _C , and the reduced image V _{c is} set as an initial value by multiplying the detected motion vector by m _c and n _{c in the} horizontal and vertical directions, respectively. ₋₁ , a motion vector is detected around the initial value, and a vector obtained by multiplying the motion vector detected on the reduced image V _C-1 in the horizontal and vertical directions by m _c-1 and n _c-1 respectively is initialized. Is a hierarchical motion vector detection method for detecting a final motion vector by repeating a motion vector detection operation around the initial value on the reduced image V _C-2 until the motion vector detection on the original image V _0. In the reduced image V _i (i is an integer satisfying 1 ≦ i ≦ c), the shape of the block for detecting the moving vector is adaptively switched to perform the moving vector detection, and the moving vector in the reduced image V _i−1 is detected. The initial value for detection Hierarchical Tekido vector detecting method according to symptoms. 2. A feature amount of each block is calculated, and a block having a feature amount difference within a certain threshold value is included in the same object and these blocks are integrated into one block.
The hierarchical motion vector detecting method according to claim 1. 3. A current frame for storing an original image of a current frame.
Frame original image frame memory and the current frame original image frame memory
Divide the original image of the current frame into k × j (k and j are natural numbers)
Split, current frame block signal and block read
Outputs the start address signal
The area division part, and the first, second, ..., c-th (c is any natural number of 1 or more)
Number) current frame reduced image frame memory and the current frame original image frame memory
The original image of the current frame is reduced horizontally and vertically.
Small coefficient m₁ , N₁ Using 1 / m₁ , 1 / n₁(M₁ ,
n₁ Is the first current frame contraction reduced to an arbitrary natural number)
Creates a small image and stores it in the first current frame reduced image memory
The first current frame reduction unit to store and the first current frame
The reduced image is reduced in the horizontal and vertical directions by m₂ ,
n₂ Using 1 / m₂ , 1 / n₂ (M₂ , N₂ Is any
Create a second current frame reduced image reduced to a natural number)
The second current frame reduced image memory to store the second
Current frame reduction unit, ..., (c-1) th reduction
Image reduction factor m in both horizontal and vertical directions_c, N_cFor
1 / m_c, 1 / n_c(M_c, N_cIs an arbitrary natural number)
Create a reduced c-th reduced image and reduce the c-th current frame
Reduction of the c-th current frame stored in the small image frame memory
Part and the first current frame reduced image stored in the frame memory
Reading k × j block data from the reduced image
The block data is further (k / m₁) × (j /
n₁) Block, then
Block integration by calculating and comparing the feature quantities
Integrated current frame block signal, output
Start address of read block data and each other
Output the integrated block information indicating the blocks integrated into
The first present frame sub-region dividing unit and the second present frame
Reduced image Reduced image stored in frame memory
Read k × j block data from the
Further data can be added to {k / (m₁ × m₂ )} × {j / (n
₁ × n₂ )} Blocks, and then
Block by calculating the characteristic amount and comparing the characteristic amounts
The integrated current frame block signal
And the read start address of the output block data
And integrated block information indicating blocks integrated with each other
A second current frame sub-region dividing unit that outputs
.. Stored in c-th current frame reduced image frame memory
Read k × j block data from the reduced image
Then, the block data is further extracted by {k / (m₁ × m
₂ × ・ ・ ・ × m_c )} × {j / (n ₁ × n₂ × ・ ・ ・ ×
n_c )} Is divided into blocks, and then the feature amount of each block
Is calculated and the feature quantities are compared to determine the block
The combined current frame block signal, output
Read block start address of block data
Output integrated block information indicating blocks that have been integrated into
The c-th current frame sub-region dividing section and the reference frame original image frame that stores the original image of the reference frame.
Memory, and references of the first, second, ..., Cth (c is an arbitrary natural number)
The frame reduced image frame memory and the reference frame original image frame memory are stored.
The original image of the reference frame is horizontally and vertically contracted.
Small coefficient m₁ , N₁ Using 1 / m₁ , 1 / n₁(M₁, N
₁Is the first reference frame reduction reduced to an arbitrary natural number)
Create an image and store it in the first reference frame reduced image memory
First reference frame reducing unit, and first reference frame
The reduced image is scaled down horizontally and vertically by m₂ , N
₂ Using 1 / m₂, 1 / n₂(M₂ , N₂ Is any nature
Create a second reference frame reduced image reduced to
Second reference frame Second reference stored in reduced image memory
Frame reduction unit, ..., (c-1) th reduced image
Reduction factor m in the horizontal and vertical directions_c, N_cUsing
1 / m_c, 1 / n_c(M_c, N_cIs an arbitrary natural number)
Created the c-th reduced image and the c-th reference frame reduced image
The c-th reference frame reduction unit stored in the image frame memory
And the vector data of the layer currently being searched are integrated as described above.
Current hierarchy vector memo written according to block information
After completing the search for all blocks in the current hierarchy,
Vector data in the current layer vector memory is copied
The vector memory including the previous layer vector memory and the integrated block output from the first current frame small area division unit.
The lock information matched the shape of the integrated block
Read the reduced image from the first reference image frame memory
The read signal as the reference frame block signal.
And the read start address at that time is the previous layer.
To the address corresponding to the current block of vector memory
The stored vector is m in the horizontal and vertical directions.₂ ,
n₂ Adds the doubled value and starts block read
First reference frame small area division output as a response signal
Section and the second current frame small area division section
With the integrated block information, in the form of integrated block
Read the reduced image that fits from the second reference image frame memory
The signal read out and read is used as the reference frame block signal.
Then, at the read start address at that time,
To the address corresponding to the current block of layer vector memory
The stored vector is m in the horizontal and vertical directions.₃,
n₃ Adds the doubled value and starts block read
Second reference frame subregion division output as a response signal
Section, ..., from the c-th current frame sub-area dividing section
The integrated block information can be
A reduced image that was in the shape of a circle
Read from the memory and read the read signal
The read start address at that time
C-th output as a block read start address signal
The reference frame sub-region division part of and the current frame image currently stored in the frame original image memory.
The frame image is divided into k × j (k and j are natural numbers),
Current frame block signal and block offset
Reference frame original image small area dividing section that outputs dress signal
And the i-th current frame small area dividing unit (i = 1, 2, ...
., C) and the current frame block signal output from the
The reference frame output from the reference frame subregion dividing unit of i.
Input the block signal and calculate the difference value of pixel values
It has a matching error calculation unit and a minimum value memory, and a difference from the matching error calculation unit.
Enter the minute value and the error value is stored in the minimum value memory.
If it is smaller than the value stored in
It is updated with the difference value, and is the i-th reference frame small area dividing unit?
Reference frame block read start address output from
From the i-th current frame area division unit
The current frame block read start address signal
The same address is used as the vector address.
Hierarchical motion vector detector with memory-stored comparator
Place.