JP2883034B2 - Correlation degree calculation device and correlation degree calculation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a correlation calculation device, a parallel correlation calculation device, and a correlation calculation method for calculating a correlation between two types of image data, which are effective for detecting a motion vector of image data.

[0002]

2. Description of the Related Art As a method for realizing image compression of a moving image, temporal redundancy is obtained by using information (motion vector) indicating where a certain part in the previous screen has moved on the current screen. There is a method to reduce the nature.

As one method for extracting the motion vector, a current frame image (reference image frame) and a previous frame image (candidate frame) are compared with each other to determine whether the current frame image is similar to a certain block (reference image block). There is a so-called block matching method of extracting a block having a high correlation (that is, having a high correlation) from a previous frame image and detecting a motion vector. This block matching method is widely used for motion compensation prediction in image compression coding.

In the block matching method, the correlation between one reference image block and a plurality of candidate blocks in the previous frame image is calculated, and the candidate block having the highest correlation is selected to detect a motion vector. . Here, in order to obtain a correlation between two image blocks, pixel data of each image block must be calculated. Further, in recent image compression techniques, in order to increase the possibility of selecting a block having a high degree of correlation, there is a tendency that the search area is increased and the number of candidate blocks is increased. Therefore, 2
There is an increasing need for a correlation degree calculating device capable of calculating the degree of correlation between two image blocks at high speed.

[0005] As a conventional correlation degree computing device, there is a device which comprises a plurality of computing units for computing the degree of correlation and operates them in parallel to increase the speed.

For example, the correlation degree computing device disclosed in Japanese Patent Laid-Open No. 6-141304 has the following configuration. When the size of the reference image block is (M × N) pixels and the number of candidate blocks is also (M × N), the reference image block includes a pixel value storage register, a multiplexer, and a difference absolute value calculator (M × N). N) operation units are arranged in an M × N matrix, and output data lines of each operation unit are connected in a pipeline via an adder, and pixel data of the reference image block and the candidate block are arranged in a predetermined order. This is supplied to the absolute difference calculator. This makes it possible to sequentially output the sum of absolute differences, which is an evaluation index of the degree of correlation, every clock cycle, thereby realizing high-speed calculation of the degree of correlation.

[0007]

However, the conventional correlation degree calculating device has the following problems.

According to the above-mentioned correlation degree calculating device, (M ×
Only the correlation degree of (M × N) candidate blocks can be calculated for the reference image block of N) pixels. Therefore, in order to increase the possibility of selecting a candidate block having a high degree of correlation, it has been extremely difficult to increase the number of candidate blocks by expanding the search area.

When an image is compressed in accordance with MPEG, which is an international standard for image compression, in order to realize high image quality with a small code amount, for example, the number of candidate blocks is set to 32 × 16 pixels for a reference image block of 32 × 16 pixels. It is common practice to expand the search area to 32 or 48 × 48. Therefore, even when the search area is expanded, a high-speed processing can be performed, and a correlation degree calculation device having a small circuit area is indispensable.

[0010] In view of the above problems, the present invention provides a correlation degree calculation device that can easily expand a search area, can execute high-speed processing even when the search area is expanded, and can be realized with a simple configuration. The task is to

[0011]

Means for Solving the Problems To solve the above problem, a solution taken by the invention of claim 1 is to apply ((mx) to a reference image block of (M × N) pixels by a block matching method. M) × L) as a correlation calculation device for calculating the correlation with the candidate blocks (where M, N,
L and m are natural numbers, L ≧ N, m ≧ 2), and the calculation of the degree of correlation is performed by pipeline processing for each candidate block group including (M × L) candidate blocks. The required pixel data and the pixel data required for the operation on the other candidate block group processed subsequent to the operation on the one candidate block group are used for pipeline processing in the same clock cycle. , Pipeline processing for each candidate block group can be performed continuously, and the correlation degree of ((m × M) × L) candidate blocks with respect to a reference image block of (M × N) pixels is one clock cycle. It becomes possible to output each time. Therefore, the search area can be easily expanded, and high-speed processing can be performed even when the search area is expanded.

A second aspect of the present invention specifically realizes the first aspect of the present invention, wherein a reference image block of (M × N) pixels is subjected to block matching ((m × N) pixels).
M) × L) as a correlation calculation device for calculating the correlation with the candidate blocks (where M, N, L, and m are natural numbers,
A search area memory for storing pixel data of a search area including (L × N, m ≧ 2) and ((m × M) × L) candidate blocks; and pixel data of a reference image block of (M × N) pixels And the calculation of the degree of correlation between the reference image block and the candidate block is performed by using the pixel data of the stored reference image block and the pixel data of each candidate block output from the search area memory as (M × L). A correlation calculation unit for performing a pipeline process for each candidate block group including the candidate blocks, wherein the search area memory stores pixel data necessary for calculation for one candidate block group and pixel data for the one candidate block group. It has a function of outputting pixel data necessary for the operation to another candidate block group to be processed subsequent to the operation to the correlation operation unit in the same clock cycle. It is. Thereby, the correlation degree calculation unit can continuously execute the calculation for each candidate block group, and the correlation degree of ((m × M) × L) candidate blocks with respect to the reference image block of (M × N) pixels. Can be sequentially output every clock cycle.

According to a third aspect of the present invention, in the correlation calculating device according to the second aspect of the present invention, the search area memory includes:
((M × M) × L) candidate blocks ((m +
The first pixel data and the first pixel data, which are pixel data necessary for the operation on the one candidate block group, from among the pixel data in the search area of (1) × M−1) columns (L + N−1) rows.
And the third pixel data and the third pixel data, which are the pixel data necessary for the operation on the other candidate block group and the second pixel data one row above and M columns right of the pixel data It has a function of outputting the fourth pixel data on the one row and the right of the M column to the correlation calculation unit in the same clock cycle.

According to a fourth aspect of the present invention, in the correlation degree calculating apparatus according to the third aspect of the present invention, the third pixel data is L rows above and M columns right of the first pixel data. And

According to a fifth aspect of the present invention, the invention of the first aspect is specifically realized, and a reference image block of (M × N) pixels is subjected to block matching ((m × N) pixels).
M) × L) as a correlation calculation device for calculating the correlation with the candidate blocks (where M, N, L, and m are natural numbers,
L ≧ N, m ≧ 2), including a main control unit for generating and outputting a memory control signal and an operation control signal based on a given clock, and ((m × M) × L) candidate blocks A search area memory that stores pixel data of the search area and outputs the stored pixel data according to an instruction of the memory control signal output from the main control unit; (M × N)
The pixel data of the reference image block of pixels is held, and the calculation of the degree of correlation between the reference image block and the candidate block is performed using the pixel data of the held reference image block and the pixel data output from the search area memory (M × L) a correlation calculation unit that performs pipeline processing for each candidate block group including candidate blocks, and a calculation control unit that controls the correlation calculation unit in accordance with an instruction of a calculation control signal output from the main control unit Wherein the search area memory includes pixel data required for an operation on one candidate block group and pixel data required for an operation on another candidate block group processed subsequent to the operation on the one candidate block group. In the same clock cycle at the same time. Accordingly, the correlation degree calculation unit can continuously execute the calculation for each candidate block group, and performs the correlation of ((m × M) × L) candidate blocks with respect to the reference image block of (M × N) pixels. Can be sequentially output every clock cycle.

According to a sixth aspect of the present invention, in the correlation degree calculation device according to the fifth aspect of the present invention, the search area memory comprises:
((M × M) × L) candidate blocks ((m +
The first pixel data and the first pixel data, which are pixel data necessary for the operation on the one candidate block group, from among the pixel data in the search area of (1) × M−1) columns (L + N−1) rows.
And the third pixel data and the third pixel data, which are the pixel data necessary for the operation on the other candidate block group and the second pixel data one row above and M columns right of the pixel data It has a function of simultaneously outputting the fourth pixel data on the one row and the right of the M column to the correlation calculation unit in the same clock cycle.

According to a seventh aspect of the present invention, in the correlation calculation device according to the sixth aspect of the present invention, the third pixel data is located L rows above and M columns right of the first pixel data. I do.

According to an eighth aspect of the present invention, in the correlation degree calculation device according to the fifth aspect of the present invention, the search area memory has a function of outputting four pixel data per one clock cycle.

According to a ninth aspect of the present invention, in the correlation degree calculating device according to the eighth aspect, the search area memory includes ((m ×
(M) × L) candidate blocks ((m + 1) × M
-1) The search area of column (L + N-1) row is (M × (L + N)
-1)) Pixel data of (m + 1) partial search areas obtained by dividing each pixel are stored, and pixel data is stored in accordance with an access instruction by a memory control signal output from the main control unit. (M + 1) storage element units to be read, and timing adjustment means for adjusting the timing and outputting the pixel data read from the (m + 1) storage element units to the correlation calculation unit, The main control unit instructs the search area memory by the memory control signal to perform two accesses in one clock cycle, and the timing adjustment unit performs one time operation from the (m + 1) storage element units. Selects two of the pixel data read by the access, outputs four pixel data per clock cycle after adjusting the timing And shall.

According to the ninth aspect of the present invention, the address can be shared in the (m + 1) storage element sections, and the pixel data required by the correlation operation section can be easily output. Further, it is possible to configure the (m + 1) storage element units with one memory.

According to a tenth aspect of the present invention, in the correlation degree calculating device of the ninth aspect, the timing adjusting means comprises the (m +
1) a first memory selector for selecting and outputting one of pixel data read out from the first to m-th storage element units among the (m) storage element units; A second memory selector for selecting and outputting one of the pixel data read from the second to (m + 1) th storage element units in the storage element unit, and a first memory selector And delay means for delaying the pixel data selected and output by the second memory selector with respect to the selectively output pixel data by M clock cycles.

According to the tenth aspect of the present invention, since the timing adjusting means is simply constituted by the two memory selectors and the delay means, the search area memory can be realized by a circuit having a small area.

According to an eleventh aspect of the present invention, in the correlation degree arithmetic unit according to the eighth aspect, the correlation degree calculating section selectively outputs different one pixel data among the four pixel data output from the search area memory. Two first selectors,
Each of the pixel data of the reference image block is held, and one of the two pixel data selected and output by the two first selectors is selected, and an evaluation value with the held pixel data is calculated. A reference image block of (M × N) pixels and each candidate are obtained by accumulating the evaluation values calculated by each of the pixel operation units, comprising N line operation units each having M pixel operation units. It is assumed that the degree of correlation with the block is determined.

According to the eleventh aspect of the present invention, even when the search area is expanded, the pixel data is supplied to the pixel operation unit every clock cycle, so that the processing can be speeded up.
In addition, since the correlation calculation unit is configured only by arranging the line calculation units having the same configuration, the configuration of the correlation calculation device is simplified.

According to a twelfth aspect of the present invention, in the correlation degree arithmetic device of the eleventh aspect, each of the pixel operation units includes a reference image storage unit for holding pixel data of an input reference image block, and the two first image storage units. A second selector for selecting one of the two pixel data selected and output by the selector, a pixel data stored in the reference image storage unit, and a pixel data selected by the second selector And an adder for adding the evaluation value calculated by the calculator and the accumulated evaluation value output from the pixel operation unit at the preceding stage, and an output data of the adder. And a register for temporarily storing the data and outputting it to the pixel operation unit at the next stage.

According to a thirteenth aspect of the present invention, in each of the line arithmetic units in the correlation degree arithmetic unit according to the eleventh aspect, the M pixel arithmetic units transfer the pixel data selected and output by the first selector. Are symmetrically arranged in two rows on both sides.

According to a fourteenth aspect of the present invention, in the correlation degree arithmetic device according to the thirteenth aspect, the evaluation value is located at an end of the pixel operation section of one column where the first selector is arranged. To the pixel operation unit located at the end where the first selector is not disposed, and subsequently to the pixel operation unit located at the other end where the first selector is not disposed. Next to the pixel operation unit, in the pixel operation units in the other columns, the first selector is arranged from the pixel operation unit located at the end where the first selector is not arranged. It is assumed that the data is sequentially transferred and accumulated to the pixel operation unit located at one end.

According to the thirteenth and fourteenth aspects of the present invention, the lengths of the lines connecting the M pixel operation units in each line operation unit and the lines between the line operation units are reduced, and the configuration of the correlation degree operation unit is simplified. Be transformed into

According to a fifteenth aspect of the present invention, in the correlation degree calculating device according to the eighth aspect, the correlation degree calculating section holds the pixel data of the reference image block, and stores the four pixel data output from the search area memory. Is selected, and (M × N) pixel operation units for calculating an evaluation value with the held pixel data are provided, and the evaluation values calculated by each pixel operation unit are accumulated. , (M × N) pixels and the degree of correlation between each of the candidate blocks.

According to the fifteenth aspect, even when the search area is expanded, the pixel data is supplied to the pixel operation unit every clock cycle, so that the processing can be speeded up.
In addition, since the correlation calculation unit is configured only by arranging the line calculation units having the same configuration, the configuration of the correlation calculation device is simplified.

According to a sixteenth aspect of the present invention, in the correlation degree calculating device according to the fifteenth aspect, the pixel operation section comprises: a reference image storage section for holding input pixel data of the reference image block;
A selector for selecting one of the four pixel data output from the search area memory, and an evaluation value between the pixel data held in the reference image storage unit and the pixel data selected by the selector. An arithmetic unit for performing arithmetic operation, an adder for adding the evaluation value calculated by the arithmetic unit and the accumulated evaluation value output from the preceding pixel operation unit, and temporarily storing output data of the adder, And a register for outputting to the pixel operation unit of the stage.

According to a seventeenth aspect of the present invention, in the correlation degree calculation device according to the eighth aspect, the correlation degree calculation unit holds M pixel data of each row of the reference image block and outputs the data from the search area memory. N operation units for calculating the evaluation values of the pixel data and the held pixel data, and the operation control unit includes N line control units for controlling the N line operation units, respectively. Shall be.

According to the seventeenth aspect of the present invention, the correlation calculation unit and the calculation control unit are configured simply by arranging the line calculation unit and the line control unit in combination, thereby simplifying the configuration of the correlation calculation unit. Is done.

According to an eighteenth aspect of the present invention, in the correlation degree arithmetic device according to the seventeenth aspect, the N line control units are cascade-connected, and the N number of operation control signals input to the arithmetic control unit are Are configured to be sequentially transferred.

According to a nineteenth aspect of the present invention, in the correlation degree arithmetic device according to the eighteenth aspect, the line arithmetic units each include:
Two first selectors for selecting and outputting different ones of pixel data among the four pieces of pixel data output from the search area memory, and pixel data of a reference image block, respectively. M pixel calculation units for selecting one of the two pixel data selected and output by the first selectors and calculating an evaluation value with the held pixel data, Each of the units includes a first control unit for controlling the two first selectors, and M second control units for controlling the M pixel operation units, respectively.

According to a twentieth aspect of the present invention, in the correlation degree calculating device according to the nineteenth aspect, each of the line controllers has the M
The second control units are cascaded, and the arithmetic control signal input to the line control unit is the M second control units.
Are sequentially transferred and output M clock cycles after being input.

According to a twenty-first aspect of the present invention, there is provided a correlation calculating device for calculating a correlation between a reference image block of (M × N) pixels and a plurality of candidate blocks by a block matching method. L is a natural number), a search area memory that stores pixel data of a search area including each candidate block to be operated, and pixel data of a reference image block of (M × N) pixels. A correlation degree calculation unit that calculates the degree of correlation with each candidate block by pipeline processing using the pixel data of the held reference image block and the pixel data of each candidate block output from the search area memory; And a calculation control unit that controls the correlation calculation unit in accordance with the calculation control signal, wherein the correlation calculation unit holds pixel data of the reference image block, respectively. And N line operation units each having M pixel operation units for calculating an evaluation value of the held pixel data and the pixel data output from the search area memory, and these (M × N) The pixel operation unit is
Evaluation of the operation result of each pixel operation unit
It is configured so that the value is sequentially added and transferred.
The arithmetic control unit is configured to include N line control units for controlling the N line arithmetic units, respectively.

According to a twenty-second aspect of the present invention, in the correlation degree arithmetic device according to the twenty-first aspect, the N line control units are cascaded, and the N number of operation control signals input to the arithmetic control unit are Are configured to be sequentially transferred.

According to a twenty-third aspect of the present invention, in the correlation degree calculating device according to the twenty-second aspect, the line control units each include:
It is assumed that M control units for controlling the M pixel operation units are provided.

According to a twenty-fourth aspect of the present invention, in each of the line control units in the correlation degree calculation device according to the twenty-third aspect, the M control units are cascaded, and the arithmetic control signal input to the line control unit is It is assumed that the M control units are sequentially transferred and output M clock cycles after being input.

[0041]

[0042]

[0043]

A solution taken by the twenty-fifth aspect of the present invention is that a reference image block of (M × N) pixels is correlated with ((m × M) × L) candidate blocks by a block matching method. As a correlation degree calculation method for calculating the degree (where M, N, L, and m are natural numbers, L ≧ N, m ≧ 2),
The ((m × M) × L) candidate blocks are divided into m candidate block groups including (M × L) candidate blocks, and each candidate block group is successively folded in the horizontal direction. A first step of performing block matching with the reference image block for the M candidate blocks and a second step of repeating the first step L times while moving in the vertical direction are performed.

According to a twenty-sixth aspect of the present invention, in the correlation degree calculating method according to the twenty-fifth aspect, pixel data required for an operation on one candidate block group and another pixel data processed following the operation on the one candidate block group It is assumed that pixel data necessary for the operation on the candidate block group is used at the same time.

[0046] In addition, solutions of the invention is taken of claim 27, the correlation between the (M × N) with respect to the reference image block of pixels, by block matching method ((m × M) × L ) number of candidate blocks As a correlation calculation device for calculating the degree (where M, N, L, and m are natural numbers, L ≧ N, m ≧ 2), the calculation of the degree of correlation is made up of (M × L) candidate blocks, In addition, the processing is performed by pipeline processing for each of the m candidate block groups connected in one of the horizontal direction and the vertical direction, and the calculation of the degree of correlation in each candidate block group is continuously folded in the one direction. The step of performing block matching with the reference image block for the M candidate blocks arranged in a row is repeated L times while moving in the other direction. Those configured for use in pipeline processing in the same clock cycle and pixel data required for operations on other candidate blocks that are processed subsequent to operation on the pixel data and the one candidate block group.

A solution taken by the invention of claim 28 is that a reference image block of (M × N) pixels is correlated with ((m × M) × L) candidate blocks by a block matching method. As a correlation degree calculation method for calculating the degree (where M, N, L, and m are natural numbers, L ≧ N, m ≧ 2),
Each of ((m × M) × L) candidate blocks is represented by (M
× L) divided into m candidate block groups consisting of candidate blocks and connected in one of the horizontal direction and the vertical direction, and for each candidate block group, successively in the one direction. A first step of performing block matching with a reference image block for the M candidate blocks arranged in a folded manner, and a second step of repeating the first step L times while moving in the other direction are performed. It is assumed that pixel data required for an operation on one candidate block group and pixel data required for an operation on another candidate block group processed subsequent to the operation on the one candidate block group are simultaneously used.

[0048]

BEST MODE FOR CARRYING OUT THE INVENTION

First Embodiment A correlation calculation device according to a first embodiment of the present invention will be described with reference to the drawings.

In the present embodiment, when a motion vector is obtained by full search using the block matching method, the correlation of each of ((m × M) × L) candidate blocks with respect to a reference image block of (M × N) pixels is determined. The degree is required. In this case, the size of the search area is ((m + 1) × M−1) × (L
+ N-1) pixels. Here, M, N, L, and m are arbitrary natural numbers (where L ≧ N, m ≧ 2).

Hereinafter, for explanation, M = 4, N = 3, L =
3, m = 2, that is, the block size of the reference image block is 4 × 3 pixels, and the number of candidate blocks is (2 × 4) × 3. It is also assumed that the sum of absolute differences of pixel data between the reference image block and the candidate block is obtained as an evaluation index of the degree of correlation.

FIG. 1 is a diagram for explaining a block matching method and a motion vector. In FIG. 1, Fp
Is a reference image frame (current screen), Bp is a reference image block, Fb is a candidate frame (previous screen), Bb is a candidate block, E is a search area, and MV is a motion vector. Further, r represents each pixel of the reference image block Bp, and C represents each pixel of the search area E.

The motion vector is information indicating where a certain part in the previous screen has moved on the current screen. In the block matching method, the reference image block B
A candidate block Bb having the highest degree of correlation with p is detected from within the search area E, and a vector indicating the positional shift is set as a motion vector. In the present embodiment, a candidate block Bb in which the sum of absolute differences, which is an evaluation index of the degree of correlation, is minimized is determined, and a motion vector MV is detected.

The outline of the calculation operation and configuration of the correlation degree calculation device according to the present embodiment is as follows.

Each operation unit for calculating the evaluation index of the degree of correlation is:
First, the data of each pixel of the reference image block is held, then the pixel data of each candidate block is received every clock cycle, and the pixel data of the held reference image block and the pixel data of the input candidate block are compared. After calculating the absolute difference value, the sum is added to the absolute difference value output from the preceding operation unit, and the addition result is output to the next operation unit. That is, each arithmetic unit has a so-called cascade connection configuration.

Further, the correlation degree calculation device according to the present embodiment includes a search order for candidate blocks, data supply means for each calculation unit for realizing the search order, means for controlling the data supply means, and The arrangement of each means is characterized.

FIG. 2A is a diagram showing the range of each candidate block Bb in the search area E. For example, the candidate block Bb0,0 corresponds to the upper left corner of the search area E, and the pixels C0,0 to
C3,0, C0,1 to C3,1 and C0,2 to C3,2.
The candidate block Bb7,2 corresponds to the lower right corner of the search area E, and the pixels C7,2 to Ca, 2, C7,3 to Ca, 3 and C7,4
~ Ca, 4. A search area E of 11 × 5 pixels is constituted by 8 × 3 candidate blocks Bb.

FIGS. 2B and 2C are diagrams showing the search order of the candidate block Bb in the present embodiment. FIG.
In (b) and (c), the pixel at the upper left corner of each candidate block Bb is set as a representative point of the candidate block Bb, and the search order of the candidate block Bb is indicated by an arrow indicating the movement of the representative point.

As shown in FIG. 2B, first, starting from the upper left candidate block Bb0,0 in the search area E, 4 (= M) candidate blocks Bb0,0, Bb1,0,
Bb2,0 and Bb3,0 are sequentially processed (step 1). Next, four candidate blocks are processed in the horizontal direction from the candidate block Bb0,1 one step down in the vertical direction. By repeating the same process 3 (= L) times (step 2),
12 (= M × L) candidate blocks Bb0,0 to Bb3,2
Process.

Next, as shown in FIG. 2C, starting from the candidate block Bb4,0, 4 (= M) candidate blocks Bb4,0, Bb5,0, Bb6,0, Bb7, in the horizontal direction. 0 is processed in order, and then the operation of moving down one step in the vertical direction and processing four candidate blocks in the horizontal direction from the candidate block Bb4,1 is repeated 3 (= L) times. Thereby, 12 (= M × L) candidate blocks Bb4,0 to Bb
Process 7,2.

When pipeline processing is performed using a conventional correlation degree calculation device, the processing shown in FIG.
The processing shown in (c) cannot be performed continuously,
Therefore, there is a period in which the correlation degree is not calculated for several cycles (12 cycles in the candidate block shown in FIG. 2).

However, according to the correlation degree calculating device according to the present embodiment, (m × M) × L (= (4 × 2) × 3) candidate blocks can be continuously processed. In the present embodiment, the description is simplified assuming that N = L = 3, but there is no problem if L is a natural number equal to or greater than N.

FIG. 3 is a block diagram showing the configuration of the correlation degree calculating device according to the present embodiment. In FIG. 3, 10 is a search area memory, 20 is a main control unit, 30 is a correlation operation unit, and 40 is an operation control unit.

The search area memory 10 includes a storage element 11a
11c, a first memory selector 12a, a second memory selector 12b, and a delay unit 13.
The pixel data of the search area is stored in the storage element units 11a to 11c, and the search area memory 10 transfers the four pixel data in the search area to the bus A1 in accordance with the memory control signal 24 output from the main control unit 20. , A2, B1, B2
Is output to the correlation degree calculation unit 30 via. Delay means 13
Are the delay circuits 13a to 13d and the M pixel delay circuit 13
e, 13f. The first memory selector 12a, the second memory selector 12b, and the delay unit 13 constitute a timing adjustment unit.

The main control unit 20 includes a clock generation circuit 2
5, a counter 26, and an address analyzer 27.
4 and outputs the operation control signal 23 to the operation control unit 40.

The clock generation circuit 25 generates a clock 24 a having the same phase as the clock 21 (odd clock), a clock 24 b having an opposite phase to the clock 21 (even clock), and twice the clock 21 based on the input clock 21. A double clock 24 c having a frequency is generated and output to the search area memory 10. The counter 26 starts counting in synchronization with the double clock 24c in accordance with the instruction of the start signal 22, and outputs count data to the address analyzer 27. The address analyzer 27 is a decoder that generates the address 24d and the operation control signal 23 based on the count data of the counter 26, outputs the operation control signal 23 to the operation control unit 40, and outputs the address 24d to the search area memory 10. I do.
The memory control signal 24 includes an odd clock 24a, an even clock 24b, a double clock 24c, and an address 24d.

The double clock 24c and the address 24d output from the main control unit 20 are given to the storage element units 11a to 11c. According to the given address 24d, the double clock 2 is output from each of the storage element units 11a to 11c.
Pixel data is read out every cycle 4c. Also,
The first and second memory selectors 12a and 12b are switched by an odd clock 24a. When the double clock 24c has an odd cycle, the first memory selector 12
a selects the pixel data read from the storage element unit 11a and outputs the selected pixel data to the delay circuit 13a;
2b selects the pixel data read from the storage element section 11b and outputs it to the delay circuit 13c. On the other hand, when the double clock 24c has an even cycle, the first memory selector 1
2a selects the pixel data read from the storage element section 11b and outputs it to the delay circuit 13b, and the second memory selector 12b selects the pixel data read from the storage element section 11c and outputs the delay circuit 13d Output to With such an operation, the storage element unit 11
Four pixel data can be read from a to 11c.

The delay circuits 13a and 13c provide the odd clock 2
4a, pixel data is input, and delay circuits 13b,
13d receives pixel data in response to an even clock 24b. The delay circuits 13a and 13b output the input pixel data to the buses A1 and A2, respectively. Further, the delay circuits 13c and 13d convert the input pixel data into M
Output to the pixel delay circuits 13e and 13f. The M pixel delay circuits 13e and 13f convert the input pixel data into M (=
4) After delaying by the clock, output to buses B1 and B2, respectively.

The correlation degree calculating section 30 receives the given (M ×
N) For the reference image block 51 of the pixel, the degree of correlation between each candidate block and the candidate block is calculated by pipeline processing using the pixel data input through the buses A1, A2, B1, and B2, and the correlation degree data 31m is calculated. Output.

In accordance with the instruction of the arithmetic control signal 23 output from the main control unit 20, the arithmetic control unit 40 stores the storage control signals 41a to 411 and the first switching signals 42a to 42l.
f and the second switching signals 43 a to 431 are output to the correlation degree calculation unit 30.

The correlation degree calculation section 30 and the calculation control section 40 will be described in more detail with reference to FIGS.

FIG. 4 is a block diagram showing the internal configuration of the correlation degree calculating section 30. As shown in FIG. 4, the correlation degree operation unit 30 is configured by 3 (= N) line operation units each having 4 (= M) pixel operation units and two first selectors. I have. The internal configurations of the line operation units 32-34 are the same, and the pixel operation units 32a-32d, 33
The internal configurations of a to 33d and 34a to 34d are the same.

The line operation unit 32 includes pixel operation units 32a to 32a
32d and first selectors 32e and 32f. The first selector 32e selects one of the two pixel data transferred on the buses A1 and A2, and
A, and the first selector 32f selects one of the two pixel data transferred on the buses B1 and B2, and
Output to 2B. The pixel calculation units 32a to 32d accumulate pixel data of the reference image block 51, respectively, read pixel data of the candidate block transferred through the bus 32A or 32B, and calculate an absolute difference between the pixel data and the pixel data of the reference image block. I do.

The pixel operation unit 32a adds the operation result to the initial value 31a (“0” in FIG. 4) of the correlation data, and outputs the addition result data 31b to the next-stage pixel operation unit 32b. The pixel operation unit 32b adds the operation result to the data 31b and outputs addition result data 31c. Similarly, the pixel operation unit 32c adds the operation result to the data 31c and outputs addition result data 31d. 32d is data 31
The calculation result is added to d, and the addition result data 31e is output. The data 31e is stored in the pixel operation unit 3 of the line operation unit 33.
3a.

The pixel operation units 33a to 33-3 of the line operation unit 33
3d, the pixel operation units 34a to 34d of the line operation unit 34 also perform the same operation, and the pixel operation unit 34d outputs correlation degree data 31m.

FIG. 5 is a circuit diagram showing the internal configuration of the arithmetic and control unit 40. As shown in FIG. 5, the arithmetic control unit 40
It is composed of 3 (= N) line control units each having 1st control unit and 4 (= M) second control units. Since the line controllers 44 to 46 have the same configuration, the configuration of the line controllers 45 and 46 is omitted in FIG.

The line control unit 44 comprises a first control unit 44e and second control units 44a to 44d. The second control units 44a to 44d are respectively provided in the line operation unit 32 shown in FIG. The pixel operation units 32a to 32d are controlled. When the operation control signal 23 is input from the main control unit 20, the first control unit 44e becomes the second control unit 44a
To 44d, and the second control units 44a to 44d output the storage control signals 41a to 41d and the second switching signals 43a to 43d to the pixel operation units 32a to 32d, respectively. In addition, the first control unit 44e controls the line operation unit 3
2 output first switching signals 42a and 42b for switching the first selectors 32e and 32f.

Further, the first control unit 44e determines that M (=
4) After the clock, output the operation control signal 44j to the line control unit 45 of the next stage. The first control unit of the line control unit 45 performs the same control, and outputs an operation control signal 45j to the next-stage line control unit 46 after M (= 4) clocks.

Here, the control of the correlation calculation section 30 by the calculation control section 40 will be described in more detail.

First, control of the first selector by the first control unit will be described. Here, the first control unit 44e
The control of the first selectors 32e and 32f according to the above will be described as an example.

As shown in FIG. 5, when the operation control signal 23 is input to the line controller 44, the first register 44f
Outputs the storage permission signal 44i as the first switching signal 42a. The first switching signal 42a is supplied to the first selector 32e as shown in FIG.
2e selects either bus A1 or bus A2,
The data of the selected bus is output to the bus 32A.

As shown in FIG. 5, four (= M) clocks after the output of the first switching signal 42a,
The second register 44g outputs the first switching signal 42b synchronized by the output signal 44o of the second control unit 44d. The first switching signal 42b is supplied to a first selector 32f as shown in FIG.
f selects one of the buses B1 and B2 and outputs the data of the selected bus to the bus 32B.

As shown in FIG. 5, the second control unit 4
When an output signal 44o is output from 4d, the OR gate 44
h, the initialization signal 44k is output again, and thereafter, the same operation is repeated.

Next, control of the pixel operation unit by the second control unit will be described. Here, the control of the pixel operation unit 32b by the second control unit 44b will be described as an example.

When the operation control signal 23 is input to the operation control unit 40, the control of storing the pixel data of the reference image block and the distribution of the pixel data of the candidate block is started.
In FIG. 5, when the operation control signal 23 is input to the line control unit 44, the first register 44f
The storage permission signal 44i is output to 4a to 44d. Also,
The operation control signal 23 is also input to the OR gate 44h,
The R gate 44h outputs an initialization signal 44k to the second control units 44a to 44d. The second control units 44a to 44d are respectively initialized by the initialization signal 44k.

FIG. 6 is a circuit diagram showing the internal configuration of the second control unit 44b. As shown in FIG. 6, the second control unit 4
4b includes an AND gate 44p, a switching circuit 44q, and a propagation circuit 44r. FIG. 7 is a circuit diagram showing the internal configuration of the second pixel operation unit 32b. FIG.
As shown in the figure, the image calculation unit 32b is connected to the reference image storage unit 8
It comprises a first and second selector 82, a difference absolute value calculator 83, a difference absolute value adder 84 and a pipeline register 85.

As shown in FIG. 6, one cycle after the initialization signal 44k is input, the second control section 44b transmits the output signal 441 of the preceding second control section 44a to the propagation circuit 44r.
At the timing of the odd clock 24a. A
The ND gate 44p receives the output signal of the propagation circuit 44r and the storage permission signal 44i as inputs, and outputs a storage control signal 41b.

The storage control signal 41b is provided to a reference image storage section 81 in the pixel operation section 32b as shown in FIG.

Further, one cycle after the initialization signal 44k is inputted, the second control unit 44b synchronizes the switching circuit 44q with the output signal 441 of the preceding second control unit 44a, and the switching circuit 44q Second switching signal 43b
Is output.

As shown in FIG. 7, the second switching signal 43b is provided to a second selector 82 in the pixel operation section 32b, and the second selector 82 outputs the second switching signal 43b.
Switch the bus to be selected according to.

The output signal 44m of the propagation circuit 44r is output to the next-stage second control unit 44c. Second control unit 44
After c operates similarly, it outputs a signal 44n to the second control unit 44d in the next stage. Thus, the second control unit 4
The operations of 4a to 44d propagate every cycle. The second control units in the line control units 45 and 46 perform the same operation.

The pixel operation unit 32b operates as follows.
As shown in FIG. 7, the absolute difference calculator 83 calculates the absolute difference between the pixel data of the candidate block selected by the second selector 82 and the pixel data of the reference image block stored in the reference image storage unit 81. Calculate the value (AE). The difference absolute value adder 84 adds the difference absolute value 86 output from the difference absolute value calculator 83 and the difference absolute value sum (SAE) 31b provided from the preceding pixel calculation unit 32a. The sum of absolute differences 31c obtained as a result of the addition is output to the next-stage pixel calculator 32c via the pipeline register 85 in the next clock cycle. This operation is
By repeating the clock cycle, correlation data 31m between the reference image block and each candidate block can be obtained.

Hereinafter, the operation of the correlation degree calculating device according to the present embodiment will be specifically described. Here, it is assumed that the degree of correlation between the reference image block Bp and the candidate block Bb as shown in FIGS. 1 and 2 is calculated.

First, pixel data of the search area E is stored in the storage elements 11a to 11c in the search area memory 10. FIG. 8 is a diagram illustrating pixel data stored in the storage element units 11a to 11c. In FIG. 8, the pixel data of the partial search area E1 is stored in the storage element 11a.
The pixel data of the partial search area E2 is stored in the storage element section 11b, and the pixel data of the partial search area E3 is stored in the storage element section 1b.
1c. At this time, the storage element units 11a to 11
In order to make the address for c common, the storage element unit 1
1a contains pixel data of the partial search area E1 as C0,0, C1,
0, C2,0, C3,0, C0,1,..., C2,4, C3,4
, And the storage element section 11b stores the pixel data of the partial search area E2 in C4,0, C5,0, C6,0, C7,
0, C4,1,..., C6,4, C7,4 in that order, and the partial search area E3 in the storage element section 11c.
Are stored in the order of C8,0, C9,0, Ca, 0, Cb, 0 (dummy data), C8,1,..., Ca, 4, Cb, 4 (dummy data).

FIG. 9 is a diagram showing pipeline processing realized by the correlation degree calculating device according to the present embodiment.
In each cycle, each pixel operation unit (PE) 32a to 34
The pixel data C in the search area E required by d is shown. The subscript of the pixel data C corresponds to the subscript of the pixel data C in the search area E shown in FIGS. In FIG. 9, the processing on the left side of the step-shaped thick line corresponds to the processing shown in FIG. 2B, and the processing on the right side of the step-shaped thick line in FIG. 2C.
Corresponds to the processing shown in FIG.

Here, for example, at time 18, the pixel data requested by each pixel operation unit (the portion surrounded by a thick broken line in FIG. 9) is C4,1, C8,0, C4,3, C0,4. There are four of them, which are indicated by circles in FIG. In the conventional correlation degree calculation device, four pixel data could not be simultaneously supplied to each pixel calculation unit as in the case of time 18, so that the processing on the right side and the processing on the left side of the step-shaped thick line were performed. There was a period of several cycles during which pipeline processing could not be performed continuously and the degree of correlation was not calculated.

The correlation calculation device according to the present embodiment implements means for supplying data to each pixel calculation unit so that pipeline processing can be performed without interruption.

FIG. 10 is a diagram for explaining the operation of the correlation degree calculating device according to the present embodiment. In the figure, (a) is a diagram for explaining the operation of the main control unit 20, (b) is a diagram showing pixel data read from each of the storage element units 11a to 11c, and (c) is a diagram of the search area memory 10. FIG. 4D is a diagram illustrating pixel data output from each bus, and FIG. 4D is a diagram illustrating a bus selected by a first selector in the correlation degree calculation unit 30.

When the start signal 22 is input, the counter 2
6 starts counting the double clock 24c output from the clock generation circuit 25, and outputs the counted data to the address analyzer 2;
7 is output. In FIG. 10A, the time is represented for each cycle of the clock 21 supplied from the outside, so that the counter 26 outputs two pieces of count data according to the timing of the odd clock 24a and the even clock 24b for each cycle. When the counter 26 starts counting, the address analyzer 27 stores the address 24d in the search area memory 10.
Is output. The address 24d output at the timing of the odd clock 24a starts from "1", while the address 24d output at the timing of the even clock 24b starts from "9" and is sequentially incremented. When each address 24d is incremented to "20", it becomes "1" in the next cycle, and is incremented again.

The pixel data read out from each of the storage element sections 11a to 11c by the address 24d is shown in FIG.
(B). For example, when the address "1" is output at the timing of the odd clock 24a, the pixel data C0,0 from the storage element 11a, the pixel data C4,0 from the storage element 11b, and the pixel data C8 from the storage element 11c. , 0 is read out, the pixel data C0,0 is output to the delay circuit 13a by the first memory selector 12a, and the second
The pixel data C4,0 is output to the delay circuit 13c by the memory selector 12b.

Next, when the address "9" is output at the timing of the even clock 24b, the pixel data C0,2 from the storage element section 11a, the pixel data C4,2 from the storage element section 11b, and the storage element section 11c. , The pixel data C4,2 is output to the delay circuit 13b by the first memory selector 12a, and the pixel data C8,2 is output to the delay circuit 13d by the second memory selector 12b.
Is output to

However, when the time is 1 to 12, the delay circuit 1
The pixel data output to 3b and 13d is unnecessary for the correlation degree calculation. In addition, at time 21 to 32, the delay circuit 1
The pixel data output to 3a and 13c is not necessary for the calculation of the degree of correlation. In FIG. 10B, pixel data actually used for the correlation calculation is surrounded by a thick line frame. Pixel data output to the delay circuit but not used for the correlation calculation is surrounded by a broken line frame.

Here, the maximum value "20" of the address 24d
Corresponds to the number of pixel data stored in each of the storage element units 11a to 11c, that is, the number of pixels in each of the partial search areas E1 to E3. Generally, it is represented by the following equation. Further, a time 13 surrounded by a thick line in FIG.
8 to 20 cycles (addresses "13" to "20" are output at the timing of the odd clock 24a,
A period during which addresses “1” to “8” are output at the timing of the even clock 24b) is a period during which the search area memory 10 outputs four pixel data. The difference between the address output at the timing of the odd clock 24a and the address output at the timing of the even clock 24b is 12. Generally, it is represented by the following equation.

[0103] Further, when the value of the counter 26 is "25" and "26", the address 24d is "13" at the timing of the odd clock 24a and "1" at the timing of the even clock 24b.
become.

Generally speaking, if the value of the counter 26 is "2
× (M × L) +1 ”and“ 2 × (M × L) +2 ”, the address 24d becomes“ M × L + 1 ”at the timing of the odd clock 24a, becomes“ 1 ”at the timing of the even clock 24b, and thereafter. M × (N−) until “M × (N + L−1)” at the timing of the odd clock 24a and “M × (N−1)” at the timing of the even clock 24b.
1) The cycle period is a period during which the search area memory 10 outputs four pixel data.

That is, as can be seen from FIG. 10B, the pixel data read from the storage elements 11a and 11b at the timing of the odd clock 24a during the first M × L (= 12) cycles, M × (N−1) (=
8) Pixel data read from the storage element units 11a and 11b at the timing of the odd clock 24a during the cycle and the storage element unit 11 at the timing of the even clock 24b.
b, 11c, followed by M ×
The pixel data read from the storage element units 11b and 11c at the timing of the even clock 24b during L (= 12) cycles is used for the calculation of the degree of correlation.

The pixel data input to the delay circuit 13a is output to the bus A1 one cycle later, and the pixel data input to the delay circuit 13b is output to the bus A2 one cycle later. The pixel data input to the delay circuit 13c is output to the M pixel delay circuit 13e one cycle later,
After being delayed by M (= 4) clocks by the pixel delay circuit 13e, it is output to the bus B1. The pixel data input to the delay circuit 13d is the M pixel delay circuit 1 after one cycle.
3f, and M (=
4) After being delayed by the clock, it is output to the bus B2.

FIG. 10 (c) shows buses A1, A2, B1,.
The pixel data output to B2 is shown. Bus B1,
The pixel data output to B2 is an M pixel delay circuit 13e,
Since the pixel data is delayed by M clocks by 13f, the pixel data is output M cycles later than the pixel data simultaneously read from the storage element units 11a to 11c. FIG.
0 (c) indicates only the pixel data within the solid thick frame in FIG. 10 (b).

Here, looking at the pixel data at time 18 (a portion surrounded by a thick line in FIG. 10C), the bus A1
Pixel data C0,4 on the bus B1 and pixel data C4,3 on the bus B1.
However, it can be seen that the pixel data C4,1 is output to the bus A2 and the pixel data C8,0 is output to the bus B2. These pixel data correspond to the pixel data circled in FIG. 8 and the pixel data surrounded by the thick broken line in FIG. Also, it corresponds to pixel data circled in FIG.

The pixel data C4,3 is delayed by four cycles from the pixel data C0,3 read out from the storage element section 11a in the same cycle (time 13), and the pixel data C0,4 output to the bus A1 It is output to the bus B1 in the same cycle. The pixel data C8,0 is delayed by four cycles from the pixel data C4,0 read from the storage element unit 11b in the same cycle (time 13) and is the same as the pixel data C4,1 output to the bus A2. It is output on bus B2 in the cycle.

By supplying the pixel data to the correlation calculation unit in this manner, the pipeline processing is not interrupted at the time of horizontal folding in the processing order of the candidate blocks shown in FIG. For example, candidate block Bb3,0
, The pixel data C4,0 and the pixel data C0,1 can be simultaneously supplied to the correlation operation unit, so that the pipeline processing is not interrupted.

Further, the pixel data C0,4 and C4,3 and the pixel data C4,1 and C8,0 are simultaneously output to the correlation operation unit 30, so that the vertical folding in the processing order of the candidate block is performed. Sometimes, pipeline processing is not interrupted. For example, when returning from the candidate block Bb3,2 to the candidate block Bb4,0, the pipeline processing is not interrupted. Therefore, the correlation degree calculator 30 can continuously calculate the correlation degree.

The correlation calculation unit 30 inputs a maximum of four pixel data per cycle from the buses A1, A2, B1, and B2 in the same cycle. First selectors 32e to 34f
10D according to the first switching signals 42a to 42f output from the arithmetic control unit 40, as shown in FIG.
One of the types of buses A1, A2, B1, and B2 is selected. The signals of the selected bus are provided to the first selectors 32e to 32e.
4f is output to the connected buses 32A to 34B. By this operation, each of the pixel operation units 32a to 34d
Are supplied to the buses in the line operation units 32 to 34, respectively.

The second selector in each of the pixel operation units 32a to 34d is a second selector to which the pixel data in the line operation unit is transferred.
One of the buses is selected, and pixel data required for calculating the degree of correlation is input. For example, in the case of the line operation unit 32, the second selectors in the pixel operation units 32a to 32d are:
One of the buses 32A and 32B is selected, and pixel data is input.

As a result of the above operation, the pixel data input to each pixel operation unit is as shown in FIG. 9, and it is possible to continuously perform the pipeline processing.

When the start signal 22 is input, the reference image block bus is connected to the reference image block Bp 12 (=
4 × 3 = M × N) pixel data r0,0 to r3,2 are converted to 12
The data is sequentially transferred over cycles, and the pixel operation units 32a to 34d
Are stored one by one. This operation is performed by the storage control signal 41a.
４１41 l. Each pixel operation unit holds the stored pixel data of the reference image block Bp for at least the number of cycles of the candidate blocks Bb (here, 8 × 3), and performs an operation with the input pixel data of the candidate block Bb. Do. As a result, the sum of absolute differences for each candidate block Bb can be obtained for each cycle in the order shown in FIG. By comparing the calculated sums of absolute differences, the candidate block having the highest correlation can be obtained, and thereby the motion vector can be obtained.

After the lapse of 8.times.3 cycles, it is also possible to similarly input each pixel data of the next reference image block Bp to the pixel operation section and calculate the degree of correlation.

As described above, according to the correlation degree calculating apparatus according to the present embodiment, the correlation degree between the reference image block of (M × N) pixels and the (M × m) × L candidate blocks is determined. Operations can be performed continuously by pipeline processing.

Next, the correlation calculation section 30 and the calculation control section 4
The arrangement configuration of 0 will be described with reference to FIGS.

In the correlation calculation device according to the present embodiment, it is necessary to arrange many pixel calculators having the same configuration. Since the arrangement configuration of the pixel operation unit greatly affects the circuit area, it is necessary to sufficiently consider the design and manufacture of the correlation operation device.

As shown in FIG. 4, line operation units 32 to 3
4 are arranged in parallel because they have the same configuration, and the wiring between the line operation units is only a line for transferring the correlation data. Further, inside each line operation unit, the pixel operation units are arranged line-symmetrically with respect to the bus for transferring the pixel data of the reference image block and the bus for transferring the pixel data of the candidate block. This arrangement can reduce the length of the bus,
The length of the wiring can be minimized by folding and connecting the line for transferring the correlation data.

Further, as shown in FIG.
Are line control units 44 to 46 for controlling each line calculation unit
Therefore, if a block is constituted by a pair of a line operation unit and a line control unit, the correlation degree operation unit 30 and the operation control unit 40 are constituted simply by arranging the blocks in parallel. In addition, even when the number of lines is increased, there is no need to add an external control circuit, and the correlation calculation device can be realized with a simple configuration.

Here, the effect of the processing order of the candidate blocks Bb as shown in FIG. 2 adopted in this embodiment will be described in comparison with a method of sequentially scanning the search area E to the end in the horizontal direction.

FIG. 11 shows the reference image block B shown in FIG.
In p and the search area E, the internal structure of a correlation degree calculation unit used to calculate the degree of correlation between the reference image block Bp and each candidate block Bb by a method of sequentially scanning the search area E to the end in the horizontal direction. FIG. FIG. 12 is a diagram showing a flow of the pipeline processing by the correlation degree calculation unit shown in FIG.

FIGS. 9 and 1 showing the processing according to the present embodiment.
Comparing with No. 2, it can be seen that the use efficiency of each pixel operation unit is significantly lower. In addition, 56 cycles are required to calculate the degree of correlation for 24 candidate blocks, and the processing is 1.5 times or more slower than 36 cycles in the present embodiment. As can be seen from FIG. 12, a delay of 7 (= m × M−1) cycles during the operation is 2
Therefore, it is necessary to provide seven stages of registers 71 and 72 as shown in FIG. Therefore, the area of the entire apparatus increases.

Therefore, by employing the processing order of the candidate blocks Bb as shown in FIG. 2, the processing speed can be increased and the device area of the correlation degree calculating device can be reduced.

(Second Embodiment) A correlation calculation device according to a second embodiment of the present invention will be described.

FIG. 13 is a block diagram showing the internal configuration of the correlation degree calculation section 35 in the correlation degree calculation apparatus according to the present embodiment. The correlation calculation device according to the present embodiment is different from the correlation calculation device shown in FIG.
1 except for the degree of correlation calculation section 35, and the other components are the same.

The correlation calculating section 35 is composed of 3 (= N) line calculating sections each having 4 (= M) pixel calculating sections. The internal configurations of the line calculation units 36 to 38 are the same, and the pixel calculation units 36a to 36d,
The internal configurations of 37a to 37d and 38a to 38d are the same.

The difference between the correlation calculation unit 35 shown in FIG. 13 and the correlation calculation unit 30 shown in FIG. 4 is that the buses A1, A2, B
The means for selecting 1 and B2 are provided by the pixel operation units 36a to 38d.
It is a point provided inside. Taking the line operation unit 36 as an example, there is no first selector, and buses A1, A2, B1,.
B2 is directly connected to the pixel operation units 36a to 36d. Further, the first switching signals 42a and 42b are also input to the pixel calculation units 36a to 36d.

FIG. 14 is a block diagram showing the internal configuration of the pixel operation unit 36b. In FIG. 14, the third selector 87 selects the buses A1, A2, B1,... According to the first switching signal 42a, 42b and the second switching signal 43b.
One of B2 is selected, and the pixel data of the selected bus is output to the difference absolute value calculator 83.

With such a configuration, the correlation calculation device according to the present embodiment can perform the same operation as the correlation calculation device according to the first embodiment.

(Third Embodiment) FIG. 15 is a block diagram showing a configuration of a parallel correlation degree calculation device according to a third embodiment of the present invention. In FIG. 15, reference numeral 61 denotes a first correlation degree calculation device, 62 denotes a second correlation degree calculation device, 63 denotes a correlation degree adder, and the first correlation degree calculation device 61 and the second correlation degree calculation device are: 4 is a correlation degree calculation device according to the first embodiment shown in FIG. The first correlation degree calculation device 61 calculates a correlation degree 1 for the reference image block 51a, and the second correlation degree calculation device 62 calculates a correlation degree 2 for the reference image block 51b. The correlation degree adder 63 adds the correlation degree 1 and the correlation degree 2 and outputs a correlation degree 3. This makes it possible to increase the processing speed when simultaneously calculating the degree of correlation when the search region is divided into two and the degree of correlation for the entire search region.

In the motion vector detection of MPEG2, it is expected that three types of motion vectors are obtained, two motion vectors obtained based on two types of fields constituting one frame and a motion vector obtained on a frame basis. Have been.

In the parallel correlation degree calculating device shown in FIG.
The evaluation value of each field can be calculated by the correlation degree calculation devices, and the evaluation value of the frame can be calculated only by adding each evaluation value. Three types of evaluation values can be calculated by such a simple configuration.
This is because the correlation degree calculation device of the present invention can calculate the evaluation value for each cycle.

In the first to third embodiments, M =
4, N = 3, m = 2, L = 3, that is, the case where the block size of the reference image block Bp is 4 × 3 pixels and the number of candidate blocks Bb is (2 × 4) × 3 has been described. However, the present invention is not limited to this, and the reference image block Bp of (M × N) pixels is ((M × M) × L)
The pipeline processing can be continuously performed on the candidate blocks Bb. Also, the same processing can be continuously performed without interruption for an arbitrary number of reference image blocks. Where M, N, L, and m are natural numbers and L
Is N or more and m is 2 or more.

Further, in the first to third embodiments, the case where the sum of absolute differences is used as the evaluation data of the degree of correlation has been described as an example. However, the present invention is not limited to this. Any device can be applied as long as it is a device that performs an evaluation calculation of the degree of correlation, such as by using a comparator or a size comparator.

In the first to third embodiments, the number of storage elements is minimized in order to reduce the area of the search area memory, and the data output from the storage elements is controlled by the timing adjusting means. Supplies pixel data to each pixel operation unit. However, there is no problem if a large number of storage element units are provided and pixel data is stored in the storage element unit in the order required by each pixel operation unit.

In each of the first to third embodiments, each pixel operation unit continuously performs pipeline processing in order to increase the operation efficiency. There is no problem if they are not necessarily continuous.

It should be noted that there is no problem if a plurality of stages of pipeline registers are inserted in order to speed up the correlation operation unit.

[0140]

As described above, according to the present invention, (M ×
N) For a reference image block of pixels, ((m × M) ×
The calculation of the degree of correlation with the L) candidate blocks can be continuously performed by pipeline processing. Further, even when the search area is expanded, the search area memory, the correlation degree calculation unit, and the calculation control unit that constitute the correlation degree calculation device can be realized with a simple configuration. Therefore, the search area can be easily expanded, a high-speed processing can be performed even if the search area is expanded, and a correlation degree calculation device having a simple configuration can be realized.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a block matching method and a motion vector, and is a diagram illustrating a reference image block, a candidate block, and a search area.

FIG. 2A is a diagram illustrating each candidate block Bb in a search area E, and FIGS. 2B and 2C are diagrams illustrating a search order of the candidate blocks Bb in the embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration of a correlation degree calculation device according to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of a correlation degree calculation unit in the correlation degree calculation device according to the first embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration of a calculation control unit in the correlation degree calculation device according to the first embodiment of the present invention.

FIG. 6 is a circuit diagram showing a configuration of a second control unit in the arithmetic control unit shown in FIG.

FIG. 7 is a block diagram illustrating a configuration of a pixel operation unit in the correlation degree operation unit illustrated in FIG. 4;

FIG. 8 is a diagram illustrating pixel data stored in each storage element unit of a search area memory in the correlation degree calculation device according to the embodiment of the present invention.

FIG. 9 is a diagram illustrating pipeline processing realized by the correlation degree operation device according to the embodiment of the present invention, and is a diagram illustrating pixel data requested by the pixel operation unit in each clock cycle.

FIGS. 10A and 10B are diagrams for explaining the operation of the correlation degree calculation device according to the first embodiment of the present invention, wherein FIG. 10A illustrates the operation of the main control unit, and FIG. FIG. 7C is a diagram showing pixel data read from a search area, FIG. 7C is a diagram showing pixel data output from a search area, and FIG. 7D is a diagram showing a bus selected by a first selector in a correlation degree calculation unit.

FIG. 11 is a block diagram illustrating an internal configuration of a correlation degree calculation unit used to calculate a degree of correlation between a reference image block and each candidate block by a method of sequentially scanning a search area to an end in a horizontal direction.

FIG. 12 is a diagram showing a flow of pipeline processing by a correlation degree operation unit shown in FIG. 11;

FIG. 13 is a block diagram showing a configuration of a correlation degree calculation unit in the correlation degree calculation device according to the second embodiment of the present invention.

FIG. 14 is a block diagram illustrating a configuration of a pixel calculation unit in a correlation calculation unit in the correlation calculation device according to the second embodiment of the present invention.

FIG. 15 is a block diagram illustrating a configuration of a parallel correlation degree calculation device according to a third embodiment of the present invention.

[Explanation of symbols]

Fp Reference image frame Bp Reference image block Fb Candidate frame Bb Candidate block E Search area 10 Search area memory 11a to 11c Storage element unit 12a First memory selector 12b Second memory selector 13 Delay means 20 Main control unit 23 Operation Control signal 24 Memory control signal 30 Correlation degree operation unit 32 to 34 Line operation unit 32a to 32d, 33a to 33d, 34a to 34d Pixel operation unit 32e, 32f, 33e, 33f, 34e, 34f First selector 40 Operation control Units 41a to 41l Storage control signals 42a to 42f First switching signals 43a to 43l Second switching signals 44 to 46 Line controllers 44a to 44d Second controllers 44e First controllers 61 First correlation degree calculation Device 62 Second Correlation Calculation Device 63 Correlation Adder 82 Second Selector

Continued on the front page (72) Inventor Katsuji Aoki 1006 Kazuma Kadoma, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd. 72) Inventor Masahiro Gion 1006 Kadoma Kadoma, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd. JP-A-5-328332 (JP, A) JP-A-6-339135 (JP, A) 1991 IEICE Spring National Convention Lecture Paper Volume 7, p. 64 IEEE Transactions on circuits and systems, Vol. 36, No. 10, p. 1309-1316 (58) Field surveyed (Int.Cl. ⁶ , DB name) H04N ^7/ 24-7/68

Claims (28)

(57) [Claims] 1. A reference image block of (M × N) pixels is subjected to block matching ((m × M) × L).
A correlation degree calculation device for calculating the degree of correlation with the candidate blocks (where M, N, L, and m are natural numbers, L ≧ N,
m ≧ 2), the correlation degree is calculated by pipeline processing for each candidate block group including (M × L) candidate blocks, and pixel data required for calculation for one candidate block group and the one candidate block A correlation degree calculation device characterized in that pixel data required for calculation for another candidate block group to be processed subsequent to calculation for a group is used for pipeline processing in the same clock cycle. 2. A reference image block of (M × N) pixels is subjected to block matching ((m × M) × L).
A correlation degree calculation device for calculating the degree of correlation with the candidate blocks (where M, N, L, and m are natural numbers, L ≧ N,
a search area memory for storing pixel data of a search area including (m × 2) and ((m × M) × L) candidate blocks, and pixel data of a reference image block of (M × N) pixels, The calculation of the degree of correlation between the reference image block and the candidate block is performed using (M × L) candidate blocks by using the pixel data of the held reference image block and the pixel data of each candidate block output from the search area memory. And a correlation operation unit for performing a pipeline process for each of the candidate block groups, wherein the search area memory is provided with pixel data necessary for the operation on the one candidate block group and the operation on the one candidate block group. A function of outputting pixel data necessary for an operation on another candidate block group to be processed to the correlation degree operation unit in the same clock cycle; Parts are correlation degree calculation apparatus characterized by being executable constructed by sequentially calculating for each candidate block group. 3. The correlation degree calculation device according to claim 2, wherein the search area memory includes ((m × M) × L) candidate blocks ((m + m)
The first pixel data and the first pixel data, which are pixel data necessary for the operation on the one candidate block group, from among the pixel data in the search area of (1) × M−1) columns (L + N−1) rows.
And the third pixel data and the third pixel data, which are the pixel data necessary for the operation on the other candidate block group and the second pixel data one row above and M columns right of the pixel data A correlation calculation device having a function of outputting fourth pixel data on one row and M column right to the correlation calculation unit in the same clock cycle. 4. The correlation degree calculation device according to claim 3, wherein the third pixel data is L less than the first pixel data.
A correlation degree calculation device characterized by being on a row and to the right of an M column. 5. A reference image block of (M × N) pixels is subjected to block matching ((m × M) × L).
A correlation calculation device for calculating the correlation with the candidate blocks (where M, N, L, and m are natural numbers and L ≧ N,
m ≧ 2), a main control unit that generates and outputs a memory control signal and an arithmetic control signal based on a given clock, and a pixel in a search area including ((m × M) × L) candidate blocks A search area memory for storing data and outputting pixel data stored according to an instruction of a memory control signal output from the main control unit; and holding pixel data of a reference image block of (M × N) pixels. The calculation of the degree of correlation between the reference image block and the candidate block is performed using the pixel data of the held reference image block and the pixel data output from the search area memory (M ×
L) a correlation degree calculation unit that performs pipeline processing for each candidate block group including candidate blocks, and a calculation control unit that controls the correlation degree calculation unit according to an instruction of a calculation control signal output from the main control unit. The search area memory includes pixel data required for an operation on one candidate block group and pixel data necessary for an operation on another candidate block group processed subsequent to the operation on the one candidate block group. The correlation degree calculation unit has a function of outputting the correlation degree calculation unit in the same clock cycle, and the correlation degree calculation unit is configured to be able to continuously execute the calculation for each candidate block group. . 6. The correlation degree calculation device according to claim 5, wherein the search area memory includes ((m × M) × L) candidate blocks ((m +
The first pixel data and the first pixel data, which are pixel data necessary for the operation on the one candidate block group, from among the pixel data in the search area of (1) × M−1) columns (L + N−1) rows.
And the third pixel data and the third pixel data, which are the pixel data necessary for the operation on the other candidate block group and the second pixel data one row above and M columns right of the pixel data A correlation calculation device having a function of outputting fourth pixel data on one row and M column right to the correlation calculation unit in the same clock cycle. 7. The correlation degree calculation device according to claim 6, wherein the third pixel data is L less than the first pixel data.
A correlation degree calculation device characterized by being on a row and to the right of an M column. 8. The correlation calculation device according to claim 5, wherein the search area memory has a function of outputting four pixel data per one clock cycle. 9. The correlation degree calculation device according to claim 8, wherein the search area memory includes ((m × M) × L) candidate blocks ((m + m)
1) × M−1) A search area of (L + N−1) rows is (M ×
(L + N-1)) (m + 1) obtained by dividing each pixel
The pixel data of each of the partial search areas is stored, and the pixel data is read out according to an access instruction by a memory control signal output from the main control unit (m + 1).
Memory element units; and timing adjustment means for adjusting the timing of the pixel data read from the (m + 1) storage element units and outputting the adjusted pixel data to the correlation degree calculation unit. Is for instructing the search area memory to perform two accesses in one clock cycle by the memory control signal. The timing adjusting unit performs one access from the (m + 1) storage element units. A correlation degree calculating device for selecting two of the read pixel data, adjusting the timing, and outputting four pixel data per one clock cycle. 10. The correlation degree calculation device according to claim 9, wherein the timing adjustment unit is configured to read the pixels read from the first to m-th storage element units in the (m + 1) storage element units. A first memory selector for selecting and outputting one of the data; and a second to a (m +) th one of the (m + 1) storage element units.
1) of the pixel data read from the storage element section of 1)
A second memory selector for selecting and outputting one of the pixel data, and for the pixel data selected and output by the first memory selector, the pixel data selected and output by the second memory selector for M clock cycles. A correlation calculation device, comprising: delay means for delaying. 11. The correlation degree calculation device according to claim 8, wherein the correlation degree calculation unit selects and outputs one different pixel data among the four pixel data output from the search area memory. Holding the pixel data of the reference image block, selecting and holding one of the two pixel data selected and output by the two first selectors. By providing N number of line calculation units each having M pixel calculation units for calculating the evaluation value with the pixel data, and accumulating the evaluation values calculated by each pixel calculation unit, An apparatus for calculating a degree of correlation, wherein the degree of correlation between a reference image block of (M × N) pixels and each candidate block is obtained. 12. The correlation degree calculation device according to claim 11, wherein the pixel calculation units each include: a reference image storage unit that stores input pixel data of a reference image block; and the two first image storage units. A second selector for selecting one of the two pixel data selected and output by the selector, and a second selector for selecting the pixel data held in the reference image storage unit.
An arithmetic unit for calculating an evaluation value with the pixel data selected by the selector, and an adder for adding the evaluation value calculated by the arithmetic unit and the accumulated evaluation value output from the preceding pixel operation unit And a register for temporarily storing output data of the adder and outputting the output data to a pixel operation unit at the next stage. 13. In each line operation unit, M pixel operation units are symmetrically arranged in two columns on both sides of a bus for transferring pixel data selected and output by the first selector. The correlation calculation device according to claim 11, wherein: 14. The correlation calculation device according to claim 13, wherein the evaluation value is located at the end of the pixel calculation unit in one column where the first selector is arranged. To the pixel operation unit located at the end where the first selector is not arranged, and then to the pixel operation unit located at the end where the first selector of another column is not arranged ,continue,
In the pixel operation units of the other columns, a pixel operation unit located at an end where the first selector is not disposed is shifted from a pixel operation unit located at an end where the first selector is arranged. A degree-of-correlation calculating device which is sequentially transferred and accumulated in a section. 15. The correlation degree calculation device according to claim 8, wherein the correlation degree calculation unit holds the pixel data of the reference image block, and outputs the four pixel data out of the four pixel data output from the search area memory. (M × N) pixel operation units for selecting one and calculating an evaluation value with the held pixel data are provided, and by accumulating the evaluation values calculated by each pixel operation unit, An apparatus for calculating a degree of correlation, wherein the degree of correlation between a reference image block of (M × N) pixels and each candidate block is obtained. 16. The correlation calculation device according to claim 15, wherein the pixel calculation unit includes: a reference image storage unit that holds input pixel data of the reference image block; A selector for selecting one of the pixel data, an arithmetic unit for calculating an evaluation value of the pixel data held in the reference image storage unit and the pixel data selected by the selector, and the arithmetic unit An adder for adding the evaluation value calculated by the above and the accumulated evaluation value output from the preceding pixel operation unit; and a register for temporarily storing output data of the adder and outputting the output data to the next stage pixel operation unit. And a correlation degree calculating device. 17. The correlation degree calculation device according to claim 8, wherein the correlation degree calculation unit holds M pixel data of each row of the reference image block,
The apparatus further comprises N line operation units for calculating an evaluation value of the pixel data output from the search area memory and the held pixel data, and wherein the operation control unit includes: N operation units for controlling the N line operation units, respectively. A degree of correlation calculation device comprising: 18. The correlation calculation device according to claim 17, wherein the N line control units are cascaded,
An apparatus for calculating a degree of correlation, wherein an arithmetic control signal input to the arithmetic control unit is sequentially transferred to the N line control units. 19. The correlation calculation device according to claim 18, wherein each of the line calculation units selectively outputs a different one of the four pixel data output from the search area memory. Two first selectors and pixel data of a reference image block are respectively held, and one of the two pixel data selected and output by the two first selectors is selected and held. M pixel operation units for calculating an evaluation value with the pixel data being processed, wherein the line control units are respectively a first control unit that controls the two first selectors, A correlation calculation device comprising: M second control units for controlling M pixel calculation units, respectively. 20. The correlation degree calculation device according to claim 19, wherein the M second control units in each line control unit are cascaded, and an arithmetic control signal input to the line control unit is A correlation degree calculation device, wherein the M second control units are sequentially transferred and output after M clock cycles after being input. 21. A correlation calculation device for calculating a correlation between a reference image block of (M × N) pixels and a plurality of candidate blocks by a block matching method (where M and N are natural numbers). A search area memory that stores pixel data of a search area including each candidate block to be operated; and pixel data of a reference image block of (M × N) pixels, and stores the pixel data of the reference image block and each candidate block. A correlation degree calculation unit that performs a calculation of the degree of correlation by pipeline processing using the stored pixel data of the reference image block and the pixel data of each candidate block output from the search area memory, according to an input operation control signal An operation control unit that controls the correlation degree operation unit, wherein the correlation degree operation unit holds pixel data of the reference image block, and holds the held pixel data. M for calculating the evaluation value of the output pixel data with the chromatography data from the search area memory
It consists of N line operation units each having N pixel operation units, and these (M × N) pixel operation units are cascaded.
Are connected, and the evaluation values of the calculation results of each pixel
The correlation calculation device is configured to be transferred while being added next, and wherein the calculation control unit includes N line control units for controlling the N line calculation units, respectively. 22. The correlation calculation device according to claim 21, wherein the N line controllers are cascaded,
An apparatus for calculating a degree of correlation, wherein an arithmetic control signal input to the arithmetic control unit is sequentially transferred to the N line control units. 23. The correlation calculation device according to claim 22, wherein the line control units each include M control units for controlling the M pixel calculation units, respectively. Degree calculation device. 24. The correlation degree calculation device according to claim 23, wherein the M control units in each line control unit are cascade-connected, and the M control calculation signals input to the line control units are the M control units. Wherein the control section is sequentially transferred and output after M clock cycles after being input. 25. For a reference image block of (M × N) pixels, ((m × M) ×
A correlation degree calculation method for calculating the degree of correlation with (L) candidate blocks (where M, N, L, and m are natural numbers, and L ≧
N, m ≧ 2), ((m × M) × L) candidate blocks are divided into m candidate block groups each including (M × L) candidate blocks. A first step of performing block matching with a reference image block for M candidate blocks that are successively folded and lined up; and a second step of repeating the first step L times while moving in the vertical direction. A method for calculating a degree of correlation characterized by performing the following. 26. The correlation degree calculation method according to claim 25 , wherein pixel data required for operation on one candidate block group and another candidate block group processed following the operation on said one candidate block group are processed. A correlation calculation method characterized by simultaneously using pixel data required for calculation. 27. A reference image block of (M × N) pixels is subjected to ((m × M) ×
A correlation degree calculation device for calculating the degree of correlation with L) candidate blocks (where M, N, L, and m are natural numbers, and L ≧ L)
N, m ≧ 2), and the calculation of the degree of correlation is performed for each of m candidate block groups consisting of (M × L) candidate blocks and connected in one of the horizontal direction and the vertical direction. The calculation of the degree of correlation in each candidate block group is performed by pipeline processing, and the step of performing block matching with the reference image block for the M candidate blocks continuously folded and arranged in one direction is performed in the other direction. It repeats L times while moving, the pixel data required for the operation on one candidate block group, and the pixel data required for the operation on another candidate block group processed following the operation on the one candidate block group. Is used for pipeline processing in the same clock cycle. 28. For a reference image block of (M × N) pixels, ((m × M) ×
A correlation degree calculation method for calculating the degree of correlation with (L) candidate blocks (where M, N, L, and m are natural numbers, and L ≧
N, m ≧ 2) and ((m × M) × L) candidate blocks are respectively assigned to (M
× L) divided into m candidate block groups consisting of candidate blocks and connected in one of the horizontal direction and the vertical direction, and for each candidate block group, successively in the one direction. A first step of performing block matching with a reference image block for the M candidate blocks arranged in a folded manner, and a second step of repeating the first step L times while moving in the other direction. And simultaneously using pixel data required for an operation on one candidate block group and pixel data required for an operation on another candidate block group processed subsequent to the operation on the one candidate block group. Correlation degree calculation method.