JP5622501B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for detecting a movement amount (deviation amount) between two images while sequentially inputting images.

FIG. 11 is a block diagram showing a conventional image processing apparatus.
In the image processing apparatus of FIG. 11, the shift amount between two images is detected while sequentially inputting the images, but here, for convenience of explanation, the image A input earlier in time and the time later The following description will be given on the assumption that the amount of deviation from the input image B is detected.

When the column direction integrated value calculation unit 101 receives an image signal indicating the image A, the block in a certain range included in the image A (for example, a block located at the center of the image A) is set as a shift amount detection target range, By referring to the pixel values (for example, luminance values) of the pixels constituting the image A and accumulating the pixel values for each column in the shift amount detection target range, each pixel in the shift amount detection target range. An integrated value of pixel values in the column is calculated.
Further, when the image signal indicating the image B is input, the column direction integrated value calculation unit 101 refers to the pixel value (for example, luminance value) of the pixels constituting the image B, and detects the above-described deviation amount detection target range ( By integrating the pixel values for each column in the same range as the shift amount detection target range in the image A, the integrated value of the pixel values in each column in the shift amount detection target range is calculated.
It should be noted that the integrated value of the pixel values in each column within the deviation detection target range of the images A and B and the intermediate integrated value until the integrated value is finally calculated are stored in the column direction integrated value buffer 102. The

When the row direction integrated value calculation unit 107 receives an image signal indicating the image A, the row direction integrated value calculation unit 107 refers to the pixel value (for example, the luminance value) of the pixels constituting the image A, and sets the row in the shift amount detection target range. By integrating the pixel values every time, an integrated value of the pixel values in each row within the shift amount detection target range is calculated.
Further, when the row direction integrated value calculation unit 107 receives an image signal indicating the image B, the row direction integrated value calculation unit 107 refers to the pixel value (for example, the luminance value) of the pixels constituting the image B and falls within the above-described deviation amount detection target range. By integrating the pixel values for each row, the integrated value of the pixel values in each row within the deviation detection target range is calculated.
It should be noted that the integrated value of the pixel values in each row in the deviation detection target range of the images A and B and the intermediate integrated value until the integrated value is finally calculated are stored in the row direction integrated value buffer 108. .

When the column direction integrated value calculating unit 101 calculates the integrated value of the pixel values in each column within the deviation detection target range of the image A, the row direction displacement feature amount calculating unit 103 is stored in the column direction integrated value buffer 102. Sequentially extract the integrated values of two rows adjacent in the row direction from the integrated values of the pixel values in each row within the detection range of the deviation amount of image A, and quantize the difference value of the integrated values of the two rows To do.
The quantized value of the difference value is stored in the row direction displacement feature amount buffer 104 as a row direction displacement feature amount in the deviation amount detection target range of the image A.
Further, the row direction displacement feature amount calculation unit 103 stores in the column direction integration value buffer 102 when the column direction integration value calculation unit 101 calculates the integration value of the pixel values in each column within the shift amount detection target range of the image B. The integrated values of two rows adjacent in the row direction are sequentially extracted from the integrated values of the pixel values in each row within the detection range of the deviation amount of the image B, and a difference value between the integrated values of the two rows is obtained. Quantize.
The quantized value of the difference value is stored in the row direction displacement feature amount buffer 104 as a row direction displacement feature amount in the shift amount detection target range of the image B.

When the row direction integrated value calculation unit 107 calculates the integrated value of the pixel values in each row within the shift amount detection target range of the image A, the column direction displacement feature amount calculation unit 109 is stored in the row direction integrated value buffer 108. The integrated values of two columns adjacent in the column direction are sequentially extracted from the integrated values of the pixel values in each column within the deviation detection target range of the image A, and the difference value between the integrated values of the two columns is quantized. To do.
The quantized value of the difference value is stored in the column direction displacement feature amount buffer 110 as the column direction displacement feature amount in the deviation amount detection target range of the image A.
Further, the column direction displacement feature amount calculation unit 109 stores the pixel value integration value in each row within the deviation amount detection target range of the image B when the row direction integration value calculation unit 107 calculates the integration value of the pixel values in the row direction integration value buffer 108. The integrated values of the two columns adjacent in the column direction are sequentially extracted from the integrated values of the pixel values in each column within the deviation amount detection target range of the image B, and the difference value of the integrated values of the two columns is obtained. Quantize.
The quantized value of the difference value is stored in the column direction displacement feature amount buffer 110 as the column direction displacement feature amount in the deviation amount detection target range of the image B.

The row direction correlation index value calculation unit 105 calculates the row direction displacement feature amount in the deviation amount detection target range of the image A calculated by the row direction displacement feature amount calculation unit 103 and the row direction in the deviation amount detection target range of the image B. By performing the correlation calculation with the displacement feature amount, a row direction correlation index value indicating how similar the displacement feature amount in the row direction in the displacement detection target range of the images A and B is calculated.
At this time, by repeating the correlation calculation while changing the shift amount in the row direction between the shift amount detection target ranges of the images A and B, the row direction correlation index value related to the shift amount in each row direction is calculated.

The column direction correlation index value calculation unit 111 calculates the column direction displacement feature amount in the shift amount detection target range of the image A calculated by the column direction displacement feature amount calculation unit 109 and the column direction in the shift amount detection target range of the image B. By calculating the correlation with the displacement feature amount, a column direction correlation index value indicating how similar the displacement feature amount in the column direction in the deviation detection target range of the images A and B is calculated.
At this time, by repeating the correlation calculation while changing the shift amount in the column direction between the shift amount detection target ranges of the images A and B, the column direction correlation index value related to the shift amount in each column direction is calculated.

When the row direction correlation index value calculation unit 105 calculates a plurality of row direction correlation index values having different row direction deviation amounts in the shift amount detection target ranges of the images A and B, a plurality of row direction correlation index values are calculated. Among the row direction correlation index values, the row direction correlation index value indicating that the displacement characteristic amounts in the row direction in the shift amount detection target ranges of the images A and B are most similar is specified.
Then, the row direction deviation amount detection unit 106 outputs the amount of deviation in the row direction when calculating the specified row direction correlation index value as the amount of deviation in the row direction between the image A and the image B.

When the column direction correlation index value calculation unit 111 calculates a plurality of column direction correlation index values having different column direction deviation amounts in the shift amount detection target ranges of the images A and B, a plurality of column direction correlation index values are calculated. Among the column direction correlation index values, the column direction correlation index value indicating that the column direction displacement feature amounts in the shift amount detection target ranges of the images A and B are most similar is specified.
Then, the column direction deviation amount detection unit 112 outputs the column direction deviation amount when calculating the identified column direction correlation index value as the column direction deviation amount between the image A and the image B.

  Similar to the image processing apparatus of FIG. 11, an image processing apparatus that calculates a correlation value between images A and B and detects a shift amount between images A and B from the correlation value is disclosed in Patent Document 1 below. It is disclosed.

JP 2008-79007 A

  Since the conventional image processing apparatus is configured as described above, a significant feature point may be present in the deviation amount detection target range of the images A and B when detecting the deviation amount between the images A and B. Thus, the amount of deviation between the images A and B can be detected with high accuracy. However, since the deviation amount detection target range is a block of a certain range located at the center of the image, significant feature points that existed in the deviation amount detection target range of the image A due to occurrence of a large image deviation. However, it may come out of the deviation amount detection target range of the image B. In such a case, since the amount of deviation between the images A and B cannot be detected with reference to the position where the significant feature point exists, the detection accuracy of the amount of deviation between the images A and B is greatly deteriorated. There was a problem.

  The present invention has been made to solve the above-described problems, and an image processing apparatus and an image processing capable of detecting a displacement amount between two images with high accuracy even when a large image displacement occurs. The purpose is to obtain a method.

An image processing apparatus according to the present invention includes a deviation amount detection target range extracting unit that extracts a plurality of deviation amount detection target ranges from an input image, and integrates pixel values for each direction in the plurality of deviation amount detection target ranges. Then, the pixel value integrating means for calculating the integrated value and the displacement feature value in a different direction that is different from the one direction in the deviation detection target range by using the integrated value for each of the plurality of deviation detection target ranges. Displacement feature amount calculating means for calculating the displacement, and calculating the correlation of the displacement feature amounts between the matching block, which is a predetermined displacement amount detection target range in the image input before the image, and a plurality of displacement amount detection target ranges And a correlation index value calculation means for calculating a correlation index value, and the deviation amount detection means corresponds to a correlation index value indicating that the correlation is highest among a plurality of correlation index values. Detecting the deviation amount Finds another direction of the deviation of the elephant range and the matching block, and the pixel value integration means integrates the pixel values for each direction in the range of one-way from the displacement amount detection target range is narrow small blocks, The integrated value of the pixel value is stored in the buffer, and a plurality of small blocks are arbitrarily combined, and the integrated value is added to calculate the integrated value of the pixel value in one direction within the plurality of deviation detection target ranges. It is a thing.

According to this invention, the deviation amount detection target range extracting means for extracting a plurality of deviation amount detection target ranges from the input image, and the pixel values are integrated for each direction in the plurality of deviation amount detection target ranges, For each of a plurality of deviation amount detection target ranges, a pixel value integration unit that calculates an integrated value, and for each of the deviation amount detection target ranges, a displacement feature amount in a different direction that is different from one direction in the deviation amount detection target range is calculated. A correlation index is calculated by calculating a correlation of displacement feature amounts between a displacement feature amount calculating means and a matching block that is a predetermined displacement amount detection target range in an image input before the image and a plurality of displacement amount detection target ranges. A correlation index value calculation means for calculating a value, and the deviation amount detection means specifies a deviation amount detection target range corresponding to the correlation index value indicating that the correlation is highest among the plurality of correlation index values. , The deviation detection target range and Finds another direction of the deviation of the quenching block, and the pixel value integration means integrates the pixel values for each direction in the range of one-way from the displacement amount detection target range is narrow small blocks, the pixel values The integrated value is stored in the buffer, and a plurality of small blocks are arbitrarily combined, and the integrated value is added to calculate the integrated value of the pixel values in one direction within the plurality of deviation detection target ranges . Therefore, even if a large image shift occurs in another direction, there is an effect that the shift amount between the two images can be detected with high accuracy.

1 is a configuration diagram illustrating an image processing apparatus according to Embodiment 1 of the present invention; It is a flowchart which shows the processing content when the image processing apparatus by Embodiment 1 of this invention detects the deviation | shift amount of a row direction. It is a flowchart which shows the processing content when the image processing apparatus by Embodiment 1 of this invention detects the deviation | shift amount of a column direction. It is explanatory drawing which shows the calculation concept of the integrated value of the pixel value in each row | line | column in the some deviation amount detection object range. It is explanatory drawing which shows the deviation amount detection object range used when detecting the deviation | shift amount of a row direction, and a matching block. It is explanatory drawing which shows the calculation concept of the row direction displacement feature-value by the row-direction displacement feature-value calculation part. It is explanatory drawing which shows the calculation concept of the correlation index value of the row direction by the row direction correlation index value calculation part. When the deviation amount in the row direction between the images A and B is detected, the amount of displacement is an explanatory diagram showing an example in which _{y 0, y 1, y 2} . It is explanatory drawing which shows the calculation concept of the integrated value of the pixel value in each row | line | column in the some deviation amount detection object range. It is a block diagram which shows the image processing apparatus by Embodiment 2 of this invention. It is a block diagram which shows the conventional image processing apparatus.

Embodiment 1 FIG.
1 is a block diagram showing an image processing apparatus according to Embodiment 1 of the present invention.
In the image processing apparatus shown in FIG. 1, the amount of misalignment between two images is detected while sequentially inputting images. Here, for convenience of explanation, the image A input earlier in time and the time later in time are detected. The following description will be given on the assumption that the amount of deviation from the input image B is detected.
In FIG. 1, when the image signal indicating the image A (or image B) is input, the column direction image dividing unit 1 divides the image A (or image B) into N pieces in the column direction, for example, and outputs the image A (or N small blocks are extracted from the image B), pixel values are integrated for each column in the N small blocks, and the integrated value of the pixel values is temporarily stored in the column direction divided integrated value buffer 2. Implement the process.
The column direction divided integrated value buffer 2 is a recording medium such as a memory for temporarily storing the integrated value of the pixel value in each column in a plurality of small blocks.

The column direction pixel value integrating unit 3 arbitrarily combines a plurality of small blocks, and adds the integrated values of the pixel values in each column in the combination target small block stored in the column direction divided integrated value buffer 2. Then, a process of calculating an integrated value of pixel values in each column within a plurality of deviation amount detection target ranges is performed.
The column direction image dividing unit 1 constitutes a deviation amount detection target range extracting unit and a column direction pixel value integrating unit, and the column direction pixel value integrating unit 3 constitutes a column direction pixel value integrating unit.

In the image processing apparatus of FIG. 1, after the column direction image dividing unit 1 extracts a plurality of small blocks, the pixel values are integrated for each column in each small block, and the column direction pixel value integrating unit 3 is the combination target. Although the example in which the integrated value of the pixel value in each column within the plurality of shift amount detection target ranges is calculated by adding the integrated value of the pixel value in each column in the small block has been shown, the column direction image dividing unit 1 Then, the pixel value integration process may not be performed.
That is, the column direction image dividing unit 1 extracts, for example, a plurality of shift amount detection target ranges that overlap in the column direction from the image A (or image B), and the column direction pixel value integrating unit 3 performs the column direction integration. The pixel values are integrated for each column in the plurality of deviation amount detection target ranges extracted by the image dividing unit 1, and the integrated value of the pixel values in each column in the plurality of deviation amount detection target ranges is calculated. Also good.

The row direction displacement feature amount calculation unit 4 uses the integrated value calculated by the column direction pixel value integration unit 3 for each of the plurality of displacement amount detection target ranges, and calculates the displacement feature amount in the row direction in the shift amount detection target range. Perform the calculation process.
That is, the row-direction displacement feature value calculation unit 4 specifies the integration value of two columns that are separated in the row direction by a predetermined distance from the integration value of each column calculated by the column-direction pixel value integration unit 3. As a displacement feature quantity in the row direction, a process of quantizing the difference value between the integrated values of the two columns is performed. The row direction displacement feature quantity calculation unit 4 constitutes a row direction displacement feature quantity calculation unit.
The row direction displacement feature amount buffer 5 is a recording medium such as a memory for storing the row direction displacement feature amount calculated by the row direction displacement feature amount calculation unit 4.

  In the row direction correlation index value calculation unit 6, the row direction displacement feature amount calculation unit 4 calculates displacement feature amounts in the row direction in a plurality of deviation amount detection target ranges (a plurality of deviation amount detection target ranges extracted from the image B). Then, the displacement feature quantity in the row direction in a plurality of deviation amount detection target ranges and a matching block that is a predetermined deviation amount detection target range in the image A (for example, a deviation amount detection target located at the center of the image A) Since the displacement feature amount in the row direction in the range (image A is input earlier in time than image B), the displacement feature amount in the row direction in the matching block has already been calculated by the row direction displacement feature amount calculation unit 4. To calculate a correlation index value between the stored and the The row direction correlation index value calculation unit 6 constitutes a row direction correlation index value calculation unit.

  The row direction deviation amount detection unit 7 selects a deviation amount detection target range corresponding to the correlation index value indicating the highest correlation among the plurality of correlation index values calculated by the row direction correlation index value calculation unit 6. A process of identifying and detecting a shift amount in the row direction between the shift amount detection target range and the matching block is performed. The row direction deviation amount detection unit 7 constitutes a row direction deviation amount detection unit.

When the image signal indicating the image A (or image B) is input to the row direction image dividing unit 8, for example, the image A (or image B) is divided into N pieces in the row direction, and the image A (or image B) is divided. N small blocks are extracted, pixel values are integrated for each row in the N small blocks, and the integrated value of the pixel values is temporarily stored in the row direction divided integrated value buffer 9. .
The row direction divided integrated value buffer 9 is a recording medium such as a memory that temporarily stores the integrated value of the pixel value in each row in a plurality of small blocks.

The row direction pixel value integration unit 10 arbitrarily combines a plurality of small blocks, and adds the integration values of the pixel values in each row in the combination target small block stored in the row direction divided integration value buffer 9. A process of calculating an integrated value of pixel values in each row within a plurality of deviation amount detection target ranges is performed.
The row direction image dividing unit 8 constitutes a deviation amount detection target range extracting unit and a row direction pixel value integrating unit, and the row direction pixel value integrating unit 10 constitutes a row direction pixel value integrating unit.

In the image processing apparatus of FIG. 1, after the row direction image dividing unit 8 extracts a plurality of small blocks, the pixel values are integrated for each row in each small block, and the row direction pixel value integrating unit 10 is a combination target. Although the example in which the integrated value of the pixel value in each row in the plurality of deviation amount detection target ranges is calculated by adding the integrated value of the pixel value in each row in the small block has been shown, The pixel value integration process may not be performed.
That is, the row direction image dividing unit 8 extracts, for example, a plurality of shift amount detection target ranges that overlap in the row direction from the image A (or image B), and the row direction pixel value integration unit 10 performs the row direction image value integration unit 10. The pixel values may be integrated for each row in the plurality of deviation amount detection target ranges extracted by the image dividing unit 8, and the integrated value of the pixel values in each row in the plurality of deviation amount detection target ranges may be calculated. Good.

The column direction displacement feature amount calculation unit 11 uses the integrated value calculated by the row direction pixel value integration unit 10 for each of the plurality of displacement amount detection target ranges, and calculates the column direction displacement feature amount in the shift amount detection target range. Perform the calculation process.
That is, the column direction displacement feature amount calculation unit 11 specifies the integration value of two rows that are separated in the column direction by a predetermined distance from the integration value of each row calculated by the row direction pixel value integration unit 10. As the displacement feature amount in the column direction, a process of quantizing the difference value between the integrated values of the two rows is performed. Note that the column-direction displacement feature value calculation unit 11 constitutes a column-direction displacement feature value calculation unit.
The column direction displacement feature amount buffer 12 is a recording medium such as a memory for storing the column direction displacement feature amount calculated by the column direction displacement feature amount calculation unit 11.

  In the column direction correlation index value calculation unit 13, the column direction displacement feature amount calculation unit 11 calculates the displacement feature amount in the column direction in a plurality of deviation amount detection target ranges (a plurality of deviation amount detection target ranges extracted from the image B). Then, the displacement feature amount in the column direction in a plurality of deviation amount detection target ranges and a matching block (for example, a deviation amount detection target located at the center of the image A in the image A) Since the displacement feature amount in the column direction in the range (image A is input earlier than image B in time), the displacement feature amount in the column direction in the matching block has already been calculated by the column direction displacement feature amount calculation unit 11. To calculate a correlation index value between the stored and the The column direction correlation index value calculation unit 13 constitutes a column direction correlation index value calculation unit.

  The column direction deviation amount detection unit 14 selects a deviation amount detection target range corresponding to the correlation index value indicating the highest correlation among the plurality of correlation index values calculated by the column direction correlation index value calculation unit 13. The process of identifying and detecting the shift amount in the column direction between the shift amount detection target range and the matching block is performed. The column direction deviation amount detection unit 14 constitutes a column direction deviation amount detection unit.

In FIG. 1, a column direction image dividing unit 1, a column direction pixel value integrating unit 3, a row direction displacement feature amount calculating unit 4, a row direction correlation index value calculating unit 6, and a row direction deviation amount detection, which are components of the image processing apparatus. Each of the unit 7, the row direction image dividing unit 8, the row direction pixel value integrating unit 10, the column direction displacement feature amount calculating unit 11, the column direction correlation index value calculating unit 13, and the column direction deviation amount detecting unit 14 is dedicated hardware. (For example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like) is assumed. However, when the image processing apparatus is configured by a computer, the column-direction image dividing unit 1 and the column Direction pixel value integration unit 3, row direction displacement feature amount calculation unit 4, row direction correlation index value calculation unit 6, row direction deviation amount detection unit 7, row direction image division unit 8, row direction pixel value integration unit 10, column direction Displacement feature amount calculation unit 11, column direction correlation index value calculation The program processing content parts 13 and column shift amount detection unit 14 is described and stored in a memory of the computer, may execute a program that the CPU of the computer is stored in the memory.
FIG. 2 is a flowchart showing the processing contents when the image processing apparatus according to the first embodiment of the present invention detects the shift amount in the row direction.
FIG. 3 is a flowchart showing the processing contents when the image processing apparatus according to the first embodiment of the present invention detects the shift amount in the column direction.

Next, the operation will be described.
In the first embodiment, each time an image is sequentially input and a new image is input, a shift amount between the currently input image and the image input once before is detected.
Here, it is assumed that the image input one time ago is the image A and the currently input image is the image B, and the amount of deviation between the image A and the image B is detected.
Hereinafter, the process at the time when the image B is input will be described, and the process similar to the process for the image B shown below has already been applied to the image A.

In the first embodiment, as shown in FIG. 5, it is assumed that the pixel size in the row direction of the images A and B is WS _h and the pixel size in the column direction is WS _v .
Further, it is assumed that the pixel signal value of the coordinates (i, j) in the image A is D ′ (i, j) and the pixel signal value of the coordinates (i, j) in the image B is D (i, j). However, the coordinate origin is the coordinate of the lower right pixel in the center of images A and B.
Further, it is assumed that the pixel size in the row direction of the shift amount detection target range is SS _h and the pixel size in the column direction is SS _v .

In FIG. 5, the shift amount detection target range (a) of the image B in which the range in the row direction is the upper part of the image and the range in the row direction are the images as the shift amount detection target range used when detecting the shift amount in the row direction. Although the shift amount detection target range (b) of the image B which is the central portion and the shift amount detection target range (c) of the image B whose row direction range is the lower portion of the image are described, (a) (b ) (C) The shift amount detection target range in which the range in the row direction is different from the shift amount detection target range is not shown.
For example, image A, the row direction of the pixel size WS _h 100 of the image B, and the row direction of the pixel size SS _h of shift amount detection target range is 50, shift amount detection target range for each shift one pixel in the row direction When provided, 51 shift amount detection target ranges are provided as the shift amount detection target ranges having the same range in the column direction.
Although FIG. 5 does not exemplify the shift amount detection target range used when detecting the shift amount in the column direction, the shift amount detection target range used when detecting the shift amount in the row direction is rotated by 90 degrees. Will be the same.

First, the detection process of the shift amount in the row direction will be described.
When the image signal indicating the image B is input, the column direction image dividing unit 1 divides the image B into N pieces in the column direction, and extracts N small blocks from the image B (step ST1 in FIG. 2). The pixel values are integrated for each column in the N small blocks, and the integrated value of the pixel values is temporarily stored in the column direction divided integrated value buffer 2 (step ST2).
For example, as shown in FIG. 4, when three types of shift amount detection target ranges including a plurality of shift amount detection target ranges having the same range in the column direction are extracted from the image B (the thick black frame in the figure is , The deviation amount detection target range groups (1) to (3) including a plurality of deviation amount detection target ranges having the same range in the column direction are shown, and the deviation amount detection target range groups (1) to (3) are shown. Includes a shift amount detection target range in which the displacement amount from the center of the image B is y ₀ , y ₁ , y ₂ ), and the image B is divided into four in the column direction, and four small pieces from the image B Blocks (1) to (4) are extracted.
Therefore, in this case, for the four small blocks (1) to (4), the pixel values are integrated for each column in each small block, and the integrated value of the pixel values is temporarily stored in the column direction divided integrated value buffer 2. To store.

The column direction pixel value integrating unit 3 arbitrarily combines a plurality of small blocks, and adds the integrated values of the pixel values in each column in the combination target small block stored in the column direction divided integrated value buffer 2. Then, an integrated value Sh (h, y _n ) of pixel values in each column within a plurality of deviation amount detection target ranges is calculated (step ST3).
As shown in FIG. 4, when four small blocks (1) to (4) are extracted from the image B, the small block (1) and the small block (2) are combined to form a small block (1). The integrated value of the pixel values in each column and the integrated value of the pixel values in each column in the small block (2) are added to the same column to be included in the shift amount detection target range group (1). An integrated value of pixel values in each column within a plurality of deviation amount detection target ranges is calculated.

Further, by combining the small block (2) and the small block (3), the integrated value of the pixel value in each column in the small block (2) and the integrated value of the pixel value in each column in the small block (3) Are added for each same column to calculate the integrated value of the pixel values in each column within the plurality of shift amount detection target ranges included in the shift amount detection target range group (2).
Further, by combining the small block (3) and the small block (4), the integrated value of the pixel value in each column in the small block (3), and the integrated value of the pixel value in each column in the small block (4) Are added for each identical column to calculate an integrated value of the pixel values in each column within the plurality of deviation amount detection target ranges included in the deviation amount detection target range group (3).

Here, an integrated value Sh (h, y _n ) of pixel values in each column within a plurality of deviation amount detection target ranges extracted from the image B is represented by the following expression (1).

Further, among the plurality of shift amount detection target ranges extracted from the image A, for example, the integrated value Sh ′ of pixel values in each column in the matching block that is the shift amount detection target range located at the center of the image A. (H ′) is represented by the following formula (2).

Although FIG. 4 shows an example where y _n = y ₀ , y ₁ , y ₂ , it is possible to increase the detection accuracy of the shift amount by increasing the number of y _n .
However, by setting N number of y _n, when you try to hold the integrated value independently for each y _n, are required integrated value large buffers.
Therefore, in the first embodiment, the column direction divided integrated value buffer 2 is used in a ring shape to reduce the capacity of the column direction divided integrated value buffer 2.
That is, in this first embodiment, since the target image to detect the shift amount is sequentially input, and holds the common overlapping portions of the column direction of the accumulated values determined from the y _n and SS _v, unnecessarily The retained part (front) holding area is reused as a newly required part (rear) holding area.
As shown in FIG. 4, when at half the interval of SS _v as lined y _n, the two integrated values Buffer A over the direction of the image, the B only will turn to use the ring, optionally It is possible to obtain an integrated value in the column direction. In this case, be any number number of y _n, compared with the case of holding the integrated values independently for each y _n, it is possible to construct a column division integration value buffer 2 at 2 times or less buffer capacity Is possible.

The row direction displacement feature amount calculation unit 4 uses the integrated value Sh (h, y _n ) calculated by the column direction pixel value integration unit 3 for each of a plurality of deviation amount detection target ranges extracted from the image B. A displacement characteristic amount P (h, y _n ) in the row direction in the shift amount detection target range is calculated (step ST4).
That is, the row direction displacement feature amount calculation unit 4 includes two columns (in the row direction that are separated by a predetermined distance from the integration value Sh (h, y _n ) calculated by the column direction pixel value integration unit 3 ( most simply identifies the integrated value of each other two columns) that next to the right and left, the displacement characteristic of the row direction P (h, as y _n), for quantizing the difference value of the integrated values of the two columns.

Here, FIG. 6 is an explanatory diagram showing a concept of calculating the displacement feature amount in the row direction by the row direction displacement feature amount calculation unit 4.
Displacement characteristic amount P in the row direction in the shift amount detection target range to be extracted from the image B (h, y _n) is expressed by the following equation (3).

Also, among the plurality of deviation amount detection target ranges extracted from the image A, for example, the displacement feature amount P ′ (h ′) in the row direction in the matching block that is the deviation amount detection target range located at the center of the image A. ) Is represented by the following formula (4).

In equations (3) and (4), fix (A) represents the truncation of the decimal point in the 0 direction of A.
Further, a step width of T _h row direction quantization, dist _h is the row direction distance between the integrated value when calculating the displacement characteristic amount.
Although the number of quantization steps and the upper and lower limits are not specified, any number can be set.

When the displacement feature amounts P (h, y _n ) and P ′ (h ′) in the row direction are expressed by the equations (3) and (4), as shown in FIG. Directional displacement feature amounts P (h, y _n ) and P ′ (h ′) are represented by “−1”, “0”, or “1”.
The row-direction displacement feature values P (h, y _n ) and P ′ (h ′) calculated by the row-direction displacement feature value calculation unit 4 can be correlated with the row-direction correlation index value calculation unit 6 in the subsequent stage. Therefore, it is stored in the row direction displacement feature amount buffer 5.

The row direction correlation index value calculation unit 6 performs the row direction displacement feature amount P (h, y _n ) in a plurality of deviation amount detection target ranges extracted from the image B and the row direction in the matching block extracted from the image A. The correlation index value Ih (k, y _n ) with the displacement feature amount P ′ (h ′) is calculated (step ST5).
That is, the row direction correlation index value calculation unit 6 uses the displacement feature amount P (h, y _n ) in the row direction in the plurality of deviation amount detection target ranges extracted from the image B and the matching block extracted from the image A. By performing a correlation calculation with the displacement feature amount P ′ (h ′) in the row direction, the displacement feature amount in the row direction in a plurality of shift amount detection target ranges extracted from the image B and the image A extracted from the image A correlation index value Ih (k, y _n) in the row direction that indicates displacement characteristic of the row direction in the matching blocks and are similar much calculated.
In this case, the image A, between B, displacement y _n and by repeating the correlation calculation while changing the shift amount k in the row direction, the row direction according to a shift amount k of each displacement y _n and row directions correlation index value Ih (k, _{y n)} is calculated.

なお、行方向相関指標値算出部６による相関演算は差分絶対値和演算に限るものではなく、相関指標となる値を算出できるのであれば、どのような演算を用いてもよい。 Here, FIG. 7 is an explanatory diagram showing the concept of calculating the correlation index value in the row direction by the row direction correlation index value calculation unit 6.
FIG. 7 shows an example in which the difference absolute value sum calculation is performed as the correlation calculation by the row direction correlation index value calculation unit 6.
If the differential absolute value sum computation, the row direction of the correlation index value Ih (k, y _n) is expressed by the following equation (5).

The correlation calculation performed by the row direction correlation index value calculation unit 6 is not limited to the absolute difference value sum calculation, and any calculation may be used as long as a value serving as a correlation index can be calculated.

When the row direction correlation index value calculation unit 6 calculates the row direction correlation index value Ih (k, y _n ), the row direction deviation amount detection unit 7 receives the row direction correlation index value from the row direction correlation index value calculation unit 6. Ih (k, y _n ) and a shift amount k in the row direction corresponding to the correlation index value Ih (k, y _n ) are received.
The row direction deviation amount detection unit 7 then indicates a correlation index indicating that the correlation is highest among the plurality of correlation index values Ih (k, y _n ) calculated by the row direction correlation index value calculation unit 6. A shift amount detection target range corresponding to a value (if the correlation calculation by the row direction correlation index value calculation unit 6 is a sum of absolute differences calculation, the minimum correlation index value) is specified, and the shift amount detection target range and the matching block detecting the row direction of the shift amount _{k min} with (step ST6).
That is, the row direction deviation amount detection unit 7 corresponds to the minimum row direction correlation index value Ih (k, y _n ) if the correlation calculation performed by the row direction correlation index value calculation unit 6 is a difference absolute value sum calculation. The shift amount _kmin in the row direction is output as the shift amount in the row direction between the images A and B.

Thus, although the deviation amount in the row direction between the images A and B is detected, unlike the conventional example, the amount of displacement y _n, can be compared with the position which a significant feature point is moving in a column direction Therefore, it is possible to improve the detection accuracy of the shift amount in the row direction without giving the detection accuracy of the shift amount in the column direction.

Here, FIG. 8 is an explanatory diagram illustrating an example in which the displacement amounts are y ₀ , y ₁ , and y ₂ when the shift amount in the row direction between the images A and B is detected.
In FIG. 8, when detecting the amount of deviation in the row direction, the only significant feature point (the letter “A” in the figure) is significantly displaced, and the deviation amount detection target range (b) of the displacement y ₁ Have gone out of the.
For this reason, even if the correlation index value Ih (k, y ₁ ) in the row direction is calculated between the displacement amount detection target range (b) of the displacement amount y ₁ and the matching block (d), the displacement amount in the row direction. Is impossible to detect.
However, in the shift amount detection target range of the amount of displacement y ₀ (a), since the only significant feature point exists, shift amount detection target range of the amount of displacement y ₀ (a) and the matching block (d ) To calculate the row direction correlation index value Ih (k, y ₀ ), it is possible to accurately detect the amount of deviation in the row direction.

Next, a process for detecting a shift amount in the row direction will be described.
When the image signal indicating the image B is input, the row direction image dividing unit 8 divides the image B into N pieces in the row direction, and extracts N small blocks from the image B (step ST11 in FIG. 3). The pixel values are integrated for each row in the N small blocks, and the integrated value of the pixel values is temporarily stored in the row direction divided integrated value buffer 9 (step ST12).
Next, the row direction image dividing unit 8 arbitrarily combines a plurality of small blocks, and adds the integrated values of the pixel values in each row in the combination target small block stored in the row direction divided integrated value buffer 9. Thus, the integrated value Sv (x _n , v) of the pixel values in each row within the plurality of deviation amount detection target ranges is calculated (step ST13).
Since the processing content of the row direction image dividing unit 8 is the same as the processing content of the column direction image dividing unit 1 if the image B is rotated by 90 degrees, detailed description thereof is omitted.

Here, an integrated value Sv (x _n , v) of pixel values in each row within a plurality of deviation amount detection target ranges extracted from the image B is expressed by the following equation (6).

Further, among the plurality of deviation amount detection target ranges extracted from the image A, for example, the integrated value Sv ′ of the pixel values in each column in the matching block that is the deviation amount detection target range located at the center of the image A. (V ′) is expressed by the following formula (7).

FIG. 4 shows an example in which y _n = y ₀ , y ₁ , y ₂ , but if image B is rotated by 90 degrees, an example in which x _n = x ₀ , x ₁ , x ₂ is shown. Will be.
Also when detecting the shift amount in the column direction, it is possible to increase the detection accuracy of the shift amount by increasing the number of _xn .
As in the case of FIG. 4, when at half the interval of SS _h as lined x _n, the two integrated values Buffer A from the left side of the image, the B only will turn to use a ring shape, It is possible to obtain an integrated value in a desired row direction. In this case, be any number is the number of x _n, in comparison with the case of holding the integrated values independently for each x _n, it is possible to configure the row direction divided accumulated value buffer 9 at twice or less buffer capacity Is possible.

The column direction displacement feature amount calculation unit 11 uses the integrated value Sv (x _n , v) calculated by the row direction pixel value integration unit 10 for each of a plurality of deviation amount detection target ranges extracted from the image B, A displacement feature amount Q (x _n , v) in the column direction in the shift amount detection target range is calculated (step ST14).
That is, the column-direction displacement feature amount calculation unit 11 includes two rows (in the column direction) that are separated by a predetermined distance from the integration values Sv (x _n , v) calculated by the row-direction pixel value integration unit 10. In the simplest case, the integrated value of two rows adjacent in the vertical direction is specified, and the difference value between the integrated values of the two rows is quantized as the displacement feature quantity Q (x _n , v) in the column direction.

The displacement feature quantity Q (x _n , v) in the column direction in the deviation detection target range extracted from the image B is expressed by the following equation (8).

Further, among the plurality of deviation amount detection target ranges extracted from the image A, for example, the displacement feature quantity Q ′ (v ′) in the column direction in the matching block that is the deviation amount detection target range located at the center of the image A. ) Is represented by the following equation (9).

In equations (8) and (9), fix (A) represents the truncation of the decimal point in the 0 direction of A.
Further, T _v is the step width in the column direction quantization, dist _v is the column direction distance between the integrated value when calculating the displacement characteristic amount.
Although the number of quantization steps and the upper and lower limits are not specified, any number can be set.

When the displacement feature quantities Q (x _n , v) and Q ′ (v ′) in the column direction are expressed by the equations (8) and (9), the displacement feature quantities Q (( x _n , v), Q ′ (v ′) is “−1”, “0” or “1”, similarly to the displacement feature amounts P (h, y _n ), P ′ (h ′) in the row direction. It is represented by
The column-direction displacement feature values Q (x _n , v) and Q ′ (v ′) calculated by the column-direction displacement feature value calculation unit 11 can be correlated by the column-direction correlation index value calculation unit 13 in the subsequent stage. Therefore, it is stored in the column direction displacement feature amount buffer 12.

The column direction correlation index value calculation unit 13 uses the column direction displacement feature amount Q (x _n , v) in the plurality of deviation amount detection target ranges extracted from the image B and the column direction in the matching block extracted from the image A. The correlation index value Iv (x _n , l) with the displacement feature amount Q ′ (v ′) is calculated (step ST15).
That is, the column direction correlation index value calculation unit 13 uses the column direction displacement feature amount Q (x _n , v) in the plurality of shift amount detection target ranges extracted from the image B and the matching block extracted from the image A. By performing a correlation operation with the displacement feature amount Q ′ (v ′) in the column direction, the displacement feature amount in the column direction in the plurality of shift amount detection target ranges extracted from the image B and the image A extracted from the image A A correlation index value Iv (x _n , l) in the column direction indicating how similar the displacement feature quantity in the column direction in the matching block is calculated.
At this time, by repeating the correlation calculation while changing the displacement amount _xn and the displacement amount l in the column direction between the images A and B, the column direction related to the displacement amount _xn and the displacement amount l in the column direction is repeated. A correlation index value Iv (x _n , l) is calculated.

なお、列方向相関指標値算出部１３による相関演算は差分絶対値和演算に限るものではなく、相関指標となる値を算出できるのであれば、どのような演算を用いてもよい。 For example, when the correlation calculation by the column direction correlation index value calculation unit 13 is a difference absolute value sum calculation, the column direction correlation index value Iv (x _n , l) is expressed by the following equation (10).

Note that the correlation calculation performed by the column direction correlation index value calculation unit 13 is not limited to the absolute difference sum calculation, and any calculation may be used as long as a value serving as a correlation index can be calculated.

When the column direction correlation index value calculation unit 13 calculates the column direction correlation index value Iv (x _n , l), the column direction deviation amount detection unit 14 receives the column direction correlation index value from the column direction correlation index value calculation unit 13. Iv (x _n , l) and a column-direction shift amount l corresponding to the correlation index value Iv (x _n , l) are received.
The column direction deviation amount detection unit 14 then selects a correlation index value (column direction correlation index value) indicating that the correlation is highest among the plurality of correlation index values calculated by the column direction correlation index value calculation unit 13. If the correlation calculation by the calculation unit 13 is the difference absolute value sum calculation, a shift amount detection target range corresponding to the minimum correlation index value) is specified, and the shift amount in the column direction between the shift detection target range and the matching block is specified. l _min is detected (step ST16).
In other words, the column direction deviation amount detection unit 14 is the column corresponding to the smallest column direction correlation index value Iv (x _n , l) if the correlation calculation by the column direction deviation amount detection unit 14 is a sum of absolute differences calculation. The displacement amount l _min in the direction is output as the displacement amount in the column direction between the images A and B.

As a result, the shift amount in the column direction between the image A and the image B is detected, but unlike the conventional example, the displacement amount _xn can be compared with the position where the significant feature point moves in the row direction. Therefore, it is possible to improve the detection accuracy of the shift amount in the column direction without giving the detection accuracy of the shift amount in the row direction.

  As apparent from the above, according to the first embodiment, the column-direction pixel value integrating unit 3 that calculates the integrated value of the pixel values in each column within the plurality of shift amount detection target ranges extracted from the image B; For each of a plurality of deviation amount detection target ranges extracted from the image B, the row direction displacement feature amount in the deviation amount detection target range is calculated using the integrated value calculated by the column direction pixel value integration unit 3. The row direction displacement feature amount calculation unit 4 and the row direction displacement feature amount in the plurality of deviation amount detection target ranges calculated by the row direction displacement feature amount calculation unit 4 and the row direction displacement feature amount in the matching block in the image A And a row direction correlation index value calculation unit 6 that calculates a correlation index value between the row direction correlation index value calculation unit 6 and the row direction deviation index detection unit 7 among the plurality of correlation index values calculated by the row direction correlation index value calculation unit 6 Shows the highest correlation. Since the shift detection target range corresponding to the correlation index value is specified and the shift amount in the row direction between the shift detection target range and the matching block is detected, a large image shift occurs in the row direction. In addition, there is an effect that the shift amount between the image A and the image B can be detected with high accuracy.

  Also, according to the first embodiment, the row direction pixel value integrating unit 10 that calculates the integrated value of the pixel values in each column within the plurality of shift amount detection target ranges extracted from the image B, and the image B is extracted. For each of the plurality of deviation amount detection target ranges, the column direction displacement feature amount is calculated using the integrated value calculated by the row direction pixel value integration unit 10 in the column direction in the deviation amount detection target range. Correlation between the column direction displacement feature amount in the plurality of deviation amount detection target ranges calculated by the calculation unit 11 and the column direction displacement feature amount calculation unit 11 and the column direction displacement feature amount in the matching block in the image A A column direction correlation index value calculation unit 13 for calculating an index value is provided, and the column direction deviation amount detection unit 14 has the highest correlation among the plurality of correlation index values calculated by the column direction correlation index value calculation unit 13. Indicates high Since the misalignment detection target range corresponding to the function index value is specified and the misalignment amount in the column direction between the misalignment detection target range and the matching block is detected, a large image misalignment occurs in the column direction. In addition, there is an effect that the shift amount between the image A and the image B can be detected with high accuracy.

In accordance with this embodiment 1, when a half of the spacing of the SS _v as lined y _n, has two integrated values Buffer A over the direction of the image, a B turn used to ring Examples showed (see Figure 4), as shown in FIG. 9, when a quarter of the distance between the SS _v as lined y _n, the upper direction of the image four integrated value buffer a, B , C, and D may be used in a ring shape to reduce the capacity of the column direction divided integrated value buffer 2.
In this case, be any number number of y _n, compared with the case of holding the integrated values independently for each y _n, it is possible to construct a column division integration value buffer 2 at four times less buffer capacity Is possible.
The same can be said for the row-direction divided integrated value buffer 9 only by rotating the image by 90 degrees.

Incidentally, _{y n} (or _{x n)} is not limited to those arranged at SS _v (or SS _h) 1/2 interval or 1/4 of the interval, SS _v commitment (or SS _h) If they are arranged at intervals of a number, the capacity of the column direction divided integrated value buffer 2 (or the row direction divided integrated value buffer 9) can be reduced.

Embodiment 2. FIG.
FIG. 10 is a block diagram showing an image processing apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG.
The row direction deviation amount detection pre-processing unit 15 outputs the correlation index value between the row direction displacement feature amounts calculated by the row direction correlation index value calculation unit 6 to the row direction deviation amount detection post-processing unit 16 and The correlation index value between the displacement feature quantities in the column direction is calculated from the correlation index value between the displacement feature quantities in the direction, and the correlation index value between the displacement feature quantities in the column direction is sent to the post-direction shift amount detection post-processing unit 18. Perform the output process.
In the second embodiment, the row direction correlation index value calculation unit is configured by the row direction correlation index value calculation unit 6 and the row direction deviation amount detection pre-processing unit 15.

  The row direction deviation amount detection post-processing unit 16 and the correlation index value between the row direction displacement feature amounts output from the row direction deviation amount detection preprocessing unit 15 and the row direction deviation amount detection preprocessing unit 17 are output. The correlation index values between the displacement feature quantities in the direction are combined (for example, addition of two correlation index values, weighted addition, multiplication, etc.), and the fact that the correlation is highest among a plurality of combined correlation index values. A process for identifying a shift amount detection target range corresponding to the correlation index value shown and detecting a shift amount in the row direction between the shift amount detection target range and the matching block is performed. It should be noted that the row direction deviation amount detection post-processing unit 16 constitutes a row direction deviation amount detection means.

The column direction deviation amount detection pre-processing unit 17 outputs the correlation index value between the column direction displacement feature amounts calculated by the column direction correlation index value calculation unit 13 to the column direction deviation amount detection post-processing unit 18, and The correlation index value between the displacement feature quantities in the row direction is calculated from the correlation index value between the displacement feature quantities in the direction, and the correlation index value between the displacement feature quantities in the row direction is sent to the processing unit 16 after detecting the displacement amount in the row direction. Perform the output process.
In the second embodiment, the column direction correlation index value calculation unit is configured by the column direction correlation index value calculation unit 13 and the column direction deviation amount detection pre-processing unit 17.

  The column direction deviation amount detection post-processing unit 18 outputs the correlation index value between the column direction displacement feature amounts output from the column direction deviation amount detection pre-processing unit 17 and the column direction deviation amount detection pre-processing unit 15. The correlation index values between the displacement feature quantities in the direction are combined (for example, addition of two correlation index values, weighted addition, multiplication, etc.), and the fact that the correlation is highest among a plurality of combined correlation index values. A process for specifying a shift amount detection target range corresponding to the correlation index value shown and detecting a shift amount in the column direction between the shift amount detection target range and the matching block is performed. The column direction deviation detection post-processing unit 18 constitutes a column direction deviation detection means.

  In FIG. 10, the column direction image dividing unit 1, the column direction pixel value integrating unit 3, the row direction displacement feature amount calculating unit 4, the row direction correlation index value calculating unit 6, and the row direction deviation amount detection, which are components of the image processing apparatus. Pre-processing unit 15, row-direction shift amount detection post-processing unit 16, row-direction image dividing unit 8, row-direction pixel value integration unit 10, column-direction displacement feature amount calculation unit 11, column-direction correlation index value calculation unit 13, column direction It is assumed that each of the misalignment detection pre-processing unit 17 and the column direction misalignment detection post-processing unit 18 is configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like). However, when the image processing apparatus is configured by a computer, the column direction image dividing unit 1, the column direction pixel value integrating unit 3, the row direction displacement feature amount calculating unit 4, the row direction correlation index value calculating unit 6, and the row Direction misalignment detection preprocessing unit 15, row direction Post-shift detection post-processing unit 16, row-direction image dividing unit 8, row-direction pixel value integration unit 10, column-direction displacement feature amount calculation unit 11, column-direction correlation index value calculation unit 13, column-direction shift amount detection pre-processing unit 17 Alternatively, a program in which the processing contents of the post-column-direction-shift detection post-processing unit 18 are described may be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.

Next, the operation will be described.
However, instead of the row direction deviation amount detection unit 7, a row direction deviation amount detection preprocessing unit 15 and a row direction deviation amount detection post-processing unit 16 are provided, and instead of the column direction deviation amount detection unit 14, the column direction deviation amount detection unit 14 is provided. Except for the point that the amount detection pre-processing unit 17 and the column direction deviation amount detection post-processing unit 18 are provided, this is the same as in the first embodiment, and here, the row direction deviation amount detection pre-processing unit 15 and the row Only the processing contents of the processing unit 16 for detecting the direction deviation amount, the pre-processing unit 17 for detecting the column direction deviation amount, and the processing unit 18 for detecting the column direction deviation amount will be described.

In the same way as in the first embodiment, the row direction deviation amount detection pre-processing unit 15 calculates the row direction correlation when the row direction correlation index value calculation unit 6 calculates the row direction correlation index value Ih (k, y _n ). correlation index value Ih (k, _{y n)} in the row direction from the index value calculation section 6 and receives the correlation index value Ih (k, _{y n)} shift amount k in the row direction corresponding to the.
Then, the row direction shift amount detection preprocessing unit 15, for each displacement amount y _n, a plurality of correlation index value Ih (k, y _n) in the most correlation index value correlation shows that high (line If the correlation calculation by the direction correlation index value calculation unit 6 is the difference absolute value sum calculation, the shift amount detection target range corresponding to the minimum correlation index value) is specified, and the shift amount detection target range and the row of the matching block to detect the direction of the deviation amount _ky min _{(y n).}
That is, if the correlation calculation performed by the row direction correlation index value calculation unit 6 is the difference absolute value sum calculation, the row direction deviation amount detection pre-processing unit 15 performs the minimum row direction correlation index value Ih for each displacement y _{n. min} (k, _{y n)} detects the, detects the correlation index value _Ih min (k, _{y n)} of the corresponding row direction shift amount _ky min _{(y n),} the minimum in the row direction of the correlation index value Ih _min (k, y _n ) and the amount of shift in the row direction ky _min (y _n ) are output to the post-shift amount detection post-processing unit 16.

The row direction shift amount detection preprocessing section 15 calculates displacement _{y n} in the row direction of the correlation index value _Ih min (k, _{y n)} from the new column of the correlation index value _J v a _{(y n)} The column direction correlation index value J _v (y _n ) is output to the column direction deviation amount detection post-processing unit 18.
The column-direction correlation index value J _v (y _n ) newly calculated by the row-direction shift amount detection pre-processing unit 15 is obtained by regarding the displacement amount y _n as the column-direction movement amount, and thus with respect to the column-direction movement amount. It can be used as a new correlation index value.

Here, the row direction shift amount detection preprocessing section 15, calculated for each displacement amount _{y n,} the correlation index value _Ih min (k, _{y n)} in the row direction correlation index value in the column direction from _J v a _{(y n)} However, the correlation index value Ih _min (k, y _n ) in the row direction may be used as the correlation index value J _v (y _n ) in the column direction as it is.
Further, the superiority or inferiority of the correlation index value Ih _min (k, y _n ) of each displacement y _n is determined (for example, the smaller the correlation index value is, the better (indicating that it is similar)). ), The smaller the correlation index value Ih _min (k, y _n ), the smaller the value may be given as the correlation index value J _v (y _n ) in the column direction.
However, this is only an example, the correlation index value J _{v (y n),} as long obtained value capable of performing the comparison between the displacement amount y _n, can also be used in any operation.

Similar to the first embodiment, the column direction deviation amount detection pre-processing unit 17 calculates the column direction correlation index value Iv (x _n , l) when the column direction correlation index value calculation unit 13 calculates the column direction correlation index value Iv (x _n , l). The index value calculation unit 13 receives a column-direction correlation index value Iv (x _n , l) and a row-direction shift amount l corresponding to the correlation index value Iv (x _n , l).
The column-direction shift amount detection pre-processing unit 17 then selects a correlation index value (column) indicating that the correlation is the highest among the plurality of correlation index values Iv (x _n , l) for each displacement amount x _n. If the correlation calculation by the direction correlation index value calculation unit 13 is a difference absolute value sum calculation, a shift amount detection target range corresponding to the minimum correlation index value) is specified, and the shift amount detection target range and a column of matching blocks A direction shift amount lx _min (x _n ) is detected.
That is, the column-direction shift amount detection pre-processing unit 17 determines the minimum column-direction correlation index value Iv for each displacement amount x _n if the correlation calculation by the column-direction correlation index value calculation unit 13 is a difference absolute value sum calculation. _Min (x _n , l) is detected, and a column-direction shift amount lx _min (x _n ) corresponding to the correlation index value Iv _min (x _n , l) is detected, and the minimum correlation index value in the column direction is detected. The Iv _min (x _n , l) and the column-direction shift amount lx _min (x _n ) are output to the column-direction shift amount detection post-processing unit 18.

Further, the column-row-direction shift amount detection preprocessing unit 17 newly calculates a row-direction correlation index value J _h (x _n ) from the column-direction correlation index value Iv _min (x _n , l) of the displacement amount x _n. Then, the correlation index value J _h (x _n ) in the row direction is output to the post-shift amount detection post-processing unit 16.
The row direction correlation index value J _h (x _n ) newly calculated by the column direction deviation amount detection pre-processing unit 17 is obtained by regarding the displacement amount x _n as the movement amount in the row direction. It can be used as a new correlation index value.

Here, the column-row-direction shift amount detection pre-processing unit 17 calculates the row-direction correlation index value J _h (x _n ) from the column-direction correlation index value Iv _min (x _n , l) for each displacement amount x _n. Although calculated, the correlation index value Iv _min (x _n , l) in the column direction may be used as it is as the correlation index value J _h (x _n ) in the row direction.
Further, the superiority or inferiority of the correlation index value Iv _min (x _n , l) of each displacement amount x _n is determined (for example, it is determined that the smaller the correlation index value, the better (shows that it is similar)). ), The smaller the correlation index value Iv _min (x _n , l), the smaller the correlation index value J _h (x _n ) in the column direction may be given.
However, this is only an example, and any calculation can be used as long as the correlation index value J _h (x _n ) provides a value that enables comparison between the displacement amounts x _n .

Row direction shift amount detection post-processing unit 16, the row direction shift amount detection preprocessing unit 15, for each displacement amount _{y n,} the correlation index value _Ih min (k, _{y n)} in the row direction displacement amount ky the row direction _min (y _n ) is received, and the row direction correlation index value J _h (x _n ) is received from the column row direction deviation amount detection pre-processing unit 17.
Then, the post-direction shift amount detection post-processing unit 16 calculates a value fx _n (ky _min (y _n )) obtained by rounding the shift amount ky _min (y _n ) in the row direction to the nearest displacement amount x _n . The correlation index values Ih _min (k, y _n ) in the row direction and J _h (fx _n (ky _min (y _n ))) are added.
_{Ih = Ih min (k, y} n) + J h (fx n (ky min (y n)))

The post-direction shift amount detection post-processing unit 16 is a correlation index value indicating that the correlation is highest among the plurality of added correlation index values Ih (the row direction correlation index value calculating unit 6 and the column direction correlation index). If the correlation calculation by the value calculation unit 13 is a sum of absolute differences calculation, a shift amount detection target range corresponding to the minimum correlation index value) is specified, and a shift in the row direction between the shift detection target range and the matching block is specified. The quantity ky _min (y _n ) is detected.
Then, the row direction deviation amount detection post-processing unit 16 outputs the row direction deviation amount ky _min (y _n ) as the row direction deviation amount between the image A and the image B.

Here, the post-direction shift amount detection post-processing unit 16 adds the correlation index values Ih _min (k, y _n ) and J _h (fx _n (ky _min (y _n ))) in the row direction. This is merely an example, and, for example, the correlation index value Ih _min (k, y _n ) in the row direction and J _h (fx _n (ky _min (y _n ))) may be weighted and added. It may be.

The column direction deviation amount detection post-processing unit 18 receives the column direction correlation index value Iv _min (x _n , l) and the column direction deviation amount lx for each displacement amount x _n from the column direction deviation amount detection preprocessing unit 17. _min (x _n ) is received, and the column direction correlation index value J _v (y _n ) is received from the row direction shift amount detection pre-processing unit 15.
Then, the column shift amount detection post-processing unit 18, a column shift amount _lx min _{(x n),} calculates the closest displacement _{y n} the rounded value _{fy n (lx min (x n} )), The correlation index value Iv _min (x _n , l) in the column direction and J _v (fy _n (lx _min (x _n ))) are added.
_{Iv = Iv min (x n,} l) + J v (fy n (lx min (x n)))

The column direction deviation detection post-processing unit 18 has a correlation index value (row direction correlation index value calculation unit 6 and column direction correlation index) indicating that the correlation is highest among the plurality of correlation index values Iv after addition. If the correlation calculation performed by the value calculation unit 13 is a difference absolute value sum calculation, a shift amount detection target range corresponding to the minimum correlation index value) is specified, and the shift in the column direction between the shift amount detection target range and the matching block is specified. The quantity lx _min (x _n ) is detected.
Then, the column-direction shift amount detection post-processing unit 18 outputs the column-direction shift amount lx _min (x _n ) as the column-direction shift amount between the images A and B.

Here, the column-direction shift amount detection post-processing unit 18 adds the column-direction correlation index values Iv _min (x _n , l) and J _v (fy _n (lx _min (x _n ))). This is merely an example, and for example, the column-direction correlation index value Iv _min (x _n , l) and J _v (fy _n (lx _min (x _n ))) may be weighted and added. It may be.

  As apparent from the above, according to the second embodiment, when the row direction deviation amount detection post-processing unit 16 detects the row direction deviation amount between the image A and the image B, the row direction correlation index value is calculated. The amount of misalignment in the row direction is detected using not only the correlation index value in the row direction calculated by the unit 6 but also the correlation index value in the row direction newly calculated from the correlation index value in the column direction. Therefore, there is an effect that it is possible to further improve the accuracy of detecting the amount of shift in the row direction between the image A and the image B.

Further, according to the second embodiment, when the column direction deviation amount detection post-processing unit 18 detects the column direction deviation amount between the image A and the image B, the column direction correlation index value calculation unit 13 calculates the amount. In addition to the column-direction correlation index value, the column-direction correlation index value newly calculated from the row-direction correlation index value is used to detect the shift amount in the column direction. There is an effect that the detection accuracy of the shift amount in the column direction between A and the image B can be further increased.
It should be noted that the embodiments can be appropriately combined, changed, omitted, etc. within the scope of the present invention.

  1 column direction image dividing unit (shift amount detection target range extracting unit, column direction pixel value integrating unit), 2 column direction divided integrated value buffer, 3 column direction pixel value integrating unit (column direction pixel value integrating unit), 4 row direction Displacement feature amount calculation unit (row direction displacement feature amount calculation unit), 5 row direction displacement feature amount buffer, 6 row direction correlation index value calculation unit (row direction correlation index value calculation unit), 7 row direction deviation amount detection unit (row Direction deviation amount detection means), 8 row direction image division unit (shift amount detection target range extraction means, row direction pixel value integration means), 9 row direction division integration value buffer, 10 row direction pixel value integration unit (row direction pixel value) Integrating means), 11 column direction displacement feature value calculating unit (column direction displacement feature value calculating unit), 12 column direction displacement feature value buffer, 13 column direction correlation index value calculating unit (column direction correlation index value calculating unit), 14 columns Direction deviation detection unit (column direction Displacement amount detection means), 15 row direction displacement amount detection pre-processing section (row direction correlation index value calculation means), 16 row direction displacement amount detection post-processing section (row direction displacement amount detection means), 17 before row direction displacement amount detection Processing unit (column direction correlation index value calculation means), 18 column direction deviation amount detection post-processing unit (column direction deviation amount detection means), 101 column direction integration value calculation unit, 102 column direction integration value buffer, 103 row direction displacement feature Amount calculation unit, 104 row direction displacement feature amount buffer, 105 row direction correlation index value calculation unit, 106 row direction deviation amount detection unit, 107 row direction integration value calculation unit, 108 row direction integration value buffer, 109 column direction displacement feature amount Calculation unit, 110 column direction displacement feature amount buffer, 111 column direction correlation index value calculation unit, 112 column direction deviation amount detection unit.

Claims (9)

A deviation amount detection target range extraction unit that extracts a plurality of deviation amount detection target ranges from the input image, and integrates pixel values for each direction in the plurality of deviation amount detection target ranges to calculate an integrated value. A displacement feature that calculates a displacement feature amount in a different direction that is different from the one direction in the displacement amount detection target range by using the integrated value for each of the plurality of displacement amount detection target ranges and the pixel value integration means. Correlation is calculated by calculating a correlation between the displacement feature amount between the amount calculation means, a matching block that is a predetermined shift amount detection target range in the image input prior to the image, and the plurality of shift amount detection target ranges. Correlation index value calculating means for calculating an index value, and specifying a shift amount detection target range corresponding to a correlation index value indicating the highest correlation among the plurality of correlation index values, and the shift amount detection target Range and above matchon And a displacement amount detecting means for detecting a displacement amount of the another direction with the block,
The pixel value integration means integrates pixel values for each direction in a small block whose range in one direction is narrower than the deviation amount detection target range, stores the integrated value of the pixel values in a buffer, and An image processing apparatus that calculates an integrated value of pixel values in the one direction within the plurality of shift amount detection target ranges by arbitrarily combining small blocks and adding the integrated values .   The displacement feature amount calculating means specifies two integrated values from the one direction separated by a predetermined distance from the one direction, out of the integrated values in the one direction calculated by the pixel value integrating means. The image processing apparatus according to claim 1, wherein a difference value between two integrated values from the one direction is quantized as the displacement feature amount in the other direction.   The image processing apparatus according to claim 1, wherein the one direction is a column direction and the other direction is a row direction.   The image processing apparatus according to claim 1, wherein the one direction is a row direction and the other direction is a column direction. The displacement characteristic amount value, the image processing apparatus according to claim 1 or claim 2 wherein wherein a is a value of either -1, 0 or 1. A deviation amount detection target range extracting step for extracting a plurality of deviation amount detection target ranges from the input image, and summing pixel values for each direction in the plurality of deviation amount detection target ranges to calculate an integrated value. A displacement feature that calculates a displacement feature amount in a different direction that is different from the one direction in the displacement amount detection target range using the integrated value for each of the plurality of displacement amount detection target ranges in the pixel value integration step. Calculating a correlation between the displacement feature amount between the amount calculation step and a matching block that is a predetermined shift amount detection target range in the image input before the image and the plurality of shift amount detection target ranges A correlation index value calculating step for calculating an index value, and specifying a deviation amount detection target range corresponding to a correlation index value indicating the highest correlation among the plurality of correlation index values, and the deviation amount detection target Model And a shift amount detection step of detecting a shift amount of the specific direction of the matching block,
The pixel value integration step integrates pixel values for each direction in a small block whose range in the one direction is narrower than the shift amount detection target range, stores the integrated value of the pixel values in a buffer, and An image processing method comprising: calculating an integrated value of pixel values in the one direction within the plurality of shift amount detection target ranges by arbitrarily combining small blocks and adding the integrated values . The image processing method according to claim 6 , wherein the one direction is a column direction and the other direction is a row direction. 7. The image processing method according to claim 6 , wherein the one direction is a row direction and the other direction is a column direction. The image processing method according to claim 6 , wherein the value of the displacement feature value is any one of −1, 0, and 1.

Images