JP2013175023A - Image positioning device and image positioning method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a highly accurate registration result regardless of the presence/absence of an edge.SOLUTION: The image positioning device includes: a featured value extraction part 2 for extracting a plurality of featured values whose types are different from an image A and an image B; a block division part 3 for dividing the image A and the image B into a plurality of blocks; and a position deviation value calculation part 4 for calculating a position deviation value between blocks having correspondence among the blocks of the image A and the blocks of the image B divided by the block division part 3 by using each of the featured values extracted by the featured value extraction part 2. A registration arithmetic part 5 performs the positioning of the image A and the image B by using the position deviation value between blocks calculated by using each featured value by the position deviation value calculation part 4.

Description

  The present invention relates to an image alignment apparatus and an image alignment method for aligning two images.

In an image alignment apparatus that aligns two images, in general, a block matching process is performed using a grayscale image of the two images to calculate a positional deviation amount between the two images. A method of performing registration (image alignment) using the amount of displacement is employed.
In recent years, a method has been proposed in which the amount of positional deviation between two images is calculated by performing matching processing using edge information of the images.

Japanese Patent Application Laid-Open No. 2004-228561 describes a method of registering by extracting an edge of an image using a Laplacian filter and performing template matching using the edge of the image.
Patent Document 2 below discloses a method of dividing an image into small regions, obtaining dispersion and edge amounts of the small regions, performing matching processing using the dispersion and edge amounts, and performing registration. Have been described.

Japanese Patent Laid-Open No. 08-161474 (paragraph numbers [0043] to [0052], figure) JP 2009-295062 A (paragraph number [0038])

  Since the conventional image alignment apparatus is configured as described above, when performing block matching processing using a grayscale image, a good matching result can be obtained in a block having many edges. In a block with few edges, a good matching result cannot be obtained. For this reason, there is a problem that the accuracy of registration deteriorates in a block with few edges.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide an image alignment apparatus and an image alignment method capable of obtaining a highly accurate registration result regardless of the presence or absence of an edge. And

  The image alignment apparatus according to the present invention uses a feature amount extraction unit that extracts a plurality of different types of feature amounts from the first image and the second image, and uses each feature amount extracted by the feature amount extraction unit. And a misregistration amount calculating means for calculating a misregistration amount between the first image and the second image, and the alignment means uses the misregistration amount calculated by the misregistration amount calculation means to The first image and the second image are aligned.

  According to the present invention, the feature amount extraction means for extracting a plurality of feature amounts of different types from the first image and the second image, and the feature amounts extracted by the feature amount extraction means, A positional deviation amount calculating means for calculating a positional deviation amount between the first image and the second image, and the alignment means uses the positional deviation amount calculated by the positional deviation amount calculating means to Since the second image is aligned, there is an effect that a highly accurate registration result can be obtained regardless of the presence or absence of an edge.

1 is a configuration diagram showing an image alignment apparatus according to Embodiment 1 of the present invention. It is a flowchart which shows the processing content (image position alignment method) of the image position alignment apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the process example of level correction. It is explanatory drawing which shows the example of a process of planarization. It is explanatory drawing which shows the example of a process of contrast correction | amendment. It is explanatory drawing which shows the filter coefficient example of a 3x3 Sobel filter. It is a block diagram which shows the image position alignment apparatus by Embodiment 2 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image alignment apparatus according to Embodiment 1 of the present invention.
In FIG. 1, when an image adjustment processing unit 1 receives image data indicating an image A (first image) and image data indicating an image B (second image), the feature amount in a subsequent feature amount extraction unit 2 is input. As pre-processing for facilitating extraction processing, image adjustment processing (for example, histogram correction processing, image frequency adjustment processing, etc.) for images A and B is performed. The image adjustment processing unit 1 constitutes an image adjustment processing unit.

The feature amount extraction unit 2 includes a plurality of feature amounts (for example, edges, feature points (for example, corner points, maximum / minimum points of intensity, different types) from the images A and B after the adjustment processing by the image adjustment processing unit 1. The process of extracting the end point of the curve, the maximum point of curvature, the isolated point, etc.) is performed. Note that the feature quantity extraction unit 2 constitutes a feature quantity extraction unit.
The block dividing unit 3 performs processing for dividing the image A and the image B into a plurality of blocks. The block dividing unit 3 constitutes block dividing means.

The misregistration amount calculation unit 4 uses the feature amounts extracted by the feature amount extraction unit 2 and has a correspondence relationship among the blocks of the image A and the blocks of the image B divided by the block division unit 3. A process for calculating the amount of misalignment is performed.
That is, the positional deviation amount calculation unit 4 calculates the correlation value between the blocks having the corresponding relationship while moving the block of the image B corresponding to the block of the image A in units of pixels, and the block having the highest correlation value is calculated. From the movement position, the amount of positional deviation between the blocks having the corresponding relationship is calculated.
The misregistration amount calculation unit 4 constitutes misregistration amount calculation means.

The registration calculation unit 5 performs registration of the image A and the image B by using the amount of positional deviation between blocks calculated by the positional deviation amount calculation unit 4 using each feature amount.
In other words, the registration calculation unit 5 calculates the position of the image unit from the positional deviation amount of the block unit of the same type of feature quantity as the calculation source among the positional deviation amounts of each block calculated by the positional deviation amount calculation unit 4. The shift amounts are calculated, and registration of the images A and B is performed using one of the shift amounts of the plurality of image units.
The registration calculation unit 5 constitutes an alignment unit.

In the example of FIG. 1, each of the image adjustment processing unit 1, the feature amount extraction unit 2, the block division unit 3, the positional deviation amount calculation unit 4, and the registration calculation unit 5, which are components of the image registration device, is dedicated hardware. Although hardware (for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer) is assumed, the image alignment apparatus may be configured with a computer.
When the image alignment apparatus is configured by a computer, the processing contents of the image adjustment processing unit 1, the feature amount extraction unit 2, the block division unit 3, the positional deviation amount calculation unit 4, and the registration calculation unit 5 are described. The program may be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 2 is a flowchart showing the processing contents (image registration method) of the image registration apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
When the image adjustment processing unit 1 receives the image data indicating the image A and the image data indicating the image B, the image adjustment processing unit 1 performs preprocessing for facilitating the feature amount extraction processing in the feature amount extraction unit 2 as image A and An image adjustment process is performed on the image B (step ST1).
That is, the image adjustment processing unit 1 takes a histogram of signal levels of each pixel constituting the image A and the image B, and performs, for example, image level correction, flattening, contrast correction, frequency adjustment, and the like.

Here, the image level correction is a process of adjusting the signal level of the image, and FIG. 3 is an explanatory diagram showing an example of the level correction process.
For example, when the maximum signal level is i _max among the signal levels i of the pixels constituting the image A (image B), the maximum signal level i _max is set to the input image. Is less than the maximum value S _max that can be taken by the signal of the pixels constituting the pixel (i _max <S _max ), the signal levels i of all the pixels constituting the image A (image B) are multiplied by S _max / i _max Thus, the maximum signal value i _max in the image A (image B) is adjusted so as to coincide with the maximum value S _max that the signal can take.
When the input image is an 8-bit image, the maximum value S _max that can be taken by a pixel signal constituting the input image is 255.

式（１）において、ｉは画像Ａ（画像Ｂ）を構成している各画素の信号レベルであり、ｈｉｓｔ（ｉ）は信号レベルｉのヒストグラムを求める関数、Ｈは画像Ａ（画像Ｂ）の縦サイズ、Ｗは画像Ａ（画像Ｂ）の横サイズである。 Image flattening is a process of flattening a histogram having amplitude, and FIG. 4 is an explanatory diagram showing an example of the flattening process.
When flattening an image, the following formula (1) is generally applied.

In Expression (1), i is a signal level of each pixel constituting the image A (image B), hist (i) is a function for obtaining a histogram of the signal level i, and H is an image A (image B). The vertical size W is the horizontal size of the image A (image B).

式（２）において、ＬＵＴ(ｉ）はヒストグラムのレベルに応じた重み付け係数であり、総和が１となるように設定されている。 However, simple flattening of the histogram may be converted into an image with high contrast if the level distribution of the image is biased.
Therefore, in order to avoid such a phenomenon, the weighted histogram may be flattened.
Equation (2) below shows an equation for weighted histogram flattening.

In Equation (2), LUT (i) is a weighting coefficient corresponding to the level of the histogram, and is set so that the sum is 1.

Image contrast correction is a process of correcting the signal level of an image using a look-up table (LUT), and FIG. 5 is an explanatory diagram showing an example of a contrast correction process.
In contrast correction of an image, the contrast can be simply increased or decreased, but the following processing may be performed.
For example, when the peak value of the histogram of the image A (image B) is obtained and the peak value is smaller than a predetermined value (for example, a value that is half of the maximum value that can be taken by the pixel signal constituting the input image) Different level expansion may be performed when the peak value is larger than a predetermined value.

The frequency adjustment of the image is a process of cutting the frequency according to the purpose.
In the frequency adjustment of the image, when the resolutions of the image A and the image B are different, a low-frequency cut process is performed on the image having the higher resolution in order to match both the resolutions.
As frequency adjustment processing, n × n low-pass filter processing may be performed, or low-frequency cut processing by Fourier transform (FFT) may be performed.

When the image adjustment processing unit 1 performs image adjustment processing on the images A and B, the feature amount extraction unit 2 extracts a plurality of feature amounts of different types from the images A and B after the adjustment processing (step ST2). .
As the feature amount extraction processing, for example, processing for extracting image edges as feature amounts, image feature points (for example, corner points, intensity maximum / minimum points, curve end points, curvature maximum points, isolated points) A process of extracting a point etc.) as a feature amount is assumed.

式（３）において、ｆｘ（ｘ，ｙ）は注目画素（ｘ，ｙ）に対する水平方向のフィルタ演算を行う関数を示し、ｆｙ（ｘ，ｙ）は注目画素（ｘ，ｙ）に対する垂直方向のフィルタ演算を行う関数を示している。
また、ｆｘ（ｘ，ｙ）、ｆｙ（ｘ，ｙ）には、画像調整処理部１による調整処理後の信号レベルが代入される。
ｆ’（ｘ，ｙ）は注目画素（ｘ，ｙ）におけるエッジの検出結果を示し、その検出結果が８ビットで表される場合、注目画素（ｘ，ｙ）にエッジがなければ０の値、注目画素（ｘ，ｙ）にエッジがあれば２５５などの値になる。 Here, processing for extracting the edge of the image A (image B) as a feature amount will be described.
As a general edge extraction method, there is a method using an n × n Sobel filter.
An arithmetic expression for extracting an edge using an n × n Sobel filter is expressed as the following Expression (3).

In Expression (3), fx (x, y) represents a function for performing a horizontal filter operation on the target pixel (x, y), and fy (x, y) represents a vertical direction with respect to the target pixel (x, y). A function for performing a filter operation is shown.
Further, the signal level after the adjustment processing by the image adjustment processing unit 1 is substituted into fx (x, y) and fy (x, y).
f ′ (x, y) indicates the detection result of the edge in the target pixel (x, y), and when the detection result is represented by 8 bits, the value of 0 is obtained if the target pixel (x, y) has no edge. If the target pixel (x, y) has an edge, the value becomes 255 or the like.

Note that Sobel is an edge detection method based on first-order differentiation, and the detection result is output as an edge gradient.
For example, in the case of a 3 × 3 Sobel filter, filter coefficients in the horizontal and vertical directions are expressed as shown in FIG.

Here, an example in which an edge is extracted using a Sobel filter has been described. However, the present invention is not limited to the Sobel filter, and for example, an edge may be extracted using a Canny filter.
When the Canny filter is used, the gradient with respect to the vicinity of 8 (four directions) of the target pixel is obtained, and the edge is connected in the direction having the strongest gradient. For this reason, the extracted edge is thinned to reduce noise.
Note that the edge detection result is represented by a binary value indicating whether or not the target pixel is an edge.

式（４）において、δは微分を示す演算記号である。
Ｌａｐｌａｃｉａｎフィルタは、２次微分フィルタであるため細かくエッジを抽出することができる。 Alternatively, an edge may be extracted using a Laplacian filter.
When a Laplacian filter is used, the sum of secondary differentials in both xy directions is calculated to extract an edge.
An arithmetic expression for extracting an edge using a Laplacian filter is expressed as the following Expression (4).

In Expression (4), δ is an arithmetic symbol indicating differentiation.
Since the Laplacian filter is a second-order differential filter, edges can be extracted finely.

  Note that the edge extraction method is not limited to the filter correction method. For example, the inverse FFT processing is performed by applying a mask to the low frequency region of the power image obtained by performing the FFT processing on the image. By doing so, the edge can be extracted.

式（５）において、ｆ（ｘ，ｙ）は注目画素（ｘ，ｙ）の信号レベル（例えば、輝度値）に相当し、画像調整処理部１による調整処理後の信号レベルである。
ｆ（ｘ＋ｉ，ｙ＋ｊ）は注目画素（ｘ，ｙ）の周辺画素（ｘ＋ｉ，ｙ＋ｊ）の信号レベルに相当し、画像調整処理部１による調整処理後の信号レベルである。
また、Ｍｏｖａｃは注目画素（ｘ，ｙ）のコーナー点らしさを評価する値である。 Next, processing for extracting feature points of the image A (image B) as feature amounts will be described.
As a general extraction method of feature point extraction processing, there is a corner detection method.
In the corner detection method, not only so-called corner points but also maximum and minimum points of intensity, end points of curves, maximum points of curvature, isolated points, and the like can be detected as feature points.
As a general corner detection method, there is a corner detection algorithm called “Moravec”, and an arithmetic expression thereof is expressed as the following Expression (5).

In Expression (5), f (x, y) corresponds to the signal level (for example, luminance value) of the target pixel (x, y), and is the signal level after the adjustment processing by the image adjustment processing unit 1.
f (x + i, y + j) corresponds to the signal level of the peripheral pixel (x + i, y + j) of the target pixel (x, y), and is the signal level after the adjustment processing by the image adjustment processing unit 1.
Movac is a value for evaluating the likelihood of a corner point of the pixel of interest (x, y).

Here, an example in which the corner detection method is a corner detection algorithm called “Moravec” has been shown, but this corner detection algorithm has a drawback that Moravec operators are limited to eight directions.
The corner detection algorithm called “SUZAN” is a corner detection algorithm in which the drawbacks of “Moravec” are improved, and “SUZAN” may be used.
SUZAN is a method of searching for a vertex having a large curvature in a convex region where the luminance value changes discontinuously.
Equation (6) below shows an arithmetic expression of a corner detection algorithm called “SUZAN”.

式（６）において、ｆ（ｘ，ｙ）は注目画素（ｘ，ｙ）の信号レベル（例えば、輝度値）に相当し、画像調整処理部１による調整処理後の信号レベルである。
ｆ’（ｘ’，ｙ’）は注目画素（ｘ，ｙ）の周辺画素（ｘ’，ｙ’）の信号レベルに相当する。
また、Ｍ_x,yは注目画素（ｘ，ｙ）を中心とするある近傍領域であり、ｔは予め設定された閾値である
式（６）では、ある近傍領域内の中心画素と輝度値の差が小さい画素の数を数えており、その極小点を特徴点としている。

In Expression (6), f (x, y) corresponds to the signal level (for example, luminance value) of the target pixel (x, y), and is the signal level after the adjustment processing by the image adjustment processing unit 1.
f ′ (x ′, y ′) corresponds to the signal level of the peripheral pixel (x ′, y ′) of the target pixel (x, y).
M _{x, y} is a certain neighboring area centered on the target pixel (x, y), and t is a preset threshold value. In Expression (6), the central pixel and luminance value in a certain neighboring area The number of pixels with a small difference is counted, and the minimum point is used as a feature point.

In addition to “Moravec” and “SUZAN”, there is a corner detection algorithm called “Harris”, and any method may be used as long as it is a method for extracting feature points.
Here, the feature quantity extraction unit 2 extracts a plurality of feature quantities of different types from the image A and the image B after adjustment processing output from the image adjustment processing section 1, and positions the plurality of feature quantities of different types. Although what is output to the shift amount calculation unit 4 is shown, the image A and the image B itself after adjustment processing output from the image adjustment processing unit 1 are output as feature amounts to the position shift amount calculation unit 4. Also good.
That is, since the image A and the image B after the adjustment process output from the image adjustment processing unit 1 represent the light and shade of the image, the light and shade itself is regarded as the feature amount of the image, and the image A after the adjustment process is processed. In addition, the image A and the image B may be output to the misregistration amount calculation unit 4 as they are without performing the process of extracting the feature amount from the image B.

The block dividing unit 3 divides the image A and the image B from which the feature amount is extracted by the feature amount extracting unit 2 into G × H blocks (step ST3).
When the block dividing unit 3 divides the image A and the image B into G × H blocks, the positional deviation amount calculating unit 4 uses the feature amounts extracted by the feature amount extracting unit 2 to use the block dividing unit 3. Among the divided blocks of the image A and the block of the image B, a positional deviation amount between the blocks having a correspondence relationship is calculated (step ST4).
If the feature amount extracted by the feature amount extraction unit 2 is, for example, three types of corner points, edges, and gray values, the amount of positional deviation between the blocks having the corresponding relationship can be calculated using the corner points (feature amounts). In addition to the calculation, an edge (feature value) is used to calculate the amount of misalignment between blocks having a corresponding relationship.
Also, the amount of positional deviation between the blocks having the correspondence relationship is calculated using the gray value (feature value).

Hereinafter, the calculation process of the positional deviation amount between the blocks by the positional deviation amount calculation unit 4 will be described in detail.
When the image A and the image B are divided into G × H blocks, the number of blocks is G × H, the block of the image A is (g, h), the block A, and the block of the image B is (g, h). This is expressed as block B. However, g = 0, 1,..., G−1, h = 0, 1,.
In this case, in the images A and B, the blocks having a correspondence relationship are the (g, h) block A and the (g, h) block B.

The misregistration amount calculation unit 4 calculates the misregistration amount between blocks for each block having a correspondence relationship in the images A and B.
That is, the positional deviation amount calculation unit 4 first calculates a correlation value between the (g, h) block A and the (g, h) block B. A method for calculating the correlation value will be described later.
Next, each pixel constituting the (g, h) block B is moved one pixel at a time in the X direction or the horizontal direction, and the (g, h) block A and the moved (g, h) block B are moved. A correlation value is calculated.
The misregistration amount calculation unit 4 includes a movement process in units of pixels of the (g, h) block B, and a process of calculating a correlation value between the (g, h) block A and the moved (g, h) block B. Repeatedly.
The above iterative process moves the pixel unit of the (g, h) block B by a preset pixel in the X-axis direction and moves by a preset pixel in the Y-axis direction, This is performed until the correlation value calculation process is completed.
Here, an example is shown in which (g, h) block B is moved in units of pixels, but (g, h) block A may be moved in units of pixels.

When the above-described repetitive processing is completed, the positional deviation amount calculation unit 4 identifies the highest correlation value among the calculated correlation values, and the (g, h) block B when the correlation value is calculated. (G, h) block A and (g, h) block B displacement amounts are calculated from the movement position of.
For example, the movement position of (g, h) block B when the highest correlation value is calculated is moved from the original position of (g, h) block B by x1 pixels in the X axis direction, and in the Y axis direction. If only y1 pixel has been moved, the amount of positional deviation between (g, h) block A and (g, h) block B is calculated as (x1, y1).

Here, the correlation value between the (g, h) block A and the (g, h) block B is calculated by the following equation (7), for example.

In equation (7), I (j, i) is the (g, h) feature quantity of each pixel constituting block A, I bar (in relation to the electronic application, above “I” in equation (7) Since “-” appended to cannot be described on the text of the specification, it is expressed as I bar) (g, h) of all pixels constituting the block A It is the average value of the feature amount.
T (j, i) is the feature value of each pixel constituting the (g, h) block B, and T bar is the average value of the feature values of all the pixels constituting the (g, h) block B. .
N is the number of pixels in the X-axis direction in the (g, h) blocks A and B, and M is the number of pixels in the Y-axis direction in the (g, h) blocks A and B.
NCC is a normal cross-correlation value of (g, h) block A and (g, h) block B.
Note that the normal cross-correlation value NCC can stably calculate the correlation value even when the two images have different shades. If the correlation value between the two blocks is obtained, the correlation value is calculated by The calculation is not limited to (7).

When the positional deviation amount calculation unit 4 calculates the positional deviation amounts of the (g, h) block A and the (g, h) block B using the respective feature amounts, the registration calculation unit 5 (g, h ) Registration of the image A and the image B is performed using the positional deviation amounts of the block A and the (g, h) block B (step ST5).
Hereinafter, the registration of the image A and the image B by the registration calculation unit 5 will be specifically described.

The registration calculation unit 5 is a calculation source when the positional deviation amount calculation unit 4 calculates the positional deviation amounts of the (g, h) block A and the (g, h) block B using the respective feature amounts. A positional deviation amount in units of images is calculated from a positional deviation amount in units of blocks having the same feature quantity type.
If the feature quantity extracted by the feature quantity extraction unit 2 is, for example, two types of corner points and gray values, the (g, h) block A and (g, h) calculated using the corner points (feature quantities) are used. h) The positional deviation amount of the image unit is calculated from the positional deviation amount of the block B, and the positional deviation amount between the (g, h) block A and the (g, h) block B calculated using the gray value (feature value). The amount of positional deviation for each image is calculated from the above.
When the image A and the image B are divided into G × H blocks, the number of blocks is G × H. Therefore, for each type of feature amount, a positional deviation amount ((g, h) block A and (G, h) G × H position deviation amounts of block B) are calculated. For this reason, as a method of calculating the positional deviation amount in image units from the positional deviation amount in block units, the average value or median value of the positional deviation amounts in G × H block units for each type of feature amount, A method of calculating the amount of positional deviation in image units is conceivable.

When the registration calculation unit 5 calculates the positional deviation amount in units of images for each type of feature amount, the registration calculation unit 5 selects one positional deviation amount from among the positional deviation amounts in units of images. Selection of one displacement amount can be performed based on the characteristics of the images A and B.
For example, a method of comparing the correlation values calculated using the respective feature amounts and selecting a displacement amount calculated using the feature amount having the higher correlation value may be considered.
Specifically, if the feature amount extracted by the feature amount extraction unit 2 is, for example, two types of edges and gray values, the number of edges present in the images A and B is greater than a predetermined threshold value. If the position deviation amount calculated using the edges (feature amount) is selected and the number of edges existing in the images A and B is smaller than a predetermined threshold value, the gray level value (feature amount) is used. Select the amount of misalignment.
Note that the number of edges existing in the images A and B may be, for example, the sum of signal values indicating edges (features) in all blocks.

When the registration calculation unit 5 selects one positional deviation amount from among a plurality of image unit positional deviation amounts, the registration unit 5 uses the positional deviation amount to convert an affine transformation coefficient (for example, rotate or translate an image). For example, a 3 × 3 filter coefficient) is calculated. Since the process itself for calculating the conversion coefficient of the affine transformation from the positional deviation amount is a known technique, detailed description thereof is omitted.
When the registration calculation unit 5 calculates the conversion coefficient of the affine transformation, the registration calculation unit 5 multiplies the image B by the conversion coefficient to perform registration that matches the position of the image B with the position of the image A.
The registration calculation unit 5 outputs image data indicating the image B after alignment.

As is apparent from the above, according to the first embodiment, the feature amount extraction unit 2 that extracts a plurality of different feature amounts from the image A and the image B, and the image A and the image B are divided into a plurality of blocks. Blocks having a correspondence relationship among the blocks of the image A and the blocks of the image B that are divided by the block dividing unit 3 by using the feature amounts extracted by the block dividing unit 3 and the feature amount extracting unit 2 A misregistration amount calculation unit 4 for calculating a misregistration amount between the blocks, and the registration calculation unit 5 uses the misregistration amount between blocks calculated by the misregistration amount calculation unit 4 using each feature amount. Thus, since the image A and the image B are aligned, it is possible to obtain a highly accurate registration result regardless of the presence or absence of an edge.
That is, a plurality of different types of feature amounts are extracted from the images A and B, and a positional shift amount is calculated using each feature amount, and one positional shift is adaptively selected from the plurality of positional shift amounts. Since the amount is selected, it is possible to perform highly accurate registration that matches the characteristics of the image.

  Further, according to the first embodiment, the image adjustment processing unit 1 that performs image adjustment processing on the images A and B and outputs the image A and image B after the adjustment processing to the feature amount extraction unit 2 is provided. Therefore, the feature amount extraction unit 2 can easily extract the feature amount, and the feature amount can be extracted with high accuracy.

Embodiment 2. FIG.
In the first embodiment, the registration calculation unit 5 calculates the image unit misregistration amount from the block unit misregistration amount that has the same type of feature quantity as the calculation source, and obtains a plurality of image unit misregistration values. Although one misregistration amount is selected from the misregistration amounts and the position of image A and image B is aligned using one misregistration amount, it is calculated using each feature amount. One positional deviation amount is selected for each block from among the positional deviation amounts between the blocks, and the positional deviation amount for each image is calculated from the selected positional deviation amount for each block. May be used to align the images A and B.

FIG. 7 is a block diagram showing an image alignment apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG.
Similar to the registration calculation unit 5 in FIG. 1, the registration calculation unit 6 uses the positional deviation amounts between the blocks calculated by the positional deviation amount calculation unit 4 using the respective feature amounts, and uses the image A and the image B. Perform the registration.
However, the registration calculation unit 6 is different from the registration calculation unit 5 in FIG. 1 from among the positional shift amounts between blocks calculated by the positional shift amount calculation unit 4 using the respective feature amounts, and 1 for each block. One misregistration amount is selected, the misregistration amount for each image is calculated from the misregistration amount for each selected block, and registration of images A and B is performed using the misregistration amount for each image unit.
The registration calculation unit 6 constitutes an alignment unit.

In the example of FIG. 7, each of the image adjustment processing unit 1, the feature amount extraction unit 2, the block division unit 3, the positional deviation amount calculation unit 4, and the registration calculation unit 6, which are components of the image registration apparatus, is dedicated hardware. Although hardware (for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer) is assumed, the image alignment apparatus may be configured with a computer.
When the image alignment apparatus is configured by a computer, the processing contents of the image adjustment processing unit 1, the feature amount extraction unit 2, the block division unit 3, the positional deviation amount calculation unit 4 and the registration calculation unit 6 are described. The program may be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.

Next, the operation will be described.
Compared to the first embodiment, since the registration calculation unit 6 is the same except that it is mounted instead of the registration calculation unit 5, the processing contents other than the registration calculation unit 6 are omitted here. .

When the positional deviation amount calculation unit 4 calculates the positional deviation amounts of the (g, h) block A and the (g, h) block B using the respective feature amounts, the registration calculation unit 6 (g, h ) Registration of the image A and the image B is performed using the amount of positional deviation between the block A and the block (g, h).
Hereinafter, the registration of the image A and the image B by the registration calculation unit 6 will be specifically described.

When the positional deviation amount calculation unit 4 calculates the positional deviation amounts of the (g, h) block A and the (g, h) block B using the respective feature amounts, the registration calculation unit 6 Then, one misregistration amount is selected from the calculated misregistration amounts.
If the feature quantity extracted by the feature quantity extraction unit 2 is, for example, two types of corner points and gray values, the (g, h) block A and (g, h) calculated using the corner points (feature quantities) are used. h) One positional deviation from among the positional deviation amount of block B and the positional deviation amounts of (g, h) block A and (g, h) block B calculated using the gray value (feature value). Select the amount.

Selection of one displacement amount can be performed based on the characteristics of the images A and B.
For example, a method of comparing the correlation values calculated using the respective feature amounts and selecting a displacement amount calculated using the feature amount having the higher correlation value may be considered.
Specifically, if there are two types of feature values extracted by the feature value extraction unit 2, for example, an edge and a gray value, the number of edges existing in (g, h) blocks A and B is predetermined. If the number of edges is larger than the threshold value, the position shift amount calculated using the edge (feature amount) is selected. If (g, h) the number of edges existing in the blocks A and B is less than the predetermined threshold value, the gray level A position shift amount calculated using a value (feature amount) is selected.
Also, the smallest positional deviation amount may be selected from among a plurality of positional deviation amounts.
In the case of a method of selecting one positional deviation amount for each corresponding block, if the image A and the image B are divided into G × H blocks, the number of blocks is G × H. Thus, G × H position shift amounts are selected.

When one registration error amount is selected for each corresponding block, the registration calculation unit 6 calculates a registration error amount in image units from the GxH block error amounts.
As a method of calculating the image unit displacement amount from the block unit displacement amount, a method of calculating the average value or median value of the G × H displacement amounts as the image unit displacement amount is considered. It is done.

When the registration calculation unit 6 calculates the positional deviation amount for each image, the registration calculation unit 6 calculates the conversion coefficient of the affine transformation by using the positional deviation amount as in the registration calculation unit 5 of FIG.
When the registration calculation unit 6 calculates the conversion coefficient of the affine transformation, the registration calculation unit 6 multiplies the image B by the conversion coefficient to perform registration that matches the position of the image B with the position of the image A.
The registration calculation unit 6 outputs image data indicating the image B after alignment.

  As can be seen from the above, according to the second embodiment, one positional deviation per block is selected from among the positional deviation amounts between blocks calculated by the positional deviation amount calculation unit 4 using the respective feature amounts. An amount of image is selected, a position shift amount for each image is calculated from the position shift amount for the selected block, and registration of images A and B is performed using the position shift amount for each image unit. Therefore, the positional deviation amount is adaptively selected in units of blocks, and as a result, there is an effect that a more accurate registration result can be obtained than in the first embodiment.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Image adjustment process part (image adjustment process means) 2 Feature amount extraction part (feature amount extraction means) 3 Block division part (block division means) 4 Position shift amount calculation part (position shift amount calculation means), 5, 6 Registration calculation unit (positioning means).

Claims (8)

  Feature amount extraction means for extracting a plurality of feature amounts of different types from the first image and the second image, and using the feature amounts extracted by the feature amount extraction means, the first image and the above Using the positional deviation amount calculation means for calculating the positional deviation amount between the second images and the positional deviation amount calculated by the positional deviation amount calculation means, the first image and the second image are aligned. An image alignment apparatus comprising: alignment means for performing Block dividing means for dividing the first image and the second image into a plurality of blocks;
The misregistration amount calculation means uses the respective feature amounts extracted by the feature amount extraction means, and uses a correspondence relationship between the block of the first image and the block of the second image divided by the block division means. Calculates the amount of misalignment between blocks,
The alignment unit calculates the positional deviation amount of each image from the positional deviation amount of the block unit having the same type of feature amount as the calculation source among the positional deviation amounts of each block calculated by the positional deviation amount calculation unit. The position of the first image and that of the second image are respectively calculated using one position shift amount among the plurality of image unit position shift amounts. Image alignment device. Block dividing means for dividing the first image and the second image into a plurality of blocks;
The misregistration amount calculation means uses the respective feature amounts extracted by the feature amount extraction means, and uses a correspondence relationship between the block of the first image and the block of the second image divided by the block division means. Calculates the amount of misalignment between blocks,
The alignment means selects one positional deviation amount for each block from among the positional deviation amounts between the blocks calculated using the respective feature amounts by the positional deviation amount calculation means, and selects the selected block unit. The position shift amount for each image is calculated from the position shift amount, and the first image and the second image are aligned using the position shift amount for each image. Image alignment device.   The misregistration amount calculation means calculates a correlation value between blocks having a corresponding relationship while moving the second image block corresponding to the first image block in units of pixels, and the correlation value becomes the highest. 4. The image alignment apparatus according to claim 2, wherein a displacement amount between blocks having a correspondence relationship is calculated from the movement position of the block.   Image adjustment processing means for performing image adjustment processing on the first image and the second image and outputting the adjusted first image and second image to the feature amount extraction means is provided. The image alignment apparatus according to any one of claims 1 to 4.   The feature amount extraction unit extracts a plurality of feature amounts of different types from the first image and the second image, and the positional deviation amount calculation unit is extracted in the feature amount extraction processing step. A misregistration amount calculation processing step for calculating a misregistration amount between the first image and the second image using each feature amount, and an alignment unit are calculated in the misregistration amount calculation processing step. An image alignment method comprising: an alignment processing step for aligning the first image and the second image using the misregistration amount. The block dividing means includes a block division processing step for dividing the first image and the second image into a plurality of blocks,
In the misregistration amount calculation processing step, using the feature amounts extracted in the feature amount extraction processing step, the first image block and the second image block divided in the block division processing step are used. , Calculate the amount of misalignment between blocks that have a correspondence,
In the alignment processing step, out of the positional deviation amounts for each block calculated in the positional deviation amount calculation processing step, the positional deviation in image units is calculated from the positional deviation amount in blocks in which the type of the feature quantity as the calculation source is the same. The amount of each is calculated, and the first image and the second image are aligned using one position shift amount among a plurality of image unit position shift amounts. The image alignment method described. The block dividing means includes a block division processing step for dividing the first image and the second image into a plurality of blocks,
In the misregistration amount calculation processing step, using the feature amounts extracted in the feature amount extraction processing step, the first image block and the second image block divided in the block division processing step are used. , Calculate the amount of misalignment between blocks that have a correspondence,
In the alignment processing step, one positional deviation amount is selected for each block from among the positional deviation amounts between blocks calculated using the respective feature amounts in the positional deviation amount calculation processing step, and the selected block 7. The positional deviation amount in units of images is calculated from the positional deviation amount in units, and the first image and the second image are aligned using the positional deviation amount in units of images. The image alignment method described.

Images