JP5945823B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus that perform image matching using a template image.

  Conventionally, an image matching technique has been used to inspect parts and the like. One example of image matching is to use a template image. In image collation using a template image, for example, a component that is known to be a non-defective product is imaged, and a template image is prepared in advance using the captured image. Then, the input image obtained by imaging the part to be inspected and the template image are collated, and the part to be inspected is inspected by comparing the input image and the template image.

  In particular, in industrial image processing, in order to detect the position and direction angle of a part to be inspected with high accuracy, a process of matching a template image given in advance with an input image with subpixel accuracy is performed. (Refer nonpatent literature 1). As such an image collation technique with subpixel accuracy, a rotation collation method is known in which collation is performed while rotating a template image with respect to an input image. The rotation matching method is an image matching technique using geometric transformation that performs geometric transformation on a template image.

  In the rotation matching method, an image similarity indicating image similarity in relation to an input image is calculated while the template image is rotated and translated, and the direction angle of the input image and the input image direction angle are calculated from the peak value of the image similarity. Coordinate values are detected. That is, according to the rotation collation method, the template image is rotated and translated at a predetermined pitch, and the rotation angle and the XY coordinate values that approximate the input image to the template image are searched, whereby the direction angle of the input image. And coordinate values are detected.

  In such a rotation collation method, in order to increase the detection accuracy, it is necessary to increase the number of collations by reducing the rotation pitch of the template image for searching for the rotation angle at which the template image most closely approximates the input image. For this reason, increasing the detection accuracy in the rotation matching method leads to a long processing time. On the other hand, if the rotation pitch of the template image is rough (enlarged) in order to shorten the processing time, it becomes difficult to obtain sufficient detection accuracy.

  Therefore, in order to reduce the processing time in the image matching method, the image similarity calculated by a predetermined calculation is calculated using a quadratic curved surface or a quadratic curve while collating with a rough rotation pitch of the template image. There is a method of interpolation (see Non-Patent Documents 2 and 3). However, according to the method of performing interpolation in this way, an error due to interpolation occurs, and the detection accuracy decreases.

  As an image matching technique that can ensure the same accuracy with a small number of matching times and can shorten the processing time compared to the conventional image matching technique using geometric transformation such as the rotation matching method, There is a technique described in Non-Patent Document 4. In the technique of Non-Patent Document 4, an eigenvalue decomposition template is generated by compressing a plurality of template images in various postures (direction angles) by eigenvalue decomposition (principal component analysis). By performing the convolution, a correlation value indicating the similarity between the images is calculated. Based on the calculated correlation value, the direction angle and the two-dimensional coordinate value for the input image are obtained.

Kazuhiko Kusumi, Shunichi Kaneko, “Practical use of machine vision technology”, IEEJ Transactions C, Vol. 124, no. 3, pp. 598-605, 2004. Masao Shimizu and Masatoshi Okutomi, “Precise Subpixel Estimation Method for Image Matching”, IEICE Transactions D-II, Vol. J84-D-II, No. 7, pp. 1409-1418, 2001. Masao Shimizu, Masatoshi Okutomi, “Meaning and Properties of Subpixel Estimation in Image Matching”, IEICE Transactions D-II, Vol. J85-D-II, No. 12, pp. 1791-1800, 2002. Takeshi Kamijo and Shinichi Uchimura, “Highly accurate image matching using eigenvalue decomposition templates”, IEEJ Transactions C, vol. 121, no. 1, pp. 237-238, 2011

  Certainly, according to the method of Non-Patent Document 4, it is possible to ensure the same accuracy and shorten the processing time as compared with the conventional image matching technique using geometric transformation. However, in the method of Non-Patent Document 4, correlation values for all pixels of the input image and correlation values for all postures (direction angles) at each pixel are calculated, and the maximum value among these correlation values is searched. Is done. For this reason, the amount of calculation becomes enormous, and there is a problem in terms of processing time in practical use.

  Therefore, the present invention maintains accuracy in an image matching technique using a correlation value indicating image similarity calculated based on an eigenvalue decomposition template image obtained by eigenvalue decomposition of a plurality of template images. However, it is an object of the present invention to provide an image processing method and an image processing apparatus capable of greatly reducing the amount of calculation and improving the processing speed.

The image processing method according to the present invention performs image processing using a template image that is an image of an object to be collated, thereby determining a value of a geometric transformation parameter of an image portion of the object to be collated existing in an input image. A method comprising the following steps.
A first image group generation step of generating a first image group composed of the template images after a plurality of geometric transformations having different geometric transformation amounts by applying geometric transformation to the template image;
An eigenvalue decomposition step of calculating eigenvalues and eigenfunctions of the same number as the template images after the plurality of geometric transformations by performing eigenvalue decomposition on the first image group;
A second image group generation step of generating a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group;
Correlation value vector generation step for generating a correlation value vector having a correlation value that is a predetermined real number representing a correlation with each image of the second image group for each image of the first image group. .
For each pixel of the input image, based on the correlation value calculated for the relationship between the lowest order image having the lowest order among the images of the second image group and the input image, the pixels of the input image A pixel excluding step of excluding from the processing target pixels having a relatively low correlation value for the lowest order image.
A feature vector generating step of generating a feature vector having the correlation value as a value of each component for the input image.
A vector selection step of selecting the correlation value vector closest to the feature vector from the plurality of correlation value vectors generated in the correlation value vector generation step;
For pixels to be processed other than the pixels excluded by the pixel exclusion step, a continuous curve is represented in a range near the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selection step. An image similarity calculating step of calculating an image similarity indicating a degree of similarity between the input image and the second image group;
A collating step of determining a value of the geometric transformation parameter based on the image similarity calculated in the image similarity calculating step;

  The image processing method of the present invention further includes an eigenfunction selection step of selecting a predetermined number of the eigenfunctions in descending order of the eigenvalues among the plurality of eigenfunctions calculated in the eigenvalue decomposition step, In the second image group generation step, the second image group is generated by performing an inner product calculation of the plurality of eigenfunctions selected in the eigenfunction selection step and the first image group.

  In the image processing method of the present invention, an image selection step of selecting a predetermined number of images in descending order of the eigenvalues among the images of the second image group generated in the second image group generation step. The correlation value vector generation step generates the correlation value vector for the image selected in the image selection step.

  In the image processing method of the present invention, the geometric transformation parameter includes a direction angle and a two-dimensional coordinate value, the geometric transformation is a rotation for changing the direction angle, and the image similarity calculation step includes the process The image similarity is calculated for all pixels of the target pixel within a range in the vicinity of the direction angle value corresponding to the correlation value vector selected by the vector selection step, and the matching step includes the processing Of all the target pixels, the position of the pixel with the largest image similarity calculated in the image similarity calculation step is determined as the value of the two-dimensional coordinate value of the geometric transformation parameter. The direction angle corresponding to the maximum value for the pixel having the largest image similarity maximum value is determined as the direction angle of the geometric transformation parameter.

  The image processing apparatus according to the present invention performs matching using an input image acquisition unit for acquiring an input image and a template image that is an image of an object to be collated, so that the object to be collated existing in the input image is detected. Image processing means for determining a value of a geometric transformation parameter of the image portion, and the image processing means adds the geometric transformation to the template image, so that the geometric transformation amounts after the plurality of geometric transformations are different. A first image group generation unit configured to generate a first image group of template images, and eigenvalue decomposition on the first image group, whereby the same number of eigenvalues as the template images after the plurality of geometric transformations and An eigenvalue decomposition unit that calculates an eigenfunction; a second image group generation unit that generates a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group; 1's A correlation value vector generation unit that generates a correlation value vector having a correlation value that is a predetermined real number representing a correlation with each image of the second image group as a value of each component for each image of the image group; and the input For each pixel of the image, based on the correlation value calculated for the relationship between the lowest order image having the lowest order among the images of the second image group and the input image, the lowest order of the pixels of the input image. A pixel excluding unit that excludes a pixel having a relatively low correlation value from an image to be processed; a feature vector generating unit that generates a feature vector having the correlation value as a value of each component for the input image; and Among the plurality of correlation value vectors generated by the correlation value vector generation unit, a vector selection unit that selects the correlation value vector that is closest to the feature vector, and the pixel exclusion unit For the pixels to be processed other than the removed pixels, the input image represented by a continuous curve in the range near the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selection unit and the input image An image similarity calculating unit that calculates an image similarity indicating a degree of similarity with the second image group, and a value of the geometric transformation parameter is determined based on the image similarity calculated by the image similarity calculating unit. And a collation unit.

  In the image processing apparatus of the present invention, the image processing means selects a predetermined number of the eigenfunctions in descending order of the eigenvalues from among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit. The second image group generation unit further includes a selection unit, and the second image group generation unit performs an inner product operation of the plurality of eigenfunctions selected by the selection unit and the first image group. Is generated.

  In the image processing apparatus of the present invention, the image processing means includes a predetermined number of images in descending order of the eigenvalues among the images of the second image group generated by the second image group generation unit. The correlation value vector generation unit generates the correlation value vector for the image selected by the image selection unit.

  According to the present invention, the accuracy is maintained in the image matching technique using the correlation value indicating the similarity of images calculated based on the eigenvalue decomposition template image obtained by eigenvalue decomposition of a plurality of template images. However, the calculation amount can be significantly reduced, and the processing speed can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic explanatory diagram of an image processing method according to an embodiment of the present invention. The flowchart which shows the outline of the image processing method which concerns on one Embodiment of this invention. Explanatory drawing of the image processing method which concerns on one Embodiment of this invention. Explanatory drawing about the narrowing down of the position in the image processing which concerns on one Embodiment of this invention. The figure which shows the correlation waveform in the image processing which concerns on one Embodiment of this invention. Explanatory drawing about the correlation value vector in the image processing which concerns on one Embodiment of this invention. The figure which shows the relationship between the correlation value vector and feature vector in the image processing which concerns on one Embodiment of this invention. Explanatory drawing about narrowing of the direction angle in the image processing which concerns on one Embodiment of this invention. 1 is a diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a control block diagram of an image processing apparatus according to an embodiment of the present invention. Explanatory drawing of the image processing of the image processing apparatus which concerns on one Embodiment of this invention. The flowchart which shows an example of the process sequence of the image process which concerns on one Embodiment of this invention. The flowchart which shows an example of the process sequence of the image process which concerns on one Embodiment of this invention. The flowchart which shows an example of the process sequence of the image process which concerns on one Embodiment of this invention. The figure which shows the result of the evaluation experiment as an Example of this invention. The figure which shows the result of the evaluation experiment as an Example of this invention.

  The present invention uses a correlation value indicating the degree of similarity of images calculated based on an eigenvalue decomposition template image obtained by eigenvalue decomposition of a plurality of template images. The geometric transformation parameter is estimated. The present invention uses the result of convolution of the eigenvalue decomposition template image when collating the input image with the eigenvalue decomposition template image, and narrows down and estimates the region (position) where the object exists in the input image. By narrowing down the range of geometric transformation parameter values (direction angle, etc.) to be performed, high-speed and high-precision collation is attempted. Embodiments of the present invention will be described below.

[Purpose of image processing method]
As shown in FIG. 1, the image processing method according to the present embodiment performs matching using a template image 1 that is an image of an object to be collated, so that the image portion of the object to be collated that exists in the input image 2. The value of the geometric transformation parameter of the target image portion 3 is determined. Here, as the geometric transformation parameter, for example, a direction angle and a two-dimensional coordinate value (XY coordinate value) are used.

  That is, in the image processing method of the present embodiment, the template image 1 is collated with respect to the input image 2, whereby the direction angle at which the image similarity, which is the degree of approximation of the template image 1 with respect to the input image 2, is the highest. And the process which calculates | requires a two-dimensional coordinate value is performed. Therefore, according to the image processing method of the present embodiment, the direction angle and the two-dimensional coordinate value of the target image portion 3 in the input image 2 with the template image 1 as a reference are obtained as a matching result.

  Specifically, as shown in FIG. 1, as the direction angle included in the collation result, for example, the inclination (rotation angle) θ of the target image portion 3 with respect to the reference line O1 indicating the direction angle of the template image 1 is obtained. It is done. As the two-dimensional coordinate value included in the collation result, for example, a target image when a predetermined position in the input image 2 (in the example illustrated in FIG. 1, the position of the upper left corner of the input image 2) is the origin O2. A two-dimensional coordinate value (XY coordinate value) of the center position C1 of the portion 3 is obtained.

  The image processing method according to the present embodiment is used, for example, for industrial image processing for inspecting components and the like. For this reason, in the image processing method of the present embodiment, the template image 1 is acquired and prepared in advance by, for example, imaging the same component as the component to be inspected by an imaging unit such as a camera. For example, in the inspection of a component, the input image 2 is acquired by imaging a component to be inspected by an imaging unit such as a camera, and is a target for collation of the template image 1. Below, the specific processing content of the image processing method of this embodiment is demonstrated.

[Outline of image processing method]
The outline of the image processing method of this embodiment will be described with reference to the flowchart shown in FIG. As shown in FIG. 2, in the image processing of this embodiment, first, the template image 1 is input (S10). Here, as described above, the template image 1 acquired by the imaging means such as a camera and prepared in advance is input to an arithmetic device such as a computer that executes the image processing method of the present embodiment.

  Next, an eigenvalue decomposition template image is generated (S20). Here, the eigenvalue decomposition template image is an eigenfunction obtained by performing geometric transformation on the template image 1 input as described above, and performing eigenvalue decomposition on the template image after the geometric transformation, and a template after geometric transformation. It is an image obtained from the image.

  Next, a dictionary is generated (S30). Here, the dictionary is a group of vectors obtained corresponding to each image of the template image after geometric transformation by collating the eigenvalue decomposition template image and the input image 2 described above.

  Next, the input image 2 is input (S40). Here, as described above, for example, the input image 2 acquired by imaging a part to be inspected by an imaging unit such as a camera is input to an arithmetic device such as a computer.

  Subsequently, image collation with three stages of narrowing is performed (S50). The narrowing performed in the process of image collation includes narrowing down pixels to be processed by estimating a region where an object exists in the input image 2, that is, an image region including the target image portion 3, and a geometry to be determined. And narrowing down the range of values of conversion parameters. As a result of this image collation, a direction angle and a two-dimensional coordinate value (XY coordinate value) for the input image 2 are detected.

  Then, based on the result of the image collation in step S50, the direction angle and the two-dimensional coordinate value for the input image 2 are output as the detected posture and the detected position, respectively (S60). Thus, in the image processing method of the present embodiment, the directional angle and the two-dimensional coordinate value for the input image 2 are adopted as the geometric transformation parameters to be determined. Hereinafter, detailed processing contents of the image processing method of the present embodiment will be described.

[Details of image processing method]
In the image processing method of the present embodiment, a process of generating a converted template image group as a first image group by applying geometric transformation to the template image 1 is performed. In other words, the post-conversion template image group is a template image 1 after geometric transformation (hereinafter referred to as “post-conversion template image”), in other words, an image group formed by adding geometric transformation to the template image 1. As the geometric transformation applied to the template image 1, for example, rotation for changing the direction angle and parallel movement for changing the two-dimensional coordinate value are performed.

  Specifically, when a rotation that changes the direction angle is performed as the geometric transformation applied to the template image 1, for example, a process of generating a template image group after conversion by rotating the template image 1 at a 1 ° pitch. Done. According to this process, 360 converted template images having different directional angles at a 1 ° pitch are generated, and the 360 converted template images are obtained as a converted template image group.

  Further, as a geometric transformation to be added to the template image 1, when a parallel movement that changes a two-dimensional coordinate value is performed, for example, when the template image 1 is 128 × 128 (pixels), A process of generating a post-conversion template image group is performed by translating the two-dimensional coordinate value) within a range of 8 × 8 (pixels) at a 1 (pixel) pitch. According to this process, 64 converted template images having different center positions (center two-dimensional coordinate values) at 1 (pixel) pitch are generated, and the 64 converted template images are converted into a group of converted template images. As obtained.

  As described above, in the image processing method according to the present embodiment, by applying geometric transformation to the template image 1, the first template image group including a plurality of transformed template images having different geometric transformation amounts is generated. An image group generation step is performed. Here, the amount of geometric transformation corresponds to the rotation amount of the image with reference to the direction angle of the template image 1 when the geometric transformation is a direction angle, and 2 of the template image 1 when the geometric transformation is a two-dimensional coordinate value. This corresponds to the amount of parallel movement of the image based on the dimensional coordinate value. Therefore, the amount of geometric transformation corresponds to the magnitude of the value of the geometric transformation parameter.

  The geometric transformation applied to the template image 1 may be, for example, image distortion such as image distortion, blur amount, depth of focus, trapezoidal distortion, etc. in addition to the direction angle and the two-dimensional coordinate value. The geometric transformation applied to the template image 1 is not limited to the case where any one of the geometric transformations as described above is used, and a plurality of geometric transformations may be used in combination.

  In the image processing method of the present embodiment, a process for calculating eigenvalues and eigenfunctions by performing eigenvalue decomposition on the converted template image group is performed. Here, the same number of eigenvalues and eigenfunctions as the number of converted template images constituting the converted template image group are calculated.

  Therefore, as described above, when the direction angle is used as the geometric transformation applied to the template image 1 and a post-conversion template image group including 360 post-conversion template images is generated, the post-conversion unit consisting of the 360 images is generated. Eigenvalue decomposition is performed on the template image group, and 360 eigenvalues and eigenfunctions are obtained. Further, when a two-dimensional coordinate value is used as a geometric transformation applied to the template image 1 and a converted template image group including 64 converted template images is generated, the converted template image group including the 64 images. Is subjected to eigenvalue decomposition, and 64 eigenvalues and eigenfunctions are obtained.

As eigenfunctions calculated by performing eigenvalue decomposition, eigenfunctions φ ₁ (θ) to φ ₃ (θ) represented by waveforms p1 to p3 are obtained, for example, as shown in FIG. These eigenfunction groups can be said to be main eigenfunction groups. In each graph showing the waveforms p1 to p3, the horizontal axis represents the direction angle (θ) with reference to the template image 1, and the vertical axis represents the eigenfunction value. As shown in the waveforms p1 to p3, the waveform obtained as the eigenfunction is a continuous waveform in which eigenfunction values continuously exist with respect to the direction angle.

In the eigenfunction, as shown by the waveforms p1 to p3 in FIG. 3, the eigenfunction value periodically fluctuates as the direction angle changes. These eigenfunctions are calculated as functions (P _n (θ)) representing image similarity by multiplying correlation values (r _n ) described later in the image processing method of the present embodiment.

  As described above, in the image processing method of the present embodiment, after the first image group generation step, eigenvalue decomposition is performed on the converted template image group, so that the same number of eigenvalues and eigenfunctions as the plurality of converted template images are obtained. An eigenvalue decomposition step for calculating is performed.

  In the image processing method of the present embodiment, a process for generating a new image group by performing an inner product operation on the plurality of eigenfunctions calculated as described above and the converted template image group is performed. The new image group generated here corresponds to the second image group in the present embodiment. In the following description, the new image group generated here is referred to as “eigenvalue decomposition template image group”.

  As the eigenvalue decomposition template image group, for example, a plurality of images as shown in eigenvalue decomposition template images E1 to E3 in FIG. 3 are obtained. Each eigenvalue decomposition template image E1 to E3 corresponds to each waveform p1 to p3 representing an eigenfunction.

  As described above, in the image processing method of the present embodiment, the eigenvalue decomposition template image group is generated by performing the inner product operation of the plurality of eigenfunctions and the converted template image group after the eigenvalue decomposition step. An image group generation step is performed.

  In the image processing method of the present embodiment, processing for calculating the image similarity with respect to the eigenvalue decomposition template image group is performed for the input image 2.

  The principle of calculating the image similarity will be described. Here, the description will be made assuming that the geometric transformation applied to the template image 1 in the first image group generation step described above is a direction angle.

  The template image 1 is T (x, y), the input image 2 is f (x, y), and the template image T (x, y) is rotated by θ, that is, the converted template image is T (x, y; θ ), The image similarity g (θ) between the template image 1 and the input image 2 is calculated by the following equation (1). Note that x and y representing each image indicate two-dimensional coordinate values of pixels in the image.

When g (θ) is expanded with the eigenfunction sequence φ _n (θ), (n = 0, 1, 2,...) along the θ direction, the following equation (2) is obtained.

  The following equation (3) is obtained by exchanging the order of the integral operation of the above equation (2).

Here, E _n (x, y) defined as the following equation (4) is an image calculated by the inner product of the converted template image T (x, y; θ) and the eigenfunction φ _n (θ), that is, It becomes an eigenvalue decomposition template image.

The image represented by the above equation (4) can be calculated separately from the input image f (x, y). Therefore, by calculating E _n (x, y) in advance, the input image f (x, y) is given thereafter, and from the result of matching with E _n (x, y), all directions The corner image similarity (g (θ)) can be calculated.

As the image similarity (g (θ)) of omnidirectional angles, for example, as shown in FIG. 3, eigenfunctions φ ₁ (θ) to φ ₃ (θ) represented by waveforms p1 to p3 are at a predetermined ratio. A waveform qs obtained by adding together is calculated. As shown in the waveform qs, the waveform obtained as the image similarity (g (θ)) for all directional angles is a continuous waveform (continuous curve) in which the image similarity is continuously present for the directional angles. That is, the image similarity (g (θ)) represents the distribution of the image similarity in the direction angle (θ) direction.

Specifically, the image similarity (g (θ)) is calculated as follows. In the example shown in FIG. 3, the eigenfunctions φ ₁ (θ) to φ ₃ (θ) are multiplied by correlation values r _{1 to} r ₃ , respectively. By multiplying eigenfunctions φ ₁ (θ) to φ ₃ (θ) by correlation values r _{1 to} r ₃ , functions P ₁ (θ) to P ₃ (θ) representing image similarity are calculated. As shown in FIG. 3, the functions P ₁ (θ) to P ₃ (θ) are waveforms q1 in which waveforms p1 to p3 as eigenfunctions φ ₁ (θ) to φ ₃ (θ) are represented at a predetermined ratio. This predetermined ratio corresponds to the correlation values r _{1 to} r ₃ . In each graph showing the waveforms q1 to q3 representing the functions P ₁ (θ) to P ₃ (θ), the horizontal axis indicates the direction angle (θ) with reference to the template image 1, and the vertical axis indicates the image similarity. . Then, by adding the functions P ₁ (θ) to P ₃ (θ), the image similarity (g (θ)) represented by the waveform qs is calculated. Here, the correlation value r _n, a predetermined real number representing the correlation between each eigenvalue decomposition template image eigenvalue decomposition template images, if the input image 2, and the eigenvalue decomposition template image eigenvalue decomposition template images This is a predetermined real number representing the correlation with the input image 2 and is defined as the following equation (5).

Thus, the correlation value r _n, f (x, y) and the input image 2 represented by, the inner product value of the eigenvalue decomposition template image group represented by the formula (4). Therefore, from the above equations (3) and (5), the image similarity (g (θ)) is expressed by the following equation (6).
g (θ) = Σr _n × φ _n (θ) (6)

As described above, in the image processing method of the present embodiment, a process of calculating an image similarity that is represented by a continuous curve and indicates the similarity between the input image 2 and the eigenvalue decomposition template image group is performed. Further, the correlation value r _n used to calculate the image similarity may be referred to as "response value" for each eigenvalue decomposition template image for the input image 2. In this specification, to calculate a correlation value r _n from the input image 2 which eigenvalue decomposition template image group called "convolution", the operation for calculating the correlation value r _n of "convolution".

  In the image processing method according to the present embodiment, based on the image similarity (g (θ)) calculated as described above, the value (real value) of the geometric transformation parameter of the input image 2 based on the template image 1. The process of determining is performed. That is, in this process, the template image 1 and the input image 2 are collated using the image similarity (g (θ)) value calculated as described above as an index.

  Specifically, as shown in FIG. 3, in the waveform qs that is the image similarity (g (θ)), the direction angle θa that maximizes the image similarity is that of the input image 2 based on the template image 1. It is determined as the value of the geometric transformation parameter. That is, the direction angle of the input image 2 based on the template image 1 is calculated as the direction angle θa as a collation result of the image processing method of the present embodiment.

  Here, the direction angle θa obtained as a matching result corresponds to the rotation angle of the image when the direction angle of the template image 1 is set as a reference (0 °). That is, in this case, in the range in which the template image 1 is rotated by 360 °, the template image 1 in the state rotated by the angle θa has the highest image similarity in relation to the input image 2.

  Further, in the image processing method of the present embodiment, after the eigenvalue decomposition step for performing eigenvalue decomposition on the post-conversion template image group as described above, among the plurality of eigenfunctions calculated in the eigenvalue decomposition step, the eigenvalues are in descending order. The eigenfunction selection step of selecting a predetermined number of eigenfunctions is performed.

  In the image processing method of the present embodiment, the same number of eigenvalues and eigenfunctions as the plurality of post-conversion template images are calculated in the eigenvalue decomposition step. That is, as described above, when 360 converted template images are obtained by applying a rotation of 1 ° pitch as geometric transformation to the template image 1, 360 eigenvalues and eigenfunctions are calculated. In this case, when all eigenfunctions are used in generating the eigenvalue decomposition template image group, the eigenvalue decomposition template image group becomes 360 eigenvalue decomposition template images.

  Therefore, a predetermined number of eigenfunctions are selected from the same number of eigenfunctions as the converted template images calculated in the eigenvalue decomposition step. The selection of the eigenfunction is performed from the one with the larger eigenvalue. That is, in the eigenfunction selection step, a predetermined number of eigenfunctions calculated in the eigenvalue decomposition step are selected from the larger eigenvalue side. Therefore, for example, when 360 eigenfunctions are calculated as described above, 60 eigenfunctions with larger eigenvalues are selected. This is based on the fact that the eigenfunction having a larger eigenvalue has a greater influence (higher contribution) on the image similarity between the input image 2 and the eigenvalue decomposition template image group that is finally calculated.

  When such an eigenfunction selection step is performed, in the second image group generation step described above, an inner product operation of the plurality of eigenfunctions selected in the eigenfunction selection step and the converted template image group is performed. Thus, an eigenvalue decomposition template image group is generated. That is, according to the eigenfunction selection step, the eigenvalue decomposition template image group is generated by using a part of the eigenvalues calculated in the eigenvalue decomposition step and using some eigenfunctions on the side having the larger eigenvalue.

  Specifically, the eigenfunction selection step corresponds to approximating the series with a predetermined number in the above equation (3). Here, when the series of Expression (3) is approximated by M truncation, Expression (3) is expressed by the following Expression (7).

At this time, by selecting an eigenfunction sequence φ _n (θ) that can approximate g (θ) with a small order M, the image similarity at all angles is M images E _n (x, y ). Then, as a method of selecting the eigenfunction sequence φ _n (θ), the eigenvalue λ _n and the eigenfunction φ _n (θ) of the post-conversion template image sequence T (x, y; θ) are calculated. Select eigenfunctions.

Specifically, θ = {θ ₁ , θ ₂ ,..., Θ _N } and vector φ _n obtained by discretization, template image T _i (x, y) = T (x, y; θ _i ) Then, eigenvector φ _n and eigenvalue λ _n can be obtained by solving the matrix eigenvalue problem of the following equation (8).
Cφ _n = λ _n φ _n (8)

  Here, the matrix C is a covariance matrix in which the i-th and j-th elements are represented by the following equation (9).

In Expression (9), μ _i is an average density value of T _i (x, y). An image E _n (x, y) obtained using this eigenvector corresponds to an eigenvalue decomposition template image.

  In the image processing method for performing each processing as described above, the calculation of the image similarity (g (θ)) represented by the waveform qs shown in FIG. When it is performed for directional angles (all postures), the amount of calculation becomes enormous, and there is a problem in terms of processing time in practical use.

  Specifically, when calculating the image similarity (g (θ)) for the omnidirectional angle, the number of converted template images is N for one pixel of the input image 2 and the above series is obtained. When the approximate order in the truncation is M, N × M operations are required. Therefore, for example, when the number N of converted template images N = 360, the number of pixels of the input image 2 is 512 × 512 (pixels), and the approximate order M = 60, the omnidirectional angle ( When image similarity (g (θ)) is calculated for all orientations, 360 × 60 × 512 × 512 = 5,662,310,400 (times), that is, about 6 billion operations are required. .

  Therefore, in the image processing method of the present embodiment, when collating images for the input image 2, the correlation value as a result of convolution is used to determine the geometric transformation parameter to be determined. 3 is narrowed down, that is, the position (two-dimensional coordinate value) is narrowed down and the posture (direction angle) is narrowed down. As described above, in the image processing method according to the present embodiment, regarding the position (two-dimensional coordinate value), among the pixels of the input image 2, a pixel that is not expected to have the target image portion 3 is regarded as a target of calculation processing in the first place. In addition, regarding the posture (direction angle), after estimating an approximate value of θ, a detailed calculation is performed on the range of the value to perform a coarse / fine search for the geometric transformation parameter.

  First, of the above-described narrowing, the narrowing of positions (two-dimensional coordinate values) will be described. In the input image 2, a portion having a high degree of matching with the template image 1 has a high correlation value that is a response value of the zeroth-order eigenvalue decomposition template image that is the lowest order. The correlation value here is calculated as a value representing the correlation between each pixel of the input image 2 and each eigenvalue decomposition template image of the eigenvalue decomposition template image group.

Therefore, as shown in FIG. 4, convolution is performed as a comparison between the input image 2 having the target image portion 3 and the zeroth-order eigenvalue decomposition template image 4 which is the lowest-order image, and each pixel of the input image 2 is checked. , calculation of the correlation value r ₀ of the zero-order is a correlation value for the lowest order image. Specifically, a convolution operation is performed by the above equation (5), and a zero-order correlation value r ₀ is obtained.

FIG. 4 shows a 0th-order correlation image 5 obtained as a result of convolution of the input image 2 and the 0th-order eigenvalue decomposition template image 4. 0-order correlation image 5, 0-order response value with the pixel corresponding to each pixel of the input image 2, i.e. illustrates a zero-order magnitude of correlation value r ₀ in color density. In the example shown in FIG. 4, in the 0th-order correlation image 5, pixels with a larger correlation value r ₀ are shown in lighter colors. Accordingly, it is possible to determine that the white portion indicated by reference numeral “5a” in FIG. 4 is a portion having a relatively large correlation value r _{0 and a} portion where the target image portion 3 exists.

Therefore, in the narrowing down position, preset zero order correlation values r ₀ a predetermined threshold value for, among the pixel input image 2, the correlation value r ₀ of the zero-order, which are calculated over a pixel a predetermined threshold, It is set as a processing target such as the image similarity (g (θ)) calculation processing as described above. That is, regarding a pixel of the input image 2, when a predetermined threshold value is Ar, a pixel having a zero-order correlation value r ₀ ≧ Ar is set as a processing target such as an image similarity (g (θ)) calculation process. The other pixels, that is, the pixels satisfying the 0th-order correlation value r ₀ <Ar are excluded from the processing target. The value of the predetermined threshold value Ar is 30 or the like, for example.

Thus, as a narrowing position, 0 for order correlation value r ₀ is pixel less than a predetermined threshold value Ar, terminate the search for geometric transformation parameter is excluded from the processing target. Then, the correlation value r ₀ of the 0th order is processed only more pixels the predetermined threshold Ar, to search for the geometric transformation parameters.

As described above, in the image processing method of the present embodiment, for each pixel of the input image 2, the 0th-order eigenvalue decomposition which is the lowest-order image having the lowest order among the eigenvalue decomposition template images which are images of the eigenvalue decomposition template image group. Based on the 0th-order correlation value r ₀ calculated for the relationship between the template image 4 and the input image 2, a pixel having a relatively low 0th-order correlation value r ₀ is excluded from the processing target among the pixels of the input image 2. A pixel exclusion step is performed.

Incidentally, in the pixel excluded step to narrow down the location, in the example described above, using a predetermined threshold value Ar set correlation value r ₀ of the zero-order correlation value r ₀ of the 0th order are pixels less than the predetermined threshold value Ar Processing that is excluded from processing targets is adopted. In this respect, the process of the pixel excluding step, based on the correlation value r ₀ of the zero-order for each pixel in the input image 2, in process for removing the relatively low pixel correlation value r ₀ of the 0th order from processed If there is, the specific processing content is not limited. The treatment of pixel excluding step, for example, may be a correlation value r ₀ of the 0th order is a process of excluding from the processing target following pixel predetermined threshold Ar, also the calculated zero-order correlation value r ₀ A process in which a predetermined calculation is performed may be a comparison target of a predetermined threshold. As described above, the position (two-dimensional coordinate value) is narrowed down.

  Next, narrowing down of the posture (direction angle) will be described. Here, in explaining the narrowing of the posture, the relationship between the approximate order M in the series truncation as described above and the estimation accuracy of the direction angle will be described with reference to FIG.

  FIG. 5 shows a correlation waveform G1 approximated with an approximate order M = 60 and an approximate order M = 10 out of the correlation waveform g (θ) obtained as the image similarity (g (θ)) of omnidirectional angles. The correlation waveform G2 is shown. In FIG. 5, the correlation waveform G1 approximated by the approximate order M = 60 is shown by a solid line graph, and the correlation waveform G2 approximated by the approximate order M = 10 is shown by a one-dot chain line graph. In the graph shown in FIG. 5, the horizontal axis represents the direction angle (θ) as a geometric transformation parameter, and the vertical axis represents the image similarity (g (θ)). The position of the peak of such a correlation waveform graph is the direction angle (posture) to be determined.

  Regarding the direction angle as a geometric transformation parameter, the larger the value of the approximate order M, the higher the direction angle can be estimated. As shown in FIG. 5, in the correlation waveform G1 of the approximate order M = 60, the waveform irregularities appear larger and clearer than the correlation waveform G2 of the approximate order M = 10. That is, the correlation waveform G2 having the approximate order M = 10 has a gentle shape as a whole, whereas the correlation waveform G1 having the approximate order M = 60 has a relatively large degree of slowness.

  Therefore, as described above, when determining the direction angle (see FIG. 3, direction angle θa) that maximizes the image similarity in the waveform of the image similarity (g (θ)) as the value of the geometric transformation parameter, The correlation waveform G1 of the approximate order M = 60 having a large unevenness is clearer than the correlation waveform G2 of the approximate order M = 10, and the peak of the waveform is clear, and higher detection accuracy is obtained for the direction angle to be determined. However, the larger the approximate order value M, the greater the amount of calculation and the longer the processing time.

  On the other hand, even if the value of the approximate order M is small to some extent, a rough direction angle (posture) can be estimated. In the example shown in FIG. 5, the direction angle θa10 determined in the correlation waveform G2 of the approximate order M = 10 is located in the vicinity of the direction angle θa60 determined in the correlation waveform G1 of the approximate order M = 60. That is, a certain degree of detection accuracy can be obtained for the determined direction angle even with the correlation waveform G2 of the approximate order M = 10.

  Therefore, as narrowing down the posture, the direction angle is determined by the correlation waveform of the approximate order M (= 10) having a relatively small value, and the range in the vicinity of the determined direction angle is searched for the direction angle as the geometric transformation parameter. set to target. That is, the direction angle to be determined as the geometric transformation parameter, in other words, the direction angle determined by the correlation waveform of the approximate order M (= 60) that is somewhat large is the correlation waveform of the approximate order M (= 10) having a relatively small value. Since it can be said that it exists in the range in the vicinity of the direction angle determined by (1), the range in the vicinity is set as the target of the direction angle search. Then, a correlation waveform having a large approximate order M (= 60) is used for a range in the vicinity of the direction angle determined by the correlation waveform having a small approximate order M (= 10), which is a target of the direction angle search. The direction angle is estimated in detail. In the following description, the direction angle determined as the reference of the neighborhood range that defines the target of the direction angle in the process of narrowing down the posture is referred to as a “first estimated direction angle”.

In the example shown in FIG. 5, if the direction angle θa10 determined in the correlation waveform G2 of the approximate order M = 10 is the first estimated direction angle θ _A1 (°), the first estimated direction angle θ _A1 (° ) Is set as a range of θ _A1 ± 5 °, for example. In this case, the search target of the direction angle as the geometric transformation parameter is an angle range of 10 ° centered on the first estimated direction angle θ _A1 , and the search target for the direction angle is the omnidirectional angle, that is, 360. The angle is narrowed down to 1/36 with respect to the angle range. Note that a numerical value (± 5 ° or the like) that defines the vicinity range with respect to the first estimated direction angle is appropriately set according to the approximate order M or the like.

In the image processing method of the embodiment, used is the correlation value a _n for each eigenvalue decomposition template image for transformed template image, the first estimated direction angle is determined. Specifically, the first estimated direction angle is obtained by the following method.

  When obtaining the first estimated direction angle, a correlation value vector corresponding to each post-conversion template image of the post-conversion template image group is calculated by comparing the post-conversion template image and the eigenvalue decomposition template image. That is, the same number of correlation value vectors as the number of converted template images in the converted template image group are calculated. A group of vectors including these correlation value vectors corresponds to the dictionary created in the image processing method of the present embodiment as described above (see S30 in FIG. 2).

  As shown in FIG. 6, in the image processing method of the present embodiment, 360 converted template images (θ = 0 to 359) having different directional angles at 1 ° pitch are generated as a converted template image group. Corresponding to each of these converted template images, the same number (360) of correlation value vectors v0 to 359 is generated.

Each component of the correlation value vector and the template image after the conversion, the correlation value a _n obtained by convolution of the eigenvalue decomposition template images. Therefore, the correlation value vector is composed of the same number of components (dimensions) as the number of eigenvalue decomposition template images 4 to be convolved with the converted template image. That is, the correlation value vector after conversion template image corresponding to the value of each θ is its converted template image, the correlation value a _n calculated in relation to eigenvalue decomposition template image 4 of each order with the components . In other words, the correlation value vector is obtained by convolving the eigenvalue decomposition template image 4 with the converted template image itself, and the convolution response values of the eigenvalue decomposition template images 4 are arranged for each converted template image. Is a vector.

Specifically, in the above equation (5), instead of f (x, y) representing the input image 2, T (x, y, θ), which is a function representing the converted template image, is used. correlation value a _n is calculated for the transformed template image. That is, the correlation value _{a n} is defined as the following equation (10).

  In the example shown in FIG. 6, among the correlation value vectors, the correlation value vector v0 corresponding to the converted template image with θ = 0, the correlation value vector v30 corresponding to the converted template image with θ = 30, and θ = 170. A correlation value vector v170 corresponding to the converted template image is shown. For the correlation value vector v0 with θ = 0, the zeroth-order correlation value representing the correlation between the converted template image with θ = 0 and the zeroth-order eigenvalue decomposition template image E0 is “5.0”. Similarly, the primary correlation value representing the correlation with the primary eigenvalue decomposition template image E1 is “2.0”, and the secondary correlation value representing the correlation with the secondary eigenvalue decomposition template image E2 is , “3.3”, and the third-order correlation value representing the correlation with the third-order eigenvalue decomposition template image E3 is “4.4”.

  Similarly, for the correlation value vector v30 with θ = 30, the zeroth-order correlation value is “5.3”, the first-order correlation value is “−2.2”, and the second-order correlation value is , “6.3”, and the third-order correlation value is “2.4”. For the correlation value vector v170 of θ = 170, the zeroth-order correlation value is “4.3”, the first-order correlation value is “4.2”, and the second-order correlation value is “2”. .3 ”and the third-order correlation value is“ −2.4 ”. In this way, 360 correlation value vectors v0 to v359 are calculated in the same number as the converted template images.

As described above, in the image processing method of the present embodiment, each converted template image in the converted template image group is a predetermined real number that represents the correlation with each eigenvalue decomposition template image 4 in the eigenvalue decomposition template image group. correlation vector generation step of generating a correlation value vector v0~v359 that the correlation value a _n and the value of each component is carried out.

  As described above, the number of components of the correlation value vector generated in this correlation value vector generation step is the same as the number of eigenvalue decomposition template images 4 to be convolved with the template image group after conversion. In the present embodiment, when the eigenfunction selection step as described above is performed, the number of eigenvalue decomposition template images 4 generated as the eigenvalue decomposition template image group corresponds to the approximate order M. Accordingly, when all the generated eigenvalue decomposition template images 4 are to be convolved, the correlation value vector has the same number of components as the number of approximate orders M. That is, in this case, when the approximate order M = 60, the number of components of the correlation value vector is 60.

  The number of components of the correlation value vector is preferably smaller than the approximate order M from the viewpoint of reducing the amount of calculation and improving the processing speed. Therefore, in the correlation value vector calculation process, K (<M) eigenvalue decomposition template images 4 that are a part of all the generated M eigenvalue decomposition template images 4 are selected and used. .

  Specifically, for example, when the approximate order M = 60, in other words, when 60 eigenvalue decomposition template images 4 are generated, about 10 to 20 eigenvalue decomposition template images 4 are selected, and the correlation value It is an object of convolution in the vector calculation process. That is, the dimension of the correlation value vector is K (<M) dimension. When K = 10, the number of components of the correlation value vector is 10.

  In the selection of the eigenvalue decomposition template image 4 as described above, for example, the size of the eigenvalue is used as in the eigenfunction selection step in the eigenfunction selection step as described above. That is, a predetermined number of eigenvalue decomposition template images 4 corresponding to eigenfunctions selected from the larger eigenvalues among the eigenvalue decomposition template images 4 having the same number as the approximate order M generated in the second image group generation step. Selected. Therefore, for example, when the selection number K of the eigenvalue decomposition template images 4 is 10, ten eigenvalue decomposition template images 4 corresponding to ten eigenfunctions are selected from the side having the larger eigenvalues. This is based on the fact that the eigenfunction having a larger eigenvalue has a greater influence (higher contribution) on the image similarity between the input image 2 and the eigenvalue decomposition template image group, which is finally calculated, as described above.

  As described above, in the image processing method of the present embodiment, when performing the correlation value vector generation step, among the eigenvalue decomposition template images 4 of the eigenvalue decomposition template image group generated in the second image group generation step, An image selection step of selecting a predetermined number (K) of eigenvalue decomposition template images 4 in descending order of eigenvalues is performed.

  The predetermined number (K) for the eigenvalue decomposition template image 4 selected in this image selection step is the approximate order M corresponding to the number of all eigenvalue decomposition template images 4 to be generated, that is, in the eigenfunction selection step described above. It is preferable that the number is smaller than the predetermined number. Therefore, when the “predetermined number” for the number of eigenfunctions selected in the eigenfunction selection step is set to “first predetermined number”, the number of eigenvalue decomposition template images 4 selected in this image selection step is The “predetermined number” is preferably “a second predetermined number smaller than the first predetermined number”. That is, even when the eigenfunction selection step described above is not performed, this image selection step is performed, and a part of the eigenvalues are generated from all (N) eigenvalue decomposition template images generated in the same number as the converted template images. By selecting the decomposition template image 4, it is possible to reduce the amount of calculation in the correlation value vector generation process.

  When the image selection step is performed, the correlation value vector generation step generates a correlation value vector using the K (<M) eigenvalue decomposition template images 4 selected in the image selection step. That is, in this case, in the correlation value vector generation step, N correlation value vectors v0 to v359 having K components are obtained by convolution of N converted template images and K eigenvalue decomposition template images 4. Is generated.

  Such calculation of the correlation value vector is performed for each pixel with respect to all the pixels to be processed (hereinafter referred to as “processing target pixels”) other than the pixels excluded by the pixel exclusion step described above.

  After N correlation value vectors have been calculated as described above, a feature vector to be compared with the correlation value vector is generated. The feature vector is calculated for each pixel for all the processing target pixels, and the correlation value of the eigenvalue decomposition template image 4 with respect to the input image 2 at each pixel position is used as each component.

Each component of the feature vector includes an input image 2, the correlation value r _n obtained by convolution of the eigenvalue decomposition template images. Therefore, the feature vector includes the same number of components (dimensions) as the number of eigenvalue decomposition template images 4 to be convolved with the input image 2. That is, the feature vector uses the correlation value rn calculated in the relationship between the input image 2 and the eigenvalue decomposition template image 4 of each _order as each component. In other words, the feature vector is a vector in which the convolution response values of the eigenvalue decomposition template images 4 are arranged for the input image 2.

Correlation value r _n of the respective components of the feature vector is calculated by the equation (5). When the feature vector Vt = {t0, t1, t2,..., T (K−1)}, t0 is a zeroth-order correlation value representing the correlation between the input image 2 and the zeroth-order eigenvalue decomposition template image E0. r ₀ and t1 are primary correlation values r ₁ and t2 representing the correlation between the input image 2 and the primary eigenvalue decomposition template image E1, and 2 is a correlation between the input image 2 and the secondary eigenvalue decomposition template image E2. The next correlation value r ₂ , t (K−1) becomes the (K−1) th order correlation value r _K−1 representing the correlation between the input image 2 and the (K−1) th order eigenvalue decomposition template image.

  Such a feature vector Vt is obtained for all pixels of the processing target pixel as described above, and is obtained as a result of image collation by translating K eigenvalue decomposition template images 4 to the position of the processing target pixel. It can be said that. Here, translating K eigenvalue decomposition template images 4 to the position of the processing target pixel converts the center position of each eigenvalue decomposition template image 4 to the two-dimensional coordinate value (XY coordinate value) of the processing target pixel. Equivalent to matching.

Thus, in the image processing method of this embodiment, the input image 2, feature vector generating step of generating a feature vector Vt of the correlation value r _n and the value of each component is carried out.

  As described above, after N correlation value vectors and feature vectors Vt are generated for each pixel of the processing target pixel, one correlation value vector is obtained from the N correlation value vectors using the feature vector Vt. A process of selecting is performed. Specifically, a correlation value vector that is closest to the feature vector Vt is selected from N correlation value vectors.

FIG. 7 schematically shows a state where the correlation value vector and the feature vector are plotted in the coordinate space. In FIG. 7, for convenience, the coordinate space is indicated by three axes of three correlation values a ₀ , a ₁ , and a ₂ , but the correlation value vector and the feature vector Vt are both K-dimensional, and the coordinate space is a correlation value. It is represented by a K-dimensional space of a _{0 to} a _(K-1) . Further, in FIG. 7, for convenience, five correlation value vectors from θ = 0 to 5 are shown for the correlation value vector, but actually there are 360 correlation value vectors from θ = 0 to 359.

  In this way, among the correlation value vectors and feature vectors Vt plotted in the common coordinate space, the correlation value vector that is closest to the feature vector Vt is selected. In the example shown in FIG. 7, the correlation value vector that is closest to the feature vector Vt is the correlation value vector V [i] 0 with θ = 0.

  As a method for obtaining a correlation value vector that most closely approximates the feature vector Vt, a method based on an algorithm called approximate neighborhood search or the like is used. As a method for obtaining a correlation value vector that is closest to the feature vector Vt, a well-known method can be used as appropriate in addition to the approximate neighborhood search, as long as it is a method for searching for the closest point from a set of points in the metric space. it can. Examples of the technique used here include a technique using a search tree, such as a technique using Kd-Tree, a technique using K-means, and a technique using vector quantization. As another example, there is a method using a table (replacement table, hash table method, etc.) that describes the correspondence between multidimensional vector values {a0, a1,..., AN} and output values corresponding thereto. Can be mentioned.

  As described above, in the image processing method of the present embodiment, a vector selection step for selecting a correlation value vector that is closest to the feature vector Vt from among the plurality of correlation value vectors generated in the correlation value vector generation step is performed. .

  In this vector selection step, the posture (direction angle) of the input image 2 is determined by comparing vectors of response values obtained by convolving the eigenvalue decomposition template image 4 for each of the N converted template images and the input image 2. Roughly explored. That is, a search for a dictionary that is a vector group of correlation value vectors is performed.

Then, the value of θ of the correlation value vector selected in this vector selection step becomes the first estimated direction angle θ _A1 described above. The value of θ of the correlation value vector selected in the vector selection step is not sufficiently accurate as the value of the direction angle as the geometric transformation parameter to be finally determined, but the first estimated direction angle θ _A1 has sufficient accuracy.

  The angle range in the vicinity of the first estimated direction angle obtained in this way is a target for searching for a direction angle as a geometric transformation parameter. As described above, the posture (direction angle) is narrowed down. The narrowing down of the posture is performed for all the pixels to be processed.

The range of the value of the direction angle θ narrowed by narrowing down the posture as described above, that is, the range of the first estimated direction angle θ _A1 ± α (°) (hereinafter referred to as “the narrowed angle range θ _A1 ± α”). The value of the direction angle is searched for by a detailed calculation. That is, the calculation of the image similarity (g (θ)) using the approximate order M (= 60) that is somewhat large as described above is performed only for the narrowing angle range θ _A1 ± α, and the image similarity ( The direction angle (see FIG. 3, direction angle θa) that maximizes the image similarity is calculated from the correlation waveform of g (θ)). Then, the calculation of the direction angle that maximizes the image similarity is performed for all the pixels to be processed, and among these pixels to be processed, the pixel having the maximum image similarity that determines the direction angle is the largest. The two-dimensional coordinates of the pixel and the direction angle determined for the pixel are determined as a position (two-dimensional coordinate value) and a posture (direction angle) as geometric transformation parameters, respectively.

Therefore, first, the correlation waveform of the image similarity (g (θ)) is calculated for the narrowed angle range θ _A1 ± α with an approximate order M that is somewhat large. Specifically, in the correlation waveform as the image similarity (g (θ)) calculated by the above formula (3) or formula (7), the portion of the narrowed angle range θ _A1 ± α is calculated. . The value of α is appropriately set depending on the value of the approximate order M or the like. From the viewpoint of reducing the amount of calculation and improving the processing speed, the value of α is 10 (°) or less, preferably 5 (°) or less. Is set.

FIG. 8 shows an example when the approximate order M = 60. The correlation waveform G1 shown in FIG. 8 is a correlation waveform calculated as M = 60 in the above equation (7). Of the correlation waveform G1 of the approximate order M = 60, a portion G1a (portion of the range B1) in the narrowed angle range θ _A1 ± α indicated by a solid line is calculated as a correlation waveform as the image similarity (g (θ)). The

That is, referring to FIG. 8, when the image similarity is calculated for all directional angles (0 to 359 °), the entire waveform including the portion indicated by the two-dot chain line in the correlation waveform G1 is calculated. The range of the direction angle to be searched is narrowed down by narrowing down the posture as described above, and the correlation waveform is calculated only for the portion G1a of the narrowed angle range θ _A1 ± α, which is the angle range obtained by narrowing down the posture. Image similarity is calculated.

As described above, in the image processing method of the present embodiment, the value of the first estimated direction angle θ _A1 that is the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selection step for the processing target pixel. An image similarity calculation step for calculating an image similarity (g (θ)) in the vicinity range, that is, the narrowing angle range θ _A1 ± α is performed.

Next, based on the correlation waveform of the image similarity (g (θ)) of the narrowed-down angle range θ _A1 ± α calculated as described above, a position (two-dimensional coordinate) as a geometric conversion parameter as a final detection result is obtained. Value) and posture (direction angle) values are determined. Specifically, it is as follows.

The number of pixels of the input image 2 is 512 × 512, and the processing target pixels after being excluded in the pixel exclusion step described above are 65,000 pixels, which is about ¼ of the total number of pixels. In this case, the orientation narrowing as described above is performed for all the 65,000 pixels, and the image similarity (g (θ)) of the narrowing angle range θ _A1 ± α is calculated. Therefore, for each pixel of 65,000 pixels, the direction angle that maximizes the image similarity (g (θ)) is calculated from the narrowed angle range θ _A1 ± α. The direction angle corresponding to the maximum image similarity (g (θ)) in the narrowed angle range θ _A1 ± α is defined as “second estimated direction angle θA2”. Among the 65,000 pixels, the position of the pixel having the largest image similarity (hereinafter referred to as “maximum image similarity”) corresponding to the second estimated direction angle θ _A2 and the second of the pixel. the estimated direction angle theta _A2 are respectively determined position of the value of the final geometric transformation parameters (2-dimensional coordinate value) and a posture (direction angle).

As described above, in the image processing method of the present embodiment, the position (two-dimensional coordinate value) and orientation (direction angle) values, which are geometric transformation parameters, are calculated based on the image similarity calculated in the image similarity calculation step. A matching step to determine is performed. That is, the collation step determines the position of the pixel having the largest maximum image similarity calculated by the image similarity calculation step among the processing target pixels as the value of the two-dimensional coordinate value of the geometric transformation parameter, and determines the maximum image similarity. The direction angle (second estimated direction angle θ _A2 ) corresponding to the maximum image similarity for the pixel having the largest degree is determined as the direction angle of the geometric transformation parameter. In such a collation step, focusing on the continuity of the image similarity resulting from the continuity of the eigenfunction, the real value of the geometric transformation parameter is determined.

  As described above, the image processing method of the present embodiment relies on the correlation value obtained by convolution with the eigenvalue decomposition template image, and the position (two-dimensional coordinate value) where the target image portion 3 exists in the input image 2, and By narrowing down the posture (direction angle) of the target image portion 3 in a multistage manner, the calculation amount is reduced while maintaining the detection accuracy of the geometric transformation parameters. The image collating method of this embodiment can be said to be a processing method that performs the following three stages of collation (search).

In the first stage, there is a high possibility that the target image portion 3 is present with respect to the input image 2 by threshold processing based on the convolution response value (correlation value r ₀ ) with the 0th-order eigenvalue decomposition template image. This is a process of narrowing down the region (XY position) and determining the processing target pixel. Pixels other than the processing target pixel determined by such processing are excluded from the geometric transformation parameter search target. That is, the first-stage process is a process for narrowing down the position (two-dimensional coordinate value) described above, and corresponds to the process in the pixel exclusion step.

  The second level is a correlation value vector obtained by convolution response values with K (<M) eigenvalue decomposition template images 4 less than the approximate order M for each of the N converted template images and the input image 2. This is a process of narrowing down the range of the direction angle to be searched based on the feature vector Vt and estimating a rough posture (direction angle). In other words, this second stage process is the process of narrowing down the posture (direction angle) described above, and the process in each step of the correlation value vector generation step, the image selection step, the feature vector generation step, and the vector selection step. Including.

  In the third stage, the positions (two-dimensional coordinate values) and postures (direction angles) narrowed down by the processes in the first and second stages are convolved with M eigenvalue decomposition template images 4 as in the conventional case. The correlation value is calculated from the response value of, and a precise search is performed based on the image similarity. That is, the third stage processing corresponds to the processing in the above-described image similarity calculation step.

  According to the image processing method of the present embodiment as described above, the correlation value indicating the degree of similarity of images calculated based on the eigenvalue decomposition template image 4 obtained by eigenvalue decomposition of the plurality of template images 1 is obtained. In the image collation technique used, the calculation amount can be greatly reduced while maintaining the accuracy, and the processing speed can be improved.

  Further, in the image processing method of the present embodiment, as described above, the eigenfunction selection step of selecting a part of the eigenfunctions in descending order of the eigenvalues is performed, thereby further reducing the processing time while ensuring the detection accuracy. Can be achieved. That is, according to the eigenfunction selection step according to the present embodiment, among the same number (N) of eigenfunctions as the converted template image, some (M pieces) of which the influence (contribution) on the image similarity is large. ) Eigenfunction is used. Thereby, in comparison with the case where all of the calculated eigenfunctions are used, sufficient accuracy can be ensured for the determined geometric transformation parameters, and the processing speed can be increased by the eigenfunctions not selected. The processing time can be further shortened.

  In this regard, in the image processing of this embodiment, the processing time and the detection accuracy can be adjusted by adjusting the value of the approximate order M. That is, when the detection accuracy is prioritized over the reduction of the processing time, the value of the approximate order M is set larger, and conversely, when the reduction of the processing time is prioritized over the detection accuracy, the value of the approximate order M is set. Set to a smaller value.

  Furthermore, in the image processing method of the present embodiment, as described above, when performing a narrowing process of the posture (direction angle), an image selection step of selecting a part of the eigenvalue decomposition template image 4 in descending order of eigenvalues is performed. Further, the processing time can be further shortened while ensuring the detection accuracy. That is, according to the image selection step according to the present embodiment, some (K) eigenvalue decomposition template images having a large influence (contribution) on the image similarity among the same number (e.g., M) of eigenfunctions. 4 is used. Thereby, in comparison with the case where all of the calculated eigenvalue decomposition template images 4 are used, sufficient accuracy can be ensured for the determined geometric transformation parameters, and the portions of the eigenvalue decomposition template images 4 that are not selected can be obtained. The processing speed can be increased and the processing time can be further shortened.

  The following experimental results have been obtained by the image processing method of the present embodiment. In the case of using the conventional method, that is, the method in which the processing target is all pixels of the input image 2 and the direction angle search target is the omnidirectional angle, the processing time (calculation time) is 7000 [msec] According to the image processing method of the present embodiment, an experimental result has been obtained that the processing time is 500 [msec]. According to this experimental result, according to the image processing method of the present embodiment, the speed can be increased by about 10 times compared to the conventional method. The result of this experiment is used in the process of narrowing down the orientation (direction angle) of 512 × 512 pixels, where the image size of the input image 2 is the standard size, the number of converted template images N = 360, the approximate order M = 60. This is obtained under the condition that the number K of eigenvalue decomposition template images 4 is K = 20.

  In addition, according to the image processing method of the present embodiment, the same accuracy can be achieved with a small number of matching times compared to a conventional image matching technique using geometric transformation such as a rotation matching method in which matching is performed while rotating a template image. Can be secured, and the processing time can be greatly shortened.

  In the image processing method of the present embodiment, the following method can be employed. In the image processing method of the present embodiment, as the number of post-conversion template image groups obtained by applying geometric transformation to the template image 1 as described above increases, the continuity of the eigenfunction increases. Therefore, in the image processing method of this embodiment, the number of post-conversion template images can be increased to infinity. That is, in the series of processes in the image processing method of the present embodiment as described above, the number of converted template images constituting the converted template image group is mathematically infinite.

In this case, the above equation (8) of the matrix eigenvalue problem becomes the following equation (11).

In the above formula (11), K (s, t) is represented by the following formula (12).

  That is, instead of the matrix eigenvalue problem, equation (11) is converted to a differential equation problem. By increasing the number of template images after conversion to infinity, the eigenfunction becomes a continuous function in a true sense. For example, this continuous function can be approximated by a polynomial and expressed analytically.

  By expressing the eigenfunction by a polynomial, an image similarity function (image similarity g) that is a weighted sum of eigenfunctions using correlation values is also expressed by a polynomial. The process of finding the maximum value of the image similarity function expressed by a polynomial may be performed by differentiating the polynomial and finding a position where it becomes zero. In this way, by increasing the number of post-conversion template images to infinity, it is possible to analytically determine the estimated value of the parameter, and the estimation accuracy of the value of the geometric parameter can be greatly increased.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include a direction angle and a two-dimensional coordinate value, and the geometric transformation applied to the template image 1 is a rotation that changes the direction angle. It is preferable to include a translation that changes the two-dimensional coordinate values.

  The direction angle and the two-dimensional coordinate value are geometric transformation parameters that are relatively frequently used in industrial image matching processing, and are two-dimensional geometric transformation parameters. The image processing method of the form can be performed. For this reason, high versatility can be obtained.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include parameters necessary for projective transformation, and the geometric transformation applied to the template image 1 includes projective transformation.

  The projective transformation is a general projective transformation including an affine transformation, and converts the coordinates (x, y) of the original image into the coordinates (x ′, y ′) on the projection plane by a predetermined transformation formula. It is. Accordingly, the parameters necessary for the projective transformation are coefficients in the general formula of the projective transformation.

  Therefore, when projective transformation is used as the geometric transformation applied to the template image 1 in the image processing method of the present embodiment, the projection transformation is added to the template image 1 in the first image group generation step, so that A template image group is generated. Then, in the collating step, the value of the coefficient in the general formula of the projective transformation is determined as the value of the geometric transformation parameter of the input image 2.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include parameters necessary for the image blurring process, and the geometric transformation applied to the template image 1 performs the image blurring process. Including.

  The image blurring process is a process for blurring an image, and is performed as follows, for example. As an image blurring process, for example, an average of colors (color values) of 3 × 3 pixels (9 pixels) centering on a pixel (x, y) to be processed is used as a color (color value) of the pixel to be processed. Is performed. Such processing is also referred to as smoothing. The image blurring process is performed, for example, for each of R (red), G (green), and B (blue) components constituting the pixel color. As described above, when the average of the colors of 3 × 3 pixels is adopted as the color of the pixel to be processed, the weights of the pixels in the averaging process of 9 pixels are all the same (“1”).

  Therefore, in image blurring processing, the weight of each of the surrounding 8 pixels is doubled with respect to the pixel to be processed, or the average of the colors of only four pixels above, below, left, and right of the pixel to be processed is processed. It may also be used as the color. In the former case, the weight of the pixel to be processed is “1” and the weight of each surrounding pixel is “2”. In the latter case, the weight of the upper, lower, left, and right pixels of the pixel to be processed is “1”. The weight of each pixel of 5 pixels is “0”.

  Thus, the weight of each pixel (numerical value such as “1” or “2”) in the averaging process performed in the image blurring process is a parameter necessary for the image blurring process. In addition, the number of pixels in the range to be averaged as the color of the pixel to be processed (vertical N × horizontal N (pieces)) is also a parameter necessary for the image blurring process. Note that the greater the value of N, the greater the degree of image blur. Parameters necessary for such an image blurring process are included in the geometric transformation parameters determined for the input image 2.

  Therefore, in the image processing method of the present embodiment, when image blurring processing is used as geometric transformation applied to the template image 1, image blurring processing is added to the template image 1 in the first image group generation step. Thus, a post-conversion template image group is generated. In the collation step, the value of the parameter necessary for the image blurring process as described above is determined as the value of the geometric transformation parameter of the input image 2. The image blurring process is not limited to the image blurring process described above, and a well-known image blurring process can be appropriately employed.

  As described above, in the image processing method of the present embodiment, by using projective transformation and image blurring processing, it is possible to shorten processing time and improve detection accuracy in more advanced image matching processing. Become.

  Specifically, with respect to the input image 2, when the part to be inspected is imaged, the imaging direction by the imaging unit such as a camera is not perpendicular to the imaging plane set in the inspection object. In the image 2, there is a case in which a part to be inspected is distorted into a trapezoid (so-called trapezoidal distortion). Such trapezoidal distortion that occurs in the input image 2 causes detection errors.

  Therefore, by using projective transformation in the image processing method of the present embodiment as described above, it is possible to accurately perform image matching even when trapezoidal distortion occurs in the input image 2. That is, by preparing in advance a plurality of template images 1 that have undergone projective transformation as a group of converted template images, an eigenvalue decomposition template image that takes into account the effect of trapezoidal distortion can be generated, and errors due to trapezoidal distortion are small. Accurate image matching can be performed.

  Similarly, regarding the input image 2, “blur” is included in the input image 2 due to the distance between the imaging means such as a camera and the inspection object. Normally, for such blurring of an image, adjustment (focusing) with movement of a camera or a lens is performed by autofocus processing or the like. However, such adjustment with movement of the camera and lens is a mechanical process, so that the processing time becomes long.

  Therefore, as described above, by using the image blurring process in the image processing method of the present embodiment, even if the input image 2 is blurred, a template that has been subjected to the blurring process in various patterns in advance. By preparing the image 1, it is possible to cope with it. That is, by preparing in advance a plurality of template images 1 subjected to blurring processing as a template image group after conversion, an eigenvalue decomposition template image that takes into account the effect of image blur can be generated, and errors due to image blurring can be generated. Therefore, it is possible to perform accurate image matching with a small amount of image. In addition, since processing that causes longer image processing time, such as autofocus processing, can be omitted, processing time for image matching can be shortened.

  The blurring process will be further described. In the actual inspection using the image processing method of the present embodiment, the input image 2 is an image caused by a focus shift of the lens of the imaging unit in addition to geometric deformation such as noise and rotation of the inspection object. Blurring occurs. When image collation using the template image 1 is performed on the input image 2 which is a blurred image, the positional deviation becomes large. That is, when the input image 2 is blurred, the image matching between the template image 1 that is not blurred and the blurred input image 2 is performed, the image matching is not successful, and a positional deviation occurs. As described above, misalignment caused as a result of image collation is a serious problem.

  Therefore, by applying blurring processing to the template image 1 as described above, the template image 1 that is intentionally blurred is used for image matching, so that even if the input image 2 is blurred, the input image 2 and The template image 1 can be matched. Thereby, it is possible to avoid the problem of misalignment due to blurring of the input image 2. That is, matching accuracy can be increased by adding blur to the template image 1 in accordance with the blurred input image 2.

  In general, the image blurring process is used for noise removal. However, in the image processing method according to the present embodiment, blurring is performed by performing a blurring process on the template image 1 that is the target of matching of the input image 2. Is used to create a template image 1 to which is added. Thus, the image processing method of the present embodiment is characterized by the use of the image blurring process.

  Regarding image blur, there can be many variations from small image blur to large image blur. In order to cope with such a large number of variations in the image, it is necessary to prepare a large number of template images 1 to which the blur has been added in accordance with the variation in the image blur. As described above, if image collation using a large number of template images 1 corresponding to the blur of all images is performed, it is considered that the calculation time becomes very long.

  In this regard, in the image processing method of the present embodiment, a template image group with various blurs is input as a template image after conversion, and these are compressed using eigenvalue decomposition to reduce the calculation time as described above. The problem can be solved. That is, since the post-conversion template image is information-compressed by eigenvalue decomposition, it is possible to suppress an increase in calculation time even if there are many variations in image blur.

  Note that the projective transformation and the image blurring processing as geometric transformation are used independently or in combination in relation to the above-described directional angle and two-dimensional coordinate values.

[Image processing device]
Next, as an application example of the image processing method of the present embodiment, an apparatus for performing the image processing method of the present embodiment will be described. In this application example, for example, in an IC chip production line, image verification is used in the process of positioning the IC chip on the IC substrate.

  As shown in FIGS. 9 and 10, the image processing apparatus 10 of the present embodiment includes a computer 11 and a camera 12. The image processing apparatus 10 images the IC chip 20 as a work with the camera 12, and uses the captured image as an input image to perform image collation using a template image prepared in advance in the computer 11. The image processing apparatus 10 according to the present embodiment detects a directional angle and a two-dimensional coordinate value for an input image as an image matching result.

  Then, by using the image collation result by the image processing apparatus 10, the IC chip 20 picked by the arm 17 from the IC chip supply tray 16 accommodating the plurality of IC chips 20 is positioned at a predetermined position on the IC substrate. Is done. The arm 17 has a configuration for picking up components, such as a robot arm or a picking arm that constitutes a part of a device that transports the IC chip 20.

  Specifically, the positioning of the IC chip 20 is performed as follows. First, a predetermined IC chip 20 in the IC chip supply tray 16 is picked by the arm 17 (see arrow A1). The picked IC chip 20 is captured by the camera 12 while being held by the arm 17. The captured image of the IC chip 20 is transmitted to the computer 11 as an input image, and image verification is performed using a template image.

  The direction angle and two-dimensional coordinate value of the input image as an image collation result by the image processing apparatus 10 correspond to the angle and position of the IC chip 20 that is the imaging target. Therefore, the arm 17 holding the IC chip 20 is rotated and translated when positioning the IC chip 20 based on the result of image collation by the image processing apparatus 10. After the arm 17 is rotated and translated, the IC chip 20 is positioned at a predetermined position on the IC substrate as the arm 17 moves in the vertical direction. Instead of rotating and translating the arm 17, a stage capable of rotating and translating may be adopted as a configuration for supporting the IC chip 20.

  As shown in FIG. 10, the computer 11 includes an arithmetic control unit 13, an input unit 14, and a display unit 15. The arithmetic control unit 13 controls a series of operations of the image processing apparatus 10. The calculation control unit 13 includes a storage unit that stores a program and the like, a development unit that expands the program and the like, a calculation unit that performs a predetermined calculation according to the program and the like, a storage unit that stores calculation results and the like by the calculation unit, and the like.

  Specifically, a configuration in which a CPU, ROM, RAM, HDD, or the like is connected by a bus, or a configuration that includes a one-chip LSI or the like is used as the arithmetic control unit 13. As the arithmetic control unit 13, in addition to a dedicated product, a commercially available personal computer, a workstation or the like in which the above program is stored is used.

  The input unit 14 is connected to the calculation control unit 13 and inputs various information / instructions related to image processing to the calculation control unit 13. For example, a keyboard, a mouse, a pointing device, a button, a switch, or the like is used as the input unit 14.

  The display unit 15 is connected to the calculation control unit 13 and displays an operation status of image processing, an input content from the input unit 14 to the calculation control unit 13, a processing result by the image processing, and the like. For example, a liquid crystal display, a CRT (cathode ray tube), or the like is used as the display unit 15.

  The camera 12 is a CCD camera, for example, and is connected to the computer 11 via a connection cable. The camera 12 takes an image of the IC chip 20 to be picked by the arm 17 by moving relative to the IC chip 20 in the IC chip supply tray 16. In the present embodiment, the camera 12 captures the IC chip 20 from above and captures a planar image of the IC chip 20.

  Imaging data of the IC chip 20 captured by the camera 12 is sent to the computer 11 and input to the arithmetic control unit 13 as an input image 22 (see FIG. 11). As described above, in the image processing apparatus 10 of the present embodiment, the camera 12 functions as an input image acquisition unit for acquiring the input image 22.

  As shown in FIG. 11, in the image processing apparatus 10 of the present embodiment, the arithmetic control unit 13 provided in the computer 11 performs collation using a template image 21 that is an image of the IC chip 20 as a collation target. Thus, it functions as an image processing means for determining the value of the geometric transformation parameter of the target image portion 23 of the IC chip 20 existing in the input image 22. Therefore, the program stored in the storage unit in the arithmetic control unit 13 includes a program for performing image matching between the template image 21 and the input image 22 captured by the camera 12.

  As a storage part of the program or the like in the arithmetic control unit 13, for example, a storage device built in the computer 11 such as a RAM, a CD (Compact Disk), an FD (Flexible Disk), an MO (Magneto-Optical Disk), a DVD ( A storage device such as a digital versatile disk (HD) or a hard disk (HD) is appropriately used.

  As illustrated in FIG. 10, the arithmetic control unit 13 includes an eigenvalue decomposition template image creation processing unit 31, a narrowing processing unit 32, and a detection processing unit 33. The eigenvalue decomposition template image creation processing unit 31 generates a converted template image group based on a template image 21 stored in advance, performs eigenvalue decomposition on the generated converted template image group, and converts the calculated eigenfunction and conversion An eigenvalue decomposition template image group is created from the subsequent template image group.

The narrowing-down processing unit 32 performs processing for narrowing down the position (two-dimensional coordinate value) and narrowing down the posture (direction angle) as described above for the input image 22. That is, the narrowing-down processing unit 32 determines the processing target pixel of the input image 22 by narrowing down the position, and calculates the first estimated direction angle θ _A1 for all the pixels of the processing target pixel in narrowing down the posture. The narrowing angle range θ _A1 ± α is derived.

The detection processing unit 33 performs the eigenvalue decomposition generated by the input image 22 and the eigenvalue decomposition template image creation processing unit 31 in the narrowing angle range θ _A1 ± α for each pixel of the processing target pixel obtained by the narrowing processing unit 32. A correlation value with the template image group is calculated. The detection processing unit 33 calculates the image similarity from the calculated correlation value, and determines the value of the geometric transformation parameter of the input image 22 based on the calculated image similarity.

  As shown in FIG. 10, the calculation control unit 13 includes a template image storage unit 41, a post-conversion template image generation unit 42, an eigenvalue decomposition unit 43, an eigenfunction selection unit 44, and an eigenvalue decomposition template image generation unit 45. The image selection unit 46, the correlation value vector generation unit 47, the pixel exclusion unit 48, the feature vector generation unit 49, the vector selection unit 50, the image similarity calculation unit 51, and the collation unit 52.

  The template image storage unit 41 stores the template image 21 (see FIG. 11). For example, the template image 21 is obtained by imaging the same components as the IC chip 20 to be imaged by the camera 12 from the same angle as the camera 12. The template image 21 is a target to be collated with the input image 22 as an image indicating the direction angle and the reference of the two-dimensional coordinate value in the step of positioning the IC chip 20 on the IC substrate.

  Specifically, the template image 21 is created so as to have the same size as the portion occupied by the IC chip 20 in the input image 22 acquired by the camera 12, that is, the target image portion 23. In other words, for the camera 12, the IC chip 20 is such that the target image portion 23, which is the portion occupied by the IC chip 20 in the input image 22 to be acquired, is comparable to the portion occupied by the IC chip 20 in the template image 21. The imaging position is set. Such a template image 21 is preset and stored in the template image storage unit 41 of the calculation control unit 13.

  The post-conversion template image generation unit 42 performs processing corresponding to the processing in the first image group generation step in the above-described image processing method. That is, the post-conversion template image generation unit 42 generates a post-conversion template image group as the first image group by performing geometric transformation on the template image 21 stored in the template image storage unit 41.

  The post-conversion template image generation unit 42 performs rotation or the like for changing the direction angle as the geometric transformation applied to the template image 21. As described above, in the image processing apparatus 10, the post-conversion template image generation unit 42 adds a plurality of (N) post-conversion template images having different amounts of geometric transformation by applying geometric transformation to the template image 21. It functions as a first image group generation unit that generates a template image group after conversion.

  The eigenvalue decomposition unit 43 performs a process corresponding to the process in the eigenvalue decomposition step in the image processing method described above. That is, the eigenvalue decomposition unit 43 calculates eigenvalues and eigenfunctions by performing eigenvalue decomposition on the converted template image group.

  The eigenvalue decomposition unit 43 calculates the same number of eigenvalues and eigenfunctions as the converted template images constituting the converted template image group. As described above, in the image processing apparatus 10, the eigenvalue decomposition unit 43 calculates eigenvalues and eigenfunctions as many as the plurality of converted template images by performing eigenvalue decomposition on the converted template image group.

  The eigenfunction selection unit 44 performs a process corresponding to the process in the eigenfunction selection step in the image processing method described above. That is, in the image processing apparatus 10, the eigenfunction selection unit 44 selects a predetermined number of eigenfunctions in descending order of eigenvalues from among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit 43. That is, the eigenfunction selection unit 44 selects some (M) eigenfunctions on the side having a larger eigenvalue from among a plurality (N) eigenfunctions calculated by the eigenvalue decomposition unit 43. .

  The eigenvalue decomposition template image generation unit 45 performs processing corresponding to the processing in the second image group generation step in the image processing method described above. That is, the eigenvalue decomposition template image generation unit 45 performs an inner product operation on the plurality of eigenfunctions calculated by the eigenvalue decomposition unit 43 and the converted template image group generated by the converted template image generation unit 42, thereby performing eigenvalue decomposition. Processing for generating a template image group is performed.

  In this way, the eigenvalue decomposition template image generation unit 45 functions as a second image group generation unit that generates an eigenvalue decomposition template image group by performing an inner product operation of the plurality of eigenfunctions and the converted template image group. . In the configuration including the eigenfunction selection unit 44 as in the image processing apparatus 10, the eigenvalue decomposition template image generation unit 45 includes a plurality of (M) eigenfunctions selected by the eigenfunction selection unit 44 and a converted template. An eigenvalue decomposition template image group is generated by performing an inner product operation with the image group.

  The image selection unit 46 performs a process corresponding to the process in the pixel selection step in the image processing method described above. That is, in the image processing apparatus 10, the image selection unit 46 selects the images of the eigenvalue decomposition template image group generated by the eigenvalue decomposition template image generation unit 45 (each eigenvalue decomposition template image 4) in descending order of eigenvalues. A predetermined number of eigenvalue decomposition template images 4 are selected. That is, according to the image selection unit 46, some (K) eigenvalues on the larger eigenvalue side of the plurality (M) eigenvalue decomposition template images 4 generated by the eigenvalue decomposition template image generation unit 45. The decomposition template image 4 is selected.

The correlation value vector generation unit 47 performs processing corresponding to the processing in the correlation value vector generation step in the image processing method described above. That is, the correlation value vector generation unit 47 is a predetermined real number representing the correlation between each image of the converted template image group (converted template image) and each image of the eigenvalue decomposition template image group (eigenvalue decomposition template image 4). generating a correlation value vector for a certain correlation values a _n and the value of each component.

  As described above, the correlation value vector generation unit 47 calculates a correlation value vector corresponding to each converted template image in the converted template image group by collating the converted template image with the eigenvalue decomposition template image. In the configuration including the image selection unit 46 as in the image processing apparatus 10, the correlation value vector generation unit 47 calculates correlation value vectors for the (K) eigenvalue decomposition template images 4 selected by the image selection unit 46. Generate.

  In this way, the correlation value vector generation unit 47 that generates the correlation value vector functions as a dictionary creation unit that creates a dictionary used in the image processing method of the present embodiment. The dictionary created here, that is, the same number of correlation value vector groups as the converted template image group is stored in the storage unit in the calculation control unit 13 or stored in a storage unit such as a RAM.

The pixel excluding unit 48 performs a process corresponding to the process in the pixel excluding step in the image processing method described above. That is, the pixel excluding unit 48 is a part that performs the above-described processing for narrowing down the positions (two-dimensional coordinate values), and excludes pixels whose 0th-order correlation value r ₀ is less than the predetermined threshold value Ar from processing targets. . The search for the geometric transformation parameter is discontinued for the pixels excluded from the processing target.

As described above, the pixel excluding unit 48 determines, for each pixel of the input image 2, the lowest order image (0th-order eigenvalue decomposition template image 4) of the eigenvalue decomposition template image group and the input image 2. Based on the correlation value r ₀ calculated for the relationship, pixels having a relatively low correlation value r ₀ for the lowest order image among the pixels of the input image 2 are excluded from the processing target.

The feature vector generation unit 49 performs processing corresponding to the processing in the feature vector generation step in the image processing described above. That is, the feature vector generating unit 49, the input image 2, and generates a feature vector Vt of the correlation values a _n and the value of each component. The feature vector Vt is a comparison target of the correlation value vector generated by the correlation value vector generation unit 47.

  The vector selection unit 50 performs processing corresponding to the processing in the vector selection step in the image processing described above. That is, the vector selection unit 50 selects a correlation value vector that most closely approximates the feature vector Vt from among the plurality of correlation value vectors generated by the correlation value vector generation unit 47.

  Specifically, the vector selection unit 50 selects a correlation value vector that most closely approximates the feature vector Vt from N = 360 correlation value vectors v0 to v359 by a method such as approximate neighborhood search.

The image similarity calculation unit 51 performs processing corresponding to the processing in the image similarity calculation step in the above-described image processing method. That is, the image similarity calculation unit 51 uses the narrowed angle range θ _A1 ± α obtained by narrowing down the posture based on the above formulas (1) to (9), and the image corresponding to the eigenvalue decomposition template image group for the input image 22. The similarity (g (θ)) is calculated. The image similarity calculation unit 51 performs the image similarity (g (θ)) for the narrowed-down angle range θ _A1 ± α for each pixel to be processed.

As described above, the image similarity calculation unit 51 near the processing target pixel in the vicinity of the value of the first estimated direction angle θ _A1 that is the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selection unit 50. In other words, the image similarity (g (θ)) is calculated within the narrowing angle range θ _A1 ± α.

  The collation unit 52 performs processing corresponding to the processing in the collation step in the above-described image processing method. That is, the collation unit 52 determines the value of the geometric transformation parameter of the input image 22 based on the image similarity (g (θ)) calculated by the image similarity calculation unit 51.

The collation unit 52 collates the template image 21 and the input image 22 with the image similarity value calculated by the image similarity calculation unit 51 as described above as an index. In the image processing apparatus 10, the collation unit 52 determines the position (two-dimensional coordinate value) and orientation (direction angle) values that are geometric transformation parameters based on the image similarity calculated by the image similarity calculation unit 51. To do. That is, the collation unit 52 determines the position of the pixel with the largest maximum image similarity calculated by the image similarity calculation unit 51 among the processing target pixels as the value of the two-dimensional coordinate value of the geometric transformation parameter, The direction angle (second estimated direction angle θ _A2 ) corresponding to the maximum image similarity for the pixel having the largest image similarity is determined as the direction angle of the geometric transformation parameter.

  As described above, according to the image collation performed by each unit included in the calculation control unit 13 in the image processing apparatus 10 of the present embodiment, the direction angle and the two-dimensional coordinate value of the input image 22 are obtained as the collation result. Specifically, as shown in FIG. 11, as the direction angle included in the collation result, for example, the inclination (rotation angle) θ of the target image portion 23 with respect to the reference line O10 indicating the direction angle of the template image 21 is obtained. It is done. As the two-dimensional coordinate value included in the collation result, for example, a target image when a predetermined position in the input image 22 (in the example illustrated in FIG. 11, the position of the upper left corner of the input image 2) is the origin O20. A two-dimensional coordinate value (XY coordinate value) of the center position C10 of the portion 23 is obtained.

  In the image processing apparatus 10 described above, the directional angle and the two-dimensional coordinate value are used as the geometric transformation parameters determined for the input image 22, but as described above, as the geometric transformation applied to the template image 21. Projective transformation or image blurring processing may be used, and parameters necessary for projective transformation or image blurring processing may be used as geometric transformation parameters.

[Processing procedure]
An example of the processing procedure of image collation performed as the image processing of this embodiment will be described with reference to the flowcharts shown in FIGS. 2 and 12 to 14.

  As shown in FIG. 2, in the image collation processing procedure as the image processing of this embodiment, first, a template image is input (S10). Here, in the image processing apparatus 10, a template image 21 obtained by imaging the IC chip 20 to be inspected by a predetermined method is input to the template image storage unit 41 and stored in advance.

  Next, an eigenvalue decomposition template image is generated (S20). Details of this eigenvalue decomposition template image generation step are shown in the flowchart of FIG.

  As shown in FIG. 12, in the eigenvalue decomposition template image generation step, first, a converted template image group is generated (S21). This step corresponds to a first image group generation step. In this step, by rotating the template image 1 at a 1 ° pitch, N = 360 converted template images (θ = 0 to 359) having different directional angles at a 1 ° pitch are generated, and the 360 converted images A post-template image is obtained as a post-conversion template image group T [i].

  In the image processing apparatus 10, the template image 21 stored in the template image storage unit 41 is rotated at a 1 ° pitch by the converted template image generation unit 42, so that N = 360 converted template images (rotated). A post-conversion template image group consisting of (image) is generated. In the example illustrated in FIG. 3, a post-conversion template image group 1N including N post-conversion template images is generated.

  Next, a plurality of eigenvalues and eigenfunctions are calculated (S22). This step corresponds to the eigenvalue decomposition step. In this step, eigenvalue decomposition is performed on the post-conversion template image group T [i] to obtain the eigenfunction φ [i] and the eigenvalue λ [i].

In the image processing apparatus 10, the eigenvalue decomposition unit 43 performs eigenvalue decomposition on the post-conversion template image group (N post-conversion template images), thereby calculating eigenvalues and eigenfunctions. Here, the same number (N) of eigenvalues and eigenfunctions as the number of post-conversion template images generated by the eigenvalue decomposition unit 43 are calculated. Therefore, when 360 converted template images are generated as described above, 360 eigenvalues and eigenfunctions are calculated. In the example shown in FIG. 3, eigenvalue decomposition is performed on the post-conversion template image group 1N, whereby an eigenfunction consisting of N eigenfunctions φ _n (θ), (n = 1, 2,... N). A group and N eigenvalues are calculated.

  Subsequently, the eigenfunction is selected based on the size of the eigenvalue (S23). This step corresponds to an eigenfunction selection step. In this step, M (for example, 60) eigenfunctions φ [i] are selected in descending order of the eigenvalue λ [i] based on the value of the eigenvalue λ [i].

In the image processing apparatus 10, the eigenfunction selection unit 44 selects a predetermined number (M) of eigenfunctions having larger eigenvalues among the N eigenfunctions calculated by the eigenvalue decomposition unit 43. For example, when 360 (N) eigenfunctions are calculated as described above, 60 (M) eigenfunctions having a larger eigenvalue are selected. In the example shown in FIG. 3, a group of eigenfunctions φ ₁ (θ), φ ₂ (θ),..., Φ _M (θ) consisting of M eigenfunctions from N eigenfunctions φ _n (θ). Is selected.

  Then, an eigenvalue decomposition template image group is generated (S24). This step corresponds to a second image group generation step. In this step, M eigenvalue decomposition template images are generated from the M eigenfunctions φ [i] selected in step S23 and the N converted template images generated in step S21.

In the image processing apparatus 10, the eigenvalue decomposition template image generation unit 45 converts the M eigenfunctions selected by the eigenfunction selection unit 44 and the converted template image generation unit 42 based on the above formula (4). The eigenvalue decomposition template image group (M eigenvalue decomposition template images) is generated by calculating the inner product with the subsequent template image group. In the example shown in FIG. 3, an eigenvalue decomposition template image group including eigenvalue decomposition template images E1 to E3 (E ₁ (x, y) to E ₃ (x, y)) is obtained.

  Next, returning to FIG. 2, a dictionary is generated (S30). Details of this dictionary generation step are shown in the flowchart of FIG.

  As shown in FIG. 13, in the dictionary generation step, first, K (<M) eigenvalue decomposition template images are selected (S31). This step corresponds to an image selection step. In this step, K (for example, 10) eigenvalue decomposition template images 4 are selected in descending order of the eigenvalue λ [i] based on the value of the eigenvalue λ [i]. The plurality of eigenvalue decomposition template images 4 selected here are defined as “dictionary template group”.

  In the image processing apparatus 10, a predetermined number (K) of eigenvalue decomposition templates having larger eigenvalues among the M eigenvalue decomposition template images 4 generated by the eigenvalue decomposition template image generation unit 45 by the image selection unit 46. Image 4 is selected. For example, when 60 eigenvalue decomposition template images 4 are generated, ten (K) eigenvalue decomposition template images 4 having larger eigenvalues are selected.

  Next, N correlation value vectors are generated (S32). This step corresponds to a correlation value vector generation step. In this step, the N converted template image groups T [i] are compared with K (<M) eigenvalue decomposition template images 4, and N correlation value vectors v [i] = {Ai0, ai1, ai2, ..., ai (K-1)} are generated. Here, it is assumed that the orientation (direction angle) of the converted template image corresponding to each correlation value vector v [i] is “s [i]”.

  In the image processing apparatus 10, the N converted template images generated by the converted template image generating unit 42 by the correlation value vector generating unit 47 and the K eigenvalue decomposition template images 4 selected by the image selecting unit 46 are used. N = 360 correlation value vectors v0 to v359 having K (for example, 10) components are generated.

  Next, returning to FIG. 2, the input image 2 is input (S40). Here, in the image processing apparatus 10, the IC chip 20 picked by the arm 17 and held by the arm 17 is imaged by the camera 12, and the imaged data is input to the arithmetic control unit 13 of the computer 11. The In the example shown in FIG. 3, the input image 2 is acquired and input.

  Subsequently, as shown in FIG. 2, image collation is performed with three stages of narrowing down (S50). Details of this three-stage narrowed image collation step are shown in the flowchart of FIG.

As shown in FIG. 14, in the three-stage narrowed image collation step, first, the 0th-order correlation image 5 is obtained by collating the input image 2 with the 0th-order eigenvalue decomposition template image 4 (S51). That is, in this step, as a result of the convolution of the input image 2 and the 0th-order eigenvalue decomposition template image 4, the 0th-order response value for the pixel corresponding to each pixel of the input image 2, that is, the 0th-order correlation value. 0-order correlation image represents the magnitude of r ₀ 5 (see FIG. 4) is obtained.

  In the image processing apparatus 10, the pixel excluding unit 48 performs a convolution operation between the input image 2 and the 0th-order eigenvalue decomposition template image 4, thereby generating a 0th-order correlation image 5.

Next, the processing target pixel is determined using the threshold value Ar in the 0th-order correlation image 5 (S52). That is, in this step relates to pixels of the input image 2, the processing target pixel a pixel correlation value r ₀ of the 0th order is equal to or greater than a threshold Ar, other pixels, i.e. zero order correlation value r ₀ is less than the threshold value Ar Are excluded from the processing target.

In the image processing apparatus 10, pixels other than the processing target pixel are detected from the processing target by threshold processing using the threshold value Ar for the zero-order correlation value r ₀ calculated for each pixel of the input image 2 by the pixel excluding unit 48. Excluded. The threshold value Ar is set and stored in advance in the pixel exclusion unit 48 or the like in the arithmetic control unit 13, for example.

  In the three-stage narrowed image collation step, these steps S51 and S52 correspond to the pixel exclusion step. That is, the processing in these steps S51 and S52 corresponds to the processing for narrowing down the position (two-dimensional coordinate value) described above.

  Next, feature vectors are generated (S53). This step corresponds to a feature vector generation step. In this step, image collation is performed by translating the dictionary template group to the position of the processing target pixel, and a feature vector Vt = {t0, t1, t2,..., T (K−1)} is generated. .

  In the image processing apparatus 10, K (for example, 10) components are obtained by a convolution operation between the input image 22 and the K eigenvalue decomposition template images 4 selected by the image selection unit 46 by the feature vector generation unit 49. A feature vector Vt is generated.

Next, the most approximate correlation value vector is selected by comparing the feature vector and the correlation value vector (S54). This step corresponds to a vector selection step. In this step, a technique such as approximate neighborhood search is used, and a correlation value vector that is most approximate to the feature vector Vt is selected from a dictionary template group, that is, 360 correlation value vector groups of θ = 0 to 359. From the correlation value vector selected here, the first estimated direction angle θ _A1 and the narrowed-down angle range θ _A1 ± α are obtained. The orientation (direction angle) s [i] of the correlation value vector selected here is defined as a “first estimated orientation”.

In the image processing apparatus 10, the correlation value vector that most closely approximates the feature vector Vt generated by the feature vector generation unit 49 from the 360 correlation value vectors generated by the correlation value vector generation unit 47 by the vector selection unit 50. Elected. Then, the vector selection unit 50 calculates the first estimated direction angle θ _A1 and the narrowing angle range θ _A1 ± α from the selected correlation value vector.

  In the three-stage narrowed-down image collation step, the processing in these steps S53 and S54 corresponds to the above-described processing for narrowing down the posture (direction angle).

Next, the image similarity is calculated for the narrowing angle range θ _A1 ± α (S55). This step corresponds to an image similarity calculation step. In this step, the image similarity (g (θ)) in the range near the value of the first estimated direction angle θ _A1 corresponding to the correlation value vector selected in step S54, that is, the narrowing angle range θ _A1 ± α. ) Is calculated. Then, a direction angle (second estimated direction angle θ _A2 ) that maximizes the image similarity (g (θ)) is calculated from the narrowed angle range θ _A1 ± α. Accordingly, the image similarity (maximum image similarity) corresponding to the second estimated direction angle θ _A2 is calculated. Here, the posture (direction angle) s [i] corresponding to the maximum image similarity is defined as a “second estimated posture”.

In the image processing apparatus 10, the image similarity calculation unit 51 determines the image similarity (g () in the range near the value of the first estimated direction angle θ _A1 corresponding to the correlation value vector selected by the vector selection unit 50. θ)) is calculated. A second estimated direction angle θ _A2 is calculated from the correlation waveform of the image similarity (g (θ)).

Then, the direction angle and the two-dimensional coordinate value (XY coordinate value) for the input image 2 are determined (S56). This step corresponds to a collation step. In this step, among the processing target pixels, the position of the pixel having the largest maximum image similarity calculated in step S55 is determined as the value of the two-dimensional coordinate value, and the maximum image for the pixel having the largest maximum image similarity is obtained. A direction angle corresponding to the similarity (second estimated direction angle θ _A2 ) is determined as the direction angle. The value of the two-dimensional coordinate determined here is the detection position, and the direction angle determined here, that is, the second estimated posture is the detection posture.

In the image processing apparatus 10, the input image 22 is calculated based on the image similarity (g (θ)) in the narrowing angle range θ _A1 ± α for the processing target pixel calculated by the matching unit 52 by the image similarity calculation unit 51. The two-dimensional coordinate value that becomes the detection position and the direction angle that becomes the detection posture are determined.

  Then, referring back to FIG. 2, the direction angle and the two-dimensional coordinate value for the input image are output as the detection posture and the detection position, respectively, based on the result of the image matching in step S50 (S60).

  In the image processing apparatus 10, the detection posture and the detection position determined by the collation unit 52 are output, for example, on the display unit 15 connected to the calculation control unit 13.

  In the processing procedure as described above, steps S53 to S55 are performed for all the pixels to be processed determined in steps S51 and S52. For this reason, in the processing procedure described above, for example, the following method is adopted. First, the two-dimensional coordinate value (x, y) is specified for the processing target pixel obtained in steps S51 and S52 in the input image.

  Then, with respect to the position (x, y) of each pixel of the processing target pixel, generation of a feature vector (S53), selection of the most approximate correlation vector (S54), and calculation of image similarity (S55) in a predetermined order. Is repeated. For example, the position (x, y) of each pixel of the processing target pixel is set while sequentially shifting one pixel at a time from the left side to the right side in each row. When the image similarity g is calculated for all the processing target pixels, based on the calculation result, in step S56, the final geometric transformation parameter value (real value), that is, the direction angle and the two-dimensional Coordinate values are determined.

  With regard to the image collation processing procedure as described above, in the image processing apparatus 10, steps S10 and S20, which are processes for creating an eigenvalue decomposition template image group based on a template image 21 stored in advance, are performed as eigenvalue decomposition templates. This is performed by the image creation processing unit 31. In addition, the narrowing-down processing unit 32 performs steps S30, S40, and S51 to S55 in steps S30, S40, and S50 that perform the process of narrowing the position and narrowing the direction angle. Then, the detection processing unit 33 performs S56 and S60 of step S50, in which the image similarity is calculated, and the direction angle and the two-dimensional coordinate value are determined and output.

  In the actual processing process, when the plurality of IC chips 20 are positioned, the eigenvalue decomposition template image group generation processing by the eigenvalue decomposition template image creation processing unit 31 is performed in advance when positioning the plurality of IC chips 20. Is called. Then, the process for determining the direction angle and the two-dimensional coordinate value by the detection processing unit 33 is repeatedly performed using the eigenvalue decomposition template image group generated in advance in the positioning process of each IC chip 20. Become.

[Evaluation experiment]
Hereinafter, evaluation experiments as examples of the present invention will be described. As shown in FIG. 15A, in this evaluation experiment, 100 sets of 512 × 512 pixel images were used as the input image 102. In the input image 102, a cross-shaped image portion whose posture (direction angle) can be specified is used as the target image portion 103. In the input image 102, the target image portion 103 exists in a black plain background. 100 sets of input images 102 were created by randomly rotating and translating the target image portion 103 in relation to other input images 102. On the other hand, the template image is an image including an image portion having the same size and shape as the target image portion 103 in the input image 102.

  Further, in this evaluation experiment, in the image processing method according to the present invention (hereinafter referred to as “proposed method”), the template image is rotated at a 1 ° pitch, so that a converted template composed of a total of 360 converted template images is obtained. A group of images was generated (N = 360). In this evaluation experiment, the approximate order M was set to 60, and the number K of eigenvalue decomposition template images used for calculating the correlation value vector and the feature vector Vt was set to 20.

  FIG. 15B shows the result of this evaluation experiment. As shown in FIG. 15 (b), in this evaluation experiment, an experiment for the conventional method was performed as a comparative example for the proposed method. Here, the conventional method is a method that does not narrow down the position and the posture in the proposed method. That is, in the conventional method, all the pixels of the input image 102 are processed, and the correlation waveform (image similarity calculation) is calculated for all directional angles (θ = 0 to 359 °). Further, as shown in FIG. 15B, in this evaluation experiment, as other comparative examples, RIPOC (Rotation Invariant Phase Only Correlation) and NCC (Normalized Cross Correlation), which are pattern matching techniques, were employed.

  In the table shown in FIG. 15B, “distance error [pix]” is a deviation of the measured value from the correct value for the position (two-dimensional coordinate value), and “angle error [deg]” is the posture (direction). This is the deviation of the measured value from the correct value for (angle). Here, the correct value for each position and orientation is a known value set in advance. Each numerical value shown in each column of “distance error” and “angle error” in the table shown in FIG. 15B is an average value of measured values obtained for 100 input images 102. The “success rate [%]” is a state in which the position measurement value is within the range of the correct value ± 5 [pix] and the posture measurement value is within the range of the correct value ± 5 [deg]. “Successful” is equivalent to the number of successful images out of 100 input images 102. “Processing time [msec]” is the time from the start of processing to the measurement of the direction angle and two-dimensional coordinate values.

  As can be seen from the result shown in FIG. 15B, the “distance error”, “angle error”, and “success rate” are all the same in the proposed method as compared to the conventional method, but “processing time”. Is significantly shortened to 1/10 or less. The “processing time” of the proposed method is comparable to that of RIPOC, which is said to be relatively fast. In addition, the “distance error” and “angle error” of the proposed method are comparable to those of RIOC, and according to the proposed method, detection accuracy similar to that of RIOC can be obtained. In comparison with NCC, the “distance error” and “angle error” of the proposed method are sufficiently small, and the “processing time” is also short.

  Another evaluation experiment will be described with reference to FIG. In this evaluation experiment, measurement was performed by adding noise to the input image. Specifically, as shown in FIG. 16A, the input image 202 used in this evaluation experiment includes a mesh pattern background 204 as noise and three curved lines 205 in addition to the target image portion 203. Have. Other conditions and the like are the same as in the evaluation experiment described above. In this way, the evaluation experiment in which the input image is relatively complicated assumes a scene close to a situation where image matching is actually used.

  As can be seen from the results shown in FIG. 16B, in the same way as the evaluation experiment described above, in the proposed method, the “distance error”, the “angle error”, and the “success” are compared with the conventional method. “Rate” is about the same, but “processing time” is greatly reduced to 1/10 or less. In addition, the “processing time” of the proposed method is comparable to that of RIPOC, which is said to be relatively fast. In addition, the “distance error” and “angle error” of the proposed method are comparable to those of RIOC, and according to the proposed method, detection accuracy similar to that of RIOC can be obtained. In comparison with NCC, the “distance error” and “angle error” of the proposed method are sufficiently small, and the “processing time” is also short.

  However, in this evaluation experiment, the “success rate” of RIPOC is significantly reduced (less than half) compared to the case of the evaluation experiment described above. In contrast, the proposed method maintains the same “success rate” in the evaluation experiment using the input image 202 including noise as in the evaluation experiment using the input image 102 having no noise. . That is, the proposed method is less affected by noise existing on the input image than the RIPOC, and the detection accuracy can be maintained even when the input image becomes complicated.

  From the results of the evaluation experiments as described above, it was proved that the proposed method can obtain a processing speed that can withstand practical use and can obtain the same level of detection accuracy in comparison with RIPOC. In addition, it was proved that the proposed method is less affected by noise on the input image than RIPOC, and that the detection accuracy can be maintained to some extent even when the input image becomes complicated. As described above, based on the results of the evaluation experiment, the proposed method comprehensively takes into consideration the effects of processing speed, detection accuracy, and noise, and compared with the conventional method for the above-described proposed method and the conventional techniques such as RIPOC and NCC. It can be said that this is an excellent technique.

  In the proposed method, the eigenvalue decomposition is not performed on the gray value of the template image group, but the template image group is converted by the proposed method after the template image and the input image are converted into the edge direction image or the incremental code image. It is also possible to perform image matching by converting into an eigenvalue decomposition template image group. That is, by performing image matching in consideration of the edge direction, it is possible to perform image matching that is robust against shielding such as fluctuations in brightness and hiding part of the object.

  The proposed method is a method for detecting, at high speed, a geometric transformation parameter such as a position and a direction angle that matches a template image from an input image. The proposed method calculates a correlation value waveform by collating images obtained by eigenvalue decomposition of a template image. That is, the proposed method performs eigenvalue decomposition on a template image group for image matching to extract principal component information, and based on this, calculates a continuous similarity curve for image similarity estimation, thereby matching time Shortening and improving collation accuracy.

  The proposed method uses the new template image reduced in number using eigenvalue decomposition and reduces the geometric difference (geometric transformation parameter value) between the image to be inspected (input image) and the template image without reducing accuracy. It can be said that the compression template matching method is estimated. Then, a second image group (eigenvalue decomposition template image group) generated by performing an inner product operation on the first image group (post-conversion template image group) that is the original template image group for the eigenfunction calculated by eigenvalue decomposition. This is used twice to generate the image and calculate the image similarity with the input image. Thereby, the image similarity represented by a continuous curve can be obtained, and it is possible to estimate the value of the geometric transformation parameter with high accuracy that cannot be obtained by the prior art.

  Then, the proposed method effectively searches the geometric transformation parameters by performing two-stage narrowing of the position (two-dimensional coordinate value) and the narrowing of the posture (direction angle) on the input image. The range, that is, the range for which the image similarity is calculated, is narrowed to further reduce the processing time.

  Application examples of the present invention include inspection and measurement of various machine parts such as automobile parts, biometrics (fingerprint authentication), character recognition (OCR), food / drug inspection, and high-accuracy alignment. IC substrate positioning (semiconductor inspection, mask exposure alignment, rotation angle alignment (alignment)), visual inspection, and the like, and the present invention has applicability in various fields. In particular, the image processing technique according to the present invention is useful in image matching processing in industrial image processing that requires high subpixel level matching accuracy and high-speed processing.

DESCRIPTION OF SYMBOLS 1 Template image 2 Input image 3 Target image part 10 Image processing apparatus 11 Computer 12 Camera (input image acquisition means)
13 Arithmetic control unit (image processing means)
DESCRIPTION OF SYMBOLS 21 Template image 22 Input image 23 Target image part 31 Eigenvalue decomposition template image creation process part 32 Narrowing process part 33 Detection process part 42 Template image generation part after conversion (1st image group generation part)
43 eigenvalue decomposition unit 44 eigenfunction selection unit 45 eigenvalue decomposition template image generation unit (second image group generation unit)
46 Image selection unit 47 Correlation value vector generation unit 48 Pixel exclusion unit 49 Feature vector generation unit 50 Vector selection unit 51 Image similarity calculation unit 52 Collation unit

Claims (7)

An image processing method for determining a value of a geometric transformation parameter of an image portion of an object to be collated existing in an input image by performing collation using a template image that is an image of an object to be collated,
A first image group generation step of generating a first image group composed of the template images after a plurality of geometric transformations by adding geometric transformation to the template image;
An eigenvalue decomposition step of calculating the same number of eigenvalues and eigenfunctions as the plurality of geometrically transformed template images by performing eigenvalue decomposition on the first image group;
A second image group generation step of generating a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group;
Correlation value vector generation step for generating a correlation value vector having a correlation value that is a predetermined real number representing a correlation with each image of the second image group for each image of the first image group. When,
For each pixel of the input image, based on the correlation value calculated for the relationship between the lowest order image having the lowest order among the images of the second image group and the input image, the pixels of the input image A pixel excluding step of excluding from the processing target pixels having a relatively low correlation value for the lowest order image;
A feature vector generation step of generating a feature vector having the correlation value as a value of each component for the input image;
A vector selection step of selecting the correlation value vector that most closely approximates the feature vector from the plurality of correlation value vectors generated in the correlation value vector generation step;
For pixels to be processed other than the pixels excluded by the pixel exclusion step, a continuous curve is represented in a range near the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selection step. An image similarity calculating step for calculating an image similarity indicating a degree of similarity between the input image and the second image group;
A collation step of determining a value of the geometric transformation parameter based on the image similarity calculated in the image similarity calculation step,
Image processing method. Of the plurality of eigenfunctions calculated in the eigenvalue decomposition step, further comprising an eigenfunction selection step of selecting a predetermined number of eigenfunctions in descending order of the eigenvalues,
The second image group generation step generates the second image group by performing an inner product operation of the plurality of the eigenfunctions selected in the eigenfunction selection step and the first image group.
The image processing method according to claim 1. An image selection step of selecting a predetermined number of images in descending order of the eigenvalues among the images of the second image group generated in the second image group generation step;
The correlation value vector generation step generates the correlation value vector for the image selected in the image selection step.
The image processing method according to claim 1 or 2. The geometric transformation parameter includes a direction angle and a two-dimensional coordinate value,
The geometric transformation is a rotation that changes a direction angle;
In the image similarity calculation step, the image similarity is calculated in a range in the vicinity of the direction angle value corresponding to the correlation value vector selected in the vector selection step for all the pixels to be processed. Calculate
In the collation step, among all the pixels to be processed, the position of the pixel having the largest image similarity calculated in the image similarity calculation step is determined as a two-dimensional geometric conversion parameter. A coordinate value is determined, and a direction angle corresponding to the maximum value for the pixel having the largest image similarity is determined as a direction angle of the geometric transformation parameter.
The image processing method according to claim 1. Input image acquisition means for acquiring an input image;
Image processing means for determining the value of the geometric transformation parameter of the image portion of the object to be collated existing in the input image by performing collation using a template image that is an image of the object to be collated;
The image processing means includes
A first image group generation unit configured to generate a first image group composed of the template images after a plurality of geometric transformations having different geometric transformation amounts by applying geometric transformation to the template image;
An eigenvalue decomposition unit that calculates eigenvalues and eigenfunctions of the same number as the template images after the plurality of geometric transformations by performing eigenvalue decomposition on the first image group;
A second image group generation unit that generates a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group;
A correlation value vector generation unit that generates a correlation value vector having a correlation value that is a predetermined real number representing a correlation with each image of the second image group as a value of each component for each image of the first image group When,
For each pixel of the input image, based on the correlation value calculated for the relationship between the lowest order image having the lowest order among the images of the second image group and the input image, the pixels of the input image A pixel excluding unit that excludes pixels having a relatively low correlation value for the lowest order image from the processing target;
For the input image, a feature vector generation unit that generates a feature vector having the correlation value as the value of each component;
A vector selection unit that selects the correlation value vector that most closely approximates the feature vector among the plurality of correlation value vectors generated by the correlation value vector generation unit;
The pixel to be processed other than the pixels excluded by the pixel excluding unit is represented by a continuous curve in a range in the vicinity of the value of the geometric transformation parameter corresponding to the correlation value vector selected by the vector selecting unit. An image similarity calculation unit that calculates an image similarity indicating a degree of similarity between the input image and the second image group;
A collation unit that determines a value of the geometric transformation parameter based on the image similarity calculated by the image similarity calculation unit,
Image processing device. The image processing means includes
Of the plurality of eigenfunctions calculated by the eigenvalue decomposition unit, further comprising an eigenfunction selection unit that selects a predetermined number of the eigenfunctions in descending order of the eigenvalues,
The second image group generation unit generates the second image group by performing an inner product operation of the plurality of eigenfunctions selected by the selection unit and the first image group.
The image processing apparatus according to claim 5. The image processing means includes
An image selection unit that selects a predetermined number of images in descending order of the eigenvalues among the images of the second image group generated by the second image group generation unit;
The correlation value vector generation unit generates the correlation value vector for the image selected by the image selection unit.
The image processing apparatus according to claim 5.