JPWO2012029662A1 - Image processing method and image processing apparatus - Google Patents

Abstract

We propose a method that can ensure the same accuracy as the rotation matching method and reduce the processing time with a smaller number of matching times than the rotation matching method that performs the matching while rotating the template image. A step of generating a first image group composed of a plurality of template images after geometric transformation with different geometric transformation amounts by applying geometric transformation to the template image 1, and a transformation by performing eigenvalue decomposition on the image group A step of calculating the same number of eigenvalues and eigenfunctions as a post-template image; a step of generating a second image group by performing an inner product operation of a plurality of eigenfunctions and the image group; and an input image and a second A step of calculating an image similarity with the image group, and a step of determining a value of a geometric transformation parameter of the input image based on the calculated image similarity.

Description

  The present invention relates to an image processing method and an image processing apparatus that perform image matching using a template image.

  Conventionally, image collation is used for software for inspecting parts and the like. One example of image matching is to use a template image. In image collation using a template image, for example, a component that is known to be a non-defective product is imaged, and a template image is prepared in advance using the captured image. Then, the input image obtained by imaging the part to be inspected and the template image are collated, and the part to be inspected is inspected by comparing the input image and the template image.

  In particular, in industrial image processing, in order to detect the position and direction angle of a part to be inspected with high accuracy, processing is performed in which a template image given in advance and an input image are collated with sub-pixel accuracy ( Non-patent document 1). As such an image collation technique with subpixel accuracy, a rotation collation method is known in which collation is performed while rotating a template image with respect to an input image. The rotation matching method is an image matching technique using geometric transformation that performs geometric transformation on a template image.

  In the rotation matching method, an image similarity indicating image similarity in relation to an input image is calculated while the template image is rotated and translated, and the direction angle of the input image and the input image direction angle are calculated from the peak value of the image similarity. Coordinate values are detected. That is, according to the rotation collation method, the template image is rotated and translated at a predetermined pitch, and the rotation angle and the XY coordinate values that approximate the input image to the template image are searched, whereby the direction angle of the input image. And coordinate values are detected.

  In such a rotation collation method, in order to increase the detection accuracy, it is necessary to increase the number of collations by reducing the rotation pitch of the template image for searching for the rotation angle at which the template image most closely approximates the input image. For this reason, increasing the detection accuracy in the rotation matching method leads to a long processing time. On the other hand, if the rotation pitch of the template image is rough (enlarged) in order to shorten the processing time, it becomes difficult to obtain sufficient detection accuracy.

  Therefore, in order to reduce the processing time in the image matching method, there is a method of interpolating the calculated image similarity using a quadratic curved surface or a quadratic curve while performing collation by coarsening the rotation pitch of the template image. Yes (see Non-Patent Documents 2 and 3). However, according to the method of performing interpolation in this way, an error due to interpolation occurs, and the detection accuracy decreases.

  As another image matching technique, a phase only correlation method is known (see Non-Patent Documents 4 to 6). In recent years, the phase-only correlation method has attracted attention as a technique that enables high-speed and high-precision alignment. However, the phase-only correlation method does not have a very good detection accuracy of the rotation angle of the image, and cannot sufficiently cope with image deformation other than image misalignment, rotation, and enlargement / reduction.

  In addition, there is a technique for performing image matching after converting a template image and an input image into an edge direction image or an incremental code image in order to perform image matching that is robust against brightness fluctuations and hiding of the input image (Non-patent Document 7). reference). However, this method does not support rotation of the input image.

Kazuhiko Kusumi, Shunichi Kaneko, “Practical use of machine vision technology”, IEEJ Transactions C, Vol. 124, no. 3, pp. 598-605, 2004. Masao Shimizu and Masatoshi Okutomi, “Precise Subpixel Estimation Method for Image Matching”, IEICE Transactions D-II, Vol. J84-D-II, No. 7, pp. 1409-1418, 2001. Masao Shimizu, Masatoshi Okutomi, “Meaning and Properties of Subpixel Estimation in Image Matching”, IEICE Transactions D-II, Vol. J85-D-II, No. 12, pp. 1791-1800, 2002. K. Taketa, T .; Aoki, Y .; Sasaki, T .; Higuchi and K.H. Kobayashi, “High-accuracy subpixel image registration based on phase-only correlation”, IEICE Transactions of Fundamentals of Fundamentals of Electronic Sciences. E86-A, no. 8, pp. 1925-1934, 2003. Hiroshi Sasaki, Kouji Kobayashi, Takafumi Aoki, Masayuki Kawamata, Tatsuo Higuchi, “Measurement of Image Rotation Angle by Rotation Invariant Phase-Only Correlation”, ITE Technical Report, Vol. 22, no. 45, pp. 55-60, 1998. Takafumi Aoki, Koichi Ito, Takuma Shibahara, Satoshi Nagashima, “High-Precision Machine Vision Based on Phase-Only Correlation—Towards Image Sensing Technology that Transcends Pixel Resolution—”, IEICE Fundamentals Review, Vol. 1, No. 1 1, pp. 30-40, 2007. Ichiro Murase and Shunichi Kaneko, “Robust Image Matching by Incremental Code Correlation”, IEICE Transactions, DII, Vol. J83, no. 5, pp. 1323-1331, 2000.

  The present invention has been made in view of the background art as described above, and has fewer collations than conventional image collation techniques using geometric transformation, such as a rotation collation method that collates while rotating a template image. It is an object of the present invention to provide an image processing method and an image processing apparatus that can ensure the same accuracy by the number of times and reduce the processing time.

  The image processing method of the present invention is an image processing method for determining a value of a geometric transformation parameter of an input image with reference to the template image by performing collation using a predetermined template image. A first image group generation step for generating a first image group composed of a plurality of template images after geometric conversion by applying geometric transformation, and eigenvalues in the first image group; Performing decomposition, eigenvalue decomposition step for calculating the same number of eigenvalues and eigenfunctions as the template images after the plurality of geometric transformations, and performing inner product operation of the plurality of eigenfunctions and the first image group And a second image group generation step for generating a second image group, and an image similarity that is expressed by a continuous curve and indicates the degree of similarity between the input image and the second image group An image similarity calculation step of calculating, based on the image similarity calculated by the image similarity calculation step, a matching step of determining the value of the geometric transformation parameters are those comprising a.

  In the image processing method of the present invention, preferably, a selection step of selecting a predetermined number of the eigenfunctions in descending order of the eigenvalues among the plurality of eigenfunctions calculated in the eigenvalue decomposition step is further included. And the second image group generation step generates the second image group by performing an inner product operation of the plurality of eigenfunctions selected in the selection step and the first image group. It is.

  In the image processing method of the present invention, it is preferable that the geometric transformation parameter includes a direction angle and a two-dimensional coordinate value, and the geometric transformation changes the rotation that changes the direction angle and the two-dimensional coordinate value. Includes translation.

  In the image processing method of the present invention, it is preferable that the geometric transformation parameter includes a parameter necessary for projective transformation, and the geometric transformation includes projective transformation.

  In the image processing method of the present invention, it is preferable that the geometric transformation parameter includes a parameter necessary for an image blurring process, and the geometric transformation includes an image blurring process.

  The image processing apparatus according to the present invention performs input image acquisition means for acquiring an input image and collation using a predetermined template image, so that the value of the geometric transformation parameter of the input image based on the template image Image processing means for determining the first image comprising the template images after the plurality of geometric transformations having different geometric transformation amounts by applying geometric transformation to the template image. A first image group generation unit that generates a group, and an eigenvalue decomposition unit that calculates the same number of eigenvalues and eigenfunctions as the template images after the plurality of geometric transformations by performing eigenvalue decomposition on the first image group A second image group generation unit that generates a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group, and the input represented by a continuous curve An image similarity calculating unit that calculates an image similarity indicating a degree of similarity between an image and the second image group, and a value of the geometric transformation parameter based on the image similarity calculated by the image similarity calculating unit. And a collation unit for determining.

  In the image processing apparatus of the present invention, it is preferable that the image processing unit selects a predetermined number of the eigenfunctions in descending order of the eigenvalues among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit. And a second image group generation unit that performs an inner product operation of the plurality of eigenfunctions selected by the selection unit and the first image group, so that the second image group is generated. A group is generated.

  According to the present invention, compared with a conventional image matching technique using geometric transformation, such as a rotation matching method that performs matching while rotating a template image, the same accuracy can be ensured with a small number of matching times. Time can be shortened.

1 is a schematic explanatory diagram of an image processing method according to an embodiment of the present invention. Explanatory drawing of the image processing method which concerns on one Embodiment of this invention. 1 is a diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a control block diagram of an image processing apparatus according to an embodiment of the present invention. Explanatory drawing of the image processing of the image processing apparatus which concerns on one Embodiment of this invention. The flowchart which shows an example of the process sequence by the image processing apparatus which concerns on one Embodiment of this invention. Explanatory drawing about an example of the process sequence by the image processing apparatus which concerns on one Embodiment of this invention. The figure which shows the precision comparison with the proposal method and rotation collation method which concern on the Example of this invention. The figure which shows the relationship between the frequency | count of collation and the error which concern on the Example of this invention. The figure which shows a table | surface about the value of the estimation error of the direction angle and position shift at the time of adding the white noise which concerns on the Example of this invention. Explanatory drawing about another Example of this invention. Explanatory drawing about a rotation collation method. Explanatory drawing about the conventional image collation technique.

  According to the present invention, a new template image and an input image obtained by eigenvalue decomposition of a template image obtained by comparing the template image with the input image while rotating the template image are obtained. It pays attention to what is obtained by collating. The present invention is intended to perform high-speed and high-precision collation by collating an input image with a new template image. Embodiments of the present invention will be described below.

  As shown in FIG. 1, the image processing method according to the present embodiment determines the value of the geometric transformation parameter of the input image 2 with reference to the template image 1 by performing collation using a predetermined template image 1. . Here, as the geometric transformation parameter, for example, a direction angle and a two-dimensional coordinate value (XY coordinate value) are used.

  That is, in the image processing method of the present embodiment, the template image 1 is collated with respect to the input image 2, whereby the direction angle at which the image similarity, which is the degree of approximation of the template image 1 with respect to the input image 2, is the highest. And the process which calculates | requires a two-dimensional coordinate value is performed. Therefore, according to the image processing method of the present embodiment, the direction angle and the two-dimensional coordinate value of the input image 2 with the template image 1 as a reference are obtained as a matching result.

  Specifically, as shown in FIG. 1, as the direction angle included in the collation result, for example, the inclination (rotation angle) θ of the input image 2 with respect to the reference line O1 indicating the direction angle of the template image 1 is obtained. . As the two-dimensional coordinate value included in the collation result, for example, the two-dimensional coordinate value (XY coordinate value) of the center position C1 of the input image 2 when the center position of the template image 1 is set as the origin O2 is obtained.

  The image processing method according to the present embodiment is used, for example, for industrial image processing for inspecting components and the like. For this reason, in the image processing method of the present embodiment, the template image 1 is acquired, for example, by imaging the same component as the component to be inspected by an imaging unit such as a camera, and the direction angle and the two-dimensional coordinate value are obtained. An image indicating a reference is prepared in advance. For example, in the inspection of a component, the input image 2 is acquired by imaging a component to be inspected by an imaging unit such as a camera, and is a target for collation of the template image 1. Below, the specific processing content of the image processing method of this embodiment is demonstrated.

  In the image processing method of the present embodiment, first, a process of generating a converted template image group as a first image group by performing geometric transformation on the template image 1 is performed. In this processing, as geometric transformation applied to the template image 1, for example, rotation for changing the direction angle and parallel movement for changing the two-dimensional coordinate value are performed.

  Specifically, when a rotation that changes the direction angle is performed as the geometric transformation applied to the template image 1, for example, a process of generating a template image group after conversion by rotating the template image 1 at a 1 ° pitch. Done. In this processing, 360 geometrically converted template images 1 (hereinafter referred to as “post-conversion template images”) having different directional angles with a 1 ° pitch are obtained as a post-conversion template image group generated in this processing. .

  Further, as a geometric transformation to be added to the template image 1, when a parallel movement that changes the two-dimensional coordinate value is performed, for example, when the template image 1 is 128 × 128 (pixels), the center position of the template image 1 (the center position) A process of generating a post-conversion template image group is performed by translating the two-dimensional coordinate value) within a range of 8 × 8 (pixels) at a 1 (pixel) pitch. In such processing, 64 converted template images having different center positions (center two-dimensional coordinate values) at a 1 (pixel) pitch are obtained as a group of converted template images generated by this processing.

  As described above, in the image processing method of the present embodiment, first, a template image group after conversion, which includes a plurality of template images 1 after geometric conversion with different amounts of geometric conversion by applying geometric conversion to the template image 1. A first image group generation step of generating is performed. Here, the amount of geometric transformation corresponds to the rotation amount of the image with reference to the direction angle of the template image 1 when the geometric transformation is a direction angle, and 2 of the template image 1 when the geometric transformation is a two-dimensional coordinate value. This corresponds to the amount of parallel movement of the image based on the dimensional coordinate value. Therefore, the amount of geometric transformation corresponds to the magnitude of the value of the geometric transformation parameter.

  The geometric transformation applied to the template image 1 may be, for example, image distortion such as image distortion, blur amount, depth of focus, trapezoidal distortion, etc. in addition to the direction angle and the two-dimensional coordinate value. In addition, as the geometric transformation applied to the template image 1, any one of the geometric transformations as described above is used, or a plurality of geometric transformations are used in combination.

  In the image processing method of the present embodiment, next, a process of calculating eigenvalues and eigenfunctions by performing eigenvalue decomposition on the converted template image group is performed. Here, the same number of eigenvalues and eigenfunctions as the number of converted template images constituting the converted template image group are calculated.

  Therefore, as described above, when the direction angle is used as the geometric transformation applied to the template image 1 and a post-conversion template image group including 360 post-conversion template images is generated, the post-conversion unit consisting of the 360 images is generated. Eigenvalue decomposition is performed on the template image group, and 360 eigenvalues and eigenfunctions are obtained. Further, when a two-dimensional coordinate value is used as a geometric transformation applied to the template image 1 and a converted template image group including 64 converted template images is generated, the converted template image group including the 64 images. Is subjected to eigenvalue decomposition, and 64 eigenvalues and eigenfunctions are obtained.

As eigenfunctions calculated by performing eigenvalue decomposition, eigenfunctions φ ₁ (θ) to φ ₃ (θ) represented by waveforms p1 to p3 are obtained, for example, as shown in FIG. These eigenfunction groups can be said to be main eigenfunction groups. In each graph showing the waveforms p1 to p3, the horizontal axis represents the direction angle (θ) with reference to the template image 1, and the vertical axis represents the eigenfunction value. As shown in the waveforms p1 to p3, the waveform obtained as the eigenfunction is a continuous waveform in which eigenfunction values continuously exist with respect to the direction angle.

In the eigenfunction, as shown by the waveforms p1 to p3 in FIG. 2, the eigenfunction value periodically varies with the change in the direction angle. These eigenfunctions are calculated as functions (P _n (θ)) representing image similarity by multiplying correlation values (r _n ) described later in the image processing method of the present embodiment.

  As described above, in the image processing method of the present embodiment, after the first image group generation step, eigenvalue decomposition is performed on the converted template image group, so that the same number of eigenvalues and eigenfunctions as the plurality of converted template images are obtained. An eigenvalue decomposition step for calculating is performed.

  In the image processing method of the present embodiment, subsequently, a process of generating a new image group by performing an inner product operation on the plurality of eigenfunctions calculated as described above and the converted template image group is performed. The new image group generated here (hereinafter referred to as “eigenvalue decomposition template image group”) corresponds to the second image group in the present embodiment.

  As the eigenvalue decomposition template image group, for example, a plurality of images as shown in eigenvalue decomposition template images E1 to E3 in FIG. 2 are obtained. Each eigenvalue decomposition template image E1 to E3 corresponds to each waveform p1 to p3 representing an eigenfunction.

  As described above, in the image processing method of the present embodiment, the eigenvalue decomposition template image group is generated by performing the inner product operation of the plurality of eigenfunctions and the converted template image group after the eigenvalue decomposition step. An image group generation step is performed.

  In the image processing method of the present embodiment, next, a process of calculating the image similarity with respect to the eigenvalue decomposition template image group is performed for the input image 2.

  The principle of calculating the image similarity will be described. Here, the description will be made assuming that the geometric transformation applied to the template image 1 in the first image group generation step described above is a direction angle.

  The template image 1 is T (x, y), the input image 2 is f (x, y), and the template image T (x, y) is rotated by θ, that is, the converted template image is T (x, y; θ ), The image similarity g (θ) between the template image 1 and the input image 2 is calculated by the following equation (1). Note that x and y representing each image indicate two-dimensional coordinate values of pixels in the image.

When g (θ) is expanded by the eigenfunction sequence φ _n (θ), (n = 0, 1, 2,...) along the θ direction, the following equation (2) is obtained.

  The following equation (3) is obtained by exchanging the order of the integral operation of the above equation (2).

Here, E _n (x, y) defined as the following equation (4) is an image calculated by the inner product of the converted template image T (x, y; θ) and the eigenfunction φ _n (θ), that is, It becomes an eigenvalue decomposition template image.

The image represented by the above equation (4) can be calculated separately from the input image f (x, y). Therefore, by calculating E _n (x, y) in advance, the input image f (x, y) is given thereafter, and from the result of matching with E _n (x, y), all directions The corner image similarity (g (θ)) can be calculated.

As the image similarity (g (θ)) of the omnidirectional angles, for example, as shown in FIG. 2, eigenfunctions φ ₁ (θ) to φ ₃ (θ) represented by waveforms p1 to p3 are at a predetermined ratio. A waveform qs obtained by adding together is calculated. As shown in the waveform qs, the waveform obtained as the image similarity (g (θ)) for all directional angles is a continuous waveform (continuous curve) in which the image similarity is continuously present for the directional angles. That is, the image similarity (g (θ)) represents the distribution of the image similarity in the direction angle (θ) direction.

Specifically, the image similarity (g (θ)) is calculated as follows. In the example shown in FIG. 2, the eigenfunctions φ ₁ (θ) to φ ₃ (θ) are multiplied by correlation values r _{1 to} r ₃ , respectively. By multiplying eigenfunctions φ ₁ (θ) to φ ₃ (θ) by correlation values r _{1 to} r ₃ , functions P ₁ (θ) to P ₃ (θ) representing image similarity are calculated. As shown in FIG. 2, the functions P ₁ (θ) to P ₃ (θ) are waveforms q1 in which waveforms p1 to p3 as eigenfunctions φ ₁ (θ) to φ ₃ (θ) are represented at a predetermined ratio. This predetermined ratio corresponds to the correlation values r _{1 to} r ₃ . In each graph showing the waveforms q1 to q3 representing the functions P ₁ (θ) to P ₃ (θ), the horizontal axis indicates the direction angle (θ) with reference to the template image 1, and the vertical axis indicates the image similarity. . Then, by adding the functions P ₁ (θ) to P ₃ (θ), the image similarity (g (θ)) represented by the waveform qs is calculated. Here, the correlation value r _n, a predetermined real number representing the correlation between the input image and each eigenvalue decomposition template image eigenvalue decomposition template image group is defined as the following equation (5).

Thus, the correlation value r _n, f (x, y) and the input image 2 represented by, the inner product value of the eigenvalue decomposition template image group represented by the formula (4). Therefore, from the above equations (3) and (5), the image similarity (g (θ)) is expressed by the following equation (6).
g (θ) = Σr _n × φ _n (θ) (6)

  As described above, in the image processing method according to the present embodiment, after the second image group generation step, the image similarity that is represented by a continuous curve and indicates the degree of similarity between the input image 2 and the eigenvalue decomposition template image group is calculated. An image similarity calculation step is performed.

  In the image processing method according to the present embodiment, based on the image similarity (g (θ)) calculated as described above, the value (real value) of the geometric transformation parameter of the input image 2 based on the template image 1. The process of determining is performed. That is, in this process, the template image 1 and the input image 2 are collated using the image similarity (g (θ)) value calculated as described above as an index.

  Specifically, as shown in FIG. 2, in the waveform qs which is the image similarity (g (θ)) of all direction angles, the direction angle θa at which the image similarity is maximum is based on the template image 1. It is determined as the value of the geometric transformation parameter of the input image 2. That is, the direction angle of the input image 2 based on the template image 1 is calculated as the direction angle θa as a collation result of the image processing method of the present embodiment.

  Here, the direction angle θa obtained as a matching result corresponds to the rotation angle of the image when the direction angle of the template image 1 is set as a reference (0 °). That is, in this case, in the range in which the template image 1 is rotated by 360 °, the template image 1 in the state rotated by the angle θa has the highest image similarity in relation to the input image 2.

  As described above, in the image processing method of the present embodiment, the template image 1 is used as a reference based on the image similarity (g (θ)) calculated in the image similarity calculation step after the image similarity calculation step. A collation step for determining the value of the geometric transformation parameter of the input image 2 is performed. In the collation step, focusing on the continuity of the image similarity due to the continuity of the eigenfunction, the real value of the geometric transformation parameter is determined.

  In the image processing method as described above, as described above, after the eigenvalue decomposition step for performing eigenvalue decomposition on the post-conversion template image group, among the plurality of eigenfunctions calculated in the eigenvalue decomposition step, predetermined values are arranged in descending order. Preferably, a selection step is performed to select a number of eigenfunctions.

  In the image processing method of the present embodiment, the same number of eigenvalues and eigenfunctions as the plurality of post-conversion template images are calculated in the eigenvalue decomposition step. That is, as described above, when 360 converted template images are obtained by applying a rotation of 1 ° pitch as geometric transformation to the template image 1, 360 eigenvalues and eigenfunctions are calculated. In this case, when all eigenfunctions are used in generating the eigenvalue decomposition template image group, the eigenvalue decomposition template image group becomes 360 eigenvalue decomposition template images.

  Therefore, a predetermined number of eigenfunctions are selected from the same number of eigenfunctions as the converted template images calculated in the eigenvalue decomposition step. The selection of the eigenfunction is performed from the one with the larger eigenvalue. That is, in the selection step, a predetermined number of eigenfunctions calculated in the eigenvalue decomposition step are selected from the larger eigenvalue side. Therefore, for example, when 360 eigenfunctions are calculated as described above, 60 eigenfunctions with larger eigenvalues are selected. This is based on the fact that the eigenfunction having a larger eigenvalue has a greater influence (higher contribution) on the image similarity between the input image 2 and the eigenvalue decomposition template image group that is finally calculated.

  When such a selection step is performed, in the second image group generation step described above, an eigenvalue is calculated by performing an inner product operation of the plurality of eigenfunctions selected in the selection step and the converted template image group. A decomposition template image group is generated. In other words, according to the selection step, an eigenvalue decomposition template image group is generated by using a part of the eigenvalues calculated in the eigenvalue decomposition step, a part of the eigenvalues having larger eigenvalues.

  Specifically, such a selection step corresponds to approximating the series by a predetermined number in Expression (3). Here, when the series of Expression (3) is approximated by M truncation, Expression (3) is expressed by the following Expression (7).

At this time, by selecting an eigenfunction sequence φ _n (θ) that can approximate g (θ) with a small order M, the image similarity at all angles is M images E _n (x, y ). Then, as a method of selecting the eigenfunction sequence φ _n (θ), the eigenvalue λ _n and the eigenfunction φ _n (θ) of the post-conversion template image sequence T (x, y; θ) are calculated. Select eigenfunctions.

Specifically, θ = {θ ₁ , θ ₂ ,..., Θ _N } and vector φ _n obtained by discretization, template image T _i (x, y) = T (x, y; θ _i ) Then, eigenvector φ _n and eigenvalue λ _n can be obtained by solving the matrix eigenvalue problem of the following equation (8).
Cφ _n = λ _n φ _n (8)

  Here, the matrix C is a covariance matrix in which the i-th and j-th elements are represented by the following equation (9).

In Expression (9), μ _j is an average density value of T _i (x, y). An image E _n (x, y) obtained using this eigenvector corresponds to an eigenvalue decomposition template image.

  According to the image processing method of the present embodiment as described above, it is equivalent to a conventional image matching technique using geometric transformation, such as a rotation matching method that performs matching while rotating a template image, with a small number of matching times. Accuracy can be ensured and processing time can be shortened. Here, the rotation collation method, which is a conventional image collation technique, will be described with reference to FIG.

  As shown in FIG. 12A, in the rotation collation method, the image similarity in relation to the input image 102 while the rotation angle (direction angle) and position (two-dimensional coordinate value) of the template image 101 are changed. Is calculated. Then, the direction angle and the two-dimensional coordinate value that maximize the image similarity are detected as the direction angle and the two-dimensional coordinate value of the input image 102.

  In graph G1 shown in the lower part of FIG. 12A, the horizontal axis represents the direction angle of the template image 101, that is, the rotation angle of the template image 101 to be rotated, and the vertical axis represents the image of the template image 101 and the input image 102. Indicates the similarity. That is, the graph G1 shown in the lower part of FIG. 12A shows a change in image similarity of the template image 101 with respect to the input image 102 due to a change in the direction angle of the template image 101.

  Further, in FIG. 12A, the middle stage shows the process of rotation of the template image 101 that rotates in accordance with the direction angle in the graph G1 shown in FIG. For example, the template image 101 is rotated 360 ° at a rotation pitch of 1 ° (only four images are shown in FIG. 10 for convenience). In FIG. 12A, the upper part shows an input image 102 that is a detection target in the rotation matching method.

  In such a rotation collation method, as shown in FIG. 12A, the direction angle P1 that maximizes the image similarity is detected as the direction angle of the input image 102. That is, according to the rotation matching method, the template image 101 is rotated at a predetermined rotation pitch, and the rotation angle (direction angle) that approximates the input image 102 is searched for the template image 101. Direction angles and the like are detected.

  In such a rotation collation method, in order to increase the detection accuracy, collation is made by reducing the rotation pitch of the template image 101 for searching for the rotation angle (direction angle P1) that the template image 101 is closest to the input image 102. It is necessary to increase the number of times. For this reason, increasing the detection accuracy in the rotation matching method leads to a long processing time. On the other hand, if the rotation pitch of the template image 101 is roughened (enlarged) in order to shorten the processing time, it becomes difficult to obtain sufficient detection accuracy.

  Therefore, as shown in FIG. 12B, in order to reduce the processing time in the image matching method, the calculated image similarity is calculated with a quadratic curved surface while performing the matching with a coarse rotation pitch of the template image 101. There is a method of interpolation using, for example.

  Specifically, in the case shown in FIG. 12B, for example, the template image 101 and the input image 102 are collated at nine directional angles R1 to R9 with a pitch of 36 °. Then, the image similarity is calculated at each of R1 to R9, and based on the image similarity at each location, the image similarity in the range between each location is interpolated by a quadric surface or the like to obtain an estimated value. A graph G2 is obtained. From this graph G2, the direction angle P2 having the maximum image similarity is detected as the estimated value of the direction angle of the input image 102.

  As shown in FIG. 12 (b), the direction angle P2 as the estimated value is a value shifted from the direction angle P1 shown in FIG. 12 (a) due to a relatively fine rotation pitch. Thus, according to the method of performing interpolation, an error due to interpolation occurs, and the detection accuracy decreases.

  Another example of the conventional image matching technique will be described with reference to FIG. As shown in FIG. 13, in the image matching technique method of this example (hereinafter referred to as “conventional method”), a plurality of registered images 201 having different direction angles are registered in advance. In the example shown in FIG. 13, four registered images 201a, 201b, 201c, and 201d having directional angles of 0 °, 30 °, 45 °, and 90 ° are registered.

The four registered images 201a, 201b, 201c, and 201d are converted into feature vectors v _{1 to} v ₄ by eigenvalue decomposition (principal component analysis), respectively. The feature vector is a registered image obtained by information compression.

Further, in the conventional method, the input image 202, similarly to the registration image 201 is converted into a feature vector _{v in} the eigenvalue decomposition. Then, for each feature vector _v 1 to v ₄ based on the registered image 201, the difference is obtained between the feature vector _{v in} based on the input image 202, registered image 201 is detected that the value of the difference is minimized. In the example shown in FIG. 13, a registered image 201 c having a direction angle of 45 ° is detected as the registered image 201 closest to the input image 202, and the direction angle 45 ° of the registered image 201 c is determined as the direction angle of the input image 202. Is done.

  In such a conventional method, in order to increase the detection accuracy, it is necessary to increase the number of registered images 201 so that the angle interval between the direction angles in the plurality of registered images 201 becomes small. As the number of registered images 201 increases, the number of comparisons for obtaining a difference between feature vectors of the registered image 201 and the input image 202 also increases, and the processing burden increases. For this reason, increasing the detection accuracy in the conventional method leads to a long processing time. On the other hand, if the number of registered images 201 is reduced in order to shorten the processing time, the detection resolution becomes coarse and it is difficult to obtain sufficient detection accuracy.

  As described above, according to the image processing method of the present embodiment, in comparison with the conventional image matching technology such as the rotation matching method and the conventional method in which it is difficult to simultaneously reduce the processing time and improve the detection accuracy. When the above processing time is taken, high detection accuracy can be obtained, and when the equivalent detection accuracy is to be ensured, the processing time can be shortened. Specifically, according to the image processing method of the present embodiment, the geometric vector is not based on the feature vector comparison as in the conventional method described above, but based on the image similarity that appears as a continuous waveform for the geometric transformation parameter to be determined. Since the conversion parameter is determined, highly accurate detection processing can be performed.

  In the image processing method of the present embodiment, as described above, the selection step of selecting a part of the eigenfunctions in descending order of the eigenvalues is performed, thereby further reducing the processing time while ensuring the detection accuracy. be able to. That is, according to the selection step according to the present embodiment, some eigenfunctions having a large influence (contribution) on the image similarity are used among the same number of eigenfunctions as the converted template images. Thereby, in comparison with the case where all of the calculated eigenfunctions are used, sufficient accuracy can be ensured for the determined geometric transformation parameters, and the processing speed can be increased by the eigenfunctions not selected. The processing time can be further shortened.

  In the image processing method of the present embodiment, the following method can be employed. In the image processing method of the present embodiment, as the number of post-conversion template image groups obtained by applying geometric transformation to the template image 1 as described above increases, the continuity of the eigenfunction increases. Therefore, in the image processing method of this embodiment, it is preferable to increase the number of converted template images to infinity. That is, in the series of processes in the image processing method of the present embodiment as described above, the number of converted template images constituting the converted template image group is mathematically infinite.

In this case, the above equation (8) of the matrix eigenvalue problem becomes the following equation (10).

In the above formula (10), K (s, t) is represented by the following formula (11).

  That is, instead of the matrix eigenvalue problem, equation (10) is converted into a differential equation problem. By increasing the number of template images after conversion to infinity, the eigenfunction becomes a continuous function in a true sense. For example, this continuous function can be approximated by a polynomial and expressed analytically.

  By expressing the eigenfunction by a polynomial, an image similarity function (image similarity g) that is a weighted sum of eigenfunctions using correlation values is also expressed by a polynomial. The process of finding the maximum value of the image similarity function expressed by a polynomial may be performed by differentiating the polynomial and finding a position where it becomes zero. In this way, by increasing the number of post-conversion template images to infinity, it is possible to analytically determine the estimated value of the parameter, and the estimation accuracy of the value of the geometric parameter can be greatly increased.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include a direction angle and a two-dimensional coordinate value, and the geometric transformation applied to the template image 1 is a rotation that changes the direction angle. It is preferable to include a translation that changes the two-dimensional coordinate values.

  The direction angle and the two-dimensional coordinate value are geometric transformation parameters that are relatively frequently used in industrial image matching processing, and are two-dimensional geometric transformation parameters. The image processing method of the form can be performed. For this reason, high versatility can be obtained.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include parameters necessary for projective transformation, and the geometric transformation applied to the template image 1 includes projective transformation.

  The projective transformation is a general projective transformation including an affine transformation, and converts the coordinates (x, y) of the original image into the coordinates (x ′, y ′) on the projection plane by a predetermined transformation formula. It is. Accordingly, the parameters necessary for the projective transformation are coefficients in the general formula of the projective transformation.

  Therefore, when projective transformation is used as the geometric transformation applied to the template image 1 in the image processing method of the present embodiment, the projection transformation is added to the template image 1 in the first image group generation step, so that A template image group is generated. Then, in the collating step, the value of the coefficient in the general formula of the projective transformation is determined as the value of the geometric transformation parameter of the input image 2.

  In the image processing method of the present embodiment, the geometric transformation parameters determined for the input image 2 include parameters necessary for the image blurring process, and the geometric transformation applied to the template image 1 performs the image blurring process. Including.

  The image blurring process is a process for blurring an image, and is performed as follows, for example. As an image blurring process, for example, an average of colors (color values) of 3 × 3 pixels (9 pixels) centering on a pixel (x, y) to be processed is used as a color (color value) of the pixel to be processed. Is performed. Such processing is also referred to as smoothing. The image blurring process is performed, for example, for each of R (red), G (green), and B (blue) components constituting the pixel color. As described above, when the average of the colors of 3 × 3 pixels is adopted as the color of the pixel to be processed, the weights of the pixels in the averaging process of 9 pixels are all the same (“1”).

  Therefore, in image blurring processing, the weight of each of the surrounding 8 pixels is doubled with respect to the pixel to be processed, or the average of the colors of only four pixels above, below, left, and right of the pixel to be processed is processed. It may also be used as the color. In the former case, the weight of the pixel to be processed is “1” and the weight of each surrounding pixel is “2”. In the latter case, the weight of the upper, lower, left, and right pixels of the pixel to be processed is “1”. The weight of each pixel of 5 pixels is “0”.

  Thus, the weight of each pixel (numerical value such as “1” or “2”) in the averaging process performed in the image blurring process is a parameter necessary for the image blurring process. In addition, the number of pixels in the range to be averaged as the color of the pixel to be processed (vertical N × horizontal N (pieces)) is also a parameter necessary for the image blurring process. Note that the greater the value of N, the greater the degree of image blur. Parameters necessary for such an image blurring process are included in the geometric transformation parameters determined for the input image 2.

  Therefore, in the image processing method of the present embodiment, when image blurring processing is used as geometric transformation applied to the template image 1, image blurring processing is added to the template image 1 in the first image group generation step. Thus, a post-conversion template image group is generated. In the collation step, the value of the parameter necessary for the image blurring process as described above is determined as the value of the geometric transformation parameter of the input image 2. The image blurring process is not limited to the image blurring process described above, and a well-known image blurring process can be appropriately employed.

  As described above, in the image processing method of the present embodiment, by using projective transformation and image blurring processing, it is possible to shorten processing time and improve detection accuracy in more advanced image matching processing. Become.

  Specifically, with respect to the input image 2, when the part to be inspected is imaged, the imaging direction by the imaging unit such as a camera is not perpendicular to the imaging plane set in the inspection object. In the image 2, there is a case in which a part to be inspected is distorted into a trapezoid (so-called trapezoidal distortion). Such trapezoidal distortion that occurs in the input image 2 causes detection errors.

  Therefore, by using projective transformation in the image processing method of the present embodiment as described above, it is possible to accurately perform image matching even when trapezoidal distortion occurs in the input image 2. That is, by preparing in advance a plurality of template images 1 that have undergone projective transformation as a group of converted template images, an eigenvalue decomposition template image that takes into account the effect of trapezoidal distortion can be generated, and errors due to trapezoidal distortion are small. Accurate image matching can be performed.

  Similarly, regarding the input image 2, “blur” is included in the input image 2 due to the distance between the imaging means such as a camera and the inspection object. Normally, for such blurring of an image, adjustment (focusing) with movement of a camera or a lens is performed by autofocus processing or the like. However, such adjustment with movement of the camera and lens is a mechanical process, so that the processing time becomes long.

  Therefore, as described above, by using the image blurring process in the image processing method of the present embodiment, even if the input image 2 is blurred, a template that has been subjected to the blurring process in various patterns in advance. By preparing the image 1, it is possible to cope with it. That is, by preparing in advance a plurality of template images 1 subjected to blurring processing as a template image group after conversion, an eigenvalue decomposition template image that takes into account the effect of image blur can be generated, and errors due to image blurring can be generated. Therefore, it is possible to perform accurate image matching with a small amount of image. In addition, since processing that causes longer image processing time, such as autofocus processing, can be omitted, processing time for image matching can be shortened.

  The blurring process will be further described. In the actual inspection using the image processing method of the present embodiment, the input image 2 is an image caused by a focus shift of the lens of the imaging unit in addition to geometric deformation such as noise and rotation of the inspection object. Blurring occurs. When image collation using the template image 1 is performed on the input image 2 which is a blurred image, the positional deviation becomes large. That is, when the input image 2 is blurred, the image matching between the template image 1 that is not blurred and the blurred input image 2 is performed, the image matching is not successful, and a positional deviation occurs. As described above, misalignment caused as a result of image collation is a serious problem.

  Therefore, by applying blurring processing to the template image 1 as described above, the template image 1 that is intentionally blurred is used for image matching, so that even if the input image 2 is blurred, the input image 2 and The template image 1 can be matched. Thereby, it is possible to avoid the problem of misalignment due to blurring of the input image 2. That is, matching accuracy can be increased by adding blur to the template image 1 in accordance with the blurred input image 2.

  In general, the image blurring process is used for noise removal. However, in the image processing method according to the present embodiment, blurring is performed by performing a blurring process on the template image 1 that is the target of matching of the input image 2. Is used to create a template image 1 to which is added. Thus, the image processing method of the present embodiment is characterized by the use of the image blurring process.

  Regarding image blur, there can be many variations from small image blur to large image blur. In order to cope with such a large number of variations in the image, it is necessary to prepare a large number of template images 1 to which the blur has been added in accordance with the variation in the image blur. As described above, if image collation using a large number of template images 1 corresponding to the blur of all images is performed, it is considered that the calculation time becomes very long.

  In this regard, in the image processing method of the present embodiment, a template image group with various blurs is input as a template image after conversion, and these are compressed using eigenvalue decomposition to reduce the calculation time as described above. The problem can be solved. That is, since the post-conversion template image is information-compressed by eigenvalue decomposition, it is possible to suppress an increase in calculation time even if there are many variations in image blur.

  Note that the projective transformation and the image blurring processing as geometric transformation are used independently or in combination in relation to the above-described directional angle and two-dimensional coordinate values.

  Next, as an application example of the image processing method of the present embodiment, an apparatus for performing the image processing method of the present embodiment will be described. In this application example, for example, in an IC chip production line, image verification is used in the process of positioning the IC chip on the IC substrate.

  As shown in FIGS. 3 and 4, the image processing apparatus 10 according to the present embodiment includes a computer 11 and a camera 12. The image processing apparatus 10 images the IC chip 20 as a work with the camera 12, and uses the captured image as an input image to perform image collation using a template image prepared in advance in the computer 11. The image processing apparatus 10 according to the present embodiment detects a direction angle and a two-dimensional coordinate value of an input image with reference to a template image as an image matching result.

  Then, by using the image collation result by the image processing apparatus 10, the IC chip 20 picked by the arm 17 from the IC chip supply tray 16 accommodating the plurality of IC chips 20 is positioned at a predetermined position on the IC substrate. Is done. Note that the arm 17 is a component-extracting configuration such as a robot arm or a picking arm that constitutes a part of a device that transports the IC chip 20, for example.

  Specifically, the positioning of the IC chip 20 is performed as follows. First, a predetermined IC chip 20 in the IC chip supply tray 16 is picked by the arm 17 (see arrow A1). The picked IC chip 20 is captured by the camera 12 while being held by the arm 17. The captured image of the IC chip 20 is transmitted to the computer 11 as an input image, and image verification is performed using a template image.

  The direction angle and two-dimensional coordinate value of the input image as an image collation result by the image processing apparatus 10 correspond to the angle and position of the IC chip 20 that is the imaging target. Therefore, the arm 17 holding the IC chip 20 is rotated and translated when positioning the IC chip 20 based on the result of image collation by the image processing apparatus 10. After the arm 17 is rotated and translated, the IC chip 20 is positioned at a predetermined position on the IC substrate as the arm 17 moves in the vertical direction.

  As shown in FIG. 4, the computer 11 includes an arithmetic control unit 13, an input unit 14, and a display unit 15. The arithmetic control unit 13 controls a series of operations of the image processing apparatus 10. The calculation control unit 13 includes a storage unit that stores a program and the like, a development unit that expands the program and the like, a calculation unit that performs a predetermined calculation according to the program and the like, a storage unit that stores calculation results and the like by the calculation unit, and the like.

  Specifically, a configuration in which a CPU, ROM, RAM, HDD, or the like is connected by a bus, or a configuration that includes a one-chip LSI or the like is used as the arithmetic control unit 13. As the arithmetic control unit 13, in addition to a dedicated product, a commercially available personal computer, a workstation or the like in which the above program is stored is used.

  The input unit 14 is connected to the calculation control unit 13 and inputs various information / instructions related to image processing to the calculation control unit 13. For example, a keyboard, a mouse, a pointing device, a button, a switch, or the like is used as the input unit 14.

  The display unit 15 is connected to the calculation control unit 13 and displays an operation status of image processing, an input content from the input unit 14 to the calculation control unit 13, a processing result by the image processing, and the like. For example, a liquid crystal display, a CRT (cathode ray tube), or the like is used as the display unit 15.

  The camera 12 is a CCD camera, for example, and is connected to the computer 11 via a connection cable. The camera 12 takes an image of the IC chip 20 to be picked by the arm 17 by moving relative to the IC chip 20 in the IC chip supply tray 16. In the present embodiment, the camera 12 captures the IC chip 20 from above and captures a planar image of the IC chip 20.

  Imaging data about the IC chip 20 captured by the camera 12 is sent to the computer 11 and input to the arithmetic control unit 13 as an input image 22 (see FIG. 5). As described above, in the image processing apparatus 10 of the present embodiment, the camera 12 functions as an input image acquisition unit for acquiring the input image 22.

  As shown in FIG. 5, in the image processing apparatus 10 according to the present embodiment, the calculation control unit 13 provided in the computer 11 performs collation using a predetermined template image 21, thereby making an input based on the template image 21. It functions as an image processing means for determining the value of the geometric transformation parameter of the image 22. Therefore, the program stored in the storage unit in the arithmetic control unit 13 includes a program for performing image matching between the template image 21 and the input image 22 captured by the camera 12.

  As a storage part of the program or the like in the arithmetic control unit 13, for example, a storage device built in the computer 11 such as a RAM, a CD (Compact Disk), an FD (Flexible Disk), an MO (Magneto-Optical Disk), a DVD ( A storage device such as a digital versatile disk (HD) or a hard disk (HD) is appropriately used.

  As shown in FIG. 4, the arithmetic control unit 13 includes an eigenvalue decomposition parameter image creation processing unit 18 and a detection processing unit 19. The eigenvalue decomposition parameter image creation processing unit 18 generates a converted parameter image group based on a template image 21 stored in advance, performs eigenvalue decomposition on the generated converted parameter image group, and converts the calculated eigenfunction and conversion An eigenvalue decomposition template image group is created from the subsequent template image group.

  The detection processing unit 19 calculates a correlation value between the input image 22 acquired by the camera 12 and the eigenvalue decomposition template image group generated by the eigenvalue decomposition parameter image creation processing unit 18. The detection processing unit 19 calculates the image similarity from the calculated correlation value, and determines the value of the geometric transformation parameter of the input image 22 based on the template image 21 based on the calculated image similarity.

  As shown in FIG. 4, the arithmetic control unit 13 includes a template image storage unit 30, a post-conversion template image generation unit 31, an eigenvalue decomposition unit 32, an eigenfunction selection unit 33, and an eigenvalue decomposition template image generation unit 34. The image similarity calculation unit 35 and the collation unit 36 are included.

  The template image storage unit 30 stores the template image 21 (see FIG. 5). For example, the template image 21 is obtained by imaging the same components as the IC chip 20 to be imaged by the camera 12 from the same angle as the camera 12. The template image 21 is a target to be collated with the input image 22 as an image indicating the direction angle and the reference of the two-dimensional coordinate value in the step of positioning the IC chip 20 on the IC substrate.

  Specifically, the template image 21 is created so that the portion (size) occupied by the IC chip 20 in the image is approximately the same in relation to the input image 22 acquired by the camera 12. In other words, for the camera 12, the imaging position of the IC chip 20 is set so that the portion occupied by the IC chip 20 in the input image 22 to be acquired is the same as the portion occupied by the IC chip 20 in the template image 21. Is done. Such a template image 21 is preset and stored in the template image storage unit 30 of the calculation control unit 13.

  The post-conversion template image generation unit 31 performs a process corresponding to the first image group generation step in the above-described image processing method. That is, the post-conversion template image generation unit 31 generates a post-conversion template image group as the first image group by performing geometric transformation on the template image 21 stored in the template image storage unit 30.

  The post-conversion template image generation unit 31 performs a rotation for changing the direction angle and a parallel movement for changing the two-dimensional coordinate value as the geometric transformation applied to the template image 21. As described above, in the image processing apparatus 10, the post-conversion template image generation unit 31 performs geometric transformation on the template image 21, thereby converting post-conversion template image groups including a plurality of post-conversion template images having different amounts of geometric transformation. Functions as a first image group generation unit for generating.

  The eigenvalue decomposition unit 32 performs processing corresponding to the eigenvalue decomposition step in the above-described image processing method. That is, the eigenvalue decomposition unit 32 calculates eigenvalues and eigenfunctions by performing eigenvalue decomposition on the converted template image group.

  The eigenvalue decomposition unit 32 calculates the same number of eigenvalues and eigenfunctions as the converted template images that constitute the converted template image group. As described above, in the image processing apparatus 10, the eigenvalue decomposition unit 32 performs eigenvalue decomposition on the converted template image group to calculate the same number of eigenvalues and eigenfunctions as the plurality of converted template images.

  The eigenfunction selection unit 33 performs processing corresponding to the selection step in the above-described image processing method. That is, in the image processing apparatus 10, the eigenfunction selection unit 33 functions as a selection unit that selects a predetermined number of eigenfunctions in descending order of eigenvalues among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit 32. . That is, the eigenfunction selection unit 33 selects some eigenfunctions on the side having a larger eigenvalue from among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit 32.

  The eigenvalue decomposition template image generation unit 34 performs processing corresponding to the second image group generation step in the image processing method described above. That is, the eigenvalue decomposition template image generation unit 34 performs an inner product operation on the plurality of eigenfunctions calculated by the eigenvalue decomposition unit 32 and the converted template image group generated by the converted template image generation unit 31, thereby performing eigenvalue decomposition. Processing for generating a template image group is performed.

  As described above, the eigenvalue decomposition template image generation unit 34 functions as a second image group generation unit that generates an eigenvalue decomposition template image group by performing an inner product operation of the plurality of eigenfunctions and the converted template image group. . In the configuration having the eigenfunction selection unit 33 as in the image processing apparatus 10, the eigenvalue decomposition template image generation unit 34 is an inner product of a plurality of eigenfunctions selected by the eigenfunction selection unit 33 and the converted template image group. By performing the calculation, an eigenvalue decomposition template image group is generated.

  The image similarity calculation unit 35 performs processing corresponding to the image similarity calculation step in the above-described image processing method. That is, the image similarity calculation unit 35 calculates the image similarity (g (θ)) for the eigenvalue decomposition template image group for the input image 22 based on the above formulas (1) to (9). That is, in the image processing apparatus 10, the image similarity calculation unit 35 calculates an image similarity that is represented by a continuous curve and indicates the degree of similarity between the input image 22 and the eigenvalue decomposition template image group.

  The collation unit 36 performs processing corresponding to the collation step in the above-described image processing method. That is, the collation unit 36 determines the value of the geometric transformation parameter of the input image 22 based on the template image 21 based on the image similarity (g (θ)) calculated by the image similarity calculation unit 35.

  The collation unit 36 collates the template image 21 and the input image 22 with the image similarity value calculated by the image similarity calculation unit 35 as described above as an index. Thus, in the image processing apparatus 10, the collation unit 36 functions as a collation unit that determines the value of the geometric transformation parameter based on the image similarity calculated by the image similarity calculation unit 35. In the image processing apparatus 10 according to the present embodiment, the matching unit 36 determines the direction angle and the two-dimensional coordinate value that maximize the image similarity as the value of the geometric transformation parameter of the input image 22 with the template image 21 as a reference. .

  As described above, according to the image collation performed by each unit included in the calculation control unit 13 in the image processing apparatus 10 of the present embodiment, the direction angle and the two-dimensional coordinate value of the input image 22 are obtained as the collation result. Specifically, as shown in FIG. 5, as the direction angle included in the collation result, for example, the inclination (rotation angle) θ of the input image 22 with respect to the reference line O10 indicating the direction angle of the template image 21 is obtained. . As the two-dimensional coordinate value included in the collation result, for example, the two-dimensional coordinate value (XY coordinate value) of the center position C10 of the input image 22 when the center position of the template image 21 is the origin O20 is obtained.

  An example of the processing procedure of image collation performed by the image processing apparatus 10 of the present embodiment will be described with reference to the flowchart shown in FIG.

  As shown in FIG. 6, in the image collation processing procedure by the image processing apparatus 10, first, the template image 21 is input (S10). In step S10, a template image 21 obtained by imaging the IC chip 20 to be inspected by a predetermined method is input to the template image storage unit 30 and stored in advance.

  Next, a converted template image group is generated (S20). In this step, the post-conversion template image generation unit 31 applies geometric transformation to the template image 21 stored in the template image storage unit 30, so that a post-conversion template image group composed of N post-conversion template images is obtained. Generated. For example, the template image 21 is rotated at a pitch of 1 °, thereby generating a converted template image group including N = 360 converted template images (rotated images). In the example shown in FIG. 2, a post-conversion template image group 1N composed of N post-conversion template images is generated.

Subsequently, a plurality of eigenvalues and eigenfunctions are calculated (S30). In this step, the eigenvalue decomposition unit 32 performs eigenvalue decomposition on the post-conversion template image group (N post-conversion template images), thereby calculating eigenvalues and eigenfunctions. Here, the same number (N) of eigenvalues and eigenfunctions as the number of post-conversion template images generated by the eigenvalue decomposition unit 32 are calculated. Therefore, when 360 converted template images are generated as described above, 360 eigenvalues and eigenfunctions are calculated. In the example shown in FIG. 2, eigenvalue decomposition is performed on the post-conversion template image group 1N, so that an eigenfunction composed of N eigenfunctions φ _n (θ), (n = 1, 2,... N). A group and N eigenvalues are calculated.

Next, an eigenfunction is selected based on the size of the eigenvalue (S40). In this step, the eigenfunction selection unit 33 selects a predetermined number (M) of eigenfunctions having larger eigenvalues among the N eigenfunctions calculated by the eigenvalue decomposition unit 32. For example, when 360 (N) eigenfunctions are calculated as described above, 60 (M) eigenfunctions having a larger eigenvalue are selected. In the example shown in FIG. 2, a group of eigenfunctions φ ₁ (θ), φ ₂ (θ),..., Φ _M (θ) consisting of M eigenfunctions from N eigenfunctions φ _n (θ). Is selected.

Then, an eigenvalue decomposition template image group is generated (S50). In this step, the eigenvalue decomposition template image generation unit 34 generates the M eigenfunctions selected by the eigenfunction selection unit 33 and the converted template image generated by the converted template image generation unit 31 based on the above equation (4). An inner product operation is performed on the image group to generate an eigenvalue decomposition template image group (M eigenvalue decomposition template images). In the example illustrated in FIG. 2, an eigenvalue decomposition template image group including eigenvalue decomposition template images E1 to E3 (E ₁ (x, y) to E ₃ (x, y)) is obtained.

  Subsequently, the input image 22 is acquired (S60). In this step, the IC chip 20 picked by the arm 17 and held by the arm 17 is imaged by the camera 12, and the imaging data is input to the arithmetic control unit 13 of the computer 11. In the example shown in FIG. 2, the input image 2 is acquired and input.

  Next, the position of the input image 22 is set (S70). In this step, the position in the image, that is, the two-dimensional coordinate value (x, y) of the pixel is specified for the input image 22 acquired in step S60.

Next, the correlation value r _n (x, y) between the input image 22 and each of the M eigenvalue decomposition template images at the position (x, y) of the input image 22 is calculated (S80). In this step, the correlation value r _n, is calculated by the equation (5). When calculating the correlation value _{r n} is the position of the input image 22 that has been set in step S70 (x, y) is used. In the example shown in FIG. 2, f (x, y) for the input image 2 represented by, according to the above equation (5), the correlation value r _n is calculated.

Subsequently, the image similarity between the input image 22 and the eigenvalue decomposition template image group at the position (x, y) of the input image 22 is calculated (S90). In this step, the image similarity calculation unit 35 calculates the position (x, y) of the input image 22 set in step S70 with respect to the eigenvalue decomposition template image group according to the above formulas (3) and (6). An image similarity (g (θ, x, y)) is calculated. In the example shown in FIG. 2, the correlation values r _{1 to} r ₃ are multiplied by the eigenfunctions φ ₁ (θ) to φ ₃ (θ), so that the functions P ₁ (θ) to P ₃ ( θ) is calculated, and these functions P ₁ (θ) to P ₃ (θ) are added to calculate the image similarity (g (θ)) represented by the waveform qs.

Thus, the position (x, y) of the input image 22 that has been set in step S70 is calculated and the calculation of the image similarity g correlation value r _n for all the rows for the two-dimensional coordinate values of the input image 22 Is called. That is, the position in the input image 22 (x, y) for all pixels to identify the calculation of the calculation and image similarity g of correlation value r _n is performed.

In this processing procedure, as shown in FIG. 7A, in step S70, the position (x, y) of the input image 22 starts from the upper left pixel 22a of the input image 22 and ends at the lower right pixel 22b. As described above, the values are set while sequentially shifting by one pixel toward the right side in each row. Then, the calculation of the calculation and image similarity g of correlation value r _n is performed for each pixel. That is, in the flowchart shown in FIG. 6, the setting of the position of the input image 22 (S70), the correlation value _{r n} is calculated in (S80), the calculation of the image similarity g (S90) is a starting point position of the input image 22 The scanning is performed from the position of the pixel 22a to the pixel 22b as the end point, and is performed for all the pixels (S100). In other words, the calculation of the correlation value _{r n} (S80), and calculates the image similarity g (S90) is, for each pixel of the input image 22 is repeated for all pixels in sequence along the pixel arrangement (S70~S100) .

Then, to the pixel 22b of the end point of the input image 22, the calculation of the calculation and image similarity g of correlation value _{r n} is performed (S100, yes), image similarity g (theta, x, y) is maximum The direction angle (θ) and the position (x, y) are determined as the direction angle (θ) and the two-dimensional coordinate value (x, y) of the input image 22 (S110). In this step, based on the image similarity (g (θ)) calculated by the image similarity calculation unit 35 by the matching unit 36, the direction angle (θ) of the input image 22 with the template image 21 as a reference, A two-dimensional coordinate value (x, y) of the center position is determined. Here, the two-dimensional coordinate value (x, y) of the direction angle (θ) and the center position at which the image similarity is maximized is determined as the value (real value) of the geometric transformation parameter of the input image 22.

  In step S110, specifically, the following processing is performed. As shown in FIG. 7B, the calculation result of the image similarity g for all the pixels of the input image 22 in steps S70 to S100, that is, the image similarity of all positions (x, y) × omnidirectional angle (θ). The calculation result of g is stored in a predetermined portion of the arithmetic control unit 13 as an image similarity table 50 that is three-dimensional array data having θ, x, and y as axes.

  The image similarity table 50 represents a three-dimensional distribution of high and low image similarity in three-dimensional coordinates with θ, x, and y as axes. The image similarity table 50 is referred to, and the direction angle (θ) and the position (x, y) at which the image similarity g (θ, x, y) is maximized are the direction angle (θ ) Is adopted as a two-dimensional coordinate value (x, y).

  The direction angle (θ) determined in step S110 corresponds to the inclination (rotation angle) θ of the input image 22 with respect to the reference line O10 indicating the direction angle of the template image 21, as shown in FIG. The two-dimensional coordinate value (x, y) determined here is the two-dimensional coordinate value of the center position C10 of the input image 22 when the center position of the template image 21 is the origin O20, as shown in FIG. It corresponds to. In the example shown in FIG. 2, the direction angle θa that maximizes the image similarity g is based on the template image 1 based on the image similarity (g (θ)) represented by the waveform qs calculated as described above. As the value of the geometric transformation parameter of the input image 2.

  In the image collation processing procedure performed by the image processing apparatus 10 as described above, steps S10 to S50, which are processes for creating an eigenvalue decomposition template image group based on the template image 21 stored in advance, are performed as eigenvalue decomposition. This is performed by the parameter image creation processing unit 18. Then, the detection processing unit 19 performs steps S60 to S110 for calculating the image similarity based on the eigenvalue decomposition template image group and the input image and determining the direction angle and the two-dimensional coordinate value.

  In the actual processing steps, when the plurality of IC chips 20 are positioned, the eigenvalue decomposition template image group generation processing by the eigenvalue decomposition parameter image creation processing unit 18 is performed in advance when positioning the plurality of IC chips 20. Is called. Then, the processing for determining the direction angle and the two-dimensional coordinate value by the detection processing unit 19 is repeatedly performed using the eigenvalue decomposition template image group generated in advance in the step of positioning each IC chip 20. Become.

  In the image processing apparatus 10 described above, the directional angle and the two-dimensional coordinate value are used as the geometric transformation parameters determined for the input image 22, but as described above, as the geometric transformation applied to the template image 21. Projective transformation or image blurring processing may be used, and parameters necessary for projective transformation or image blurring processing may be used as geometric transformation parameters.

  The embodiment of the present invention described above is a preferable specific example of the present invention, and does not limit the present invention. The components in the embodiment can be appropriately replaced with existing components. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

  Examples of the present invention will be described below. In this embodiment, a 128 × 128 pixel image is used as the template image. In addition, a template image group consisting of a total of 360 template images after conversion was generated by rotating the template images at a pitch of 1 °.

  In the present embodiment, as the input image, the above-described template image is subjected to a position shift of −0.5 to 0.5 pixels in the x direction and the y direction and a rotation of 0 to 180 °, whereby a test image is obtained. Was generated. In this example, 43560 test images (input images) were generated in total by changing the positioning deviation at a 0.1 pixel pitch and rotating at a 0.5 ° pitch.

  In this example, in the image processing method according to the present invention (hereinafter referred to as “proposed method”), collation was performed with the order M = 60 (see the above formula (7)). In the present embodiment, the rotation verification method is adopted as a comparative example of the proposed method. As the rotation collation method, a method was adopted in which the template image is collated by rotating a total 360 ° while rotating the template image at a pitch of 6 °. Therefore, in both the proposed method and the rotation collation method, the collation is performed 60 times in one search. For this reason, the calculation amount is the same between the proposed method and the rotation matching method.

  In this embodiment, for the direction angle and displacement (two-dimensional coordinate value) of the test image, the actual direction angle and displacement of the test image of the values (estimated values) calculated by the proposed method and the rotation verification method, respectively. The error for was calculated.

  FIG. 8 shows the relationship between the direction angle of the test image and the error regarding the direction angle and the positional deviation. In FIG. 8A, the horizontal axis represents the direction angle of the test image, and the vertical axis represents the error with respect to the direction angle. In FIG. 8B, the horizontal axis indicates the direction angle of the test image, and the vertical axis indicates an error regarding the positional deviation. In each figure of FIG. 8, the graph shown by a solid line is data about the proposed method (Proposed), and the graph shown by a broken line is data about the rotation matching method (Rotational matching). 8A and 8B, each angle of the test image (each value on the horizontal axis) obtained by rotating at 0 ° to 180 ° with a 1 ° pitch for each of the error values of the direction angle and the positional deviation. The error value at (the value on the vertical axis) is the average value of errors at each position of 0.1 pixel pitch in the range of -0.5 to 0.5 pixels.

  As can be seen from the graphs of FIGS. 8 (a) and 8 (b), in both the proposed method and the rotation matching method, the number of matching times is 60 and the processing time is the same. In either case, the error is extremely small. Specifically, the direction angle error was 0.03 ° in the proposed method and 0.36 ° in the rotation matching method on average. Further, the error of the positional deviation was 0.04 pixels in the proposed method on average and 0.09 pixel in the rotation matching method on the average.

  Further, as can be seen from the graphs of FIGS. 8A and 8B, in the rotation matching method, since the image similarity obtained at a 6 ° pitch is interpolated, a periodic error occurs at an interval of 6 °. Has occurred. On the other hand, since the interpolation is not performed in the proposed method, it is possible to obtain high estimation accuracy for the direction angle and the positional deviation without causing a periodic error as in the rotation matching method.

  Thus, according to the proposed method, extremely high accuracy can be obtained as long as no periodic error occurs and the processing time (number of verifications) is comparable in comparison with the rotation verification method.

  Next, FIG. 9 shows the relationship between the number of verifications and the error. In the graph shown in FIG. 9, the horizontal axis represents the number of verifications required for one search, and the vertical axis represents the estimated directional angle error. Here, the number of collations corresponds to the truncation order M in the case of the proposed method (see Expression (7)), and in the case of the rotation collation method, the number of collations is determined by the rotation pitch, and is obtained by dividing 360 ° by the rotation pitch. It becomes.

  In FIG. 9, a graph indicated by a solid line is data on the proposed method, and shows an average error of the direction angle when the order M is changed from 10 to 360. In FIG. 9, a graph indicated by a broken line is data about the rotation matching method, and shows an average error of the direction angle when the rotation pitch is changed.

  From the solid line graph in FIG. 9, it can be seen that in the proposed method, the direction angle error converges from the vicinity of the number of collations of 60 (corresponding to the order of M = 60). The error value at this time was 0.03 °. On the other hand, it can be seen from the broken line graph in FIG. 9 that in the rotation matching method, the direction angle error converges when the number of matching times is around 120 times. From these experimental results, it was found that according to the proposed method, the same accuracy can be obtained with about half the number of collations of the rotation collation method.

  Subsequently, in order to verify the tolerance of the proposed method against noise, an experiment was performed by adding white noise (σus, 0, 10, 20, 30) to the test image. In this experiment, a phase-only correlation method (see Non-Patent Documents 4 to 6) was adopted as a comparative example of the proposed method in addition to the rotation matching method. The phase-only correlation method is a technique that is currently considered to have the highest accuracy, and has been put into practical use for industrial use. Note that the order M = 60 (the number of collations was 60) for the proposed method, and the collation was performed at a 6 ° pitch (the number of collations was 60) for the rotation collation method.

  FIGS. 10A and 10B show estimation errors of the direction angle and the positional deviation for each of the proposed method, the rotation matching method, and the phase only correlation method. FIG. 10A shows the estimation error of the direction angle (°) when white noise is added, and FIG. 10B shows the estimation error of the positional deviation amount (pixel) when white noise is added. .

  10 (a) and 10 (b), it can be seen that the error value does not change even when the amount of white noise increases in the proposed method and the rotation matching method. Similarly, FIGS. 10A and 10B show that the proposed method has the smallest error value and the detection accuracy is good among the proposed method, the rotation matching method, and the phase-only correlation method.

  On the other hand, as can be seen from FIGS. 10 (a) and 10 (b), the phase-only correlation method has a small error with respect to misalignment and a relatively good accuracy, but the direction angle accuracy is better than the other two methods. It can not be said. Furthermore, with respect to the phase-only correlation method, good results were obtained when the noise was small, but relatively good results were obtained when the noise was small. However, when the noise increased, the performance deteriorated significantly. This is considered to be because in the phase-only correlation method, a process of dividing the frequency resolution by the spectral intensity is included, so that the noise of the high-frequency component is amplified and adversely affects the matching. From these experimental results, according to the proposed method, high accuracy can be obtained in comparison with the rotation matching method and the phase-only correlation method, with the estimation accuracy being hardly influenced by the magnitude of noise.

  Another embodiment of the present invention will be described. In recent years, SIFT (Scale-Invariant Feature Transform) has been used as a high-speed and high-precision pattern matching technique. SIFT has been very successful in the field of face recognition and augmented reality (AR). SIFT is a method of performing pattern matching by extracting point group information called feature points from two images and associating the feature point groups of the respective images.

  In order to detect feature points by SIFT, DOG (Differential Of Gaussian) processing is performed. Specifically, by applying Gaussian filters with different amounts of blur to the input image to create a plurality of blurred images and detecting feature points from these image groups (DOG image groups), the image blurs. And feature points that are robust to scale changes can be detected. In SIFT, if the number of blurred images is increased, the detection accuracy is improved, but the calculation amount is increased.

  In SIFT, an operation of blurring an image can be performed by collating a template image given by a Gaussian function. Therefore, the eigenvalue decomposition template image obtained by compressing a template image group of Gaussian functions with different amounts of blur, such as the eigenvalue decomposition template image in the above-described embodiment, is used for the above-mentioned DOG processing, and calculation is performed while maintaining detection accuracy. The amount can be reduced. In this case, since the template image of the Gaussian function can be described continuously with the blur amount parameter, the eigenvalue decomposition template image can be described with an analytical expression by solving the problem of Expression (11). Since the eigenvalue decomposition template image is expressed by an analytical expression, the peak position of the DOG image group can also be obtained analytically. The method of the present invention can detect feature points with high accuracy without numerical errors at high speed.

  FIG. 11 shows the result of detecting feature points in this example. In this embodiment, feature points are detected for an input image 301 as shown in FIG. If the feature points are detected correctly, a total of 16 feature points 305 are detected as shown in the image 304 in FIG. In FIG. 11, the detected feature points are represented by a graphic in which x is surrounded by ○.

  In the image 302 shown in FIG. 11B, there are a total of eight feature points 305a. Eight feature points 305a in the image 302 are feature points detected by creating five DOG images by conventional SIFT processing. Similarly, in the image 303 shown in FIG. 11C, there are a total of 12 feature points 305b. Twelve feature points in the image 303 are feature points detected by creating 15 DOG images. From these detection results, 5 DOG image groups are normally used in SIFT, but this does not provide good detection accuracy, and it is understood that all feature points cannot be detected even if the number of DOG images is increased to 15. .

  On the other hand, the result of detecting the feature points by the method according to the present invention is an image 304 shown in FIG. In the detection of feature points in the case of the image 304, the number of eigenvalue decomposition template images is 5, and the amount of calculation is the same as in the case of the image 302 shown in FIG. Nevertheless, when the method according to the present invention is used, all 16 feature points 305 are correctly detected. From this detection result, it can be seen that according to the method of the present invention, it is possible to correctly detect all feature points without increasing the number of eigenvalue decomposition template images and the amount of calculation.

  In the present invention, the eigenvalue decomposition is not performed on the gray value of the template image group, but the template image and the input image are converted into the edge direction image or the incremental code image, and then the template according to the present invention is used. Image collation can also be performed by converting an image group into an eigenvalue decomposition template image group. That is, by performing image matching in consideration of the edge direction, it is possible to perform image matching that is robust against shielding such as fluctuations in brightness and hiding part of the object.

  From the examples of the present invention as described above, it was proved that the present invention is excellent in both accuracy and calculation amount (processing time).

  The present invention has proposed a new method for rapidly detecting a geometric transformation parameter such as a position and a direction angle corresponding to a template image from an input image. The proposed method calculates a correlation value waveform by collating images obtained by eigenvalue decomposition of a template image. That is, the present invention performs eigenvalue decomposition on a template image group for image matching, extracts principal component information, and calculates a continuous similarity curve for estimating similarity of images based on this, thereby matching time. Shortening and improving collation accuracy.

  In the present invention, a geometrical difference (value of a geometric transformation parameter) between an inspection target image (input image) and a template image is reduced without lowering accuracy by using a new template image in which the number of sheets is reduced using eigenvalue decomposition. It can be said to be a compression template matching method to be estimated. Then, a second image group (eigenvalue decomposition template image group) generated by performing an inner product operation on the first image group (post-conversion template image group) that is the original template image group for the eigenfunction calculated by eigenvalue decomposition. This is used twice to generate the image and calculate the image similarity with the input image. Thereby, the image similarity represented by a continuous curve can be obtained, and it is possible to estimate the value of the geometric transformation parameter with high accuracy that cannot be obtained by the conventional method.

  Applications of the present invention include inspection and measurement of various machine parts such as automobile parts, biometrics (fingerprint authentication), and positioning of semiconductor wafers and IC substrates (semiconductor inspection, mask exposure) that require high-precision alignment. Alignment, rotation angle alignment (alignment)), appearance inspection, and the like, and the present invention has applicability in various fields.

DESCRIPTION OF SYMBOLS 1 Template image 2 Input image 10 Image processing apparatus 11 Computer 12 Camera (input image acquisition means)
13 Arithmetic control unit (image processing means)
18 eigenvalue decomposition parameter image creation processing unit 19 detection processing unit 21 template image 22 input image 31 converted template image generation unit (first image group generation unit)
32 eigenvalue decomposition unit 33 eigenfunction selection unit 34 eigenvalue decomposition template image generation unit (second image group generation unit)
35 Image similarity calculation unit 36 Verification unit

The image processing method of the present invention is an image processing method for determining a value of a geometric transformation parameter of an input image with reference to the template image by performing collation using a predetermined template image. A first image group generation step for generating a first image group composed of a plurality of template images after geometric conversion by applying geometric transformation, and eigenvalues in the first image group; Eigenvalue decomposition step of calculating eigenvalues representing the same number of eigenvalues as the template images after the plurality of geometric transformations and eigenfunctions representing the relationship between the amount of the geometric transformations based on the template images and eigenfunction values by performing decomposition And a second image group generation step of generating a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group, An image similarity is expressed by a curve indicating the degree of similarity between the second image group and the input image, the correlation value is the inner product value of the image similarity g, and the said input image second image group the _{r n,} calculates the g =? r eigenfunctions as _{φ n n × φ n, (} n = 0,1,2, ···) and the image similarity calculation step of calculating by at the image similarity calculation step And a collation step of determining a value of the geometric transformation parameter based on the image similarity.

The image processing apparatus according to the present invention performs input image acquisition means for acquiring an input image and collation using a predetermined template image, so that the value of the geometric transformation parameter of the input image based on the template image Image processing means for determining the first image comprising the template images after the plurality of geometric transformations having different geometric transformation amounts by applying geometric transformation to the template image. A first image group generation unit configured to generate a group, and eigenvalue decomposition on the first image group, whereby the same number of eigenvalues as the template images after the plurality of geometric transformations and the template image are used as a reference. this performing the eigenvalue decomposition unit configured to calculate a unique function representing the relationship between the amount and the eigenfunction value of geometric transformation, the inner product calculation between the plurality of the eigenfunction first image group In a second image group generating unit that generates a second image group, the image similarity is expressed by the continuous curve showing the degree of similarity between the second image group and the input image, the image similarity g, a correlation value is the inner product value _{r n} of the said input image second image group, the eigenfunctions as _{φ n g = Σr n × φ} n, (n = 0,1,2, ·· The image similarity calculation unit calculated by (1) and a collation unit that determines the value of the geometric transformation parameter based on the image similarity calculated by the image similarity calculation unit.

In Expression (9), μ _i is an average density value of T _i (x, y). An image E _n (x, y) obtained using this eigenvector corresponds to an eigenvalue decomposition template image.

As shown in FIG. 4, the arithmetic control unit 13 includes an eigenvalue decomposition template image creation processing unit 18 and a detection processing unit 19. Eigenvalue decomposition template image creation processing unit 18, based on the template image 21 stored in advance, to generate the converted template image group subjected to eigenvalue decomposition to the generated converted template image group, converted calculated eigenfunction An eigenvalue decomposition template image group is created from the subsequent template image group.

The detection processing unit 19 calculates a correlation value between the input image 22 acquired by the camera 12 and the eigenvalue decomposition template image group generated by the eigenvalue decomposition template image creation processing unit 18. The detection processing unit 19 calculates the image similarity from the calculated correlation value, and determines the value of the geometric transformation parameter of the input image 22 based on the template image 21 based on the calculated image similarity.

In the image collation processing procedure performed by the image processing apparatus 10 as described above, steps S10 to S50, which are processes for creating an eigenvalue decomposition template image group based on the template image 21 stored in advance, are performed as eigenvalue decomposition. This is performed by the template image creation processing unit 18. Then, the detection processing unit 19 performs steps S60 to S110 for calculating the image similarity based on the eigenvalue decomposition template image group and the input image and determining the direction angle and the two-dimensional coordinate value.

In the actual processing step, if the positioning of the plurality of IC chips 20 is performed, generation processing of eigenvalue decomposition template images by eigenvalue decomposition template image creation processing unit 18 in advance row upon positioning of a plurality of IC chips 20 Is called. Then, the processing for determining the direction angle and the two-dimensional coordinate value by the detection processing unit 19 is repeatedly performed using the eigenvalue decomposition template image group generated in advance in the step of positioning each IC chip 20. Become.

DESCRIPTION OF SYMBOLS 1 Template image 2 Input image 10 Image processing apparatus 11 Computer 12 Camera (input image acquisition means)
13 Arithmetic control unit (image processing means)
18 eigenvalue decomposition template image creation processing unit 19 detection processing unit 21 template image 22 input image 31 converted template image generation unit (first image group generation unit)
32 eigenvalue decomposition unit 33 eigenfunction selection unit 34 eigenvalue decomposition template image generation unit (second image group generation unit)
35 Image similarity calculation unit 36 Verification unit

Claims (7)

An image processing method for determining a value of a geometric transformation parameter of an input image based on the template image by performing collation using a predetermined template image,
A first image group generation step of generating a first image group composed of the template images after a plurality of geometric transformations by adding geometric transformation to the template image;
An eigenvalue decomposition step of calculating the same number of eigenvalues and eigenfunctions as the plurality of geometrically transformed template images by performing eigenvalue decomposition on the first image group;
A second image group generation step of generating a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group;
An image similarity calculating step for calculating an image similarity that is represented by a continuous curve and indicates a similarity between the input image and the second image group;
A collation step of determining a value of the geometric transformation parameter based on the image similarity calculated in the image similarity calculation step,
An image processing method. A selection step of selecting a predetermined number of the eigenfunctions in descending order of the eigenvalues among the plurality of eigenfunctions calculated in the eigenvalue decomposition step;
The second image group generation step generates the second image group by performing an inner product operation of the plurality of eigenfunctions selected in the selection step and the first image group.
The image processing method according to claim 1. The geometric transformation parameter includes a direction angle and a two-dimensional coordinate value,
The geometric transformation includes a rotation that changes a direction angle and a translation that changes the two-dimensional coordinate value.
The image processing method according to claim 1 or 2. The geometric transformation parameters include parameters necessary for projective transformation,
The geometric transformation includes a projective transformation,
The image processing method according to claim 1. The geometric transformation parameters include parameters necessary for image blurring processing,
The geometric transformation includes image blur processing,
The image processing method according to claim 1. Input image acquisition means for acquiring an input image;
Image processing means for determining a value of a geometric transformation parameter of the input image based on the template image by performing collation using a predetermined template image,
The image processing means includes
A first image group generation unit configured to generate a first image group composed of the template images after a plurality of geometric transformations having different geometric transformation amounts by applying geometric transformation to the template image;
An eigenvalue decomposition unit that calculates eigenvalues and eigenfunctions of the same number as the template images after the plurality of geometric transformations by performing eigenvalue decomposition on the first image group;
A second image group generation unit that generates a second image group by performing an inner product operation of the plurality of eigenfunctions and the first image group;
An image similarity calculation unit that calculates an image similarity represented by a continuous curve and indicating a similarity between the input image and the second image group;
A collation unit that determines a value of the geometric transformation parameter based on the image similarity calculated by the image similarity calculation unit,
Image processing device. The image processing means includes
A selection unit that selects a predetermined number of the eigenfunctions in descending order of the eigenvalues among the plurality of eigenfunctions calculated by the eigenvalue decomposition unit;
The second image group generation unit generates the second image group by performing an inner product operation of the plurality of eigenfunctions selected by the selection unit and the first image group.
The image processing apparatus according to claim 6.

Images