JP2004326728A - Device, method and program of pattern detection, and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a low cost and speedy pattern detection method without any complicated processing such as inverse Fourier transform in an image processing by which a registered image is retrieved from an input image. <P>SOLUTION: The pattern detection method comprises a transforming step, discretizing step, discrete Fourier transforming step, calculating steps, and sensing step. In the transforming step, rectangular coordinates of the registered image and input image are transformed into logarithm/polar coordinate respectively. In the discretizing step, discretizing is performed in a given sampling period. In the discrete Fourier transforming step, the transformation is performed with a discretized registered image and input image. In the calculating steps, calculation of a phase limitation composition is performed on the basis of the registered image and input image which have been discrete Fourier transformed. Calculating a corrected input image is performed after multiplying a complex conjugate to the input image to which the discrete Fourier transforming is performed. Calculating a correlation value is performed with the corrected input image and the registered image. Detecting a pattern of the registered image in the input image is performed on the basis of the calculated correlation value. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique relating to data processing of an image or the like used for pattern search, pattern matching, position detection, and the like, and searches for predetermined registered data registered in advance from input data to be searched. The present invention relates to a pattern detection device, a pattern detection method, a pattern detection program, and a computer-readable recording medium.

  Image processing is used in various fields such as manufacturing and mounting of electronic components in factories. In particular, pattern matching for converting a video captured by a video or the like into data and searching for a desired pattern from the data is used for applications such as positioning. FIG. 1 shows an example of pattern matching. The input image shown in FIG. 1 is configured in a matrix arrangement in which an area is divided into a plurality of pixels G. As shown in this figure, matching is performed by a technique such as a normalized correlation method while moving a registered image RF1 registered in advance in an area SA1 of an input image by a predetermined amount to obtain coordinates at which an image OB to be detected is located.

  Various techniques for detecting such an image pattern have been developed. In general, the more detailed the feature amount is extracted from image data, the more accurate the detection becomes possible, but the required amount of calculation and data becomes enormous, and high-speed, large-capacity equipment that can cope with this is required. Therefore, an image search system that has specifications that can be mounted on an embedded device or the like and that has accuracy and speed that can withstand practical use is desired, and research is progressing.

  For example, in the above-described normalized correlation method, the degree of coincidence between a registered image and an input image is obtained by a normalized correlation that calculates the relationship between two data groups. In the matching by the normalized correlation, the correlation value between the input image and the registered image is calculated while shifting the registered image by one pixel within the search area, and it is considered that the registered image exists at a position where the maximum correlation value is obtained. Find the degree of coincidence.

  Regarding the normalized correlation method, for example, a pattern extraction device using phase-only correlation has been developed (Patent Document 1). An outline of this technique is shown in FIG. First, a registered image to be searched from an input image is subjected to Fourier transform, thereby separating it into an amplitude component and a phase component (frequency component s). Here, the amplitude component of the image is information relating to shading of pixels constituting the image, and one phase component is information relating to the structure of the image. The phase-only correlation method is also called a Fourier phase correlation, and removes an information amount of an amplitude component in a process of calculating a correlation using a Fourier transform. In other words, the similarity of the images is determined by handling the phase components related to the shape, not the shading of the images.

  On the other hand, the input image is similarly Fourier-transformed to extract a phase component. Then, the phase components of the registered image and the input image are combined. Then, a correlation image between the registered image and the input image can be obtained by performing an inverse Fourier transform on this.

  The phase-only correlation method is superior in the characteristic of identifying the same one as compared with a method of performing a normal correlation. FIG. 3 shows the results of autocorrelation obtained by performing cross-correlation with the same registered image and input image for (a) normal correlation and (b) phase-only correlation. In the autocorrelation, the correlation value should originally be the maximum, but as shown in FIG. 3A, the normal correlation has a broad waveform in which the correlation value changes irregularly. On the other hand, in the phase-only correlation, a delta function-like peak is obtained in the center, and it is clearly determined that the images are the same. Further, in this method, the displacement can be accurately detected. FIG. 4 shows a result of calculating a phase-only correlation for an input image obtained by horizontally moving a registered image. As shown in this figure, the position of the delta function-like peak is shifted according to the position shift of the input image. The position of the peak corresponds to the direction of displacement and the amount of displacement. As described above, by measuring the peak position of the phase-only correlation, it is possible to detect a position shift extremely accurately.

  In the normalized correlation method, the cross-correlation between images can be obtained as described above. However, since this method is not invariant to rotation as it is, it cannot be used for detecting an image including rotation. In the above example, if the registered images that are registered are present in the same orientation in the input image to be searched, pattern matching can be performed by sequentially scanning the input images. However, if the registered image exists in the input image in a tilted or rotated state, it cannot be detected by simply scanning as it is by normal pattern matching. In this case, it is necessary to perform a repeat matching search in which the registered image is rotated at a predetermined angle and pattern matching is repeated for each rotation. In this method, there is a problem that the amount of calculation is enlarged in accordance with the pitch of the rotation angle and the necessary arithmetic processing becomes enormous.

  On the other hand, by converting the x and y coordinates of each pixel in the image into polar coordinates, a rotation-invariant normalized correlation can be obtained (Non-Patent Document 1). For example, in the pattern extraction device using the above-described phase-only correlation, first, as shown in FIG. 5, when a registered image to be searched from an input image is first subjected to polar coordinate conversion, the rotational movement amount is converted into a parallel movement amount as shown in FIG. Is done. By performing the Fourier transform based on the polar coordinate system, the amount of parallel movement can be calculated, and by performing the inverse Fourier transform thereof, the amount of rotation can be calculated. For example, for a discrete value Fourier transform X (k), an inverse Fourier transform x (n) can be obtained by the following equation (1).

However, the above-described phase-only correlation method has a problem that an inverse Fourier transform is required in any case. In the pattern detection method, it is essential to reduce the processing load and increase the speed in consideration of practical use. In the phase-only correlation method, after Fourier transform is performed once to synthesize phase components, an inverse Fourier transform is performed to return to a correlation image. However, the execution of the inverse Fourier transform involves the operation of a trigonometric function and the like, so that there has been a problem that the operation is complicated and the cost for execution is increased. The execution of a complicated operation imposes a processing load, and a high-level circuit corresponding to the processing load is required. In addition, a higher processing speed is required in response to an increase in the processing amount, so that power consumption is also increased. As described above, when high specifications are required in terms of hardware such as processing speed, storage area, and power supply capacity, it is difficult to implement the circuit in a practical level circuit. For example, when considering mounting on a small embedded device, the available computing devices and storage capacity are naturally limited due to restrictions on cost, power supply capacity, mounting area, and the like. And there is a problem that mounting is difficult.
Patent No. 3035654 Yoshifumi Sasaki et al. "Performance Evaluation of Phase-Only Image Matching Invariant to Rotation and Scale Change" The 45th Automatic Control Alliance Lecture (November 26-27, 2002)

  The present invention has been made in view of such a problem. An object of the present invention is to provide a pattern detection device, a pattern detection method, a pattern detection program, and a computer-readable recording medium that can search for data such as a registered image in a rotating state and that can be realized at low cost. is there.

  In order to achieve the above object, a pattern detection device according to claim 1 of the present invention is a pattern detection device that performs pattern detection in image processing for searching a registered image registered in advance from an input image, and An input image storage unit 1 for inputting and holding an input image, a registered image storage unit 2 for storing a registered image to be detected, and respective image data from the input image storage unit 1 and the registered image storage unit 2 Is read, a Fourier transform is performed based on the polar coordinates of each pixel constituting the registered image and the input image, a phase component is extracted based on the registered image and the input image subjected to the Fourier transform, and a phase component obtained by combining the phase components is obtained. The composite complex conjugate is added to the Fourier-transformed input image to correct the input image, and a correlation value between the corrected input image and the registered image is calculated. Characterized in that it comprises an image processing unit 3 for performing pattern detection of the registered image from the input image based on the correlation value.

  Thus, even if the registered image is included in the input image in a rotated state, in other words, even if there is a phase shift between the input image and the registered image, pattern detection that eliminates the influence of the phase shift is realized. . In particular, since the calculation can be performed without performing the inverse Fourier transform, the calculation cost can be reduced.

  A pattern detecting apparatus according to a second aspect of the present invention is a pattern detecting apparatus for performing pattern detection in image processing for searching for a registered image registered in advance from an input image, and externally inputs and holds an input image. Image storage unit 1 for storing a registered image to be detected, read image data from the input image storage unit 1 and the registered image storage unit 2, read the registered image A polar coordinate conversion unit that converts the rectangular coordinates of each pixel constituting the image into polar coordinates, a sampling unit that samples the polar coordinates of the registered image and the input image converted by the polar coordinate conversion unit at a predetermined sampling cycle, and A discrete Fourier transform unit for performing a discrete Fourier transform on the registered image and the input image sampled by the sampling unit, Based on the registered image and the input image that have been Fourier transformed by the discrete Fourier transform unit, a phase-only composition unit that computes phase-only composition, and the input image that has been Fourier transformed by the Fourier transform unit is computed by the phase-only composition unit. A complex conjugate integrating section for integrating the complex conjugate of the phase-only synthesis to calculate a corrected input image, and a correlation for calculating a correlation value between the input image corrected by the complex conjugate integrating section and the registered image. An arithmetic unit and a pattern detecting unit that detects a pattern of a registered image from an input image based on the correlation value calculated by the correlation calculating unit.

  Furthermore, the pattern detecting device according to claim 3 of the present invention is the pattern detecting device according to claim 1 or 2, wherein the polar coordinate conversion unit converts the orthogonal coordinates of each pixel into logarithmic / polar coordinates. .

  This realizes pattern detection in which the influence of the phase shift is eliminated not only for image rotation but also for scaling.

  A pattern detection method according to a fourth aspect of the present invention is a pattern detection method for searching registered data registered in advance from input data, and performs a Fourier transform based on polar coordinates of each data constituting the registered data. Performing a Fourier transform based on polar coordinates of each data constituting the input data, calculating a phase-only combination from the registered data and the input data subjected to the Fourier transform, and applying the Fourier transform to the input data subjected to the Fourier transform. Calculating the corrected input data by integrating the complex conjugates of the phase-only synthesis, calculating the correlation value between the corrected input data and the registered data, and calculating the input data based on the calculated correlation value. And detecting registered data from the inside.

  As a result, even if there is a phase shift in the registered data in the input data, pattern detection that eliminates the influence of the phase shift is realized. In particular, since the calculation can be performed without performing the inverse Fourier transform, the calculation cost can be reduced.

  Further, a pattern detection method according to claim 5 of the present invention is a pattern detection method in image processing for searching a pre-registered registered image from an input image, wherein the Fourier transform is performed based on polar coordinates of each pixel constituting the registered image. Performing a transform, performing a Fourier transform based on polar coordinates of each pixel constituting the input image, calculating a phase-only combination from the registered image and the input image subjected to the Fourier transform, and performing the Fourier transform. Calculating the corrected input image by integrating the complex conjugate of the phase-only synthesis to the input image; calculating the correlation value between the corrected input image and the registered image; and And performing a pattern detection of the registered image from the input image.

  Thus, even if the registered image is included in the input image in a rotated state, in other words, even if there is a phase shift between the input image and the registered image, pattern detection that eliminates the influence of the phase shift is realized. . In particular, since the calculation can be performed without performing the inverse Fourier transform, the calculation cost can be reduced.

  Furthermore, a pattern detection method according to claim 6 of the present invention is a pattern detection method in image processing for searching a registered image registered in advance from an input image, wherein the orthogonality of each pixel constituting the registered image and the input image is determined. Converting the coordinates into polar coordinates, respectively, discretizing the registered image and the input image displayed in polar coordinates at a predetermined sampling cycle, and performing discrete Fourier transform on the discretized registered image and the input image, respectively. Calculating a phase-only combination based on the registered image and the input image subjected to the discrete Fourier transform, and calculating a corrected input image by integrating the complex conjugate of the phase-only combination with the input image subjected to the discrete Fourier transform Calculating a correlation value between the corrected input image and the registered image; and calculating the calculated phase value. Based on the values, characterized in that it comprises a step of performing pattern detection of the registered image from the input image.

  Furthermore, the pattern detection method according to claim 7 of the present invention is the pattern detection method according to claim 6, wherein sampling is performed when each of the registered image and the input image displayed in polar coordinates is discretized at a predetermined sampling cycle. The method is characterized in that smoothing is performed on each pixel of the registered image and the input image based on pixels around the pixel around the pixel.

  Thereby, the quantization error is smoothed, and the polar coordinate conversion in which the target pixel is weighted is realized.

  Furthermore, the pattern detection method according to claim 8 of the present invention is the pattern detection method according to claim 6 or 7, wherein when the registered image and the input image displayed in polar coordinates are each discretized at a predetermined sampling cycle, For each pixel of the sampled registered image and input image, a point on orthogonal coordinates that satisfies a predetermined condition is aggregated by adding the amplitude component of each pixel to a corresponding point on polar coordinates. I do.

  Thereby, the polar coordinate data discretized by sampling can be collected.

  Furthermore, a pattern detection method according to a ninth aspect of the present invention is the pattern detection method according to any one of the fifth to eighth aspects, wherein an amplitude component is calculated when calculating a correlation value between the corrected input image and the registered image. Are integrated for pixels having a predetermined threshold value or more.

  As a result, data having a small amplitude component in the registered image can be excluded from the calculation, and the amount of calculation can be reduced to contribute to a reduction in processing load and an increase in speed.

  A pattern detection method according to a tenth aspect of the present invention is the pattern detection method according to any one of the fifth to ninth aspects, wherein each pixel is calculated when a correlation value between the corrected input image and the registered image is calculated. The emphasis processing based on the amplitude component is performed by taking a weighted average based on the displacement amount of the amplitude component.

  As a result, data having a small amplitude component in the registered image can be excluded from the calculation, the amount of calculation can be reduced, the effect of errors can be suppressed and the operation can be stabilized, and the calculation can be performed based on the data having a large amplitude component. The performance is also improved.

  Furthermore, the pattern detection method according to claim 11 of the present invention is the pattern detection method according to any one of claims 4 to 10, wherein the coordinates of each data constituting the registration data and the input data are respectively converted to logarithmic / polar coordinates. It is characterized by doing.

  This realizes pattern detection in which the influence of the phase shift is eliminated not only for image rotation but also for scaling.

  A pattern detection program according to a twelfth aspect of the present invention is a pattern detection program for retrieving registered data registered in advance from input data, wherein the computer performs a Fourier search based on polar coordinates of each data constituting the registered data. A function of performing a transform, a function of performing a Fourier transform based on polar coordinates of each data constituting the input data, a function of calculating a phase-only combination from the registered data and the input data subjected to the Fourier transform, and a function of performing the Fourier transform. A function of calculating the corrected input data by integrating the complex conjugate of the phase-only synthesis to the input data, a function of calculating a correlation value between the corrected input data and the registered data, and a function of calculating the correlation value based on the calculated correlation value. , A pattern detection program for realizing a function of detecting registered data from input data.

  Thus, even if the registered image is included in the input image in a rotated state, in other words, even if there is a phase shift between the input image and the registered image, pattern detection that eliminates the influence of the phase shift is realized. . In particular, since the calculation can be performed without performing the inverse Fourier transform, the calculation cost can be reduced.

  Further, a pattern detection program according to a thirteenth aspect of the present invention is a pattern detection program in an image processing for searching a registered image registered in advance from an input image, the computer being provided with a polar coordinate of each pixel constituting the registered image. A function for performing a Fourier transform based on the polar coordinates of each pixel constituting the input image, a function for performing a phase-only combination from the registered image and the input image subjected to the Fourier transform, A function to calculate the corrected input image by integrating the complex conjugate of the phase-only synthesis into the converted input image, a function to calculate the correlation value between the corrected input image and the registered image, and a function to calculate the calculated correlation. This is a pattern detection program for realizing a function of detecting a pattern of a registered image from an input image based on a value.

  Still further, a pattern detection program according to a fourteenth aspect of the present invention is a pattern detection program in image processing for searching a pre-registered registered image from an input image, and stores, in a computer, each of the registered image and the input image. A function of converting the rectangular coordinates of pixels into polar coordinates, a function of discretizing a registered image and an input image displayed in polar coordinates at a predetermined sampling period, and a discrete Fourier transform of the discretized registered image and input image, respectively Based on the registered image and the input image subjected to the discrete Fourier transform, and a function to calculate the phase-only combination.The corrected input is obtained by integrating the complex conjugate of the phase-only combination with the discrete Fourier-transformed input image. A function for calculating the image, a function for calculating the correlation value between the corrected input image and the registered image, and a function for calculating the correlation value. Based on the correlation value, a pattern detecting program for realizing a function of performing pattern detection of the registered image from the input image.

  Still further, a pattern detection program according to a fifteenth aspect of the present invention is the pattern detection program according to the fourteenth aspect, wherein each of the registered image and the input image displayed in polar coordinates is sampled when discretized at a predetermined sampling cycle. The method is characterized in that smoothing is performed on each pixel of the registered image and the input image based on pixels around the pixel around the pixel.

  Thereby, the quantization error is smoothed, and the polar coordinate conversion in which the target pixel is weighted is realized.

  Furthermore, a pattern detection program according to claim 16 of the present invention is the pattern detection program according to claim 14 or 15, wherein when the registered image and the input image displayed in polar coordinates are each discretized at a predetermined sampling cycle, For each pixel of the sampled registered image and input image, a point on orthogonal coordinates that satisfies a predetermined condition is aggregated by adding the amplitude component of each pixel to a corresponding point on polar coordinates. I do.

  Thereby, the polar coordinate data discretized by sampling can be collected.

  Furthermore, a pattern detection program according to a seventeenth aspect of the present invention is the pattern detection program according to any one of the fourteenth to sixteenth aspects, wherein an amplitude component is calculated when a correlation value between the corrected input image and the registered image is calculated. Are integrated for pixels having a predetermined threshold value or more.

  As a result, data having a small amplitude component in the registered image can be excluded from the calculation, and the amount of calculation can be reduced to contribute to a reduction in processing load and an increase in speed.

  Furthermore, a pattern detection program according to claim 18 of the present invention is the pattern detection program according to any one of claims 14 to 17, wherein each pixel is calculated when calculating a correlation value between a corrected input image and a registered image. The emphasis processing based on the amplitude component is performed by taking a weighted average based on the displacement amount of the amplitude component.

  As a result, data having a small amplitude component in the registered image can be excluded from the calculation, the amount of calculation can be reduced, the effect of errors can be suppressed and the operation can be stabilized, and the calculation can be performed based on the data having a large amplitude component. The performance is also improved.

  Furthermore, a pattern detection program according to a nineteenth aspect of the present invention is the pattern detection program according to any one of the twelfth to eighteenth aspects, wherein the coordinates of each data constituting the registration data and the input data are respectively converted to logarithmic / polar coordinates. It is characterized by doing.

  This realizes pattern detection in which the influence of the phase shift is eliminated not only for image rotation but also for scaling.

  According to a twentieth aspect of the present invention, a computer-readable recording medium records the pattern detection program according to any one of the twelfth to nineteenth aspects. Recording media include magnetic disks such as CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, and DVD + RW, and optical disks. , A magneto-optical disk, a semiconductor memory, and other media capable of storing programs. The above-mentioned program also includes a form that can be downloaded via a network.

  According to the pattern detection device, the pattern detection method, the pattern detection program, and the computer-readable recording medium of the present invention, a registered image of an arbitrary shape can be searched at high speed from an input image regardless of rotation or enlargement / reduction. In particular, in the pattern detection device, the pattern detection method, the pattern detection program and the computer-readable recording medium of the present invention, the rotation of the registered image can be performed by multiplying the phase-only correlation by the complex conjugate without using the inverse Fourier transform. Robust (invariant) pattern detection for scaling has been obtained. As a result, the calculation of the inter-image correlation value can be mainly performed by the product-sum operation, and a complicated operation such as a trigonometric function required for performing the inverse Fourier transform can be omitted. As a result, the computational load is reduced and the processing is reduced in weight, so that the circuit configuration for the computation can be simplified to reduce power consumption and improve the processing speed. Since the amount of calculation itself is greatly reduced in combination with the fact that conversion is not required, an inexpensive pattern detection that contributes to higher speed and can be implemented on a relatively small-scale device is realized. Further, by adding a process based on the amplitude component, more accurate pattern detection can be obtained. In particular, by ignoring data having a small amplitude and emphasizing data having a large amplitude, there is an advantage that a difference is emphasized and a reliable pattern detection is easily obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below exemplifies a pattern detection device, a pattern detection method, a pattern detection program, and a computer-readable recording medium for embodying the technical idea of the present invention. The invention does not specify a pattern detection device, a pattern detection method, a pattern detection program, and a computer-readable recording medium as follows.

  Further, in this specification, in order to make it easy to understand the claims, the numbers corresponding to the members described in the embodiments will be referred to as “claims” and “means for solving the problems”. Column). However, the members described in the claims are not limited to the members of the embodiments. In addition, the size, positional relationship, and the like of the members illustrated in each drawing may be exaggerated for clarity of description. Further, in the following description, the same names and reference numerals denote the same or similar members, and a detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are configured by the same member and one member also serves as the plurality of elements, or conversely, the function of one member may be determined by a plurality of members. It can also be realized by sharing.

  In this specification, a pattern detection device refers to hardware capable of mainly performing pattern detection using a phase-only correlation method. This hardware has a configuration as shown in FIG. 7, for example, and is composed of general-purpose or dedicated parts, circuits, and devices. However, besides this, a pattern detection device that incorporates software, programs, plug-ins, objects, libraries, applets, compilers, etc. into a general-purpose circuit or computer and enables pattern detection using the phase-only correlation method is also included in the pattern detection device. Including. That is, the embodiment of the present invention can be configured as hardware or can be realized as software. When implemented as software, the pattern detection method using the phase-only correlation method can be executed as a program on general-purpose or customized dedicated computers, workstations, terminals, portable electronic devices, PDAs, pagers, smartphones, and other electronic devices. Is done.

[Pattern detector]
A pattern detection device according to an embodiment of the present invention will be described with reference to FIG. The pattern detection device shown in FIG. 7 includes an input image storage unit 1 for externally inputting image data, a registered image storage unit 2 for storing a registered image to be subjected to pattern detection, an input image storage unit 1 and a registered image storage unit 2, an image processing unit 3 for reading each image data and performing various processes required for pattern detection, and a control unit for controlling the image processing unit 3 to perform pattern detection and performing necessary settings 4, a display unit 5 for displaying a pattern detection result, and an input unit 6 for a user to operate and set the pattern detection device. Note that the relationship and connection form between the members are not limited to the example shown in FIG. 7. For example, the image processing unit 3 and the control unit 4 may be replaced, or the input unit 6 and the display unit 5 may be shared. It is.

  The input image storage unit 1 stores an image input from an image source 7 connected thereto as data. For example, a real-time image captured by an imaging camera or a previously recorded reproduced image is used as an input image, which is converted from an analog signal to a digital signal by an A / D converter, for example, to obtain binary data or multi-tone data. Image data such as gray scale data. As the input image storage unit 1, a memory that stores input image data, for example, a RAM that is a volatile memory is used.

  The registered image storage unit 2 stores data of an image to be searched by pattern detection. Prior to execution of pattern detection, a desired registered image serving as a search reference is registered in advance. For example, similarly to the input image storage unit 1, the image data including the registered image is input by connecting to an external image source 7 (for example, by switching to the input image storage unit 1). The user specifies an area to be a registered image from the captured image. The designated area is stored in the registered image storage unit 2 as a registered image. As the registered image storage unit, a nonvolatile ROM, a volatile RAM, or the like can be used.

  The image processing unit 3 serves as a work area for reading input image data and registered image data from the input image storage unit 1 and the registered image storage unit 2 and performing pattern detection. The control unit 4 connected to the image processing unit 3 specifies an address for calling image data from the input image storage unit 1 and the registered image storage unit 2. The control unit 4 includes a CPU, a system LSI, an ASIC, and the like. The image processing unit 3 reads necessary image data according to an address designated by the control unit 4 and performs pattern detection. In order to perform pattern detection, the image processing unit 3 and the control unit 4 convert a rectangular coordinate of each pixel constituting the registered image and the input image into polar coordinates, and a registration converted by the polar coordinate converting unit. A sampling unit that samples the polar coordinates of the image and the input image at a predetermined sampling cycle, a discrete Fourier transform unit that performs a discrete Fourier transform on the registered image and the input image sampled by the sampling unit, and a Fourier transform that uses the discrete Fourier transform unit Based on the registered image and the input image obtained, a phase-only synthesis unit that calculates phase-only synthesis, and integrates the complex conjugate of the phase-only synthesis calculated by the phase-only synthesis unit with the input image Fourier-transformed by the Fourier transform unit And a complex conjugate accumulator for calculating the corrected input image. The input image and the correlation calculation unit for calculating a correlation value between the registration image, based on the correlation values calculated by the correlation calculating unit, functions as pattern detecting unit for performing pattern detection of the registered image from the input image.

  The result of the pattern detection performed by the image processing unit 3 and the control unit 4 is displayed on the display unit 5. Alternatively, it is output to an externally connected device for subsequent processing. On the other hand, the user operates the pattern detection device from the input unit 6. As the input unit 6, various pointing devices such as a mouse, a keyboard, a slide pad, a tablet, a digitizer, a light pen, a numeric keypad, a jog dial, a touch pad, an accu point, and a touch panel can be used. For example, when the pattern detection device is implemented by a computer, the input unit 6 is an input / output device such as a mouse or a keyboard connected to the computer, the display unit 5 is a display, the image processing unit 3 and the control unit 4 are a CPU, and an input image storage. The unit 1 and the registered image storage unit 2 correspond to a memory.

[Phase-only correlation method]
Next, a pattern detection method using the phase-only correlation method according to Embodiment 1 of the present invention will be described with reference to the flowcharts of FIGS. The pattern matching or pattern search is a method of searching for an image in which a registered image to be searched is registered in advance, and an input image as a search target area is scanned to search for a registered pattern. In the present embodiment, pattern detection based on rotation-invariant image correlation is performed by applying phase-only correlation. Specifically, the target registered image is converted to a polar coordinate system, and after performing a discrete Fourier transform in the circumferential direction, the phase difference between the images due to the rotational displacement is corrected using the phase-only correlation, thereby rotating the image. An invariant correlation value between images is obtained. Hereinafter, the basic principle will be described with reference to FIG.

[Polar coordinate conversion step]
First, an input image to be subjected to pattern detection and a registered image registered in advance are respectively expressed in a polar coordinate system. In step S1, an input image obtained from an external image source or the like is obtained from the input image storage unit by the image processing unit, and I ₁ (x, y) on the orthogonal coordinate system (x, y), and the registered image storage unit Is given as I ₂ (x, y). In step S2, these orthogonal coordinates are converted into polar coordinates (ρ, θ) based on the following equation 2, and the input image on the polar coordinates is I ₁ (ρ, θ), and the registered image is I ₂ (ρ, θ). Expressed as Here, ρ is an absolute value, and θ is an argument. If the input image and the registered image are directly given in the polar coordinate system without going through the rectangular coordinates, step S1 can be omitted.

[Discrete Fourier Transform]
Next, in step S3, discretization of polar coordinates is performed. Specifically, for each of the input image I ₁ (ρ, θ) and the registered image I ₂ (ρ, θ), the number M of sampling intervals _Tr in the radial direction (distance or displacement) and the circumferential direction (angle or angle) Phase) at 2π / N, and discrete values I ′ ₁ (r, t), I ′ ₂ (r, t) (r = 0, 1,..., M−1; t = 0, 1, ..., N-1). Further, in step S4, a discrete Fourier transform is performed on the discrete value along the phase t. As a result, F ₁ (r, s) and F ₂ (r, s) expressed by the following equation (3) are obtained.

Here, the input image I ₁ is when the registered image I ₂ were those theta ₁ rotates around a point of polar coordinates r = 0, the number 4 relationship is established.

In this case, if the aliasing due to the high frequency component is neglected, the relationship of Formula 5 is established between F ₁ (r, s) and F ₂ (r, s) of Formula 3.

[Phase-only synthesis]
In step S5, based on the discrete Fourier transform _{F 1} and _{F 2} of the input image _{I 1} and the registered image _{I 2,} is calculated on the basis of these phase-only synthesis R (s) is the number 6. As shown in Equation 6, the magnitude is set to 1 by dividing the product of F ₁ (r, s) by the complex conjugate of F ₂ (r, s) by the amplitude component, in other words, the amplitude component. Here, the luminance information of the image is ignored by the normalization, and only the phase component for each angle is extracted.

[Correction of input image]
From 6, it is possible to extract the phase shift due to rotation angle theta ₁ at each frequency s. In step S6, using the above R (s) to correct the input image F ₁ as in equation 7, returned to the input image F _'1 in the form in which the influence of the phase difference due to the rotation _(r, s) be able to. Equation 7 is obtained by multiplying the Fourier-transformed F ₁ (r, s) by the complex conjugate of R (s) obtained in Equation 6. Accordingly, in this state, if the correlation value between the input image and the registered image is obtained in step S7, it is possible to obtain the correlation value in a state in which the influence of the rotation is eliminated. Then, in step S8, the control unit detects an image based on the correlation value, and performs output to the display unit as necessary.

[Application example]
With the above method, it is possible to obtain a correlation value in which a phase shift due to rotation has been canceled. However, in practice, in addition to noise and quantization error at the time of image acquisition, an error occurs at the time of conversion to a polar coordinate system because image data is usually obtained discretely on orthogonal coordinates. Accordingly, an algorithm for actually calculating the correlation value between images in consideration of the above basic principle will be described below with reference to FIG.

[Consolidation on polar coordinates]
In steps S′1 and S′2, the input image and the registered image are acquired and the conversion from the rectangular coordinates to the polar coordinates is performed as in FIG. Then, in step S′3-1, the data obtained in the polar coordinates is discretized and integrated. Here, a range having the same size as the registered image was cut out from the input image, and conversion from rectangular coordinates to polar coordinates and discretization were performed using Expression 2 and a sampling interval _Tr , 2π / N. At this time, in order to reduce the above-described various errors, a point on the orthogonal coordinates (x, y) satisfying the relationship of the following equation 8 is set to a corresponding point on the polar coordinates (r, t), and the luminance value of each pixel is set. Aggregated by adding. As a result, the discretized image data is aggregated into discretized steps on the polar coordinates.

[Smoothing on polar coordinates]
Further, in step S′3-2, smoothing using the vicinity 4 of an arbitrary pixel on the polar coordinates is performed as in the following equation 9. In this example, a smoothing filter as shown in FIG. 10 is used. That is, the pixel adjacent to the target pixel in the upper, lower, left and right directions is smoothed by 1/4 times.

[Averaging of discrete Fourier transform and phase-only synthesis]
In step S'4, a discrete Fourier transform is performed on the image data converted into the polar coordinates in the circumferential direction according to Equation 3. Further, in step S'5, a phase-only combination is obtained based on Expression 6. However, when the same image is originally rotated by a fixed angle, the value of Expression 6 should be invariable in the radial direction, but actually varies due to various errors. Therefore, in step S′5, an average in the radial direction is obtained, and in the following calculation, the average value R _ave (s) is set to the value of Expression 6.

[Emphasis processing based on amplitude component]
In step S′6, the input image is corrected using Expression 7, and F ′ ₁ (r, s) is obtained. In step S'7, the input image F _'1 (r, s) and the corrected reference image _F 2 (r, s) the cross-correlation c of the (r), is calculated from the following equation number 10. Based on this correlation value, rotation-invariant image detection is performed in step S'8.

  However, in the above equation, Re (*) is the real part of *. Gs and G (r) are as shown in the following equation (11).

  As described above, by taking the weighted average by G (r), which is the pixel density at each radius, the correlation in a portion having a high pixel density in the registered image is emphasized. In other words, data having a small amplitude component (image luminance information) is ignored. In Equation 11, for data in which G (r) is 0, c (r) in Equation 10 is not considered, and is removed from the calculation target. The data of 0 is a black pixel in a binary image and is data without image information. By removing such pixels from the search target, high-speed and high-accuracy pattern detection can be realized. In general, accurate data cannot be detected from data having a low or zero pixel value among the pixels constituting an image, but rather the calculation result becomes unstable and errors increase. Therefore, such data is eliminated to stabilize the calculation, and the required amount of calculation is reduced, thereby contributing to speeding up. In addition, by emphasizing data having a large amplitude component, a correlation based on a bright portion having a high luminance in an image is obtained. By performing the emphasis processing based on the amplitude component in this manner, only data having an amplitude (spectrum) is to be calculated, and pattern detection with high speed and robustness to rotation is reliably realized.

  In the above-described pattern detection method, the pattern detection is executed in a form in which the rotation component is canceled without considering the rotation angle of the registered image in detail. For this reason, it is not necessary to scan the image in consideration of the rotation, and since only the detection of the translation component is handled, the number of calculations is significantly reduced, and a high-speed operation with a low load is realized. In addition, since it is not necessary to perform inverse Fourier transform and it is not necessary to perform a step of extracting a correlation peak for angle detection, it is possible to realize a pattern detection that can be easily performed at a high speed in this regard.

  In this method, rotation-independent pattern detection can be performed, but rotation angle itself is not detected. It goes without saying that in an application that requires detection of the rotation angle, an algorithm for separately calculating the rotation angle can be used together or added. This pattern detection method can be used for high-speed and inexpensive pattern search in an image processing apparatus. For example, the present invention can be applied to image extraction and fingerprint collation in a product inspection process. Instead of detailed position detection and rotation angle detection, pre-processing can be used for rough position detection, and then, if necessary, detailed pattern detection including angle detection near the detection position can be performed. .

[Simulation experiment results]
Next, the results of a simulation experiment of pattern detection for various images to confirm the effectiveness when the above-described method is used will be described.

[Correlation of simple figures]
First, for each of the test images 1 and 2 including the simple figures shown in FIGS. 11 and 12, the correlation is evaluated by the pattern detection method according to the above-described embodiment, and this method has uniqueness with respect to rotation. And the difference in correlation values with similar figures were verified. Each image used in the present embodiment is a binary image of 128 × 128 pixels, and the number of grid points in polar coordinates is 32 in both the radial direction and the circumferential direction. In this example, the test image 1 is a registered image, the test image 1 and the test image 2 are input images 1 and 2, respectively, and the input image is rotated from 0 to π / 2 in steps of π / 18. The value was determined.

  The correlation values obtained for each rotation angle are shown in the graph of FIG. In the figure, the upper polygonal line indicates the autocorrelation of the test image 1, and the lower polygonal line indicates the cross-correlation between the test image 1 and the test image 2. As is apparent from this figure, even when the image is rotated, the correlation value of the autocorrelation of the registered image 1 itself always exceeds the correlation value with the input image 2, and the minimum value of the autocorrelation and the input image A sufficient difference is also observed between the maximum value of the correlation value and the correlation value of 2. Therefore, when pattern detection is performed by the above method, it can be confirmed that the test image 1 and the test image 2 can be sufficiently distinguished. Since the threshold value of the correlation value for actually performing pattern detection is determined by the input image and the registered image, it is statistically processed from the obtained correlation value data.

  In FIG. 13, ideally, it is desired that the upper correlation value remains constant irrespective of the rotation angle, but the autocorrelation value becomes 1 except for the positions where the rotation angle is 0 and π / 2. Not. That is, it is not completely invariant to rotation. This is because the error at the time of the polar coordinate conversion described above and the quantization error of the pixel position when the image is sequentially rotated to generate the input image occur, and the original image and the rotated image do not exactly match. It is thought that it is. Particularly, in this simulation experiment, since the binary image is used, the influence is large. However, when a multi-valued image of 256 gradations or the like is used in an actual application, such an effect is considered to be small because anti-aliasing is applied by an optical system at the time of image acquisition.

[Advanced correlation]
Next, in order to confirm that the pattern detection method according to the above-described embodiment is effective even in more advanced image detection, a correlation value is calculated using an image as shown in FIGS. 15 was compared with the autocorrelation when the test image 3 was used as the registered image, and the correlation value when the test image 4 shown in FIG. 15 was used as the input image. The results are shown in the graph of FIG. Each of the test images in FIGS. 14 and 15 has the same distribution of the number of pixels at a position at a fixed radius from the center of the image. For this reason, if the input registered image 3 and the input image 4 are evaluated with normal correlation values, they are determined to be the same. Therefore, when the pixel distribution at the same radius from the center is viewed in the frequency domain, it cannot be determined unless the phase component is considered, and advanced pattern detection is required. Even in such advanced image detection, a clear difference between the autocorrelation and the autocorrelation is observed at all angles, as shown in FIG. A clear difference was observed between the minimum value of the correlation and the effectiveness of the correlation value method considering the phase component was confirmed.

[Evaluation of robustness against noise]
Next, in order to evaluate the resistance to noise contained in the image, 500 pixels are randomly selected from the image shown in FIG. 11 and the pixel value of the selected pixel is inverted to add noise. Was done. The image after noise addition is shown as a test image 5 in FIG. In the case of actual product inspection or the like, it is unlikely that such spike noise is directly included in the original image obtained from the CCD camera. However, when an edge is detected using a Sobel filter or the like as preprocessing after acquiring an image as a multi-valued image, unevenness in the density of the image in the original image may become such noise. Similarly, the same processing was performed on the input image 2, and an evaluation test of the same correlation value was performed as the input image 6 (not shown). The results are shown in the graph of FIG.

  From this simulation result, as in the above, the difference is observed in the autocorrelation of the test image 5 and the correlation value between the test image 6 and it can be confirmed that the two can be distinguished. Note that, when looking at FIG. 18, the correlation value of the registered image 5 itself decreases due to the rotation as compared with the result of FIG. It is considered that this is because the high frequency component of the image increased due to random noise, and the rate at which Equation 5 did not hold increased. However, in this case, the correlation value between the test image 5 and the test image 6 is also reduced similarly to the autocorrelation value, and the identification is sufficiently possible by this method. However, in practice, it is considered rare to handle an image to which more noise is added, and a pattern search with practically sufficient accuracy can be realized by this embodiment. In each of the examples described above, pattern detection is performed on an image that is difficult to discriminate for evaluation purposes. However, in actual pattern detection, a clearer difference is detected, and pattern detection with sufficient accuracy is performed. Is realized.

[Extraction from input image containing multiple objects]
Next, as a simulation in a more practical case, evaluation was performed on a case where a target character was extracted from the test image 7 including a plurality of characters as shown in FIG. Here, “F” of the test image 8 shown in FIG. 20 is searched as a registered image from an input image in which a plurality of alphabets are arranged obliquely and horizontally. The test image 7 in FIG. 19 includes three “F” s having various inclinations. 19, the size of the image is 512 × 512 pixels, and FIG. 20 is 128 × 128 pixels in the same manner as described above. Since this evaluation test aims at evaluating the rotation invariance of the pattern detection method according to the embodiment, the pattern scanning in the vertical and horizontal directions is simply performed sequentially for each pixel. In order to speed up the scanning, it can be combined with other known or future developed techniques in actual applications. The calculation result is shown in the graph of FIG.

  In FIG. 21, the x and y axes are the numbers of pixels shifted vertically and horizontally in x and y orthogonal coordinates with the origin at the upper left in the test image 7 shown in FIG. The correlation value at the position is plotted. Here, only those having a correlation value of 0.85 or more are displayed for easy understanding. In the graph of FIG. 21, three peaks are recognized, and these peaks respectively correspond to the position of “F” included in the test image 7 of FIG. 19, that is, the upper, middle, and lower positions. As described above, according to the embodiment of the present invention, it has been confirmed that the registered image is properly detected from the input image regardless of the tilt or the rotation. In addition, the numerical value indicated by the arrow at each peak in FIG. 21 indicates the x coordinate, the y coordinate, and the correlation value at the detection position in order from the left.

[processing speed]
Next, the processing speed of the pattern detection method according to the embodiment of the present invention will be examined. Regardless of the method of the present invention, when detecting an image including a displacement due to rotation using a general normalized correlation method, the registered images are sequentially rotated by an appropriate step size as described above, and after rotation, Cross-correlation between the registered image and the input image is performed. In this case, the processing time increases as the step size of the rotation is reduced. However, since the processing time is used in combination with the binary search or the like, the processing time does not necessarily increase in inverse proportion to the step size. However, processing time required for image rotation conversion is always required. Here, in order to confirm that the processing speed is improved by the pattern detection method according to the present embodiment as compared with the conventional normalized correlation method, one cross-correlation is obtained for the same set of images. The processing time required for this is compared between the pattern detection method according to the present embodiment and a repeat matching search method that calculates a normalized correlation for image rotation conversion. The image used for comparison was a combination of test image 1 and test image 2 described above. Each process is performed 1000 times, and the average value is obtained and compared. In the pattern detection method according to the embodiment of the present invention, since the image size is reduced to 1/16 at the time of polar coordinate conversion, the normalized correlation is similarly reduced to 1/4 in both the vertical and horizontal directions.

  The computer used for measuring the processing time used Pentium III (registered trademark) having an operating frequency of 1.2 GHz as the CPU, FreeBSD R4.7 as the OS, and gcc 2.95.4 as the Compiler.

  As a result, the average of the processing time for 1000 times was 1.9 ms in the pattern detection method according to the present embodiment, and 11.1 ms in the method combining the normal normalized correlation and the rotation conversion. From this, it was confirmed that the pattern detection method according to the present embodiment is at least five times faster than normal normalized correlation in one process. It is considered that this greatly contributes to the fact that the calculation time for performing the rotation conversion is unnecessary in the method of the present embodiment. Therefore, the amount of computation to be processed is reduced, and the processing speed is increased.

  Note that also in the pattern detection method according to the present embodiment, coordinate conversion from the rectangular coordinates to the polar coordinate system is necessary. However, once the registered image and the size after conversion to the polar coordinate system are determined, the correspondence between pixels is calculated in advance, and at each search, a conversion table etc. is prepared to uniquely convert to the corresponding pixel. Therefore, there is no need to perform a matrix operation. Also, even when a binary search or the like is used in combination, if a correlation value that is not rotation-invariant is used in the conventional method, a plurality of processes are required to correspond to an arbitrary rotation angle. The processing time difference becomes even greater.

  In the above example, an example in which the target image is fixed and the input image is rotated has been described. However, it is needless to say that the same result can be obtained by rotating the target image.

  As described above, it has been confirmed by various simulations that the phase-only correlation is used to define a rotation-invariant inter-image correlation value, thereby enabling high-speed detection independent of the rotation of the input image. Was. Moreover, the pattern detection method according to this embodiment does not require inverse Fourier transform, and mainly uses complex conjugates that can be executed by a product-sum operation. As a result, the processing load is greatly reduced, and a system that can be implemented in a small-scale embedded system can be realized by reducing the processing weight and increasing the processing speed. In particular, high-speed execution is possible by using a processor having a multiply-accumulate operation instruction which has been available at a low cost in recent years.

[Scale]
In addition, by performing log transformation in the radial direction together with polar coordinate transformation, it is possible to cope with not only rotation of the registered image but also enlargement / reduction. For enlargement / reduction by log conversion, a known method such as Non-Patent Document 1 described above or a method developed in the future can be used as appropriate. As a result, high-speed and robust pattern detection is realized even with respect to rotation, enlargement, and reduction of the registered image. Hereinafter, as Embodiment 2 of the present invention, an example of pattern detection that is invariant to scaling and rotation will be described.

  The flow charts of FIGS. 22 and 23 show a flow of detection of a pattern that is invariable in scale and rotation using the phase-only correlation method. In this method, the target registered image is transformed into a logarithmic / polar coordinate system, and a discrete Fourier transform is performed in the radial / circumferential direction. By performing the correction, a correlation value between images that is invariable in scale and rotation is obtained. Hereinafter, the basic principle will be described with reference to FIG.

[Log / Polar coordinate conversion step]
First, an input image to be subjected to pattern detection and a registered image registered in advance are respectively expressed in a logarithmic / polar coordinate system. In step S ″ 1, an input image acquired from an external image source or the like is acquired by the image processing unit from the input image storage unit, and I ₁ (x, y) and a registered image are acquired on a rectangular coordinate system (x, y). The registered image obtained from the storage unit is given as I ₂ (x, y) .In step S ″ 2, these orthogonal coordinates are converted into logarithmic / polar coordinates (ρ, θ) based on the following equation 12, The input image on the logarithmic / polar coordinates is expressed as I ₁ (ρ, θ), and the registered image is expressed as I ₂ (ρ, θ). Here, ρ is an absolute value, and θ is an argument. Unlike Equation 2, ρ = 0 is a singular point. If the input image and the registered image are directly given in a logarithmic / polar coordinate system without going through rectangular coordinates, step S ″ 1 can be omitted.

[Two-dimensional discrete Fourier transform]
Next, in step S ″ 3, logarithmic / polar coordinate discretization is performed in the same manner as in FIG. 8. M samples at a sampling interval _Tr in the radial direction (distance or displacement) and 2π / N in the circumferential direction (angle or phase). , And N discrete values I ′ ₁ (r, t), I ′ ₂ (r, t) (r = 0, 1,..., M−1; t = 0, 1,. , N-1) In step S ″ 4, a two-dimensional discrete Fourier transform is performed on the obtained discrete values along r and t. As a result, F ₁ (u, s) and F ₂ (u, s) expressed by the following equation 13 are obtained, and terms of a radial component and a circumferential component appear.

Here, if the input image I ₁ is obtained by scaling the registered image I ₂ by ρ ₁ around the point of logarithmic / polar coordinates ρ = 0 and rotating θ ₁ , the relationship of Expression 14 is established.

In this case, if the aliasing due to the high frequency component is neglected, the relationship of Formula 15 is established between F ₁ (u, s) and F ₂ (u, s) of Formula 13.

[Phase-only synthesis]
Next, in step S ″ 5, based on the discrete Fourier transforms F ₁ and F ₂ of the input image I ₁ and the registered image I ₂ , the phase-only combination R (u, s) is calculated based on Expression 16. As shown in FIG. 16, by dividing the product of F ₁ (u, s) by the complex conjugate of F ₂ (u, s) by the amplitude component, the magnitude is set to 1, in other words, the amplitude component, Here, the luminance information of the image is ignored by the normalization, and only the phase component for each angle is extracted.

[Correction of input image]
From Equation 16, it is possible to extract a phase shift due to the scaling ratio ρ ₁ and the rotation angle θ _{1 at} each of the frequencies u and s. In step S "6, the above R (u, s) using the corrected as the input image F ₁ the number 17, the input image F of the form in which the influence of the phase difference due to scaling and rotation _'1 Equation (17) is obtained by multiplying the Fourier-transformed F ₁ (u, s) by the complex conjugate of R (u, s) obtained in Equation (16). In this state, if the correlation value between the input image and the registered image is obtained in step S ″ 7, it is possible to obtain the correlation value in a state where the influence of the enlargement / reduction and rotation is eliminated. Then, in step S ″ 8, the control unit detects an image based on the correlation value, and performs output to the display unit as necessary.

[Application example]
By the above method, it is possible to obtain a correlation value in which a phase shift due to scaling / rotation has been canceled. However, in practice, in addition to noise and quantization error at the time of image acquisition, an error occurs at the time of conversion to a logarithmic / polar coordinate system because image data is usually obtained discretely on orthogonal coordinates. Therefore, an algorithm for actually calculating a correlation value between images in consideration of the above basic principle will be described below with reference to FIG.

[Consolidation on logarithmic and polar coordinates]
In steps S ″ ′ 1 and S ″ ′ 2, as in FIG. 22, the input image and the registered image are obtained, and the transformation from the orthogonal coordinates to the logarithmic / polar coordinates is performed. Then, in step S ″ ′ 3-1, data obtained in logarithmic / polar coordinates are discretized and aggregated. Here, a range having the same size as the registered image is cut out from the input image, and Equation 12 and the sampling interval _Tr are extracted. The conversion from the rectangular coordinates to the logarithmic / polar coordinates and the discretization were performed using 2π / N, and at this time, in order to reduce the above-mentioned various errors, the rectangular coordinates (x, y ) Are aggregated by adding the luminance value of each pixel to the corresponding point on the logarithmic / polar coordinates (r, t), whereby the discretized image data is It will be aggregated to the step size on polar coordinates.

[Smoothing on logarithmic and polar coordinates]
Further, in step S ″ ′ 3-2, smoothing using the vicinity 4 of an arbitrary pixel on logarithmic / polar coordinates is performed as in the above equation 9. Also in this example, a smoothing filter as shown in FIG. , The pixel adjacent to the target pixel in the upper, lower, left and right directions is smoothed by 1/4.

[Averaging of discrete Fourier transform and phase-only synthesis]
In step S ″ ′ 4, two-dimensional discrete Fourier transform is performed on the image data converted to logarithmic / polar coordinates in the radial and circumferential directions according to equation 13. Further, in step S ″ ′ 5, the phase is limited based on equation 16. Ask for composition. Here, when the same image is scaled at a fixed magnification and rotated by a fixed angle, Equation 16 can be separated into a component only for scaling and a component only for rotation, and can be expressed by Equation 19 below.

  Therefore, the correction may be performed using only the DC components R (u, 0) and R (0, s) after the Fourier transform. Here, Equation 17 is corrected using the value of Equation 19 as the value of Equation 16. When the input image is different from the registered image, Expression 14 does not hold, and the value of Expression 16 reflects an image difference, and an independent phase difference appears at each frequency. Cannot be represented by the product of components. Therefore, in this case, even if the correction according to Equation 19 is performed, the image does not approach the registered image.

[Correlation between images]
In step S ″ ′ 6, the input image is corrected using Expression 17 to obtain F ′ ₁ (u, s). In step S ″ ′ 7, the corrected input image F ′ ₁ (u, s) is obtained. The cross-correlation c of the registered image F ₂ (u, s) is calculated from the following equation (20). Based on this correlation value, in step S ″ ′ 8, image detection that is invariable in scale and rotation is performed.

Here, in the above equation, Re (*) is the real part of *. G ₁ and G ₂ are as shown in the following equation (21).

  In Equation 20, e (u, s) represents the correlation between images by the phase component. If this portion is set to 1, it becomes the same as the normalized correlation using only the amplitude component (luminance information) of the image. In the above-described pattern detection method, the pattern detection is performed in a form in which the scaled / rotated components of the registered image are canceled. Therefore, it is not necessary to scan the image in consideration of the enlargement / reduction and rotation, and since only the detection of the translation component is handled, the number of calculations is significantly reduced, and a high-speed operation with a low load is realized. In addition, since it is not necessary to perform inverse Fourier transform and it is not necessary to perform a step of extracting a correlation peak for angle detection, it is possible to realize a pattern detection that can be easily performed at a high speed in this regard. In this method, pattern detection independent of enlargement / reduction and rotation can be executed, but detection of the enlargement / reduction ratio and the rotation angle itself are not performed. It is needless to say that in an application that requires detection of the enlargement / reduction ratio and the rotation angle, an algorithm for separately calculating the enlargement / reduction ratio and the rotation angle can be used or added.

[Simulation experiment results]
Next, the results of a simulation experiment of pattern detection for various images to confirm the effectiveness when the above-described method is used will be described.

[Correlation of simple figures]
First, correlation is evaluated by the pattern detection method according to the second embodiment for each of the test image 1 and the test image 2 including the graphics shown in FIGS. 11 and 12 similar to the above. We verified the uniqueness and the correlation value difference between similar figures. Here, the test image 1 is a registered image, the test image 1 and the test image 2 are each input images, and the input image is 0.8, 1.0, and 1.2 times the registered image. The correlation value was obtained for the case where the image was enlarged and reduced, and further rotated from 0 to π / 2 in steps of π / 18. The results are shown in the graphs of FIGS. FIG. 24 shows the correlation values obtained for each rotation angle for the input image enlarged to 0.8 times, FIG. 25 to 1.0 times without enlargement / reduction, and FIG. ing. In these figures, the upper polygonal line indicates the autocorrelation of the test image 1, and the lower polygonal line indicates the cross-correlation between the test image 1 and the test image 2.

  As is clear from this figure, even when the image is enlarged / reduced / rotated, the correlation value of the autocorrelation of the registered image 1 itself always exceeds the correlation value of the registered image 2 and the minimum value of the autocorrelation , And a maximum value of the correlation value with the registered image 2. Therefore, when pattern detection is performed by the above method, it can be confirmed that the test image 1 and the test image 2 can be sufficiently distinguished. Since the threshold value of the correlation value for actually performing pattern detection is determined by the input image and the registered image, it is statistically processed from the obtained correlation value data. In the example of the figure, by setting the correlation value 0.85 as the threshold value, it is possible to identify the images in all of FIGS.

  In FIGS. 24 to 26, ideally, it is desired that the upper correlation value remains constant irrespective of the enlargement / reduction ratio and the angle of rotation. Are not equal to 1 except at positions of 0 and π / 2. That is, it is not completely invariant to scaling and rotation. This is because the error at the time of the logarithmic / polar coordinate conversion described above and the quantization error of the pixel position when the image is sequentially enlarged / reduced / rotated to generate the input image occur, and the original image and the converted image are strictly This is probably because they no longer match. Particularly, in this simulation experiment, since the binary image is used, the influence is large. However, when a multi-valued image of 256 gradations or the like is used in an actual application, such an effect is considered to be small because anti-aliasing is applied by an optical system at the time of image acquisition.

[Comparative test]
Next, in order to confirm the effectiveness of the pattern detection method according to Embodiment 2 in using the phase component, the correlation value was calculated without considering the phase component. More specifically, the test images in FIGS. 11 and 12 are used as registered images and input images in the same manner as described above, and the conditions such as the enlargement / reduction ratio and the rotation angle are the same as those in FIGS. The correlation value when s) was set to 1 was calculated. The results are shown in FIGS. As is clear from these figures, in this method, since only the amplitude information is compared without using the phase information, the difference between the images is reduced as a whole, and the correlation value is increased. However, since the minimum value of the correlation value for the test image 1 itself is smaller than the maximum value of the correlation value for the test image 2, the test image 1 and the test image 2 cannot be distinguished by a single threshold. From this, the effectiveness by the correlation value in consideration of the phase component by the pattern detection method according to the second embodiment was confirmed.

[Evaluation of robustness against noise]
Next, in order to evaluate the resistance to noise included in the image, the background of the input image was not plain but had a specific pattern, and pattern detection was performed. Here, a correlation value was calculated using the test image 9 obtained by synthesizing the image shown in FIG. 12 with the hatched background pattern as shown in FIG. 30 as an input image and the test image 1 in FIG. 11 as a registered image. Note that when the correlation value is directly calculated from FIG. 11 and FIG. 30, the correlation value is significantly reduced because the area of the background pattern is larger than the plain portion of the registered image. Therefore, in this example, it is assumed that the background pattern is known, the difference between the input image and the background pattern is obtained, and the difference is set as the target of correlation. The results are shown in FIGS. In the test image 1 of FIG. 11 to be compared in FIGS. 31 to 33, the correlation is calculated using the image synthesized with the background as the input image. The correlation value does not become 1. When the results shown in these figures are compared with the above-described FIGS. 24 to 26, the correlation value for the test image 1 itself decreases due to the image deterioration due to the removal of the background portion, and the correlation value for the test image 9 in FIG. Some have increased. For this reason, it is more difficult to distinguish sharply than when there is no background. However, in this example as well, the threshold can be set to 0.85 so that the two can be distinguished sufficiently.

[Extraction from input image containing multiple objects]
Next, as a simulation in a more practical case, evaluation was performed on a case where a target character is extracted from the test image 10 in which a plurality of characters are included in various sizes and inclinations as shown in FIG. Here, “F” of the test image 11 shown in FIG. 35 is searched as a registered image from an input image in which a plurality of alphabets are arranged obliquely and horizontally. The test image 10 of FIG. 34 includes three “F” s having various inclinations in addition to other alphabets. In FIG. 34, the size of the image is set to 512 × 512 pixels, and in FIG. 35, the size is set to 128 × 128 pixels as described above. In this evaluation test, the purpose is to evaluate the enlargement / reduction and rotation invariance of the pattern detection method according to the second embodiment of the present invention. Therefore, for pattern scanning in the vertical and horizontal directions, simply search sequentially for each pixel. It is carried out. In order to speed up the scanning, it can be combined with other known or future developed techniques in actual applications. The calculation result is shown in the graph of FIG.

  In FIG. 36, the x and y axes are the number of pixels shifted vertically and horizontally in x, y orthogonal coordinates with the origin at the upper left in the test image 10 shown in FIG. The correlation value at the position is plotted. Here, for the sake of clarity, only data having a correlation value of 0.6 or more is displayed. A plurality of peaks are recognized in the graph of FIG. 36. Of these peaks, three particularly prominent peaks correspond to the position of “F” included in the test image 10 of FIG. 34, that is, the upper, middle, and lower positions. ing. As described above, according to the second embodiment of the present invention, it has been confirmed that the registered image is properly detected from the input image regardless of the tilt, rotation, and enlargement / reduction. The numerical values shown at the respective peaks in FIG. 36 indicate the x coordinate, the y coordinate, and the correlation value at the detection position in order from the left. The exact positions of the three “F” in FIG. 34 are (180, 40), (180, 180), and (180, 320) with x and y coordinates from top to bottom, respectively. The magnifications are 1.2, 1.0, and 0.8, and the rotation angles are + 30 °, 0 °, and −45 ° when the clockwise direction is positive. Note that the F at the center where the rotation angle is 0 ° completely matches the detection result, but the F that has been scaled and rotated is detected with a shift of one pixel from the true value. One possible reason for this is that the shape of the character itself is degraded by the enlargement / reduction / rotation of the character performed when the input image is generated, in addition to the various errors as described above.

[processing speed]
Finally, the processing speed of the pattern detection method according to the second embodiment of the present invention will be discussed. Here, as in Embodiment 1, in order to confirm that the processing speed is improved by the pattern detection method according to Embodiment 2 as compared with the conventional normalized correlation method, the same image set is used. The processing time required to obtain one cross-correlation was compared between the pattern detection method according to the second embodiment and the repeat matching search method that calculates a normalized correlation for image scaling / rotation conversion. The image used for the comparison is a combination of the test image 1 and the test image 2 described above, each of which is processed 1000 times, and the average value is obtained and compared. In the pattern detection method according to the second embodiment of the present invention, the size of the image is reduced to 1/16 at the time of logarithmic / polar coordinate conversion. The size is also reduced to 1/16 of the original image. The computer used for measuring the processing time was the same as in the first embodiment.

  As a result, the average of the processing time 1000 times was 2.9 ms in the pattern detection method according to the second embodiment, and 10.5 ms in the method combining the normal normalized correlation and the scaling / rotation conversion. From this, it was confirmed that the pattern detection method according to the second embodiment is at least three times faster in one process than the normal normalized correlation. It is considered that this greatly contributes to the fact that the calculation time for performing the scaling / rotation conversion is not required in the method of the second embodiment. Therefore, the amount of computation to be processed is reduced, and the processing speed is increased. For reference, the processing time in the case where only the normalized correlation was simply performed without performing the conversion was 0.5 ms. However, in this case, of course, it is not possible to detect an image including enlargement / reduction / rotational deviation.

  In the pattern detection method according to the second embodiment as well, coordinate conversion from orthogonal coordinates to a logarithmic / polar coordinate system is required. However, if a conversion table or the like is used as in the first embodiment, it is not necessary to perform a matrix operation. Absent. Also, in the above example, an example in which the target image is fixed and the input image is enlarged / reduced / rotated has been described, but it goes without saying that the same result can be obtained even if the target image is enlarged / reduced / rotated.

  As described above, by using the phase-only correlation, it is possible to define a scaling value / rotation-invariant image correlation value, thereby realizing high-speed detection independent of the scaling / rotation of the input image. Was confirmed by various simulations.

  Further, in the above example, an example using a binary image has been described. However, the present invention can be applied to a multi-value image such as a gray scale image (shade image) such as 8 gray levels, 16 gray levels, and 256 gray levels, and a full color image. Needless to say, there is. Such a multivalued image has a larger amount of information than a binary image, and a search with higher accuracy and reliability can be obtained. In the case of a color image, a correlation is obtained based on predetermined information such as RGB components, luminance, and chromaticity.

  Furthermore, in the above example, the calculation in two dimensions has been described for simplicity, but it is needless to say that the calculation can be applied not only to two dimensions but also to one or more dimensions. When a search is performed by extracting a large number of feature amounts, a multidimensional search is performed, and the calculation amount is further increased. In such a case, however, according to the pattern detection method according to the embodiment of the present invention, In addition, a high-speed rotation-invariant search at a low cost, or a search invariant to scaling can be realized.

  Further, in the present specification, a description has been given of retrieval of a planar image as an example. However, the present invention can be applied not only to retrieval of two-dimensional images, but also to retrieval of stereoscopic images, audio, and other multimedia data. As described above, the present invention can be used for pattern detection in which an N-dimensional pattern is collated based on a spatial frequency characteristic and a difference between a registered pattern and an input pattern or a movement pattern is extracted. Therefore, the pattern detection device, the pattern detection method, the pattern detection program, and the computer-readable recording medium of the present invention are included in the scope of right to search data other than images regardless of their names.

  By using the present invention, pattern matching and image recognition can be performed at lower cost and at higher speed. Since it can be realized by a product-sum operation, it can be easily applied to, for example, an embedded device.

FIG. 9 is a schematic diagram illustrating an example of a pattern matching technique. It is the schematic which shows the pattern extraction using a phase only correlation. FIG. 7 is a schematic diagram showing a result of calculating an autocorrelation of a registered image by a normal correlation and a phase-only correlation. FIG. 9 is a schematic diagram illustrating a result of calculating a phase-only correlation for an input image obtained by horizontally moving a registered image. It is the schematic which shows the pattern extraction which used the phase-only correlation by transforming an image into polar coordinates. It is the schematic which shows a mode that a rotational movement amount is converted into a parallel movement amount by polar coordinate conversion. FIG. 1 is a block diagram illustrating a configuration of a pattern detection device according to an embodiment of the present invention. 5 is a flowchart illustrating a pattern detection method according to the first embodiment of the present invention. 9 is a flowchart illustrating a modification of the pattern detection method of FIG. FIG. 3 is a schematic diagram illustrating the concept of a smoothing filter used for smoothing on polar coordinates. FIG. 4 is an image diagram showing a test image 1; FIG. 9 is an image diagram showing a test image 2; 6 is a graph showing a result of calculating an autocorrelation of the test image 1 and a cross-correlation with the test image 2 by the pattern detection method according to the first embodiment of the present invention. FIG. 9 is an image diagram showing a test image 3; FIG. 9 is an image diagram showing a test image 4. 6 is a graph showing a result of calculating an autocorrelation of the test image 3 and a cross-correlation with the test image 4 by the pattern detection method according to the first embodiment of the present invention. FIG. 9 is an image diagram showing a test image 5. 6 is a graph showing a result of calculating an autocorrelation of the test image 5 and a cross-correlation with the test image 6 by the pattern detection method according to the first embodiment of the present invention. FIG. 9 is an image diagram showing a test image 7. FIG. 9 is an image diagram showing a test image 8. 6 is a graph showing a result of detecting a test image 8 from a test image 7 by the pattern detection method according to the first embodiment of the present invention. 9 is a flowchart illustrating a pattern detection method according to Embodiment 2 of the present invention. 9 is a flowchart illustrating a pattern detection method according to another embodiment of the present invention. 11 is a graph showing a result of calculating an autocorrelation of the test image 1 and a cross-correlation with the test image 2 at a reduction ratio of 0.8 by the pattern detection method according to the second embodiment of the present invention. 13 is a graph showing the result of calculating the autocorrelation of the test image 1 and the cross-correlation with the test image 2 at the same magnification by the pattern detection method according to the second embodiment of the present invention. 13 is a graph showing a result of calculating an autocorrelation of the test image 1 and a cross-correlation with the test image 2 at a magnification of 1.2 times by the pattern detection method according to the second embodiment of the present invention. 25 is a graph showing a result of calculating a correlation value without considering a phase component under the conditions of FIG. 24. 26 is a graph showing a result of calculating a correlation value without considering a phase component under the conditions of FIG. 25. 27 is a graph showing a result of calculating a correlation value without considering a phase component under the conditions of FIG. 26. FIG. 9 is an image diagram showing a test image 9. 9 is a graph showing a result of calculating an autocorrelation of the test image 1 and a cross-correlation with the test image 9 at a reduction ratio of 0.8 by the pattern detection method according to the second embodiment of the present invention. 13 is a graph showing the result of calculating the autocorrelation of the test image 1 and the cross-correlation with the test image 9 at the same magnification by the pattern detection method according to the second embodiment of the present invention. 13 is a graph showing a result of calculating an autocorrelation of the test image 1 and a cross-correlation with the test image 9 at a magnification of 1.2 times by the pattern detection method according to the second embodiment of the present invention. FIG. 4 is an image diagram showing a test image 10. FIG. 7 is an image diagram showing a test image 11. 9 is a graph showing a result of detecting a test image 11 from a test image 10 by the pattern detection method according to the second embodiment of the present invention.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 1 ... Input image storage part 2 ... Registered image storage part 3 ... Image processing part 4 ... Control part 5 ... Display part 6 ... Input part 7 ... Image source

Claims (20)

A pattern detection device that performs pattern detection in image processing of searching for a registered image registered in advance from an input image,
An input image storage unit (1) for inputting and holding an input image from outside,
A registered image storage unit (2) for storing a registered image to be detected,
Each image data is read from the input image storage unit (1) and the registered image storage unit (2), and the Fourier transform is performed based on the polar coordinates of each pixel constituting the registered image and the input image. The phase component is extracted based on the input image, and the complex conjugate of the phase-only synthesis obtained by synthesizing the phase component is integrated into the Fourier-transformed input image to correct the input image, and the corrected input image and the registered image And an image processing unit (3) for performing a pattern detection of the registered image from the input image based on the calculated correlation value,
A pattern detection device comprising: A pattern detection device that performs pattern detection in image processing of searching for a registered image registered in advance from an input image,
An input image storage unit (1) for inputting and holding an input image from outside,
A registered image storage unit (2) for storing a registered image to be detected,
A polar coordinate conversion unit that reads each image data from the input image storage unit (1) and the registered image storage unit (2), and converts the rectangular coordinates of each pixel forming the registered image and the input image into polar coordinates,
A sampling unit that samples the polar coordinates of the registered image and the input image converted by the polar coordinate conversion unit at a predetermined sampling cycle,
A discrete Fourier transform unit for performing a discrete Fourier transform on the registered image and the input image sampled by the sampling unit,
Based on the registered image and the input image Fourier-transformed in the discrete Fourier transform unit, a phase-only synthesis unit that calculates a phase-only synthesis,
A complex conjugate integration unit for integrating the complex conjugate of the phase-only synthesis calculated by the phase-only synthesis unit to the input image that has been Fourier-transformed by the Fourier transform unit, and calculating the corrected input image,
A correlation operation unit that calculates a correlation value between the input image and the registered image corrected by the complex conjugate integration unit,
Based on the correlation value calculated by the correlation calculation unit, a pattern detection unit that performs pattern detection of the registered image from the input image,
A pattern detection device comprising: The pattern detection device according to claim 1 or 2,
A pattern detection apparatus, wherein the polar coordinate conversion unit converts the orthogonal coordinates of each pixel into logarithmic / polar coordinates. A pattern detection method for searching registered data registered in advance from input data,
Performing a Fourier transform based on the polar coordinates of each data constituting the registration data;
Performing a Fourier transform based on polar coordinates of each data constituting the input data;
Calculating a phase-only synthesis from the Fourier-transformed registration data and the input data;
Integrating the Fourier-transformed input data with the complex conjugate of phase-only synthesis to calculate the corrected input data;
Calculating a correlation value between the corrected input data and the registered data;
Detecting registered data from the input data based on the calculated correlation value;
A pattern detection method comprising: A pattern detection method in image processing of searching for a registered image registered in advance from an input image,
Performing a Fourier transform based on the polar coordinates of each pixel constituting the registered image;
Performing a Fourier transform based on the polar coordinates of each pixel constituting the input image;
Calculating a phase-only combination from the Fourier-transformed registered image and the input image;
Integrating the Fourier-transformed input image with the complex conjugate of phase-only synthesis to calculate a corrected input image;
Calculating a correlation value between the corrected input image and the registered image;
Performing a pattern detection of the registered image from the input image based on the calculated correlation value;
A pattern detection method comprising: A pattern detection method in image processing of searching for a registered image registered in advance from an input image,
Converting the rectangular coordinates of each pixel constituting the registered image and the input image into polar coordinates,
Discretizing the registered image and the input image displayed in polar coordinates at a predetermined sampling cycle,
Discrete Fourier transforming each of the discretized registered image and the input image;
Calculating a phase-only synthesis based on the discrete Fourier transformed registered image and the input image;
A step of integrating the complex conjugate of the phase-only synthesis into the discrete Fourier transformed input image to calculate a corrected input image;
Calculating a correlation value between the corrected input image and the registered image;
Performing a pattern detection of the registered image from the input image based on the calculated correlation value;
A pattern detection method comprising: The pattern detecting method according to claim 6, wherein
When each of the registered image and the input image displayed in polar coordinates is discretized at a predetermined sampling period, the pixels of the sampled registered image and the input image are smoothed based on the pixels around the pixel. A pattern detection method characterized by performing conversion. It is a pattern detection method of Claim 6 or 7, Comprising:
When each of the registered image and the input image displayed in polar coordinates is discretized at a predetermined sampling period, a point on the orthogonal coordinates that satisfies a predetermined condition with respect to each pixel of the sampled registered image and the input image is corresponded. A pattern detection method, wherein aggregation is performed by adding an amplitude component of each pixel to a point on polar coordinates. The pattern detection method according to any one of claims 5 to 8,
When calculating a correlation value between a corrected input image and a registered image, a pattern detection method is characterized in that integration is performed on pixels whose amplitude components are equal to or greater than a predetermined threshold value. The pattern detection method according to any one of claims 5 to 9,
When calculating a correlation value between a corrected input image and a registered image, a weighted average is obtained based on a displacement amount of an amplitude component of each pixel, thereby performing an enhancement process based on the amplitude component. . The pattern detection method according to any one of claims 4 to 10,
A pattern detection method characterized by converting coordinates of each data constituting registration data and input data into logarithmic and polar coordinates. A pattern detection program for searching registered data registered in advance from input data,
A function of performing a Fourier transform based on polar coordinates of each data constituting the registration data,
A function of performing a Fourier transform based on the polar coordinates of each data constituting the input data,
A function of calculating a phase-only combination from the Fourier-transformed registration data and input data,
A function of integrating the complex conjugate of the phase-only synthesis with the Fourier-transformed input data, and calculating the corrected input data;
A function of calculating a correlation value between the corrected input data and the registered data,
A function of detecting registered data from input data based on the calculated correlation value,
Pattern detection program for realizing. A pattern detection program in image processing for searching for a registered image registered in advance from an input image, the computer comprising:
A function of performing a Fourier transform based on the polar coordinates of each pixel constituting the registered image,
A function of performing a Fourier transform based on the polar coordinates of each pixel constituting the input image,
A function of calculating a phase-only combination from the registered image and the input image subjected to the Fourier transform,
A function of integrating the complex conjugate of the phase-only synthesis into the Fourier-transformed input image, and calculating a corrected input image;
A function of calculating a correlation value between the corrected input image and the registered image,
A function of detecting a pattern of a registered image from an input image based on the calculated correlation value,
Pattern detection program for realizing. A pattern detection program in image processing for searching for a registered image registered in advance from an input image, the computer comprising:
A function of converting the rectangular coordinates of each pixel constituting the registered image and the input image into polar coordinates,
A function of discretizing the registered image and the input image displayed in polar coordinates at a predetermined sampling cycle,
A function of performing discrete Fourier transform on the discretized registered image and the input image, respectively,
A function of calculating phase-only synthesis based on the registered image and the input image subjected to the discrete Fourier transform,
A function of integrating the complex conjugate of the phase-only synthesis into the discrete Fourier transformed input image, and calculating a corrected input image;
A function of calculating a correlation value between the corrected input image and the registered image,
A function of detecting a pattern of a registered image from an input image based on the calculated correlation value,
Pattern detection program for realizing. A pattern detection program according to claim 14,
When each of the registered image and the input image displayed in polar coordinates is discretized at a predetermined sampling period, the pixels of the sampled registered image and the input image are smoothed based on the pixels around the pixel. A pattern detection program characterized in that the pattern detection is performed. It is a pattern detection program of Claim 14 or 15, Comprising:
When each of the registered image and the input image displayed in polar coordinates is discretized at a predetermined sampling period, a point on the orthogonal coordinates that satisfies a predetermined condition with respect to each pixel of the sampled registered image and the input image is corresponded. A pattern detection program for integrating by adding the amplitude component of each pixel to a point on polar coordinates. The pattern detection program according to any one of claims 14 to 16,
A pattern detection program for calculating, when calculating a correlation value between a corrected input image and a registered image, for pixels whose amplitude components are equal to or greater than a predetermined threshold. The pattern detection program according to any one of claims 14 to 17,
A pattern detection program for calculating a correlation value between a corrected input image and a registered image, performing a weighting average based on a displacement amount of an amplitude component of each pixel, thereby performing an emphasis process based on the amplitude component. . The pattern detection program according to any one of claims 12 to 18, wherein
A pattern detection program for converting coordinates of each data constituting registration data and input data into logarithmic and polar coordinates. A computer-readable recording medium that records the pattern detection program according to claim 12.

Images