JP2010097436A - Positioning method for pattern model in image processing, image processing apparatus, image processing program and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To stably obtain a result of precise positioning regardless of the graphics of a search target image. <P>SOLUTION: There is provided a positioning method of a pattern model in image processing for, when searching and positioning a search object similar to a preregistered image from search target images by using a pattern model corresponding to the pertinent registered image, calculating an error value to be used for calculation by a method of least square with a position relation between a point under consideration and a corresponding point corresponding to the point under consideration, and for solving simultaneous equations obtained by the method of least square from those values, and for carrying out positioning with higher precision than an initial position given with prescribed precision. The method of least square includes an item which becomes minimum when it is matched with an ideal position and an item which becomes minimum when it is matched with a trial parameter as the variables of the method of least square. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention uses a pattern model corresponding to a registered image registered in advance to perform precise positioning with higher accuracy when searching for a search target similar to the registered image from the search target images. The present invention relates to a pattern model positioning method, an image processing apparatus, an image processing program, and a computer-readable recording medium in image processing to be performed.

  An image processing apparatus that processes an image picked up by an image pickup element is generally an image pickup apparatus that picks up an image processing object (hereinafter also referred to as “work”), and an image data storage that stores data of the image picked up by the image pickup apparatus. And image data processing means for processing the image data stored in the image data storage means. For example, in an image processing apparatus in which the image pickup apparatus is constituted by a CCD camera, luminance data (so-called multi-value data) such as 256 gradations and 1024 gradations based on the respective charge amounts of a large number of charge-coupled elements constituting the image pickup surface. Thus, the size, position, rotation angle, etc. of the workpiece that is the search object can be obtained. Conventionally, as a technique for processing image data and searching for a search target by image processing, a difference search using a total value of absolute values of pixel difference values between images or a normalized correlation value between images is searched. The normalized correlation search used for the above is known. In these searches, a search object to be searched is registered in advance as a template image, and a search for the search object is executed from the search object image based on the image. In these search processes, an area-based search using image data has been the mainstream. However, such a conventional region-based search based on the density of an image has a problem that it is easily influenced by an illuminance change at the time of image capturing.

  On the other hand, a method has also been proposed in which edge extraction processing is performed on a registered image and a search target image and a search is performed based on edge information. In this method, since edge data based on the amount of change in density value is used instead of the density value of the pixels constituting the image data, there is an advantage that it is difficult to be affected by fluctuations in illuminance during imaging. In particular, in recent years, edge-based pattern search using edges as a feature amount has attracted attention due to its robustness and has been put to practical use in industrial applications and the like.

  As a technique for improving the processing speed of pattern search, a “coarse to fine” approach is known. In other words, first, a rough search is performed using a low-resolution image (course image), a rough position is specified, and then a high-resolution image (fine image) is used to perform detailed positioning to obtain position and posture accuracy. Is increasing. When performing edge-based search using course-to-fine template matching, the original data is compressed (also called “thinning” etc.) and the search is performed with coarse data to specify the approximate position. Pyramid search that performs a search using detailed data is used. The concept of the pyramid search is shown in FIG. As shown in this figure, a rough search (called “rough search” or the like) is performed using a low-resolution image with a high reduction ratio, and after obtaining a rough position, an intermediate reduction ratio with an increased resolution. Then, a search in the vicinity is performed, and finally a detailed search is performed on an image having an original size or a reduction ratio close thereto. As described above, in the pyramid search, generally, a plurality of images with different resolutions are prepared, and first, the approximate position is detected from the image with the lowest resolution. In the subsequent processing, the search range is narrowed down to the vicinity of the previous detection position while gradually increasing the resolution. As a result, the accuracy of the detection position increases each time the processing stage proceeds, and finally, a highly accurate position can be detected with the original image or a resolution close thereto. In this method, the position can be detected efficiently by searching a wide search range with an image having a low resolution.

  An image processing apparatus disclosed in Patent Document 1 is known as a technique related to precise positioning for obtaining a position and orientation with high accuracy by such course-to-fine template matching. In such precision positioning, the least square method has been widely used.

  In precision positioning using a conventional least square method, the distance between a point and a tangent is calculated as an error function. Based on FIG. 47 and FIG. 48, the case of performing pattern matching of a workpiece with a part cut out in a planar shape will be described as an example of precision positioning using a conventional least square method. In these drawings, FIG. 47 shows a state in which a rough search has been performed and a certain degree of positioning has been completed, and FIG. 48 shows a state in which fine positioning has been performed and the pattern model PM has been attempted to be superimposed on the search target image. . In these drawings, the thick line indicates the pattern model PM, and the thin line indicates the edge point (input edge point IE) of the search target image. Further, the attention point of the input edge point IE is indicated by a broken line arrow, and the tangent SL of the input edge point is indicated by a broken line. The actual tangent line SL is shorter and exists in a state where it overlaps the fine line pattern model PM. However, in these drawings, the tangent line SL is displayed longer for convenience of explanation.

  In the least square method, the distance between the input edge point and the tangent SL is used as an error function, and the pattern model PM is moved (rotated) so that this distance becomes small. Now, it is assumed that the pattern model PM and the input edge point are overlapped to some extent as a result of the rough search as shown in FIG. If further precise positioning is performed from this state, the pattern model PM is rotated clockwise (or the input edge point side is rotated clockwise) as shown in FIG. Expected to match.

As described above, in the alignment using the least square method, the error value is moved and rotated in the direction in which the error value becomes zero. However, as a result of diligent research by the present inventor, there was a phenomenon in which movement and rotation in a direction in which the accuracy deteriorates when the least square method is used for precise positioning under specific conditions. For example, when precise positioning is performed for a highly symmetrical figure such as a circle shown in FIG. 85, the error value hardly changes even if the circle is rotated around the center of rotation, so it rotates in either direction. It will be possible. For this reason, it is possible to rotate in the direction opposite to the original direction of rotation, and as a result, the error value does not converge and precise positioning fails. The rough position obtained by the rough search is given as the initial position of the fine positioning, but the situation that the result of the fine positioning becomes worse than this rough position may occur, which is an obstacle to stable pattern search. It was.
Japanese Patent No. 3759983 JP-A-7-128017 US Pat. No. 6,408,109 B1

  The present invention has been made in view of such a situation. A main object of the present invention is to provide an image data compression method, a pattern model positioning method in image processing, an image processing apparatus, and an image processing capable of stably obtaining a result of precise positioning regardless of a figure of a search target image. To provide a recording medium readable by a program and a computer.

Means for Solving the Problems and Effects of the Invention

  In order to achieve the above object, a pattern model positioning method in the first image processing uses a pattern model corresponding to a registered image registered in advance, and a search target similar to the registered image from search target images. When performing positioning by searching for an error value used in the least-squares method calculation based on the positional relationship between the point of interest and the corresponding point corresponding to the point of interest, and simultaneous values obtained by these methods using the least-squares method A pattern model positioning method in image processing that solves an equation and performs positioning with higher accuracy than an initial position given with a predetermined accuracy, and is minimized when the least squares method matches an ideal position. It is possible to include a term and a term that becomes minimum when it matches a trial parameter that is a least squares variable. In this way, in the least squares method, by adding a term that increases the error value as the amount of change in the trial parameter increases, the solution is searched in such a direction that both terms become smaller. It is possible to avoid the situation of rotating or diverging in a different direction and stabilizing the positioning result.

  The pattern model positioning method in the second image processing uses a pattern model corresponding to a registered image registered in advance to search for a search object similar to the registered image from the search target image and perform positioning. A pattern model positioning method in image processing in which positioning is performed with higher accuracy than an initial position given with a predetermined accuracy, and a search target image is obtained and registered in the search target image An initial position corresponding to the image is obtained, and according to the initial position, the pattern model is arranged so as to overlap the search target image, and individual correspondence on the search target image corresponding to each segment constituting the pattern model The minimum step defined as the first error value is the step of obtaining the edge point and the distance between each segment and its corresponding edge point. A first error function for calculating a cumulative value of the first error value of multiplication, and a change amount when changing a trial parameter which is a variable in the first error function is a second error value, and the second error value And a second error function for calculating a cumulative value of the first error function, and calculating the least squares arithmetic expression, so that the sum of the first error function and the second error function is minimized. And a step of performing precise positioning with higher accuracy than the accuracy of. As a result, in the least squares calculation, the second error value is set to increase as the amount of change in the trial parameter for obtaining the ideal position increases, so that a situation in which a large displacement of the trial parameter is solved can be avoided. The stability of precision positioning results can be improved.

Furthermore, in the pattern model positioning method in the third image processing, when the second error function is P _{i as} the trial parameter of the least square method and the initial value of the trial parameter is P _0i , P _i −P _0i Can be a function of

  Furthermore, the pattern model positioning method in the fourth image processing includes the second error function,

And can.

  Furthermore, the fifth image processing apparatus uses a pattern model corresponding to a registered image registered in advance to search for a search object similar to the registered image from the search target image and perform positioning. An image processing apparatus for positioning with higher accuracy than an initial position given in advance with a predetermined accuracy, comprising: an image input means for acquiring a registered image and a search target image; and a registered image acquired by the image input means Contour extraction means for extracting a contour, and a plurality of edge points are extracted from the contour extracted by the contour extraction means, and adjacent edge points are continuously connected among the plurality of extracted edge points. A chain creating means for creating a chain, and a segment for creating a segment approximated by a line segment and / or an arc for each of one or more chains created by the chain creating means. A plurality of base points are set on the segment created by the segment creating means and the segment creating means, and a corresponding point search line having a predetermined length passing through the base point and substantially orthogonal to the contour is assigned to each base point. A pattern model construction means for constructing a pattern model of the registered image, and an initial position corresponding to the registered image included in the search target image obtained by the image input means, and the pattern model is obtained according to the initial position. Each corresponding edge point on the search target image corresponding to each segment constituting the pattern model is obtained by overlapping the search target image, and the distance between each segment and the corresponding edge point is set as the first distance. By calculating a least square method defined as an error value so that the accumulated value of the error value is minimized, the initial position given initially is calculated. A precision positioning means for performing precise positioning with higher accuracy, and a least squares error function used in the precision positioning means includes a first error function that defines the first error value, and And a second error function having a second error value as the amount of change from the trial parameter of the variable in the least square method. In this way, by adding the term of the trial parameter change to the least squares method expression, the second error value increases as the trial parameter change amount increases, so that a large displacement of the trial parameter is solved. The situation can be avoided and the stability of precision positioning results can be improved.

  Furthermore, the sixth image processing program uses a pattern model corresponding to a registered image registered in advance to search for a search object similar to the registered image from the search target image and perform positioning. An image processing program for positioning with higher accuracy than an initial position given in advance with a predetermined accuracy, from an image input function for acquiring a registered image and a search target image, and a registered image acquired by the image input function A contour extraction function for extracting a contour, and a plurality of edge points are extracted from the contour extracted by the contour extraction function, and the adjacent edge points are continuously connected among the extracted plurality of edge points. A chain creation function for creating a chain and a segment approximated by a line segment and / or arc for each of one or more chains created by the chain creation function. The segment creation function to be formed, and a plurality of base points are set on the segment created by the segment creation function, and for each base point, a corresponding point search line having a predetermined length that passes through the base point and is substantially orthogonal to the contour A pattern model construction function for constructing a pattern model of a registered image to which an image is assigned, an initial position corresponding to a registered image included in a search target image obtained by the image input function, and the pattern according to the initial position The model is arranged so as to overlap the search target image, and each corresponding edge point on the search target image corresponding to each segment constituting the pattern model is obtained, and the distance between each segment and the corresponding edge point is determined. By calculating the least square method defined as the first error value so that the accumulated value of the error value is minimized, the initial value is given. And a precision positioning function for performing precise positioning with higher accuracy than the initial position, and a least squares error function used in the precision positioning function defines the first error value. And a second error function in which the amount of change from the trial parameter of the variable in the least square method is a second error value. In this way, by adding the term of the trial parameter change to the least squares method expression, the second error value increases as the trial parameter change amount increases, so that a large displacement of the trial parameter is solved. The situation can be avoided and the stability of precision positioning results can be improved.

  Furthermore, a seventh computer-readable recording medium stores the above program. Recording media include CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray, etc. A medium that can store a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and other programs is included. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Further, the recording medium includes a device capable of recording the program, for example, a general purpose or dedicated device in which the program is implemented in a state where the program can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or hardware may be executed by hardware such as a predetermined gate array (FPGA, ASIC), or program software. And a partial hardware module that realizes a part of hardware elements may be mixed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the following embodiments exemplify a pattern model positioning method, an image processing apparatus, an image processing program, and a computer-readable recording medium in image processing for embodying the technical idea of the present invention. Therefore, the present invention does not specify a pattern model positioning method, an image processing apparatus, an image processing program, and a computer-readable recording medium in image processing as follows. Further, the present specification by no means specifies the members shown in the claims to the members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the embodiments are not intended to limit the scope of the present invention unless otherwise specified, and are merely explanations. It's just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing.

  The connection between the image processing apparatus used in the embodiment of the present invention and a computer, a printer, an external storage device and other peripheral devices for operation, control, display, and other processing connected thereto is, for example, IEEE 1394, RS-232x, RS-422, serial connection such as USB, parallel connection, or communication via network such as 10BASE-T, 100BASE-TX, 1000BASE-T, etc. Do. The connection is not limited to a physical connection using a wire, but may be a wireless connection using a wireless LAN such as IEEE802.1x, radio waves such as Bluetooth (registered trademark), infrared light, optical communication, or the like. Furthermore, a memory card, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be used as a recording medium for exchanging data or storing settings. In this specification, the term “image processing device” is used to include not only a device main body that performs edge extraction, pattern matching, and the like, but also a contour extraction system that combines peripheral devices such as a computer and an external storage device.

Further, in this specification, a pattern model positioning method, an image processing apparatus, an image processing program, and a computer-readable recording medium in image processing include edge extraction, measurement area setting, edge point connection system itself, and various images. However, the present invention is not limited to an apparatus or method that performs input / output, display, calculation, communication, and other processes related to imaging and acquisition in hardware. An apparatus and method for realizing processing by software are also included in the scope of the present invention. For example, by incorporating software, programs, plug-ins, objects, libraries, applets, compilers, modules, macros that run on specific programs, etc. into general-purpose circuits or computers, edge extraction or edge point connection itself, or processing related to this The apparatus and system that enable the image processing correspond to any one of the pattern model positioning method, the image processing apparatus, the image processing program, and the computer-readable recording medium in the image processing of the present invention. In this specification, the computer includes a general-purpose or dedicated computer, a workstation, a terminal, a portable electronic device, PDC, CDMA, W-CDMA, FOMA (registered trademark), GSM, IMT2000, fourth generation, and the like. Mobile phones, PHS, PDAs, pagers, smartphones and other electronic devices. Further, in the present specification, the program is not limited to a program that is used alone, an aspect that functions as a part of a specific computer program, software, service, etc., an aspect that is called and functions when necessary, an environment such as an OS, etc. It can also be used as a mode provided as a service, a mode that operates resident in the environment, a mode that operates in the background, and other support programs.
(Rough flow of image processing)

  FIG. 1 shows a block diagram of the image processing apparatus 100. The image processing apparatus 100 registers an image to be searched in advance as shown in FIG. 2, creates a pattern model from the registered image, and in actual operation, the pattern is extracted from the input search target images. Find the position corresponding to the model. Here, an outline of the operation when creating the pattern model is shown in the flowchart of FIG. In the present embodiment, as shown in FIG. 2A, the user sets an area for creating a pattern model, that is, a pattern window PW, for the registered image RI to be searched (step S301 in FIG. 3). The image including the area where the pattern window PW is set is reduced as necessary as shown in FIG. 2B (step S302). Further, a pattern model PM as a search pattern corresponding to the registered image RI is created from the reduced pattern window RPM as shown in FIG. 2C (step S303). As described above, the image processing apparatus creates in advance a pattern model PM corresponding to the registered image RI to be searched from the search target images prior to the actual search operation.

  Further, the pattern model PM is reduced to a reduction rate for the first edge angle bit image and a reduction rate (described later) for the second edge angle bit image as shown in FIG. Step S304). Such a reduced pattern model RPM can be created in advance at the time of registration, or can be performed each time during operation.

On the other hand, the outline of the operation during operation is shown in the flowchart of FIG. During operation, after inputting the search target image OI in step S401 (FIG. 2 (e)), in step S402, the search target image OI is reduced as necessary to obtain a reduced search target image ROI (FIG. 2 (f). )). In step S403, an edge angle bit image EB is generated from the reduced search target image ROI (FIG. 2 (g): details will be described later). In step S404, an edge angle bit reduced image REB obtained by reducing the edge angle bit image EB is created (FIG. 2 (h)). In step S405, a pattern search is performed on the edge angle bit reduced image REB obtained in this way using the pattern model obtained at the time of registration.
(Image processing apparatus 100)

Next, the configuration of the image processing apparatus 100 will be described. An image processing apparatus 100 shown in the block diagram of FIG. 1 includes an image processing apparatus main body 100A, an image input unit 1 that inputs an image, a display unit 3 that displays an image and various data, and a user's various operations. An operation unit 2 and an output unit 5 constituting an output interface for outputting the result of image processing by the image processing apparatus main body 100A to the outside are provided. The image input means 1 is composed of an imaging means such as a CCD. The input image input from the image input unit 1 is taken into the image processing apparatus main body 100A via the A / D conversion unit. The display unit 3 displays an original image of the input image and an edge image obtained by performing image processing on the original image by, for example, edge image processing.
(Image processing apparatus main body 100A)

The image processing apparatus main body 100A includes a storage unit 4 that stores various data and a calculation unit 6 that performs various calculations related to image processing. In addition to the registered image input from the image input unit 1 via the A / D conversion unit, the storage unit 4 stores a pattern model generated using the registered image as a template image, a search target image, and a pattern model from the registered image. It has a work area used for holding and storing temporary data created at the time of generation and search.
(Calculation means 6)

The computing means 6 performs edge detection on the image and extracts a contour, a contour extracting means 62 for creating a contour, a chain creating means 63 for creating a chain from the contour information, a segment creating means 68 for creating a segment from the chain, and the like And various means. As other means, an edge angle bit image creating means 69, an edge angle bit image reducing means 78, an image reducing means 77 for reducing an image to a predetermined magnification, a pattern model constituting means 70 for constituting a pattern model, and a rough search are performed. Examples include a coarse search means 71, and a fine positioning means 76 that performs high-precision positioning within a search target image using a pattern model of a registered image based on edge information (details will be described later). As described above, the calculation unit 6 registers a pattern model and performs a search on a search target image using the registered pattern model. In order to perform such an operation, the chain creation means 63, the segment creation means 68, and the pattern model construction means 70 are used when registering the pattern model. The coarse search means 71 and the fine positioning means 76 are used during actual operation. Further, the image reduction means 77, the contour extraction means 62, the chain creation means 63 and the like are used for both the registration operation and the operation operation. The edge angle bit image creation means 69 and the edge angle bit image reduction means 78 are used during the operation. used. The chain creating means 63 is used only at the time of registration in the present embodiment, but it is also possible to chain the search target images during driving.
(Outline extraction means 62)

  The contour extracting unit 62 includes an edge angle / edge intensity image creating unit 60 that creates an edge angle image and an edge strength image as edge images, and a thinning unit 61 that thins edge points. Specifically, the edge angle / edge strength image creation means 60 applies to the multivalued image (original image of the original size or the reduced image reduced by the image reduction means 77 described later) input by the image input means 1. A known filter for extracting edge points, for example, a Sobel filter for each of the X and Y directions, and an edge intensity image (X direction edge intensity image component) in each XY direction. In addition, a Y-direction edge intensity image component) is generated, and a two-dimensional edge angle image is generated from the edge intensity images in the XY directions. The thinning means 61 thins the edge points of the edge image created by the edge angle / edge strength image creating means 60 by, for example, edge strength non-maximal point suppression processing.

In this specification, the edge strength is a numerical value indicating the degree of whether one pixel is a part of an edge (dark → light). In general, the edge strength is calculated from the pixel value of the pixel of interest and nine pixels located around it. The edge angle indicates the direction of the edge in one pixel, and is generally calculated from the edge intensities in the X and Y directions obtained by the above-described Sobel filters in the X and Y directions.
(Chain creation means 63)

On the other hand, the chain creating means 63 and the segment creating means 68 are used when generating a pattern model using a part of the registered image as a template image. The chain creating means 63 in this embodiment includes an edge chain forming means 64 and a chain filter means 66. Specifically, the edge chaining unit 64 creates a chain by connecting a plurality of adjacent edge points among a plurality of edge points included in the edge image generated by the contour extracting unit 62. The chain filter unit 66 filters the plurality of chain groups generated by the edge chaining unit 64 with various chain feature quantities.
(Segment creation means 68)

  The segment creation means 68 includes an edge chain segmentation means 65 and a segment selection means 67. The edge chain segmentation unit 65 performs approximation on each chain created by the edge chaining unit 64 and filtered by the chain filter unit 66 to create a segment. The segment here is a line segment and / or an arc that approximates the chain by the least squares method. The segment selection means 67 performs filtering on the segment.

Furthermore, the segment selection means 67 can also have a pattern feature selection function. That is, more stable positioning can be realized by changing the selection criteria of the segments constituting the pattern model in accordance with the pattern characteristics obtained from the search object. Details will be described later.
(Pattern model construction means 70)

The pattern model construction unit 70 is a unit that generates a pattern model to be stored in the storage unit described above. Specifically, the registered image is processed by the above-described contour extraction unit 62, chain creation unit 63, and segment creation unit 68, and each generated segment is processed by the pattern model construction unit 70. Details of the pattern model will be described later.
(Image reduction means 77)

  On the other hand, the image reduction means 77 is a means for reducing the registered image and the search target image. This reduction ratio is automatically set by the automatic reduction ratio determination mode. Specifically, as described above, in the present embodiment, the user sets an area desired to be a pattern model for a registered image. Here, the reduction ratio is automatically determined according to the size set by the user, that is, the size of the pattern window PW for designating the pattern model. For example, when the pattern window PW is rectangular, the reduction ratio is determined according to the length of the short side.

  In this specification, “increasing the reduction ratio”, “large reduction ratio”, and “high reduction ratio” are to increase the degree of reduction or the compression ratio, for example, a reduced image with a reduction ratio of 1/8. Is reduced to a reduction ratio of 1/16. Conversely, “reducing the reduction ratio” or “small reduction ratio” and “low reduction ratio” mean that the degree of reduction is suppressed, for example, the reduced image is brought closer to the same size image side, for example, reduction This means that an image with a rate of 1/16 is reduced to 1/8. The “reduction ratio equal to or greater than the same size” does not mean to enlarge the same-size image, but means an equal-size image, that is, a reduction ratio higher than the reduction ratio 1 or an equal-size image with a reduction ratio of 1.

  In addition to such an automatic reduction ratio determination mode, a manual reduction ratio determination mode in which the user selects and determines a desired reduction ratio can also be used. In the example of the user interface shown in FIG. 5, the user is made to select from a plurality of different reduction rates, here 1/2, 1/4, and 1/8 as the global search reduction rates. The reduction ratio may be designated by an arbitrary numerical value. Further, an image showing how the original image is deformed when the image is reduced at the reduction ratio selected here may be displayed together. In this manner, the local search reduction rate (intermediate reduction rate) and the precise positioning reduction rate can be set in addition to the global search reduction rate (global reduction rate).

  Furthermore, a reduction rate for local search can be automatically added. Specifically, when the ratio between the global search reduction ratio and the local search reduction ratio is larger than a predetermined value, an additional reduction ratio (for local search) is selected between the global search reduction ratio and the local search reduction ratio. A local search (additional local search) set to (reduction ratio) is automatically added (see FIG. 10). Further, the additional local search can be performed a plurality of times. In this way, the ratio between the global search reduction ratio and the local search reduction ratio can be maintained within a predetermined value, and the search speed can be shortened.

The image reduction unit 77 is used for a registered image for generating a pattern model and a search target image that is reduced at the same reduction rate as that set for the registered image. In other words, the search target image is also reduced at the reduction rate set for the registered image.
(Edge angle bit image creation means 69)

  The edge angle bit image creation unit 69 generates the search target image and the reduced image obtained by reducing the search target image by the image reduction unit 77 by the edge angle / edge strength image creation unit 60 of the calculation unit 6 described above. An edge angle bit image is generated by using the edge angle image. That is, the edge angle bit image creation means 69 is used to generate an edge angle bit image of the search target image during driving. In other words, it is not used when registering a pattern model.

More specifically, as shown in FIG. 6, 8-bit data obtained by dividing an edge angle value of 0 to 360 ° into 8 units in units of 45 ° (FIG. 6B) is provided for each pixel of the edge angle image. It is determined whether the corresponding edge angle is a bit position assigned to each of these eight divided areas (FIG. 6A), and a flag of 1 is set at the determined bit position. Thus, the edge angle image is converted into an edge angle bit image (FIG. 6C). In the present embodiment, the edge angle bit image generating unit 69 generates an edge angle bit image using the edge angle image generated by the edge angle / edge strength image generating unit 60 of the calculating unit 6 (for details, see FIG. And will be described later based on FIG. 30). However, the present invention is not limited to this method, and the edge angle bit image may be generated by the edge angle bit image creating means 69 using, for example, the edge angle image thinned by the thinning means 61 of the computing means 6.
(Edge angle bit image reduction means 78)

  The edge angle bit image reduction means 78 is a means for reducing the edge angle bit image generated by the edge angle bit image creation means 69 described above. Thereby, an edge angle bit reduced image is obtained.

  The reduction rate of the edge angle bit reduced image in the present embodiment is determined according to the size of the pattern window PW in which the user sets an area that the user wants to make a pattern model for the registered image. Therefore, when the reduction rate in the edge angle bit image reduction means 78 is automatically determined, the size of the pattern window PW set for the registered image is reflected. However, the present invention is not limited to this method, and it goes without saying that the reduction rate in the edge angle bit image reduction means 78 may be set directly by checking the reduction rate of the pattern model.

  This situation will be described in comparison with the conventional method. When the edge angle bit image is reduced by the conventional method, for example, when the reduction ratio is 1/8 of the original image size of the search target image, 8 × The region of 8 pixels, that is, the edge angle of one pixel representing 64 pixels or the edge angle of one pixel obtained by reducing the average value of 64 pixels was used as the representative value of the region. On the other hand, in the reduction of the edge angle bit image by the edge angle bit image reduction means 78 according to the present embodiment, for example, when the reduction ratio is 1/8 with respect to the original of the search target image, 8 × 8 pixels. The bit position of each of the 64 pixels in the area, that is, the position of the angle bit is stored as it is by OR operation, and as shown in FIG. 7, one flag is set at the bit positions corresponding to all the angles held by the 64 pixels. Maintain a standing position. As a result, the information of the edge angle bit image can be held so as not to be damaged even after the image is reduced, and the accuracy can be obtained in the search using the pattern model.

  Here, the OR operation area in which the OR operation is performed is determined according to a reduction ratio for reducing the edge angle bit image. For example, when an edge angle bit reduced image obtained by reducing an edge angle bit image to 1/8 is generated, an 8 × 8 area is an OR operation area. That is, the edge angle bit image is divided into 8 × 8 OR operation areas. In each OR operation area, the result of OR operation of the edge angle bits of all the pixels included in the OR operation area is obtained in each OR operation area. The reduced edge angle bit data is represented. A set of reduced edge angle bit data for each OR operation area obtained in this manner is an edge angle bit reduced image.

The OR operation means a sum operation, particularly a bit sum. Further, when performing an OR operation of pixels, in addition to a simple bit sum, a lower limit value may be provided for the number of pixels having edge angle bits. For example, when the number of pixels having the edge angle bit to be added is less than a predetermined pixel number threshold value, this edge angle bit is ignored. That is, the edge angle bits that are extremely smaller than the others, such as one to several pixels to be added, may be noise or error. By ignoring, it is possible to generate a highly reliable edge angle bit reduced image based only on the angle bits that are considered to be highly reliable.
(Other OR operations, saturation addition)

Furthermore, operations other than the above sum operation can be used as the OR operation. Here, saturation addition will be described as an example. For example, when performing reduction processing and expansion processing by OR operation, OR operation is performed on pixel data of nxn edge angle bit images. An extension of this calculation is “saturation addition is performed on the bits corresponding to each angle, and the next representative value is determined from the result”. Here, the saturation addition refers to an addition process in which an upper limit is provided for the addition result, and when the upper limit is exceeded after normal addition, clipping is performed to the upper limit value. For example, if the upper limit is 100,
10 + 89 = 99
11 + 89 = 100
12 + 89 = 100
10 + 100 = 100
For example, the upper limit value is limited to 100.

  Next, a specific example of reduction processing using saturation addition will be described with reference to FIG. Here, FIG. 52A is an edge angle image in which each pixel has an edge angle, and FIG. 52B is an edge angle section for expressing each pixel constituting the edge angle image by 8-bit edge angle bit data. The schematic diagram which shows is shown, respectively. Note that the edge angle division in which the angles are divided in order to express the edge angle direction with the edge angle bits will be described later with reference to FIG. In the example shown in FIG. 52 (b), E2.5, E, SE, S, SW, W, NW, N, NE are divided into 8 equal parts with a shift of 22.5 ° from the horizontal or vertical direction, and 45 ° clockwise from the right. The edge angle sections are labeled, and edge angle bit labels are assigned to the respective edge angle sections as 0, 1, 2, 3, 4, 5, 6, and 7, respectively. An edge angle bit image obtained by expressing the nine pixels a to i constituting the edge angle image of FIG. 52A by edge angle bits according to the edge angle section of FIG. 52B is as follows. It has edge angle bit data.

754343210
a: 00001000
b: 10000000
c: 10000000
d: 00001000
e: 10000000
f: 10000000
g: 00001000
h: 00000100
i: 10000101

Then, in the edge angle bit image in the new reduced image, all the bits a to i corresponding to the respective angles are saturated and added. Here, the upper limit of the saturation addition is set to 3. For example, the reduced edge angle bit data e ′ at the position e is e ′ = a + b + c + d + e + f + g + h + i
The result is as follows.

77665554433221100
e ′: 1100000011100001; binary display e ′: 3 0 0 0 3 2 0 1; decimal display

  A characteristic of the saturation addition process appears in the calculation result. That is, all 3 or more are clipped to 3. When the result of the sum is equal to or greater than a predetermined clipping threshold, the addition for clipping at that value is called saturation addition. The reduced edge angle bit data e 'can be used as it is in the search process. For example, reduced edge angle bit data e ″ in which only two or more are set to 1 and the others are set to 0 can be expressed as follows and used for the search.

754343210
e ″: 1000100; binary display (pattern model X for coarse search means)

The pattern model construction means 70 creates two types of pattern models: a coarse search pattern model X used by a coarse search means 71 described later and a fine positioning pattern model Y used by a fine positioning means 76 ( FIG. 8). As the coarse search pattern model X, a base point is determined on each segment based on a predetermined condition, and an angle in which the direction of the edge is set in the normal direction of the segment at each base point is set. .
(Pattern model Y for precision positioning)

On the other hand, as the fine positioning pattern model Y, in addition to the information of the above-described coarse search pattern model X, each base point is a segment type parameter (for example, a line segment or an arc) representing each corresponding segment as data. Is a kind of segment normal extended from the above-mentioned base points in the normal direction of the segment at each base point and in the direction of the edge, and further having a predetermined length. A line segment having a line segment (hereinafter referred to as “corresponding point search line”) is generated. The corresponding point search line is extended back and forth in the normal direction around the base point. In addition, it is preferable that at least one base point is set on each segment. In other words, a segment for which no base point is set need not be included in the pattern model.
(Coarse search means 71)

  The search during operation is performed by the coarse search means 71 and the fine positioning means 76 using a dedicated pattern model created at the time of registration. The coarse search means 71 is means for performing a coarse search during operation. The coarse search can be performed only once, or multiple times with different reduction ratios. When performing multiple coarse searches, the second coarse search should be based on more detailed data that is closer to the original size, such as reducing the reduction rate compared to the first coarse search, that is, increasing the resolution. Is preferred. In the second coarse search, it is preferable to narrow the scanning range based on the first coarse search result. Here, an outline of the search during operation is shown in the flowchart of FIG. 9 and the schematic diagram of FIG. In the present embodiment, as shown in step S901 in FIG. 9, the first coarse search (global search) for searching the entire area of the search target image using the second pattern model reduced at the global reduction rate. I do. After that, as shown in step S902, the “detection candidate” region obtained by the first coarse search and its peripheral region are converted into search target images having an intermediate reduction rate lower than that of the first coarse search. A second coarse search (local search) is performed. Further, in step S903, precise positioning is performed by the precision positioning means 76 using the pattern model Y for precise positioning.

It should be noted that the order in which the pattern model is created at the time of registration and the order in which the search is executed at the time of operation do not necessarily match. For example, at the time of registration, a pattern model having a low reduction rate (close to the original size) is created in order (first pattern model → second pattern model, etc.). Thereby, the situation where detailed information is lost as the image is reduced can be greatly reduced. In contrast, during operation, the search is performed in order from the pattern model having a high reduction ratio (low resolution, high compression ratio, etc.) (first coarse search → second coarse search). This makes it possible to search efficiently with a course-to-fine approach. As a result, during operation, the first coarse search is performed using the second pattern model, and then the second coarse search is performed using the first pattern model.
(First coarse search)

Next, an outline of the coarse search performed by the coarse search means 71 will be described. First, in the first coarse search, a rough search is performed on the entire area of the search target image using the registered pattern model, and a rough position, that is, a detection candidate is extracted. Here, the pattern model generated from the registered image having the same reduction ratio is scanned with respect to the edge angle bit reduced image of the search target image generated by the edge angle bit image reducing means 78 described above. Specifically, for example, an attitude having a specific rotation angle from the upper left of the edge angle bit reduced image in order to scan the entire area of the edge angle bit reduced image reduced to 1/8 times from the original size. The pattern model set to is scanned in the lower right direction. As a result, a detection candidate region similar to the pattern model is specified for the entire region of the edge angle bit reduced image. In this scan, the same scan is executed using all of a plurality of different rotational postures set individually for the pattern model. That is, the scan with the rotation angle changed is repeated a plurality of times. As a result, all detection candidate regions similar to the pattern model are extracted as matching candidates for the pattern model. FIG. 11 shows how the edge angle bits change as a result of rotating the pattern model. In this example, the change (b) → (d) of the edge angle bit when the global search pattern model (a) is rotated 60 ° clockwise (c) is shown. In addition, each of the detection candidates calculates an evaluation value (score value) indicating the degree of similarity, and a candidate having a score value higher than a certain threshold is extracted as a detection candidate. Each detection candidate has information on its position and orientation, that is, XY coordinates of the pattern model, angle θ, and score. Details of the score will be described later.
(Second coarse search)

Next, the coarse search means 71 performs a second coarse search based on the result of the first coarse search. The second coarse search uses a reduction rate that is smaller than the reduction rate of the search target image used in the first coarse search described above, in other words, that has a large amount of information. For example, by using an edge angle bit reduced image having a reduction ratio of 1/4, all pattern models extracted from the edge angle bit reduced image having a reduction ratio of 1/8 used in the first coarse search are used. Only for the peripheral areas of similar detection candidates, the detection candidates are narrowed down using the pattern model of the same reduction ratio. Since the second coarse search is a search performed using a part of the edge angle bit reduced image, the detection candidates can be narrowed down efficiently. Thus, efficient search is possible by performing a plurality of stages of coarse search. That is, the entire area is scanned by the first coarse search, a rough position is specified, and then the second coarse search is performed on the image with the reduced reduction ratio, the specified region (detection candidate or “targetness”) or Perform in the vicinity. Such multi-stage coarse search can maintain both search accuracy and shorten search time. The local search can be performed a plurality of times according to the reduction rate.
(Precise positioning means 76)

  As described above, after obtaining the detection candidate by the coarse search means 71 or the “more likely detection candidate” obtained by further coarse search in the vicinity thereof, the fine positioning means 76 performs the fine positioning. In precise positioning, a search is performed using an original size image that has not been reduced, or an image with a reduction rate close to the original size that is lower than the reduction rate used in the coarse search. Here, the original size image is also handled as a reduced image with a reduction ratio of 1. In the coarse search, the search target image is an edge angle bit image, whereas in the fine positioning, the original image or a reduced image thereof is used.

  The precise positioning means 76 is arranged so that the corresponding point search line of the precise positioning pattern model Y is superimposed on the search target image with respect to the detection candidate region similar to the pattern model obtained on the search target image. . More specifically, the search target is searched with respect to the detection candidate region similar to the pattern model, which is narrowed down by searching the edge angle bit reduced images having two different reduction ratios and the pattern model using the coarse search means 71 described above. Search means for performing a final positioning by performing a search using a pattern model having the same magnification as the image.

  The pattern model used in the coarse search means 71 and the fine positioning means 76 described above is not an image (hereinafter referred to as “image data”) itself as data used for image processing, but at each edge point obtained from the image data. The data is a one-dimensional list of at least the corresponding X coordinate position, Y coordinate position, and edge angle. The “pattern model” is composed of such data, and the processing speed can be increased by using the pattern model composed of the list data instead of the image data. On the other hand, the image data is so-called image data. For example, if the data size is 640 × 480 bits, a value for each coordinate position must be held for each pixel on the entire surface. On the other hand, when the pattern model is used, the pattern model can be composed of only the position and angle corresponding to the edge portion of the edge image. For this reason, the data amount is relatively small, and the required processing amount can be reduced. Therefore, using a pattern model instead of image data enables faster processing.

  In this specification, the registered image is the original image to be searched. On the other hand, the pattern model is the above listed data suitable for searching for searching for the same image as the registered image from the search target images. In this embodiment, the pattern model has information on the XY coordinates and the angle θ of each point constituting the pattern model. As will be described later, the data constituting the pattern model in the precise positioning includes information on the corresponding point search line described later in addition to the information on the XY coordinates and the angle θ of each point.

  Each member shown in FIG. 1 can be implemented by integrating a plurality of members or dividing one function into individual members. For example, an image reducing unit 77 that reduces an image may be integrated with the contour extracting unit 62. Each function of the calculation means 6 can be processed by a single CPU, LSI, or the like. For example, the functions may be distributed to a dedicated FPGA that performs preprocessing and a dedicated DSP that performs image processing. . In this image processing apparatus 100, image processing is executed by an image processing unit configured by a DSP or the like, and display of image images and search results is processed by a CPU. In this way, the processing is speeded up by performing distributed processing of each function. The member that performs each function can be arbitrarily configured.

  The image input unit 1 obtains an input image that is captured and created by an external device and is a target of image processing. In addition to being acquired from an external device by communication or I / O, the input image data can also be input in the form of a data file via a recording medium. Further, the image processing apparatus itself may have a function of capturing an input image. In this case, the image input unit 1 functions as an imaging unit or an image generation unit. When a camera using a solid-state image sensor such as a CCD or CMOS is used as an imaging means and a workpiece such as an electronic component to be image-processed is imaged, the reflected light of the light irradiated on the workpiece and the light irradiated on the background Since the amount of reflected light is different, there is a difference in the charge amount of the solid-state imaging device between the portion corresponding to the workpiece and the portion corresponding to the background. That is, since a difference in brightness occurs between the work and the background, this brightness difference can be detected as the contour or edge of the work. It is also possible to input the contour data of the registered image as the CAD data of the workpiece. The data input from the image input unit 1 in this way is A / D converted as necessary and sent to the image processing apparatus main body.

  The storage unit 4 holds data such as a parameter file storing various data necessary for calculations such as various conversions and template matching, a pattern model of a registered image, and an edge image for a search target image. As described above, the storage unit 4 can be used as an area for storing input image data in addition to holding the setting contents. As the storage means 4, a semiconductor element such as a DRAM or a flash memory, or a fixed storage means such as a hard disk can be used.

  The operation unit 2 is an input device for operating the image processing apparatus 100. For example, when the user manually designates a processing area by operating the mouse 81 and the keyboard 82, the input device functions as the processing area designation unit 5. On the other hand, the processing area can be automatically specified by calculating with the calculating means 6 based on the image processing on the image processing apparatus 100 side.

  The input device is connected to the image processing apparatus 100 in a wired or wireless manner, or is fixed. Examples of general input devices include various pointing devices such as a mouse, keyboard, slide pad, track point, tablet, joystick, console, jog dial, digitizer, light pen, numeric keypad, touch pad, and accu point. Further, these input devices operate the contour extraction program in a mode in which a computer in which the contour extraction program is installed is connected to the image processing apparatus 100, or in a mode in which the computer in which the contour extraction program is installed is used as the image processing device or the contour extraction device. In addition, it can be used to operate the image processing apparatus itself and its peripheral devices. Furthermore, using a touch screen or touch panel on the display itself that displays the interface screen, the user can directly input or operate the screen by hand, or use voice input or other existing input means, Or these can also be used together. In the example of FIG. 1, the input device is configured by a pointing device such as a mouse or a keyboard.

  As the display means 3, an external display such as a liquid crystal monitor or a CRT monitor can be used. If a type having an input function such as a touch panel is used, the display means and the operation means can be used together. The display means 3 can be incorporated in the image processing apparatus itself in addition to the externally connected form.

The above configuration is an example. For example, the image processing apparatus itself may include a display unit, an operation unit, and the like. Each member may be shared by a single member or may be integrated with the calculation unit 6. . Hereinafter, an example in which an edge extraction program is installed in a general-purpose computer and edge connection processing and contour extraction processing are executed will be described.
(Detailed procedure for image processing)

  This image processing apparatus performs preprocessing (expansion, contraction, smoothing, Sobel, etc.) on the registered image (also referred to as a standard image or reference image) acquired by the image input means 1 and the search target image. Then, an edge is extracted as feature extraction. Then, an edge-based pattern search is performed during operation using the pattern model obtained from the registered image. In the present embodiment, as described above, a pattern model is registered in advance (FIG. 3), and the search target image is processed during actual driving (FIG. 4). In this way, the processing can be speeded up by distributing the processing at the time of registration and operation.

The operation at the time of registration will be described in more detail. By processing the registered image by the above-described contour extracting means 62, the contour portion of the registered image is extracted as an edge and expressed by a set of points having a width of about 1 pixel. A registered image edge image is generated. The registered image edge image is temporarily held in the registered image edge image memory of the storage unit 4. Furthermore, a pattern model is generated by the segment creation means 68 and the pattern model construction means 70 from the registered image edge image. The pattern model used for the search is held in the pattern model memory of the storage means 4 and is called up as necessary.
(Create pattern model at registration)

  FIG. 12 shows a flowchart of a pattern model creation procedure at the time of registration. In creating the pattern model, as described above, the master image of the target image to be extracted from the search target image is designated as a registered image and temporarily stored in the storage unit 4. Therefore, a pattern window is set for the registered image (step S1201). After determining the reduction ratio for this registered image (step S1202), the image reduction means 77 attempts to reduce the image (step S1203). Further, a contour is extracted from the reduced image. Specifically, edge extraction processing and chaining processing are performed (step S1204). Further, segmentation for connecting the chains is performed (step S1205). As described above, the edge data is segmented by the contour extracting unit 62, the chain creating unit 63, and the segment creating unit 68 on the reduced image. Then, a pattern model is generated by the pattern model construction means 70 (step S1206).

As described above, in the present embodiment, two types of pattern models, ie, a rough search pattern model X and a fine positioning pattern model Y are generated as pattern models. The data structure of the pattern model for coarse search is composed of the coordinate position and edge angle of each edge point with respect to the arbitrarily set origin, and the data structure of the pattern model for fine positioning Is composed of coordinate positions in the X direction and Y direction of each edge point with respect to an arbitrarily set origin, edge angles, and corresponding point search lines described later.
(Edge angle)

The edge angle indicates an edge density gradient direction at an edge point. Here, in order to indicate the edge angle, 0 ° to 360 ° are expressed in 256 levels.
(Edge strength)

Note that the edge strength of each edge point is composed only of edge points having a strength value larger than a predetermined strength value in the present embodiment, and therefore does not have that value as data. However, the present invention is not limited to this method. For example, in the search algorithm, when evaluation or weighting by score calculation to be described later is performed based on the similarity of the edge strength values, the data structure of the pattern model is changed. Data of edge strength may be held as a value.
(Determination of image reduction ratio)

  In addition, when using a reduced image at the time of registration, the selection of the reduction rate is very important because the reduction rate affects the operation. Here, while maintaining the feature points of the image necessary for the search, noise is deleted, and an appropriate reduction ratio that can shorten the search time is set. For example, in the balance between search accuracy and search time, when the search time is sacrificed to some extent and the search accuracy is emphasized, a specific reduction rate with a relatively low reduction rate is set. Alternatively, an optimum reduction ratio determined by trial and error by the user can be used. In the present embodiment, it is possible to switch between an auto mode in which the reduction ratio is automatically determined and a manual mode in which the user designates the reduction ratio. Here, in the auto mode, the reduction ratio is determined based on the length of the shorter side defining the rectangular area of the pattern window PM. In the manual mode, if the user selects the optimum value while visually checking how the pattern model is actually deformed according to the reduction ratio, the setting operation can be facilitated as a sensation. In the example of the user interface in FIG. 13, when the manual mode is selected, a reduction ratio can be selected from a drop-down list, and an image shows how an image is deformed according to the selected reduction ratio. As the reduction ratio is increased, the corners of the image become distorted and the deformation becomes more severe. The user can select an appropriate reduction ratio according to the application and purpose while referring to such a change in image.

Alternatively, as another embodiment, a method of automatically determining the balance between search accuracy and search time may be employed. For example, a search accuracy, a search time, or both are presented as choices as items that are emphasized by the user, and after the user makes a selection, an appropriate setting is automatically performed according to the selection.
(Coarse search pattern model X)

  The coarse search is a search for efficiently narrowing down an area in which there is a high possibility that the same image as the registered image exists in the search target image as a detection candidate prior to precise positioning described later. For this reason, the rough search achieves the above-described object by utilizing an original search target image or an image reduced from a registered image. More specifically, as in the pyramid search shown in FIG. 87, a coarse search is performed using a reduced image obtained by reducing an original size image. In this rough search, a low-resolution image with a high reduction ratio is used. Then, a precise value is determined for the coarse position (detection candidate) obtained by the coarse search. In this precise positioning, an image with a high resolution and a lower reduction rate than that of the coarse search is used.

  Here, the procedure for creating the rough search pattern model X will be described. In step S1201, the master image of the target image to be detected from the search target image is temporarily stored in the registered image memory of the storage unit 4 as a registered image. More specifically, the position and size of a necessary part as a registered image is used for the image captured from the imaging unit and displayed on the display unit, using a rectangular pattern window PW as shown in FIG. The user makes settings on the screen.

  Next, a reduction ratio when the image is reduced by the image reduction unit 77 is determined for the registered image (step S1202). In the present embodiment, the global reduction rate of the pattern model to be described later is determined by the size of the pattern window PW, that is, the number of pixels included in the rectangle.

  That is, when the number of pixels as a registered image is relatively large, even if the reduction ratio is set high, the loss of feature points in the image is small, so the reduction ratio is set high to some extent. On the other hand, when the number of pixels as a registered image is relatively small, if the reduction ratio is high, there is a high possibility that feature points in the image will be lost. Therefore, the reduction ratio is set low to some extent. The optimal reduction ratio is preferably set to such a degree that noise in the image is removed by the reduction and the feature points of the image are not lost.

  As another method for determining the reduction ratio, for example, a registered image is shifted by a predetermined amount in the X direction and the Y direction with respect to the same registered image. Here, when the autocorrelation changes slowly, it can be determined that the correlation value is difficult to change if matching is performed to some extent, and therefore the reduction ratio is increased. On the other hand, when the autocorrelation changes abruptly, it can be determined that the image has a characteristic that the correlation value is likely to change, so the reduction rate is kept low. Thus, the reduction rate can be determined based on the autocorrelation.

  Based on the reduction ratio determined as described above, the registered image of the original size is reduced by the image reduction means 77 (step S1203). In detail, the image reduction means 77 reduces the image at an intermediate reduction ratio located between the original size of the image and the global reduction ratio, not the final reduction ratio determined by the size of the pattern window PW. Is executed. Here, the image is reduced by the image reduction unit 77 at the intermediate reduction rate, and the edge extraction unit 62, the chain generation unit 63, and the segment generation unit 68 further perform edge segmentation (steps S1204 and S1205). ). Note that the intermediate reduction rate corresponds to the reduction rate in the second coarse search.

Here, after reducing the original size image first, the reason for the edge data conversion and segmentation of the image is compared to the procedure for reducing the size of the original size image after converting it to edge data and segmentation. This is because noise can be reduced while preserving the feature points of the original image. This method also matches the purpose of the coarse search for extracting detection candidate regions similar to the pattern model. For this reason, the intermediate reduction ratio can be set manually so that the user can manually determine the segment while looking at the segment so that the noise can be reduced to some extent while retaining the feature points of the original size image. Or it can also be set as the automatic setting determined automatically according to the size of the pattern window PW.
(Create first pattern model)

In step S1206, the pattern model construction unit 70 uses the data segmented by the segment creation unit 68 to generate the first pattern model at the intermediate reduction rate described above. This first pattern model is for local search in the coarse search (step S902 in FIG. 9). FIG. 8 shows an example of each pattern model. As shown in FIG. 8A, the first pattern model determines a base point on each segment existing in the edge image data based on a predetermined condition, and further, a segment for each base point. The edge model point is defined in which the edge angle is set in the normal direction and the edge direction.
(Create second pattern model)

  Further, the pattern model construction unit 70 uses the data segmented by the segment creation unit 68 to generate the second pattern model at the above-described global reduction rate (FIG. 8B). As in the first pattern model, the second pattern model for coarse search determines a base point on each segment existing in the edge image data based on a predetermined condition, and for each base point, Thus, the edge angle is set in the normal direction of the segment and in the direction of the edge. Even if these pattern models are not created individually according to the reduction ratio, only one pattern model can be created, and during operation, the pattern model can be enlarged and reduced according to the reduction ratio of the search target image.

  The difference between the first pattern model and the second pattern model is that when there are a plurality of base points set on each segment, the distance between the base points is as shown in FIGS. 8 (a) and 8 (b). Compared to the first pattern model, the second pattern model is longer. This is because the two pattern models have different reduction rates. The degree of difference in the distance between the base points is governed by the difference between the intermediate reduction rate and the global reduction rate.

  Here, as shown in FIG. 9, the specific coarse search is performed on the entire range of the search target image using the second pattern model of the global reduction ratio (step S901). ), And “local search” (step S902) performed using the first pattern model of the intermediate reduction rate only for the candidate region of the detection candidate extracted by this “global search”.

In the description of the present embodiment, the coarse search is performed by one “global search” using one global reduction rate pattern model and one “local search” using one intermediate reduction rate pattern model. “Search”. Specifically, the intermediate reduction ratio with respect to the original size of the image is
(1) If 1 / √2 or less, the global reduction rate is ½ times the intermediate reduction rate,
(2) In the case of 1 / √2 times to 1/4 times, the global reduction rate is 1/3 times the intermediate reduction rate, and (3) in the case of 1/4 or more, the global reduction rate is set to the intermediate reduction rate. Each is set to 1/4 times. Thus, by determining the global reduction rate according to the intermediate reduction rate, both the search efficiency and the saving of the feature points from the original image are achieved.

  Here, the local search can be performed a plurality of times. As in (3) above, when the intermediate reduction ratio is remarkably high with respect to the original size of the image, for example, when the intermediate reduction ratio is set to 1/4 times the original size of the image, the global reduction ratio Is 1/16 times the original size of the image. In this case, since the ratio between the global reduction rate and the intermediate reduction rate is different by a factor of four, the candidate area of the detection candidate extracted using the second pattern model of the global reduction rate and its surroundings are set as the minimum search. The “local search” to be performed may take too long.

  Therefore, in the present embodiment, when the ratio between the intermediate reduction ratio and the global reduction ratio is larger than twice, the search is automatically performed once more so that the ratio between the adjacent reduction ratios becomes two times or less. An additional intermediate reduction ratio is set so as to add more or more. That is, the time required for one local search is shortened by performing the local search twice or more. Such an automatic addition of the intermediate reduction rate (reduction rate for local search) can be set by checking the “Automatically add reduction rate for local search” option on the user interface screen shown in FIG. 5, for example.

As described above, the pattern model is a data structure including coordinate positions and edge angles in the X direction and Y direction of each edge point with respect to an arbitrarily set origin. Therefore, when an additional intermediate reduction ratio is set, the pattern model corresponding to the additional intermediate reduction ratio is generated from the segment in the same manner as the first pattern model and the second pattern model described above. The difference between the additional intermediate reduction ratio pattern model and the first pattern model or the second pattern model is that when there are a plurality of base points set on each segment, the distance between the base points is the first pattern. It is longer than that of the model and shorter than that of the second pattern model. Even if the pattern model is reduced, since the pattern model itself is composed of data such as coordinate positions, information loss due to the reduction can be reduced as compared with reduction of image data.
(Global search)

Here, for convenience of explanation, the operation during the search operation will also be described. As shown in step S901 in FIG. 9, a “global search” using a global reduction rate pattern model is executed to extract candidate areas for detection candidates. Then, a “local search” is performed on the extracted candidate areas of the detection candidates using the second pattern model having the next highest reduction ratio after the global reduction ratio. Thereby, the candidate area | region of a detection candidate with higher precision is narrowed down based on a search result. After that, using the pattern model of the intermediate reduction rate that is set in descending order of the reduction rate, a step of performing “local search” on the narrowed detection candidate region and narrowing down the detection candidate region repeat.
(Precise positioning reduction rate)
(Manual reduction rate determination mode)

As described above, in the manual reduction ratio determination mode in which the user selects the reduction ratio using the user interface of FIG. 5, the user can select one intermediate reduction ratio, one global reduction ratio, and precise positioning. The precise positioning reduction rate when using the means 76 can be selected. However, even in this case, if the ratio between the selected intermediate reduction ratio and the global reduction ratio is larger than two times, a further intermediate reduction ratio based on the above-described conditions is automatically set between these reduction ratios. You may set so that it may be generated automatically. The precision positioning reduction ratio when using the precision positioning means 76 is the lowest value among the reduction ratios set for the coarse search when the user selects a reduction ratio candidate, or even lower than this. The value to be selected is limited so that only the reduction ratio value (including the original size of the image) can be selected. As a result, it is possible to avoid a situation in which precise positioning is performed with coarse data having a higher reduction ratio than the previous coarse search.
(Pattern model for precise positioning)

As shown in step S903 in FIG. 9, the fine positioning search is performed at the last in the “local search” with respect to one or a plurality of detection candidate candidate areas narrowed down by the “local search” of the coarse search described above. Precise positioning is performed using the pattern model of the final intermediate reduction ratio used or a reduction ratio lower than this (including the original size of the image). Preferably, the reduction ratio of the pattern model used for precise positioning is an equal-size image, that is, an original image of an original size.
(Edge sharpness)

  As described above, the original image search target image is not necessarily used for precise positioning, but reduced at a reduction rate (precise positioning reduction rate) that does not exceed the final intermediate reduction rate used last in the previous local search. Images can also be used. As a result, a suitable search result can be expected even when the search target image is blurred.

  For example, it can be said that the sharper the waveform of the luminance data of the edge image of the original size, the higher the sharpness, and vice versa. Therefore, when the sharpness of the edge portion is lower than a predetermined value, that is, when the edge is spread over a predetermined width in the width direction and the image is blurred, an appropriate precision positioning reduction ratio is set. By reducing the image to this reduction ratio, the sharpness of the image can be reduced and the edge sharpness can be increased. As a result, stable positioning accuracy can be obtained.

For example, in the case of a binary image with sharp pixel changes as shown in FIG. 53, the contour or edge of the image is a so-called step edge in which the pixel density or pixel value changes stepwise as shown in FIG. is there. For this reason, as shown in FIG. 55, the edge intensity changes tend to appear sharply, and accurate positioning can be expected. On the other hand, when the binary image is unclear, the change in the boundary portion becomes gentle as shown in FIG. 56, and as a result, the edge intensity change becomes an uneven curve as shown in FIG. As described above, there is a problem that the edge detection accuracy is affected by slight fluctuations in the surrounding environment, and stable edge detection cannot be performed, and the reliability of image processing such as image recognition is reduced. Therefore, in this embodiment, the sharpness of the edge is improved by reducing to an appropriate positioning reduction rate.
(Determine edge sub-pixel coordinates)

  Specifically, in precise positioning using edge information, highly accurate positioning is performed based on a pattern model generated based on edge position information extracted from a registered image and edge position information extracted from a search target image. . For this reason, edge position information is extremely important.

Conventionally, as a method for determining the edge sub-pixel position, techniques such as Patent Documents 2 and 3 are known. In these methods, sub-pixel coordinates are obtained using quadratic function interpolation using a total of three pieces of data, that is, the edge intensity of the target pixel and the edge intensity of two pixels located around the target pixel. FIG. 58 shows an outline of a subpixel coordinate calculation method according to Patent Document 2. In this figure, EMc (Edge Magnitude center) is an edge intensity value of the target edge point that is an edge point remaining after non-maximal point suppression processing (nonmaxsupress) for edge thinning. Further, EMf (Edge Magnitude forward) is an estimated edge strength value at an arrow in the direction of the edge angle of the target edge point. Further, EMb (Edge Magnitude backward) is an estimated edge intensity value at a circle in the direction of the edge angle of the target edge point. Furthermore, the subscript ₁ of EMf ₁ and EMb ₁ is given to the feature quantity of the edge point in the cross direction, and the subscript ₂ of EMf ₂ and EMb 2 is given to the feature quantity of the edge point in the diagonal direction. . In the following example, an angle EAc <45 ° between the edge angle and the horizontal direction will be considered. Other angles can be obtained in consideration of symmetry. Here, the following equation holds.

EMf = EMf ₁ * (1-tanEAc) + EMf ₂ * tanEAc
EMb = EMb ₁ * (1-tanEAc) + EMb ₂ * tanEAc

  Using the three edge intensity data of EMc, EMf, and EMb in the above equation, the offset amount of the subpixel position from the EMc position can be calculated by the following equation.

x = (EMf-EMb) / (2 (2EMc-EMf-EMb))
y = x * tan θ

  As described above, the subpixel position can be calculated. Next, FIGS. 55 and 57 show the state of the focused edge strength and the state of the edge strength in a blurred image. As is clear from FIG. 55, the edge strength that is in focus to some extent has a sharp peak, and the edge position can be clearly determined. On the other hand, in the case of an unclear and blurry image as shown in FIG. 57, the vicinity of the maximum is in a considerably flat state, and the influence on the subpixel coordinates due to the error of the edge strength is increased. In order to evaluate the influence, the case where EMf = EMb and EMf has one gradation is considered. The relationship between edge strength error and subpixel position is shown in the following equation.

  In the above function, the situation near x = a + 1 is examined as follows.

  In the above equation, if X = c−a,

It becomes. This can be expressed as a graph as shown in FIG. As shown in this figure, when the edge strength becomes flat in the vicinity of EMf = EMb, X → 0, that is, c → a, the influence of the edge strength error on the edge position error increases. . A similar tendency can be seen even when EMf ≠ EMb. Therefore, in this embodiment, the sharpness of the edge point is calculated, and the image data reduction ratio for calculating the edge sub-pixel position is calculated using this value. Further, edge extraction is performed using the image data reduced to the image data reduction rate. By reducing the image data in this way, the gentle waveform as shown in FIG. 57 becomes a sharp waveform as shown in FIG. 55, the edge position is stabilized, and the positioning accuracy can be improved.

The procedure for determining the image data reduction ratio based on the sharpness of the edge point will be described based on the flowchart of FIG. First, in step S6001, an edge image is created from the same size image. In step S6002, the edge of the edge image is thinned. In step S6003, the sharpness of each edge point of the edge image is calculated. Edge thinning processing is performed for non-maximum point suppression processing prior to edge connection processing. In step S6004, the average value of the sharpness of the thinned target edge point is calculated. In step S6005, the reduction rate of the image data is determined based on the average value of the sharpness. Here, the image data reduction ratio is determined so that the position accuracy of the edge point is maintained at a predetermined level or higher.
(Edge model function)

Next, the model function of the edge is examined. It is assumed that many of the extracted edges are step-like edges and can be expressed by the following edge model function. The edge model function σ _s is defined as the sharpness of the edge point. In this case, the ideal type of edge can be expressed by the following equation.

A graph of the above function is shown in FIG. The ideal shape (profile) of the edge shown in this figure is plotted by the following equation (when I = 2, l = 0, and σ _s = 0.6 along the x-axis):

(Pattern search procedure using image data reduction ratio)

Next, a pattern search procedure in specific image processing using the image data reduction ratio will be described with reference to the flowcharts of FIGS. In these drawings, FIG. 62 shows an operation at the time of registration, and FIG. 63 shows an operation at the time of operation.
(Operations during registration with image data reduction ratio applied)

First, the operation at the time of registration will be described with reference to FIG. 62. In step S6201, a pattern model for rough search is created. In step S6202, edge extraction is performed on the same-size image, and a standard deviation σ _s is obtained for each extracted edge point. When obtaining the standard deviation σ _s , it is assumed that the shape of the edge actually extracted from the image data approximates the model shown in FIG. Here, in order to calculate the sub-pixel position of each edge at an edge point equal to or higher than a predetermined edge strength threshold, the edge strengths EMb, EMc, EMf of three adjacent points B, C, F shown in FIG. 64 are used. The following expression which is an approximate expression of the second derivative of the logarithmic edge strength is used.

The standard deviation σ _s can be calculated from the following equation using t in the above equation.

If t in the above equation is used, it is important that the standard deviation σ _s can be expressed by a monovalent function, and the above equation does not have a special meaning.

On the other hand, in step S6203, the edge extracted from the image data is chained and segmented by the chain creation means 63 and the segment creation means 68 to create a temporary pattern model. Further, in step S6204, an average value σ _a using only σ _{s of} each edge used when creating the temporary pattern model is calculated. In step S6205, using this average value σ _a , the image data reduction ratio r is obtained by the following equation. Here, the precision positioning reduction rate is obtained.

  In this example, the logarithmic difference value of the edge strength is obtained, and the reduction degree is calculated from the difference value. However, the present invention is not limited to this example, and the image data reduction ratio can also be calculated using various approximate values related to the logarithmic differential value of the edge strength. The approximate value here includes an approximate value of the differential value of the edge strength.

When the image data reduction ratio r is obtained in this way, in step S6206, the registered image is reduced again according to the precision positioning reduction ratio, and in step S6207, a pattern model for precise positioning is created from the reduced registration image. To do.
(Operation during operation using image data reduction ratio)

Next, the operation at the time of operation performed in response to the registration operation using the image data reduction ratio will be described based on FIG. First, in step S6301, the search target image is reduced to a reduced search target image for precise positioning using the fine positioning reduction ratio obtained above. On the other hand, in step S6302, a reduced image for coarse search is created from the search target image, and in step S6303, the position and orientation of the detection candidate are obtained using the coarse search pattern model and the reduced image for coarse search. Finally, in step S6304, precise positioning is executed using the precision positioning pattern model, the precision positioning reduction search target image, and the position and orientation of the detection candidate.
(Image reduction pre-processing and post-processing)

  Note that the image data is reduced so that the edge position information is maintained as much as possible even after the image data is reduced. Specifically, sub-sampling is performed after applying a low-pass filter corresponding to the image data reduction rate. By performing edge extraction on image data after sub-sampling, it is possible to obtain accurate edge position information.

  As described above, it is possible to suppress a decrease in accuracy caused by unclearness such as blurring of the original image. Further, by reducing the image data to reduce the amount of data, there is also a secondary advantage that the subsequent processing can be performed at high speed and light load.

  In this embodiment, the precise positioning reduction ratio is automatically set to a reduction ratio whose upper limit is the final intermediate reduction ratio (first reduction ratio) used last in the “local search”. The pattern model used for precise positioning is generated from the registered image reduced so as to be the same as the precision positioning reduction rate. Thus, the precise positioning reduction ratio is determined in the range from the original size of the image to the final intermediate reduction ratio used last in the local search based on the edge sharpness of the edge image of the original size.

By adjusting the precision positioning reduction ratio in this way, it is possible to maintain the sharpness at a certain level or more and to ensure the stability of the positioning accuracy by using the sharpness in a high state.
(Operation when registering a pattern model)

  Returning to the description of the pattern model registration operation again, the procedure for registering the precision positioning pattern model will be described with reference to the flowchart of FIG. The fine positioning pattern model is created using the same image as the registered image described in the above-described coarse search pattern model. First, in step S1401, prior to processing by the image reducing unit 77, the edge extracting unit 62 generates an original size edge image for the registered image, and evaluates the sharpness of the edge. Based on this, an optimum reduction ratio is determined in step S1402. Based on the optimum precise positioning reduction ratio determined for the registered image in this way, the image is reduced by the image reduction means 77 in step S1403. Further, edge segmentation is performed on the image reduced at the reduction ratio determined by the image reduction means 77 (including a reduction ratio of 1) by the contour extraction means 62, the chain creation means 63, and the segment creation means 68. (Steps S1404 and S1405). Specifically, an edge extraction process for extracting an edge and a chaining process for creating a chain from edge points are performed on the reduced image, and a segmentation for connecting the chains is further performed. Next, the pattern model construction unit 70 uses the data segmented by the segment creation unit 68 to generate a precise positioning pattern model at the determined precise positioning reduction rate (step S1406).

Further, the fine positioning pattern model determines a base point on each segment existing in the edge image data based on a predetermined condition, similarly to the above-described coarse search pattern model (step S1407). Further, an angle is set for each base point (step S1408). Here, an angle is set in the normal direction of the segment and in the direction of the edge. Furthermore, for each base point, the type of the segment where the base point is provided (segment type such as a line segment or arc), a parameter that represents the segment (a parameter that can define a segment consisting of a line segment or arc), and the normal of the segment Line segment information having a predetermined angle in the normal direction of the segment, that is, an angle close to the edge angle and corresponding segment search line is set (step S1409).
(Line length of corresponding point search line)

  Here, the line length of the corresponding point search line given to each base point is set to the same length for each base point. The length is determined by the ratio between the final intermediate reduction ratio used last in the local search and the fine positioning reduction ratio used in the fine positioning. In other words, if the ratio between the final intermediate reduction ratio and the precision positioning reduction ratio is large, the length of the line segment is long, and if the ratio is small, the length of the line segment is short.

For example, if the final intermediate reduction ratio used last in the local search is 1/4 times the original size of the image, and the fine positioning reduction ratio is the same magnification, the final intermediate reduction ratio and the fine positioning Since the ratio with the reduction ratio is four times, one pixel in the local search corresponds to four pixels in the fine positioning. Therefore, the length of the line segment in the precision positioning pattern model is set to a length that covers 4 pixels in the positive and negative directions of the edge from the base point. However, since the length of this line segment affects positioning accuracy and search time, it is not always necessary to cover all the corresponding number of pixels by the ratio of the reduction ratio. For example, the line length of the corresponding point search line is set short according to the required processing time. Or conversely, it may have a length longer than the corresponding number of pixels. For example, by setting a margin to the line segment length corresponding to the reduction ratio, further processing stability can be achieved.
(Change in segment length of corresponding point search line)

In addition, the length of the corresponding point search line can be changed so that it is not equal in the front and rear directions with respect to the base point, and one of them is lengthened or shortened. This process is performed by the pattern model construction means 70 and the like. Here, an example in which the length of the corresponding point search line is changed will be described with reference to FIGS. In these figures, FIG. 27 shows the case where the length from the base point is equal, and FIG. 28 shows the case where the length from the base point is different. In these drawings, the corresponding point search line generated in the inner rectangular area is filtered. As shown in FIG. 27, if the length of the corresponding point search line extended from the base point is constant before and after the base point, it overlaps with the inner rectangular shape, which may cause an erroneous determination. Therefore, if the corresponding point search line is set only in the outward direction as shown in FIG. 28 without extending the corresponding point search line in the inward direction, an accurate search result with fewer erroneous determinations can be expected. .
(Set interval of corresponding point search line)

  The corresponding point search line is set on the segment except for the end. This is because such an end has a great influence when the position is shifted. Therefore, stable processing is expected by setting a corresponding point search line by excluding a portion having a large influence when the position is shifted.

  The interval and the number of corresponding point search lines are determined according to the required pattern search processing speed and accuracy. Pattern search accuracy can be maintained by setting at least one corresponding line search line for each line segment or arc constituting the segment. In the simplest case, one base point is arranged at the center on the segment, and base points are set at equal intervals on the segment from that point. In addition, the accuracy can be improved by thinning out the base point setting for a portion of the segment where the edge angle is ambiguous and setting the base point densely in a portion that is reliably detected.

Further, it is preferable that at least one corresponding point search line is always assigned to the center of the segment. As a result, one of the segments constituting a part of the pattern model is guaranteed to set a corresponding point search line even if it is short.
(Preprocessing for search target images during driving)

  The operation at the time of registering the pattern model, that is, the creation of the pattern model for coarse search and fine positioning has been described (FIGS. 3 and 12). During operation, a search using these pattern models is performed (FIG. 4). In the search, a predetermined pre-process is performed on the search target image input from the imaging unit. Here, a procedure for performing pre-processing on the search target image in the search during driving will be described based on the flowchart of FIG.

  First, in step S1501, based on the input search target image, the image reduction unit 77 uses the intermediate reduction rate (first reduction rate) of the first pattern model for coarse search used for the registered image at the time of registration. To generate a reduced image.

  On the other hand, in step S1502, the edge angle / edge strength image creating means 60 of the contour extracting means 62 generates an edge angle image and an edge strength image. Further, the thinning means 61 generates a thinned edge angle image based on the edge angle image and the edge intensity image.

  In step S1503, the edge angle bit image creation unit 69 corresponds to the intermediate reduction rate of the first pattern model for coarse search based on the edge angle image generated by the contour extraction unit 62 and thinned. Create an edge angle bit image. Needless to say, the edge angle bit image generated here is applied to “local search” using the first pattern model for coarse search in the search operation.

  Further, in step S1504, the edge angle bit image reduction means 78, based on the edge angle bit image generated by the edge angle bit image creation means 69, reduces the global reduction rate of the second pattern model for “global search”. The edge angle bit reduced image corresponding to is generated.

  As described in the description of setting the intermediate reduction ratio of the coarse search, an additional pattern model is created based on the additional intermediate reduction ratio between the initial intermediate reduction ratio and the global reduction ratio. If so, as an optional step S1505, also in this pre-processing, the edge angle bit image reducing unit 78 adds an additional value based on the edge angle bit image generated by the edge angle bit image generating unit 69. An edge angle bit reduced image corresponding to the intermediate reduction rate of the pattern model is generated.

  Note that, in the above-described preprocessing during operation, the search target image is processed in the reverse order to the global search and local search of the coarse search performed during operation and the fine positioning (see FIG. 8). The order of creating the pattern model is not particularly limited, and it is needless to say that the pattern model for coarse search may be created after the pattern model for precise positioning is created first. On the other hand, during operation, a coarse search is performed using an image with a high reduction ratio, and a fine search is performed with an image close to the original size by gradually reducing the reduction ratio.

After the pre-processing during driving is completed as described above, the coarse search global search and local search are performed using the generated edge angle bit reduced image, edge angle bit image, etc., and the coordinates of the detection candidate are obtained. After that, precise positioning is performed (FIG. 9).
(Details of each operation during registration)

  The outline of the operation at the time of registration and operation has been described above. Next, details of the image processing operation at the time of registration will be described. At the time of registration, the edge angle / edge strength image creating means 60 applies a Sobel filter to the registered image, obtains the edge strength and the edge angle for each point constituting the registered image, the edge strength, the edge angle, Edge information including the edge position is calculated. Based on the edge information, thinning processing is performed to obtain edge points. As a specific example of the thinning process, an edge strength non-maximum point suppression process can be used. Here, the line shape is 1 pixel wide by thinning.

  The edge point can also be obtained with the accuracy of the subpixel position. For example, the subpixel position can be calculated using quadratic function interpolation (see, for example, Japanese Patent Laid-Open No. 7-128017).

  Further, the obtained edge points are connected to create a continuous chain. Here, the edge chaining means 64 performs an edge connection process for connecting adjacent edge points whose edge angles are substantially in the same direction, and creates a continuous line segment element (chain). The resulting chain also has xy subpixel coordinates. Each chain is a set of edge points, and a chain index is assigned to each chain as an identifier for distinguishing the chains.

  Further, the edge chain segmentation means 65 generates a segment by approximating the chain. The segment is obtained by fitting the chain with a line segment and an arc using the least square method. In the fitting, first, approximation with a line segment is performed, and when an approximation error due to the line segment exceeds a predetermined threshold, switching to fitting with an arc is performed. If the error does not decrease even when approximated by a circular arc, the fitting result of the line segment is used. In this way, the process of sequentially fitting with a combination of line segments and arcs is repeated, and when the error of the fitting result exceeds the threshold, if the data obtained up to that point is sufficiently long, the segment as a continuous line To do. Since the edge point is obtained at the subpixel position, the segment can also be obtained at a highly accurate position in the subpixel order.

Here, a segment approximating the chain with a line segment and an arc is formed. A segment of a line segment can be expressed by an equation representing a straight line (for example, ax + by + c = 0) and end point coordinates. On the other hand, arc segments can be expressed by center coordinates and radius, start angle and end angle, and the like. For example, an arc segment is represented by the center coordinates (x _o , y _o ) and the radius r _o of the arc at (x−x _o ) ₂ + (y−y _o ) ₂ = _{ro 2} . Base points are set at predetermined intervals on each segment generated in this manner.

In the above, an example of approximation by a line segment or an arc as a segment has been described. However, the present invention is not limited to this, and a conic curve, a spline curve, a Bezier curve, or the like can be used as appropriate. As a result, a pattern model can be created as a single or a combination thereof based on a certain geometric shape such as a circular shape, an elliptical shape, a triangular shape, or a rectangular shape, so that pattern search creation and subsequent processes can be simplified.
(Reduction of pattern model)

Also, the pattern model is reduced during search operation. This reduction rate is a reduction rate for reducing the search target image for rough search during operation, which will be described later. Since there is this reduction processing, the base points as the model edge points of the pattern model are set with an interval between the base points so that the same coordinates are not designated as a result of the reduction processing. As a result, the pattern model of FIG. 16 becomes as shown in FIG.
(Differences between the coarse search pattern model and the fine positioning pattern model)

  The coarse search pattern model and the fine positioning pattern model are each created from an original size registered image (or a reduced image thereof). In other words, the segment of the coarse search pattern model is not created from the segment of the fine positioning pattern model, and the two segments do not necessarily match. Further, since the pattern model for coarse search and the pattern model for fine positioning are different in size, the distance between the base points is also different. The difference in distance obtained by converting the base points to the same magnification depends on the reduction ratio.

  Further, in the coarse search pattern model, the coordinates of the base point and the direction of the edge (angle information) at the base point are given. In other words, an angle close to the edge direction is set in the normal direction of the segment not having the length information of the corresponding point search line. In this coarse search using the coarse search pattern model, the pattern model is placed on the image to be searched, and it is confirmed whether or not there is an edge at the position of the base point and whether the edge direction matches the direction of the pattern model. .

  On the other hand, in the fine positioning pattern model, in addition to the coordinates of the base point and the direction of the edge at the base point, which are the rough search pattern model, a corresponding point of a predetermined length extending through the base point in a direction substantially orthogonal to the segment. It has a search line (that is, one that defines the length of the segment normal) and a segment type (for example, an attribute such as a line segment or an arc). This difference corresponds to the processing content of each search. That is, in precise positioning, a corresponding edge is searched for within the range of the corresponding point search line. As described above, the precision positioning pattern model functions as a corresponding edge point selection unit that selects a corresponding edge point corresponding to the base point.

Note that it is not always necessary to create a segment in the contour extraction. It is also possible to set the corresponding point search line directly from the chain instead of the segment. For example, when three base points are set for a certain contour, three edge points constituting the chain corresponding to the contour are set at equal intervals, and corresponding point search lines are set in each normal direction. With this method, high-speed processing is expected as long as segments are not created, but the accuracy is somewhat lowered because the chain is not approximated by straight lines or arcs. In particular, the chain simply has the edge points connected and may be inferior in linearity. On the other hand, since the segment is approximated by a straight line or arc, a more accurate calculation result can be obtained and the positioning accuracy can be stabilized.
(Details of coarse search during operation)

Next, a description will be given of an operation at the time of an operation for actually searching for a matching part from the search target image using the pattern model registered as described above. First, details of a procedure for obtaining a rough position and posture by a rough search will be described based on the flowchart of FIG. In this embodiment, the coarse search is divided into a global search and a local search, and detection candidates are obtained.
(Step S1801 Reduction of search target image)

First, in step S1801, the search target image to be searched is reduced in accordance with the reduction ratio of the registered image. For example, the search target image shown in FIG. 19A is reduced to the same magnification as the registered image to obtain a reduced search target image shown in FIG. Here, the image is first reduced to an intermediate reduction ratio for coarse search. In other words, the reduction to the global reduction rate with a large reduction rate is not performed first, but the reduction is first performed to an intermediate reduction rate with a small reduction rate.
(Step S1802 Acquisition of edge angle image and edge strength image)

  In step S1802, an edge strength image and an edge angle image are obtained from the reduced search target image by the edge calculation unit. As an edge calculation method, a Sobel filter or the like can be used.

  Here, the Sobel method will be described. The Sobel method uses a 3 × 3 matrix as an operator (kernel). This method extracts a pixel value (for example, luminance) obtained by multiplying a pixel value (for example, luminance) by a coefficient and adds it as a pixel value at the center point for surrounding points centered on the target point. This method is a filter in the horizontal direction and the vertical direction, and has a characteristic of being resistant to noise because it includes a smoothing operation. The kernel used in the Sobel filter is shown below.

As a result, an edge strength image and an edge angle image of the search target image are obtained.
(Step S1803 Creation of edge angle bit image of search target image)

In step S1803, an edge angle bit image creating unit 69 creates an edge angle bit image from the edge angle image and the edge strength image. The edge angle bit image is image data expressing the edge angle of each point constituting the edge angle image as angle bit information. As a result, an edge angle bit image is obtained. The conversion from the edge angle image to the edge angle bit image will be described later.
(Step S1804: Reduction of edge angle bit image)

In step S 1804, the obtained edge angle bit image is reduced by the edge angle bit image reduction means 78. The reduction ratio is set to the global reduction ratio when creating the second pattern model for global search, and to the intermediate reduction ratio when creating the first pattern model for local search. As a result, a reduced image of the edge angle bit image is obtained.
(Step S1805 Execution of global search)

Next, a global search is performed on the edge angle bit reduced image reduced in step S1804 using a pattern model reduced in advance. Specifically, the search is performed in the entire range while changing the angle of the pattern model so as to scan from the upper left to the lower right. Thereby, a detection candidate region is extracted. The position of the detection candidate is expressed by, for example, xy coordinates, and the posture is expressed by, for example, an angle θ. Here, the detection candidates are performed by score calculation, and the coarse search means 71 moves the reduced pattern model with the degree of freedom of the search position and posture, and calculates the score at each position and posture.
(Score calculation)

In the global search, the score is calculated by comparing the edge angle bit image obtained by reducing the search target image to the global reduction ratio and the edge angle bits of each edge point included in the pattern model. Is done. At the time of search, the position and angle data of the base point are changed in accordance with the position and posture where the score is calculated. After that, the angle is converted in the same manner as the edge angle bit image, the pixel value of the reduced edge angle bit image data is ANDed, and the total number of remaining bits is the maximum of the expected total value. The value divided by the value is used as the degree of coincidence and is calculated by the coarse search means 71. It is also possible to assign a plurality of bits in the angular direction and add the concept of weight.
(Step S1806 Execution of local search)

  Further, a local search is performed on the detection candidate area found in the global search. In the local search, a pattern model for local search having a reduction rate lower than that of the pattern model for global search is used. The edge angle bit image as the search target image is also a reduced image reduced at a local search reduction rate lower than the global search reduction rate.

In addition, in the local search, not only the detection candidate area found in the global search is used as it is, but also a certain area in the vicinity thereof, for example, surrounding pixels such as 3 × 3 pixels and 5 × 5 pixels are also locally A search may be performed. As a result, stable search results can be expected.
(Expansion treatment)

  That is, in order to stabilize the score calculation result, expansion processing can be performed during the coarse search. In general, when the reduction rate of the search target image is reduced and the accuracy is increased, the score tends to be greatly reduced even if the position is slightly shifted. In order to avoid an abrupt change in the score, the rotation angle may be finely changed. In this case, however, there is a disadvantage that the processing amount increases. Therefore, in consideration of the balance between the reduction in processing amount and the improvement in accuracy, the edge angle bit image that is the search target image is expanded by a predetermined amount. For example, the number of pixels is expanded by a predetermined number of pixels, for example, 2 × 2 pixels obtained by doubling one pixel with respect to the XY direction of the pixels. Thereby, a rapid change in score value due to a slight angle shift is suppressed, and a stable score value can be obtained.

  In this way, based on the calculated score, the coarse position in the reduced search target image of the reduced pattern model is determined. Moreover, the said process can be repeated as needed and the precision of a coarse position can be improved. That is, not only simply dividing the rough search into two times, the global search and the local search, but also performing the local search multiple times, gradually reducing the reduction rate of the search target image, and using a larger reduced search target image, More accurate positioning can be performed.

The global search is normally performed only once because the search range is wide and the processing amount is large. However, the global search can be performed a plurality of times depending on the required accuracy and tact time. As a search method, known search techniques such as edge search, normalized correlation search, generalized Hough transform, and geometric hashing can be used.
(Details of precision positioning during operation)

  As described above, the coarse search is performed to obtain the position and orientation data of the detection candidate where the pattern model exists, and then the precise positioning is performed by the precision positioning means 76. Next, the detailed procedure of precise positioning will be described in detail based on the flowchart of FIG.

  First, in step S2001, based on the coarse position and orientation of the detection candidate obtained by the coarse search, the fine positioning pattern model is superimposed on the search target image. Here, it is preferable to use the original size search target image and the fine positioning pattern model with the position and orientation finally obtained by the coarse search as the start position and start orientation. However, precise positioning can also be performed with a reduction ratio higher than the original size (reduction ratio 1) and lower than the reduction ratio used last in the coarse search.

In step S2002, points corresponding to corresponding edge points are obtained along the corresponding point search line of the precision positioning pattern model. The corresponding point search line is a line segment having a predetermined length extending in the normal direction of the segment as described above, and a starting point that is one of both ends of the line segment is a search start point and an end point is a search end point. Here, edge calculation is first performed along the corresponding point search line to obtain an edge vector. As an edge calculation method, a Sobel can be used as described above. From the edge vector obtained by this edge calculation, the edge angle, edge strength, edge position, etc. of each point on the corresponding point search line are obtained. The edge vector is a vector representation of the edge strength and direction, and can be represented as ( _Ex , _Ey ). For example, as shown in FIG. 46, when the edge strength is E _M and the edge angle is θ _E , the edge angle θ _E = Atan (Ey / E _x ) and the edge strength E _M = sqrt (E _x2 + E _y2 ) can be expressed.
(Corresponding edge point search processing)

  Also, the corresponding edge point corresponding to the segment including the base point of the corresponding point search line is obtained based on the information such as the edge vector, the edge angle, the edge strength, and the edge position. As an example of a method for determining the corresponding edge point, it can be performed at high speed by using the above-described edge vector. As another method, the calculation can be performed using the edge strength and the edge angle as described below, but in this case, the calculation of Atan is required as described later, and the calculation becomes complicated. Hereinafter, a procedure for obtaining the corresponding edge point using the edge strength and the edge angle will be described.

  First, a point corresponding to a maximum point where the edge strength is larger than a predetermined edge strength threshold and the absolute value of the difference between the edge angle and the base point is smaller than the predetermined edge angle threshold is set as a corresponding edge point candidate. Further, a point closest to the base point among the corresponding edge point candidates is finally set as a corresponding edge point.

  Further, the subpixel position of the edge of the corresponding edge point is obtained (step S2003), an error value is obtained using the position and the geometric data of the segment, and the least square method calculation is performed (step S2004). Is obtained (step S2005). As an example of the error value, the distance between the corresponding edge point and the straight line is the error value in the line segment, and the absolute value of the difference between the distance between the corresponding edge point and the center position and the radius is the error value in the arc segment.

As described above, the error value and weight value used in the least square method calculation are calculated by the precision positioning means 76, and simultaneous equations obtained by the least square method are obtained from the values. Here, the least square method is applied so that the segment has an ideal shape and the error between the corresponding points corresponding to the segment is minimized. Furthermore, simultaneous equations are solved to obtain a highly accurate position and orientation.
In this way, the correction amount Δx of the position X, the correction amount Δy of the position Y, the correction amount Δθ of the angle θ, and the correction amount Δs (s: scale) of the scale s are obtained.

In precise positioning, the base point is overlaid on the search target image using the position and orientation data obtained by the coarse search. Then, an edge calculation such as a Sobel filter is performed along the corresponding point search line to obtain an edge vector. The edge vector is expressed as a result of applying the Sobel filter, and can be expressed by (S _x , S _y ) or the like. The edge strength E _M can be expressed as E _M = √ (S _x2 + S _y2 ), and the edge angle θ _E can be expressed as θ _E = Atan (S _y / S _x ). Furthermore, the edge angle, edge strength, and position of the pixel on the corresponding edge point are obtained from the edge vector, and the corresponding edge point corresponding to the segment including the base point of the corresponding edge point is determined from these edge vector, edge angle, edge strength, and position. It is determined by the positioning means 76.

  This situation will be described with reference to FIG. 21A. First, the pattern model PM indicated by a bold solid line is precisely positioned at the position of a detection candidate of a search target image (edge angle bit reduced image EABR indicated by a broken line) obtained by a rough search. The positioning means 76 superimposes them. A corresponding edge point TT corresponding to the base point KT is obtained along a corresponding point search line TL that passes through the base point KT set on the pattern model PM and is substantially perpendicular to the segment of the pattern model. In FIG. 21A, the corresponding point search line TL is indicated by a thin solid line. The corresponding point search line TL is a virtually set line and does not actually draw a line. The corresponding edge point TT is an intersection of the corresponding point search line TL and the reduced image EABR. Here, the corresponding edge point TT can determine the subpixel coordinate position. Using this position and the geometric data of the segment, the precise positioning means 76 performs precise positioning.

  Specifically, the relationship between the geometric data of the segment (in this case, a line segment) and the corresponding edge point is used as an evaluation value, and score calculation is performed so that the accumulated value of the evaluation value is minimized or maximized. As an evaluation value, a distance can be typically used, and a precise position can be obtained by performing a least-squares calculation so that the error value is minimized with this distance as an error value. As the distance, the Euclidean distance between the segment and its corresponding edge point can be used. That is, when the segment is a line segment, the distance between the corresponding edge point and the straight line is used, and when the segment is an arc, the absolute value of the difference between the distance between the corresponding edge point and the center position and the radius is used. By solving simultaneous equations obtained by the method of least squares, it is possible to obtain a highly accurate position and orientation. The evaluation value is not limited to the distance, and may be an angle formed by the base point and the corresponding edge point of the base point.

  FIG. 21B shows the state of the corresponding edge point search process in which the precise positioning means 76 obtains the corresponding edge point. In this figure, as in FIG. 21A, the broken line indicates the reduced image EABR of the search target image, the bold solid line indicates the pattern model PM, and the thin solid line indicates the corresponding point search line TL set on the base point KT. . Here, the coordinate positions x and y to which the Sobel filter is applied in the 3 × 3 pixel region SR of the reduced image EABR are obtained. The center coordinates of this calculation are performed using Bresenham's straight line data generation algorithm. In the example of FIG. 21B, the pixel B is extracted as the corresponding edge point for the model edge point A.

  The method shown in FIG. 21B is different from Patent Document 1 described above in that the corresponding edge point is arranged with a point appropriately selected on the segment from which the corresponding edge point is automatically extracted. In particular, Patent Document 1 does not define a seek line arrangement method. Further, when determining the corresponding edge point, not only the edge strength but also the data such as the edge angle and the edge position are used, thereby improving the reliability of the corresponding edge point. In addition, the kernel used in the processing for obtaining the edge angle and edge strength is made more compact, and the computation processing is lightly loaded. Furthermore, the position can be obtained with subpixel accuracy for the corresponding edge point. In addition, by using the method of least squares, there is an advantage that it can cope with models of various shapes.

  As described above, high-speed and high-precision positioning in pattern search is possible. In particular, in this method, by changing the line segment length of the corresponding point search line, the search range of the corresponding edge point by the corresponding point search line can be easily changed, and an advantage that necessary stability can be adjusted is obtained. That is, when the iterative least square method is applied, the line length of the corresponding point search line is gradually shortened, so that higher-precision and high-speed positioning can be easily realized.

  In addition, since the pattern model is expressed by segments, the wavy phenomenon at the edge position as shown in FIG. 22 can be eliminated. In other words, when performing precise positioning between points, the position may be greatly shifted because it is paired with a high wave part and a low wave part, but such influence can be reduced.

  Alternatively, or in addition to this, the edge angle threshold value may be changed each time the least square method is repeated. That is, more stable positioning can be realized by gradually reducing the edge angle threshold value in accordance with the number of times of the least squares method.

  When the pattern model is superimposed on the search target image based on the initial position obtained by the coarse search or the precise positioning start position obtained by the coarse search again, the search target image is unprocessed so-called original image data, As the pattern model, a pattern model corresponding to the original image data is used.

  With this method, it is not necessary to convert all the pixels of the original image data of the search target image into edge image data, and the processing speed can be increased. In particular, in-line processing requiring a tact time, such low-load and high-speed processing is preferable. If it is more efficient to extract the edge data in advance as a whole, it goes without saying that the pattern search can be performed by converting all points of the search target image into the edge image data.

  Further, it is not necessary to arrange all of the pattern models so as to overlap the search target image, and it is sufficient to arrange at least the corresponding point search lines. In particular, since the corresponding point search line is a straight line, it can be easily obtained by calculation. Therefore, in this specification, “superimpose” does not necessarily mean that images are actually superimposed, but is used to mean a process of determining corresponding edge points according to corresponding point search lines. In addition, the term “placed in an overlapping manner” here is for explaining that it is easy to grasp the corresponding position of each image by overlapping as described above, and is only a virtual thing in calculation. Needless to say, it is not necessary to actually superimpose data.

  According to this method, a highly accurate edge-based search can be realized as compared with the conventional method. In the method of Patent Document 1 described above, the direction of the edge and the angle component are not examined, and only the direction of the edge defined in advance is seen, so stability in a complicated shape cannot be expected. On the other hand, in the method according to the present embodiment, the edge direction is important, so that the reliability of the corresponding edge point can be improved. Furthermore, since the difference is calculated using a small kernel filter such as a Sobel filter in this embodiment, an edge can be detected even if the workpiece is elongated. As described above, a stable edge-based search that can cope with a search object having a complicated shape can be realized as compared with the technique of Patent Document 1.

  In addition, when the position of the detection candidate in the search target image of the pattern model, that is, the initial position of the local search is obtained by the coarse search, the pattern search is performed by reducing the search target image, so that the load can be reduced and the speed can be increased. There is. However, since some information is lost due to the reduction, there is a possibility that the accuracy may be lowered. Therefore, it is desirable to perform the reduction so as to maintain the information amount (details will be described later). Further, the initial position of the pattern model can be manually specified by the user in addition to the form obtained by the coarse search.

  In the above example, a point refers to a point constituting a search target image or a registered image, that is, one pixel (pixel), but a plurality of pixels (for example, four pixels) can be handled as one point. Needless to say. Therefore, in this specification, a point means one or a predetermined number of pixels.

  The term “based on the base point” is used to include not only performing edge detection on the base point but also performing edge detection near the base point. For example, edge detection is performed within a specified range such as 1 to 10 pixels around the base point.

Furthermore, a segment refers to a continuous finite length line composed of a combination of line segments and / or arcs. In addition to line segments and arcs, conic curves, spline curves, Bezier curves, and the like can also be combined. The corresponding point search line data includes the coordinates of the base point, the angle and length of the corresponding point search line (least square method).

  The least squares method applies a linear error function to the line segment. The straight line error function is a least square method in which the distance between a point and a straight line is an error function. An arc error function is applied to the arc segment. The arc error function is a least square method in which the distance between a point and an arc is an error function. Details of this will be described later.

  As a problem of the least squares method, there is a point that if even one large value exists, the accuracy is extremely deteriorated due to this point. Therefore, in this method, the influence is suppressed by using a weighted least square method in which weight values are given so as to reduce the weights for such points.

Further, as the degree of freedom used in the least square method, X direction movement, Y direction movement, rotation, enlargement / reduction, skew, aspect, and the like can be used. By selecting these, not only translation in the xy direction but also rotation, enlargement / reduction, distortion, etc. of the registered image can be handled.
(Generalization of least squares error function)

Here, the least squares error function is generalized and expanded. First error function _{E (p o, p 1,} ···, p n) is affine parameter p _o, p _1, considered ..., and is determined by p _n (e.g. _{p 0 = x, p 1 =} y etc). Here, it is assumed that the error function E (p _o , p ₁ ,..., P _n ) is minimized with the optimum affine parameters p _o o, p ₁ o,..., P _n o (o: optimized). . At this time, the error function _{E (p o, p 1,} ···, p n) to be represented by the following equation.

The meanings of the parameters in the above equation are as follows.
i: Index of the corresponding edge point
ω _i is a weight determined by the positional relationship between the corresponding edge point and the model, and is defined to approach 0 when the distance between the corresponding edge point and the line segment and the straight line is far, for example.
e (p ₀ , p ₁ ,..., p _n ): an individual error function determined by the geometric distance between the corresponding edge point and the model, and the distance between the corresponding edge point and the line segment and the straight line, etc. expressed.
p _{0 to} p _n : Affine parameters such as parallel x movement, parallel y movement, rotation angle, scale value, etc.

The error function _{E (p o, p 1,} ···, p n) affine parameters to minimize _{p o o, p 1 o,} ···, in order to obtain the p _n o, sufficiently close to the value displacement from affine parameter _{p 0t, p 1t, ···,} p nt but determined by precise positioning conducted in coarse search and the previous expected

Ask for.

_{Δp 0, Δp 1, ···,} Δp n is obtained by solving the system of equations below.

As described above, by using not only the edge strength of the image but also the angle of the edge, the direction component can be taken into account, and a more stable positioning resistant to the noise component can be achieved. In particular, by using differential processing for image data, a stable search that is hardly affected by fluctuations in illuminance is possible.
(Corresponding point search line filter processing)

  In particular, when a corresponding point search line search process is performed on a registered image and it is highly likely that it is difficult to select a corresponding point search line, such a position is preferably excluded from the pattern search. For example, when there is a registered image as shown in FIG. 23, if a corresponding point search line is automatically set for this pattern, the result is as shown in FIG. As shown in this figure, corresponding point search lines are set not only in the peripheral contour but also in a portion with a difference in density located near the inner center. When edge matching is performed on the corresponding point search line set in the vicinity of the center, there are many portions having similar edge angles. As a result, when a pattern search is performed using the corresponding point search line, there is a high possibility that the corresponding edge point becomes ambiguous by detecting the corresponding edge point.

  In the present embodiment, by eliminating such an ambiguous corresponding point search line, a stable and highly accurate search is possible. Specifically, the corresponding point search line filter means filters the corresponding point search line set in such an unfavorable region, and if there is something similar in edge strength and edge angle, this is excluded. To do. FIG. 25 shows an example of the result of filtering the corresponding point search line from FIG.

  An example of a procedure in which the corresponding point search line filter unit performs the corresponding point search line filter processing on the corresponding point search line candidates will be described with reference to the flowchart of FIG. First, in step S2601, a corresponding point search line generation process is performed on the registered image to generate a corresponding point search line candidate. Next, in step S2602, corresponding point search line candidates are placed on the registered image. When the corresponding point search line is arranged at the created position, the corresponding point search line candidate can be detected near the middle of the corresponding point search line candidate.

  In step S2603, corresponding point search line candidate searches are performed along the corresponding point search line candidates, and the number of corresponding point search line candidates is counted. In addition, when the filtering process is performed in step S2604 and the number of corresponding point search line candidates is 2 or more, it is determined that the corresponding point search line in the corresponding point search line is highly ambiguous, and the corresponding point search line is determined. Exclude from the candidate. In step S2605, the remaining corresponding point search line is set as the final corresponding point search line. By this process, an uncertain corresponding point search line is eliminated, and a more stable pattern search result is expected.

When the precise positioning process is repeated, the corresponding point search line selected by the corresponding point search line filter processing once performed is reduced in order to shorten the line length of the corresponding point search line according to the number of repetitions. This information can be used in a repetitive process. Alternatively, when the corresponding point search line is shortened, the corresponding point search line filter process is performed in the same manner, and the corresponding point search line selected as a result can be recorded.
(Corresponding point search line)

In addition, by changing the line length of the corresponding point search line, it is possible to improve the precision and speed of precision positioning. The line segment length of the corresponding point search line is determined by the difference in the reduction ratio between the coarse search and the fine positioning. For example, when precise positioning is performed with an equal magnification image and the final coarse search is 1/4 times, the length is about 2 * 4 = 8 pixels.
(Chain filter means 66)

  In the above example, at least one base point is set for all segments. In the present embodiment, when generating a segment, a highly reliable segment is constructed in advance by selecting a chain constituting the segment, and a base point is set for each segment. Such a specific chain is selected or excluded by the chain filter means 66 shown in the block diagram of FIG. References for the chain filter 66 to select a chain include average edge strength, chain length, and the like.

  This chain filter means 66 performs chain selection at the time of registration and operation. At the time of registration, a chain worthy of constituting a segment is extracted. Specifically, even if a segment is created from a short chain that does not meet the predetermined chain length threshold or a chain that has a low average edge strength that does not meet the predetermined edge strength threshold, the reliability of the segment data is low. It is anticipated that we will filter out these chains.

  On the other hand, during operation, since a short chain is highly likely to be noise, whether or not to use the chain is selected depending on the situation of the target image. For example, the user sets a length threshold and eliminates short chains. Hereinafter, these details will be described sequentially.

  First, when filtering a chain based on the average edge strength, the chain filter means 66 calculates the average edge strength of the edge points included for each chain, and compares the average edge strength with a preset average edge strength threshold. In other words, high accuracy is achieved by eliminating chains with low average edge strength, segmenting only chains with average edge strength above a certain level, setting a base point for the obtained segments, and pattern modeling A pattern search based on edges is expected, and the search accuracy can be improved. The average edge strength threshold can be set by the user to include an edge strength sufficient to identify the contour of the pattern model.

  When filtering by chain length, the chain filter means 66 performs filtering by comparing the length of each chain with a preset chain length threshold. That is, by selecting only a chain having a certain length or more and eliminating a shorter chain, a pattern search based on a stable edge can be performed, which contributes to an improvement in accuracy.

  On the other hand, the segment selection means 67 which filters with respect to the segment comprised with the chain can also be utilized. As a criterion for selecting a segment by the segment selecting unit 67, the average edge strength, the length of the segment, whether or not there is a segment having an edge angle similar to the vicinity, as in the chain filter unit 66, or the ideal edge angle is selected. And the like. Further, instead of simply eliminating short segments uniformly, the segment filtering method may be changed according to the combination of line segments and arcs constituting the segments. For example, a sufficiently long segment is extracted from a set of segments, a combination thereof is evaluated, and when there are one or more arc segments, if there are one or more line segments, short segments are unnecessary and are eliminated or deleted. Further, when all segments are line segments, if there are three or more long segments, sufficient accuracy can be maintained even if other short segments are excluded. In this way, the filtering by the segment selecting means 67 is changed according to the combination constituting the segment, and an appropriate segment is selected and the search can be executed more efficiently.

  Here, the segment length refers to the length of a straight line or a curved line from the end of the line segment or the arc of the segment constituting the pattern model. In addition, segment length thresholds can be individually provided for the line segment length and the arc length. The segment length threshold is set according to the registered image, the required accuracy, or the like, or can be set based on the average segment length of the registered image or the search target image.

Furthermore, when filtering based on whether or not there is a segment having an edge angle similar to the neighborhood, another segment having an edge angle similar to the neighborhood exists for the edge angle of the edge point included in each segment. The segment selection means 67 determines whether or not the segment exists. In other words, considering the possibility that the pattern search results may not be stable for segments having the same edge angle, the stability of the pattern search results is improved by setting such a base point by eliminating such segments. be able to.
(Temporary corresponding edge point)

  Next, a procedure for obtaining the coordinates of the corresponding edge point corresponding to the base point in the precise positioning will be described with reference to FIG. Here, first, a corresponding edge point is found on the corresponding point search line TL, then a corresponding edge point that is another pair is found, an average coordinate is obtained from the two, and the coordinates are determined as true corresponding edge point coordinates. To do.

  Specifically, in FIG. 29, a base point KT is set at a part of the arc-shaped segment (position indicated by a circle in the figure), and the corresponding point search line TL passing through this point is extended from the upper left to the lower right. Yes. First, the edge strength of each point of the search target image is examined along the corresponding point search line TL. In the example of FIG. 29, the edge strength is obtained based on the base point KT. Here, the edge strength of each of the four points of the grid-like vertices a, b, c and d including the base point KT is examined. Then, the subpixel position of the point having the highest edge strength is obtained as a temporary corresponding edge point. As a result, if e is calculated as a temporary corresponding edge point as a sub-pixel position of point a, for example, a pair point corresponding to this temporary corresponding edge point e is selected next. The pair points are selected so that the base point KT is sandwiched between the temporary corresponding edge point e and the pair point. Here, it is assumed that f is selected as the pair point. Further, average coordinates are obtained from these provisional corresponding edge points e and their pair points f. Then, the obtained average coordinates are treated as true corresponding edge point coordinates.

By doing in this way, it can suppress that a corresponding edge point is calculated in a wavy shape, and can obtain the stable calculation result. That is, when performing precise positioning between points, the pairing occurs at a high wave and the pairing occurs at a low wave, so that the position may be greatly shifted, but the influence can be reduced by the above method.
(Method to obtain the coordinates of the corresponding edge point using the nearby edge point)

  The method for obtaining the coordinates of the corresponding edge point in units of subpixels is not limited to the above, and other methods can be used. For example, the corresponding edge point is searched for in the pixel unit on the corresponding point search line, and the obtained point is set as the temporary corresponding edge point. It is also possible to obtain a plurality of neighboring edge points around this temporary corresponding edge point in units of pixels, obtain the subpixel coordinates of the temporary corresponding edge point and the plurality of neighboring edge points, and then obtain the average coordinates of these. it can. With this method, since the position of the true corresponding edge point can be obtained using a plurality of neighboring edge points close to the temporary corresponding edge point, the coordinate position of the corresponding edge point can be accurately obtained in subpixel units. . In addition, the coordinate position of the corresponding edge point can be determined more simply and accurately by using a plurality of neighboring edge points appropriately selected around the temporary corresponding edge point without using the corresponding point search line.

  The selection of neighboring edge points can be edge points located on the same contour or segment around the provisional corresponding edge point. Further, by using the edge angle of the temporary corresponding edge point, a neighboring edge point having a similar edge angle can be obtained. Preferably, edge points close to the left and right with the edge angle direction of the temporary corresponding edge point as the center are respectively selected as neighboring edge points. The distance from the temporary corresponding edge point to the neighboring edge point is preferably close, for example, within 2 pixels, preferably about 1 pixel. This is because if it is too far away, the accuracy will deteriorate. Hereinafter, the procedure for obtaining the coordinates of the corresponding edge point using the neighboring edge point will be described with reference to the schematic diagram of FIG. 65 and the flowchart of FIG.

  First, in step S6601, a temporary corresponding edge point is searched for on a pixel basis on the corresponding point search line. In FIG. 65, a white circle indicated by diagonal lines is the base point KT, and the corresponding position of the segment constituting the contour is found on the corresponding point search line TL passing through the base point KT. The search is performed on the basis of pixel units, that is, the intersection of the grid-like grid in FIG. 65. Here, (x, y) = (2, 3) is selected in pixel coordinates.

  In step S6602, neighboring edge points are selected in pixel units around the temporary corresponding edge points in pixel units. Here, since the edge angle of the tentative corresponding edge point has a vector in the direction overlapping the corresponding point search line, the edge points located on the left and right sides of the corresponding point search line, the edge having a predetermined value or more Edge points having strength are selected as neighboring edge points. Specifically, a first neighboring edge point located on the right side and a second neighboring edge point located on the left side are selected. In the example of FIG. 65, the pixel coordinates of the first neighboring edge point are (2, 2), and the pixel coordinates of the second neighboring edge point are (1, 3). At this time, since the edge angle of each neighboring edge point is an edge point near the temporary corresponding edge point, there is a high possibility that it has a similar edge angle, and it is not necessary to confirm it. Of course, it is also possible to select a neighboring edge point after confirming that the edge angle similarity is high. If there is no edge point having an edge strength greater than or equal to a predetermined value at a nearby position, the neighboring edge point is not selected. In this case, a true corresponding edge point to be described later is calculated using only the obtained edge points (the temporary corresponding edge point and the other neighboring edge point). In this example, two left and right neighboring edge points are selected in total, but the number of neighboring edge points can be selected to be 1, or 3 or more. However, considering the balance between accuracy and calculation processing, two is preferable.

  In step S6603, the coordinates of the true corresponding edge point are calculated based on the temporary corresponding edge point and the neighboring edge point. Here, the coordinate position of the true corresponding edge point is determined as the average coordinate from the sub-pixel positions of the temporary corresponding edge point and the neighboring edge point. In the example of FIG. 65, TP1 indicated by a black circle which is a subpixel position of the temporary corresponding edge point, a hatched circle TP2 which is a subpixel position of the first neighboring edge point, and a subpixel position of the second neighboring edge point. The corresponding edge point is determined from the average of the three points of the cross-hatched circle TP3. The sub-pixel position of each edge point is calculated in advance from the surrounding pixel values. As a sub-pixel position calculation method, a known method can be used. For example, the pixel value around 3 × 3 centering on each edge point or information on adjacent edge points in the edge angle direction is used. Thus, the subpixel position can be calculated. Note that the timing for calculating the sub-pixel position of each edge point is not particularly limited, and may be immediately after the edge point in units of pixels is determined or immediately before the average coordinate calculation.

As described above, the corresponding edge point can be calculated in units of subpixels from the average of the subpixel coordinates of the three edge points. With this method, three edge points can be easily extracted from the provisional corresponding edge points, so that highly accurate corresponding edge points using many edge points can be determined. For example, if the number of base points is 10, the number of corresponding edge points is usually 10; however, in the above method, 30 points are added by adding left and right points to each of the 10 temporary corresponding edge points. Since the corresponding edge point can be calculated from the edge point, the accuracy can be improved by the averaging effect. In particular, compared to the method of obtaining the average coordinates from the pair point and the temporary corresponding edge point described above with reference to FIG. 29, the average coordinates are obtained by three points obtained by adding two neighboring edge points to the temporary corresponding edge point. This is advantageous. In the above description, three points are obtained and averaged, but each of the three points can be individually used for precise positioning calculation.
(Conversion from edge angle image to edge angle bit image)

  Next, conversion from an edge angle image to an edge angle bit image will be described with reference to FIGS. When taking the course-to-fine approach, it is not easy to set a reduced image for the first coarse search. This is because the feature amount information necessary for the search may be lost due to the reduction of the image. Particularly in edge-based search, edge angle information is important for improving search accuracy. Therefore, in the present embodiment, when the image is reduced by the image reduction means 61, the edge angle bit image creation means 69 and the edge angle bit image reduction means 78 that can maintain the edge angle information even if the reduction ratio is increased are provided. Therefore, the data amount is reduced while leaving a sufficient feature amount, and the processing speed is increased.

Hereinafter, a procedure for acquiring edge angle information and holding it when the image reducing means 61 reduces the image will be described. First, the edge strength is examined for each edge point of the edge angle image, and when the edge strength is larger than the set edge strength threshold, the bit corresponding to the direction of the edge angle is set to 1, and the others are set to 0. . For example, let us consider a case where the edge angle bit image generating unit 69 generates an edge angle bit image from a 2 × 2 pixel edge angle image composed of four pixels (edge points) of the pixels a to d shown in FIG. Each of the pixels a to d has an edge strength greater than a threshold value, and each has an edge angle indicated by an arrow. This edge angle is expressed by eight kinds of edge angle bits of 0 to 7 according to the edge angle classification representing the correspondence between the edge angle and the bit.
(Edge angle bit image)

  When this edge angle image is converted into an edge angle bit image, the edge angle information is converted into edge angle bits. The edge angle bit is a code obtained by dividing the edge angle direction into predetermined angles. The edge angle classification shown in FIG. 31 can be the same as that shown in FIG. In the example of FIG. 31, as the edge angle direction, the density gradient direction is divided into 8 sections in 45 ° increments, and an edge angle bit is assigned to each section. Alternatively, it is not limited to this example, it is rotated 22.5 ° counterclockwise from the posture shown in FIG. 31, is divided into 8 equal parts by shifting 22.5 ° from the horizontal or vertical direction, and is 45 ° clockwise from the right. E, SE, S, SW, W, NW, N, NE are labeled with edge angle sections, and edge angle bit labels are assigned to each edge angle section as 0, 1, 2, 3, 4, 5, respectively. 6 and 7 may be given (FIG. 52 (b) described above). Of course, such division is merely an example, and for example, it can be divided into 16 equal parts, 4 equal parts, or even 3 equal parts and 5 equal parts.

  From the above, when the edge angle image data shown in FIG. 30 is converted into a 2 × 2 edge angle bit image based on the edge angle section of FIG. 31, the result is as shown in FIG. Thus, for the four edge points distinguished by the labels a, b, c, and d, a bit is set in the edge angle section corresponding to each edge angle.

In addition, as a method of acquiring an edge angle bit image, as described above, in addition to a method of processing only a portion having a certain edge strength threshold value or more, an edge angle bit image having a certain width is acquired. In order to achieve this, there is a method in which an edge strength image and an edge angle image are used for thinning, and an edge angle bit image as described above is obtained using the thinned edge angle image. The method of performing the thinning process has a merit that noise can be easily eliminated because the contour portion of the search target can be limited although the processing time is relatively longer than the above-described method.
(Reduce edge angle bit image)

  In this way, after the image is expressed by the edge angle bit image, the data is reduced so that the information on the edge angle remains sufficiently. Specifically, for each edge angle bit label, synthesis is performed so as to take OR or bit sum of edge angle bits of each pixel. For example, if the 2 × 2 data to be reduced is in the state shown in FIG. 32, the data is reduced to vertical ½ × horizontal ½ (= 1/4), and four pixels a to d are set to 1 of a ′. When expressed in pixels, it is as shown in FIG. As shown in this figure, the edge angle bits of the pixels a to d are combined with the edge angle bit reduced image in which the edge angle bits of the pixels a to d are combined. A bit stands. The processing for holding the edge angle information in the edge angle bit reduced image is performed by the edge angle bit image reducing means 78. As a result, the edge angle information is maintained after the reduction while reducing the amount of data, so that sufficient features can be left in the search even if the reduction ratio is increased by repeating the reduction.

  With the above compression processing, if the reduction ratio is suppressed so as to leave a sufficient feature amount in the search, which is a conventional problem, the situation where the search processing speed becomes insufficient is improved. Even if the reduction ratio for performing edge detection is fixed at, for example, ½, a sufficiently high-speed search becomes possible by the reduction processing of the edge angle bit image. The reduction ratio for performing edge detection can be automatically determined based on at least one of the size of the registered image and the feature data of the pattern model. Also, the user may set it uniquely.

  Here, an example of a procedure for generating an edge angle bit reduced image by reducing the edge angle bit image will be described with reference to FIGS. 34 to 37. When creating an edge angle bit reduced image, there is a clipping problem as a point to be noted. That is, if the reduction process is simply performed by the sub-sampling process, there is a possibility that the score calculation described later may vary greatly due to a slight shift in the start coordinates of the process and the position in the input image to be searched. The following two methods can be considered as a reduction method for avoiding this clipping problem.

  The first method is a method of performing expansion after performing reduction processing. This method will be described with reference to FIGS. 34 and 35, with an example of a process of reducing to 1 / n (n = 2). First, an OR operation is performed on nxn edge angle bit data included in each square area of FIG. The result is replaced with edge angle bit data representing the edge angle bit data of each of the previous nxn regions. By performing this process, the image is reduced to 1 / n of the original image.

  If this is the case, a clipping problem may occur. Therefore, the reduced image is expanded. Here, as shown in FIG. 35, an OR operation is performed on the edge angle bit data of each square mxm (m = 2 in this example) of the reduced image, and the result is obtained for each mxm area. Is replaced with edge angle bit data representative of. This process does not reduce the image. In this example, m = 2, but it is possible to increase m in accordance with the expected variation in the shape and size of the search target.

  The other method is a method in which the range of the OR operation performed in the reduction process is widened and reduced, and the subsequent expansion is not performed. This method will be described with reference to FIG. 36 in the case of processing to reduce to 1 / n (n = 2). An OR operation is performed on edge angle bit data of (n + m) x (n + m) (n = 1, m = 1) included in each square area in FIG. The result is replaced with edge angle bit data representing nxn edge angle bit data near the center of the previous region. By performing this process, the image is reduced to 1 / n of the original image. FIG. 37 shows a case where n = 2 and m = 1 in the second method.

Here, as shown in FIG. 36, when normal 2 × 2 reduction is repeated without expansion, a problem of the cutout position may occur. That is, when the upper left coordinate of the search area changes by one pixel, the set of pixels at the time of reduction changes, and the score calculation decreases even though the registered image and the search target image are the same image. Can happen. On the other hand, when the expansion is performed, there is an advantage that such a problem does not occur.
(Edge angle bit conversion processing at angle boundary)

  Furthermore, in the edge angle bit conversion, it is expected that the edge angle bit when the edge angle is in the vicinity of the angle boundary will have two bits corresponding to the two angle areas constituting the boundary, thereby improving the stability. it can. For example, in the edge angle bit image of FIG. 31 obtained by converting the edge angle image composed of the four pixels a to d shown in FIG. 30 described above, if the edge angle is near the boundary between E and SE, The edge angle bit of the E section may be set or the edge angle bit of the SE may be set. Due to such blurring, it is expected that non-essential effects will occur in the coincidence calculation. Therefore, when the edge angle is at the boundary, the edge angle bits of both edge angle sections that divide the boundary are set to 1. This eliminates blurring due to noise, and a stable calculation result of coincidence can be expected. Specifically, when the edge angle is located within a predetermined width (for example, 5.625 °) centered on the boundary of the edge angle section, both edge angle bits facing the boundary are set to 1.

Note that the edge angle bit conversion processing at the angle boundary is performed only for the edge angle bit conversion processing to be searched, and is not performed in the edge angle bit conversion of the pattern model. This is because if the same processing is performed for the conversion of the edge angle of the pattern, an unnatural weight change occurs for each edge point.
(Edge angle bit adjacent processing)

  In the above example, only one edge angle bit is set in the conversion from the edge angle image to the edge angle bit image. However, the edge angle bit is also set for the adjacent edge angle sections around the corresponding edge angle section. Such edge angle bit adjacency processing can also be performed. For example, a 1-bit edge angle bit is assigned to the corresponding edge angle section and the edge angle sections adjacent to the left and right. Alternatively, weighting may be performed so that 2 bits are assigned to the corresponding edge angle section, and 1 edge angle bit is assigned to the edge angle sections adjacent to the left and right. Further, the fuzzy effect is such that the edge angle of the pattern model and the edge angle of the image to be searched are sufficiently coincident, such that the edge angle bit is 1 when the edge angle is slightly shifted, and 0 when the edge angle is large. Can also be weighted.

  Even when such edge angle bit adjacency processing is performed, if the edge angle exists near the boundary as described above, the influence of blurring due to noise is considered, so the edge angle bit of the search target image is at the boundary of the edge angle section Can avoid the influence of blurring by setting 2 bits for each edge angle segment adjacent to the boundary.

In the above example, 8 bits are used as the angular resolution of the edge angle. However, the present invention is not limited to this, and conversion with higher angular resolution can be performed, such as 16 bits or 32 bits.
(Parallelization)

Further, by parallelizing values obtained by converting the edge angle of the pattern model into edge angle bits, the search process can be speeded up. Examples of parallelizing the edge data of the pattern model are shown in FIGS. FIG. 38 is a conceptual diagram of the pattern model before parallel processing, and FIG. 39 is a conceptual diagram of the pattern model after parallel processing. As shown in these figures, a plurality of edge angle bit data of one base point on the pattern model are arranged side by side to achieve parallelization, and the processing speed can be increased. Here, a general CPU constituting the calculation unit can perform parallel processing from 4 bytes to 8 bytes, and can perform 4 to 8 times speed processing. As described above, the parallel search can speed up the coarse search.
(Coarse search using edge-angle bit reduced images)

  Hereinafter, a procedure for performing a rough search using such reduced data will be described. Here, it is assumed that an edge angle bit reduced image of 1/8 is obtained by performing reduction. For example, if the edge-extracted bit-reduced image has a size of 640 × 480 and the edge extraction is 320 × 240, which is 1/2, the edge subjected to compression processing of 1/8 The size of the angle bit image is 40 × 30. The search for the edge angle bit reduced image is performed as follows. First, at the time of pattern model registration before search processing, data as shown in FIG. 40 is created. As shown in this figure, the pattern model is held as an array of edge data having position and angle information. FIG. 40A shows an example of a pattern model, and FIG. 40B shows an example of a pattern model of edge data. In FIG. 40B, x and y are edge coordinate positions, and θ is an edge angle. This pattern model is held as an array of edge data having coordinate position and angle information as shown in FIG.

Using the above data and the edge angle bit image of the search target image, the position and orientation of the pattern are repeatedly changed, and the degree of coincidence is sequentially calculated for each position and orientation of the pattern. This calculation is performed as follows. First, an affine transformation value representing a position and orientation for which the degree of coincidence is to be examined is determined. This affine transformation value is created in consideration of the scale of the pattern model and the scale of the search target image. Using the affine transformation values, the edge positions x _i and y _i and the edge angle θ _i are transformed. Here, the converted edge positions are X _i and Y _i , and the converted edge angle is φ _i (i is a subscript indicating the index of the edge). The degree of coincidence is calculated by converting the edge angle φ _i into bit data in the same manner as the search target image. The equation for calculating the coincidence S is expressed by the following equation.

In this way, the precision positioning means 76 compares the edge strength and edge angle of each edge point included in the search target image and the pattern model, and calculates the degree of coincidence. A high degree of coincidence S indicates a high probability that a pattern model exists at that position and orientation. In this way, the situation where the processing rate is not sufficient if the search is performed as it is with the reduction ratio determined so as to leave a sufficient feature amount for performing the search is improved. In addition, even if the reduction ratio for performing edge extraction is fixed to 1/2, for example, a sufficiently high-speed search becomes possible, so that it is not necessary to determine a reduction ratio for performing edge extraction, which is generally complicated. Can also be obtained.
(Reduction process of search target image)

With such reduction processing, sufficient features can be left in the search even if the reduction ratio is high. When edge extraction processing such as a Sobel filter is performed on a search target image, the original size of the same size image has a lot of noise, which may be inappropriate for extracting a feature for a coarse search. Therefore, in this embodiment, edge extraction processing is performed after a reduced image is created first. Thereby, an averaging effect can be obtained by reducing the image data, which contributes to noise reduction. Here, the initial reduction ratio (first reduction ratio) is set to ½ of the original size. With this size, it is possible to obtain a noise reduction effect by averaging while leaving sufficient features necessary for the search.
(Polarity in the edge direction)

  Furthermore, in this method, the presence or absence of polarity in the edge direction can be set. Therefore, the processing method for the edge angle can be changed according to the polarity. Conventionally, such a concept of polarity is not considered, and only the concept of the direction (angle) of the edge is used. Therefore, for example, as a result of being treated as 0 to 180 °, vectors having different orientations cannot be distinguished and erroneous search occurs. There was a problem. On the other hand, the above-described method can cover 0 to 360 ° by considering the concept of polarity, and a more accurate search is realized.

In the case of ignoring the polarity, it can be easily realized by the process of simultaneously setting the reverse direction bit in the edge angle bit conversion on the search target side. Alternatively, it is possible to assign bits evenly with respect to the edge direction instead of the direction. For example, in coarse search, by assigning 8-bit edge angle bits, which are edge resolutions, equally to the edge direction, it is possible to obtain a search result in which the edge direction resolution is more important than the edge polarity. .
(Precise positioning for rotation angle)

  Next, a procedure for precise positioning of the arc segment by the least square method using the distance between the point and the arc as an error function will be described with reference to FIGS. Here, a case is considered in which precise positioning is performed with respect to a search target image using a pattern in which a circular shape with a part cut away is represented by an arc segment and a line segment as shown by a thick line in FIG. 41 as the pattern model PM. The pattern model PM shown in FIG. 41 is composed of one arc segment and one line segment, and such a workpiece shape is generally used for a wafer or the like provided with an orientation flat surface. FIG. 42 shows a state in which a rough search is performed on the search target image with the pattern model PM and the pattern model PM is arranged at the position of the detection candidate and is positioned to some extent. Here, a state is considered in which the pattern model PM is almost superimposed on the edge point of the search target image, and only the line segment of the cutout portion is inconsistent. In these drawings, the thick line indicates the pattern model PM, and the thin line indicates the edge point (input edge point IE) of the search target image. Further, the attention point of the input edge point IE is indicated by a broken line arrow, and the tangent SL of the input edge point is indicated by a broken line. The actual tangent line SL is shorter and exists in a state where it overlaps the fine line pattern model PM. However, in these drawings, the tangent line SL is displayed longer for convenience of explanation.

  When precise positioning is performed from this state, even if the least square method is applied to the input edge point at the position indicated by the arrow in FIG. It is expected that the error will not necessarily grow. Here, the least square method using the distance between the point and the arc as an error function is applied to the arc segment instead of the point and the line. Specifically, the least square method is applied with the absolute value of the difference between the center of the arc-shaped pattern model and the radius having the edge point of the search target image as an arc as an error value. That is, the error function of the arc segment used in the least square method is the difference between the ideal radius centered on the arc segment and the distance between the center of the arc and the corresponding edge point. The error function of this arc segment can be expressed by the following equation.

  As a result, as shown in FIG. 43, the distance between the input edge point indicated by the arrow and the arc model does not change much. That is, it means that the counterclockwise rotation of the pattern model PM as shown in FIG. 43 occurs as a sufficient solution. As a result, high angle accuracy can be expected with a small number of precise positionings in an arc-shaped workpiece. 42 and 43, the arc region indicated by the dotted line actually overlaps the fine line of the pattern model, but for the sake of convenience of explanation, these figures are displayed with a slight shift from the pattern model.

On the other hand, in the conventional least square method, the distance between the input edge point and the tangent line is used as an error function, and the pattern model is moved so that this distance becomes small. For this reason, the pattern model may not be rotated in the correct rotation direction, but the positional deviation may be increased. For example, there is a case where it cannot be expected to rotate the pattern model PM counterclockwise as shown in FIG. 48 from the state of FIG. That is, when the pattern model PM is rotated as shown in FIG. 48 from the relationship between the input edge point at the position indicated by the arrow in FIG. 47 and the tangent, the input edge point is in a direction away from the tangent. It was a result that could not occur as a solution. In contrast, in the present embodiment, since the least square method using the distance between the point and the arc as an error function is employed as described above, high angle accuracy can be achieved with a small number of precise positionings on a circular workpiece. I can expect.
(Corresponding edge point generation processing)

  Further, FIG. 44 shows an example in which the corresponding point search line generation processing is performed on the pattern model PM of FIG. In this way, one base point is assigned to the center on the arc segment and line segment, and is set by excluding the vicinity of the edge at regular intervals from that point, and from the inside in the normal direction of the segment on each base point. A corresponding point search line is set outward. FIG. 45 shows a state in which a rough search is performed on the search target image with this pattern model, and the pattern model PM is superimposed on the position and orientation of the determined detection candidate.

  In this state, edge extraction is performed along each corresponding point search line to search for corresponding edge points for each segment. Since the corresponding edge point is a point where each corresponding point search line and the edge point (thin line) of the search target image intersect, for example, the point indicated by x in FIG. 45 is the corresponding edge point. In the example of FIG. 45, since the arc part is almost overlapped with the pattern model arranged at the position of the detection candidate obtained as a result of the coarse search and the input edge point, the corresponding edge point of the arc segment is almost identical to the arc segment. I'm doing it. For this reason, even if the arc-shaped workpiece is relatively rotated, the error function of the arc does not increase, so this degree of freedom does not hinder the rotation. Therefore, it can be expected that rotational movement is the solution.

On the other hand, many of the corresponding edge points of the line segment are not on the line segment. In the least-squares method for line segments, the distance between the point and the line is an error function, so it is speculated that rotating the counterclockwise as a whole will reduce the value of the error function. . From these facts, it can be seen that the solution obtained by the method of least squares can sufficiently be expected to obtain counterclockwise rotation.
(Weighting process when there are multiple corresponding edge point candidates)

Furthermore, when there are a plurality of corresponding edge point candidates corresponding to the base point, each corresponding edge point is weighted to improve the precision of precision positioning when the determination of the corresponding edge point is ambiguous. Can do. This will be described with reference to FIG. Here, consider an example in which precise positioning is performed using a pattern model PM in which two vertical lines are included in a rectangle as shown by a thick line in FIG. In this case, it is assumed that segments SG ₁ and SG ₂ as shown by thin lines in FIG. 49B are obtained by the coarse search. Here, only _two segments SG ₁ and SG ₂ positioned on the left and right sides of the workpiece in FIG. 49B are considered. The corresponding point search lines TTL ₁ and TTL ₂ of these segments are set in the normal direction (edge direction) of the segments SG ₁ and SG ₂ passing through the base points KT ₁ and KT ₂ as described above. For each segment SG _1, SG _2, when the reference point KT _1, KT corresponding point search line TTL ₁ in each of the _2, TTL ₂ set as indicated by broken lines, on each corresponding point search lines TTL _1, TTL ₂ A corresponding corresponding edge point candidate is obtained. Here, there is only one corresponding edge point candidate TT _A for the right segment SG ₁ , but there are _two corresponding edge point candidates TT _B and TT _C for the left segment SG ₂ . The determination of the corresponding edge point selects the closest corresponding edge point among the corresponding edge point candidates, so that TT _A and TT _B are the corresponding edge points. Here, it can be said that in the example of FIG. 49 (b) because for the left segment SG ₂ there are a plurality of corresponding edge point candidates includes ambiguity. In this case, there is a problem in which direction the segment, that is, the pattern model is moved. In particular, in the example of FIG. 49B, if TT _B is selected as the corresponding edge point, if the entire segment, that is, the pattern model, moves to the right, it moves in the direction opposite to the desired precision positioning. It is not preferable.

Therefore, when the least square method is calculated, each corresponding edge point is weighted. Since In the example of FIG. 49 (b), the corresponding edge point TT _A is only one existing on the corresponding point search line TTL ₁ base point KT ₁ for the right segment SG _1, the weight "1.0 "give. On the other hand, with respect to the left side of the segment SG _2, on the corresponding point search line TTL ₂ base point KT _2, the corresponding edge point candidates TT _B on the left and right sides of the base point KT _2, TT _C is located. Therefore, the weighting of the corresponding edge point TT _B about the base point KT _2, set according to the distance from the base point KT ₂ to each corresponding edge point candidates. As an example of an expression for determining the weight, as shown in FIG. 49 (d), the distance between the first corresponding edge point candidate and the base point is d ₁ , and the distance between the second corresponding edge point candidate and the base point is d. ₂ (d ₁ ≦ d ₂ )

Weight W = 1−α (d ₁ / d ₂ )
However, 0 <α <1
And In the above formula, W = 1 when there is one corresponding edge point candidate, and it becomes smaller when there are a plurality of candidates. In this way, when the number of corresponding edge point candidates is one, that is, when weighting is considered to be unambiguous, movement in a more probable direction can be expected. When there are two or more corresponding edge point candidates, the closest corresponding edge point candidate is set as the corresponding edge point, and the positional relationship between the corresponding edge point and the corresponding edge point candidate sandwiches the ideal point. Equation W = 1−α (shorter distance) / (longer distance), and W = 1 when no ideal point is sandwiched. Such weighting work and determination of the segment movement direction after weighting are performed using the precision positioning means 76 included in the computing means 6 of FIG. 1 as the weighting computing means.
(Score calculation)

  In addition, when performing precise positioning calculation using the least square method, a score representing similarity can be calculated at the same time. That is, the least square method is applied to the corresponding point search line, and the score is calculated by the ratio of the number of corresponding points and the number of base points in the final least square method. In simple terms, a score obtained by dividing the number of corresponding points by the number of base points is regarded as a similarity, and the score can be obtained by the following equation.

S: score n: number of corresponding points m: number of base points

  Also, the similarity between the ideal edge angle of the base point and the edge angle of the corresponding edge point can be reflected in the score. In this case, the ratio of the sum of the weights obtained from the difference between the edge angle of the corresponding edge point in the final least square method and the edge angle of the corresponding base point and the number of base points is calculated as a score. . Specifically, it can be calculated by the following formula.

S: score n: number of corresponding points m: number of base points ω (x): 1 when x = 0, monotonically decreasing as x becomes larger θ _i : edge angle θ _i ^{p of} corresponding points: corresponding points Angle of base point corresponding to (ideal angle of base point)

  Here, for example, when the segment is a line segment, the ideal edge angle of the base point is the normal direction of the line segment and overlaps with the corresponding point search line. In this method, the smaller the difference between the edge angle of each corresponding edge point and the ideal edge angle of the base point corresponding to the corresponding edge point, the closer the weight is to 1, and vice versa. Is set as follows. For example, the weight is set so that the angle difference is 0 when the angle difference is 0 ° to 18 °, 0.1 when the angle difference is 18 ° to 36 °, and 1 when the angle difference is 162 ° to 180 °. In this manner, the weight is calculated for each base point, and the score is calculated by averaging the finally obtained weights.

In the example of FIG. 49B, the weight 1 is assigned to the corresponding edge point A on the right side and the weight “0.9” is assigned to the corresponding edge point B on the left side. As a result, an action to move the pattern model to the left side works for the right segment SG ₁ , and an action to move the pattern model to the right side works for the left segment SG ₂ . Therefore, when these are combined, the weight to be moved to the left side of the segment is 1.0 and the weight to be moved to the right side is 0.9. As a result, it is moved to the left side as shown in FIG. become. Similarly, weighting is performed again from the state of FIG. 49 (c), and the movement process based on the weighting result is repeated to determine the final precision positioning. As described above, the direction in which the pattern model is moved during positioning is evaluated by weighting according to the distance between the corresponding edge point candidate and the base point, so that the pattern model can be moved in a relatively reliable direction. Reliability and stability can be expected.
(Segment selection method considering positioning direction)

  Such a case can occur particularly when the segment of the line segment is concentrated in a specific direction. In the example of FIG. 49B, there are only segments in the vertical direction (Y-axis direction), and accurate positioning can be expected in the X-axis direction, but there are no segments of line segments extending in the horizontal direction (X-axis direction). Therefore, the Y-axis direction cannot be defined, and positioning in this direction is ambiguous. Therefore, when selecting the segments that make up the pattern model, by consciously selecting the intersecting segments such as the X-axis direction and the Y-axis direction so that the positioning direction is not concentrated in a specific direction. Stable positioning results can be expected. Hereinafter, a method for selecting a segment in consideration of the positioning direction will be described with reference to the flowchart of FIG.

  First, in step S4901, in a state where a plurality of segment candidates are obtained, the segment candidates are sorted in length order. In the case of an arc segment, the arc length is defined as the segment length.

  In step S4902, the longest segment candidate is selected as a segment and set as a reference segment. The normal direction of the reference segment is set as a reference angle. If an arc segment is selected, the reference angle is disabled. When the reference angle is invalid, the conjugate segment is selected only by the length of the segment candidate regardless of the angle.

  In step S4903, segment candidates that are conjugate segments with respect to the reference segment are extracted. Here, it is searched from the reference angle whether there is a segment candidate included in a predetermined angle range, here the first angle range. An example of the first angle range is shown in FIG. 51A. A range of ± 45 ° centered on the normal direction (90 °) of the reference segment, that is, a range of 45 ° to 135 °, and a total of 90 °. Segment candidates included in the range are extracted.

  In the example of the segment candidate shown in FIG. 51 (b), the segment candidates with a circle are extracted, and the segment candidates with a cross are excluded. If a line segment is selected as the conjugate segment, the reference angle is the normal direction of that segment. Becomes valid. If the segment is not a line segment but an arc, it is extracted unconditionally. This is because, in the case of an arc, it can be expected that the angle change is large, so that it can be useful information for positioning. In the case of an arc, the reference angle state does not change.

  If a segment candidate is extracted, the process advances to step S4904-1 to select the longest segment candidate among these as a segment and set it as a conjugate segment with respect to the reference segment. In step S4905, it is determined whether or not the number of selected segments has reached the specified number. If it has been reached, the process ends. If not, the process proceeds to step S4906, and a conjugate segment is newly set as a reference segment. Then, the process returns to step S4903 to repeat the process. When an arc segment is selected as the conjugate segment, the selection of the conjugate segment when the arc segment is used as the reference segment is selected only by the length regardless of the angle as described above when the reference angle is invalid. If it is valid, the same process as described above is performed to extract the conjugate segment.

  On the other hand, if there is no segment candidate included in the first angle range in step S4903, the process proceeds to step S4904-2, and is included in the second angle range that is wider than the first angle range. A similar search is performed for segment candidates. As an example within the second angle range, in the example of FIG. 51A, the range is 40 ° to 140 °, which is ± 5 ° wider than the first angle range. If a segment candidate is found here, the process jumps to step S4904-1 to select the longest segment as described above and set it as a conjugate segment.

  If not found even within the second angle range, in step S4904-3, a search is similarly performed to find whether there is a segment candidate included in the third angle range that is wider than the second angle range. To do. As an example within the second angle range, in the example of FIG. 51A, a range of 35 ° to 145 °, which is wider by ± 5 ° than the second angle range, is set. If a segment candidate is found here, the process jumps to step S4904-1 in the same manner, and the longest segment is selected and set as a conjugate segment. If not found, the process returns to step S4902, and the longest segment among the segment candidates is selected again as the reference segment. Note that the numerical value of the angle range, the number of times of setting in the case of the angle range, etc. can be changed as appropriate. For example, if no segment is found in step S4904-3, a search with a wider angle range may be performed. If step S4904-3 is omitted and no segment is found in step S4904-2, it is possible to immediately return to step S4902 and reset the reference segment.

In this way, the operation of selecting a conjugate segment close to the orthogonal direction with respect to the reference segment is repeated, and as a result, segments with dispersed angles are selected, so that positioning stability can be improved. Such segment selection is performed by the segment selection means 67 of the segment creation means 68.
(Pattern feature selection function)

Furthermore, it is possible to provide a pattern feature selection function capable of changing the selection criteria of the segments constituting the pattern model in accordance with the pattern features obtained from the search object. Specifically, this is particularly effective for a registered image as shown in FIG. In the registered image of FIG. 67, different characters and numbers are displayed in a grid-like frame. When the pattern window PW is set in such an image, the pattern model to be created has many segments SG _W set in the frame portion as shown in FIG. 67, and conversely, the segment SG is set in the characters in the frame. _M is not set much. In such a pattern model, positioning is performed only in the frame portion. As a result, characters in the frame are ignored or not emphasized, recognition becomes difficult, and the position is shifted in units of frames. There was a risk of failure.

  This is due to segment selection criteria. That is, in the conventional concept, a long segment is preferentially selected from the viewpoint of eliminating the noise component and improving the positioning accuracy by placing importance on the sharply detected edge point. . This is because the short outline is considered to have more noise components, and conversely, the longer line segment is set to automatically select from the longer line segment based on the assumption that the edge information is correctly extracted. In other words, there has been no image processing method or the like that enables selection with priority on short line segments. For this reason, in the example as shown in FIG. 67, since the edge portion of the frame portion surrounded by the straight line tends to be relatively easy and clear, the segment generated in the frame portion is easily selected. A positioning failure occurred. In particular, in rough search, since it is a simple search, it does not use all the extracted contour information such as edges, chains, segments, etc., and only some contours are selected. If it does not contribute to accurate positioning, such a problem of false detection occurs.

In contrast, in the present embodiment, by providing a function that can be selected in ascending order from a short contour, an appropriate search result corresponding to the registered image can be obtained. In addition, the noise component is realized by setting a threshold value and eliminating a line segment having a predetermined length or less. As a result, it is possible to obtain a highly reliable search result while effectively eliminating noise components.
(Sort by contour length)

  Next, two methods for selecting such a short contour will be described. First, a method of sorting the contours in order of length and selecting a predetermined number from the shortest ones will be described based on the user interface screen of FIG. FIG. 68 shows a setting of a pattern feature selection function for appropriately selecting a contour for constructing a pattern model of a registered image in an image processing program for searching a portion similar to the registered image from search target images using a pattern model. 4 is an image diagram showing a user interface of a screen 200. FIG. In this screen, the user makes settings for the coarse search and the fine positioning for each of the edge strength upper limit 81, the edge strength lower limit 82, the contour length lower limit 83, the contour selection number 84, and the contour registration order 85 as setting items. be able to. Among these, the items regarding the pattern feature selection function are the contour length lower limit 83, the contour selection number 84, and the contour registration order 85.

The edge strength upper limit 81 and the edge strength lower limit 82 define a range for detection as an edge, and a filtering condition is specified so as to exclude those whose edge strength is higher than the upper limit value or lower than the lower limit value.
(Contour length lower limit setting means)

The contour length lower limit 83 functions as contour length lower limit setting means for setting a lower limit value for detection as a contour. That is, the contour whose length is shorter than the lower limit defined by the contour length lower limit 83 is filtered by the length filter. Thereby, noise that appears as a short outline can be eliminated. Also, by making this value adjustable by the user, the strength of filtering can be appropriately adjusted according to the use environment and application. Depending on the environment, the contour length lower limit may be a fixed value.
(Selection number determination means)

The contour selection number 84 functions as selection number determining means for defining the number of contours to be selected. By defining the upper limit of the number of contours used as a pattern model, the pattern model can be simplified, the processing amount can be reduced, and the search time can be shortened. In addition, if the number of selections is increased, the processing amount increases, but more accurate search results can be expected. By setting the number of contour selections to a fixed value (for example, 50), the setting operation can be simplified as described above.
(Selection order determination means)

  The contour registration order 85 functions as a selection order determination unit that can switch the contour selection order between the ascending order and the descending order of the contour length. Accordingly, it is possible to perform more flexible and highly accurate image processing by setting an appropriate selection method in ascending order or descending order of length according to the image to be subjected to image processing.

  By setting the above setting items, from among a plurality of contours, while filtering an extremely short contour that is close to noise, it is possible to select from the shortest one in order, effectively eliminating noise components, An effective pattern model can be constructed by appropriately selecting segments that affect the positioning accuracy.

For example, consider an example in which a segment constituting a pattern model is selected by setting a pattern window PW for a registered image in which different characters and numbers are displayed in a grid-like frame as shown in FIG. . In the case of the setting as shown in FIG. 70, the contour registration order 85 is set to select “descending order of length”, that is, from the longest to the shortest, so the segment of the frame portion as shown in FIG. SG _W will be much selected, small selection of segments SG _M portions of letters or numbers that are important to identify, can not be expected accuracy of the positioning in this state.

On the other hand, by changing the setting so that the contour registration order 85 is selected in the “ascending order of length”, that is, from the shortest to the longest as shown in FIG. 71, the characters in the frame as shown in FIG. become segment SG _M numbers is selected lot, can build a pattern model having the contour information suitable for such registered image.

  In the example of FIG. 68, each setting item can be set individually for each of the coarse search and the fine positioning, but can be set by only one of them, and an image processing program can be set for a specific item. Or it is good also as a structure which prescribe | regulates by the image processing apparatus side and prohibits the adjustment by a user. In particular, operability can be improved by reducing the number of setting items so that even a user who is not familiar with the operation can easily use it.

  In addition to the segment, other contour information such as a chain can be used as the contour. For example, when the chain obtained by the chain creation means 63 is used as contour information without approximating a line segment or an arc segment, it can be selected from a short chain as described above as the chain selection criterion. The selection order can be switched in ascending order and descending order of the length, and a method such as eliminating a chain shorter or longer than a predetermined threshold can be applied, and the same effect can be obtained. Next, a specific procedure for sorting in order of contour length will be described based on the flowcharts of FIGS. 73 and 74.

  First, the case of sorting using the segment length will be described with reference to FIG. First, the contour is extracted. Here, first, in step S7301, a sobel is performed on the registered image to obtain an edge angle image and an edge strength image, respectively, and in step S7302, edge points are thinned using the edge angle image and the edge strength image. Find contour points. Specifically, after an edge angle image and an edge strength image are created by the edge angle / edge strength image creation means 60 of the contour extraction means 62, the fine line of the edge points is subjected to a non-maximum point suppression process of the edge strength by the thinning means 61. To do. In step S7303, a chain is created by the chain creation means 63. Specifically, the edge chaining means 64 connects adjacent edge points to create a chain. Further, as necessary, the chain filter means 66 performs filtering with various feature quantities of the chain.

  In step S7304, a segment is created. Here, the edge chain segmenting means 65 of the segment creating means 68 creates a segment in which each chain is approximated by a line segment or an arc. In step S7305, short segments are filtered. Here, the segment selection means 67 serves as the contour length lower limit setting means, and removes those whose segment length is equal to or less than the lower limit value and removes noise components.

  Finally, in step S7306, the segments are sorted by segment length, and the segments constituting the pattern model are selected in order from the shortest segment length. Here, the segment selection unit 67 functions as a contour sort unit that sorts the contours in the long order, sorts the segments in the length order, and further selects the selection order defined by the selection order determination unit, in this case, the ascending order of the segment lengths, that is, short. The number specified by the selection number determining means is selected in order from the one. As a result, a pattern model suitable for the registered image as shown in FIG. 72 is constructed.

  On the other hand, an example of using the chain as it is without using a segment as an outline will be described based on the flowchart of FIG. Here, the procedure from extracting contour points to creating a chain is the same as in FIG. That is, first, in step S7401, a sobel is performed on the registered image to obtain an edge angle image and an edge intensity image, and in step S7402, edge points are thinned using the edge angle image and the edge intensity image. Then, contour points are obtained, and a chain is created in step S7403.

  In step S7404, the short chain is filtered. Here, in accordance with the lower limit set by the contour length lower limit setting means, the chain filter means 66 of the chain creating means 63 excludes chains shorter than the contour length lower limit 83. After that, in step S7405, the chains are sorted in order of chain length, and the chains constituting the pattern model are selected in order of increasing chain length. Here again, the chain filter means 66 functions as the contour sorting means, sorts the chains in order of length, and further defines the selection order defined by the selection order determination means, in this case the selection number determination means in ascending order of the chain length. Select as many chains as possible. As a result, a pattern model suitable for a registered image as shown in FIG. 72 is constructed with a chain.

With this method, the process of approximating the segment once can be omitted, and the process can be simplified accordingly. On the other hand, the chain becomes a connected body of random line segments that are not approximated by a certain geometric figure such as a line segment or an arc, and each subsequent process becomes complicated. Which one to select is determined depending on whether the registered image is a simple figure, the detection accuracy of the edge point, and the like.
(Filter long contours)

In the above example, the sorting is performed in the order of the length of the contour. However, the pattern model can be constructed by eliminating the long contour without sorting. Hereinafter, this method will be described based on the user interface screen of FIG. FIG. 75 is also an image diagram showing a user interface of a setting screen 300 for setting a pattern feature selection function of a registered image in the image processing program. In this screen, an edge length upper limit 86, in addition to an edge strength upper limit 81, an edge strength lower limit 82, and a contour length lower limit 83 are listed as setting items. Since the edge strength upper limit 81, the edge strength lower limit 82, and the contour length lower limit 83 are the same as those in FIG. 68 described above, detailed description thereof is omitted.
(Contour length upper limit setting means)

  The contour length upper limit 86 functions as contour length upper limit setting means for setting the upper limit value of the contour. That is, the contour longer than the upper limit value defined by the contour length upper limit 86 is filtered. As a result, the long contour can be intentionally excluded and only the short contour can be left to construct the pattern model. As a result, the same effect as the case where the short contour is preferentially selected can be obtained.

  In the example of FIG. 75, the selection number determining means is not provided, and a preset contour having a predetermined value is automatically selected. However, it is also possible to provide a selection number determining means and set the number of contours manually by the user.

In this way, a pattern model in which short contours are preferentially selected can be constructed without sorting in length order, and an effective pattern search can be realized even for registered images as shown in FIG. For example, under the setting conditions as shown in FIG. 76, a pattern model in which many frame segments SG _W are selected as shown in FIG. 77 is used, but the contour length upper limit 86 is set to 100 to 20 as shown in FIG. by changing, it can be constructed laden pattern model segments SG _M letters and numbers Attempts on as shown in Figure 79.

  Here, the procedure for filtering a long contour will be described based on the flowcharts of FIGS. First, FIG. 80 illustrates a case where a segment is used as an outline. In this method, as in the case of sorting by the above-described contour length, contour points are extracted to create a chain. That is, first, in step S8001, a sobel is performed on the registered image to obtain an edge angle image and an edge intensity image, and then in step S8002, the edge point is thinned using the edge angle image and the edge intensity image. Find contour points. Further, a chain is created in step S8003, and then a segment is created in step S8004.

  Finally, in step S8005, the long segment and the short segment are deleted. Here, the segment selection means 67 serves as the contour length lower limit setting means, and in addition to excluding those whose segment length is equal to or lower than the lower limit value, the segment selection means 67 also functions as a contour length upper limit setting means. Also delete longer segments. As a result, it is possible to select a short segment after removing the noise component, so that a pattern model including a segment effective for positioning is constructed in the case shown in FIG. Although the number of selected segments is a fixed value, as described above, the selection number determining means can be separately provided and manually set by the user.

  An example in which a segment is configured with a chain instead of a segment will be described with reference to FIG. Again, the procedure from extracting contour points to creating a chain is the same as in FIG. 74 described above. That is, in step S8101, the registered image is subjected to a sobel, and an edge angle image and an edge strength image are obtained. In step S8102, the edge point is thinned using the edge angle image and the edge strength image to obtain a contour point, and step S8103 is obtained. Create a chain with

In step S8104, the long chain and the short chain are deleted. Here, the chain filter means 66 functions as a contour length upper limit setting means as a contour length lower limit setting means as a contour length lower limit setting means, and also functions as a contour length upper limit setting means, and also deletes chains longer than the contour length upper limit 86. As a result, it is possible to select a short chain after removing noise components, so that a pattern model including a chain effective for positioning a registered image as shown in FIG. 79 can be constructed. The number of chains to be selected is a fixed value. However, as described above, the number of selections can be manually set by a user by separately providing a selection number determining means.
(Combination with segment selection function considering segment direction)

  Furthermore, the pattern feature selection function can be used in combination with the above-described segment selection function in consideration of the positioning direction or angle. That is, after sorting the segments in order of length or after filtering long segments, the method shown in the flowchart of FIG. 50 or the like is used as a method for determining which segment is selected for pattern model construction. A method of selecting a conjugate segment close to the orthogonal direction with respect to the segment can be adopted. This method can be used for segment selection and does not apply to chains. This is because the chain is not approximated by a line segment or an arc, and thus does not have an angle or direction like a segment.

  Hereinafter, an example in which the normal direction of a segment is considered when selecting a segment after sorting or filtering will be described with reference to the flowcharts of FIGS. First, FIG. 82 shows an example of sorting by segment length. Again, the procedure from extracting contour points to create chains and segments, filtering short ones of the obtained segments, and sorting them is the same as in FIG. 73 described above. That is, first, in step S8201, a sobel is performed on the registered image to obtain an edge angle image and an edge intensity image, and then in step S8202, edge points are thinned using the edge angle image and the edge intensity image. Find contour points. Further, a chain is created in step S8203, then a segment is created in step S8204, and then a short segment is filtered in step S8205. In step S8206, the filtered segments are sorted by segment length and arranged in ascending order.

  In this state, in step S8207, selection is performed in order from the shortest segment while considering the normal direction of the segment. Here, a specific procedure after S8206 in FIG. 82 is shown in the flowchart in FIG. This flowchart is substantially the same as the flowchart of FIG. 50 except that in step S8401, the sort order is not the descending order of the segment length, but the ascending order in which the order is ascending, and the shortest segment is selected in step S8402. The difference is that it is selected as a segment candidate and set as a reference segment, and in step S8404-1, the shortest segment is selected as a segment candidate and set as a conjugate segment. The rest is the same as in FIG. 50, and detailed description thereof is omitted.

  According to this method, a particularly short segment is effective for a registered image effective for positioning, and segments having a relationship of intersecting normal angles are consciously selected by alternately selecting substantially orthogonal segments. Since the selected pattern model can be constructed, stable positioning results in the vertical and horizontal directions can be expected.

Similarly, an example in which the normal direction of the segment is taken into account in the method of filtering a long segment will be described based on the flowchart of FIG. Again, the procedure from extracting contour points to creating chains and segments and filtering the short and long segments of the obtained segments is the same as in FIG. That is, first, in step S8301, a sobel is performed on the registered image to obtain an edge angle image and an edge intensity image, and then in step S8302, edge points are thinned using the edge angle image and the edge intensity image. Find contour points. Further, a chain is created in step S8303, then a segment is created in step S8304, and then a short segment and a long segment are filtered in step S8305. In this state, in step S8306, in the same manner as in FIG. 84, selection is performed in order from the shortest segment while considering the normal direction of the segment. In this case, the work of sorting by segment length in step S8306 is necessary anyway, and from this point, it can be said that the method of FIG. 82 described above is efficient. The other effects are the same as in FIG. 82. As a result of selecting conjugate segments close to the orthogonal direction with respect to the reference segment, segments with dispersed angles are selected for the pattern model. It can be improved.
(Improved stability of least squares method)

  Next, a method for improving the stability of the least square method performed in the precision positioning process will be described. The least square method is roughly classified into a linear type and a non-linear type. Among these, in the linear least square method, a solution can be uniquely obtained theoretically. On the other hand, in the non-linear least square method, since approximation is generally performed up to a quadratic expression, the approximate solution is not always correct. In some cases, in the fine positioning process, the position may be moved or rotated in a direction in which the accuracy is worse than the position of the detection candidate obtained by the coarse search. For example, when precise positioning is performed on a highly symmetrical figure such as the circle shown in FIG. 85 (the center coordinates of the outer circle and the center coordinates of the inner circle are slightly different), the center of the circle is used as the rotation axis. Since the error value hardly changes even if it is rotated, it may be rotated in the direction opposite to the original direction of rotation, and the change in angle is large, resulting in a large deviation in the amount of translation. There is also.

  In a general solution of a nonlinear least square method approximated by a quadratic equation, an error function E (pi) is approximated to a trial parameter set pi that is a variable of the least square method, and a quadratic power of a trial parameter. The approximate error function approximated in the above is created, and the procedure of finding a set of trial parameters pi that minimizes the error function using the approximate error function is taken.

  In such a nonlinear least square method, the following inverse Hessian method has been proposed as a solution for obtaining a solution having a small error value.

  This is a method of obtaining an approximate error function, obtaining a minimum trial parameter set from the error function, and obtaining a more accurate trial parameter set at the next stage at a time. However, when a solution with a small error value is obtained using the inverse Hessian method, the following problems may occur. This will be described with reference to FIG. In each of FIGS. 86A and 86B, a solid line indicates an error function, and a broken line indicates an approximate error function obtained by approximating the error function. P1 provided on the curve indicating the error function in FIGS. 86 (a) and 86 (b) indicates the position (x, y, θ) obtained by the above-described rough search, and is in the vicinity of this P1. What is obtained as a quadratic function based on the value of the error function is a broken-line quadratic curve indicating an approximate error function indicated by a broken line.

  Further, in the case shown in FIG. 86 (a), it is a diagram for explaining the case where the inverse Hessian method is appropriately applied, and the position of the smallest error value of the broken-line quadratic curve indicating this approximate error function, that is, Since P2 is close to the position P having the smallest error value of the error function indicated by the solid line, a case where a more accurate position P2 having a smaller error value is obtained is shown.

  On the other hand, in the case shown in FIG. 86 (b), it is a diagram for explaining a case where the inverse Hessian method has acted improperly, and is the position of the smallest error value of the dashed quadratic curve indicating the approximate error function, that is, Since P2 is obtained as a position away from the position P having the smallest error value of the error function indicated by the solid line, a case where a position P2 having an incorrect error value is obtained is shown.

Of the above, in the case shown in FIG. 86 (b), if an attempt is made to obtain a solution as it is by the inverse Hessian method, a large displacement of the trial parameter may be obtained as described above, and as a result, the precision of precision positioning There was a problem that decreased. The inventor of the present application has found a technique for restricting movement and rotation with respect to the pattern model as a technique for suppressing the occurrence of the case where the reverse Hessian method acts inappropriately as shown in FIG. 86 (b). In other words, a new term is added to the error function as a solution of the least square method based on the inverse Hessian method. Specifically, in addition to the above-described term relating to the distance (first error function), a term (second error function E _d ) that increases the error value due to the displacement from the trial parameter is added. Thereby, an extremely large change is suppressed by the second error function E _d , and an appropriate approximation as shown in FIG. 86A can be expected. In this way, conditions for converging in the correct direction in the precision positioning step can be added to avoid rotation and divergence in an unintended direction and improve positioning reliability. The second error function is a function of P _i −P _0i when the trial parameter of the least square method is P _i . An example of simultaneous equations of the error function E (P) obtained as a result is shown in the following equation.

In the above equation, the overall error function E (P) is the first error function E _o indicating the distance between each segment and its corresponding edge point, and the amount of change when the trial parameter in this first error function is changed. Is the second error value, and is expressed by the sum of the second error function E _d for calculating the cumulative value of the second error value. In this way, in the calculation of the least square method, in addition to the term that becomes the minimum when it matches the ideal position, the second error function E _d is added, so that the solution is solved in a direction that both terms become smaller. Since an extremely large change in the trial parameter is suppressed, it is possible to avoid a situation such as rotation or divergence in the wrong direction during precise positioning, and an advantage that the processing result is stable can be obtained.

  A pattern model positioning method, an image processing apparatus, an image processing program, and a computer-readable recording medium in image processing according to the present invention include position detection such as a work coordinate position and a rotation angle in image processing used in the FA field, It can be suitably applied to position measurement, recognition, identification, determination, inspection, etc., such as outer diameter, inner diameter, and width. For example, it can be used for positioning an electrode for bonding an electrode of an IC.

It is a block diagram which shows an example of an image processing apparatus. It is a schematic diagram which shows the outline | summary of the operation | movement at the time of the operation | movement at the time of registering and searching using a pattern model. It is a flowchart which shows the outline | summary of the operation | movement at the time of pattern model registration. It is a flowchart which shows the outline | summary of the operation | movement at the time of the driving | running which actually searches. It is an image figure which shows the user interface screen which sets the reduction rate in manual reduction rate determination mode. It is a schematic diagram which shows the outline | summary of an edge angle bit. It is a schematic diagram which shows a mode that an edge angle bit image is produced | generated and compressed. FIG. 8A is a schematic diagram showing an example of each pattern model. FIG. 8A shows a local search pattern model, FIG. 8B shows a global search pattern model, and FIG. 8C shows a precise positioning pattern model. Show. It is a flowchart which shows the outline | summary of the procedure of the search at the time of driving | operation. It is a schematic diagram which shows the concept of the search at the time of driving | operation. It is a schematic diagram which shows a mode that an edge angle bit changes as a result of rotating a pattern model. It is a flowchart which shows the creation procedure of the pattern model at the time of registration. It is an image figure which shows the user interface screen which sets these automatically according to the priority of search precision and search time. It is a flowchart which shows the procedure which registers the pattern model for precise positioning. It is a flowchart which shows the procedure which performs the pre-processing with respect to a search object image at the time of a driving | operation. It is an image figure which shows the pattern model in which the corresponding point search line was set. It is a schematic diagram which shows the state which thinned out the pattern model of FIG. It is a flowchart which shows the procedure which performs a pattern search at the time of operation. FIG. 19A is an example of a search target image, and FIG. 19B is an image diagram showing a reduced search target image reduced to the same magnification as the registered image of FIG. 6A. It is a flowchart which shows the procedure of the pattern search at the time of operation. It is a schematic diagram which shows the corresponding edge point search process which calculates | requires a corresponding edge point. It is a schematic diagram explaining the ripple phenomenon of an edge position. It is a schematic diagram which shows an example of the registration image which may be difficult for selection of a corresponding point search line. It is a schematic diagram which shows the state which set the corresponding point search line automatically with respect to the pattern of FIG. It is a schematic diagram which shows the result of having filtered the corresponding point search line from FIG. It is a flowchart which shows the procedure of a corresponding point search line filter process. It is an image figure which shows the state which set the corresponding point search line whose length from a base point is equal. It is an image figure which shows the state which set the corresponding point search line from which the length from a base point differs. It is a schematic diagram explaining the procedure which calculates | requires the coordinate of a corresponding edge point. It is a schematic diagram which shows the edge angle image comprised by 4 pixels of pixel ad. It is a schematic diagram which shows the edge angle division which defines an edge angle bit. It is a schematic diagram which shows the edge angle bit image which converted the edge angle image of FIG. It is a schematic diagram which shows the edge angle bit reduced image which reduced the edge angle bit image of FIG. It is a schematic diagram explaining a mode that the original edge angle bit image is reduced in units of 2 × 2 pixels. It is a schematic diagram explaining a mode that it expands after the reduction process of FIG. It is a schematic diagram explaining a mode that the original edge angle bit image is reduced to 1/2. It is a schematic diagram explaining a mode that the original edge angle bit image is reduced to 1/3. It is a conceptual diagram which shows the pattern model before parallel processing. It is a conceptual diagram which shows the pattern model after parallel processing. It is a conceptual diagram which shows the data of pattern model registration. It is an image figure which shows an example of the pattern model which expressed the circle | round | yen notched partially by the arc segment and the line segment. FIG. 42 is an image diagram showing a state in which a rough search is performed on the input image using the pattern model of FIG. 41 and a certain degree of positioning is performed. FIG. 43 is an image diagram showing a state in which fine positioning is performed by applying a least square method using a distance between a point and an arc as an error function from the state of FIG. It is an image figure which shows the example which performed the corresponding point search line production | generation process with respect to the pattern model of FIG. FIG. 45 is an image diagram illustrating a state in which a rough search is performed on the search target image with the pattern model of FIG. 44 and the pattern models are overlaid at the determined position and orientation. It is a schematic diagram showing an edge vector ( _Ex , _Ey ) having an edge strength E _M and an edge angle θ _E. It is a conceptual diagram which shows the state which performed the rough search with respect to the circular shaped workpiece | work, and completed the positioning to some extent. FIG. 48 is a conceptual diagram illustrating a state in which precise positioning is performed from FIG. 47 and an attempt is made to superimpose a pattern model on a search target image. It is a schematic diagram explaining the weighting process when there are a plurality of corresponding edge point candidates. It is a flowchart which shows the selection procedure of the segment considered in the positioning direction. It is a schematic diagram explaining a mode that the setting of an angle range is updated in FIG. 52A and 52B are schematic diagrams illustrating an example of reduction processing using saturation addition, in which FIG. 52A is an edge angle image in which each pixel has an edge angle, and FIG. 52B is 8-bit edge angle bit data of each pixel. The schematic diagram which shows the edge angle division to represent is shown, respectively. It is an image figure which shows an example of a binary image. It is a graph which shows the pixel value of FIG. It is a graph which shows the intensity | strength change of the edge of an image with high sharpness. It is a graph which shows the pixel value of a blurred image. It is a graph which shows the intensity | strength change of the edge of an image with low sharpness. It is a schematic diagram which shows the calculation method of the subpixel coordinate which concerns on patent document 2. FIG. It is a graph which shows the relationship between the error of edge strength, and a subpixel position. It is a flowchart which shows the procedure which determines the image data reduction rate based on the sharpness of an edge point. It is a graph which shows the model function of an edge. It is a flowchart which shows the operation | movement at the time of registration using an image data reduction ratio. It is a flowchart which shows the operation | movement at the time of the driving | operation using the image data reduction rate. It is a schematic diagram which shows the edge strength of 3 points | pieces B, C, and F which adjoin. It is a schematic diagram which shows the procedure which calculates | requires the coordinate of a corresponding edge point using a near edge point. FIG. 66 is a flowchart showing a procedure for obtaining coordinates of corresponding edge points in FIG. 65. FIG. It is an image figure which shows a mode that a pattern model is set to the target image by which the character was displayed in the frame. It is an image figure which shows the user interface screen which sets the pattern feature selection function sorted in order of length in an image processing program. It is an image figure which shows the segment selected as a pattern model in the registration image in which a different character and number are displayed in the checkered frame. FIG. 69 is an image diagram showing a state in which the contour registration order is set to “length descending order” on the user interface screen of FIG. 68. FIG. 71 is an image diagram showing a state in which the contour registration order is set to “length ascending order” on the user interface screen of FIG. 70. FIG. 70 is an image diagram showing a state in which a segment is selected under the setting condition of FIG. 71 in the registered image of FIG. It is a flowchart which shows the procedure sorted by the length of a segment. It is a flowchart which shows the procedure sorted in order of the length of a chain. It is an image figure which shows the user interface screen which sets the pattern feature selection function which filters a long outline in an image processing program. FIG. 76 is an image diagram showing a state where the contour length upper limit is set higher on the user interface screen of FIG. 75. FIG. 70 is an image view showing a state where a segment is selected under the setting condition of FIG. 76 in the registered image of FIG. 69. FIG. 76 is an image diagram showing a state where the contour length upper limit is set lower on the user interface screen of FIG. 75. FIG. 70 is an image diagram showing a state in which a segment is selected in the registered image of FIG. It is a flowchart which shows the procedure which filters a long segment. It is a flowchart which shows the procedure which filters a long chain. It is a flowchart which shows the procedure which selects a segment after sorting by segment length. It is a flowchart which shows the procedure which selects a segment after filtering. It is a flowchart which shows the selection procedure of the segment which considered the normal line direction of the segment. It is a schematic diagram which shows the example which performs precise positioning with respect to a highly symmetrical figure. It is a schematic diagram which shows a mode that an error function is approximated by the inverse Hessian method. It is a schematic diagram which shows the concept of a pyramid search.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 100A ... Image processing apparatus main body 200, 300 ... Image processing program setting screen 1 ... Image input means 2 ... Operation means 3 ... Display means 4 ... Storage means 5 ... Output means 6 ... Calculation means 60 ... Edge angle Edge strength image creation means 61 ... thinning means 62 ... contour extraction means 63 ... chain creation means 64 ... edge chaining means 65 ... edge chain segmentation means 66 ... chain filter means 67 ... segment selection means 68 ... segment creation means 69 ... edge angle bit image creation means 70 ... pattern model construction means 71 ... coarse search means 76 ... fine positioning means 77 ... image reduction means 78 ... edge angle bit image reduction means 81 ... edge strength upper limit 82 ... edge strength lower limit 83 ... contour length Lower limit 84 ... Contour selection number 85 ... Contour registration order 86 ... Contour length upper limit RPM Reduction patterns window ROI ... reduced image to be searched EB ... edge angle bit image REB ... edge angle bit reduced image EABR ... reduced image KT ... base TL, TTL _1, TTL ₂ ... corresponding point search line TT ... corresponding edge point a, b, c, d ... vertex e ... provisional corresponding edge point f ... pair point TT _A , TT _B , TT _C ... corresponding edge point candidate RI ... registered image PW ... pattern window PM ... pattern model OI ... search target images KT ₁ , KT ₂ ... Base points SG ₁ , SG ₂ , SG _W , SG _M ... Segment d ₁ ... Distance d ₂ between first corresponding edge point candidate and base point ... Distance E _M between second corresponding edge point candidate and base point ... Edge strength θ _E Edge angle IE Input edge point SL Tangent SR Area EMc Edge strength value EMf of target edge point Estimated edge intensity value EMb indicated by arrow Estimated edge indicated by a circle Di-strength values EMf ₁ , EMb ₁ ... Cross point edge feature EMf ₂ , EMb _2. Diagonal edge point feature EAc... Angle TP1, TP2, TP3. Edge point sub-pixel position

Claims (7)

When searching for a search object similar to the registered image from the search target image using the pattern model corresponding to the registered image registered in advance and performing positioning, the attention point and the correspondence corresponding to the attention point The error value used in the least square method calculation is calculated based on the positional relationship with the point, the simultaneous equations obtained by the least square method are solved from these values, and positioning is performed with higher accuracy than the initial position given with a predetermined accuracy. A pattern model positioning method in image processing,
A pattern in image processing, wherein the least square method includes a term that is minimum when it matches an ideal position, and a term that is minimum when it matches a trial parameter that is a variable of the least square method. Model positioning method. When searching for a search object similar to the registered image from the search target image using the pattern model corresponding to the registered image registered in advance and positioning it, the initial position is given with a predetermined accuracy. A pattern model positioning method in image processing for positioning with high accuracy,
Acquiring a search target image, acquiring an initial position corresponding to a registered image included in the search target image, and arranging the pattern model so as to overlap the search target image according to the initial position, Obtaining individual corresponding edge points on the search target image corresponding to each segment constituting the model;
A first error function for calculating a cumulative value of the first error value in the least square method in which the distance between each segment and its corresponding edge point is defined as a first error value;
A second error function for calculating a cumulative value of the second error value by setting the change amount when changing the trial parameter which is a variable in the first error function as a second error value;
And performing precise positioning with higher accuracy than the accuracy of the given initial position so that the sum of the first error function and the second error function is minimized. When,
A pattern model positioning method in image processing, comprising: A pattern model positioning method in image processing according to claim 1 or 2,
Positioning of a pattern model in image processing, wherein the second error function is a function of P _i −P _0i where P _{i is} a trial parameter of the least square method and P _0i is an initial value of the trial parameter. Method. A method for positioning a pattern model in image processing according to claim 3,
The second error function is
A pattern model positioning method in image processing, characterized by: When searching and positioning a search object similar to the registered image from the search target images using a pattern model corresponding to the registered image registered in advance, the initial position given with a predetermined accuracy is used. Is an image processing apparatus that performs positioning with high accuracy,
Image input means for acquiring registered images and search target images;
Contour extraction means for extracting a contour from a registered image acquired by the image input means;
A chain creating unit that extracts a plurality of edge points from the contour extracted by the contour extracting unit and creates a continuous chain by connecting adjacent edge points among the plurality of extracted edge points;
Segment creation means for creating segments approximated by line segments and / or arcs for one or more chains created by the chain creation means;
A registered image pattern in which a plurality of base points are set on the segment created by the segment creating means, and a corresponding point search line having a predetermined length passing through the base point and substantially orthogonal to the contour is assigned to each base point. Pattern model construction means for constructing a model,
An initial position corresponding to a registered image included in the search target image acquired by the image input unit is acquired, and the pattern model is arranged so as to overlap the search target image according to the initial position, and the pattern model Each corresponding edge point on the search target image corresponding to each segment constituting the segment is calculated, a least square method defined by using a distance between each segment and the corresponding edge point as a first error value is calculated, and the error is calculated. Precision positioning means for performing precise positioning with higher accuracy than the initially given initial position by making the cumulative value of the value minimum;
With
A least squares error function used in the precision positioning means includes a first error function that defines the first error value, and a second error value that is a change amount from a trial parameter of a variable in the least square method. And an error function. When searching and positioning a search object similar to the registered image from the search target images using a pattern model corresponding to the registered image registered in advance, the initial position given with a predetermined accuracy is used. Is an image processing program for positioning with high accuracy,
An image input function for acquiring registered images and search target images;
A contour extraction function for extracting a contour from a registered image acquired by the image input function;
A chain creation function for extracting a plurality of edge points from the contour extracted by the contour extraction function and creating a continuous chain by connecting adjacent edge points among the extracted plurality of edge points;
A segment creation function for creating segments approximated by line segments and / or arcs for one or more chains created by the chain creation function;
A registered image pattern in which a plurality of base points are set on the segment created by the segment creation function, and a corresponding point search line having a predetermined length passing through the base point and substantially orthogonal to the contour is assigned to each base point. Pattern model construction function to construct a model,
An initial position corresponding to a registered image included in the search target image acquired by the image input function is acquired, and the pattern model is arranged to overlap the search target image according to the initial position, and the pattern model Each corresponding edge point on the search target image corresponding to each segment constituting the segment is calculated, a least square method defined by using a distance between each segment and the corresponding edge point as a first error value is calculated, and the error is calculated. A precision positioning function for performing precise positioning with higher accuracy than the initially given initial position by making the cumulative value of the value minimum,
Is realized on a computer,
A least squares error function used in the precision positioning function includes a first error function that defines the first error value, and a second error value that is a change amount from a trial parameter of a variable in the least square method. An image processing program comprising an error function.   A computer-readable recording medium on which the program according to claim 6 is recorded.