JP2009250739A - Method for processing image and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To immediately and accurately detect a position of a rotated mark in an image. <P>SOLUTION: In step S1, an area of an input image having a high degree of similarity to a template is detected by matching using a projection pattern so as to extract a center of the detected area as a candidate point. In step S2, regarding each candidate point, an image of a plurality of areas which have predetermined shapes each centering around the candidate point and have different rotational angles is extracted as a matching target image. In step S3, matching between the extracted matching target image and the template is performed so as to obtain a matching score indicative of a degree of similarity. In step S4, a position and a rotational angle of a mark of the input image is detected on the basis of the result of the matching. This method can be applied to, for example, a device for detecting a position of an image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing method and program, and more particularly, to an image processing method and program for detecting the position of a predetermined mark in an image.

  Conventionally, in a process of printing a circuit pattern on a substrate or the like, the substrate is positioned using an alignment mark that is printed in advance on the substrate. Template matching is often used as one of means for detecting the position of the alignment mark on the substrate (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2005-300322

  However, in the conventional template matching, in order to be able to accurately detect the position of the alignment mark even if the alignment mark is rotated due to the rotation of the substrate or the like, for example, in addition to the alignment mark facing the normal direction, In addition, since a plurality of templates indicating rotated alignment marks are prepared and matching is performed on the entire image obtained by photographing the substrate by the number of templates, the processing takes time.

  The present invention has been made in view of such a situation, and makes it possible to quickly and accurately detect the position of a rotated mark in an image.

  An image processing method according to an aspect of the present invention detects an input image region having a high similarity to a template by matching a pixel signal pattern within a predetermined area of the input image with a template signal pattern, and the center of the detected region Candidate point extracting step for extracting candidate points, and for each of the candidate points, images of a plurality of regions having a predetermined shape centered on the candidate point and having different rotation angles rotated around the candidate point A matching target image extracting step for extracting a matching target image, a matching step for performing matching between the extracted matching target image and a template, obtaining a matching score representing a similarity between the matching target image and the template, and a matching score being maximum The position of the target image to be matched is the mark position in the input image And a mark detecting step of detecting in.

  The program according to one aspect of the present invention detects a region of an input image having a high similarity to the template by matching a signal pattern of a pixel in a predetermined area of the input image with a signal pattern of the template, and selects the center of the detected region as a candidate Candidate point extraction step for extracting points, and for each of the candidate points, matching images of a plurality of regions having a predetermined shape centered on the candidate point and having different rotation angles rotated around the candidate point A matching target image extracting step for extracting as a target image, a matching step for performing matching between the extracted matching target image and a template and obtaining a matching score representing a similarity between the matching target image and the template, and a matching that maximizes the matching score The position of the target image is the mark position in the input image. The process including the mark detection step of detecting causing a computer to execute Te.

  In one aspect of the present invention, a region of an input image having a high degree of similarity to a template is detected by matching a signal pattern of a pixel in a predetermined area of the input image and a signal pattern of the template, and the center of the detected region is a candidate. For each of the candidate points, images of a plurality of regions having a predetermined shape centered on the candidate point and having different rotation angles rotated around the candidate point are extracted as matching target images. , Matching the extracted matching target image with the template, obtaining a matching score representing the similarity between the matching target image and the template, and detecting the position of the matching target image having the maximum matching score as the mark position in the input image Is done.

  According to one aspect of the present invention, the position of a rotated mark in an image can be detected quickly and accurately.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of an image processing apparatus to which the present invention is applied. The image processing apparatus 1 shown in FIG. 1 includes, for example, a position where a mark that matches an image (hereinafter referred to as a mark) indicated in a given template exists in an image input from the outside (hereinafter referred to as an input image). , And an angle at which the mark is rotated (hereinafter also referred to as a rotation angle) is detected. For example, the image processing apparatus 1 detects the position and rotation angle of an alignment mark used for positioning the substrate in an image obtained by photographing the substrate.

  As will be described later with reference to FIGS. 4 and 5, the candidate point extraction unit 11 of the image processing apparatus 1 may have a mark indicated by the template in the input image, and is an area to be searched for a mark. Candidate points indicating the positions of the candidate points are extracted, and information indicating the coordinates of the extracted candidate points is supplied to the matching target image extraction unit 12.

  As will be described later with reference to FIG. 6 and the like, the matching target image extraction unit 12 is an area having a predetermined shape centered on the candidate point and rotated about the candidate point. Images in a plurality of regions having different angles (hereinafter referred to as matching target images) are extracted from the input image. The matching target image extraction unit 12 supplies the matching unit 13 with information indicating the extracted matching target images and the coordinates of candidate points corresponding to the rotation angles of the matching target images.

  As will be described later with reference to FIGS. 7 and 8, the matching unit 13 performs matching of the extracted matching target image and the template in units of pixels, the matching result for each matching target image, and each matching target image. Information indicating the coordinates of the candidate point corresponding to the rotation angle is supplied to the mark detection unit 14.

  As will be described later with reference to FIG. 2 and the like, the mark detection unit 14 detects the position and rotation angle of the mark in the input image based on the matching result of each matching target image and the template in units of pixels. The mark detection unit 14 supplies information indicating the detected mark position and rotation angle to the coordinate conversion unit 15.

  As will be described later with reference to FIG. 9 and the like, the coordinate conversion unit 15 converts the coordinates of the mark position represented in the coordinate system of the matching target image into coordinates in the coordinate system of the input image. The coordinate conversion unit 15 outputs the coordinates of the mark position converted into the coordinate system of the input image and information indicating the rotation angle of the mark to the outside.

  In the following, the coordinate system of an image other than the matching target image (for example, an input image, a template, etc.) is set such that the upper left corner of the image is the origin, the horizontal direction of the image is the x-axis direction, and the vertical direction is the y-axis direction. A coordinate system in which the right direction is the positive direction of the x axis and the downward direction is the positive direction of the y axis is used. Further, the coordinate system of the matching target image is set to the origin of the center of the matching target image, the horizontal direction of the matching target image is the x′-axis direction, the vertical direction is the y′-axis direction, and the right direction is a positive x′-axis. Let the direction and the upper direction be a coordinate system with the positive direction of the y ′ axis.

  Next, image processing executed by the image processing apparatus 1 will be described with reference to the flowchart of FIG. This process is started, for example, when an input image to be processed is input to the candidate point extraction unit 11. Hereinafter, it is assumed that a template indicating a mark to be searched is given in advance to the image processing apparatus 1 and the shape of the template is rectangular.

  In step S1, the candidate point extraction unit 11 extracts candidate points by matching using a projection pattern. Note that a projection pattern is a one-dimensional image that represents the total value of pixel values for each line in the x-axis direction (horizontal direction) in a two-dimensional image, and the distribution of the calculated total value in the y-axis direction (vertical direction). A pixel signal pattern (hereinafter referred to as a projection pattern in the y-axis direction) or a total pixel value for each line in the y-axis direction (vertical direction) is obtained, and the obtained total value in the x-axis direction (horizontal direction) ) Is a one-dimensional pixel signal pattern (hereinafter referred to as a projection pattern in the x-axis direction). That is, the projection pattern indicates a distribution of pixel values in units of lines in the horizontal or vertical direction, and can be said to be a pattern in which an image is projected in the x-axis or y-axis direction.

  Here, a specific example of the projection pattern will be described with reference to FIG. In the template T1 of FIG. 3, square marks M1t to M4t having the same size are shown. The mark M1t and the mark M3t, and the mark M2t and the mark M4t are arranged at the same position in the x-axis direction, and the mark M1t and the mark M2t, and the mark M3t and the mark M4t are arranged at the same position in the y-axis direction. Has been. When the pixel value of the template T1 is 256 gradations, the pixel values of the marks M1t to M4t are 0 (ie, black), and the pixel values of other regions are 255 (ie, white), the x axis of the template T1 The direction projection pattern is represented by a pattern 51. That is, the projection pattern 51 is a binary pulse having a minimum value at the x coordinate of the column in which the mark M1t and the mark M3t, and the mark M2t and the mark M4t exist, and a maximum value at the x coordinate of the other columns. It becomes the upper waveform.

  On the other hand, in the region R1, marks M1r to M4r similar to the marks M1t to M4t of the template T1 are shown. However, the marks M1r to M4r in the region R1 rotate counterclockwise as compared with the marks M1t to M4t in the template T1. As a result, the projection pattern in the x-axis direction of the region R1 is represented by a pattern 52 different from the pattern 51. That is, the projection pattern 52 changes in accordance with the number of pixels included in the marks M1r to M4r in each column, and the value becomes the minimum at the x coordinate of the column where the pixels included in the marks M1r to M4r are the most. The value is the maximum in the x coordinate of the column where the pixels included in M1r to M4r do not exist.

  The candidate point extraction unit 11 detects a region of the input image having a high similarity to the template by matching the projection pattern of the input image and the projection pattern of the template, and extracts the center of the detected region as a candidate point. Here, a specific example of candidate point extraction processing will be described with reference to FIGS. Hereinafter, the size of the input image Gi shown in FIG. 4 is defined as width wg × height hg, and the size of a template (not illustrated) is defined as width wt × height ht.

  First, as shown in FIG. 4, the candidate point extraction unit 11 sets a window W1 having a width wg × height ht at the upper end of the input image Gi, and generates a projection pattern in the x-axis direction of the image in the window W1. To do. Next, as shown in FIG. 5, the candidate point extraction unit 11 sets a window W2 having a width wt × height ht of the same size as the template at the left end of the window W1, and the x-axis of the image in the window W2 The projection pattern in the direction and the projection pattern in the x-axis direction of the template are matched, and a matching score representing the similarity between the image in the window W2 and the template is obtained. The candidate point extraction unit 11 repeatedly performs the same processing while shifting the window W2 to the right by one pixel until the window W2 comes to the right end of the window W1, and the image between each area image in the window W1 and the template. Find matching score. Candidate point extraction unit 11 uses, as a candidate point, a midpoint in a region where the matching score is maximal in window W1 and is equal to or greater than a predetermined threshold (that is, a region where the similarity to the template is equal to or greater than a predetermined threshold). Extract. Therefore, a plurality of candidate points may be extracted in the window W1, or no candidate points may be extracted.

  The candidate point extraction unit 11 repeats the above-described process until the window W1 comes to the lower end of the input image Gi while shifting the window W1 pixel by pixel, and extracts candidate points from the entire input image. The candidate point extraction unit 11 also performs the same process using the projection pattern in the y-axis direction for the y-axis direction, and extracts candidate points. Then, the candidate point extraction unit 11 supplies information indicating the coordinates of the extracted candidate points to the matching target image extraction unit 12. Note that the coordinates of the extracted candidate points are represented by the coordinate system of the input image.

  In addition, since the variation of the value of the projection pattern due to the rotation of the image is small, by performing the matching using the projection pattern, even if the mark is rotated in the input image as compared with the case of performing the matching on a pixel basis, A decrease in matching score between the region where the mark exists and the template is suppressed. In addition, when matching is performed using a projection pattern, the amount of calculation is smaller than when matching is performed in units of pixels. Therefore, it is possible to quickly extract appropriate candidate points.

  In addition to the total value of pixel values for each line, for example, an average value of pixel values for each line can be used as the projection pattern value.

  Returning to FIG. 2, in step S <b> 2, the matching target image extraction unit 12 extracts a plurality of matching target images having different rotation angles around each candidate point. Specifically, the matching target image extraction unit 12 first selects one of the candidate points as a candidate point for attention. The matching target image extraction unit 12 is a rectangular area that is slightly larger (for example, several pixels larger) than the template centered on the target candidate point, and each side is parallel to each side of the input image (hereinafter, An image in a basic extraction area) is extracted from the input image as a matching target image.

  Further, the matching target image extracting unit 12 performs processing for extracting, as a matching target image, an image in an area obtained by rotating the basic extraction area by a predetermined angle around the candidate point of interest using affine transformation. Execute for the rotation angle of. For example, the matching target image extracting unit 12 selects an image in a region obtained by rotating the basic extraction region at an interval of Δθ within a range of ± θ with respect to a vertical line passing through the candidate point of interest centered on the candidate point of interest. Extracted as matching target image. In this case, 2θ ÷ Δθ + 1 matching target images are extracted for each candidate point.

  The matching target image extraction unit 12 performs the same processing for all candidate points, and extracts matching target images having the same number and rotation angle for each candidate point. The matching target image extraction unit 12 supplies the matching unit 13 with information indicating the extracted matching target images and the coordinates of candidate points corresponding to the rotation angles of the matching target images.

  Here, a specific example of the matching target image extraction process will be described with reference to FIG. For example, consider a case where the center Ps1 of the mark M11r on the same input image as the cross-shaped mark M11t shown in the square template T11 is selected as the candidate point of interest. As shown on the left side of FIG. 6, it is assumed that the mark M11r rotates counterclockwise as compared with the mark M11t.

  The matching target image extraction unit 12 first extracts an image in the basic extraction region R13 that is a square region centered on the candidate point Ps1 and is slightly larger than the template T11 as a matching target image. Then, the matching target image extraction unit 12 rotates the basic extraction region R13 counterclockwise by a region R11, counterclockwise by a region R12 rotated by θ2 (where θ2 <θ1), and clockwise by θ2. The image in the region R14 rotated by only the angle R1 and the image in the region R15 rotated clockwise by θ1 are extracted as matching target images. Thereby, five candidate images Gm11 to Gm5 shown on the right side of FIG. 6 are extracted for the candidate point Ps1.

  The reason why the image in the region larger than the template is extracted as the matching target image is that the position of the extracted candidate point does not necessarily coincide with the center of the mark due to the influence of rotation.

  Returning to FIG. 2, in step S <b> 3, the matching unit 13 executes matching of each matching target image and the template in units of pixels. Specifically, the matching unit 13 selects one of the matching target images as the target matching target image. The matching unit 13 performs matching in units of pixels with the template while shifting the region to be matched with the template up and down or left and right one pixel at a time in the target matching target image.

  Here, with reference to FIG. 7, a specific example of the matching process of the matching target image and the template in the pixel unit will be described. As shown in FIG. 7, the matching unit 13 arranges a window W21 having a width wt × height ht of the same size as the template in the upper left corner of the matching target image Gm21, and the image in the window W21 and the pixel unit of the template And a matching score representing the similarity between the image in the window W21 and the template is obtained. The matching unit 13 repeats the same process until the window W21 comes to the right end of the matching target image Gm21 while shifting the window W21 to the right by one pixel. In addition, when the window W21 comes to the right end of the matching target image Gm21, the matching unit 13 shifts the window W21 down by one pixel and returns it to the left end of the matching target image Gm21. It repeats until it comes to the lower right corner of the target image Gm21. Thereby, the matching score between the image in each area | region of the matching target image Gm21 and a template is calculated | required.

  The matching unit 13 executes the above-described processing for all matching target images. The matching unit 13 supplies a matching result for each matching target image and information indicating the coordinates of candidate points corresponding to the rotation angle of each matching target image to the mark detection unit 14.

  The matching result for each matching target image includes a combination of the matching score in each region of the matching target image and the coordinates of the center of each region. For example, when considering the region in the window W21 of the matching target image Gm21 in FIG. 7, the matching score between the image in the region in the window W21 and the template, and the center Rm of the region in the window W21 (FIG. The combination of coordinates in 8) is included in the matching result.

  Note that the coordinates of the center of each region of the matching target image are represented by the coordinate system of the matching target image. For example, as shown in FIG. 8, the coordinates (Xr, Yr) of the center Rm of the region in the window W21 have the candidate point Ps21 as the origin, the horizontal direction of the matching target image Gm21 as the x ′ axis, and the vertical direction It is represented by a coordinate system with y ′ axis.

  Returning to FIG. 2, in step S <b> 4, the mark detection unit 14 detects the mark position and the rotation angle of the input image based on the result of matching in pixel units. Specifically, the mark detection unit 14 selects one of the candidate points as the attention candidate point, and for the attention candidate point, the matching target image including the region where the matching score is maximized and the maximum value of the matching score are determined. Ask. The mark detection unit 14 performs the same processing for all candidate points, and obtains a combination of a matching target image including a region where the matching score is maximized and a maximum value of the matching score, one for each candidate point.

  Further, the mark detection unit 14 compares the maximum value of the matching score obtained for each candidate point, and obtains a matching target image (hereinafter referred to as a mark detection image) including a region where the maximum value of the matching score is maximum. The mark detection unit 14 uses the rotation angle of the mark detection image as the rotation angle of the mark in the input image, and detects the center of the region where the matching score is maximum in the mark detection image as the mark position in the input image. The mark detection unit 14 supplies the coordinate conversion unit 15 with information indicating the coordinates of the center (that is, the candidate point) of the mark detection image and the coordinates of the rotation angle and the mark position in the input image. Note that the coordinates of the center of the mark detection image are represented by the coordinate system of the input image, and the coordinates of the mark position are represented by the coordinate system of the mark detection image.

  In step S5, the coordinate conversion unit 15 converts the coordinates of the mark position into the coordinate system of the input image. Here, with reference to FIG. 9, the details of this coordinate conversion processing will be described. In FIG. 9, the point Pm represents the detected mark position of the input image, and the candidate point Ps31 represents the center of the mark detected image from which the mark position Pm is detected. Further, the rotation angle of the mark detection image is set to −θm (θm in the counterclockwise direction). Further, a coordinate system represented by the x ′ axis and the y ′ axis with the candidate point Ps31 as the center is set as the coordinate system of the mark detection image. Further, the coordinates of the mark position Pm in the coordinate system of the mark detection image are (Xm ′, Ym ′).

  Mark position in a coordinate system (hereinafter referred to as a rotational coordinate system) represented by an x ″ axis and a y ″ axis obtained by rotating the coordinate system of the mark detection image clockwise by θm around the candidate point Ps31 The coordinates (Xm ″, Ym ″) of Pm are obtained by the following equations (1) and (2). Note that the x ″ axis is parallel to the x axis of the coordinate system of the input image, and the y ″ axis is parallel to the y axis of the input image.

Xm ″ = Xm ′ × cos θm−Ym ′ × sin θm (1)
Ym ″ = Xm ′ × sin θm + Ym ′ × sin θm (2)

  If the coordinates of the candidate point Ps31 in the coordinate system of the input image are (Xs, Ys), the coordinates (Xm, Ym) of the mark position Pm in the coordinate system of the input image are expressed by the following equations (3) and (4). Is required.

Xm = Xs + Xm '' (3)
Ym = Ys−Ym ″ (4)

  The coordinate conversion unit 15 outputs the coordinates of the mark position in the coordinate system of the input image obtained as described above and information indicating the rotation angle of the mark in the input image to the subsequent apparatus. Thereafter, the image processing ends.

  In this way, the area for pixel-by-pixel matching can be reduced, and a matching target image with a different rotation angle is extracted for each candidate point, and matching with the template is performed. Corners can be detected quickly and accurately. In addition, since it is not necessary to prepare a plurality of templates, the memory capacity required for the templates can be reduced.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, the program is installed in a general-purpose computer from a program recording medium.

  FIG. 10 is a block diagram illustrating an example of a hardware configuration of a computer that executes the above-described series of processing by a program.

  In a computer, a central processing unit (CPU) 201, a read only memory (ROM) 202, and a random access memory (RAM) 203 are connected to each other by a bus 204.

  An input / output interface 205 is further connected to the bus 204. The input / output interface 205 includes an input unit 206 composed of a keyboard, mouse, microphone, etc., an output unit 207 composed of a display, a speaker, etc., a storage unit 208 composed of a hard disk or nonvolatile memory, and a communication unit 209 composed of a network interface. A drive 210 for driving a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is connected.

  In the computer configured as described above, the CPU 201 loads, for example, the program stored in the storage unit 208 to the RAM 203 via the input / output interface 205 and the bus 204 and executes the program. Is performed.

  The program executed by the computer (CPU 201) is, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (DiGittal Versatile Disc), etc.), a magneto-optical disk, or a semiconductor. The program is recorded on a removable medium 211 that is a package medium composed of a memory or the like, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  The program can be installed in the storage unit 208 via the input / output interface 205 by attaching the removable medium 211 to the drive 210. The program can be received by the communication unit 209 via a wired or wireless transmission medium and installed in the storage unit 208. In addition, the program can be installed in the ROM 202 or the storage unit 208 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

1 is a block diagram showing an embodiment of an image processing apparatus to which the present invention is applied. It is a flowchart for demonstrating the image process performed by an image processing apparatus. It is a figure for demonstrating a projection pattern. It is a figure for demonstrating the extraction process of a candidate point. It is a figure for demonstrating the extraction process of a candidate point. It is a figure for demonstrating the extraction process of a matching target image. It is a figure for demonstrating the matching process of a pixel unit. It is a figure which shows the coordinate system of a matching target image. It is a figure for demonstrating coordinate transformation. It is a figure which shows the structural example of a computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 Candidate point extraction part, 12 Matching object image extraction part, 13 Matching part, 14 Mark detection part, 15 Coordinate conversion part

Claims (4)

In an image processing method of an image processing apparatus for detecting a position of a mark that matches a mark indicated on a template in an input image,
A candidate for detecting a region of the input image having a high similarity to the template by matching a signal pattern of a pixel in a predetermined area of the input image and a signal pattern of the template, and extracting a center of the detected region as a candidate point A point extraction step;
For each of the candidate points, a matching target for extracting, as matching target images, images of a plurality of regions having a predetermined shape centered on the candidate point and having different rotation angles rotated about the candidate point An image extraction step;
A matching step of performing matching between the extracted matching target image and the template and obtaining a matching score representing a similarity between the matching target image and the template;
An image processing method, comprising: a mark detection step of detecting a position of the matching target image having the maximum matching score as a position of the mark in the input image. In the candidate point extraction step, a projection pattern indicating a distribution of pixel values in units of lines is used as a signal pattern of pixels in a predetermined area of the input image, and a distribution of pixel values in units of lines of the template is used as the signal pattern of the template. The image processing method according to claim 1, wherein a projection pattern is used. The image processing method according to claim 1, wherein in the mark detection step, a rotation angle of the matching target image that maximizes the matching score is detected as a rotation angle of the mark in the input image. In a program for causing a computer to execute processing for detecting a position of a mark that matches a mark indicated on a template in an input image,
A candidate for detecting a region of the input image having a high similarity to the template by matching a signal pattern of a pixel in a predetermined area of the input image and a signal pattern of the template, and extracting a center of the detected region as a candidate point A point extraction step;
For each of the candidate points, a matching target for extracting, as matching target images, images of a plurality of regions that have a predetermined shape centered on the candidate point and are rotated about the candidate point and having different rotation angles An image extraction step;
A matching step of performing matching between the extracted matching target image and the template and obtaining a matching score representing a similarity between the matching target image and the template;
A program that causes a computer to execute a process including a mark detection step of detecting a position of the matching target image that maximizes the matching score as a position of the mark in the input image.

Images