JP2014006613A - Neighborhood search method and similar image search method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology by which, in a neighborhood search method for searching an image in the neighborhood of a predetermined reference position in a feature amount space and a search method for similar images, neighborhood search or search for similar images can be efficiently performed in the feature amount space without the need of high arithmetic capacity.SOLUTION: For one feature amount X1, a predetermined range including a value of a reference image Is is selected as a neighborhood range R1 in the feature amount X1, and images I1 and I2 in the range are selected as candidate images. For another feature amount X2, a neighborhood range R2 including a value of the reference image Is is set, and images I1 and I3 in the range are selected as candidate images. The candidate image I1 that is selected in common in the respective feature amount is specified as a neighborhood image of the reference image Is.

Description

  The present invention relates to a neighborhood search method for searching for an image in the vicinity of a predetermined position in a feature quantity space when an image is represented by a plurality of types of feature quantities, and a similar image search method using the principle. It is.

  For example, in the field of manufacturing technology such as semiconductors and printed circuit boards, in order to detect defects contained in products and analyze / evaluate them, the object to be evaluated is imaged through a microscope, etc. Research has been done on calculating the quantity and performing automatic classification. In this type of classification technique, in order to find similar ones from a plurality of images, a neighborhood search is performed in a feature amount space using a plurality of types of feature amounts as coordinate axes. In other words, since it can be said that images in positions close to each other in the feature amount space have many common features, finding an image located in the vicinity of the feature amount space for one image is as follows: This corresponds to extracting an image similar to the image.

  As such a proximity search technique, for example, there is one described in Patent Document 1. In the technique described in Patent Document 1, a distance between two points in a multidimensional feature amount space is approximately calculated by an octagon distance, and a neighborhood search is performed based on the value. In this technique, by using the octagon distance, high-precision (for example, double-precision floating-point arithmetic) that is necessary for calculating the Euclidean distance, which is a stricter distance, is avoided.

JP 2011-086124 A

  However, in the above-described prior art, a considerable amount of computation is required to obtain the octagon distance between two points, and in particular, in a multi-dimensional feature amount space, the amount of computation increases exponentially according to the number of dimensions. When there are many types of feature amounts, that is, when the number of dimensions of the feature amount space is large, or when the number of points for distance calculation is large, the processing amount for the distance calculation becomes enormous. For this reason, although the above-mentioned conventional technique does not require calculation accuracy in hardware, there is a problem that it takes a long time for processing unless high-speed processing is possible.

  The present invention has been made in view of the above-described problems, and is a feature that does not require a high degree of computing capability in a neighborhood search method and a similar image search method for searching for an image in the vicinity of a predetermined reference position in a feature amount space. It is an object of the present invention to provide a technique capable of efficiently performing a proximity search or a similar image search in a quantity space.

  According to a first aspect of the present invention, an image is represented by a plurality of feature amounts, and is located in the vicinity of a predetermined reference coordinate point in a multidimensional feature amount space having each of the plurality of feature amounts as one coordinate axis. In order to achieve the above-mentioned object, the feature value corresponding to the coordinate axis is within a predetermined neighborhood range from the value of the reference coordinate point on the coordinate axis. A candidate extraction step of performing an extraction process for selecting an image in a candidate image as a candidate image, and images selected as the candidate images in all of the plurality of coordinate axes targeted by the candidate extraction step. And a neighborhood image specifying step of specifying as a neighborhood image of the reference coordinate point.

  In the invention configured as described above, for each of a plurality of feature amounts representing an image, focusing on only the coordinate axis corresponding to the feature amount in the feature amount space, the value of the feature amount is within the vicinity range from the reference coordinate point. The image at is a candidate image on the coordinate axis. The processing necessary for this is only the calculation of the feature amount and the simple comparison of the scalar amount for each type of feature amount. Then, from among candidate images selected for each feature amount, a common image is searched for in all feature amounts, and this is set as a neighborhood image.

  As a method for determining the degree of similarity between a plurality of images, a method for obtaining a distance between points corresponding to each image in the feature amount space has been generally used. For example, a Euclidean distance, a Manhattan distance, and a Mahalanobis are used. Distance etc. are used for this purpose. However, there are various feature quantities for representing the features of the defect image, and the dimension of the feature quantity space may exceed 100. When the dimension of the feature amount space increases in this way, the amount of computation for obtaining the above-mentioned distance between a plurality of images becomes enormous, and for example, a processor having high computation capability such as floating point processing becomes essential.

  On the other hand, in the processing of the present invention described above, the degree of similarity between two defect images is determined by comparing the values of one feature amount by the number of types of feature amounts. For this reason, the amount of calculation is greatly reduced as compared with the conventional method, and advanced calculation capability is not required. That is, according to the present invention, it is possible to efficiently perform a neighborhood search in the feature amount space without requiring a high level of computing ability. From this point, the present invention can be suitably applied for the purpose of searching for an image having a predetermined feature from many images or finding images having features similar to each other.

  In the present invention, an image selected as a candidate image in all the feature quantities that are candidates for the candidate extraction process is set as a neighborhood image, but the candidate extraction process is executed in all of a plurality of types of feature quantities representing the image. Is not required. That is, the concept of the present invention includes a mode in which the candidate extraction step is executed for a part selected according to the purpose among a large number of feature quantities, and a candidate image that is common among these feature quantities is used as a neighborhood image.

  In the present invention, when searching for a neighborhood image from a plurality of images, for example, for one coordinate axis, a neighborhood range may be set according to a distribution mode of feature value values corresponding to the coordinate axis in the plurality of images. Good. In a plurality of images, the distribution of the value of a certain feature value depends on the contents of the collected images, and it is difficult to predict in advance. In addition, since the neighborhood range set without reflecting the actual distribution mode is used, the neighborhood range may be inappropriately determined. Such a problem can be avoided in advance by setting a neighborhood range according to the distribution mode of the feature value values in the collected image and performing a neighborhood search. As described above, the neighborhood range is not determined in advance in a uniform manner, but is determined according to the distribution of the feature value values in a plurality of images, thereby making it possible to perform a neighborhood search in accordance with the contents of the actual image.

  More specifically, for example, a numerical value range of a section obtained by equally dividing a numerical value range including a maximum value and a minimum value between a plurality of images of a feature amount corresponding to one coordinate axis into a plurality of predetermined division numbers. , It can be set as a neighborhood range for the coordinate axis corresponding to the feature amount. In this way, the distribution range from the maximum value to the minimum value of the feature amount is equally divided into a plurality of sections, and the image should be a candidate image only by determining which section the feature value value is in It can be determined whether or not. This further simplifies the process.

  Further, for example, an image in which the coordinate value on the coordinate axis is in the same section as the reference coordinate point may be set as the candidate image. By doing so, an image having a feature value close to the coordinate value of the reference coordinate point is selected as a candidate image. At this time, it is only necessary to have information about which section each image belongs to, and it becomes possible to select a candidate image without using the value of the feature amount itself, so that the processing is further simplified.

  Further, for example, an image in which the coordinate value on the coordinate axis is in any of a plurality of continuous sections including the section to which the coordinate value of the reference coordinate point belongs may be used as the candidate image. When the candidate images are limited to only images in the same section as the reference coordinate point, it is difficult to find a candidate image that is common among the feature amounts. This tendency is particularly strong in a high-dimensional feature amount space, and it may not be determined as “neighboring” except for an image whose position in the feature amount space substantially coincides with the reference coordinate point. By expanding the range of the candidate image to a plurality of sections including the section to which the coordinate value of the reference coordinate point belongs, it is possible to perform a neighborhood search according to the mode of the image.

  According to a second aspect of the present invention, there is provided a similar image search method for searching for images that are similar to each other from a plurality of images. To achieve the above object, one of the plurality of images is selected as a reference image. A reference setting step for setting, as a reference coordinate point, a point where the reference image is located in a multi-dimensional feature amount space having a plurality of feature amounts each representing the reference image as one coordinate axis; A proximity search step of searching for a neighborhood image of the reference coordinate point from images other than the reference image among the plurality of images by a search method, and an image specified as the neighborhood image of the reference coordinate point as the reference image And a similar image specifying step of specifying as a similar image similar to.

  In the invention configured as described above, when one of a plurality of images is used as a reference image, a neighborhood image is searched by the above-described neighborhood search method using a point where the reference image is located in the feature amount space as a reference coordinate point. The As a result, an image that is in the vicinity of the reference image in the feature amount space, that is, an image similar to the reference image is searched efficiently, and in this case, as in the case of the above-described proximity search method, a high calculation capability is not required.

  In the present invention, for example, among the plurality of feature amounts, the difference between the maximum value and the minimum value between the plurality of images is larger than a threshold set in advance for the feature amount. The corresponding coordinate axis may be the target of the candidate extraction process. As described above, it is not essential to perform the candidate extraction process for all the feature quantities representing the image. For example, with respect to a feature quantity that has almost no difference in value between images, it is obvious that each image becomes a candidate image without performing a candidate image extraction step. There is no point in performing the candidate extraction step for such a feature quantity, but there is a possibility that the result will be adversely affected (for example, whether or not it is a neighborhood based on only a small difference in the feature quantity). By using a feature amount having a certain level of difference between a plurality of images as a candidate extraction step, such a problem can be avoided and the processing can be simplified.

  Also, for example, prior to the reference setting step, a plurality of feature amounts are calculated for each of a plurality of images, and a table that associates the value of the feature amount with an image having the value is created for each of the plurality of feature amounts. In the neighborhood searching step, the neighborhood image may be searched based on the table. As described above, the table in which the value and the image are associated with each feature amount is useful for searching for a neighborhood image of the reference image, regardless of which of the plurality of images is selected as the reference image. It will be a thing. This is because another image having a value close to the value of the feature amount in the reference image can be immediately derived by referring to the table. In particular, when the search process is repeatedly performed while sequentially changing the reference image, the same table can be referred to for all the reference images, and the effect is particularly remarkable.

  According to the present invention, advanced calculation is performed by selecting a candidate image in the vicinity range for each feature amount, and setting an image that is a candidate image in common for each feature amount as a vicinity image of the reference coordinate point. It is possible to efficiently perform a neighborhood search in the feature amount space without requiring capability.

It is a figure which shows the concept of the defect image which is the process target of this embodiment. It is a flowchart which shows the outline of the flow of the process in this embodiment. It is a figure which shows the example of distribution of the image in feature-value space. It is a figure which shows the range of the vicinity in this embodiment. It is a flowchart which shows the pre-processing for searching a near image. It is a figure which shows the concept of normalization and quantization of the value of a feature-value. It is a figure which shows the example of the table which linked | related the quantized feature-value and the image number. It is a flowchart which shows the grouping process of a defect image. 1 is a diagram showing a schematic configuration of an inspection system to which the present invention can be preferably applied.

  Hereinafter, a method for presenting a defect image according to an embodiment of the present invention will be described. In this embodiment, when automatic defect classification is performed in which a plurality of defect images are classified into several defect categories by an automatic learning algorithm, the process is executed as a previous stage process. More specifically, a process for supporting a user's work of finding a typical image that includes typical defects and can be a teacher image in automatic learning from a plurality of defect images collected by imaging an inspection target such as a substrate. It is. When the similar images are grouped and presented from a large number of collected defect images, the neighborhood search method and the similar image search method of the present invention are applied to the processing for associating images having similar features. Has been. Prior to specific description of the present embodiment, the concept that is the premise of the present embodiment and the following description will be described.

  FIG. 1 is a diagram showing a concept of a defect image that is a processing target of the present embodiment. As shown in FIG. 1A, for each defect image belonging to the defect image group IG collected in advance (hereinafter, simply referred to as “image”), each image is individually distinguished. Serial numbers and unique image numbers (1, 2,...) Are assigned as identification codes. In the following description, the total number of images in the defect image group IG is P (P is a natural number of 2 or more), and any one of the images is represented as an image number p (p is a natural number, 1 ≦ p ≦ P). To do.

  The feature of the defect included in each image is expressed by a plurality of feature amounts. As shown in FIG. 1B, in this embodiment, the feature of the defect is represented by N types (N is a natural number of 2 or more) of feature amounts, and a code X1 using a capital letter X for each of these various feature amounts. , X2, ..., XN. Any one of these features is represented by a code Xn (n is a natural number, 1 ≦ n ≦ N).

  The value of the feature amount corresponding to the defect image with the image number p is represented by the symbol xpn using the lowercase letter x. Therefore, for example, N types of feature values corresponding to the defect image of image number 1 are represented by codes x11, x12,..., X1N, respectively. Further, the value of the feature quantity X1 corresponding to the image numbers 1, 2,..., N is represented by x11, x21,. Further, when paying attention to one feature amount Xn, the maximum value among the feature amount values x1n, x2n,..., XPn corresponding to each defect image is represented by a symbol Mn, and the minimum value is represented by a symbol mn.

  FIG. 2 is a flowchart showing an outline of the flow of processing in this embodiment. First, a plurality of different defect images are acquired (step S101), and a defect image group IG is constituted by these defect images. Each defect image includes defects such as pinholes and foreign matters appearing on the appearance of the semiconductor substrate to be inspected, and is captured on a plurality of images taken at different positions on the same substrate or on different substrates. In addition, the defect image group IG can be configured by collecting a large number of images.

  A plurality of feature amounts X1,..., XN are calculated for each of the plurality of defect images thus obtained (step S102). Thereby, the feature of the defect contained in each defect image is quantitatively represented by the N types of feature amounts. Assuming a multidimensional feature amount space having a plurality of feature amounts as coordinate axes, each defect image can be plotted in the feature amount space as coordinate points having the feature amount values as coordinate values. Images having similar features of defects included in the defect image are positioned at positions close to each other in the feature amount space. In other words, the defect features (eg, shape, size, etc.) of multiple images located close to each other in the feature space are similar and are likely to be defects based on the same type of origin. Can be estimated.

  In order to accurately perform automatic defect classification using a learning algorithm, it is necessary to find an image (typical image) including a typical defect that becomes a teacher image from a plurality of collected defect images. Even if the defect is the same, its appearance is slightly different, and a technique for automatically obtaining a typical image by computer image processing or the like has not been established. For this reason, the selection of typical images is left to the judgment based on the experience of skilled users. At this time, the user needs to carefully scrutinize a large number of images without any clues in advance and select a typical image, and the work load is very large. It is convenient if information on defect tendency and correlation that can be automatically grasped from the collected images is provided to the user.

  Therefore, in this embodiment, the defect images are grouped so that the images at positions close to each other in the feature amount space are in the same group (step S103). Thereby, images having similar characteristics are associated with each other. For each group formed in this manner, some of the images belonging to the group are presented to the user as typical image candidates. At this time, among some of the formed groups, a large group, that is, one having a large number of images belonging to the group is presented in priority order (step S104).

  The number of images to be presented is arbitrary, and may be all images belonging to the same group or a part thereof. As a mode of presentation, a method for displaying an image to be presented on a display unit such as a display may be used, and for example, a method for outputting a list of image numbers of defect images belonging to the same group. Also good. Further, information that can grasp the size of the group, for example, information on the number of defect images belonging to the group may be further added.

  The reason why a group having a large number of images belonging is preferentially presented is as follows. It means that the larger the group of images having similar characteristics, the more “typical” defects having the characteristics are generated. Therefore, in the learning algorithm based on the teacher image, it is desirable that a typical image corresponding to the group is included in the teacher image. For this reason, images are preferentially presented from a large group.

  By such presentation, only images having features similar to each other are narrowed down and presented from a large number of unorganized defect images that are not associated with each other at all. Therefore, the user only has to select some typical images from the presented images, and the user's workload in specifying the typical images is greatly reduced. That is, much of the work for finally selecting a typical image from an unorganized image group is automatically and objectively performed. The number of typical images selected from images belonging to one group is arbitrary. For example, all images in the same group, or all except for those apparently having different origins may be used as typical images. Good.

  In addition, the user does not need to check all of the defect images similar to each other, and looks at the characteristic part grasped from some of the presented images to determine whether the defect is typical. Just judge. On the other hand, even if an image having a special defect with few images with similar characteristics is presented as a group even though it is small, such a special defect is not overlooked and is made a typical image. It is possible to make the user determine whether or not.

  In addition, when a typical image is selected only by the user's visual observation, it is specified that contradicts the feature of the image that is quantitatively determined based on the feature amount, and learning based on this may cause confusion in the classification result. obtain. However, in this embodiment, similar images and images that are not so are sorted in advance, and the user can specify a typical image after grasping the information, and thus contradicts the quantitative judgment in this way. The possibility of making a designation is extremely low.

  Next, a method for finding a group of images having similar characteristics to each other from a large number of collected defect images will be described. As described above, in general, it can be said that a plurality of images located at close positions in the feature amount space are similar to each other. Therefore, it is possible to estimate the degree of similarity between images by calculating the mutual distance in the feature amount space for a plurality of images. Euclidean distances, Manhattan distances, Mahalanobis distances, and the like are known as parameters representing such a degree of similarity. However, the amount of calculation for obtaining these distances between a large number of images in a high-dimensional space is enormous. In addition, it is known as a “dimensional curse” that comparison of distances between defect images may make little sense in a high-dimensional feature space. In this embodiment, since there is no distance calculation in a multidimensional space, such a problem does not occur, and the degree of similarity between images is estimated by a simpler method. The method will be described below.

  FIG. 3 is a diagram illustrating an example of an image distribution in the feature amount space. Here, a two-dimensional feature amount space using two types of feature amounts X1 and X2 will be described as an example. However, the following concept can be extended and applied to a higher-dimensional feature amount space. As an example, let us consider a case where 20 defect images (P = 20) are plotted in accordance with the value of the feature quantity in a two-dimensional feature quantity space having the feature quantities X1 and X2 as coordinate axes. In FIG. 3, a circle with a number inside indicates the position of each defect image in the feature space, and the number indicates the image number of each defect image.

  The 20 images with image numbers 1 to 20 are arranged at respective positions in the coordinate space according to the values of the feature amounts X1 and X2. Here, as indicated by broken lines in FIG. 3, the images at close positions can be grouped together to divide the entire image into several groups. For example, a group of images represented by image numbers 1, 4, 7, 13, 17, and 20 are close to each other and constitute one cluster as a whole. One defect image is set as a reference image, and other defect images in the vicinity thereof are included in the same group as the reference image, and another defect image in the vicinity of the other defect image is also referred to as the reference image. Same group. It should be noted that an isolated image (for example, image number 8) that is far from other images may be regarded as one group having the isolated image as a single component.

  FIG. 4 is a diagram showing the vicinity range in this embodiment. In the concept of this embodiment, when one defect image is a reference image Is, a neighborhood range R1 and a feature amount appropriately determined on the coordinate axis corresponding to the feature amount X1 with the position of the reference image in the feature amount space as the center. An image within the range of the neighborhood range R2 appropriately determined on the coordinate axis corresponding to X2 is searched, and if there is such an image, it is determined as a neighborhood image close to the reference image.

  In this example, the images in the vicinity range R1 in the coordinate axis corresponding to the feature amount X1 with respect to the reference image Is include images indicated by symbols I1 and I2, respectively. Of these, the coordinate axis corresponding to the feature amount X2 is also included. The image I1 in the vicinity range R2 is set as the vicinity image of the reference image. Since the image I2 is out of range on the coordinate axis corresponding to the feature amount X2, it is not regarded as a neighborhood. On the other hand, the image I3 is within the vicinity range R2 on the coordinate axis corresponding to the feature amount X2, but is not considered to be near because it is outside the range on the coordinate axis corresponding to the feature amount X1. Naturally, the image I4 which is out of the range in both coordinate axes is not near.

  Although it can be said that the determination based on such a criterion is not as strict as the determination based on the Euclidean distance described above, the determination can be performed by repeating independent calculation for each coordinate axis, and the number of dimensions increases. There is an advantage that the calculation amount is not so large as compared with the above distance calculation in which the calculation amount increases exponentially. In addition, the purpose of the determination here is to support the user's determination by grouping and displaying images that are considered to be related to each other. For the purpose of searching for nearby images, the above determination method is Necessary and sufficient accuracy can be obtained. That is, here, the advantage of the small amount of processing is given priority over the accuracy of calculation.

  FIG. 5 is a flowchart showing preprocessing for searching for a neighborhood image. This process is a process in which the relationship between the feature values of each defect image is arranged in advance for each type of feature value in order to smoothly search for the neighborhood relationship between a large number of defect images. This process corresponds to steps S101 and S102 and a part of step S103 in the process shown in FIG. When a plurality of defect images are acquired (step S201), sequential image numbers are assigned as codes for identifying the defect images (step S202). Then, N types of feature amounts are calculated for each defect image (step S203).

  Next, one feature quantity Xn is selected from N types, and the respective defect images are arranged based on the magnitude relationship of the feature quantity values x1n, x2n,..., XPn. Here, for example, it is assumed that they are arranged in ascending order (step S205). In the image arrangement, it is not necessary to arrange actual defect images, and it is only necessary to obtain a sequence of image numbers arranged in the order of feature value values.

  Then, the difference between the maximum value Mn and the minimum value mn of these feature quantities is obtained (step S206). This difference is compared with a predetermined threshold value a (step S207), and if it exceeds the threshold value a, the next steps S208 to S211 are executed, while if not more than the threshold value, these steps are skipped.

  In steps S208 to S211, in order to facilitate the comparison of the feature value between the defect images, the feature value is normalized and quantized, and a table in which the value is associated with the image number is created. To do. However, when the difference between the maximum value Mn and the minimum value mn of the feature amount is equal to or less than the threshold value, it is considered that there is no significant difference between the images for the feature amount, and such work is omitted. As the threshold value a, for example, a value that is 1000 times the minimum positive number that can be expressed by a double-precision floating point type can be used. When the difference between the maximum value Mn and the minimum value mn is equal to or less than the threshold value a thus determined, the difference in value between the defect images becomes even smaller, and each coordinate axis corresponding to the feature amount is different for each coordinate axis. This is because they can be regarded as being very close without comparing between images. By excluding feature quantities that hardly affect the result in this way, the substantial number of dimensions becomes smaller than the original N dimensions, and the amount of computation can be further reduced.

  In step S208, the difference value δ between adjacent images in an ascending order of feature value values is obtained, and the difference value is averaged Δn by accumulating the value δ and dividing by the number of data. Is calculated. Thereby, the range of the spread of the feature value is obtained. At this time, if the value δ is smaller than the predetermined threshold value bn, it is excluded from the cumulative addition because the difference in feature quantity between adjacent images is substantially zero, and is also excluded from the number of data. The threshold value bn can be, for example, (Mn−mn) / 1000. In this way, in the next feature quantity quantization to be performed, it is possible to apply an appropriate quantization step that does not become an unnecessarily fine step depending on the variation in the value of the feature quantity between images. It becomes possible.

In step S209, the quantization division number Kn on the coordinate axis corresponding to the feature amount Xn is obtained. The quantization division number Kn is expressed by the following formula:
Mn ≦ Kn ・ Δn
The smallest integer that satisfies the relationship Further, in step S210, based on the quantization division number thus determined, the value xpn of the feature amount Xn in each defect image is expressed by the following equation:
ypn = Int {(xpn-mn) /Δn+0.5}
Normalized and quantized by Here, Int {x} is a function that returns the integer part of the variable x as a value. The reason for adding 0.5 to an integer is to round off the decimal part.

  As a result, the feature value xpn of each defect image distributed in the range from the minimum value mn to the maximum value Mn on the coordinate axis corresponding to the feature value Xn is normalized to take discrete values from 1 to Kn. It is converted into a quantized value ypn. As a result, information regarding a minute difference in feature value between images is lost, but this information is not necessary in the application as in this embodiment, and hardly affects the result. Then, a table in which the feature value ypn quantized in this way is associated with the image number of the defect image taking the value is created (step S211).

  Such processing is performed for all types of feature values (step S212). That is, while the feature quantities to be selected are sequentially changed, the above-described feature quantity values are normalized and quantized, and a table based on this is created. In addition, since some feature amounts may be excluded in step S207, the table is not necessarily created for all N types of feature amounts. In the following, the number of feature quantities for which a table has been created is represented by N ′ (≦ N).

  FIG. 6 is a diagram showing a concept of normalization / quantization of feature value. FIG. 7 is a diagram showing an example of a table in which quantized feature amounts are associated with image numbers. As the defect image group in these drawings and the description thereof, the same one as shown in FIG. 3 is used. The quantization of the feature value in the two-dimensional feature space means that the position of the coordinate point representing each image in the feature space is the closest among the lattice points provided at equal intervals as shown by broken lines in FIG. This corresponds to moving the object to the position. Further, normalizing the feature value corresponds to scaling the coordinate axis so that the minimum value of the quantized feature value is 1 and the maximum value is Kn in one coordinate axis. In the example of FIG. 6, the quantization division number Kn (that is, K1) is 9 on the horizontal axis corresponding to the feature quantity X1 (n = 1), and the quantization is performed on the vertical axis corresponding to the feature quantity X2 (n = 2). The number of divisions Kn (that is, K2) is 10. However, these are not limited to those shown as an example, and the number of quantization divisions is determined according to the distribution of feature values on each coordinate axis as described above. is there.

  The table illustrated in FIG. 7 indicates where each defect image with image numbers 1 to 20 is positioned in the feature amount space as a result of the normalization and quantization as described above. FIG. 7A is a table for the feature quantity X1, and FIG. 7B is a table for the feature quantity X2. In the figure, “division number” has the same meaning as the normalized and quantized feature value, and “image number corresponding to the division” has the same feature value as the division number as a result of normalization and quantization. Is the image number of the defect image. For example, referring to FIG. 7A, the value of the feature quantity normalized and quantized on the coordinate axis corresponding to the feature quantity X1 is 1 in three image numbers 4, 7, and 10. It is shown.

  As described above, if a table in which the feature value and the image number are associated with each other is created, when one image number is designated as the reference image, the reference image and the feature value are stored. It is possible to immediately read out the image numbers of other defective images close to the table.

  Specifically, this table is used as follows. For example, consider a case in which a defect image represented by image number 1 is used as a reference image and a neighboring image in the vicinity thereof is searched in the feature amount space. When, for example, image number 1 is designated as the image number corresponding to the reference image, those whose feature values are in the vicinity of the reference image value are specified as candidate images from each table. Here, the same section as the reference image and a section adjacent to the same section are set as the vicinity range. Then, as is apparent from FIG. 7A, the feature quantity X1 belongs to the same section (section number 2) as the image number 1, and 5 of image numbers 1, 11, 13, 18, 20 In addition, there are a total of five that belong to adjacent sections (section numbers 1 and 3): 4, 7, 10, 14, and 17. That is, the candidate images for the feature amount X1 are images identified by image numbers 1, 4, 7, 10, 13, 14, 17, 18, and 20, respectively.

  On the other hand, as can be seen from FIG. 7 (b), the feature quantity X2 belongs to the same partition number 2 as the reference image as image numbers 1, 7, and 16, and belongs to the adjacent partition numbers 1 and 3. Section numbers 4, 13, 17, and 20. Accordingly, the candidate images for the feature amount X2 are images specified by image numbers 1, 4, 7, 13, 16, 17, and 20, respectively.

  An image in the vicinity in the feature amount space is an image in the vicinity range of the reference image on all coordinate axes (feature amounts). Therefore, the product represented by the image numbers 4, 7, 13, 17, and 20 obtained by taking the product set of candidate images with the feature amounts X1 and X2 and excluding the reference image itself is the reference image (image number 1). It can be said that it is a neighborhood image corresponding to). Note that the feature amount space in this case is an N′-dimensional space excluding the coordinate axis that is determined to have a small difference between the maximum value Mn and the minimum value mn from the original N-dimensional space (step S207 in FIG. 5). Using this principle, defect images are grouped as follows in this embodiment.

  FIG. 8 is a flowchart showing the defect image grouping process. This process corresponds to step S103 in FIG. First, a defect image that has not yet been grouped is selected from a plurality of collected defect images, and designated as a reference image (step S301). An appropriate group label for identifying the group is assigned to the reference image (step S302). In order not to give the same label to different groups, it is assumed that the group label at this time is a new one not assigned to an existing group.

  Next, a neighborhood image is searched for the reference image. That is, based on the above principle, the N′-plane table created earlier is referred to, and each feature value whose value is in the vicinity of the reference image value is extracted as a candidate image (step S303). ). Then, those extracted as candidate images in all the tables on the N ′ plane are specified (step S304), and those obtained by excluding the reference image itself from them are specified as neighboring images corresponding to the reference image (step S305). ). Of the neighboring images thus identified, the same group label as that of the reference image is assigned to the unassigned group label (step S306). As a result, the reference image and its neighboring images are associated as the same group.

  According to the gist of the present embodiment, the same group label as the reference image is given to an image in the vicinity of the vicinity image even if the image is far from the reference image beyond the vicinity range. Need to be done. Therefore, the search for the neighborhood image and the label assignment for the neighborhood image of the reference image are subsequently performed. Specifically, the image numbers of the neighboring images with respect to the reference image obtained above are added to the stack (step S307), and those images are sequentially designated as new reference images (step S309), while the neighborhood of the image is displayed. Image search and group label assignment to the found neighboring images are performed (steps S303 to S306). If a new neighborhood image is found, it is added to the arithmetic processor stack (step S307). By repeating this until the stack becomes empty (step S308), the association between the images via the neighboring images starts from the first reference image, and the same group label is assigned to them. go.

  When the stack is empty, there are no more images left in the vicinity of the group. That is, the spread of the group converges. Therefore, the process returns to step S301 (step S310), and the above processing is repeated using an image that has not yet been grouped as a new reference image. At this time, a label different from the previous group label is given. The above process is repeated until any group label is given to all defect images. In this way, all images are grouped. The method of presenting the grouping result to the user is as described above.

  In the above-described example, the description has been made as the “neighboring range” up to the same section as the reference image or the section adjacent to the reference image in the normalized and quantized feature value. Naturally, the result of grouping differs depending on how the neighborhood range is set. That is, when the neighborhood range is widened, even images with relatively large differences are grouped into the same group. Conversely, if the neighborhood range is narrowed, even a slight difference in characteristics will be treated as another group. In particular, in a high-dimensional feature amount space, a small difference in appearance tends to be a large distance in the feature amount space and each image tends to be isolated. According to the knowledge of the inventor of the present application, in the defect image group of about Kn = 100, about ± 10 to ± 50 sections based on the same section as the reference image, although it varies depending on the number of defect images and the distribution of the feature amount. Good results have been obtained when is included in the neighborhood range.

  FIG. 9 is a diagram showing a schematic configuration of an inspection system to which the present invention can be preferably applied. This inspection system 1 is an inspection system that performs inspection of defects such as pinholes and foreign matters appearing on the appearance of a semiconductor substrate S to be inspected, and automatically classifies detected defects. The inspection system 1 performs the defect inspection based on the image data from the image pickup device 2 and the image pickup device 2 that picks up the inspection target area on the substrate S, and when the defect is detected, the defect belongs to the category to which the defect should belong. And a host computer 5 as a control unit having an inspection / classification function for automatically classifying (ADC) automatic defect classification and a function for controlling the entire operation of the inspection system 1. The imaging device 2 is incorporated in the production line of the substrate S, and the inspection system 1 is a so-called inline system.

  The imaging device 2 moves the stage 22 relative to the imaging unit 21 that acquires image data by imaging the inspection target region on the substrate S, the stage 22 that holds the substrate S, and the imaging unit 21. The imaging unit 21 includes a stage driving unit 23, and an imaging unit 21 forms an image by an illumination unit 211 that emits illumination light, an optical system 212 that guides the illumination light to the substrate S and receives light from the substrate S, and an optical system 212. An imaging device 213 that converts the image of the substrate S thus converted into an electrical signal is included. The stage drive unit 23 includes a ball screw, a guide rail, and a motor. The apparatus control unit 501 provided in the host computer 5 controls the stage drive unit 23 and the imaging unit 21, so that the inspection target region on the substrate S is changed. Imaged.

  The host computer 5 implements each functional block shown in FIG. 1 by software by executing a control program read in advance. In addition to the apparatus control unit 501, the host computer 5 includes functional blocks such as a defect detection unit 502, a defect classification unit (ADC) 503, a feature amount calculation unit 504, a teaching unit 505, a determination unit 506, and a calculation unit 507. I have. Further, the host computer 5 displays a storage unit 510 for storing various data, an input receiving unit 511 such as a keyboard and a mouse for receiving operation inputs from the user, and a display for displaying visual information for the user such as operation procedures and processing results. Part 512 and the like. Although not shown in the figure, a reading device for reading information from a computer-readable recording medium such as an optical disk, a magnetic disk, a magneto-optical disk, etc. Are suitably connected via an interface (I / F) or the like.

  The defect detection unit 502 performs defect detection by finding a unique area in the inspection target area while processing the image data of the inspection target area. When the defect detection unit 502 detects a defect from the inspection target area, image data of the defect and various data used for the inspection are temporarily stored in the storage device 510.

  On the other hand, the defect classification unit 503 performs, in software, a process of classifying detected defects using learning algorithms such as SVM (Support Vector Machine), neural network, decision tree, and discriminant analysis. To do. The feature amount calculation unit 504 calculates a feature amount that characterizes the defect based on the image data of the detected defect. A teaching unit 505 gives teaching data for causing the defect classification unit 503 to machine-learn the algorithm.

  The determination unit 506 and the calculation unit 507 each perform processing for calculating the accuracy indicating the validity of the classification result classified by the defect classification unit 503. Specifically, the determination unit 506 determines whether or not each feature value of the defect image is appropriate for each classification category. The calculation unit 507 calculates the accuracy based on the determination result of the determination unit 506.

  The present invention can be suitably applied to the inspection system configured as described above. That is, when the defect classification unit 503 performs machine learning by using some of a large number of defect images captured by the imaging device 2 as typical images, the main task is to assist the user in finding the typical image from the defect images. The invention can be used. The candidates for typical images obtained by grouping can be presented to the user by display on the display unit 512 of the host computer 5. When a typical image is designated by the user, the teaching unit 505 creates teaching data based on the information and gives it to the defect classification unit 503.

  As described above, in this embodiment, when finding a plurality of defect images having features similar to each other, when one defect image is used as a reference image, an image in the vicinity of the reference image in the feature amount space is displayed. The method of searching is taken. The search for the neighborhood image is not based on the distance calculation in the feature amount space, but for each of the coordinate axes (that is, each feature amount) constituting the feature amount space, images in the neighborhood range from the reference image are candidate images. And selecting a candidate image common to each feature amount as a neighborhood image.

  According to such a search method, it is possible to select candidate images for the feature amount only by comparing numerical values of one feature amount, and by extracting a common image from candidate images for each feature amount. A neighborhood image is identified. For this reason, the amount of processing is greatly reduced as compared with the conventional search method involving distance calculation in the feature amount space, and a high degree of arithmetic processing capability is not required. This advantage is particularly remarkable when the number of dimensions of the feature amount space is large or when the number of images to be collected is large.

  As described above, in this embodiment, step S303 in FIG. 8 corresponds to the “candidate extraction step” in the neighborhood search method according to the present invention, while step S304 corresponds to the “neighboring image specifying step”. . These correspond to the “neighbor search step” in the similar image search method according to the present invention, and step S301 corresponds to the “reference setting step”. Step S305 corresponds to a “similar image specifying step”.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, the value of each feature value of the defect image is normalized and quantized, and then the neighborhood image is searched for the reference image. It is not an essential requirement in the present invention to normalize or quantize the value of. Only one of these processes may be executed.

  Therefore, for example, for each feature amount, a range including the minimum value to the maximum value of the feature amount value among a plurality of defect images is equally divided into a plurality of sections, and the feature value value of each image is The same effect can be obtained even if a table as shown in FIG.

  Moreover, although the said embodiment is an image classification apparatus which test | inspects and classifies the defect of a semiconductor substrate, the image classification apparatus which is an application object of this invention is not only the apparatus which test | inspects a semiconductor substrate, but another target object, for example, a print It may be a device for inspecting a substrate, a glass substrate or the like, or a surface inspection device for inspecting the surface state of various materials.

  In the above embodiment, the present invention is applied to an apparatus for selecting images having similar characteristics from defect images on a semiconductor substrate or the like. However, the image targeted by the present invention is limited to such defect images. However, the present invention can be applied to all techniques for handling various images evaluated based on feature amounts.

  The present invention is suitable for a technique for handling an image represented by a plurality of feature amounts, and in particular, an object of searching for an image having a predetermined feature from many images or finding images having features similar to each other. It can be suitably applied to.

DESCRIPTION OF SYMBOLS 1 Inspection system 2 Imaging device 503 Defect classification | category part 504 Feature-value calculation part 505 Teaching part 506 Judgment part 507 Calculation part 512 Display part S301 Reference | standard setting process S303 Candidate extraction process, neighborhood search process S304 Neighborhood image specification process, Neighborhood search process S305 Similarity Image identification process

Claims (8)

In a neighborhood search method for searching for an image located in the vicinity of a predetermined reference coordinate point in a multidimensional feature amount space in which an image is represented by a plurality of feature amounts and each of the plurality of feature amounts is a coordinate axis.
For one of the coordinate axes, an extraction process for selecting, as a candidate image, an image in which a feature value corresponding to the coordinate axis is within a predetermined vicinity range from a value of the reference coordinate point on the coordinate axis, a plurality of the coordinate axes Candidate extraction process to be executed as a target;
A neighborhood search method comprising: a neighborhood image identification step that identifies an image selected as the candidate image on all of the plurality of coordinate axes targeted by the candidate extraction step as a neighborhood image of the reference coordinate point . Searching the neighborhood image from the plurality of images,
The neighborhood search method according to claim 1, wherein the neighborhood range is set for one coordinate axis in accordance with a distribution mode of feature value values corresponding to the coordinate axes in the plurality of images.   A numerical range of a section obtained by equally dividing a numerical range including a maximum value and a minimum value between the plurality of images of a feature amount corresponding to one coordinate axis into a plurality of predetermined division numbers corresponds to the feature amount. The neighborhood search method according to claim 2, wherein the neighborhood range with respect to a coordinate axis is used.   The neighborhood search method according to claim 3, wherein an image having a coordinate value on the coordinate axis in the same section as the reference coordinate point is set as the candidate image.   The neighborhood search method according to claim 3, wherein an image having a coordinate value on the coordinate axis in any of a plurality of continuous sections including a section to which the coordinate value of the reference coordinate point belongs is used as the candidate image. In a similar image search method for searching for images similar to each other from a plurality of images,
One of the plurality of images is selected as a reference image, and a point where the reference image is located in a multi-dimensional feature amount space having a plurality of feature amounts representing the reference image as one coordinate axis is used as a reference coordinate point. A reference setting process to be set;
A neighborhood search step of searching for a neighborhood image of the reference coordinate point from images other than the reference image among the plurality of images by the neighborhood search method according to any one of claims 1 to 5.
A similar image search method comprising: a similar image specifying step of specifying an image specified as the neighborhood image of the reference coordinate point as a similar image similar to the reference image.   Among the plurality of feature amounts, for the difference between the maximum value and the minimum value between the plurality of images larger than a threshold value set in advance for the feature amount, a coordinate axis corresponding to the feature amount is used. The similar image search method according to claim 6, which is a target of the candidate extraction step. Prior to the reference setting step, the plurality of feature amounts are calculated for each of the plurality of images, and the feature amount value and the image having the value are associated with each of the plurality of feature amounts. The similar image search method according to claim 6, wherein in the neighborhood search step, the neighborhood image is searched based on the table.

Images