JP6161256B2 - Image analysis support apparatus and method - Google Patents

Description

  The present invention relates to an image analysis support apparatus and method.

  Currently, image processing technology is applied in various technical fields. For example, it is applied to inspection devices and measuring devices in the semiconductor manufacturing field, and to diagnostic devices and analysis devices in the medical field. In these apparatuses, image processing technology is used for the purpose of automatically extracting and specifying a defective part or a part to be measured from an image taken with an electron microscope, an image taken with an X-ray, or an image obtained with an optical camera. The

  However, it is generally not easy to construct and optimize an image processing algorithm according to the characteristics of the image and the characteristics of the part to be specified. In fact, to optimize the combination of parameter values and processing functions used in this kind of image processing algorithm, verification work for many image samples is required, and operator skill for verification work is also required. .

  In order to improve these problems, for example, Patent Document 1 discloses a method of specifying a region where an image is to be extracted and applying a neural network. Patent Document 2 discloses a method for generating a teaching image and an automatic optimization method for an image processing algorithm using a genetic algorithm.

JP 2000-339463 A JP 2006-337152 A

  Patent Document 1 shows that a representative point such as a center point of a region is used as a method for designating a desired region. However, if a good extraction result is not obtained in advance, the center point of a region is indicated. It is not always easy to prepare position information. Moreover, their positional accuracy is not always important. This method is particularly problematic when it is desired to optimize the image processing algorithm for a new object.

  On the other hand, in Patent Document 2, a method for preparing an appropriate target image is not clear. In addition, the method of using the difference value from the target image as the evaluation value of the image processing algorithm generally has a problem of preparing the target image. Moreover, it is generally not easy to create a suitable target image. Further, if the created target image is not appropriate, the evaluation value is lowered only by a small difference in area and shape. In addition, for example, in the case of setting that ignores a minute difference, a minute part cannot be evaluated. Further, if the algorithm is forced to approach an inappropriate target image, there is a possibility that useless processing will occur in image processing.

  The present invention has been made in consideration of the above situation, and provides an image analysis support apparatus and method which can easily create a target used for evaluation of result data and can automatically optimize an image processing algorithm.

  An image analysis support apparatus according to the present invention applies a predetermined image processing algorithm to input image data and outputs a result data, and target data used for evaluation of the result data are set to the target data. The target data setting unit that accepts as coordinate points on the image corresponding to the image data input for setting, and the result data and target data are logically compared, and the result data having a higher degree of match with the target data is higher. A comparison evaluation unit that calculates an evaluation value and an automatic optimization unit that automatically determines a setting value of an image processing algorithm based on the evaluation value.

  According to the present invention, the cost of development and adjustment of an image processing algorithm can be suppressed, and inspection accuracy and measurement accuracy can be stabilized. Problems, configurations, and effects other than those described above will become apparent from the following description of embodiments.

The figure which shows the function structure of the image analysis assistance apparatus which concerns on an Example. The figure which shows the keyboard, mouse | mouth, display, and stylus pen as a target data setting part which concerns on an Example. The figure explaining the example of designation | designated of the target data with respect to a sample image. The figure which shows the example of an image of result data. The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the example of evaluation classification of result data (evaluation by the link between coordinate points). The figure which shows the sample image example whose object is ambiguous. The figure which shows the example of designation | designated of the target data with respect to the sample image example whose object is ambiguous. The figure which shows the evaluation classification example of result data (evaluation by the number of coordinate points). The figure which shows the evaluation classification example of result data (evaluation by the number of coordinate points). The figure which shows another example of a setting of the target data with respect to the sample image example whose object is ambiguous. The figure which shows the evaluation classification example of result data (evaluation by an area). FIG. 6 is a diagram illustrating another sample image example according to the first embodiment. The figure which shows the example of designation | designated of the target data surrounding the site | part made into extraction object. The figure which shows the example of designation | designated of the target data surrounding the site | part made into extraction object. The figure which shows the example of a setting of the target data surrounding the site | part made into extraction object. The figure which shows the example of a setting of the target data surrounding the site | part made into extraction object. The figure which shows the example of a setting of the target data surrounding the site | part made into extraction object. The figure which shows the process function example which a candidate extraction part has. The figure which shows the 1st process structural example of a candidate extraction part. The figure which shows the 2nd process structural example of a candidate extraction part. The flowchart which shows the process sequence example of an automatic optimization part. FIG. 10 is a diagram illustrating a template image used in the second embodiment. The figure explaining the example of designation | designated of the target data with respect to a sample image. The figure explaining the example of a setting of the tolerance | permissible_range circle | round | yen with respect to a coordinate point. The figure explaining the link relation between coordinate points. The figure explaining the link relation between coordinate points. The figure which shows the example of a setting of the tolerance | permissible_range with respect to a coordinate point in case the object has shifted | deviated doubly. The figure which shows the example of a setting of the tolerance | permissible_range with respect to a coordinate point in case the object has shifted | deviated doubly.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, embodiment mentioned later is shown as an illustration and should not be interpreted limiting the invention to the contents of each example.

[Example 1]
In this embodiment, an image analysis support apparatus that extracts foreign matter, scratches, or manufacturing defect portions attached to a product from an image obtained by imaging the product (for example, a semiconductor wafer) will be described.

[Device configuration]
FIG. 1 shows a configuration of an image analysis support apparatus according to the embodiment. The image analysis support apparatus 100 includes a target data setting unit 101, a comparative evaluation unit 103, an imaging unit 104, a candidate extraction unit 106, and an automatic optimization unit 108. Among these, the imaging unit 104 and the candidate extraction unit 106 are used for image analysis, and the target setting unit 101, the comparative evaluation unit 103, and the automatic optimization unit 108 are used for automatic optimization of the image processing algorithm.

  The imaging unit 104 is a device that captures an image of an inspection target. The imaging unit 104 is, for example, an electron microscope, an optical microscope, an X-ray imaging apparatus, a CMOS sensor camera, or the like, and sensitivity, exposure time, distortion correction, and the like are adjusted by setting data 109. The imaging unit 104 is used not only at the time of analysis but also at the time of creating the target data 102. The imaging unit 104 outputs image data 105. The image data 105 is stored in a hard disk, memory, or other storage device (not shown).

  The candidate extraction unit 106 extracts candidate areas from the image data 105 according to the image processing algorithm set in the setting data 109, and outputs the extracted images of the candidate areas as result data 107. Note that the coordinate information can also be output as the result data 107 as in an embodiment described later. The parameters of the imaging unit 104 and the image processing algorithm are adjusted based on the setting data 109.

  The target data setting unit 101 is used to create target data 102. The target data 102 is data prepared for evaluation of the result data 107 output from the candidate extraction unit 106, and includes, for example, position information of coordinate points included in a part to be extracted from the inspection target. The target data 102 is set for the image data input for the setting. Note that the target data setting unit 101 is also used to input relationships and conditions to be satisfied by designated coordinate points (for example, relationships between a plurality of coordinate points, relationships / conditions with a specific region, etc.).

  The comparative evaluation unit 103 outputs a comparison result between the target data 102 and the result data 107 as an evaluation value 110. The evaluation value 110 is used to evaluate the image processing algorithm of the candidate extraction unit 106. The comparative evaluation unit 103 outputs an evaluation value 110 having a higher value as the target data 102 and the result data 107 match the evaluation condition.

  The automatic optimization unit 108 inputs the evaluation value 110 from the comparative evaluation unit 103. The automatic optimization unit 108 changes the setting data 109 when the evaluation value 110 is less than a predetermined threshold value, and sets the setting data 109 as a final result when the evaluation value is equal to or greater than the predetermined threshold value. By changing the setting data 109, the image processing algorithm of the candidate extraction unit 106 and the parameters of the imaging unit 104 are optimized.

  The other components are realized by a computer or a program constituted by, for example, a microprocessor, a hard disk, a memory, a display, and the like.

[Detailed configuration of target data setting section]
FIG. 2 shows a detailed configuration of the target data setting unit 101. FIG. 2 shows a specific configuration example of the target data setting unit 101. The keyboard 201 is used when inputting coordinates numerically. The mouse 202 and the stylus pen 204 are used when designating coordinate points in the image data on the screen displayed on the display 203. The mouse 202 and the stylus pen 204 are generally called a pointing device. Other pointing devices include trackballs and touchpads.

[Sample image and target data setting example]
FIG. 3 shows the operation when setting the target data 102 on the screen of the display 203. The target data 102 is prepared by an operator to optimize the setting data 109 so that the image processing algorithm of the candidate extraction unit 106 automatically extracts an expected area from the image data 105. In the window of the screen, a sample image 301 for product inspection and coordinate points set by the operator are displayed. A sample image 301 is a display example of the image data 105. In the example of FIG. 3, the foreign matter 302 is attached to the surface of the product, and the manufacturing defect portion 303 and two scratches 307 and 308 also exist.

  In the case of the present embodiment, the operator uses the stylus pen 204 to set the target data 102 by designating coordinate points in the foreign matter 302, the defective portion 303, and the scratches 307 and 308. A coordinate point 304 indicates a coordinate point designated by the operator in correspondence with the foreign object 302, and a coordinate point 305 and a coordinate point 306 indicate coordinate points designated by the operator in correspondence with the defect portion 303. In this embodiment, these coordinate points are marked with “x”. When a plurality of coordinate points 305 and 306 marked with “x” are connected by a line segment, it indicates that the plurality of coordinate points connected should be extracted simultaneously in one region. That is, it indicates that a plurality of coordinate points that are connected should exist in an area labeled in the same group. On the other hand, the coordinate points 304 that are not connected by line segments indicate that they are used for extraction of isolated regions independent of other coordinate points.

  The coordinate points 309 to 311 correspond to the scratch 307, and the coordinate points 312 to 314 correspond to the scratch 308. In this embodiment, these coordinate points 309 to 314 are marked with “+”. Among these, a plurality of coordinate points 309 to 311 or 312 to 314 connected by line segments indicate that an area including one or more coordinate points is extracted. Note that there is no connection between the group in which the coordinate points 309 to 311 are designated and the group in which the coordinate points 312 to 314 are designated, and is a different group. This indicates that both groups are extracted as different regions.

  Note that the distinction between “x” and “+” and the drawing of a connection line indicating group designation may be selected from the menu at the top of the window or may be selected from the keyboard 201.

  Specific contents of the target data 102 in this embodiment are “×” (coordinate point 304), “×” (coordinate point 305, coordinate point 306), “+” (coordinate point 309, coordinate point 310, coordinate Point 311), “+” (coordinate point 312, coordinate point 313, coordinate point 314).

  Also, a number for distinguishing the group is assigned to each coordinate, and (coordinate point 304, x0), (coordinate point 305, x1), (coordinate point 306, x1), (coordinate point 309, +0), ( Coordinate points 310, +0), (coordinate points 311, +0), (coordinate points 312, +1), (coordinate points 313, +1), and (coordinate points 314, +1) can also be expressed.

  Coordinate values may be directly input from the keyboard 201 to the character input box 320 in case the coordinates are difficult to specify with a stylus pen, such as in the case of a very small foreign object.

  The target data 102 only needs to be prepared before processing by the automatic optimization unit 108 is started, and may be built in a device different from the device that executes the automatic optimization processing. Further, when good result data 107 for the sample image 301 is obtained in advance by an existing adjusted apparatus or the like, the result data 107 may be converted to create the target data 102.

[Example image of result data]
FIG. 4 shows an example of the image 401 of the result data 107 corresponding to the sample image 301. An image 401 is a processing result output from the candidate extraction unit 106 that has received the image data 105 corresponding to the sample image 301. In the image 401, candidate areas 402, 403, 404, 405, and 406 are output as extraction parts. These areas are shaded in the figure. In FIG. 4, coordinate points 304 to 306 and coordinate points 309 to 314 designated as the target data 102 are also shown for easy understanding of the relationship with the target data 102.

  Candidate area 402 includes coordinate point 304 inside, but does not include other coordinate points. In this sense, the candidate area 402 is an extraction result as expected. The candidate area 403 is an independent area that does not include any coordinate point of the target data 102, and is an unexpected erroneous extraction result. The candidate area 404 includes the coordinate point 306 inside, but does not include the coordinate point 305 that should be included at the same time in the same area, which is an insufficient extraction result. The candidate area 405 does not include the coordinate point 309 inside, but includes the two coordinate points 310 and 311 marked with “+”, and the extraction result is sufficiently as expected. The candidate area 406 includes three coordinate points 312 to 314 marked with “+” inside, and is an extraction result as expected.

[Evaluation example of comparative evaluation unit 103]
The comparative evaluation unit 103 evaluates the situation of the image 401 of the result data 107 by a numerical value. An example of the method will be described below.

  When the number of coordinate points of the target data 102 is m and the number of candidate areas extracted from the image 401 of the result data 107 is n, the comparative evaluation unit 103 calculates the evaluation value 110 by the following formula, for example.

Evaluation value = (m−number of coordinate points not meeting expectations) × (n−number of candidate areas not meeting expectations) / (m × n)

  The evaluation value 110 takes a value from 0 to 1, and becomes larger as the result data 107 logically matches the expectation indicated by the target data 102. In the example of FIG. 4, the number of coordinate points of the target data 102 is 9, the number of extracted candidate regions is 5, the coordinate point 305 does not match the expectation, and the candidate region 403 and the candidate region 404 match the expectation. Absent. For this reason, the evaluation value 110 is given by (9-1) × (5-2) / (9 × 5) ≈0.53.

  This calculation method of the evaluation value is an example showing a basic concept, taking into consideration the shape and area of the candidate area, the positional relationship and distance from the coordinate point specified by the target data 102, the processing time of the candidate extraction unit 106, and the like. May be calculated.

  The evaluation value is preferably calculated by preparing a plurality of sets of the image data 105 and the target data 102. In that case, as the evaluation value 110, an average value of evaluation values calculated for a plurality of sets of prepared image data 105 and target data 102 can be used. That is, when the number of sample image data is N and the evaluation value for the sample k is Vk, the comparative evaluation unit 103 calculates the evaluation value 110 by the following equation, for example.

(Evaluation value for all samples) = (ΣVk) / N
(Where k = 1, 2,... N)

  Hereinafter, the difference in logical conditions when a group is set for a plurality of coordinate points marked with “×” and “+” (when connected by line segments) will be described with reference to FIGS. Detailed description will be made using this method.

  Of these, a plurality of coordinate points for which one group is designated among the “x” marks are expected in the result data 107 to be extracted as one region in which candidate regions including all these coordinate points are connected. To do. In addition, it is expected to be extracted as an isolated region independent of other coordinate points not designated in the same group.

  FIG. 5 shows a case where the target data 102 specifies that two coordinate points marked with “x” are included in different groups (two coordinate points are connected by a line segment in the target data 102). It is an example of the evaluation result of (when there is no). In the four examples A to D of the extracted candidate areas (hatched parts), only the example A meets the expectation. Therefore, the comparative evaluation unit 103 calculates a high evaluation value for Example A, and calculates a lower evaluation value than Example A for Examples B to D.

  FIG. 6 shows a case where two coordinate points with an “x” mark are designated as one group in the target data 102 (when two coordinate points are connected with a straight line in the target data 102). It is an example of an evaluation result. In the four examples A to D of the extracted candidate areas, only example B meets the expectations. Therefore, the comparative evaluation unit 103 calculates a high evaluation value for the example B, and calculates a lower evaluation value for the example A, the example C, and the example D than the example B.

  FIG. 7 shows a case where two coordinates are designated as one group among the three coordinate points marked with “x” in the target data 102 and the other one is designated as another group (target data 102). Shows a case where two of the three coordinate points are connected by a line segment). In the four examples A to D of the extracted candidate regions, only example B meets all expectations. Accordingly, the comparative evaluation unit 103 calculates a high evaluation value for the example B, and calculates a lower evaluation value than the example B for the example A, the example C, and the example D.

  On the other hand, for a plurality of coordinate points designated as one group among the “+” marks, the result data 107 is expected to extract a candidate area including any coordinate point. In this case, it does not matter whether the candidate areas are connected or not, and there may be coordinate points not included in the candidate areas. On the other hand, it is expected that other coordinates not designated in the same group are extracted as independent isolated regions.

  FIG. 8 shows a case where two coordinate points with a “+” mark are designated to be included in different groups (when the two coordinate points are not connected by a line segment in the target data 102). . In this case, the coordinates are the same as the two “x” coordinates in FIG. In the four examples A to D of the extracted candidate regions, only example A meets expectations. Therefore, the comparative evaluation unit 103 calculates a high evaluation value for Example A, and calculates a lower evaluation value than Example A for Examples B to D.

  FIG. 9 shows a case where two coordinate points with a “+” mark are designated as the same group (when two coordinate points are connected by a line segment in the target data 102). In the four examples A to D of the extracted candidate regions, Example A, Example B, and Example C meet expectations. Therefore, the comparative evaluation unit 103 calculates an equally high evaluation value for Examples A to C, and calculates a lower evaluation value for Example D than Examples A to C.

  FIG. 10 shows a case where two coordinate points are designated as one group and three remaining coordinate points are designated as another group (in the target data 102). 2 of 3 coordinate points are connected by a line segment). In four examples A to D of the extracted candidate regions, examples A and B meet expectations. Accordingly, the comparative evaluation unit 103 calculates an equally high evaluation value for Example A and Example B, and calculates a lower evaluation value for Example C and Example D than for Example A and Example B.

  As described above, the user can easily set the logical condition to be satisfied by the result data 107 obtained by applying the image processing algorithm flexibly and simply by properly using the “x” mark and the “+” mark. This setting is simpler than when a target image is prepared as in the conventional example.

  Next, a more flexible method for setting the target data 102 will be described with reference to FIGS. FIG. 11 is an example of an image whose extraction target is ambiguous. The image data 105 is ambiguous because the object itself is not of a clear shape, or the object is a foreign object or floating object that exists in a three-dimensionally out-of-focus position, or because of vibrations that occur during imaging. Occurs when blurred.

  When such an object is to be extracted by image processing, it is not so important what region shape, size, or position it is. However, when it is desired to extract an area of a certain size, the target data 102 is designated as shown in the setting example of FIG. In FIG. 12, coordinate points with “x” marks at six locations in the target area are set as one group, and “3+” is designated. “3+” is a designation that three or more coordinates only need to be included. Accordingly, the setting example in FIG. 12 means that one connected region including three or more of the six coordinate points with the “x” mark is expected to be extracted.

  In Examples A to F of the six candidate regions (hatched portions) shown in FIG. 13, Examples A to C including three or more coordinate points in the candidate region meet expectations. In Example D, only two coordinate points are included in the candidate area, and in Example E, the candidate area is extracted in two. Further, in Example F, the candidate area is not extracted in the first place.

  In this case, the comparative evaluation unit 103 calculates an equally high evaluation value for Examples A to C, and calculates an evaluation value lower than Examples A to C for Examples D to F. Note that, as in Example C in FIG. 13, the order of the coordinate points on the line segment that designates belonging to the same group is not related to the expected result.

  On the other hand, when the coordinate point is specified by the “+” mark instead of the “x” mark, the extraction of candidate regions including three or more coordinate points as a whole is expected regardless of the number of candidate regions and the connection. It becomes. An example is shown in FIG. In example A to example F of the six candidate areas (hatched parts) shown in FIG. 14, examples A, B, C, and E that include three or more coordinates in the candidate area meet expectations. . The comparative evaluation unit 103 calculates an equally high evaluation value for Example A to Example C and Example E, and calculates an evaluation value lower than Example A to Example C and Example E for Example D and Example F.

  As another setting method for extracting an area of a certain size in an ambiguous extraction target image as shown in FIG. 11, an expected size may be designated by an area. Here, such an example will be described with reference to FIGS. 15 and 16.

  FIG. 15 shows an example in which extraction of a region having an area of 300 pixels (pixels) to 350 pixels is specified. FIG. 15 means that one coordinate region including six coordinate points with “x” marks is expected to be extracted with an area of a specified range. In the example A to example C of the three candidate regions (hatched parts) shown in FIG. 16, the area of the example B is 318 pixels, which matches the expected area range. In Example A, the area of the extracted region is 670 pixels, which is too larger than the expected area range, and in Example C, the area of the extracted region is 152 pixels, which is too smaller than the expected area range. Each of the three examples A to C includes six coordinates of the “x” mark and is a connected region, but the comparative evaluation unit 103 calculates a high evaluation value for example B, For example A and example C that do not match the specification, an evaluation value lower than that of example B is calculated.

  The method for specifying the region to be extracted as the target data 102 in the target setting unit 101 has been described above. Next, a case in which a part (non-extraction target) that is not desired to be extracted will be described.

  FIG. 17 shows how the target data 102 is set on the screen of the display 203. In FIG. 17, a sample image 1501 acquired for product inspection and coordinate points set by the operator are displayed inside the window of the screen.

  A sample image 1501 corresponds to the image data 105. In the case of the sample image 1501, foreign matter 1502 is attached to the product. The sample image 1501 includes a large hollow foreign object 1503. Although the product 1504 is not an extraction target, it has a similar appearance characteristic as a foreign substance, and is close to the foreign substance 1503. Therefore, the product 1504 is a pattern that is highly likely to be erroneously extracted in the image processing extraction process.

  In this case, the coordinate point 1508 of the “x” mark is specified for the foreign object 1502, and three coordinate points 1509, 1510, and 1511 are specified for the foreign object 1503. The designation method is the same as in the above example. On the other hand, if it is important that the product 1504 is not extracted, a coordinate point 1517 marked with “◇” is designated as a non-extraction target. If it is important that the hollow portion of the foreign material 1503 is not extracted, coordinate points 1515 and 1516 marked with “◇” are designated as the same group as non-extraction targets. The evaluation method of the comparative evaluation unit 103 for the coordinate points 1508 to 1511 with the “x” mark may be the same as described above. For coordinate points 1515 to 1517 marked with “◇”, a high evaluation value is output when they are not included in the extracted area. Coordinate points 1515 and 1516 of the same group may be given a high evaluation value when both are not included in the extraction region, or may be set to a high evaluation value when any one or more are not included in the extraction region. May be. Alternatively, as in the case of the “×” mark and the “+” mark, the difference between the two coordinate points may be distinguished by the “◇” mark and the “□” mark, and the number specified (such as “2+”). May be added.

  Furthermore, another setting method of the target data 102 will be described with reference to FIGS. FIG. 18 shows that there are two ambiguous objects (candidate area 1601 and candidate area 1602). When trying to cause the candidate extraction unit 106 to extract these two objects, the coordinate point indicated by “●” is connected by a line segment to draw a closed region, and one candidate is placed inside the drawn closed region. Expect the region to be extracted.

  In FIG. 18, the candidate area 1601 is surrounded by four coordinate points 1603 to 1606 marked with “●” and four line segments, and the candidate area 1602 is marked with three marks marked with “●”. Coordinate points 1607 to 1609 are surrounded by three line segments.

  In this case, the comparative evaluation unit 103 calculates a high evaluation value when one independent candidate region is extracted in each of the two closed spaces surrounded by the coordinate points indicated by “●” marks. FIG. 18 shows an example in which the coordinate points designated by the “●” mark are connected by line segments to enclose a predetermined candidate area. However, as shown in FIG. 19, between the coordinate points designated by the “●” mark. May be surrounded by a predetermined curve. The setting method of the target data 102 shown in FIG. 18 and FIG. 19 is slightly troublesome because the number of coordinate points to be specified is larger than the method described above, but the target data 102 is an object like “x” mark or “+” mark. It is excellent in workability when it is not easy to specify coordinate points inside an object.

  In addition, when a non-object (object that is not desired to be extracted) is not included and a plurality of objects are approaching, a method of specifying the number of regions is added as shown in FIG. May be. FIG. 20 is an example of designating that there are seven objects to be extracted inside a closed space surrounded by coordinate points 1800 to 1803 and four lines. For this reason, the number designation 1804 of “7” is added to the screen. The comparative evaluation unit 103 extracted seven candidate regions from the result data 107 in the region specified by the target data 102 (in the closed space surrounded by a line between the coordinate points specified by the “●” mark). When a high evaluation value is calculated. In this case, the target data 102 can be set with a smaller number of coordinates than when using the “x” mark and the “+” mark, and high workability can be expected.

  Furthermore, the target data 102 can be set such that if “0” is set in the number designation, an object to be ignored such as noise is not extracted. An example is shown in FIG. In the case of FIG. 21, a plurality of noises are included inside a closed space surrounded by three coordinate points 1900 to 1902 and three lines designated by the “●” mark. Here, the target data 102 is set by adding the number designation 1903 of “0” to the closed space. The comparison / evaluation unit 103 that has input the target data 102 does not extract a candidate area within the area specified by the target data 102 (within the closed space surrounded by a line between the coordinate points specified by the “●” mark). A high evaluation value is calculated.

  Next, a setting example of the target data 102 that specifies that a non-target region (a region that is not desired to be extracted) is included in the region surrounded by coordinates will be described. FIG. 22 shows an example. In the case of FIG. 22, there are six parts that do not want to be extracted in the target 2005 to be extracted. These six parts are surrounded by coordinate points 2001 to 2004 with “◯” marks, and the number designation 2000 of “6” is added. The comparative evaluation unit 103 that has input the target data 102 calculates a high evaluation value when six parts are not extracted inside the designated closed space.

  Similarly to the case shown in FIG. 21, for a region that is not an extraction target, a closed space is set by connecting the coordinate points of the “◯” mark with a line, and “0” is specified for the number to specify noise. Such a part may be ignored.

[Processing Image of Candidate Extraction Unit 106]
FIG. 23 shows an image of processing functions built in the candidate extraction unit 106. The candidate extraction unit 106 performs various processes on the image data 105 and outputs the processing results as result data 107. Therefore, the candidate extraction unit 106 includes one or more types of image processing functions 2102, 2103, 2104, 2105,. The image processing functions 2102, 2103, 2104, 2105... Execute predetermined processing on the processing results of the image data 105 and other processing functions and output the result data 107. The processing content includes, for example, noise reduction processing, luminance correction processing, edge enhancement processing, color separation processing, two-image difference calculation processing, two-image sum calculation processing, smoothing processing, binarization processing, and the like. The details of each processing function are adjusted by the parameter value given as the setting data 109. In addition, the setting data 109 also specifies which of the built-in image processing functions is used and how the input and output are connected.

  24 and 25 show connection examples of the image processing function. FIG. 24 shows a case where “processing a” 2102 and “processing e” 2200 are connected in series and used. In this case, the candidate extraction unit 106 executes “process a” 2102 on the image data 105, executes the process “process e” on the result, and outputs the result as result data 107. FIG. 25 shows a case where “processing a” 2102, “processing b” 2103, and “processing c” 2104 are used in combination. In this case, the candidate extraction unit 106 executes “processing a” 2102 and “processing b” 2103 on the image data 105, and further executes “processing c” 2104 on the two processing results. Is output as result data 107. As described above, the candidate extraction unit 106 can output the result data 107 obtained by performing various processes by changing the processing configuration and parameters according to the setting data 109.

[Processing of the automatic optimization unit 108]
FIG. 26 shows an example of the optimization processing operation executed by the automatic optimization unit 108. It is assumed that target data 102 and a target evaluation value (target evaluation value) are set in advance as a premise of the processing operation of the automatic optimization unit 108. The target evaluation value is, for example, 0.9 or 1.0.

  First, the automatic optimization unit 108 performs initialization processing (S01). In the initialization process, the automatic optimization unit 108 randomly generates the processing configuration and parameter plan of the candidate extraction unit 106 as setting data 109.

  Next, the automatic optimization unit 108 causes the candidate extraction unit 106 to perform image processing, and outputs the result data 107 (S02).

  Next, the automatic optimization unit 108 acquires the evaluation value 110 from the comparative evaluation unit 103 that inputs the result data 107 and the target data 102 set in advance and executes a predetermined evaluation (S03).

  Next, the automatic optimization unit 108 determines whether or not the evaluation value 110 has reached the target evaluation value (S04). When the evaluation value 110 has reached the target evaluation value, the automatic optimization unit 108 terminates the optimization process using the setting data for which the evaluation value can be obtained as an optimization result (S06). On the other hand, when the evaluation value 110 does not reach the target evaluation value, the automatic optimization unit 108 generates a new processing configuration and parameter proposal based on the acquired evaluation value 110 and updates the setting data 109. (S05).

  For example, when the evaluation value 110 is not 0 (zero), the automatic optimization unit 108 changes only a part of the setting data 109 at random, and when the evaluation value 110 is 0 (zero), the automatic optimization unit 108 , The entire setting data is regenerated at random as in S01. Thereafter, the automatic optimization unit 108 returns to the image processing execution step (S02) and repeats a series of processes.

  If a processing configuration and parameters that can be expected to have a certain result are known in advance, the automatic optimization unit 108 may give the processing configuration and parameters to the setting data 109 instead of the step of S01. In step S05, the automatic optimization unit 108 may compare the current evaluation value 110 with the previous evaluation value 110 and adjust the processing configuration or parameters used in the next plan. It is also possible to cause the candidate extraction unit 106 to execute a plurality of plans at a time, compare the results (evaluation value 110), and refer to the comparison result for generation of a new plan. The automatic optimization unit 108 may regenerate the processing configuration and parameters according to a genetic algorithm or a neural network.

[Summary]
As described above, in this embodiment, the image analysis support apparatus capable of changing the processing configuration and parameters according to the setting data allows (1) the link relationship between the coordinate points and the coordinate points between the coordinates to be extracted from the image data 105 ( (2) The evaluation value 110 that becomes higher as the result data 107 of the candidate extraction unit 106 logically matches the target data 102 is a predetermined threshold value (target) When the value is less than (evaluation value), the setting data 109 is changed. When the value is equal to or greater than a predetermined threshold value, a function for automatically adjusting the setting data 109 at that time as a final result is adopted.

  As described above, in this embodiment, the link relationship between coordinate points is designated as the target data 102. For this reason, it is not necessary to prepare an appropriate target image as in the prior art, and it is possible to significantly reduce the labor and time required to create the target data necessary for the evaluation of the result data 107 as compared with the conventional example. . Further, in this embodiment, even when a good extraction result is not obtained in advance or when the extension of the region to be extracted is not clear, it is possible to designate the region including the coordinate point, and an image that can be input as an analysis target The range of data can be expanded.

  Further, in this embodiment, since the evaluation value for the result data 107 is calculated according to the logical operation matching degree between the target data 102 and the result data 107 given by the link relationship between coordinate points, simple pattern matching is Differently, it is possible to evaluate the result data 107 including relationships and conditions required between individual parts and a plurality of parts.

  For example, a high evaluation value can be given when the region extracted as the result data 107 includes a specified number or more of a plurality of coordinate points specified by the target data 102. For example, when a plurality of coordinate points specified by the target data 102 exist in the same region of the result data 107, a high evaluation value can be given. Also, for example, if a specified number of coordinate points are included, even if those coordinate points are dispersed in a plurality of areas, a high evaluation value is given, or when coordinate points are dispersed in a plurality of areas on the contrary It is also possible not to give a high evaluation value. In addition, a high evaluation value can be given when an excluded portion designated by coordinate points is included in one area, or a high evaluation value can be given when a plurality of independent areas exist.

  As described above, in this embodiment, when the coordinates as the target data 102 are designated, logical conditions (for example, “+” mark, “x” mark, number condition, group relation) can be designated at the same time. This makes it easier to develop and adjust image processing algorithms than in the prior art.

  Note that the logical conditions that can be input as the target data 102 may include gradation information (hue, gradation value (for example, white / black)). Of course, the comparative evaluation unit 103 has a logical operation function that gives a high evaluation value to the result data 107 that matches various logical conditions that can be designated as the target data 102.

[Example 2]
In this embodiment, an image analysis support apparatus that detects a position of a predetermined pattern from an image obtained by capturing a product (for example, a semiconductor wafer) will be described. The apparatus configuration is the same as that of the first embodiment.

  In the present embodiment, a case will be described in which the candidate extraction unit 106 detects a position of a pattern that matches a specific pattern from the image data 105 by a template matching method. For this reason, the candidate extraction unit 106 in the present embodiment includes a pattern matching process as one of the processes. In this embodiment, the result data 107 output from the candidate extraction unit 106 is always candidate coordinates.

[Template image]
FIG. 27 shows an example of a template image 2501 used for template matching. A template image 2501 shown in FIG. 27 includes a cross-shaped position detection pattern 2502.

[Sample image and target data setting example]
FIG. 28 shows the operation when setting the target data 102 on the screen of the display 203. A sample image 2601 to be inspected is displayed in the window of the screen. The sample image 2601 includes a cross-shaped pattern 2603 similar to the pattern 2502 of the template image 2501 (FIG. 27). The operator designates a portion 2602 that matches the template image 2501 with the stylus pen 204. In this embodiment, for specifying the part 2602, for example, a square having the same size as the template image 2501 is displayed in advance, and the part 2602 is specified by a method of appropriately moving the display position of the displayed square with a stylus pen. At this time, the position of the part 2602 is designated by a coordinate point 2604 indicated by a “x” mark in the drawing. In the case of a sample in which a plurality of patterns to be searched exist in the image, the operator may set a plurality of coordinates in the target data 102.

  By the way, the pattern of an image given as a search target is not always clear. For example, as shown in FIG. 29, a pattern similar to the pattern of the template image 2501 is damaged, and it may be difficult to predict the position at the time of pattern matching, or high position accuracy may not be questioned. Therefore, in this embodiment, a first allowable range circle 2702 and a second allowable range circle 2703 centering on the coordinate point 2704 of the “×” mark given as the target data 102 are set. Here, the first allowable range circle 2702 is a range that is sufficiently allowed for the coordinate point 2704. On the other hand, the second allowable range circle 2703 outside the first allowable range circle 2702 is a range that is allowed to detect that a pattern is detected although it is displaced from the coordinate point 2704.

  The comparative evaluation unit 103 calculates a high evaluation value if the candidate coordinates of the result data 107 are located within the first allowable range circle 2702, and the first evaluation range circle 2702 and the second allowable range circle 2703 are calculated. If it is located in between, a lower evaluation value is given than when it is located within the first tolerance circle 2702.

  Also in this embodiment, when there are a plurality of coordinate points specified by the target data 102 in the sample image, a logical relationship can be defined between the plurality of coordinate points. FIG. 30 shows a case where there are a plurality of parts including a pattern similar to the template pattern in the image.

  For example, if the coordinate points 2802 and 2803 for designating the positions of the two parts 2800 and 2801 are designated by “+” marks and the respective coordinate points are connected by line segments, the logical condition is that the two parts belong to the same group. Can be specified as In this case, since the coordinate point is a “+” mark, it is set that any one or more of the two coordinate points 2802 and 2803 have been successfully matched. In the case of this setting, the comparative evaluation unit 106 gives a predetermined evaluation value if the candidate coordinates are within an allowable range for either one, and gives a higher evaluation value if both are within the allowable range, or those Give the average value. In FIG. 30, if coordinate points 2802 and 2803 are designated by “x” marks, it is set to require successful matching at both the two coordinates 2802 and 2803.

  Next, an example in which a logical relationship is defined between more coordinate points will be described. FIG. 31 shows an example in which there are four parts (2900 to 2903) including a pattern similar to the template pattern in the image, and the logical condition that the matching should be successful at some coordinates among them is specified. It is.

  In FIG. 31, the positions of the four patterns are designated as the same group by connecting the coordinate points 2904 to 2907 of the “+” mark with line segments. Furthermore, as shown in the figure, if a number designation 2908 of “2+” is added, it is possible to set that any two or more matching needs to be successful.

  FIG. 32 and FIG. 33 show the target data 102 even when the target is shifted in one direction in the image used for specifying the target data 102 due to vibration at the time of imaging and the manufacturing process of the target. It is an example which adjusts the tolerance | permissible_range of according to it.

  FIG. 32 shows an example in which an allowable range 3001 extended in the pattern displacement direction in the image 3003 is set by being surrounded by a coordinate point and a line segment of the “●” mark. In this case, the comparison evaluation unit 106 operates to calculate a high evaluation value if there is only one candidate coordinate in the result data 107 within this range. FIG. 33 is an example in which the number designation 3100 of “2” is added. FIG. 33 is expected to detect a matching position for each of two shifted patterns, and the comparative evaluation unit 106 calculates a high evaluation value if there are two candidate coordinates of the result data 107 within this range. To do.

[Summary]
As described above, even in the case of the present embodiment, a simple method of simply designating the position of a pattern to be extracted by pattern matching with one or a plurality of coordinate points and setting an allowable range for each coordinate point. Thus, the target data 102 suitable for the evaluation of the pattern matching result can be input. Also in the present embodiment, a logical relationship between a plurality of coordinate points can be designated as the target data 102. For this reason, development of an image processing algorithm and adjustment of a processing configuration in which a high evaluation value is calculated for the desired result data 107 can be realized with less manpower and time.

[Other embodiments]
In the above-described embodiment, the image analysis support apparatus that processes an image acquired from a product has been described. However, the present invention can also be applied to an image analysis support apparatus used for diagnosis of a medical image obtained by counting the number of cells or an affected area. . For example, when the image data is a captured image of a semiconductor device, the present invention can be applied to an electronic scanning inspection apparatus, an electronic scanning measurement apparatus, and an electronic scanning review apparatus. The charged particles used for the inspection are not limited to electrons but may be ions. For example, when the image data is a medical image, the present invention can be applied to a medical diagnostic apparatus and a medical analysis apparatus.

  In addition, this invention is not limited to the Example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of a certain embodiment can be replaced with a configuration of another embodiment, and a configuration of another embodiment can be added to a configuration of a certain embodiment. Moreover, it is also possible to add, delete, or replace another configuration for a part of the configuration of each embodiment.

  Moreover, you may implement | achieve some or all of each structure, a function, a process part, a process means, etc. which were mentioned above as an integrated circuit or other hardware, for example. Each of the above-described configurations, functions, and the like may be realized by the processor interpreting and executing a program that realizes each function. That is, it may be realized as software. Information such as programs, tables, and files for realizing each function can be stored in a memory, a hard disk, a storage device such as an SSD (Solid State Drive), or a storage medium such as an IC card, an SD card, or a DVD.

  Control lines and information lines indicate what is considered necessary for the description, and do not represent all control lines and information lines necessary for the product. In practice, it can be considered that almost all components are connected to each other.

  DESCRIPTION OF SYMBOLS 101 ... Target data setting part, 102 ... Target data, 103 ... Comparison evaluation part, 104 ... Imaging part, 105 ... Image data, 106 ... Candidate extraction part, 107 ... Result data, 108 ... Automatic optimization part, 109 ... Setting data , 110 ... evaluation value, 201 ... keyboard, 202 ... mouse, 203 ... display, 204 ... stylus pen.

Claims (14)

A candidate extraction unit that applies a predetermined image processing algorithm to input image data and outputs result data; and
A target data setting unit that accepts target data used for evaluation of the result data as coordinate points on an image corresponding to the image data input for setting the target data;
Comparing the result data and the target data logically, a comparison evaluation unit that calculates a higher evaluation value for the result data having a higher degree of matching with the target data;
An automatic optimization unit that automatically determines a setting value of the image processing algorithm based on the evaluation value;
Have
When the result data is for extracting one or a plurality of parts in the image corresponding to the image data, the target data is link information that specifies whether or not a plurality of coordinate points belong to the same group including
An image analysis support apparatus characterized by that . The image analysis support device according to claim 1 ,
The link information is indicated as a line segment connecting coordinate points belonging to the same group on the target data designation input screen. The image analysis support device according to claim 1 ,
A coordinate point indicating that a high evaluation value may be given if any one of a plurality of coordinate points associated with the same group is included in the corresponding part is displayed on the first target data designation input screen. An image analysis support device characterized by being represented by the symbol. The image analysis support apparatus according to claim 3 ,
Specifying a logical condition indicating that a high evaluation value may be given if one or more corresponding parts are included in one or more corresponding parts among a plurality of coordinate points associated with the same group. An image analysis support apparatus characterized by being expressed by specifying the number of coordinate points on an input screen. The image analysis support device according to claim 1 ,
A coordinate point indicating that a high evaluation value may be given if a plurality of coordinate points associated with the same group are included in the same corresponding part is represented by a second symbol on the target data designation input screen. An image analysis support device characterized by the above. The image analysis support device according to claim 5 ,
On the designation input screen for the target data, a logical condition indicating that a high evaluation value may be given if one or more of the designated number of coordinate points associated with the same group is included in one corresponding part An image analysis support apparatus characterized by being expressed by specifying the number of coordinate points in. The image analysis support device according to claim 1,
When the result data is for extracting a part in the image corresponding to the image data, the logical area indicating that a high evaluation value may be given to the corresponding part including the coordinate point, the area of the corresponding part is An image analysis support device characterized by specifying. A candidate extraction unit that applies a predetermined image processing algorithm to input image data and outputs result data; and
A target data setting unit that accepts target data used for evaluation of the result data as coordinate points on an image corresponding to the image data input for setting the target data;
Comparing the result data and the target data logically, a comparison evaluation unit that calculates a higher evaluation value for the result data having a higher degree of matching with the target data;
An automatic optimization unit that automatically determines a setting value of the image processing algorithm based on the evaluation value;
Have
Coordinates indicating that when the result data is for extracting one or a plurality of parts in an image corresponding to the image data, a high evaluation value may be given when the coordinate points are not included in the corresponding parts A point is represented by a third symbol on the target data designation input screen.
An image analysis support apparatus characterized by that . A candidate extraction unit that applies a predetermined image processing algorithm to input image data and outputs result data; and
A target data setting unit that accepts target data used for evaluation of the result data as coordinate points on an image corresponding to the image data input for setting the target data;
Comparing the result data and the target data logically, a comparison evaluation unit that calculates a higher evaluation value for the result data having a higher degree of matching with the target data;
An automatic optimization unit that automatically determines a setting value of the image processing algorithm based on the evaluation value;
Have
The result data is for extracting one or a plurality of parts in an image corresponding to the image data, and a plurality of coordinate points form a closed space on the target data designation input screen. The comparative evaluation unit gives a high evaluation value when the corresponding part is included in the closed space.
An image analysis support apparatus characterized by that . The image analysis support device according to claim 9 ,
A logical condition indicating that a high evaluation value may be given if the specified number of parts are included in the closed space is expressed by specifying the number of parts on the designation input screen of the target data. Image analysis support device. The image analysis support device according to claim 1,
The automatic optimization unit changes the setting data when the evaluation value is less than a predetermined threshold value, and sets the setting data as a final result when the evaluation value is equal to or more than a predetermined threshold value. Image analysis support device. The image analysis support device according to claim 1,
When the result data is given as one or a plurality of coordinate values, the target data is given as target coordinate points and permissible range information for each coordinate point. The image analysis support device according to claim 12 ,
The target data includes link information that designates whether or not a plurality of coordinate points belong to the same group. A process of applying a predetermined image processing algorithm to input image data and outputting result data;
Processing for receiving target data used for evaluation of the result data as coordinate points on an image corresponding to the image data input for setting the target data;
The result data and the target data are logically compared, and the result data having a higher degree of coincidence with the target data calculates a higher evaluation value;
Based on the evaluation value, it has a processing for automatically determining the set values of the image processing algorithms,
When the result data is for extracting one or a plurality of parts in the image corresponding to the image data, the target data is link information that specifies whether or not a plurality of coordinate points belong to the same group including,
Image analysis support method.

Images