JP5447641B2 - Model image acquisition support apparatus, model image acquisition support method, and model image acquisition support program - Google Patents

Description

  The present invention is directed to an image processing apparatus that performs matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject.

  In the FA (Factory Automation) field, so-called visual sensors are practically used as devices for optically inspecting defects such as workpieces that are measured and recognizing characters printed on the surface. It has become. As a typical example of a technique for performing such inspection and recognition, matching processing based on a preset model image is generally used (see, for example, Patent Document 1).

  In this matching process, a similarity (typically, a preset model image) among measurement images (measurement images) obtained by imaging a subject such as a semi-finished product in a manufacturing process or a product before shipment (typically) , A search process for searching for a region having the highest correlation value is executed. Therefore, the matching process is also referred to as a search process.

Japanese Patent Laid-Open No. 11-085907

  In an actual production line, there are cases where the imaging position of the workpiece flowing on the production line slightly shifts or the lighting environment fluctuates. Even when such a disturbance factor occurs, it is necessary to stably perform the matching process. Therefore, when actually applying the above-described visual sensor to a production line, what model image is selected is important.

  At present, such a model image is set by a skilled worker based on experience. Specifically, some model image is set in advance, and based on this set model image, matching processing is performed on the actually acquired measurement image. The process of changing to an image is repeated, and finally a model image that seems to be optimal is selected. Therefore, the operation of searching for a model image is repeated repeatedly until a stable matching result can be obtained. Therefore, there is a problem that it takes a lot of time and labor to select a model image. In addition, there is a problem that the visual sensor cannot be actually operated during the selection of the model image.

  Also, in reality, it is not time and costly to completely check whether the selected model image can be stably matched for all the workpieces that can flow through the production line. Often it is possible. Therefore, it is necessary to adopt model images whose stability has only been confirmed for a limited number of measurement images. For this reason, after the actual operation is started, it may be found that the selected model image cannot be stably matched.

  Furthermore, the operation of what model image is cut out from the original image or which model image is selected varies depending on the skill level of the operator.

  Accordingly, the present invention has been made to solve these problems, and an object thereof is image processing for performing matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject. To provide a model image acquisition support device, a model image acquisition support method, and a model image acquisition support program that can acquire an optimum model image more quickly and easily.

  According to an aspect of the present invention, there is provided an apparatus for assisting acquisition of a model image, which is directed to an image processing apparatus that performs matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject. provide. The model image acquisition support apparatus includes: an extracting unit that extracts a plurality of model image candidates that are model image candidates from a reference image obtained by imaging a subject that can be a model; an input unit that receives a measurement image; and a measurement image On the other hand, a processing means for generating a plurality of trial results by performing matching processing based on each of a plurality of model images, an evaluation means for generating an evaluation result by evaluating each of the plurality of trial results, and an evaluation result Output means for separately outputting a plurality of model image candidates.

  Preferably, the model image acquisition support apparatus further includes means for setting a model image candidate corresponding to the selected evaluation result as a model image of the image forming apparatus in response to a user selection for the output evaluation result. .

  Preferably, the evaluation unit includes a unit that receives a condition to be satisfied by the evaluation result, and a unit that determines a model image candidate that best matches the condition among the plurality of trial results.

Preferably, the extraction unit extracts a plurality of model image candidates from the same reference image.
Preferably, the extracting unit extracts a plurality of model image candidates from which at least one of a size, a position, and a direction of the region to be extracted is different from the same reference image.

  Preferably, the extracting unit extracts a model image candidate from the reference image by performing a matching process based on the initially set model image.

  More preferably, the extraction unit acquires an initially set model image from the image processing apparatus.

  Preferably, the input unit receives at least one of an expected class and an expected value associated with the measurement image in addition to the measurement image. The evaluation unit evaluates the corresponding trial result based on the expected value and / or the expected class associated with each measurement image.

  More preferably, the expected class is a non-defective image whose associated measurement image should be matched with the model image, or a defective product image whose associated measurement image should not be matched with the model image. Indicates the category.

More preferably, the reference image is a non-defective image included in the measurement image.
Preferably, the processing unit generates, for each of the plurality of model image candidates, a trial result group including a plurality of trial results obtained by performing processing on the plurality of measurement images, and the evaluation unit includes the plurality of models. Means for calculating a statistical output for the corresponding trial result group for each of the image candidates;

  More preferably, the statistical output includes at least one of an average value, an intermediate value, a maximum value, a minimum value, and a variance value.

  Preferably, the input unit receives a plurality of measurement images, and the model image acquisition support device is further included in a plurality of model image candidates for optimizing a matching processing result for the whole of the plurality of measurement images. Search means for searching for a combination of two or more model image candidates.

  According to another aspect of the present invention, an apparatus for assisting acquisition of a model image directed to an image processing apparatus that performs matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject. I will provide a. The model image acquisition apparatus includes an extraction unit that extracts a plurality of model image candidates that are model image candidates from a reference image obtained by imaging a subject that can be a model, an input unit that receives a measurement image, and a measurement image. On the other hand, a processing unit that generates a plurality of trial results by performing matching processing based on each of a plurality of model images, an evaluation unit that generates an evaluation result by evaluating each of the plurality of trial results, Determining means for determining a model image most suitable for a predetermined condition.

  According to still another aspect of the present invention, the acquisition of a model image directed to an image processing apparatus that performs matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject is supported. Provide a method. The model image acquisition support method includes a step of extracting a plurality of model image candidates as model image candidates from a reference image obtained by imaging a subject that can be a model, a step of receiving a measurement image, A step of generating a plurality of trial results by performing matching processing based on each of the plurality of model images, a step of generating an evaluation result by evaluating each of the plurality of trial results, and a plurality of evaluation results. And outputting a model image candidate separately.

  According to still another aspect of the present invention, the acquisition of a model image directed to an image processing apparatus that performs matching processing based on a preset model image with respect to a measurement image obtained by imaging a subject is supported. Provide a program. The program includes a step of extracting a plurality of model image candidates as model image candidates from a reference image obtained by imaging a subject that can be a model, a step of receiving a measurement image, and a measurement image A step of generating a plurality of trial results by performing matching processing based on each of the plurality of model images, a step of generating an evaluation result by evaluating each of the plurality of trial results, and a plurality of models. And a step of outputting image candidates separately.

  According to the present invention, an optimal model image can be acquired more quickly and easily with respect to an image processing apparatus that performs a matching process based on a preset model image with respect to a measurement image obtained by imaging a subject.

It is a figure which shows the whole system according to embodiment of this invention. It is a schematic block diagram of the computer which implement | achieves the model image acquisition assistance apparatus according to embodiment of this invention. 1 is a schematic configuration diagram of a computer that realizes an image processing apparatus according to an embodiment of the present invention. It is an example of the workflow in the case of operating the visual sensor according to embodiment of this invention on a production line. It is a figure for demonstrating the outline | summary of the model image acquisition process in the model image acquisition assistance apparatus according to embodiment of this invention. It is a figure which shows an example of the extraction process of the model image candidate according to embodiment of this invention. It is the schematic which shows the function structure of the assistance apparatus and image processing apparatus according to embodiment of this invention. It is a figure which shows the example of a screen display in the setting mode provided by the image processing application according to embodiment of this invention. It is a figure which shows the example of a screen display in the operation mode displayed on the monitor of the image processing apparatus according to embodiment of this invention. It is a figure which shows an example (the 1) of the user interface provided by the evaluation process application of the assistance apparatus according to embodiment of this invention. It is a figure which shows an example (the 2) of the user interface provided by the evaluation process application of the assistance apparatus according to embodiment of this invention. It is a figure which shows an example (the 3) of the user interface provided by the evaluation process application of the assistance apparatus according to embodiment of this invention. It is a figure which shows an example (the 4) of a user interface provided by the evaluation process application of the assistance apparatus according to embodiment of this invention. It is a figure which shows an example (the 5) of the user interface provided by the evaluation process application of the assistance apparatus according to embodiment of this invention. It is a functional block diagram which shows the control structure of the assistance apparatus according to embodiment of this invention. It is a figure which shows the file structure produced | generated in the control structure of the assistance apparatus shown in FIG. It is a figure which shows an example of the input interface of the conditions which select the most suitable thing among model image candidates in the assistance apparatus according to embodiment of this invention. It is a flowchart (the 1) which shows the whole process in the system according to embodiment of this invention. It is a flowchart (the 2) which shows the whole process in the system according to embodiment of this invention. It is a flowchart (the 3) which shows the whole process in the system according to embodiment of this invention. It is a flowchart (the 4) which shows the whole process in the system according to embodiment of this invention. It is a flowchart (the 5) which shows the whole process in the system according to embodiment of this invention. It is a flowchart which shows the principal part of the whole process in the system according to the modification 1 of embodiment of this invention. It is a flowchart which shows the principal part of the whole process in the system according to the modification 2 of embodiment of this invention. It is a figure for demonstrating the outline | summary of the matching process according to the modification 3 of embodiment of this invention. It is a figure for demonstrating the outline | summary of the model image set acquisition process in the matching process according to the modification 3 of embodiment of this invention. It is a figure for demonstrating the outline | summary of the method 1 according to the modification 3 of embodiment of this invention. It is a flowchart which shows the procedure of the method 1 according to the modification 3 of embodiment of this invention. It is a flowchart which shows the procedure of the method 2 according to the modification 3 of embodiment of this invention. It is a figure for demonstrating the outline | summary of the method 3 according to the modification 3 of embodiment of this invention. It is a figure which shows an example of the trial result in the method 3 according to the modification 3 of embodiment of this invention. It is a flowchart which shows the procedure of the method 3 according to the modification 3 of embodiment of this invention. It is a figure which shows an example of the trial creation process according to the modification 4 of implementation of this invention. It is a block diagram for demonstrating the content of the process item according to the modification 5 of implementation of this invention. It is a figure for demonstrating optimization of the model image according to the modification 5 of implementation of this invention.

  Embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals and description thereof will not be repeated.

<Overview>
In the embodiment of the present invention, a configuration in the case where the model image acquisition support device according to the present embodiment is applied to an image processing device called a so-called visual sensor will be exemplified. This visual sensor typically receives a measurement image obtained by imaging a measurement target object (hereinafter also referred to as “work”) that is a subject flowing through a production line or the like, By performing matching processing based on the set model image, it provides workpiece defect inspection and character string discrimination.

  The model image acquisition support apparatus according to the embodiment of the present invention first extracts a plurality of model image candidates that are model image candidates from a reference image obtained by imaging a subject that can be a model. Then, the model image acquisition support apparatus according to the present embodiment generates a plurality of trial results by performing a matching process based on each of the extracted model images on the measurement image, and further, the plurality of trials An evaluation result is generated by evaluating each result. Finally, the model image acquisition support apparatus according to the present embodiment outputs the generated evaluation result separately for a plurality of model image candidates. For example, if all the target measurement images are images to be matched with the model image, it is possible to determine that the model image having the highest degree of matching with the target measurement image is optimal.

  In addition, the model image acquisition support apparatus according to the present embodiment accepts an expected class and / or an expected value associated with a measurement image. In this specification, “expected class” and “expected value” mean the contents of a processing result to be generated when matching processing is performed on a corresponding measurement image.

  Typically, the “expected class” is “good” in which the workpiece shown in the target measurement image includes a model image, or the workpiece shown in the target measurement image is different from the model image. Includes the category of “defective”. In other words, it indicates a classification of whether the target measurement image is a “non-defective product image” that should be matched with the model image, or a “defective product image” that should not be matched with the model image. In the following description, the measurement image associated with the “non-defective product” expected class is also referred to as “non-defective product image”, and the measurement image associated with the “defective product” expected class is also referred to as “defective product image”.

  Further, typically, the “expected value” includes a value indicating the degree of similarity (typically, a correlation value) between the target measurement image and the set model image. That is, when the workpiece shown in the target measurement image contains a model image, the “expected value” shows a relatively high value, and the workpiece shown in the target measurement image contains a model image. If not, the “expected value” indicates a relatively low value.

As described above, in this specification, the terms “non-defective product” and “defective product” are determined to match the target measurement image with the model image in a relative relationship with the set model image. Including whether or not it should be. For this reason, since matching processing based on different model images is often executed for the same measurement image a plurality of times, the “expected class” and / or “expected value” is one class and / or value. Not exclusively.

  Upon accepting such an expected class and / or expected value, the model image acquisition support apparatus according to the present embodiment, based on the generated trial results and the input expected class and / or expected value, An evaluation result is generated by evaluating each of a plurality of trial results. That is, an image having a high matching degree with respect to a good product image and a low matching degree with respect to a defective product image is evaluated as an optimum model image. And the model image acquisition assistance apparatus according to this Embodiment outputs this produced | generated evaluation result according to a some model image candidate.

  In the present invention, “output” means providing the content to a user or device that actually determines the model image, typically representing the content on a display device such as a display, This includes expressing the content on a paper medium by a printer device or the like, transmitting data representing the content to an external device, storing data representing the content in a storage device or the like.

  By the process as described above, it is possible to quickly and easily determine an optimal result of the matching process for an actual measurement image among a plurality of model images.

  In the following description, for the sake of simplification, “expected class” and “expected value” are collectively referred to as “expected class”. However, the term “expected class” includes “expected class” and “expected value” in the above-mentioned meaning.

<Overall device configuration>
FIG. 1 is a diagram showing an entire system according to an embodiment of the present invention.

  Referring to FIG. 1, the system according to the present embodiment includes a model image acquisition support apparatus (hereinafter also simply referred to as “support apparatus”) 100 and an image processing apparatus 200.

  The image processing apparatus 200 is electrically connected to the imaging unit 8 and the photoelectric sensor 4. The imaging unit 8 acquires a measurement image in which the workpiece 2 is captured by imaging the workpiece 2 that is a subject conveyed on a conveyance line 6 such as a belt conveyor. The image processing apparatus 200 stores the measurement image acquired by the imaging unit 8 and performs matching processing based on a preset model image for each measurement image, and the processing result (for example, matching OK or (Matching NG judgment) is output.

The imaging timing of the workpiece 2 by the imaging unit 8 is detected by the photoelectric sensors 4 (light receiving unit and light projecting unit) arranged on both sides of the conveyance line 6. That is, the photoelectric sensor 4 includes a light receiving unit and a light projecting unit disposed on the same optical axis, and the light receiving unit detects that light emitted from the light projecting unit is shielded by the work 2. A trigger signal indicating the imaging timing is output. In addition to an optical system such as a lens, the imaging unit 8 includes a CCD (Coupled Charged Device) and a CMOS (Complementary Metal Oxide Semiconductor) sensor.
Includes an image sensor. The measurement image acquired by the imaging unit 8 may be a black and white image or a color image.

On the other hand, the support apparatus 100 receives a measurement image and setting parameters (including an initial setting model image) stored in the image processing apparatus 200, and executes a model image acquisition process for determining an optimal model image. Then, the support apparatus 100 can set the model image acquired by the model image acquisition process, the setting parameter, and the like in the image processing apparatus 200. As means for exchanging data between the support apparatus 100 and the image processing apparatus 200, a method using communication means (wired or wireless) such as USB (Universal Serial Bus) or Ethernet (registered trademark), SD ( For example, a method using a removable storage medium such as a Secured Digital card may be employed.

  As will be described later, all or part of the functions of the support apparatus 100 may be incorporated into the image processing apparatus 200. However, the support apparatus 100 and the image processing apparatus 200 are separated from each other by separating them. The process of determining the optimum model image can be executed independently on the desk in the office.

<Hardware configuration>
(1 Model image acquisition support device)
The support apparatus 100 according to the present embodiment is typically implemented by executing a program in which a computer is installed. Alternatively, some or all of the functions provided by the computer executing the program may be embodied as a dedicated hardware circuit.

  FIG. 2 is a schematic configuration diagram of a computer that realizes model image acquisition support apparatus 100 according to the embodiment of the present invention.

Referring to FIG. 2, a computer that implements support device 100 includes a monitor 102 as a display device, a keyboard 103 and a mouse 104 as input devices, and a CPU (Central Processing Unit) 105 as an arithmetic device (processor). , A memory 106 and a fixed disk 107 as storage devices, and an FD driving device 111 and a CD-ROM driving device 113 as data reading devices from a recording medium. These units are connected to each other via a bus so that data communication is possible.

  A program executed by the support apparatus 100 (computer) is typically stored and distributed in a flexible disk (FD: Flexible Disk) 112 or a CD-ROM (Compact Disk Read Only Memory) 114, or connected to a network. It is distributed in the form of download from the distributed distribution server device. The programs stored in the flexible disk 112 and the CD-ROM 114 are read from the FD driving device 111 and the CD-ROM driving device 113, respectively, and temporarily stored in the fixed disk 107. Further, the data is expanded from the fixed disk 107 onto the memory 106 and executed by the CPU 105.

  The CPU 105 performs various operations by sequentially executing programmed instructions. The memory 106 temporarily stores various types of information according to the program execution by the CPU 105. The fixed disk 107 is a non-volatile storage device that stores a model image to be set, a measurement image to be processed, various setting values, and the like in addition to a program executed by the CPU 105.

  The keyboard 103 receives a command from the user corresponding to the input key. The mouse 104 receives a command from the user according to an operation such as click or slide. As described above, the command received by the keyboard 103 and the mouse 104 is given to the CPU 105.

In addition, other output devices such as a printer may be connected to the support apparatus 100 as necessary.

(2 Image processing device)
Similar to support apparatus 100, image processing apparatus 200 according to the present embodiment is typically implemented by executing a program in which a computer is installed. Alternatively, some or all of the functions provided by the computer executing the program may be embodied as a dedicated hardware circuit.

  FIG. 3 is a schematic configuration diagram of a computer that implements image processing apparatus 200 according to the embodiment of the present invention.

  Referring to FIG. 3, a computer that realizes image processing apparatus 200 includes a main body 201, a monitor 202 as a display device, and a keyboard 203 and a mouse 204 as input devices. The main body 201 includes a CPU 205 as an arithmetic device (processor), a memory 206 and a fixed disk 207 as storage devices, and an FD driving device 211 and a CD-ROM driving device 213 as data reading devices from a recording medium. . Further, the main body unit 201 includes a camera interface unit 209, a control information interface unit 215, and a sensor interface unit 217 as interfaces for exchanging signals with the outside of the main body unit 201. These units are connected to each other via a bus so that data communication is possible.

  A program executed by the image processing apparatus 200 (computer) is also typically downloaded from a distribution server apparatus or the like that is stored and distributed on a flexible disk (FD) 212 or CD-ROM 214 or connected to a network. It is distributed in the form. The programs stored in the flexible disk 212 and the CD-ROM 214 are read from the FD driving device 211 and the CD-ROM driving device 213, respectively, and temporarily stored in the fixed disk 207. Further, the data is expanded from the fixed disk 207 to the memory 206 and executed by the CPU 205.

  Among the configurations included in the main body unit 201, the CPU 205, the memory 206, the fixed disk 207, the FD drive device 211, and the CD-ROM drive device 213 are the same as the corresponding parts of the above-described support device 100, so The explanation will not be repeated.

  The camera interface unit 209 mediates data communication between the CPU 205 and the imaging unit 8. More specifically, the camera interface unit 209 includes an image buffer, temporarily accumulates measurement image data captured by the imaging unit 8 and continuously transmitted, and accumulates at least one frame of measurement image data. Then, the accumulated data is transferred to the memory 206 or the fixed disk 207. In addition, the camera interface unit 209 gives an imaging command to the imaging unit 8 in accordance with an internal command generated by the CPU 205.

The control information interface unit 215 mediates data communication between the CPU 205 and a control device (not shown) (typically, a programmable logic controller (PLC)) that controls the production line. The control information interface unit 215 receives line information and the like from an external control device and outputs them to the CPU 205. The sensor interface unit 217 receives a trigger signal from the above-described photoelectric sensor and outputs the trigger signal to the CPU 205.

  Since other configurations are the same as those of support device 100 described above, detailed description will not be repeated.

<Installation procedure>
First, a workflow when a visual sensor to which the support device 100 according to the present embodiment is applied is operated on a production line will be described.

  FIG. 4 is an example of a workflow when the visual sensor is operated on the production line. As shown in FIG. 4, first, there are a purpose determination phase PH1 and an installation phase PH2 as procedures until the visual sensor is installed on the production line. As a procedure after the visual sensor is installed on the production line, there are an initial imaging phase PH3, an installation adjustment phase PH4, an actual imaging phase PH5, an initial setting phase PH6, an adjustment phase PH7, and an operation (improvement) phase PH8. Note that the initial imaging phase PH3 and the installation adjustment phase PH4 proceed in parallel, and the full-scale imaging phase PH5, the initial setting phase PH6, and the adjustment phase PH7 also proceed in parallel.

  In the purpose determination phase PH1, the operator determines what kind of work is to be inspected and what items are to be inspected for the work. Further, the worker determines which range of the entire workpiece is to be inspected.

  In the subsequent installation phase PH2, the worker examines the lens and camera installation methods according to the installation environment, secures a power source for driving the imaging unit and the like, and installs necessary equipment.

  In the initial imaging phase PH3, the installation adjustment phase PH4, the full-scale imaging phase PH5, and the initial setting phase PH6, the operator selects the types of lenses and cameras for the image processing apparatus, and items to be processed by the image processing apparatus. select. In addition, a model image corresponding to a processing item used for matching processing is registered. Further, the operator sets initial values of parameters for each processing item. After such various settings have been made, a series of flow operations is confirmed for a measurement image obtained by taking a test image.

  With the above operation, a series of settings (selection of an imaging unit, an imaging environment, etc.) for imaging a workpiece is completed. Then, model images and parameters set in the image processing apparatus are optimized by trial and error. That is the adjustment phase PH7. In this phase, the operator verifies the result obtained by image processing on a test line or the like, and optimizes the model image and parameters based on the verification contents.

  After the model image and parameters have been adjusted in the adjustment phase PH7, the operation moves to the operation (improvement) phase PH8 and the operation of the visual sensor is started. Above, the model image and / or parameters are changed. The erroneously detected product means, for example, a product that is erroneously determined as a non-defective product or a product that is erroneously determined as a non-defective product. In the present specification, the term “stable” means a state in which the occurrence of erroneous detection products as described above is small.

  The support device 100 according to the present embodiment aims to improve the efficiency of processing in the adjustment phase PH7 and the operation (improvement) phase PH8 among these phases. In addition, once the adjustment of the model image and / or parameters is completed, if a lot change occurs during operation, it is necessary to adjust the model image and / or parameters again. Also, the support device 100 according to the present embodiment can be utilized. Furthermore, in the operation (improvement) phase PH8, by using the support device 100 according to the present embodiment in-line, the model image can be dynamically changed according to the state in the production line.

<Outline of model image acquisition procedure>
First, an overview of model image acquisition processing in support device 100 according to the present embodiment will be described.

  FIG. 5 is a diagram for describing an overview of model image acquisition processing in model image acquisition support apparatus 100 according to the embodiment of the present invention.

(1) Matching With reference to FIG. 5, first, a plurality of model image candidates that are model image candidates are extracted from a reference image obtained by imaging a subject that can be a model. This reference image is typically composed of a non-defective image OK_IMG. This reference image may be the same as a non-defective image OK_IMG included in a trial target image group described later. This model image candidate is extracted by performing a matching process on the reference image (non-defective image) based on the initial setting model image that is initially set in the image processing apparatus 200.

(2) Model Image Candidate Extraction The above-described matching process corresponds to a kind of search process, and a region having a high similarity (correlation value) with the initially set model image is extracted as a model image. Although a plurality of model images are extracted, as this extraction method, a plurality of model image candidates may be extracted from the same reference image, or model image candidates are extracted from each of the plurality of reference images. Or they may be combined as appropriate.

  When extracting a plurality of model image candidates from the same reference image, as shown in FIG. 6, the same model image is extracted according to a plurality of different conditions (for example, extraction size, extraction position, extraction direction (rotation direction), etc.). A plurality of model images may be extracted from the reference image. Alternatively, model images may be extracted from a plurality of reference images according to the same conditions.

  FIG. 6 is a diagram showing an example of model image candidate extraction processing according to the embodiment of the present invention. As shown in FIG. 6A, a plurality of model images MDL1-1, MDL1-2, MDL1-3,... With different extraction sizes can be extracted from the same reference image REF. Alternatively, as shown in FIG. 6B, a plurality of model images MDL2-1, MDL2-2,... With different extraction directions can be extracted from the same reference image REF.

  Since support device 100 according to the present embodiment determines an optimal model image from among a plurality of model image candidates, it is preferable to prepare more model images.

(3) Trial Referring again to FIG. 5, each trial result is obtained by executing matching processing according to the plurality of model images thus extracted on the measurement image actually acquired by the visual sensor. Get. The trial result typically includes a similarity (correlation value) between each of the measurement images and each of the extracted model images. Further, the trial result may include a determination result of matching OK or matching NG by comparing the calculated similarity with a preset threshold value.

(4) Trial Result Evaluation Further, an evaluation result is generated by evaluating each trial result of the matching process according to each model image based on the expected class associated with the target measurement image. More specifically, the higher the degree that the trial result obtained when matching processing according to a certain model image is performed on the non-defective image OK_IMG is “matching OK”, the more appropriate the model image can be evaluated. . Similarly, it can be evaluated that the model image is more appropriate as the trial result obtained when the matching process according to a certain model image is performed on the defective product image NG_IMG is “matching NG”.

  In contrast, the higher the degree of misjudgment that the trial result obtained when matching processing according to a certain model image is performed on the non-defective image OK_IMG is “matching NG”, the more inappropriate the model image is. Can be evaluated. Similarly, the higher the degree of misjudgment that the trial result obtained when matching processing according to a certain model image is performed on the defective product image NG_IMG is “matching OK”, the more inappropriate the model image is evaluated. it can.

(5) Determination of Optimal Model Image Supporting apparatus 100 according to the present embodiment statistically evaluates the degree of coincidence / mismatch between such a trial result and the expected class (or expected value) of the target measurement image. And based on such an evaluation result, the optimal thing is determined among the extracted model images. Alternatively, it assists in determining an optimal model image. That is, the support apparatus 100 according to the present embodiment outputs an evaluation result for each of a plurality of model image candidates, and the user arbitrarily selects an optimal model image based on the evaluation result, as well as the present embodiment. Any of the configurations in which the support device 100 automatically determines a model image that best matches a predetermined condition among the evaluation results can be employed.

  Note that “optimum model image” in this specification means searching for a model image that can obtain the result expected by the user. As the object to be searched for, a model image that matches a condition expected (set) by the user is determined by evaluating a measurement image acquired in advance. Therefore, when a new measurement image is acquired or when the user changes the condition, another model image may be determined as “optimal”.

  Furthermore, the model image acquired in this way can be transferred to the image processing apparatus 200 and set.

  The output evaluation result may include statistical outputs such as an average value, an intermediate value, a maximum value, a minimum value, and a variance value.

<Functional configuration>
FIG. 7 is a schematic diagram showing functional configurations of support apparatus 100 and image processing apparatus 200 according to the embodiment of the present invention.

  With reference to Fig.7 (a), the image processing apparatus 200 according to this Embodiment contains the measurement image acquisition application 10 and 30 A of image processing applications. The measurement image acquisition application 10 acquires a measurement image obtained by imaging a workpiece by giving a predetermined command to the imaging unit 8 and the camera interface unit 209 (FIG. 3). Further, the image processing application 30A performs image processing (mainly matching processing in the present embodiment) on the measurement image acquired by the measurement image acquisition application 10 according to a preset model image and parameters. , Output the processing result.

On the other hand, the support apparatus 100 includes an evaluation processing application 20 and an image processing application 30B. The function of the image processing application 30B is basically the same as that of the image processing application 30A. That is, the image processing application 30B functions as a kind of “simulator” for realizing the same processing as the matching processing executed by the image processing apparatus 200. The evaluation processing application 20 is a main part that provides the model image acquisition processing according to the present embodiment, extracts a plurality of model images from the reference image, and performs matching processing (trial) based on each model image. 30B is executed. Then, the evaluation processing application 20 supports the acquisition of an optimal model image by evaluating the trial result by the image processing application 30B based on the expected class associated with the target measurement image.

  That is, in the system configuration shown in FIG. 7A, it is possible to support acquisition of an optimal model image by the support device 100 alone based on the measurement image (and the reference image) acquired by the image processing device 200. .

  On the other hand, FIG. 7B shows a system configuration according to another embodiment of the present embodiment. In this system, support device 100 # does not include image processing application 30B, but includes only evaluation processing application 20. In this configuration, the evaluation processing application 20 supports the acquisition of an optimal model image by cooperating with the image processing application 30A of the image processing apparatus 200. That is, the support apparatus 100 and the image processing apparatus 200 are connected in a state where data communication is possible, and the evaluation processing application 20 of the support apparatus 100 gives various instructions to the image processing application 30A of the image processing apparatus 200 to perform matching processing. The image processing application 30A of the image processing apparatus 200 may return the processing result to the evaluation processing application 20 of the support apparatus 100. As described above, the processing can be advanced by synchronizing the support apparatus 100 and the image processing apparatus 200.

  In the following description, the image processing application 30A included in the image processing apparatus 200 and the image processing application 30B included in the support apparatus 100 are simply referred to as “image processing application 30”, focusing on the commonality of the functions. Also collectively referred to.

<User interface of image processing application>
First, an example of a user interface provided by the image processing application 30 (FIG. 7) will be described in order to facilitate understanding of the matching processing related to image processing.

  FIG. 8 is a diagram showing a screen display example in the setting mode provided by the image processing application 30 according to the embodiment of the present invention. This setting mode is basically selected in the above-described initial setting phase PH6. The display screen shown in FIG. 8 is provided by the cooperation of the CPU 205 of the image processing apparatus 200 (or the CPU 105 of the support apparatus 100) and a graphic board (not shown). Such a display screen is realized by a GUI (Graphical User Interface) program incorporated as a part of an OS (Operating System), and the user can use the keyboard 103 (or 203) or the mouse 104 (or 204). It also provides an environment for performing various user settings using the cursor on the screen operated by.

  As shown in FIG. 8, in the mode for performing the search using the matching process, as an example, “model registration”, “area setting”, “detection point”, “reference position”, “measurement parameter”, “output parameter” A screen on which a total of six tabs corresponding to the setting items can be selected is displayed.

  Referring to FIG. 8, when a “model registration” tab 301 is selected, a setting screen 300 is displayed. This setting screen 300 includes a model registration area 301 #, an image display area 304, an entire display area 306, and a display control icon group 308.

In the image display area 304, a measurement image acquired by the imaging unit 8 is displayed. In the entire display area 306, the measurement image acquired by the imaging unit 8 is displayed as in the image display area 304. However, the entire measurement image is displayed in the entire display area 306 independently of the display range in the image display area 304. Further, the display range and display accuracy of the measurement image displayed in the image display area 304 are changed according to a user operation (enlargement or reduction) on the display control icon group 308.

  On the other hand, a model parameter setting area 310, a model edit button 330, a registered figure display box 332, a model parameter setting area 310, and a model registration image area 340 are displayed in the model registration area 301 #.

  When the user sets a model image, the operation is performed in a state where a reference image obtained by imaging a subject that can be a model is displayed in the image display area 304 and the entire display area 306.

  First, when the user operates the mouse 204 or the like and presses the model editing button 330, a drawing tool dialog (not shown) is displayed. The user operates this drawing tool dialog to designate a range to be set as a model image on the reference image (measurement image) displayed in the image display area 304. FIG. 8 shows a case where a rectangular range including the character string “ABC” is set as the model image MDL on the image display area 304. If any model image has been set, the shape of the set model image is displayed in the registered figure display box 332 (“rectangle” in the case of FIG. 8). Note that the shape of the model image to be set is not limited to a rectangle, and may be any shape such as a circle, a sector, or an arbitrary polygon.

  In addition, when changing the setting of a set model image, the user presses a necessary button in the model registration image area 340. A reference image (measurement image) used for setting the model image is stored, and only parameters relating to the set model image can be changed later. More specifically, when the registration screen display button 342 is pressed, the reference image (measurement image) used for setting the model image is displayed. When this button 342 is pressed again, the display switches to the display of the currently input measurement image. When the model re-registration button 344 is pressed, the set model image is left as it is and is reset as a model image with other parameters changed. Further, when the delete button 346 is pressed, the set model image is deleted.

  Model parameter setting area 310 accepts selection of a search mode as a setting item. The search mode is selection of an algorithm for calculating the similarity with the model image. In this search mode, either “correlation” or “shape” can be selected by operating the radio button 312. “Correlation” is an algorithm for measuring similarity by calculating a correlation value with a model image after normalizing the brightness of an input measurement image. On the other hand, “shape” is an algorithm for measuring the similarity based on the degree of coincidence with the contour shape of the model image. Generally, more stable measurement is possible in the “correlation” mode.

  When “Correlation” is selected in this search mode, “Rotation”, “Stability”, and “Accuracy” can be set. On the other hand, when “shape” is selected in the search mode, “rotation range” and “stability” can be set.

In “Rotation”, when the workpiece rotates, a plurality of model images obtained by rotating a set model image by a predetermined angle are internally generated, and the similarity is calculated based on the generated model images. Parameters related to the process to be measured are specified. That is, when the rotation check box 314 is checked, the rotation process is validated. Then, when the rotation range (rotation angle upper limit value and rotation angle lower limit value) and the step angle are respectively input to the numerical value box 315, it is specified that a model image rotated by the step angle over the rotation range range is generated. Is done. In general, the smaller the step angle, the higher the stability, but the longer the processing time. It is also possible to set a smart mode capable of performing a rotation search at high speed.

  In “Stability”, which of the measurement stability and the processing speed is prioritized is set. That is, the slide bar 316 is set to any value within a range of a predetermined width (for example, 1 to 15), and the processing time is shortened as the set value is smaller, and the stability is increased as the value is larger. It is done.

  In “Accuracy”, which of the measurement position accuracy and the processing speed is prioritized is set. That is, the slide bar 318 is set to any value in a range of a predetermined width (for example, 1 to 3). The smaller the set value, the shorter the processing time, and the larger the value, the higher the accuracy. .

  After the contents as described above are set by the user, when an OK button 307 is pressed, it is reflected as an internal parameter of the image processing apparatus 200. If the cancel button 309 is pressed, the unreflected parameter is reset. When the “area setting” tab 302 is selected following the setting of the model image, a setting screen for designating a range to search for a portion matching the model image is displayed (not shown). On this setting screen, the user can set an arbitrary range as a search area on the image display area 304. Although the entire input measurement image can be used as the search area, it is preferable to limit the search area to a specific range from the viewpoint of reducing the processing time.

  When the “measurement parameter” tab 303 is selected following the input of the area setting, it is determined that the similarity (correlation value) in each measured coordinate matches the model image (is “OK”). Accept the conditions to do.

<Expected class of measurement image>
In a visual sensor or the like, matching OK is output for a measurement image that includes substantially the same content as a set model image, and includes content that is similar to, but essentially different from, a registered model image. Matching NG must be output for the measurement image.

  For example, it is assumed that a model image MDL including a character string “ABC” is set as shown in FIG. Consider the case where the measurement image shown in FIG. 9 is input in such a case.

  FIG. 9 is a diagram showing a screen display example in the operation mode displayed on the monitor of the image processing apparatus 200 according to the embodiment of the present invention. As shown in FIG. 9A, when a measurement image obtained by capturing a work on which the character string “ABC” included in the set model image MDL is printed is input, the corresponding area has the highest similarity. It is detected as a region having (correlation value). At this time, matching OK must be determined based on the similarity (correlation value) measured for the detected area. On the other hand, as shown in FIG. 9B, when a measurement image obtained by imaging a work on which a character string “BCD” different from the character string “ABC” included in the set model image MDL is printed is input. Must be determined as matching NG.

Therefore, when the model image MDL including the character string “ABC” is set, the expected class of the measurement image shown in FIG. 9A is “good product (good product image)” or “OK”. The expected class of the measurement image shown in (b) is “defective product (defective product image)” or “
NG ".

  As described above, in support device 100 according to the present embodiment, it is assumed that an expected class such as a non-defective product or a defective product is known for each measurement image.

  Note that, in the parameter adjustment phase of the image processing apparatus 200, as a method of actually acquiring the measurement image, a test work is flowed on the production line and images are continuously captured. At this time, the expected class may be input to each of the acquired measurement images after the user confirms the contents. Alternatively, from the viewpoint of further labor saving, a workpiece that should be detected as “good” and a workpiece that should be detected as “defective” are distinguished in advance, and only “good” workpieces are continuously detected. It is efficient to take a picture, and then continuously photograph only the “defective product”. When such a method is used, a measurement image obtained by imaging a “good” workpiece and a measurement image obtained by imaging a “defective” workpiece are stored in different folders, respectively. By doing so, both can be easily distinguished.

<User interface of evaluation processing application>
Next, an example of a user interface provided by the evaluation processing application 20 (FIG. 7) of the support apparatus 100 will be described with reference to FIGS. 10 to 14 are provided by the cooperation of the CPU 105 of the support apparatus 100 and a graphic board (not shown). Such a display screen is realized by a GUI (Graphical User Interface) program incorporated as part of an OS (Operating System), and the GUI uses a cursor on the screen operated by the user with the keyboard 103 or the mouse 104. Provides an environment for various user settings.

(1 Fluctuation setting)
First, in the support apparatus 100, when the user instructs execution of the evaluation processing application 20 (FIG. 7), an input screen 400A as shown in FIG. 10 is displayed. In this input screen 400A, a button 402 for synchronizing the evaluation processing application 20 and the image processing application 30, a button 404 for inputting a measurement image and an expected class associated with the measurement image, and an adjustment target are designated. A button 406 for performing selection is displayed so as to be selectable.

  When the user operates the mouse 104 or the like to press the button 402, a setting dialog (not shown) is displayed. The user selects the image processing application 30 for executing the trial on this setting dialog. That is, the user selects one of the image processing application 30A installed in the image processing apparatus 200 and the image processing application 30B installed in the support apparatus 100 as the image processing application 30 used for executing the trial. . By this selection, synchronization is established between the evaluation processing application 20 and the image processing application 30A or 30B.

Next, referring to FIG. 11, when the user operates the mouse 104 or the like and presses the button 404, a dialog 414 for inputting a target measurement image and an expected class associated with the measurement image is displayed. Is done. Here, each measurement image exists as one file, and each file is a folder that is distinguished by its expected class (in this example, “good product” (OK) or “defective product” (NG)). Assume that the data are stored in 416A and 416B, respectively. At this time, when the user designates a higher folder 416 (in the example shown in FIG. 11, the folder name is “111”) in which the target measurement image is stored, the “111” folder 416 is subordinate to “ The measurement image in the “OK” folder 416A is associated with the expected class of “non-defective product”, while the measurement image in the “NG” folder 416B is associated with the expected class of “defective product”. Is specified.

  In addition, another form of the input method of the expected class associated with the measurement image and the measurement image will be described later.

  Next, referring to FIG. 12, when the user operates the mouse 104 or the like and presses the button 406, an item for inputting a model image acquisition process target and a parameter for searching for an optimal model image are input. A dialog 420 is displayed. This dialog 420 includes a pull-down menu 442 for selecting an adjustment target unit, a pull-down menu 444 for selecting a reference target unit, and a pull-down menu 446 for selecting an adjustment mode. In the pull-down menu 442, designation of a processing item in which the model image to be adjusted is used is accepted. The pull-down menu 444 accepts designation of a processing item to be referred to when processing is performed using the result of other image processing. In the pull-down menu 446, the designation of the adjustment mode is accepted. This adjustment mode typically includes three modes: a parameter variation mode, an image variation mode, and a region variation mode. When the parameter variation mode is selected, it is selected when optimizing the parameters related to image processing. When the image variation mode is selected, it is selected when optimizing the model image used for the matching process. When the region variation mode is selected, the region variation mode is selected when the region to be extracted as a model image used for the matching process is optimized.

  By these operations, the model image (initial setting model image) that has already been set for the selected matching process and the current values of the related parameters are acquired. If the support apparatus 100 is connected to the image processing apparatus 200 so that data communication is possible, the contents set in the image processing apparatus 200 are transferred to the support apparatus 100.

  When the setting of the contents as described above is completed, a button 409 for setting / changing an initially set model image, a button 410 for setting a model image candidate to be tried, and an instruction to start trial And a button 412 for reflecting the selected model image on the image processing apparatus 200 and the like are displayed in a selectable manner.

  Next, when the user operates the mouse 104 or the like to press the button 409, a dialog (not shown) for accepting selection of the initial setting model image is displayed. The user designates a file to be used as the default model image by performing an operation on this dialog.

  Next, when the user operates the mouse 104 or the like to press the button 410, a dialog (not shown) for accepting reference image settings is displayed. The user designates a file to be used as a reference image by operating this dialog. Typically, when a non-defective image is used as the reference image, the user designates a folder in which the non-defective image is stored. Of course, when a reference image is prepared independently of a non-defective image, the file of the reference image may be directly specified. When the reference image is designated by the user in this way, a dialog 430 as shown in FIG. 13 is displayed. In this dialog 430, a table including a number field 432 and an image file path field 434 is displayed. In the number field 432 and the image file path field 434, image file names to be used as reference images are listed together with file path names. The number (ID) displayed in the number field 432 is identification information for specifying the reference image in the internal processing, and is uniquely determined for each reference image.

When the user operates the mouse 104 or the like and presses the trial addition button 422, a model image is extracted from the reference image set at that time, and the trial number is extracted from the extracted model image. Are allocated internally. The reference image number display 426 corresponding to the number of trials is updated according to the trial number assigned internally. When the close button 424 is pressed, the dialog 430 is closed.

  Further, when the user operates the mouse 104 or the like and presses the button 408 for instructing the start of the trial, the parameter evaluation process is started according to the variation setting input in the dialog 430.

(2 Evaluation results)
FIG. 14 shows an evaluation result screen 400B displayed when all trials are completed. FIG. 14 shows the result of all trials being completed. Not only is the result displayed all at once after the trials are completed, but also the frame itself is displayed at the start, and as the trials progress. Corresponding numerical values may be sequentially displayed.

  On the evaluation result screen 400B, an evaluation result is output for each model image candidate. More specifically, the evaluation result screen 400B includes a trial number field 452, a tried field 454, a false detection field 456, a non-defective image false detection field 458, a bad product image false detection field 460, and a maximum measurement. A table including a time field 462, a non-defective product image correlation value average field 464, a non-defective product image correlation value average field 466, a non-defective product image correlation value 3σ field 468, and a non-defective product image correlation value 3σ field 470 is displayed.

  In the trial number field 452, trial numbers assigned to the model image candidates generated previously are displayed in ascending order. In the tried field 454, a check box indicating whether or not a trial for the corresponding model image candidate has been executed is displayed.

  In the erroneous detection field 456, the non-defective product image erroneous detection field 458, and the defective product image erroneous detection field 460, the total number of erroneously detected results of the corresponding model image candidates is displayed. More specifically, in the non-defective product image detection field 458, the total number of erroneously determined non-defective images as “defective products” is displayed, and in the defective product image detection field 460, the defective product image is “good”. The total number of erroneous judgments is displayed. A value obtained by adding up the two false detection numbers is displayed in the false detection field 456.

  The maximum measurement time field 462 displays the maximum value of the processing time measured in the trial execution stage for each model image candidate. When processing is executed by the image processing application 30B of the support apparatus 100, the processing time may differ from the time actually measured by the image processing application 30A of the image processing apparatus 200. This corresponds to the processing time expected to be required to generate an image processing result at 200. The time displayed in the maximum measurement time field 462 serves as an index for considering the actual production line tact time and the like when selecting an optimal model image candidate.

The non-defective product image correlation value average field 464, the non-defective product image correlation value average field 466, the non-defective product image correlation value 3σ field 468, and the non-defective product image correlation value 3σ field 470 are measured as feature quantities for the respective measurement images. A statistical output value for the similarity (correlation value) is displayed. That is, the average value of all correlation values measured for a plurality of input non-defective images is displayed in the non-defective image correlation value average field 464, and the plurality of input non-defective image correlation value fields 466 are input. An average value of all correlation values measured for the defective product image is displayed. The non-defective product image correlation value 3σ field 468 displays 3σ values indicating the degree of variation of the overall correlation values measured for the plurality of input non-defective products images. The defective product image correlation value 3σ field 470 displays A 3σ value indicating the degree of variation of the entire correlation value measured for the plurality of input defective images is displayed.

  On the evaluation result screen 400B, among the model image candidates, model image candidates having relatively high feature amounts included in the corresponding processing result group are output in a mode different from other model image candidates. In the example shown in FIG. 14, an asterisk mark 453 is displayed for the trial number “4” having the highest quality image correlation value average.

  As shown in FIG. 14, the determination result and the statistical output are displayed in a list for each of the plurality of tried model image candidates, so that the user can easily select an optimal model image. In general, the one having the highest non-defective image correlation value and the largest difference from the defective image correlation value is capable of stable matching processing. In the example shown in FIG. It can be said that the image is optimal.

  After the optimal model image is determined in this way, the user operates the mouse 104 or the like to input the trial number of the determined model image candidate and then presses the button 412 (FIG. 13). The image is reflected on the image processing apparatus 200.

  Through such a series of procedures, the user can more quickly and easily determine a model image used for matching processing in the image processing apparatus 200.

<Input method of expected class>
As an input method of the expected class associated with the measurement image, the configuration in which the expected class is given to the folder name in which each measurement image is stored has been exemplified, but other configurations as described below may be adopted.

(1) Configuration Using File Name of Measurement Image An expected class can be assigned by embedding characters indicating the expected class in a part of the file name of the measurement image. For example, a file name “OK_xxx.jpg” is given for the above-mentioned non-defective image, and a file name “NG_xxx.jpg” is given for a defective image, so that both can be identified. . By adopting such a method, the expected class can be individually input for the measurement image. In addition, when a correlation value or the like is input as an expected value, the same method can be used.

(2) Configuration Using Image File Header Part An expected class can be assigned by embedding an expected class in the header part of a measurement image. For example, in the case of the jpeg format, a header part is prepared in accordance with the Exif standard, and therefore an expected class classification (OK or NG) and / or an expected value can be stored in this part.

(3) Configuration Using Definition File In addition to the measurement image, an expected class can be input by preparing a definition file describing the expected class and / or expected value of each measurement image. In this definition file, an expected class classification (OK or NG), a numerical value indicating an expected class, a condition, and the like are described in association with identification information such as a file name of each measurement image. When this definition file is used, a plurality of expected classes can be defined for one measurement image. For example, considering the case where the matching process of the character string “ABC” and the matching process of the character string “BCD” are continuously performed on the same measurement image, the measurement image including the character string “ABC” When the matching process for the character string “ABC” is considered to be a non-defective image, the matching process for the character string “BCD” is a defective image. Therefore, when multiple matching processes are executed for the same measurement image, it is necessary to assign an expected class to each processing item. In such a case, using a definition file is more effective. Is.

(4) Configuration set by user individually / collectively In particular, when an expected class of non-defective product / defective product is input to the measurement image, it may be input when the measurement image is selected on the support apparatus 100. That is, when the target measurement image is selected, the user selects whether the selected file / folder is a non-defective product or a defective product. Here, if the selection is made in units of files, the expected classes are individually assigned, and if the selection is made in units of folders, the expected classes are collectively given.

<Control structure>
FIG. 15 is a functional block diagram showing a control structure of support apparatus 100 according to the embodiment of the present invention. FIG. 15 shows a configuration when the evaluation processing application 20 and the image processing application 30B are mounted on the support apparatus 100 (see FIG. 7A). The control structure shown in FIG. 15 is typically provided by the CPU 105 of the support apparatus 100 executing a program. FIG. 16 is a diagram showing a file structure generated in the control structure of the support apparatus 100 shown in FIG.

  Referring to FIG. 15, support device 100 according to the present embodiment includes, as its control structure, input unit 1010, model image candidate generation unit 1020, processing unit 1030, evaluation unit 1040, and output unit 1050. Including. The functions of the input unit 1010, the evaluation unit 1040, and the output unit 1050 are provided by the evaluation processing application 20, the function of the processing unit 1030 is provided by the image processing application 30, and the function of the model image candidate generation unit 1020 is The evaluation processing application 20 and the image processing application 30 are provided jointly.

  The input unit 1010 accepts designation of a target measurement image from the user and an expectation class associated with each measurement image. The input unit 1010 can copy the specified measurement image to itself, but if the specified measurement image is accessible, the input unit 1010 acquires the target measurement image at a necessary timing, and The entity data is output to the processing unit 1030. More specifically, the input unit 1010 generates an image list 1011 in which a target measurement image and an image number to be used internally are associated with each other based on the specification of the measurement image, and the measurement corresponding to the image number. An expected class list 1012 in which an expected class for an image is associated is generated. As shown in FIG. 16A, the image list 1011 describes the position and file name where the target measurement image exists in association with the image number. As shown in FIG. 16B, the expected class list 1012 describes an expected class (in this example, a non-defective product (OK) or a defective product (NG)) in association with the image number.

  Next, the model image candidate generation unit 1020 acquires a reference image (non-defective image) from the input unit 1010 in response to the user setting, and extracts a plurality of model image candidates 40. More specifically, the model image candidate generation unit 1020 includes an input interface unit 1021, a model image extraction unit 1022, and a communication unit 1023. The input interface unit 1021 displays a dialog 420 as shown in FIG. 12 and a dialog 430 as shown in FIG. 13 in response to a user operation. Further, the input interface unit 1021 outputs the change settings input by the user to the dialogs 420 and 430 to the model image extraction unit 1022. That is, the input interface unit 1021 accepts a variation setting including a search range ARA for extracting a model image. The model image extraction unit 1022 extracts a plurality of model image candidates according to the variation setting designated by the user. These model image candidates are output to the processing unit 1030.

The communication unit 1023 is configured to be capable of data communication with the image processing apparatus 200, and acquires processing items, initial setting model images, parameter values, and the like set in the image processing apparatus 200 in response to a user operation. The communication unit 1023 can also transfer the model image and parameter values determined by the support apparatus 100 to the image processing apparatus 200.

  Next, the processing unit 1030 generates a plurality of processing results by performing matching processing on the specified measurement image according to the plurality of model images, respectively. More specifically, the processing unit 1030 includes a measurement image selection unit 1031, a model image candidate selection unit 1032, a similarity calculation unit 1033, a determination unit 1034, and a processing controller 1036.

  Processing of each unit in the processing unit 1030 is controlled by the processing controller 1036. That is, when a plurality of measurement images and expected classes associated with them are input to the input unit 1010, the results of matching processing for each of the plurality of measurement images are sequentially output for each of the plurality of model image candidates. Thus, the processing controller 1036 controls the measurement image selection unit 1031 and the model image candidate selection unit 1032 as appropriate.

  The measurement image selection unit 1031 sequentially selects target measurement images from the input unit 1010 according to a command from the processing controller 1036 and outputs the selected measurement images to the similarity calculation unit 1033. The model image candidate selection unit 1032 also sequentially selects target model image candidates from the model image candidate generation unit 1020 in accordance with a command from the processing controller 1036 and outputs the selected model image candidates to the similarity calculation unit 1033.

  The similarity calculation unit 1033 performs a matching process on the measurement image selected by the measurement image selection unit 1031 based on the model image candidate selected by the model image candidate selection unit 1032, so that the measurement image of the target is measured. The similarity (correlation value) is calculated. The similarity calculation unit 1033 outputs the calculated similarity (correlation value) to the determination unit 1034.

  The determination unit 1034 generates a processing result for the target measurement image by comparing the similarity (correlation value) calculated by the similarity calculation unit 1033 with a predetermined threshold value. Typically, the determination unit 1034 determines that the matching is OK if the similarity (correlation value) is equal to or greater than a threshold value. Otherwise, the determination unit 1034 determines that it is a matching NG.

  The determination unit 1034 generates a determination result list 1035 indicating the determination result together with the similarity calculated by the similarity calculation unit 1033. As shown in FIG. 16C, the determination result list 1035 describes the similarity of the target measurement image and the determination result (in this example, OK or NG) in association with the image number. The determination result list 1035 is generated for each attempted model image candidate.

  The processing controller 1036 gives a command to the measurement image selection unit 1031 and the model image candidate selection unit 1032 according to the execution of the trial for the model image candidate.

  Next, the evaluation unit 1040 evaluates each of a plurality of processing results generated separately for each model image candidate based on the expected class associated with the measurement image, and generates the evaluation result. More specifically, the evaluation unit 1040 includes a comparison unit 1041, a match / mismatch counter 1043, an abort processing unit 1044, a statistical processing unit 1045, a histogram generation unit 1046, and a determination unit 1047.

The comparison unit 1041 compares the expected class for each measurement image acquired from the input unit 1010 with the determination result generated by the determination unit 1034, and evaluates whether or not the contents match. More specifically, the comparison unit 1041 generates an evaluation result list 1042 indicating the evaluation result in association with the image number. As shown in FIG. 15D, the evaluation result list 1042
Describes the correspondence between the determination result of the target measurement image and the expected class in association with the image number. As a description example, “OK-OK” indicating that the evaluation result for the measurement image to which the expected class of the non-defective product (OK) is assigned is OK (OK) or the expected class of the good product (OK) is given. For example, “OK-NG” indicating that the evaluation result for the measured image is a non-defective product (NG). Only one of the match and mismatch may be described. However, if they do not match, that is, if a misjudgment occurs, the details cannot be analyzed. It is preferable to record. The evaluation result list 1042 is generated for each attempted model image candidate.

  The match / mismatch counter 1043 calculates the degree of match with the corresponding expected class for the evaluation results included in the plurality of evaluation result lists 1042 generated by the comparison unit 1041. More specifically, the match / mismatch counter 1043 counts the number of evaluation results included in the evaluation result list 1042 that do not match the corresponding expected class (those that are erroneously detected). Note that the number of erroneously determined “defective” (OK) measurement images as “defective” (NG) and the number of “defective” (NG) measurement images erroneously determined as “good” (OK), respectively. It is preferable to count separately.

  The censoring processing unit 1044 accepts an allowable upper limit value specified by the user, and for any model image candidate, the number of processing results that do not match the corresponding expected class is specified during the generation of the processing results for the measurement image. If the allowable upper limit value is exceeded, generation of processing results for the remaining measurement images for the model image candidate is discontinued. This allowable upper limit value is set according to the stability required for the target image processing apparatus. That is, when the number of erroneous detections exceeds the allowable upper limit value, it can be determined that the model image candidate cannot perform stable image processing, and therefore no further evaluation is required. In this way, the time required for the entire trial can be reduced by aborting the trial for the model image candidate that is clearly ineligible based on the request of the application destination. Note that the abort processing unit 1044 gives an abort instruction to the processing controller 1036 of the processing unit 1030.

  Alternatively, in a certain trial, evaluation is performed on a predetermined number of measurement images, and when the average value of the result is worse than the result of the trial executed earlier, generation of the processing result for the trial is aborted. Also good.

  The statistical processing unit 1045 calculates a statistical output for the evaluation result calculated by the processing unit 1030. More specifically, the statistical processing unit 1045 calculates a statistical amount of correlation values (for example, average value, intermediate value, maximum value, minimum value, variance value) included in the determination result list 1035 calculated by the similarity calculation unit 1033. , Standard deviation, etc.) for each model image candidate.

  The histogram generation unit 1046 generates, for each model image candidate, histogram (frequency distribution) data obtained by dividing the feature amount into predetermined sections based on the statistical amount calculated by the statistical processing unit 1045.

  The determination unit 1047 determines the most appropriate model image candidate generated by the model image candidate generation unit 1020 according to the conditions specified by the user. More specifically, the determination unit 1047 accepts a condition that the evaluation result should satisfy, and among the process results generated by performing the process according to each of the plurality of model image candidates, the determination unit 1047 most closely matches the specified condition. A model image candidate is determined. The processing in the determination unit 1047 will be described later.

Next, the output unit 1050 outputs the evaluation result generated by the evaluation unit 1040 for each of the plurality of model image candidates. More specifically, the output unit 1050 provides various output forms such as the table output function, the histogram output function, and the scatter diagram output function described above as an example. These output forms are appropriately switched according to the user operation. The output unit 1050 outputs, as a table output function, the degree of match with the corresponding expected class for the evaluation result calculated by the match / mismatch counter 1043 and the number of processing results that do not match the corresponding expected class. . Further, as this table output function, the output unit 1050 outputs the processing time measured together with the evaluation result.

  Further, the output unit 1050 has, as a table output function, a model image candidate having a relatively high feature amount (correlation value) included in the corresponding processing result group among a plurality of model image candidates as another model image candidate. May be output in different ways. That is, it is preferable that the model image candidate having the highest average correlation value is output conspicuously by red display or blinking display. This means that a high feature quantity (correlation value) means that the features of the measurement image can be successfully extracted by image processing according to the model image candidate, so such model image candidates are selected by the user with priority. This is to encourage them to do this.

<Condition specification>
Next, referring to FIG. 17, an example of an input interface for a condition for the determination unit 1047 (FIG. 15) of the evaluation unit 1040 to select the most appropriate model image candidate will be described.

  After setting the model image candidates on the dialog 420 shown in FIG. 13, the user sets which item has priority in the dialog 499 as shown in FIG. 17. In the dialog 499, a list of items output on the evaluation result screen 400B shown in FIG. 14 is displayed, and the user operates a pull-down menu to set a priority order indicating whether priority should be given to the necessary items. To do. In the example shown in FIG. 17, “number of false detections: low” is set to priority “1”, and “non-defective image correlation value average: high” is set to priority “2”.

  When the conditions as shown in FIG. 17 are set, the determination unit 1047 searches for a model image candidate with the smallest number of erroneous detections with reference to the generated evaluation result for each model image candidate. As a result of this search, if it can be narrowed down to one, the model image candidate is determined as the optimum parameter set. On the other hand, if the number of erroneous detections cannot be reduced to one, the model image candidate having the highest average of the non-defective image correlation values is determined as the optimum model image.

  As a method for setting conditions by the user, various methods other than the method using the user interface shown in FIG. 17 can be adopted.

<Processing procedure>
18 to 22 are flowcharts showing the overall processing in the system according to the embodiment of the present invention. The flowcharts shown in FIGS. 18 to 22 are typically realized by the CPU 105 of the support apparatus 100 reading a program stored in advance in the fixed disk 107 or the like into the memory 106 and executing it. In the flowcharts shown in FIGS. 18 to 22, the evaluation processing application 20 and the image processing application 30 are divided and expressed in order to clarify the execution subject of each processing, but a program in which these applications are integrated is executed. May be. Alternatively, as described above, the support apparatus 100 can use the image processing application 30 </ b> A installed in the image processing apparatus 200. In this case, the system including the support apparatus 100 and the image processing apparatus 200 according to the present embodiment corresponds to the model image acquisition support apparatus according to the present invention. In this case, the CPU 205 of the image processing apparatus 200 also implements the function of the image processing application 30A by reading the program stored in advance in the fixed disk 207 or the like into the memory 206 and executing it.

  18 to 22, first, when an activation is instructed, the evaluation processing application 20 (the CPU 105 of the support apparatus 100 (FIG. 2); the same applies hereinafter) is shown in FIG. 10 after executing the initialization process. The input screen 400A is displayed on the monitor 102 (step S100). Subsequently, the evaluation processing application 20 determines whether or not the button 402 on the input screen 400A has been pressed (step S102). If button 402 has not been pressed (NO in step S102), the process of step S102 is repeated.

  When the button 402 is pressed (YES in step S102), the evaluation processing application 20 accepts the selection of the image processing application 30 for executing the trial (step S104). When the image processing application 30 is selected, the evaluation processing application 20 executes synchronization establishment processing with the image processing application 30 that is the selection destination. Specifically, the evaluation processing application 20 transmits a synchronization establishment request to the image processing application 30 that is the selection destination. In response to the synchronization establishment request, the selected image processing application 30 requires the image processing application 30 (the CPU 105 (FIG. 2) of the support apparatus 100 or the CPU 205 (FIG. 3) of the image processing apparatus 200; the same applies hereinafter). Information is returned to the evaluation processing application 20 (step S106). The synchronization of both is established by the response of the data from the image processing application 30.

  Subsequently, the evaluation processing application 20 determines whether or not the button 404 on the input screen 400A has been pressed (step S108). If button 404 is not pressed (NO in step S108), the process of step S108 is repeated.

  When the button 404 is pressed (YES in step S108), the evaluation processing application 20 displays the dialog 414 shown in FIG. 11 on the monitor 102 (step S110). Subsequently, the evaluation processing application 20 receives the measurement image to be a target and designation of an expectation class associated with the measurement image (step S112). That is, the user designates the “OK” folder in which the measurement image associated with the “non-defective” expected class is stored, and is associated with the “defective” expected class as necessary. The “NG” folder in which the measured image is stored is designated. In the model image acquisition support apparatus according to the present embodiment, a model image can be determined as long as at least a non-defective image exists. Therefore, it is not always necessary to designate a defective image.

  When the measurement image is set, the evaluation processing application 20 generates an image list 1011 (FIG. 16A) in which the target measurement image is associated with the image number to be used internally, and also corresponds to the image number. An expected class list 1012 (FIG. 16B) that associates the expected class for the measurement image to be generated is generated (step S114). Then, the evaluation processing application 20 closes the dialog 414.

  Subsequently, the evaluation processing application 20 determines whether or not the button 406 on the input screen 400A has been pressed (step S116). If button 406 is not pressed (NO in step S116), the process in step S116 is repeated.

When the button 406 is pressed (YES in step S116), the evaluation processing application 20 is a dialog for inputting an item to be subjected to model image acquisition processing and parameters for searching for an optimal model image. 420 (FIG. 12) is displayed on the monitor 102 (step S118). Subsequently, the evaluation processing application 20 accepts designation of a processing item used by the model image to be adjusted, which is input by the user using the pull-down menu 442 (step S120). Furthermore, the evaluation processing application 20 accepts designation of a processing item to be referred to when the user inputs using the pull-down menu 444 and performs processing using the result of other image processing (step S122). Further, the evaluation processing application 20 accepts designation of the adjustment mode input by the user using the pull-down menu 446 (step S124).

  After inputting the target items for the model image acquisition process and the parameters for searching for the optimum model image, the evaluation processing application 20 sets the initial model image and parameter values to the selected image processing application 30. A request is made (step S126). In response to this request, the image processing application 30 responds to the evaluation processing application 20 with the currently set model image (initial setting model image) and the current values of the related parameters (step S128).

  Subsequently, the evaluation processing application 20 determines whether or not the button 409 on the input screen 400A has been pressed (step S130). If button 409 has not been pressed (NO in step S130), the process proceeds to step S136.

  If the button 409 is pressed (YES in step S130), the evaluation processing application 20 displays the dialog 430 shown in FIG. 12 on the monitor 102 (step S132). Subsequently, the evaluation processing application 20 accepts designation of a reference image from which a model image is to be extracted (step S134). That is, the user sets a reference image file from which a model image is extracted by operating the mouse 104 or the like. When this setting is completed, the evaluation processing application 20 closes the dialog 430.

  Subsequently, the evaluation processing application 20 determines whether or not the button 410 on the input screen 400A has been pressed (step S136). If button 410 has not been pressed (NO in step S136), the process of step S130 is repeated.

  When the button 410 is pressed (YES in step S136), the evaluation processing application 20 extracts a plurality of model image candidates from the reference image as shown below. Specifically, the evaluation processing application 20 obtains a command for causing the matching processing based on the initial setting model image to the image processing application 30 after acquiring the list of file names of the designated reference images. (Step S138). At this time, information for designating an area (search range) for extracting the model image from the reference image may be added. Furthermore, the evaluation processing application 20 gives the image processing application 30 designation of the image file name corresponding to the first image number of the reference image (step S140). That is, the evaluation processing application 20 causes the image processing application 30 to search and extract a plurality of model image candidates from the reference image that is a non-defective image based on the initial setting model image.

  In response to the notification from the evaluation processing application 20, the image processing application 30 loads the designated measurement image (step S142). Subsequently, the image processing application 30 extracts a model image candidate by executing a matching process based on the currently set initial setting model image with respect to the loaded measurement image (step S144). Thereafter, the image processing application 30 returns the extracted model image candidate to the evaluation processing application 20 (step S146).

The evaluation processing application 20 stores the model image candidate received from the image processing application 30 in the storage device (memory 106 and / or fixed disk 107) (step S148). Subsequently, the evaluation processing application 20 determines whether or not extraction of model image candidates for all reference images (typically, measurement images that are “good”) is completed (step S150). If extraction of model image candidates for all reference images has not been completed (NO in step S150), the evaluation processing application 20 refers to the image list 1011 and is next to the current image number. The designation of the image file name corresponding to the image number is given to the image processing application 30 (step S152). Thereafter, the processing after step S142 is repeated.

  If the extraction of model image candidates for all reference images has been completed (NO in step S150), the evaluation processing application 20 displays a dialog 430 shown in FIG. 13 on the monitor 102 (step S154). .

  Subsequently, the evaluation processing application 20 determines whether or not the button 422 on the dialog 430 has been pressed (step S158). If the button 422 is pressed (YES in step S158), the evaluation processing application 20 assigns a trial number to the selected model image candidate (step S160). That is, the evaluation processing application 20 associates the selected model image candidate with the trial number among the model image candidates stored in the storage device. Subsequently, the evaluation processing application 20 updates the number display 426 on the dialog 430 in accordance with the assigned trial number (step S162).

  If the button 422 has not been pressed (NO in step S158), or after the processing in step S162, the evaluation processing application 20 determines whether or not the button 424 on the dialog 430 has been pressed (step S164). If button 424 has not been pressed (NO in step S164), the processes in and after step S156 are repeated. On the other hand, when button 424 is pressed (YES in step S164), evaluation processing application 20 closes dialog 430 (step S166).

  Subsequently, the evaluation processing application 20 determines whether or not the button 408 on the input screen 400A has been pressed (step S168). If button 408 has not been pressed (NO in step S168), the process of step S168 is repeated.

  When the button 408 is pressed (YES in step S168), the evaluation processing application 20 starts trials for a plurality of model image candidates as shown below. Specifically, the evaluation processing application 20 loads the model image candidate corresponding to the smallest trial number (trial number [0]) from the storage device (step S170). Subsequently, the evaluation processing application 20 gives the image processing application 30 a command for performing matching processing based on the loaded model candidate image and the model candidate image (step S172). Further, the evaluation processing application 20 refers to the image list 1011 and gives the image processing application 30 designation of the image file name corresponding to the first image number (step S174). That is, the evaluation processing application 20 causes the image processing application 30 to execute matching processing based on model image candidates.

  In response to the notification from the evaluation processing application 20, the image processing application 30 loads the designated measurement image (step S176). Subsequently, the image processing application 30 executes matching processing based on the model image candidate received from the evaluation processing application 20 with respect to the loaded measurement image (step S178). Thereafter, the image processing application 30 returns the similarity obtained by the matching process to the evaluation processing application 20 (step S180).

  The evaluation processing application 20 stores the similarity received from the image processing application 30 in the storage device as the determination result list 1035 in association with the trial number and the image number (step S182). The evaluation processing application 20 compares the received similarity with a preset threshold value, calculates a determination result for the target measurement image, and adds the determination result to the determination result list 1035 (step S184). Furthermore, the evaluation processing application 20 compares the calculated determination result with the expected class associated with the target measurement image, associates the evaluation result with the trial number and the image number, and evaluates the evaluation result list 1042. Is stored in the storage device (step S186).

  Subsequently, the evaluation processing application 20 refers to the image list 1011 and determines whether or not the trial for the current model image candidate for all measurement images has been completed (step S188). If the trials for all measurement images have not been completed (NO in step S188), the evaluation processing application 20 refers to the image list 1011 and corresponds to the image number next to the current image number. The designation of the image file name of the measurement image to be performed is given to the image processing application 30 (step S190). Thereafter, the processing from step S176 onward is repeated.

  When trials for all measurement images have been completed (YES in step S188), the evaluation processing application 20 refers to the determination result list 1035 and the evaluation result list 1042, and calculates the current trial number. A statistic about the similarity and the evaluation result is calculated (step S192).

  Subsequently, the evaluation processing application 20 determines whether or not the trial for all the model image candidates to be tried has been completed (step S194). When the trials for all the model image candidates to be tried have not been completed (NO in step S194), the evaluation processing application 20 selects the model image candidate corresponding to the trial number next to the current trial number. Is loaded from the storage device (step S196). Thereafter, the processing after step S174 is repeated.

  When trials for all model image candidates to be tried have been completed (YES in step S194), the evaluation processing application 20 determines the optimum based on the calculated similarity and / or evaluation result. A model image is determined (step S198). Subsequently, the evaluation processing application 20 displays an evaluation result screen as shown in FIG. 14 on the monitor 102 based on the similarity calculated for each trial number, the evaluation result, and their statistics (step S200). ).

  Subsequently, the evaluation processing application 20 determines whether or not the button 412 on the input screen 400A has been pressed (step S202). When the button 412 is pressed (YES in step S202), the evaluation processing application 20 transfers the selected model image candidate to the image processing application 30A of the connection destination image processing apparatus 200 (step S204). ). The image processing application 30 sets the received model image candidate as a new model image (step S206).

  If the button 412 has not been pressed (NO in step S202), or after the processing in step S206, the evaluation processing application 20 determines whether or not the termination of the evaluation processing application 20 has been instructed (step S208). If termination of evaluation processing application 20 is not instructed (NO in step S208), the processing in step S202 and subsequent steps is repeated.

  If the end of evaluation processing application 20 is instructed (YES in step S208), the process ends.

<Operational effects of this embodiment>
According to the embodiment of the present invention, the model image candidate is extracted from each reference image only by the user specifying the reference image from which the model image is to be extracted, and each extracted model image candidate is used. A list of evaluation contents of the results of the actual matching process is displayed. Therefore, the user can easily select an optimal model image based on the evaluation results displayed in a list.

  That is, the user only specifies a reference image that can be a model image, and the model image acquisition support apparatus automatically performs processing for other than that, so the user sticks to the site and makes adjustments over a long period of time. There is no need. Therefore, even when there are a large number of reference images that may be adopted as model images, an optimal model image can be determined more quickly and easily.

  In addition, since the evaluation results are displayed in a list so that the statistical evaluation results can be compared among the model image candidates, an optimal model image can be objectively determined regardless of the operator's experience. Therefore, an appropriate model image can be determined by an operator who has a certain level of knowledge without inferior to the work performed by a skilled worker.

<Modification 1>
In the flowchart showing the overall process in the support device 100 according to the above-described embodiment, the procedure for sequentially executing the matching process on the target measurement image after setting each model image candidate is illustrated. After loading the measurement image, the matching process may be sequentially executed for each of the model image candidates (Modification 1). This method is effective in environments that require time to load and transfer measurement images, and when evaluating model image candidates in parallel while capturing a new measurement image by capturing a workpiece flowing through the production line. It is.

  The overall process in the system according to the first modification is changed from the overall process procedure (FIGS. 18 to 22) in the system according to the present embodiment to the process shown in the flowchart of FIG. Is equivalent to Therefore, in the following description, the description of the processing procedure similar to that of the present embodiment will not be repeated. Further, in the flowchart shown in FIG. 23, the same step numbers are used for steps including processes similar to the processes of the steps included in the flowchart shown in FIG.

  Referring to FIG. 23, first, the evaluation processing application 20 refers to the image list 1011 and gives the image processing application 30 designation of the image file name corresponding to the first image number (step S174). . Subsequently, the evaluation processing application 20 loads the model image candidate corresponding to the smallest trial number (trial number [0]) from the storage device (step S170). Furthermore, the evaluation processing application 20 gives the image processing application 30 a command for performing matching processing based on the loaded model candidate image and the model candidate image (step S172).

In response to the notification from the evaluation processing application 20, the image processing application 30 loads the designated measurement image (step S176). Subsequently, the image processing application 30 executes matching processing based on the model image candidate received from the evaluation processing application 20 with respect to the loaded measurement image (step S178). Thereafter, the image processing application 30 returns the similarity obtained by the matching process to the evaluation processing application 20 (step S180).

  The evaluation processing application 20 stores the similarity received from the image processing application 30 in the storage device as the determination result list 1035 in association with the trial number and the image number (step S182). The evaluation processing application 20 compares the received similarity with a preset threshold value, calculates a determination result for the target measurement image, and adds the determination result to the determination result list 1035 (step S184). Furthermore, the evaluation processing application 20 compares the calculated determination result with the expected class associated with the target measurement image, associates the evaluation result with the trial number and the image number, and evaluates the evaluation result list 1042. Is stored in the storage device (step S186).

  Subsequently, the evaluation processing application 20 determines whether or not the trial for all the model image candidates to be tried has been completed (step S194). When the trials for all the model image candidates to be tried have not been completed (NO in step S194), the evaluation processing application 20 selects the model image candidate corresponding to the trial number next to the current trial number. Is loaded from the storage device (step S196). Thereafter, the processing after step S172 is repeated.

  On the other hand, when trials for all model image candidates to be tried have been completed (YES in step S194), the evaluation processing application 20 refers to the image list 1011 and performs the measurement for all measurement images. It is determined whether or not the trial is completed (step S188A). If trials for all measurement images have not been completed (NO in step S188A), the evaluation processing application 20 refers to the image list 1011 and corresponds to the image number next to the current image number. The designation of the image file name of the measurement image to be performed is given to the image processing application 30 (step S190). Thereafter, the processing after step S170 is repeated.

  When trials for all measurement images have been completed (YES in step S188A), the evaluation processing application 20 refers to the determination result list 1035 and the evaluation result list 1042, and calculates for each trial number. Statistics about the similarity and the evaluation result are calculated (step S192A). Thereafter, the process proceeds to step S198.

<Modification 2>
Depending on the number of erroneous detections in the above-described matching process, a process of aborting trials for model image candidates that are considered inappropriate may be added (Modification 2). The overall process in the system according to the second modification is changed from the overall process procedure (FIGS. 18 to 22) in the system according to the present embodiment to the process shown in the flowchart in FIG. Is equivalent to Therefore, in the following description, the description of the processing procedure similar to that of the present embodiment will not be repeated. The flowchart shown in FIG. 24 corresponds to the flowchart shown in FIG. 21 with steps S230, S232, and S234 added.

  After execution of step S186, the evaluation processing application 20 counts the number of false detections when the calculated determination result does not match the expected class associated with the target measurement image (step S230). Subsequently, the evaluation processing application 20 determines whether or not the number of erroneous detections counted in step S230 exceeds a specified allowable upper limit value (step S232).

When the counted number of erroneous detections exceeds the specified allowable upper limit value (step S232)
In the case of YES), the evaluation processing application 20 transmits to the image processing application 30 a command for aborting trials for the remaining measurement images for the current model image candidate (step S234). Then, the process proceeds to step S196.

<Modification 3>
In the above-described embodiment, the configuration for performing the matching process mainly based on one model image has been illustrated. By the way, when the acquired measurement image is not stable (unstable) due to various disturbances, a plurality of model images are prepared in advance, and matching processing is performed by performing matching processing based on these model images. Can be further stabilized. More specifically, matching processing based on a plurality of model images is performed on the measurement image, and if matching processing based on at least one model image is successful, the entire measurement image on the target Assuming that matching is OK, an application is assumed. Such matching processing is called flexible search or the like.

  FIG. 25 is a diagram for describing the outline of the matching processing according to the third modification of the embodiment of the present invention. FIG. 25A shows the case of matching OK as a whole, and FIG. 25B shows the case of matching NG as a whole.

  Referring to FIG. 25A, matching is performed between a plurality of model images MDL1 to MDL5 prepared in advance (hereinafter, these model images are also referred to as “(optimum) model image set”) and measurement images. Each process is performed. That is, for each of the model images MDL1 to MDL5, the similarity (correlation value) with the measurement image is calculated. Further, matching OK or matching NG is determined by comparing the calculated similarity (correlation value) with a preset threshold value. When it is determined that the matching is OK for at least one model image (two model images MDL2 and MDL4 in the example shown in FIG. 25A), the matching processing for the entire measurement image is “OK”. Is determined.

  On the other hand, as shown in FIG. 25B, when all the model images MDL1 to MDL5 are determined to be matching NG, the matching processing for the entire measurement image is determined to be “NG”.

  For such an application that performs a matching process using a plurality of model images, what kind of combination of model images is set is important. Therefore, in Modification 3 of the present embodiment, a configuration that can determine an optimal model image set more quickly and easily will be described.

  FIG. 26 is a diagram for describing an overview of the model image set acquisition process in the matching process according to the third modification of the embodiment of the present invention.

  Referring to FIG. 26, the support apparatus according to the present modification determines a plurality of model image candidates as an “optimum model image set” from among model image candidates extracted from a plurality of reference images. By using such an optimal model image set, matching processing can be performed stably even if the acquired measurement image is affected by various disturbances. That is, the support device according to the present modification searches for a combination of two or more model image candidates included in the plurality of model image candidates for optimizing the matching processing result for the entire plurality of measurement images. To do. In the following, three typical techniques will be exemplified.

(Method 1)
Method 1 is to select model image set candidates from the model image candidates shown in FIG. 26 as trial targets, and select the one that can perform the most stable matching process from the created trial targets.

  FIG. 27 is a diagram for describing an overview of Method 1 according to the third modification of the embodiment of the present invention. As shown in FIG. 27, in Method 1 of the present modification, combinations of model image candidates configured by a predetermined number are determined brute force, and each combination is set as a trial target. That is, in the example shown in FIG. 27, combinations of three model image candidates are sequentially generated, and a trial number is assigned to each combination. And the trial similar to the above is repeatedly performed using the model image candidate which assigned each trial number. Then, the execution result is displayed to the user. Since the user interface for providing the trial result is the same as that in FIG. 14 described above, detailed description will not be repeated.

  In the example shown in FIG. 27, the configuration for creating a predetermined number of model image candidate combinations has been illustrated, but the number of model image candidates used for the combination may also vary.

The procedure of Method 1 as described above can be outlined in the flowchart shown in FIG.
FIG. 28 is a flowchart showing a procedure of method 1 according to the third modification of the embodiment of the present invention. The flowchart shown in FIG. 28 is typically realized by the CPU 105 of the support apparatus 100 reading a program stored in advance in the fixed disk 107 or the like into the memory 106 and executing it.

  The evaluation processing application 20 (the CPU 105 of the support apparatus 100 (FIG. 2); the same applies hereinafter) accepts designation of a reference image from which a model image is to be extracted (step S400). Subsequently, the evaluation processing application 20 extracts a plurality of model image candidates from the designated reference image (step S402). The processing content of step S402 is the same as the processing content of steps S138 to S152 of FIG. 19 described above.

  Thereafter, the evaluation processing application 20 determines a brute force combination composed of a predetermined number from the extracted model image candidates (step S404). Furthermore, the evaluation processing application 20 assigns a trial number to each determined combination (step S406).

  Furthermore, the evaluation processing application 20 executes a trial for the model image candidate set corresponding to each trial number (step S408). The processing content of step S408 is the same as the processing content of steps S170 to S196 of FIG. 21 described above.

  After the trial is completed, the evaluation processing application 20 displays an evaluation result screen indicating the similarity calculated for each trial number, the evaluation result, and their statistics on the monitor 102 (step S410). Furthermore, the evaluation processing application 20 determines whether or not the user has selected any model image candidate set (step S412).

  When the user selects any model image candidate set (YES in step S412), the evaluation processing application 20 uses the selected model image candidate set as the image processing application 30A of the connection destination image processing apparatus 200. (Step S414). The image processing application 30 registers the received model image candidate as a model image set for flexible search.

According to Method 1 according to the present modification, the combination of model image candidates can be evaluated brute force, so that the optimal model image candidate set can be determined reliably.

(Method 2)
In Method 2, first, a model image that can be most stably matched among the model image candidates shown in FIG. 26 is determined, and subsequently, a measurement image that is a matching NG with respect to the determined model image is extracted. Thus, another model image that is matched with the extracted measurement image is determined in a complementary manner. The procedure of Method 2 will be described with reference to FIG.

  FIG. 29 is a flowchart showing a procedure of technique 2 according to the third modification of the embodiment of the present invention. The flowchart shown in FIG. 29 is typically realized by the CPU 105 of the support apparatus 100 reading out a program stored in advance in the fixed disk 107 or the like to the memory 106 and executing it.

  The evaluation processing application 20 (the CPU 105 (FIG. 2) of the support apparatus 100; the same applies hereinafter) accepts designation of a reference image from which a model image is to be extracted (step S450). Subsequently, the evaluation processing application 20 extracts a plurality of model image candidates from the designated reference image (step S452). The processing content of step S452 is the same as the processing content of steps S138 to S152 of FIG. 19 described above.

  Thereafter, the evaluation processing application 20 assigns a trial number to each of the extracted plurality of model image candidates (step S454). Then, the evaluation processing application 20 executes a trial for the model image candidate corresponding to each trial number (step S456). The processing content of step S456 is the same as the processing content of steps S170 to S196 of FIG. 21 described above. After completion of the trial, the evaluation processing application 20 determines an optimal model image (first model image) among the target model image candidates based on the calculated similarity and / or evaluation result (step S458). . Subsequently, the evaluation processing application 20 extracts a measurement image that becomes a matching NG with respect to the first model image determined in step S458 (step S460).

  Further, the evaluation processing application 20 performs a trial on the measurement image extracted in step S460 with respect to other model image candidates excluding the first model image determined in step S458 among the extracted model image candidates. Is executed (step S462). The processing content of step S462 is the same as the processing content of steps S170 to S196 of FIG. 21 described above. After completion of the trial, the evaluation processing application 20 determines an optimal model image (second model image) among the target model image candidates based on the calculated similarity and / or evaluation result (step S464). .

  Subsequently, the evaluation processing application 20 determines whether or not there is a measurement image that is a matching NG with respect to the second model image determined in step S464 (step S466). If there is no measurement image matching NG with respect to the second model image (NO in step S466), the evaluation processing application 20 has determined in step S464 and the first model image determined in step S458. The second model image is determined as a model image candidate set (step S468).

  On the other hand, when there is a measurement image that matches NG with respect to the second model image (YES in step S466), evaluation processing application 20 repeats the processing in step S460 and subsequent steps. In other words, the evaluation processing application 20 adds the second model image until there is no measurement image that is matched NG.

According to the method 2 according to the present modification, a model image candidate set composed of a minimum number of model images can be determined, so that more efficient matching processing can be performed.

(Method 3)
In Method 3, trials are executed for each of the model image candidates shown in FIG. 26, and an optimal model image candidate combination is determined based on the trial results.

  FIG. 30 is a diagram for describing an overview of method 3 according to the third modification of the embodiment of the present invention. FIG. 31 is a diagram showing an example of a trial result in the technique 3 according to the third modification of the embodiment of the present invention.

  As shown in FIG. 30A, in the method 3 of the present modification, first, a trial for a measurement image is executed for each of a plurality of extracted model image candidates. Then, as shown in FIG. 31, the similarity (correlation value) for each target measurement image is calculated for each model image candidate. Based on the trial results as shown in FIG. 31, a combination is determined such that the similarity (correlation value) for all measurement images can maintain a predetermined value. As a typical determination processing method, model image candidates having the highest similarity (correlation value) among the respective measurement images are identified (corresponding to the hatched portion shown in FIG. 31), and these identified models are also identified. A set of image candidates is determined as a model image candidate set. That is, in the example shown in FIG. 31, the model image candidate 1 has the highest similarity (correlation value) for the measurement images 1, 2, and 5, and the model image candidate 2 has the highest similarity for the measurement images 3 and 4. (Correlation value) is high. As a result, as shown in FIG. 30B, model image candidate 1 and model image candidate 2 are determined as a model image candidate set.

The procedure of Method 3 as described above can be outlined in the flowchart shown in FIG.
FIG. 32 is a flowchart showing a procedure of technique 3 according to the third modification of the embodiment of the present invention. The flowchart shown in FIG. 32 is typically realized by the CPU 105 of the support apparatus 100 reading a program stored in advance in the fixed disk 107 or the like into the memory 106 and executing it.

  The evaluation processing application 20 (the CPU 105 of the support apparatus 100 (FIG. 2); the same applies hereinafter) receives the designation of the reference image from which the model image is to be extracted (step S500). Subsequently, the evaluation processing application 20 extracts a plurality of model image candidates from the designated reference image (step S502). The processing content of step S502 is the same as the processing content of steps S138 to S152 of FIG. 19 described above.

  Thereafter, the evaluation processing application 20 assigns a trial number to each of the extracted plurality of model image candidates (step S504). Then, the evaluation processing application 20 executes a trial for the model image candidate corresponding to each trial number (step S506). The processing content of step S506 is the same as the processing content of steps S170 to S196 of FIG. After completion of the trial, the evaluation processing application 20 temporarily stores the similarity (correlation value) for each target measurement image for each model image candidate (step S508). That is, a table as shown in FIG. 31 is generated internally. Subsequently, the evaluation processing application 20 determines an optimal combination of model image candidates based on the table generated in step S508 (step S510). This combination is determined in consideration of factors such as the number of model image candidates and the number of measurement images that have become matching NG. Further, linear programming or the like can be adopted as a combination determination algorithm. In step S510, a combination of a plurality of model image candidates may be determined.

  Thereafter, the evaluation processing application 20 displays, on the monitor 102, an evaluation result screen showing the similarity calculated for each model image candidate, the evaluation result, and their statistics, together with the combination of the model image candidates determined in step S510. It is displayed (step S512). Furthermore, the evaluation processing application 20 determines whether or not the user has selected any model image candidate set (step S514).

  If the user selects any model image candidate set (YES in step S514), the evaluation processing application 20 uses the selected model image candidate set as the image processing application 30A of the connection destination image processing apparatus 200. (Step S516). The image processing application 30 registers the received model image candidate as a model image set for flexible search.

  According to the method 3 according to the present modification, the combination can be determined based on the similarity (correlation value) for each of the model image candidates, so that the optimal model image candidate set can be determined more efficiently.

<Modification 4>
In the above-described embodiment, the configuration in which a plurality of model image candidates can be extracted from the same reference image has been illustrated. On the other hand, depending on the target processing item, it may be preferable to use different image processing parameters associated with the same model image as model image candidates. Hereinafter, such an example will be illustrated.

  When performing the search process based on the similarity (correlation value) with the model image as described above, since it is less necessary to optimize the parameters associated with the model image, a plurality of model image candidates are prepared in advance. It is sufficient to determine the most suitable of these. However, depending on the processing item, there is a close relationship with the parameters set for the model image.

  For example, in an edge code-based search process called edge code model search, it is necessary to optimize the edge mask level and edge level when registering the model. In other words, it is necessary to adjust the corresponding parameter for each model to be registered. Furthermore, if there are many edges registered in the model, the amount of matching increases, which greatly affects the measurement time. Therefore, it is necessary to adjust the parameters in consideration of the tact time and the like.

  In such a processing item, since the disturbance caused by the parameters is large, even if only a plurality of model image candidates are changed and evaluated, parameters such as the edge mask level and the edge level are not changed in conjunction with each other. There are cases where it is not possible to acquire settings that meet the conditions required by the user.

  Therefore, in such a case, a trial is performed for a combination of a plurality of model image candidates and a plurality of parameters.

FIG. 33 shows an example of trial creation processing according to the fourth modification of the present invention.
As shown in FIG. 33 (a), there are a plurality of model image candidates and a plurality of parameter candidates as variation factors for performing optimization. Therefore, as shown in FIG. 33B, combinations of model image candidates and parameter candidates are sequentially performed as trials 1-1, 1-2, 1-3, 1-4, 2-1, 2-2. , 2-3, 2-4,..., And by executing each of the set trials, an optimal combination of the model image and the corresponding parameter can be extracted.

  According to this modification, the corresponding parameters can be optimized together with the model image, so that even a more complicated processing item can be set appropriately in a short time.

<Modification 5>
In the above-described embodiment, the configuration in which the image capturing unit 8 executes the matching process on the measurement image itself obtained by capturing the workpiece or the like has been mainly illustrated. On the other hand, depending on the target processing item, preprocessing such as a filter (for example, a color gray filter) may be applied to the measurement image acquired by the imaging unit 8. In this case, it is necessary to register a model image corresponding to the preprocessing applied to the measurement image. Hereinafter, such an example will be illustrated.

  FIG. 34 is a block diagram for illustrating the contents of process items according to the fifth modification of the present invention. FIG. 35 is a diagram for explaining optimization of a model image according to the fifth modification of the embodiment of the present invention.

  Referring to FIG. 34A, assume that model image extraction unit 520 extracts a model image from a reference image according to a predetermined condition and registers it in advance. On the other hand, the measurement image acquired by the imaging unit is output to the similarity calculation unit 500 after the filter processing desired by the user is performed in the preprocessing unit 510. The similarity calculation unit 500 calculates (searches) a similarity (correlation value) with a model image registered in advance for the measurement image input from the preprocessing unit 510. Here, if a setting different from the setting used when registering the model image is given to the pre-processing unit 510, the registered model image is pre-processed as shown in FIG. The later measurement image may not be appropriate. That is, as shown in FIG. 35A, when the model image is not optimized in accordance with the change in the preprocessing conditions, the color density of the two images is different, so that a large shift may occur in the matching processing result.

  Therefore, as shown in FIG. 34B, the reference image is input to the preprocessing unit 522 that performs the same filtering process as the preprocessing unit 510, and the filtered image output from the preprocessing unit 522 is input. A model image is registered based on the reference image. As described above, when the model image is optimized in accordance with the change in the preprocessing conditions, an appropriate matching process is executed as shown in FIG.

  According to the present modification, the model image is registered in accordance with the preprocessing for the measurement image that is the target of the matching processing, so that appropriate matching processing can be executed without being affected by the preprocessing.

<Modification 6>
In the above-described embodiment, in order to speed up the process of searching for an optimal model image, when the measurement result for a certain trial satisfies a predetermined condition, the configuration for aborting the trial measurement in the middle is illustrated. However, other speed-up methods may be applied.

  For example, in the above-described embodiment, the process of extracting model image candidates from all the target non-defective images by performing the matching process with the initial setting model image is exemplified (for example, see FIG. 5). Further, by adding further conditions, only a part may be extracted as model image candidates.

More specifically, a predetermined number can be used as model image candidates from those having a relatively high similarity (correlation value) with the initially set model image. Further, the model image candidates may be extracted every predetermined number in descending order of similarity (correlation value) with the initially set model image. Alternatively, a model image candidate having a similarity (correlation value) with the initial setting model image within a predetermined range (typically in the vicinity of the average value) may be used.

  Furthermore, since the average image extracted by the model image candidate is often influenced by the initial setting model image, after extracting the model image candidate based on the initial setting model image, the average image among them is the most average A model image may be specified, and model image candidates may be extracted again based on the specified model image.

According to this modification, an optimal model image can be determined at higher speed.
[Other embodiments]
The program according to the present invention may be a program module that is provided as a part of a computer operating system (OS) and that calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. . In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. Note that the program product includes the program itself and a recording medium in which the program is stored.

  Furthermore, part or all of the functions realized by the program according to the present invention may be configured by dedicated hardware.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

2 work, 4 photoelectric sensor, 6 transport line, 8 imaging unit, 10 measurement image acquisition application, 20 evaluation processing application, 30, 30A, 30B image processing application, 40 model image candidate, 100 model image acquisition support device (support device) , 102, 202 monitor, 103, 203 keyboard, 104, 204 mouse, 106, 206 memory, 107, 207 fixed disk, 111, 211 FD drive, 112, 212 flexible disk, 113, 213 CD-ROM drive, 200 Image processing apparatus 201 main body unit 209 camera interface unit 215 control information interface unit 217 sensor interface unit 1010 input unit 1011 image list 1012 expected class list 1020 model Image candidate generation unit, 1021 input interface unit, 1022 model image extraction unit, 1023 communication unit, 1030 processing unit, 1031 measurement image selection unit, 1032 model image candidate selection unit, 1033 similarity calculation unit, 1034 determination unit, 1035 determination result List, 1036 processing controller, 1040 evaluation unit, 1041 comparison unit, 1042 evaluation result list, 1043
Match / mismatch counter, 1044 abort processing unit, 1045 statistical processing unit, 1046
Histogram generation unit, 1047 determination unit, 1050 output unit.

Claims (6)

An apparatus for supporting acquisition of a model image, directed to an image processing apparatus that performs matching processing based on a preset model image and parameters for a measurement image obtained by imaging a subject,
Extraction means for extracting a plurality of model image candidates that are candidates for the model image from a reference image obtained by imaging a subject that can be a model;
Setting means for setting a plurality of parameter candidates that correspond to each of the plurality of model image candidates and that are candidates for the parameters;
An input means for receiving a measurement image;
Processing means for generating a plurality of trial results by performing the matching processing based on each of the plurality of model image candidates and the plurality of parameter candidates for the measurement image,
A model image acquisition support device that extracts an optimal combination of the model image and the corresponding parameter based on the plurality of trial results. 2. The processing unit generates the plurality of trial results by adjusting each of the plurality of model image candidates with each of the plurality of parameter candidates and performing the matching process with the measurement image. Model image acquisition support device. The matching process includes a search process based on an edge code;
The parameters include parameters related to edge, the model image acquisition support apparatus according to claim 1 or 2. The input means receives a plurality of the measurement images,
The plurality of measurement images include a non-defective image that should be matched with the model image, and a defective product image that should not be matched with the model image,
The model image acquisition support apparatus further includes processing means for generating a plurality of trial results by performing the matching processing based on each of the plurality of model image candidates for the measurement image, and the processing means An index including a number of non-defective images that do not match the model image among the plurality of non-defective images and a number of defective images that match the model image among the plurality of defective images is predetermined. The model image acquisition support apparatus according to any one of claims 1 to 3, wherein generation of the plurality of trial results is terminated when a condition is reached. A method for supporting directed to an image processing apparatus for performing based Kuma etching process to preset the model image and the parameters for the measurement image obtained by imaging an object, the acquisition of the model image,
Extracting a plurality of model image candidates that are candidates for the model image from a reference image obtained by imaging a subject that can be a model;
Setting a plurality of parameter candidates corresponding to each of the plurality of model image candidates and serving as the parameter candidates;
Receiving a measurement image;
Generating a plurality of trial results by performing the matching process based on each of the plurality of model image candidates and the plurality of parameter candidates for the measurement image,
A model image acquisition support method for extracting an optimal combination of the model image and the corresponding parameter based on the plurality of trial results. A program for supporting acquisition of a model image directed to an image processing apparatus that performs a matching process based on a model image and parameters set in advance for a measurement image obtained by imaging a subject, To the processor,
Extracting a plurality of model image candidates that are candidates for the model image from a reference image obtained by imaging a subject that can be a model;
Setting a plurality of parameter candidates corresponding to each of the plurality of model image candidates and serving as the parameter candidates;
Receiving a measurement image;
Generating a plurality of trial results by performing the matching process based on each of the plurality of model image candidates and the plurality of parameter candidates, with respect to the measurement image,
A model image acquisition support program for extracting an optimal combination of the model image and the corresponding parameter based on the plurality of trial results.