JP2023077051A - Setting device and setting method - Google Patents

Abstract

To obtain a machine learning network which reduces user's labor at learning time of a machine learning network, excludes dependency on individual skills, and has high detection performance.SOLUTION: A setting device comprises: a display control unit which displays an image to be learned by a machine learning network on a display unit; an input unit which receives defective information inputted by either formula of a first annotation formula which assigns a label indicating that it is a non-defective article image or a defective article image, and a second annotation formula which designates a defective part when the image displayed on the display unit is the defective article image, with respect to the image displayed on the display unit; and a processor which makes the machine learning network learn both of the non-defective article image or the defective article image to which the label was assigned by the first annotation formula, and the defective article image having the defective information inputted by the second annotation formula.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a setting device and a setting method for a visual inspection apparatus.

For example, Patent Literature 1 discloses a processing device that uses computer-based machine learning to determine whether a work is good or bad. The processing device of Patent Document 1 performs supervised machine learning on non-defective product data to generate a non-defective product learning model, and performs supervised machine learning on defective product data to generate a defective product learning model. After generation, the data of the work to be judged is input, and it is configured to be able to judge whether the work is a good product or a defective product by the non-defective product learning model and the defective product learning model. is also called a work visual inspection device.

JP 2019-204321 A

By the way, when supervised learning is performed on defective product data, it is necessary for the user to clearly indicate which part of the work is defective, that is, so-called annotation. Since the judgment performance of the defective product learning model generated by defective product learning tends to improve as the annotation is performed in detail, it is desirable to perform the annotation in more detail.

However, the defective part is not always a simple shape, and often has various shapes and is located at various positions on the workpiece. The task of specifying is difficult. If the designation of the area corresponding to the defective portion is inaccurate, this results in deterioration of the determination performance of the defective product learning model. In addition, as long as humans are annotating, there are cases where it is difficult to judge whether an area should be annotated as a defective part on the defective product image, or an area larger than the area corresponding to the defective part is annotated. In some cases, on the contrary, it is conceivable that an area smaller than the area corresponding to the defective portion is annotated. Such differences in annotations by users can result in differences in the performance of the finally generated defective product learning model. In other words, it is ideal to make the annotations more detailed, but the more the annotations are made, the more dependent the annotation becomes, and as a result, there is a risk that a defective product learning model with good judgment performance cannot be obtained.

In addition, in order to generate a defective product learning model, for example, dozens or hundreds of defective product images are required. This was also a problem.

The present disclosure has been made in view of this point, and its purpose is to reduce the user's time and effort when learning a machine learning network, and to eliminate the dependence on individuals, while having a machine with high detection performance. The object is to obtain a learning network.

In order to achieve the above object, in one aspect of the present disclosure, a machine learning network generated by learning a non-defective product image corresponding to a non-defective product and a defective product image corresponding to a defective product is provided with a work to be inspected. It is possible to presuppose a setting device for inputting a photographed work image and setting a visual inspection device for determining the quality of the work image. The setting device includes a display control unit that causes a display unit to display an image for learning by the machine learning network, and a label indicating that the image displayed on the display unit is a non-defective product image or a defective product image. an input unit that receives defect information input by either a first annotation method that specifies a defective part when the image displayed on the display unit is a defective product image; with a processor. The processor causes the machine learning network to learn both the non-defective product image or the defective product image labeled by the first annotation method and the defective product image having the defect information input by the second annotation method. , the parameters of the machine learning network can be adjusted.

According to this configuration, in the first annotation method, images displayed on the display unit can be sorted into non-defective product images and defective product images by label designation. Annotation becomes possible. On the other hand, in the second annotation method, since the defective part can be specified on the defective product image, it is possible to make an annotation that specifies the defective part in more detail than in the first annotation method. In other words, the defective product image includes a defective product image that can be determined as defective simply by attaching a label, and a defective product image that cannot be determined as defective unless the defective part is clearly specified. In the former case, the annotation can be easily performed without the user's trouble, while in the latter case, the user can annotate in detail.

The input unit according to another aspect includes, as the second annotation method, a region specification method of specifying the defective portion by enclosing the defective portion of the defective product image, and a defective portion specifying method of tracing the defective portion of the defective product image. can be selected from a precise designation method that designates in a free form.

According to this configuration, when the region designation method is selected, the defective portion can be easily designated by generating a frame surrounding the defective portion using, for example, a pointing device. Further, when the defective portion is, for example, a free-form, the precise designation method can be selected to precisely designate the free-form defective portion. In other words, since it is possible to switch between rough designation and precise designation of defective parts as needed, it is possible to effectively learn a machine learning network using defective product images while reducing the user's labor. can.

An input unit according to another aspect can execute an automatic extraction method for automatically extracting a defective portion within a designated area by enclosing a defective portion of a defective product image and the periphery of the defective portion.

According to this configuration, the user can automatically extract the defective portion simply by encircling the defective portion and its surroundings, thereby further reducing the user's trouble.

Since the input unit according to another aspect can accept correction of the shape of the region automatically extracted by the automatic extraction method after the execution of the automatic extraction method, for example, the automatically extracted region can change the shape of the defective part The user can correct the shape of the automatically extracted area so that it matches the shape of the defective area, thereby enabling more precise annotation.

A processor according to another aspect performs a single machine learning process on a non-defective product image or a defective product image labeled by the first annotation method and a defective product image on which a defective portion is designated by the second annotation method. Inputs to the network and parameters of the machine learning network can be adjusted so that a single machine learning network can be trained on multiple defective images annotated in different ways.

A processor according to another aspect can perform a verification process of inputting verification image data to the machine learning network generated by learning defective product images and determining whether the verification image data is good or bad. After the verification process, if the verification image data corresponding to a defective product is not determined to be defective, the processor outputs the verification image data in which a defective portion is designated by the second annotation method. An update process can be performed for causing the machine learning network to learn and updating the parameters of the machine learning network.

In other words, there is room for further learning for a machine learning network that does not determine verification image data corresponding to defective products as defective. By allowing such a machine learning network to learn the verification image data in which the defective portion is precisely specified by the second annotation method, the judgment performance of the machine learning network can be further improved.

In another aspect, the display control unit can cause the display unit to display a list of images that are not determined to be defective among images labeled as defective product images by the first annotation method. . The input unit accepts a selection input of an image for which a defective portion is to be designated by the second annotation method from among the images displayed as a list on the display unit, and determines whether the selected image is defective by the second annotation method. Accept designation of location.

In other words, the fact that there are images that are not determined as defective among the images labeled as defective images means that there is room for further learning in the machine learning network. In such a case, it is conceivable to let the machine learning network learn images that have not been judged as defective, but if the machine learning network is allowed to learn them as they are, the learning effect will hardly improve. In this configuration, an image that is not determined to be defective despite being labeled as a defective product image can be presented to the user by displaying it on the display unit. An image suitable for precisely specifying the defective portion can be selected from among them by the second annotation method, and the machine learning network can be trained with the image in which the defective portion is precisely specified, thereby improving the learning effect.

In another aspect, the processor inputs a plurality of verification image data different from each other to a machine learning network to determine whether each verification image data is good or bad; The determination result can be displayed on the display unit and presented to the user.

According to this configuration, the processor can acquire the determination result by causing the machine learning network to perform the pass/fail determination of the plurality of verification image data. Since this determination result is displayed on the display unit, the user can grasp whether the images designated as non-defective products and defective products are separated. If the determination result that the image is determined to be a non-defective product can be separated from the determination result that the image is determined to be a defective product, the user can determine that the machine learning network has sufficiently learned. If it can be separated, the user can judge that the learning of the machine learning network is insufficient and further learning is necessary.

In another aspect, a cumulative histogram based on the number of times the image is determined to be a non-defective image and the number of times the image is determined to be a defective image is generated and displayed on the display unit.

According to this configuration, as a result of pass/fail judgment of the verification image data by the machine learning network, the user is presented with a histogram based on the number of times the image data for verification is judged to be a non-defective image and the number of times the image is judged to be a defective image. be able to. This allows the user to easily determine whether or not the images designated as non-defective products and defective products are separated.

A processor according to another aspect is configured to be capable of executing a process of increasing a distance between an abnormality degree map of an image having non-defective product information and an abnormality degree map of an image having defective product information by a predetermined distance or more.

According to this configuration, the learning effect using the image having non-defective product information and the image having defective product information is further improved.

A processor according to another aspect can execute at least one of a process of reducing the distance between the abnormality maps of images having non-defective product information and the process of reducing the distance between the abnormality maps of images having defective product information. is configured to

According to this configuration, the learning effect using a plurality of images having non-defective product information and the learning effect using a plurality of images having defective product information are further improved.

A processor according to another aspect, the machine so that each pixel value in the degree of anomaly map generated by inputting a non-defective product image having non-defective product information input by the first annotation method into the machine learning network is 0 Parameters of the learning network can be adjusted.

A processor according to another aspect specifies, in an anomaly map generated by inputting a defective product image having defective product information input by the second annotation method into the machine learning network, by the second annotation method The parameters of the machine learning network can be adjusted so that the pixel value of the portion corresponding to the non-defective portion is 0.

As described above, defect information is obtained by either the first annotation method, which assigns a label indicating whether the image is a non-defective product image or the defective product image, or the second annotation method, which designates the defective part of the defective product image. It is possible to input and adjust the parameters of the machine learning network based on the defect information, reducing the user's labor during learning, eliminating the dependence on individuality, and having high detection performance. can be obtained.

1 is a schematic diagram showing the configuration of a setting device for an appearance inspection apparatus according to an embodiment of the present invention; FIG. 3 is a block diagram showing the hardware configuration of the setting device; FIG. 10 is a flow chart showing an example of an annotation procedure; FIG. 4A shows a case where a label is given to a defective product image by the first annotation method, FIG. 4B shows a case where a label is added to a non-defective product image by the first annotation method. It is a figure which shows the example which designated the defective part of the defective product image by the 2nd annotation method. FIG. 6A shows an example of enclosing a defective portion with a circular frame passing through three points, and FIG. 6B shows an example of enclosing a defective portion with a circular frame generated by designating a center point; FIG. 6C shows an example of enclosing a defective portion with a polygon, and FIG. 6D is a diagram showing an example of enclosing a defective portion with a frame generated by a freehand tool. It is a figure explaining the detail of a 2nd annotation method. FIG. 10 is a diagram illustrating a case of designating a defective portion according to an example using a first annotation method and a second annotation method; FIG. 10 is a diagram illustrating another example of designating a defective portion by a first annotation method and a second annotation method; Fig. 10 is a flow chart showing a first example of a parameter adjustment procedure for a machine learning network; FIG. 10 is a diagram showing an example of a user interface screen displaying the results of learning parameter analysis; FIG. 10 is a diagram showing an example of a user interface screen displaying images of defective products that have not been determined as defective; FIG. 10 is a diagram showing an example of a user interface screen for executing the second annotation method; FIG. 10 is a diagram showing an example of a user interface screen after updating parameters; FIG. 10 is a flow chart showing a second example of a parameter adjustment procedure for a machine learning network; FIG. FIG. 10 is a diagram showing an example of an output image output from a neural network for anomaly extraction; It is explanatory drawing of a learning method. It is explanatory drawing of a learning method.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

FIG. 1 is a schematic diagram showing the configuration of a setting device (hereinafter simply referred to as a setting device) 1 for an appearance inspection apparatus according to an embodiment of the present invention. The setting device 1 is a device that enables a user to make various settings (details of which will be described later) of the visual inspection apparatus. A visual inspection device is a device for judging the quality of a work image obtained by imaging a work to be inspected, such as various parts and products, and can be used at a production site such as a factory. can. Specifically, a machine learning network is built inside the visual inspection device, and this machine learning network is generated by learning a non-defective product image corresponding to a non-defective product and a defective product image corresponding to a defective product. It is A work image obtained by imaging a work to be inspected is input to the generated machine learning network, and the machine learning network can judge whether the work image is good or bad.

The entire workpiece may be inspected, or only part of the workpiece may be inspected. Also, one workpiece may include a plurality of inspection targets. Also, the workpiece image may include a plurality of workpieces.

The setting device 1 includes a control unit 2 serving as a device main body, an imaging unit 3 , a display device (display section) 4 and a personal computer 5 . The personal computer 5 is not essential and can be omitted. The personal computer 5 can be used instead of the display device 4 to display various information and images, and the functions of the personal computer 5 can be incorporated into the control unit 2 or into the display device 4 .

In FIG. 1, a control unit 2, an imaging unit 3, a display device 4, and a personal computer 5 are shown as an example of the configuration of the setting device 1, but any plurality of these may be combined and integrated. can also For example, the control unit 2 and the imaging unit 3 can be integrated, or the control unit 2 and the display device 4 can be integrated. Also, the control unit 2 can be divided into a plurality of units and some of them can be incorporated into the imaging unit 3 and the display device 4, or the imaging unit 3 can be divided into a plurality of units and some of them can be incorporated into other units. .

Since the hardware of the setting device 1 and the hardware of the visual inspection device have many parts in common, the setting device 1 can be set to have the visual inspection function by incorporating a program for realizing each function of the visual inspection device. The apparatus 1 may be used, or a program for realizing each function of the setting apparatus 1 may be installed in the appearance inspection apparatus to provide an appearance inspection apparatus having a setting function, which will be described later. Moreover, the setting device 1 and the appearance inspection device may be devices separated from each other. In this case, the setting device 1 and the appearance inspection device can be used by connecting them so as to be able to communicate with each other.

(Configuration of imaging unit 3)
As shown in FIG. 2, the imaging unit 3 includes a camera module (imaging section) 14 and an illumination module (illumination section) 15, and is a unit that acquires a workpiece image. The camera module 14 includes an AF motor 141 that drives an imaging optical system, and an imaging board 142 . The AF motor 141 is a part that automatically performs focus adjustment by driving the lens of the imaging optical system, and can perform focus adjustment by conventionally known methods such as contrast autofocus. The imaging substrate 142 includes a CMOS sensor 143 as a light receiving element that receives light incident from the imaging optical system. The CMOS sensor 143 is an imaging sensor configured to acquire a color image. Instead of the CMOS sensor 143, for example, a light receiving element such as a CCD sensor can also be used.

The illumination module 15 includes LEDs (light emitting diodes) 151 as light emitters that illuminate an imaging area including a workpiece, and an LED driver 152 that controls the LEDs 151 . The light emission timing, light emission time, and light emission amount of the LED 151 can be arbitrarily controlled by the LED driver 152 . The LED 151 may be provided integrally with the imaging unit 3, or may be provided separately from the imaging unit 3 as an external illumination unit.

(Configuration of display device 4)
The display device 4 has a display panel such as a liquid crystal panel or an organic EL panel. A work image, a user interface image, and the like output from the control unit 2 are displayed on the display device 4 . Moreover, when the personal computer 5 has a display panel, the display panel of the personal computer 5 can be used as a substitute for the display device 4 .

(operating device)
The operation device for the user to operate the setting device 1 includes, for example, the keyboard 51 and the mouse 52 of the personal computer 5, but is not limited to these, and can accept various operations by the user. Any device that is configured to For example, a pointing device such as the touch panel 41 of the display device 4 is also included in the operating device.

The user's operation of the keyboard 51 and mouse 52 can be detected by the control unit 2 . Further, the touch panel 41 is a conventionally well-known touch-type operation panel equipped with, for example, a pressure-sensitive sensor, and the user's touch operation can be detected by the control unit 2 . The same is true when using other pointing devices.

(Configuration of control unit 2)
The control unit 2 includes a main board 13 , a connector board 16 , a communication board 17 and a power supply board 18 . The main substrate 13 is provided with a display control section 13a, an input section 13b for receiving image defect information, and a processor 13c. The display control unit 13a and the input unit 13b can be configured by an arithmetic processing device mounted on the main substrate 13, for example. Further, the display control unit 13a, the input unit 13b, and the processor 13c may be configured by one arithmetic processing unit, or the display control unit 13a, the input unit 13b, and the processor 13c may be configured by separate arithmetic processing units. good too.

The display control unit 13a, the input unit 13b, and the processor 13c control the operations of the connected substrates and modules. For example, the processor 13 c outputs a lighting control signal for controlling lighting/lighting out of the LED 151 to the LED driver 152 of the lighting module 15 . The LED driver 152 adjusts the light amount of the LED 151 and the like while switching between lighting/lighting out of the LED 151 and adjusting the lighting time according to the lighting control signal from the processor 13c.

The processor 13c also outputs an imaging control signal for controlling the CMOS sensor 143 to the imaging board 142 of the camera module 14 . The CMOS sensor 143 starts imaging in response to an imaging control signal from the processor 13c, adjusts the exposure time to an arbitrary time, and performs imaging. That is, the imaging unit 3 captures an image within the visual field range of the CMOS sensor 143 according to the imaging control signal output from the processor 13c. If an object is within the field of view, it can also be imaged. For example, the setting device 1 can capture a non-defective product image corresponding to a non-defective product and a defective product image corresponding to a defective product using the imaging unit 3 as images for learning a machine learning network. The learning image may not be an image captured by the imaging unit 3, but may be an image captured by another camera or the like.

On the other hand, during operation of the visual inspection apparatus, the image of the work can be imaged by the imaging unit 3 . In addition, the CMOS sensor 143 is configured so as to be able to output a live image, that is, a currently captured image at a short frame rate at any time.

When the CMOS sensor 143 finishes capturing an image, the image signal output from the imaging unit 3 is input to the processor 13c of the main substrate 13, processed, and stored in the memory 13d of the main substrate 13. . Details of specific processing contents by the processor 13c of the main board 13 will be described later. Note that the main board 13 may be provided with a processing device such as FPGA or DSP. A processor 13c integrated with a processing device such as FPGA or DSP may be used.

The connector board 16 is a part that receives power supply from the outside via a power connector (not shown) provided on the power interface 161 . The power supply board 18 is a part that distributes the power received by the connector board 16 to each board and modules. do. The power supply board 18 has an AF motor driver 181 . The AF motor driver 181 supplies drive power to the AF motor 141 of the camera module 14 to achieve autofocus. The AF motor driver 181 adjusts power supplied to the AF motor 141 according to an AF control signal from the processor 13c of the main board 13. FIG.

The communication board 17 is a part that executes communication between the main board 13 and the display device 4 and the personal computer 5, communication between the main board 13 and an external control device (not shown), and the like. The external control device can be, for example, a programmable logic controller or the like. Communication may be wired or wireless, and either form of communication can be realized by conventionally known communication modules.

The control unit 2 is provided with a storage device (storage section) 19 such as a solid state drive, a hard disk drive, or the like. The storage device 19 stores program files 80, setting files, and the like (software) for enabling the hardware to execute various controls and processes, which will be described later. The program files 80 and setting files can be stored in a storage medium 90 such as an optical disk, for example, and the program files 80 and setting files stored in this storage medium 90 can be installed in the control unit 2 . The program file 80 may be downloaded from an external server using a communication line. The storage device 19 can also store, for example, the image data, parameters for constructing a machine learning network of the visual inspection apparatus, and the like.

(annotation)
The setting device 1 adjusts the parameters of the machine learning network of the visual inspection apparatus by inputting a non-defective product image corresponding to a non-defective product and a defective product image corresponding to a defective product to the machine learning network. For example, it is possible to adopt a method of randomly determining the initial values of the parameters of the machine learning network, and adjusting the parameters while feeding back the image recognition error output from the machine learning network.

Prior to learning the machine learning work, the user determines whether the image input to the machine learning network is a good product image or a defective product image, and if the image input to the machine learning network is a defective product image, where is it? It is necessary to specify whether it is a defective part. This is the annotation by the user himself/herself, and in this embodiment, annotation can be executed by a plurality of methods including the first annotation method and the second annotation method.

As shown in FIG. 3, when the user performs an annotation execution start operation, the process proceeds to step SA1, where the user selects an image to be annotated. For example, a plurality of images including a workpiece W as shown in FIG. 4 are stored in the setting device 1, and the user selects a desired image from among them. After that, when proceeding to step SA2, FIG. 4A and FIG. As shown in 4B, the display control unit 13a generates an annotation execution user interface screen 200 and causes the display device 4 to display it. An annotation execution user interface screen 200 is provided with an image display area 201 for displaying an image for learning by the machine learning network. FIG. 4A shows the case where the defective product image 202 is displayed in the image display area 201, and the black circled portion is the defective portion 202a. Also, FIG. 4B shows a case where a non-defective product image 203 is displayed in the image display area 201. FIG. This is the display step. That is, before the user executes the annotation, the display control unit 13a causes the display device 4 to display an image for the machine learning network to learn, so that the user can , annotation can be performed.

Next, in step SA3, the user makes an annotation according to the first annotation method. In the first annotation method, the image displayed on the display device 4 is labeled as a non-defective product image or a defective product image. Specifically, FIG. In 4A, a defective product image 202 is displayed. As shown on the right side, the defective product image 202 has a defective product label A1 with, for example, "NG" as a label indicating that it is a defective product image. to give This defective product label A1 is defective information input by the first annotation method.

On the other hand, FIG. In 4B, a non-defective product image 203 is displayed, so the image 203 is given a non-defective product label A2 indicating that it is a non-defective product image, for example, "OK", as shown on the right side. This non-defective item label A2 is the non-defective item information input by the first annotation method. Any labeling method may be used by the user. For example, a button labeled "NG" and a button labeled "OK" may be provided on the annotation execution user interface screen 200, and these buttons desired labels A1 and A2 may be given by the operation of .

As a labeling method according to the first annotation method, labels may be attached to images one by one. The defective product label A1 may be given collectively. Also, as for non-defective product images, a plurality of non-defective product images may be similarly stored in a specific folder, and the non-defective product images in the folder may collectively be given the non-defective product label A2.

In the case of the non-defective product image 203, steps SA4 and SA5 in the flow chart shown in FIG. 3 are omitted and the process proceeds to step SA6. At step SA6, the input unit 13b receives non-defective item information input by the first annotation method. The input unit 13b can store the non-defective product image 203 displayed on the display device 4 and the non-defective product information (label A2) in the storage device 19 in association with each other.

In the case of the defective product image 202, steps SA4 and SA5 of the flowchart shown in FIG. 3 may be skipped and the process may proceed to step SA6, or steps SA4 and SA5 may be executed and the process may proceed to step SA6. When steps SA4 and SA5 are omitted, in step SA6, the input unit 13b receives defect information input by the first annotation method. The input unit 13b can cause the storage device 19 to store the defective product image 202 displayed on the display device 4 and the defective product information (the defective product label A1) in association with each other.

Steps SA4 and SA5 are an example of a second annotation method for designating a defective portion when the image displayed on the display device 4 is the defective product image 202 . Only one of step SA4 and step SA5 may be executed, or both may be executed. When proceeding from step SA3 to step SA4 or step SA5, the user may perform a predetermined operation, or the application of the defective label A2 may be detected and the process may proceed automatically.

Step SA4 is an example of a method of specifying the defective part 202a by enclosing the defective part 202a of the defective product image 202 displayed on the display device 4. This is an example of a precise designation method in which the defective portion 202a of the defective product image 202 is traced to designate the defective portion 202a in a free form. The user can select either the area specification method or the precise specification method.

After step SA3, as shown in FIG. 5, the display control unit 13a causes the display device 4 to display a user interface screen 200 for annotation execution, and an image display area 201 displays a defective product image with a defective product label A1. 202 is displayed. A designation method selection window 204 is displayed on the annotation execution user interface screen 200 . The designation method selection window 204 is provided with a first button 204a for selecting an area designation method, a second button 204b for selecting a precise designation method, and a third button 204c for selecting an eraser tool. It is When it is detected that the first button 204a has been operated by the user, the process proceeds to step SA4 of the flowchart shown in FIG. 3, and when it is detected that the second button 204b has been operated by the user, the process proceeds to step SA5. By operating the third button 204c, it is possible to cancel at least part of the specifying operation by the area specifying method or the precise specifying method, or to erase part of the specified area.

In step SA4, the user operates the pointer 205 (shown in FIG. 5) of the mouse 52 and the click button (not shown) of the mouse 52 displayed on the display device 4 to surround the defective part 202a of the defective product image 202. . In the example shown in FIG. 5, the user operates the pointer 205 and the click button of the mouse 52 to generate a rectangular frame 206 having a size surrounding the defective portion 202a. When generating the rectangular frame 206, for example, by placing the pointer 205 at the position corresponding to the upper left vertex and dragging it diagonally to the lower right, the frame 206 of the desired size and shape is generated at the desired position. can. The size and position of the frame 206 can be changed by operating the mouse 52 .

Also, FIG. As shown in 6A, three points 207 may be designated by a pointer 205 on the defective product image 202, a circular frame 208 passing through these three points 207 may be generated, and the defective portion 202a may be surrounded by this frame 208. . Also, FIG. As shown in 6B, a point 210 may be designated on the defective product image 202 by a pointer 205, a circular frame 211 centered on this point 210 may be generated, and the defective portion 202a may be surrounded by this frame 211. FIG.

Also, FIG. As shown in 6C, a plurality of points 212 may be designated by a pointer 205 on the defective product image 202, a frame 213 shaped to connect the points 212 may be generated, and the frame 213 may surround the defective portion 202a. Also, FIG. As shown in 6D, a free-form frame 214 may be generated by moving the pointer 205 on the defective product image 202 so as to surround the defective portion 202a, and the frame 214 may surround the defective portion 202a. FIG. The example shown in 6D is a method of designating an area by free drawing using a so-called freehand tool. Which designation method to use may be determined according to the shape and size of the defective portion 202a.

As described above, in step SA4, the positions, shapes, and sizes of frames 206, 208, 211, 213, and 214 surrounding the defective portion 202a are acquired as defect information. For example, by acquiring the center coordinates and the lengths of the vertical and horizontal sides of the frame 206 shown in FIG. Similarly, in the case shown in FIG. 6, by obtaining information on frames 208, 211, 213, and 214, the defective portion 202a in the defective product image 202 can be identified. The information specifying the defective portion 202a is the defect information.

When the user designates the defective portion 202a in step SA4, the process proceeds to step SA6. At step SA6, the input unit 13b receives the defect information input at step SA4. The input unit 13b can cause the storage device 19 to store the defective product image 202 displayed on the display device 4 in association with the defective product information input in step SA4.

When the user operates the second button 204b in the designation method selection window 204 on the annotation execution user interface screen 200 shown in FIG. 5 to proceed to step SA5, FIG. As shown in 7A, a fill tool 220 is displayed on the annotation-performing user interface screen 200 . The fill tool 220 can be freely moved by the user operating the mouse 52 . The user can specify the defective portion 202a in a free form by moving the paint tool 220 so as to trace the defective portion 202a in the defective product image 202. FIG. By tracing the defective portion 202a with the paint-out tool 220, portions other than the defective portion 202a are less likely to be included, enabling more precise annotation than the region designation method in step SA4. By obtaining the position, shape, and size of the region filled with the filling tool 220, the defective portion 202a in the defective product image 202 can be identified. The information specifying the defective portion 202a is the defect information.

When the user designates the defective portion 202a with the painting tool 220 at step SA5 of the flowchart shown in FIG. 3, the process proceeds to step SA6. At step SA6, the input unit 13b receives the defect information input at step SA5. The input unit 13b can cause the storage device 19 to store the defective product image 202 displayed on the display device 4 in association with the defective product information input in step SA5.

In step SA5, for example, FIG. The defective portion 202a may be specified with a magnet tool 221 as shown in 7B. The magnet tool 221 can be moved by operating the mouse 52, and when moved to the vicinity of the defective portion 202a, it moves so as to be automatically attracted to the defective portion 202a. Since the defective portion 202a is contained within the frame 222 that connects the plurality of magnet tools 221, the defective portion 202a can be specified precisely while reducing the burden on the user. The position, shape, and size of the frame 222 are defect information.

Further, in step SA5, for example, FIG. A GrabCut tool 223, such as that shown in 7C, may be used to designate the defect location 202a. When the defective portion 202a and the periphery of the defective portion 202a are enclosed by the GrabCut tool 223, the area is designated, and an automatic extraction method is executed to automatically extract the defective portion 202a within the designated region. In the automatic extraction method, only the defective portion 202a is specified, and the surrounding area of the defective portion 202a is not specified. Therefore, the defective portion 202a is automatically precisely specified as indicated by the white circle on the right. This reduces the burden on the user.

However, in the case of the GrabCut tool 223, the precise designation of the defective portion 202a may fail, and in this case, the area around the defective portion 202a is also included. FIG. 7C, in which the circular defect 202a is incorrectly specified to be nearly rectangular in this example. In such a case, after the execution of the automatic extraction method, the user finely specifies the foreground and background by correcting the stroke or correcting the click, and the shape of the area automatically extracted by the GrabCut tool 223 is corrected. The input unit 13b accepts.

Alternatively, the defective portion 202a may be designated by AI Assisted designation. In the case of AI Assisted designation, after roughly designating and extracting the contour of the defective portion 202a, the inside of the extracted portion is designated by a Fill tool or the like. Thereby, the defective portion 202a can be automatically extracted. After automatically extracting the defective portion 202a, it is also possible to make minor corrections.

After step SA6, the process proceeds to step SA7 to determine whether or not the number of annotated images has reached the required number. If the determination is NO and the required number is not reached, the process proceeds to step SA1. On the other hand, when the determination is YES and the required number of annotations is completed, the annotation is terminated. Steps SA1 to SA7 are input steps for receiving defect information input by either the first annotation method or the second annotation method.

(Specific example of annotation)
FIG. 8A shows the case where the defective product image 202 has one linear defective portion 202a. FIG. 8B shows the state in which the defective product label A1 is given by the first annotation method in step SA3. FIG. 8C shows a case where a rectangular frame 206 is generated by the region designation method of step SA4, and the defective portion 202a is designated by enclosing the defective portion 202a with the generated frame 206. FIG. FIG. 8D shows a case where a frame 213 connecting a plurality of points is generated by the region designation method of step SA4, and the defective portion 202a is designated by enclosing the defective portion 202a with the generated frame 213. FIG. FIG. 8E shows a case where a frame 230 surrounding the defective portion 202a is generated using a freehand tool in the precise designation method of step SA5, and the defective portion 202a is designated by this frame 230. FIG. After the first annotation method, the user may perform annotation by the region designation method or may perform annotation by the precise designation method. After the annotation by the region designation method, the annotation by the region designation method may be canceled and then the annotation by the precise designation method may be performed.

FIG. 9A shows the case where the defective product image 202 has two linear defective portions 202a that intersect with each other. FIG. 9B shows the state in which the defective product label A1 is given by the first annotation method in step SA3. FIG. 9C shows a case where a rectangular frame 206 is generated by the area designation method of step SA4, and the defective portion 202a is designated by enclosing the defective portion 202a with the generated frame 206. FIG. FIG. 9D shows a case where a frame 213 connecting a plurality of points is generated by the region designation method of step SA4, and the defective portion 202a is designated by enclosing the defective portion 202a with the generated frame 213. FIG. FIG. 9E shows a case where a frame 230 surrounding the defective portion 202a is generated using a freehand tool in the precise designation method of step SA5, and the defective portion 202a is designated by this frame 230. FIG. FIG. 9D and FIG. In the example shown in FIG. 9E, FIG. As compared with the example shown in 9C, it becomes possible to specify the defective portion 202a precisely.

FIG. 9F shows the case where two rectangular frames 206 are generated by the area designation method of step SA4 and the defective portion 202a is designated by the two frames 206. FIG. FIG. 9G shows a case where a rectangular frame 206 and a frame 213 connecting a plurality of points are generated by the region designation method of step SA4, and the defective portion 202a is designated by the frames 206 and 213. FIG. In this way, the defective portion 202a may be designated by combining a plurality of types of frames 206 and 213. FIG.

(First example of parameter adjustment procedure for machine learning network)
The processor 13c causes the machine learning network to learn both the defective product image 202 having the defect information input by the first annotation method and the defective product image 202 having the defect information input by the second annotation method. That is, the defective product image 202 to which the defective product label A1 indicating that it is a defective product image 202 by the first annotation method and the defective product image 202 to which the defective part 202a is specified by the second annotation method are combined into a single image. and adjust the parameters of the machine learning network based on the defect information of each defective product image 202 .

A first example of the parameter adjustment procedure of the machine learning network will be specifically described below based on the flowchart shown in FIG. The parameter adjustment of the machine learning network may be performed by the user, by the manufacturer of the setting device 1, or on the cloud.

At step SB1 after the start, annotation is performed according to the first annotation method. At step SB2, the processor 13c causes the machine learning network to learn the non-defective product image 203 to which the non-defective product label A2 is assigned by the first annotation method and the defective product image 202 to which the defective product label A1 is assigned by the first annotation method. That is, the processor 13c adjusts the parameters of the machine learning network based on the non-defective product information of each non-defective product image 203 and the defective product information of each defective product image 202 . The number of non-defective product images 203 and defective product images 202 to be learned by the machine learning network in step SB2 can be set to any number. Step SB2 is an adjustment step for adjusting the parameters of the machine learning network based on the defect information of the defective product image.

After that, the process proceeds to step SB3, and the processor 13c executes verification processing. Specifically, the processor 13c adds a plurality of mutual Different verification image data is input and a machine learning network is made to determine the quality of the verification image data. At step SB3, the processor 13c generates a user interface screen 300 as shown in FIG. Step SB3 is a verification step of inputting the verification image data to the machine learning network after the parameters have been adjusted and executing quality judgment of the verification image data.

The user interface screen 300 is provided with a verification image data display area 301 for displaying verification image data. In the verification image data display area 301, the verification image data input to the machine learning network by the processor 13c is displayed in a list format. Verification image data marked with "OK" is a non-defective product image, and verification image data marked with "NG" is a defective product image.

A user interface screen 300 is provided with an enlarged image display area 302 . Among the verification image data displayed in the verification image data display area 301 , the verification image data selected by the user is enlarged and displayed in the enlarged image display area 302 .

A learning parameter analysis area 303 is provided on the user interface screen 300 . In other words, the processor 13c obtains a plurality of determination results by causing the machine learning network to determine the quality of the plurality of verification image data. The display control unit 13a causes the display device 4 to display a plurality of determination results acquired by the processor 13c as the learning parameter analysis area 303, so that the user can grasp the degree of inference performance of the machine learning network.

In this example, a case will be described in which a cumulative histogram is displayed as the display form of the determination result. In this case, for example, the processor 13c acquires, as the determination results, the number of times the image is determined to be a non-defective image and the number of times the image is determined to be a defective image. The processor 13c generates a cumulative histogram based on the number of times an image is determined to be a good image and the number of times an image is determined to be a defective image. The display control unit 13a causes the display device 4 to display the cumulative histogram generated by the processor 13c. In the learning parameter analysis area 303 shown in FIG. 11, the frequency distribution of non-defective product images and the frequency distribution of defective product images are displayed in graph format. In the graph shown in FIG. 11, the area of the non-defective product image (area indicated as OK in the figure) and the area of the defective product image (area indicated as NG in the figure) cannot be separated. In such a case, it is considered that the inference performance of the machine learning network is insufficient, so in step SB4 it is determined that the parameter adjustment in step SB2 has not yielded the desired result. On the other hand, if it is determined in step SB4 that the desired result has been obtained, the process proceeds to step SB5. "When it is determined that a desired result has been obtained" will be described later. "Operation" in step SB5 means putting the appearance inspection apparatus into an operable state, and the operation may be started immediately or may be in a standby state.

At Step SB6, to which the determination at Step SB4 is NO, the user is urged to perform the second annotation. That is, a message display area 304 is provided on the user interface screen 300 shown in FIG. In this example, the area of the non-defective product image and the area of the defective product image cannot be separated, and it is considered that the inference performance of the machine learning network is insufficient. A method for improving performance is generated by the processor 13c and displayed on the display device 4 by the display control unit 13a. Methods for improving the inference performance of the machine learning network include, for example, "reviewing the feature size", "considering image variations", and "refining annotations".

As described above, the second annotation method enables more detailed annotation than the first annotation method. Therefore, displaying the display "refine the annotation" in the message display area 304 prompts the user to execute the second annotation method, or as an option for improving the inference performance of the machine learning network. It is to present to the user that there is a second annotation.

A user interface screen 300 shown in FIG. 11 is provided with an image number display area 305 . A table is displayed in the number-of-images display area 305, and the table is provided with a column of "label designation". In the "label specification" column, the number of images that could be detected as defective images and the number of images that could not be detected as defective images among the defective product images 202 to which the defective product label A1 was assigned by the first annotation method. is displayed. In this example, the number of images that could be detected as defective images is "80", and the number of images that could not be detected as defective images is "6". In other words, as a result of the pass/fail judgment in the verification process, there are six images that were not judged as defective even though they are verification image data corresponding to defective products, which means that the inference performance of the machine learning network is insufficient. can be determined to be If the number of images that could not be detected as defective images is "0", it can be determined that the inference performance of the machine learning network is sufficient, and operation can be started with that machine learning network.

The user can use the mouse 52 to click the number “6” indicating the number of images that could not be detected as defective images in the table displayed in the image number display area 305 . When the number "6" is clicked with the mouse 52, as shown in FIG. 12, the display control unit 13a displays the six images that were not determined as defective despite being verification image data corresponding to defective products. Displayed on the display device 4 . Specifically, the six images are displayed as a list in the verification image data display area 301 of the user interface screen 300 . When the number of images is large, all the images can be browsed by the user by scrolling up and down or left and right.

When the user executes the second annotation, a desired image is selected from the six images displayed in the verification image data display area 301 of the user interface screen 300 shown in FIG. This selection operation can be performed by an operating device such as the mouse 52, for example. In other words, the input unit 13b receives a selection input of an image for which a defective portion is specified by the second annotation method from among the images displayed as a list on the display device 4. FIG.

When the input unit 13b receives an image selection input, the display control unit 13a causes the selected image to be displayed in the enlarged image display area 302 of the user interface screen 300 as shown in FIG. Window 204 is also displayed. The user uses the tools in the designation method selection window 204 to perform annotation by the area designation method and annotation by the precise designation method. In this manner, the input unit 13b accepts designation of a defective portion for the selected image by the second annotation method, and the accepted result is stored in the storage device 19 together with the image. After completing the designation of one image by the second annotation method, another image may be selected, and the defective portion may be designated by the second annotation method on the other image. The above is the processing of step SB6 in the flowchart shown in FIG.

After step SB6, the process proceeds to step SB7. At Step SB7, an update process is executed to update the parameters of the machine learning network by learning the verification image data in which the defective portion was designated by the second annotation method at Step SB6. When executing learning, the user may operate the execution button 306 on the user interface screen 300 . That is, the parameters of the machine learning network are changed by causing the machine learning network to learn the image in which the defective portion is designated by the second annotation method, which is more detailed than the first annotation method. This improves the inference performance of the machine learning network.

Whether or not the inference performance of the machine learning network is sufficient, that is, whether or not it is determined that a desired result has been obtained is displayed in the learning parameter analysis area 303 of the user interface screen 300 shown in FIG. This can be confirmed by looking at the graph shown. In the graph shown in FIG. 14, the region of the non-defective product image and the region of the defective product image can be completely separated, and the machine learning network has the inference performance that can clearly distinguish between the non-defective product image and the defective product image. I understand.

In addition, as shown in FIG. 14, the table displayed in the image number display area 305 is provided with a column of "specify square area" and a column of "specify details". “Rectangular area specification” is a field indicating the number of images that could be detected as defective images and the number of images that could not be detected as defective images after annotation by the area specification method. Since the number of images detected as defective images is "87", it can be seen that all images that should be detected as defective images have been detected as defective images. Further, "detailed designation" is a field indicating the number of images that could be detected as defective images and the number of images that could not be detected as defective images after annotation by the precise designation method. For example, if the number of images that could not be detected as defective images is other than "0" in the "Rectangle Area Designation" column, the user may perform annotation using the precise designation method. The number of images that could not be detected as defective images can be set to "0" in the "detailed specification" column.

The above is the description of the setting method using the setting device 1 . If sufficient inference performance can be obtained with only the first annotation method, the system can be put into operation without executing the second annotation method, thus not imposing an unnecessary burden on the user. Also, if sufficient inference performance cannot be obtained with only the first annotation method, the region designation method, which is relatively less burdensome, can be used for annotation to confirm the inference performance. As a result, when sufficient inference performance is not obtained, annotation by the precise designation method can be executed only for necessary images, thus reducing the user's burden.

(Second example of parameter adjustment procedure for machine learning network)
A second example of the parameter adjustment procedure of the machine learning network will be specifically described based on the flowchart shown in FIG. 15 . Step SC1 after the start is the same as step SB1 in the flow chart shown in FIG. After that, in step SC2, annotation is performed according to the second annotation method. At step SC3, the processor 13c generates a non-defective product image 203 to which the non-defective product label A2 is assigned by the first annotation method, a defective product image 202 to which the defective product label A1 is assigned by the first annotation method, and a defective product image 202 which is assigned the defective product label A1 by the first annotation method. A machine learning network is trained on images with specified locations. That is, the processor 13c adjusts the parameters of the machine learning network based on the non-defective product information of each non-defective product image 203 and the defective product information of each defective product image 202 .

After that, the process proceeds to step SC4, and the processor 13c executes verification processing. Specifically, the processor 13c inputs a plurality of verification image data different from each other to the machine learning network generated in step SC3, and causes the machine learning network to determine whether the verification image data is good or bad. Steps SC5 and SC6 are the same as steps SB4 and SB5 in the flowchart shown in FIG.

At Step SC7, to which Step SC5 is judged as NO, the user is prompted to re-annotate. For example, if only the region specification method is performed on a certain image by the second annotation method, the user is prompted to perform annotation by the precise specification method on that image. In addition, the user may be prompted to perform annotation using the precise specification method more precisely.

After step SC7, the process proceeds to step SC8. In step SC8, an update process is executed to update the parameters of the machine learning network by learning the verification image data re-annotated in step SC7. The parameters of the machine learning network are changed by having the machine learning network learn images in which the defect locations are specified by re-annotation. This improves the inference performance of the machine learning network.

(Details of neural network for anomaly extraction)
The machine learning network is composed of, for example, a neural network for anomaly extraction as conceptually shown in FIG. The anomaly extraction neural network outputs an anomaly degree as a heat map (abnormality degree map) indicating whether each pixel forming the input image is abnormal. This heatmap becomes the output image. Therefore, when a non-defective product image is used as an input image, the output image is an anomaly degree map in which all pixels with the same pixel value are configured. ) becomes an abnormality degree map in which pixel values differ from those of other pixels.

In this embodiment, even if the second annotation method is not used for precise designation, and even if multiple annotation methods are mixed such as the first annotation method and the second annotation method, learning that can extract only the defective part is performed. method is adopted. That is, as shown in FIG. 17, the processor 13c generates at least three different images including both a non-defective product image having non-defective product information and a defective product image having defective product information input by the first annotation method. is input to a machine learning network to generate an anomaly map. In the example shown in FIG. 17, the first input image, the second input image, and the third input image are input to the same machine learning network as one group. The first input image and the second input image are good product images, and the third input image is a bad product image. Labels A1 and A2 are given to the first input image, the second input image, and the third input image by the first annotation method.

The processor 13c inputs the non-defective product image having the non-defective product information input by the first annotation method to the machine learning network, and sets the parameters of the machine learning network so that each pixel value in the abnormality map generated is 0. adjust. In this example, since the first input image and the second input image are non-defective images, each pixel value in the first output image and the second output image is restricted to zero.

In addition, the processor 13c is configured to be capable of executing at least one of a process of reducing the distance between the images having non-defective product information and the process of reducing the distance between the abnormality maps of the images having defective product information. It is In this example, since the first input image and the second input image are non-defective images, processing is performed to reduce the distance between the abnormality degree maps of the first output image and the second output image. If the first input image and the second input image are defective images, processing is performed to reduce the distance between the abnormality degree maps of the first output image and the second output image.

In addition, the processor 13c is configured to be able to execute a process of increasing the distance between the abnormality degree map of the image having non-defective product information and the abnormality degree map of the image having defective product information by a predetermined distance or more. In this example, since the second input image is a non-defective product image and the third input image is a defective product image, the abnormality degree maps of the second output image and the third output image are separated by a predetermined amount or more.

Specifically, the following calculation formulas and ideas (for example, calculation methods 1 and 2 below) can be applied. First, the Loss function is defined. The pixel value average of the corresponding abnormality degree map of each input image can be calculated by the following formula.

In addition, n is the number of pixels of the abnormality degree map H, and x and y are pixel coordinate values of the abnormality degree map H.

The average pixel values of the anomaly maps are used, and the distance between the anomaly maps is defined as the absolute value of the difference between the average pixel values.

Calculation method 1 (Method of directly limiting the distance between anomaly degree maps)

α is a hyperparameter that represents the distance margin, and λ is a hyperparameter that represents the strength of the Loss restriction for the non-defective part of the anomaly degree map.

Calculation method 2 (Method of indirectly limiting the distance between the anomaly degree maps)

It should be noted that the distance between the anomaly maps can also be defined as the Euclidean distance between the vectors by converting the anomaly maps into one-dimensional vectors. Also, learning can be performed by minimizing the Loss function.

The above is the case of learning images labeled by the first annotation method, but learning can also be performed on images with defect locations designated by the second annotation method. That is, the processor 13c inputs the defective product image having the defective product information input by the second annotation method to the machine learning network, and in the abnormality degree map generated by the machine learning network, The parameters of the machine learning network are adjusted so that the pixel value of the portion corresponding to is 0.

A specific description will be given based on FIG. FIG. 18 shows a case where a defective portion is designated by the second annotation method for the third input image. In this example, the case where the frame 206 surrounding the defective portion is generated by the area designation method is shown, but the defective portion may be designated by the precise designation method. As shown as a mask image under the third input image, the pixel values of the area corresponding to the outside of the frame 206 are set to 0. In the third output image, the pixel values are set to 0 in areas other than the area specified as the defective area.

The definition of the Loss function is as described above, but the pixel average values inside and outside the annotation region are calculated for the third input image.

Also, the average pixel values of the anomaly degree maps are used, and the distance between maps is defined as the absolute value of the difference between the average pixel values.

Calculation method 1 (Method of directly limiting the distance between anomaly degree maps)

Calculation method 2 (Method of indirectly limiting the distance between the anomaly degree maps)

(Action and effect of the embodiment)
As described above, according to the present embodiment, in the first annotation method, images displayed on the display device 4 can be sorted into non-defective product images and defective product images by label designation. It is possible to annotate a defective product image that should be determined as. On the other hand, in the second annotation method, since the position and shape of the defective portion can be specifically specified on the defective product image, it is possible to specify the defective portion more clearly than in the first annotation method. As a result, the user can annotate a defective product image that should be determined to be defective as a whole without requiring the user's trouble, and the user can annotate the defective product image that should clearly specify the defective part. can be done. As a result, it is possible to obtain a machine learning network with high detection performance by eliminating individual dependence while reducing the user's labor at the time of learning the machine learning network.

The above-described embodiments are merely examples in all respects and should not be construed in a restrictive manner. Furthermore, all modifications and changes within the equivalent scope of claims are within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the present invention can be used, for example, when setting an appearance inspection apparatus for inspecting the appearance of a work.

1 setting device 13a display control unit 13b input unit 13c processor

Claims (15)

A work image of a work to be inspected is input to a machine learning network generated by learning non-defective product images corresponding to non-defective products and defective product images corresponding to defective products, and the quality of the work image is determined. A setting device for setting the appearance inspection device to be performed,
a display control unit for displaying an image for learning by the machine learning network on a display unit;
A first annotation method of adding a label indicating a non-defective product image or a defective product image to the image displayed on the display unit; an input unit that receives defect information input by one of the second annotation method that specifies the
Let the machine learning network learn both the non-defective product image or the defective product image labeled by the first annotation method and the defective product image having the defect information input by the second annotation method, and a processor for adjusting parameters of the learning network. The setting device according to claim 1,
The input unit includes, as the second annotation method, a region specification method in which the defective portion is designated by enclosing the defective portion in the defective product image, and a free-form defective portion by tracing the defective portion in the defective product image. A setting device for an appearance inspection device, configured to be able to select a precise specification method to specify. The setting device according to claim 1,
The input unit is configured to be capable of executing, as the second annotation method, an automatic extraction method for automatically extracting the defective portion within a designated area by enclosing the defective portion of the defective product image and the periphery of the defective portion. A setting device for a visual inspection device. The setting device according to claim 3,
The setting device of the appearance inspection apparatus, wherein the input unit receives correction of the shape of the region automatically extracted by the automatic extraction method as the second annotation method after execution of the automatic extraction method. The setting device according to any one of claims 1 to 4,
The processor inputs a non-defective product image or a defective product image labeled by the first annotation method and a defective product image to which a defective part is specified by the second annotation method into the single machine learning network. , a setting device for a visual inspection device for adjusting the parameters of the machine learning network. The setting device according to any one of claims 1 to 5,
The processor
inputting verification image data to the machine learning network generated by learning a non-defective product image or a defective product image labeled as a non-defective product image or a defective product image by the first annotation method; a verification process for determining whether the verification image data is good or bad;
As a result of the pass/fail determination in the verification process, if the verification image data corresponding to the defective product is not determined to be defective, the verification image data in which the defective portion is specified by the second annotation method is selected. A setting device for a visual inspection apparatus configured to be capable of executing an update process of causing the machine learning network to learn image data and updating parameters of the machine learning network. The setting device according to any one of claims 1 to 6,
The display control unit causes the display unit to display a list of images that are not determined to be defective among the images labeled as defective product images by the first annotation method,
The input unit accepts a selection input of an image for which a defective portion is to be designated by the second annotation method from among the images displayed as a list on the display unit, and determines whether the selected image is defective by the second annotation method. A setting device for a visual inspection device that accepts location designations. The setting device according to any one of claims 1 to 6,
The processor inputs a plurality of verification image data different from each other to the machine learning network generated by learning the defective product image labeled as the defective product image by the first annotation method. to acquire the judgment result by executing the pass/fail judgment of each verification image data,
The display control unit is a setting device for a visual inspection apparatus, which causes the display unit to display the determination result acquired by the processor. The setting device according to claim 8,
The processor acquires the number of times the image is determined to be a non-defective image and the number of times the image is determined to be a defective image as the determination results, and the number of times the image is determined to be a non-defective image and the image is determined to be a defective image. generating a cumulative histogram based on the determined frequencies and
A setting device for a visual inspection apparatus, wherein the display control unit causes the display unit to display the cumulative histogram generated by the processor. A setting device according to any one of claims 1 to 9,
The processor
At least three images that include both a non-defective product image having non-defective product information and a defective product image having defective product information input by the first annotation method and are different from each other are input to the machine learning network to provide an anomaly map to generate
A setting device for a visual inspection apparatus configured to be capable of executing a process of separating an abnormality degree map of an image having non-defective product information and an abnormality degree map of an image having defective product information by a predetermined distance or more. The setting device according to claim 10, wherein
The processor is configured to be able to execute at least one of a process of reducing the distance between the images having non-defective product information and the process of reducing the distance between the abnormality maps of the images having defective product information. A setting device for visual inspection equipment. The setting device according to claim 10 or 11,
The processor inputs the non-defective product image having the non-defective product information input by the first annotation method to the machine learning network so that each pixel value in the abnormality map generated is 0 of the machine learning network A setting device for visual inspection equipment that adjusts parameters. A setting device according to claim 12, wherein
The processor inputs the defective product image having the defective product information input by the second annotation method to the machine learning network, and the defect location specified by the second annotation method in the anomaly degree map generated by the machine learning network A setting device for a visual inspection device that adjusts the parameters of the machine learning network so that the pixel value of the portion corresponding to the area other than the above is 0. A work image of a work to be inspected is input to a machine learning network generated by learning non-defective product images corresponding to non-defective products and defective product images corresponding to defective products, and the quality of the work image is determined. A setting method for a visual inspection apparatus to perform
a display step of displaying an image for learning by the machine learning network on a display unit;
a first annotation method of adding a label indicating a non-defective product image or a defective product image to the image displayed on the display unit in the display step; an input step of receiving defect information input by any one of a second annotation method for designating a defect location in a case;
Let the machine learning network learn both the non-defective product image or the defective product image labeled by the first annotation method and the defective product image having the defect information input by the second annotation method, and an adjusting step of adjusting parameters of the learning network. In the setting method according to claim 14,
After having the machine learning network learn the defective product image to which the label indicating the defective product image is given by the first annotation method, the verification image data is input to the machine learning network to generate the verification image. a verification step for performing pass/fail judgment of data;
As a result of the pass/fail judgment in the verification step, if the verification image data corresponding to the defective product is not judged to be defective, after specifying the defective portion by the second annotation method, and an updating step of causing the machine learning network to learn the verification image data in which the defective portion is designated by the 2 annotation method, and updating parameters of the machine learning network.

Images