JP7128107B2 - Image inspection device generation device, image inspection device, image inspection device generation program, and image inspection device generation method - Google Patents

Description

The present invention relates to an image inspector generation device, an image inspection device, an image inspector generation program, and an image inspector generation method.

Conventionally, as a classifier generation device, CNN generation means for generating a CNN (Convolutional Neural Network) by deep learning based on first teacher data including a plurality of first images showing defects, and CNN generation means A feature amount acquisition unit for inputting each of a plurality of second images showing defects to an intermediate layer constituting a CNN and acquiring a plurality of types of feature amounts for each of the plurality of second images; and a classifier generating means for generating a classifier by machine learning based on second teacher data including at least some of the acquired plural types of feature amounts (Patent Reference 1). Patent Literature 1 describes that a classifier generated by this classifier generation device classifies an image of an inspection target that includes a defect.

Japanese Patent Application Laid-Open No. 2018-5640

In the case of an image inspection apparatus that inspects an image using a classifier generated based on a CNN generated by deep learning, such as the classifier described in Patent Document 1, the accuracy of image classification and inspection by the classifier In order to increase , it was necessary to prepare a wide variety of training images and generate a classifier based on deep learning of the training images. For this reason, an image inspection apparatus with high inspection accuracy requires a large number of images.

Therefore, the present invention provides an image inspector generation apparatus, an image inspector generation program, and an image inspector generation method that can generate an image inspector with high inspection accuracy without requiring a large amount of images. With the goal.

An image inspector generation apparatus according to an aspect of the present invention includes an image inspector generation unit that generates an image inspector that inspects an inspection object included in an image by machine learning based on an inspection standard for the inspection object and a learning image. , and the test criteria include information about the state variables of the test object, the state values of the state variables, and the class into which the test object is classified.

According to this aspect, an image inspector for inspecting the inspection object included in the image is generated by machine learning based on the inspection standard for the inspection object and the learning image, and the inspection standard includes the state variables of the inspection object, the state Contains information about the state values of variables and the class into which the study object is classified. In this way, by performing machine learning based on the inspection standard and the learning image, which are logical information, the learning efficiency of machine learning can be improved compared to the case of performing machine learning based on the learning image. becomes possible. Therefore, it is possible to generate an image inspection device with high inspection accuracy without requiring a large amount of learning images as compared with the conventional art.

In the above aspect, the image inspector includes a detector and a classifier, and the image inspector generator includes a detector generator that generates a detector for detecting inspection objects included in the image, and a detector generator that detects inspection objects in the image. and a classifier generator that generates a classifier that classifies the state of the inspected object.

According to this aspect, a detector is generated for detecting an inspection object contained in an image, and a classifier is generated for classifying the state of the inspection object detected in the image. In this way, by separately generating a detector for detecting an inspection object contained in an image and a classifier for classifying the state of the detected inspection object, the generation of the detector by machine learning can be performed using the inspection object On the other hand, generation of a classifier by machine learning does not require learning images of different variations such as the number, position, range, etc. of inspection objects.

In the above aspect, the classifier generator may generate the classifier by machine learning based on partial images of inspection objects detected in the image.

According to this aspect, the classifier is generated by machine learning based on sub-images of inspection objects detected in the images. As a result, the number, position, range, etc., of the inspection objects in the training images are abstracted. Therefore, by generating a classifier by machine learning based on partial images of different variations of inspection object states, a large amount of training images can be generated. A classifier generation unit that generates a classifier can be easily realized without the need for

The above aspect may further include an input unit for inputting inspection criteria.

According to this aspect, it further includes an input unit for inputting inspection criteria. This makes it possible to easily set the inspection criteria.

Further, an image inspection apparatus according to another aspect of the present invention includes an image inspection device generated by the image inspection device generation device described above, and inspects an inspection image using the image inspection device.

According to this aspect, the inspection image is inspected using the image inspector. As a result, an image inspection apparatus with high inspection accuracy can be easily realized.

In the above-described aspect, the image inspector is generated by machine learning based on a plurality of inspection criteria, and as a result of inspection using the image inspector, when it is determined that there is a defect in the inspection image, the image inspector determines the plurality of inspection criteria. Among them, an output unit may be further provided for outputting an inspection standard that is the cause of the determination that there is a defect.

According to this aspect, when it is determined that there is a defect in the inspection image as a result of inspection using the image inspector, the inspection standard that caused the defect to be determined among the plurality of inspection standards is output. be done. As a result, when it is determined that there is a defect, the inspection results using the image inspector are more descriptive than when the inspection criteria that caused the defect are not output.

Further, an image inspector generating program according to another aspect of the present invention is an image inspector generating program to be executed by a computer, and is included in an image by machine learning based on an inspection standard for an inspection object and a learning image. An image inspector generation step for generating an image inspector for inspecting the inspection object, wherein the inspection criteria include state variables of the inspection object, state values of the state variables, and class information into which the inspection object is classified.

According to this aspect, an image inspector for inspecting the inspection object included in the image is generated by machine learning based on the inspection standard for the inspection object and the learning image, and the inspection standard includes the state variables of the inspection object, the state Contains information about the state values of variables and the class into which the study object is classified. In this way, by performing machine learning based on the inspection standard and the learning image, which are logical information, the learning efficiency of machine learning can be improved compared to the case of performing machine learning based on the learning image. becomes possible. Therefore, it is possible to generate an image inspection device with high inspection accuracy without requiring a large amount of learning images as compared with the conventional art.

In the aforementioned aspect, the image inspector includes a detector and a classifier, and the image inspector generation step includes a detector generation step for generating a detector for detecting inspection objects contained in the image; and a classifier generation step of generating a classifier that classifies states of the inspected objects.

According to this aspect, a detector is generated for detecting an inspection object contained in an image, and a classifier is generated for classifying the state of the inspection object detected in the image. In this way, by separately generating a detector for detecting an inspection object contained in an image and a classifier for classifying the state of the detected inspection object, the generation of the detector by machine learning can be performed using the inspection object On the other hand, generation of a classifier by machine learning does not require learning images of different variations such as the number, position, range, etc. of inspection objects.

In the aforementioned aspect, the classifier generation step may comprise generating the classifier by machine learning based on sub-images of the inspection object detected in the image.

According to this aspect, the classifier is generated by machine learning based on sub-images of inspection objects detected in the images. As a result, the number, position, range, etc., of the inspection objects in the training images are abstracted. Therefore, by generating a classifier by machine learning based on partial images of different variations of inspection object states, a large amount of training images can be generated. A classifier generation step for generating a classifier can be easily realized without requiring

The above aspect may further include an input step for inputting inspection criteria.

According to this aspect, further comprising an input step for inputting inspection criteria. This makes it possible to easily set the inspection criteria.

Further, an image inspection device generation method according to another aspect of the present invention is an image inspection method for generating an image inspection device for inspecting an inspection object included in an image by machine learning based on an inspection standard for the inspection object and a learning image. The test criteria include state variables of the test object, state values of the state variables, and class information into which the test object is classified.

According to this aspect, an image inspector for inspecting the inspection object included in the image is generated by machine learning based on the inspection standard for the inspection object and the learning image, and the inspection standard includes the state variables of the inspection object, the state Contains information about the state values of variables and the class into which the study object is classified. In this way, by performing machine learning based on the inspection standard and the learning image, which are logical information, the learning efficiency of machine learning can be improved compared to the case of performing machine learning based on the learning image. becomes possible. Therefore, it is possible to generate an image inspection device with high inspection accuracy without requiring a large amount of learning images as compared with the conventional art.

According to the present invention, it is possible to generate an image inspection device with high inspection accuracy without requiring a large amount of images.

FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system according to one embodiment. FIG. 2 is a diagram illustrating a schematic configuration for generating the image inspector shown in FIG. FIG. 3 is a diagram illustrating a first example of generating an image inspector by machine learning. FIG. 4 is a diagram illustrating a second example of generating an image inspector by machine learning. FIG. 5 is a diagram illustrating a third example of generation of the image inspector 33 by machine learning. FIG. 6 is a flow chart showing a schematic operation of generating an image inspector of the image inspector generating apparatus according to one embodiment.

Embodiments of the present invention are described below. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, the drawings are schematic. Therefore, specific dimensions and the like should be determined by referring to the following description. In addition, it goes without saying that there are portions with different dimensional relationships and ratios between the drawings. Furthermore, the technical scope of the present invention should not be construed as being limited to this embodiment.

First, with reference to FIGS. 1 to 5, an example of configuration of an image inspector generation apparatus and an image inspection apparatus according to an embodiment of the present invention will be described. FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system 100 according to one embodiment. FIG. 2 is a diagram illustrating a schematic configuration for generating the image inspector 33 shown in FIG. FIG. 3 is a diagram illustrating a first example of generation of the image inspector 33 by machine learning. FIG. 4 is a diagram illustrating a second example of generation of the image inspector 33 by machine learning. FIG. 5 is a diagram illustrating a third example of generation of the image inspector 33 by machine learning.

As shown in FIG. 1, the image inspection system 100 includes a network NW, an image inspector generation device 50, and an image inspection device 90. As shown in FIG.

The image inspection device generation device 50 and the image inspection device 90 are connected so as to be able to communicate with each other via a network NW. The network NW is, for example, the Internet, a LAN (Local Area Network), a dedicated line, a telephone line, an in-house network, Bluetooth (registered trademark), Wi-Fi (Wireless Fidelity), a mobile communication network, other communication lines, or A combination of these and the like. Moreover, the connection between the image inspection generator generation device 50 and the image inspection device 90 and the network NW may be wired or wireless.

In this embodiment, FIG. 1 shows an example in which one network NW is included in the image inspection system 100, but the present invention is not limited to this. For example, two or more networks may be included in the image inspection system 100 .

The image inspection device generation device 50 is a computer that serves as a high-level controller for the image inspection device 90 . The image inspector generation device 50 is configured including a controller such as a PLC (Programmable Logic Controller), for example.

The image inspector generation device 50 includes, for example, a communication unit 10, an input unit 20, a storage unit 30, and a control unit 40. The image inspector generation device 50 further includes a bus 51 configured to transmit signals and data between each section of the image inspector generation device 50 .

The communication unit 10 is configured to communicate (transmit and receive) data via the network NW. More specifically, the communication unit 10 is configured to communicate with the image inspection apparatus 90 connected to the network NW based on one or more predetermined communication methods via the network NW.

The input unit 20 is for inputting inspection criteria 32, which will be described later. More specifically, the input unit 20 is configured to input inspection criteria to the image inspector generation device 50 by a user's operation. The input unit 20 includes, for example, an interface and an input device that is an external device. The interface is configured to exchange data and signals with an external device. Contains interface connection terminals. Input devices are, for example, devices such as keyboards, keypads, mice, trackballs, touch panels, and microphones.

For example, when the user operates an input device such as a keyboard, keypad, mouse, trackball, touch panel, microphone (including voice operation using a microphone), the input unit 20 outputs data corresponding to the operation. or output a signal. Data or signals output by the input unit 20 are input to the control unit 40 . The control unit 40 generates the inspection standard 32 based on this data or signal, so that the inspection standard can be input to the image inspector generating device 50 . Thus, by providing the input unit 20 for inputting the inspection standard 32, the inspection standard 32 can be easily set.

The storage unit 30 is configured to store programs, data, and the like. The storage unit 30 includes, for example, a hard disk drive, a solid state drive, and the like. The storage unit 30 stores in advance various programs executed by the control unit 40 and data necessary for executing the programs.

The storage unit 30 also stores learning data 31 , a plurality of inspection standards 32 , and a plurality of image inspectors 33 . The learning data 31 includes a plurality of learning images and teacher data assigned to the corresponding learning images. Each inspection criterion 32 is for an inspection object contained in the image. One or a plurality of inspection criteria 32 are set for each inspection object.

The control unit 40 is configured to control the operation of each unit of the image inspector generation device 50 such as the communication unit 10, the input unit 20, the storage unit 30, and the like. Further, the control unit 40 is configured to implement each function described later by executing a program stored in the storage unit 30 or the like. The control unit 40 includes, for example, a processor such as a CPU (Central Processing Unit), a memory such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a buffer storage device such as a buffer.

Further, the control unit 40 includes, for example, an image inspector generation unit 45 as its functional configuration.

The image inspector generating unit 45 generates the image inspector 33 for inspecting the inspection object included in the image by machine learning based on the inspection standard 32 for the inspection object and the learning image included in the learning data 31. It is configured. More specifically, the image inspector 33 includes a detector 33 a and a classifier 33 b , and the image inspector generator 45 includes a detector generator 41 and a classifier generator 42 .

The detector generation unit 41 is configured to generate a detector 33a for detecting an inspection object included in the image by machine learning based on the inspection standard 32 and the learning image included in the learning data 31 .

The classifier generating unit 42 generates a classifier 33b that classifies the state of the inspection object detected in the inspection image by machine learning based on the inspection standard 32 and the learning image included in the learning data 31. It is configured.

In this way, an image inspector 33 is generated that includes a detector 33a and a classifier 33b. The control unit 40 generates the image inspector 33 for each inspection object, and the generated image inspector 33 is stored in the storage unit 30 .

In this way, by separately generating the detector 33a for detecting the inspection object contained in the image and the classifier 33b for classifying the state of the detected inspection object, the detector 33a can be generated by machine learning. On the other hand, generation of the classifier 33b by machine learning does not require learning images with different variations in the number, position, range, etc. of inspection objects. Therefore, it is possible to generate an image inspection device 33 with high inspection accuracy without requiring a large amount of learning images as compared with the conventional art.

Here, generation of an image inspector by machine learning will be described with reference to FIGS. 2 to 5. FIG. In the following description, "other than" may be indicated using the symbol "￢".

When the information of the inspection standard is input by the user's operation, the control unit 40 generates the inspection standard 32 for the inspection object. The generated inspection standard 32 is stored in the storage unit 30 . The inspection criteria 32 are described, for example, in a format such as "NG if the state variable X of the inspection object A is the state value X1" or "NG if the state variable Y of the inspection object B is the state value Y1". Note that the number of inspection criteria 32 is not limited to one for each inspection object, and a plurality of inspection criteria 32 may be input and set.

The inspection criteria 32 include information on the "inspection object", the "state variables", the "state values" of the state variables, and the class into which the image is classified. For example, the control unit 40 classifies the inspection object into an “OK” class and an “NG class” using the information of “inspection object”, “state variable”, and “state value” of the state variable input by the user (user). '' class, the inspection criteria 32 may be generated for classification into one of the ``NG'' classes.

In this way, by performing machine learning based on the inspection standard 32, which is logical information, and the learning image, the learning efficiency of machine learning is improved compared to the case of performing machine learning based on the learning image. becomes possible. Therefore, it is possible to generate an image inspection device 33 with high inspection accuracy without requiring a large amount of learning images as compared with the conventional art.

The inspection object is the part of the object that appears in the image. Inspection objects are, for example, screw holes, cables, needles, containers, rings, electrodes, cutting surfaces, expiration date stamps, and the like.

A state variable is a variable of the study object. State variables are, for example, width, length, area, size, direction, number, color, shape, surface condition, date, and the like.

A state value is the value of the state variable. The status values are, for example, 100 pixels, parallel, 30 degrees, 4, red, variable, burr, scratch, dirt, cloudy, October 17, and so on.

The detector 33a and the classifier 33b are generated using an arbitrary machine learning model algorithm. The detector 33a is generated using, for example, the first neural network NN1 shown in FIG. 2, and the classifier 33b is generated using, for example, the second neural network NN2 shown in FIG.

The first neural network NN1 is a neural network for detecting an inspection object included in the learning image, for example, "inspection object A". The first neural network NN1 is obtained by transfer learning an object recognition neural network ORNN generated by learning in advance. The object recognition neural network ORNN uses an object detection algorithm such as SSD (Single Shot Multibox Detector) and is trained to detect general objects including "inspection object A".

Transfer learning is performed by inputting the learning image LG included in the learning data 31 to the first neural network NN1 and learning so that the first neural network NN1 detects "inspection object A".

Specifically, the detection result of the first neural network NN1 is input to the first determiner DD1. The first determiner DD1 compares the detection result of the first neural network NN1 with the first teacher data LD1 included in the learning data 31 and corresponding to the learning image LG, and determines whether or not they match. be. The first teacher data LD1 is an annotation given to the learning image LG, that is, related information. In the example shown in FIG. 2, the label "OK" or "NG" is given as the first teacher data LD1. By feeding back the determination result of the first determiner DD1 to the first neural network NN1, the weight of the first neural network NN1 is corrected. By performing such machine learning on a plurality of learning images LG, a first neural network NN1 for detecting "inspection object A" is generated.

On the other hand, the second neural network NN2 is a neural network for classifying the state of the inspection object, for example, the state of the state variable "X" of "inspection object A". The second neural network NN2 is obtained by transfer-learning the state recognition neural network SRNN generated by learning in advance. The state recognition neural network SRNN is trained to classify general states including the state of state variable "X" of "inspection object A" using a predetermined state classification algorithm.

In the transfer learning, the learning image LG included in the learning data 31 is input to the second neural network NN2, and the second neural network NN2 learns to classify the state of the state variable "X" of the "inspection object A". It is done by Alternatively, instead of inputting the learning image LG, information such as the position and number of the "inspection object A" in the learning image LG can be obtained from the detection result of the first neural network NN1. A partial image PLG of the “inspection object A” may be cut out from the image LG, and this partial image PLG may be input to the second neural network NN2. As a result, the number, position, range, etc. of the inspection objects in the learning image LG are abstracted. The classifier generator 42 that generates the classifier 33b can be easily realized without requiring the learning image LG.

The classification result of the second neural network NN2 is input to the second determiner DD2. In the second determiner DD2, the classification result of the second neural network NN2 is compared with the second teacher data LD2 included in the learning data 31 and corresponding to the learning image LG, and it is determined whether or not they match. be. The second teacher data LD2, like the first teacher data LD1, is related information attached to the learning image LG. In the example shown in FIG. ” label is attached. By feeding back the determination result of the second determiner DD2 to the second neural network NN2, the weight of the second neural network NN2 is corrected. By performing such machine learning on a plurality of learning images LG, a second neural network NN2 for classifying the state of the state variable "X" of "inspection object A" is generated.

Although the example shown in FIG. 2 shows an example in which the first neural network NN1 and the second neural network NN2 are used to generate the detector 33a and the classifier 33b, the present invention is not limited to this. For example, a support vector machine, a logistic regression, a deep neural network, or the like may be used as a machine learning model algorithm for generating the detector 33a and the classifier 33b.

(First embodiment)
In the first embodiment, a case of generating an image inspector 33 for inspecting a screw hole of an object will be described. As shown in FIG. 3, the learning image LG1 includes images of screw holes SH1, SH2, and SH3. As the inspection criteria, for example, "NG if the three screw holes are not at the predetermined positions" and "NG if the shape of the screw holes is deformed" are input and set. In this case, the first neural network NN1 shown in FIG. 2 detects screw holes SH1, SH2, SH3 included in the learning image LG1. A label of "OK" or "NG" is given as the first teacher data LD1 corresponding to the learning image LG1. A determination result obtained by comparing the detection result of the screw holes SH1, SH2, SH3 of the first neural network NN1 and the label of the first teacher data LD1 is fed back to the first neural network NN1. As a result, the detection rate of the screw holes SH1, SH2, SH3 can be increased, and the allowable range of the predetermined positions of the screw holes SH1, SH2, SH3 can be learned.

Also, the partial images of the screw holes SH1, SH2, SH3 of the learning image LG1 are input to the second neural network NN2, and the second neural network NN2 classifies the shapes of the screw holes SH1, SH2, SH3. As the second teacher data LD2 corresponding to the learning image LG1, a label of "deformation" or "￢deformation" is given. A determination result obtained by comparing the shape classification result of the screw holes SH1, SH2, SH3 of the second neural network NN2 and the label of the second teacher data LD2 is fed back to the second neural network NN2. As a result, it is possible to improve the classification accuracy of whether or not the shapes of the screw holes SH1, SH2, and SH3 are deformed, and to learn the permissible range of shape deformation of the screw holes SH1, SH2, and SH3.

(Second embodiment)
In the second embodiment, the case of generating the image inspector 33 for inspecting the expiration date printed on the object will be described. As shown in FIG. 4, the learning image LG2 includes an image of the expiration date print DA1. As inspection criteria, for example, "NG if the expiration date printing position is not at a predetermined position" and "NG if the expiration date printing character string is not "2018.10.30"" are input and set. . In this case, the first neural network NN1 shown in FIG. 2 detects the expiration date print DA1 included in the learning image LG2. A label of "OK" or "NG" is given as the first teacher data LD1 corresponding to the learning image LG2. A judgment result obtained by comparing the detection result of the expiration date print DA1 of the first neural network NN1 and the label of the first teacher data LD1 is fed back to the first neural network NN1. As a result, it is possible to distinguish, for example, water droplets that look like characters from character strings, increase the detection rate of the best-before date print DA1, and learn the print position of the best-before date print DA1 and the allowable range of the predetermined position. can.

Also, the partial image of the expiration date print DA1 of the learning image LG2 is input to the second neural network NN2, and the second neural network NN2 classifies the character string of the expiration date print DA1. A label of “2018.10.30” or “￢2018.10.30” is given as the second teacher data LD2 corresponding to the learning image LG2. A result of comparison between the character string classification result of the expiration date print DA1 of the second neural network NN2 and the label of the second teacher data LD2 is fed back to the second neural network NN2. As a result, easily confused characters such as "1" and "7" are identified, the classification accuracy or identification accuracy of the character string of the expiration date print DA1 is improved, and the character pattern of the character string of the expiration date print DA1 is learned. be able to.

(Third embodiment)
In the third embodiment, the case of generating the image inspector 33 for inspecting the coating material APP of the object will be described. As shown in FIG. 5, the learning image LG3 includes an image of the application material APP. As an inspection criterion, for example, "NG if the minimum width of the coating material APP is less than a predetermined value VA1" is input and set. In this case, the first neural network NN1 shown in FIG. 2 detects the coating material APP included in the learning image LG3. A label of "OK" or "NG" is given as the first teacher data LD1 corresponding to the learning image LG3. A determination result obtained by comparing the detection result of the coating material APP of the first neural network NN1 and the label of the first teacher data LD1 is fed back to the first neural network NN1. As a result, it is possible to prevent erroneous detection of a place other than the coating material APP or failure to detect the coating material APP, thereby increasing the detection rate of the coating material APP.

Also, the partial image of the coating material APP in the learning image LG3 is input to the second neural network NN2, and the second neural network NN2 classifies the minimum width of the coating material APP. A label of “predetermined value VA1 or more” or “less than predetermined value VA1” is given as the second teacher data LD2 corresponding to the learning image LG3. A result of comparison between the classification result or estimation result of the minimum width of the coating material APP of the second neural network NN2 and the label of the second teacher data LD2 is fed back to the second neural network NN2. This makes it possible to learn an appropriate range for the minimum width of the coating material APP.

The structures and formats of the learning data 31, the inspection standard 32, and the image inspector 33 are not limited to the examples described above. For example, the learning data 31, the inspection standard 32, and the image inspector 33 may each be simple data or a database. Further, when at least part of the learning data 31, the inspection standard 32, and the image inspection device 33 is a database, normalization may be performed to subdivide the data group unit.

Returning to the description of FIG. 1 , the image inspection apparatus 90 includes a communication section 60 , a storage section 70 , an output section 80 and a control section 85 . The image inspection apparatus 90 further includes a bus 91 configured to transmit signals and data between each section of the image inspection apparatus 90 . The image inspection device 90 includes the image inspector 33 generated by the image inspector generator 50 and is configured to inspect the inspection image 71 using the image inspector 33 .

The communication unit 60 is configured to communicate (transmit and receive) data via the network NW. More specifically, the communication unit 60 is configured to be able to communicate with the image inspector generator 50 connected to the network NW based on one or more predetermined communication methods via the network NW.

The storage unit 70 is configured to store programs, data, and the like. The storage unit 70 includes, for example, a hard disk drive, a solid state drive, and the like. The storage unit 70 stores in advance various programs executed by the control unit 85 and data necessary for executing the programs.

The storage unit 70 also stores a plurality of inspection standards 32 , a plurality of image inspectors 33 , and a plurality of inspection images 71 . The inspection standard 32 and the image inspector 33 are transmitted from the image inspector generator 50 described above. The image inspection apparatus 90 uses the communication unit 60 to receive the inspection standard 32 and the image inspector 33 from the image inspector generator 50 via the network NW, and stores the received inspection standard 32 and image inspector 33 in the storage unit 70. be memorized.

Output unit 80 is configured to output inspection criteria 32 . The output unit 80 includes, for example, an interface and an output device that is an external device. The interface is configured to exchange data and signals with an external device, and includes, for example, connection terminals for standardized interfaces such as USB, HDMI (registered trademark), and IEEE1394. The output device includes, for example, a display device such as a liquid crystal display, an EL (Electro Luminescence) display, a plasma display, etc., and an audio device such as a speaker. Among the information input from the control unit 85, the output device displays information such as text, images, and moving images such as letters, numbers, and symbols on the display device, and outputs audio information from the speaker to display the information. output is possible.

The control section 85 is configured to control the operation of each section of the image inspection apparatus 90 such as the communication section 60 , the storage section 70 and the output section 80 . Further, the control unit 85 is configured to implement an image inspection function, which will be described later, by executing a program stored in the storage unit 70 or the like. The control unit 85 includes, for example, a processor such as a CPU, memories such as ROM and RAM, and buffer storage devices such as buffers.

The control unit 85 also has an image inspection function of inspecting the inspection image 71 using the image inspector 33 . For example, when inspecting a screw hole in an object, the control unit 85 inspects the inspection image 71 using the image inspector 33 generated in the first embodiment described above. In this case, the detector 33a detects the screw holes from the inspection image 71, and the control unit 85 detects the defect if it matches the inspection standard 32, for example, "NG if the positions of the three screw holes are not at the predetermined positions." If there is a defect and if it does not match the inspection criteria 32, it is determined that there is no defect. This checks the number and position of the screw holes.

By inspecting the inspection image 71 using the image inspector 33 in this manner, the image inspection apparatus 90 with high inspection accuracy can be easily realized.

In addition, the control unit 85 classifies the shape of the screw hole by the classifier 33b, and if it matches the inspection standard 32, for example, "NG if the shape of the screw hole is deformed", there is a defect, and it does not match the inspection standard 32. If so, it is determined that there is no defect. Thereby, the presence or absence of shape deformation of the screw hole is inspected. Furthermore, when it is determined that there is a defect in the screw hole of the inspection image 71 as a result of the determination, the control unit 85 causes the output unit 80 to output the inspection standard 32 that caused the determination that there is a defect. As a result, when it is determined that there is a defect, the inspection result using the image inspector 33 is more descriptive than the case where the inspection standard 32 that is the cause of the defect is not output. In this case, for example, when there are a plurality of screw holes, the control unit 85, in addition to the inspection standard 32, displays the location of the deformed screw hole so that it can be known which screw hole is determined to be deformed. , for example, coordinate values and an image with a mark attached to the corresponding location may be output.

Next, the operation of the image inspector generating apparatus according to one embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing a schematic operation of generating the image inspector 33 of the image inspector generator 50 according to one embodiment.

For example, when generation of the image inspector 33 is selected by a user's operation, the control unit 40 of the image inspector generator 50 executes image inspector generation processing S200 shown in FIG.

In the following description, it is assumed that the learning data 31 and the inspection standard 32 are stored in advance in the storage unit 30 of the image inspector generation device 50 .

First, the detector generation unit 41 generates the detector 33a for detecting the inspection object included in the image by machine learning based on the inspection standard 32 and the learning image LG included in the learning data 31 (S201). .

Next, the classifier generator 42 generates a classifier 33b that classifies the state of the inspection object detected in step S201 by machine learning based on the inspection standard 32 and the learning image LG (S202).

Note that the teacher data corresponding to the learning image LG is not included in the learning data 31, but may also be input by the user while viewing the learning image LG displayed on a display device or the like. , and may be given to the learning image LG.

Next, the control unit 40 generates the image inspector 33 including the detector 33a generated in step S201 and the classifier 33b generated in step S202, and stores the image inspector 33 in the storage unit 30. (S203).

After step S203, the control unit 40 terminates the image inspector generation processing S200.

Note that when there are a plurality of inspection objects to be inspected, the control unit 40 repeats steps S201 to S203 the number of times corresponding to the number of inspection objects to generate the image inspector 33 for each inspection object.

Further, the image inspector 33 stored in the storage unit 30 in step S203 may be transmitted to the image inspector 90 immediately after step S203, or when a plurality of image inspectors 33 are generated, These image inspection devices 33 may be collectively transmitted to the image inspection device 90.

Exemplary embodiments of the invention have been described above. According to the image inspector generating apparatus 50, the image inspector generating program, and the image inspector generating method according to one embodiment of the present invention, the image An image inspector 33 is generated for inspecting inspection objects contained in , inspection criteria 32 containing information about the state variables of the inspection object, the state values of the state variables, and the class into which the inspection object is classified. In this way, by performing machine learning based on the inspection standard 32, which is logical information, and the learning image LG, the learning efficiency of machine learning is improved compared to the case of performing machine learning based on the learning image. It becomes possible to let Therefore, it is possible to generate an image inspection device 33 with high inspection accuracy without requiring a large amount of learning images LG as compared with the conventional art.

Each of the embodiments described above is for facilitating understanding of the present invention, and is not for limiting interpretation of the present invention. Each element provided in each embodiment and its arrangement, materials, conditions, shape, size, etc. are not limited to those illustrated and can be changed as appropriate. Also, it is possible to partially replace or combine the configurations shown in different embodiments.

(Appendix)
1. An image inspector generating unit 45 for generating an image inspector 33 for inspecting an inspection object included in an image by machine learning based on the inspection standard 33 for the inspection object and the learning image LG,
The test criteria 32 include information about the state variables of the test object, the state values of the state variables, and the class into which the test object is classified.
Image inspector generator 50 .
7. An image inspector generation program to be executed by a computer,
An image inspector generating step for generating an image inspector 33 for inspecting an inspection object included in the image by machine learning based on the inspection standard 32 for the inspection object and the learning image LG,
The test criteria 32 include information about the state variables of the test object, the state values of the state variables, and the class into which the test object is classified.
Image inspector generation program.
11. An image inspector generating step for generating an image inspector 33 for inspecting an inspection object included in the image by machine learning based on the inspection standard 32 for the inspection object and the learning image LG,
The test criteria 32 include information about the state variables of the test object, the state values of the state variables, and the class into which the test object is classified.
Image inspector generation method.

DESCRIPTION OF SYMBOLS 10... Communication part, 20... Input part, 30... Storage part, 31... Data for learning, 32... Inspection standard, 33... Image inspection device, 33a... Detector, 33b... Classifier, 40... Control part, 41... Detection Device generation unit 42 Classifier generation unit 45 Image inspection device generation unit 50 Image inspection device generation device 51 Bus 60 Communication unit 70 Storage unit 71 Inspection image 80 Output Part 85... Control part 90... Image inspection device 91... Bus 100... Image inspection system A... Inspection object APP... Coating material B... Inspection object DA1... Expiration date print DD1... First determiner , DD2 . Network, ORNN... Object recognition neural network, PLG... Partial image, S200... Image inspector generation processing, SH1, SH2, SH3... Screw hole, SRNN... State recognition neural network, VA... Value, X... State variable, X1... State Value, Y... State variable, Y1... State value.

Claims (7)

an image inspector generation unit that generates an image inspector that inspects the inspection object included in the image by machine learning based on the inspection standard for the inspection object and the learning image;
the inspection criteria includes information on the state variables of the inspection object, the state values of the state variables, and the class into which the inspection object is classified;
The image inspector includes a detector and a classifier,
The image inspector generation unit generates a detector generation unit that generates the detector that detects the inspection object included in the image, and a classifier that classifies the state of the inspection object detected in the image. and a classifier generator that
The classifier generation unit is a partial image of the inspection object detected in the image, and generates the classifier by machine learning using as input a partial image cut out from the image.
Image inspector generator. further comprising an input unit for inputting the inspection criteria;
The image inspector generating apparatus according to claim 1 . The image inspector generated by the image inspector generation device according to claim 1 or 2 ,
inspecting an inspection image using the image inspector;
Image inspection equipment. The image inspector is generated by machine learning based on a plurality of the inspection criteria,
When it is determined that the inspection image has a defect as a result of inspection using the image inspector, an output unit that outputs, among the plurality of inspection standards, the inspection standard that caused the determination that there is a defect. further comprising
The image inspection apparatus according to claim 3 . An image inspector generation program to be executed by a computer,
an image inspector generation step of generating an image inspector for inspecting the inspection object included in the image by machine learning based on the inspection standard for the inspection object and the learning image;
the inspection criteria includes information on the state variables of the inspection object, the state values of the state variables, and the class into which the inspection object is classified;
The image inspector includes a detector and a classifier,
The image inspector generation step includes a detector generation step for generating the detector for detecting the inspection object included in the image, and a classifier for classifying the state of the inspection object detected in the image. and a classifier generation step for
The classifier generation step is a partial image of the inspection object detected in the image, and includes generating the classifier by machine learning using as input a partial image cut out from the image.
Image inspector generation program. further comprising an input step for inputting the inspection criteria;
The image inspector generating program according to claim 5 . an image inspector generation step of generating an image inspector for inspecting the inspection object included in the image by machine learning based on the inspection standard for the inspection object and the learning image;
the inspection criteria includes information on the state variables of the inspection object, the state values of the state variables, and the class into which the inspection object is classified;
The image inspector includes a detector and a classifier,
The image inspector generation step includes a detector generation step for generating the detector for detecting the inspection object included in the image, and a classifier for classifying the state of the inspection object detected in the image. and a classifier generation step for
The classifier generation step is a partial image of the inspection object detected in the image, and includes generating the classifier by machine learning using as input a partial image cut out from the image.
Image inspector generation method.

Images