JP2021096124A - Inspection system and inspection program - Google Patents

Abstract

To provide an inspection system with which an inspection speed and inspection accuracy are balanced in accordance with the nature of an inspection object or the inspection items to be carried out.SOLUTION: The inspection system executes at least one inspection item among a plurality of set inspection items on an object. The inspection system comprises: a selection unit for selecting, on the basis of a set condition, whether to execute inspection of the object using a general prelearned model having nodes as output layers that indicate inspection results respectively corresponding to the plurality of inspection items, or execute inspection of the object after selecting one or more individual prelearned models that indicate inspection results of one of the plurality of inspection items; and an inspection execution unit for executing inspection of the object by an inspection scheme selected by the selection unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to inspection systems and inspection programs.

There is known an apparatus that captures an image of a woven fabric to be inspected to be conveyed with an imaging device and detects an abnormality in the appearance of the woven fabric from the obtained image (see, for example, Patent Document 1). Recently, a trained model that inputs an image and outputs various judgment results has reached the practical stage.

Japanese Unexamined Patent Publication No. 06-102199

When inspecting multiple inspection items for an inspection target, if a trained model that outputs judgment results for multiple inspection items at once is used, the accuracy of the inspection results will drop due to reasons such as insufficient learning. There was a case. On the other hand, if the trained model that has been sufficiently learned for each test item is used individually, the accuracy of the test result is improved, but there is a problem that the time and effort required to repeat the test for each test item increases. there were.

The present invention has been made to solve such a problem, and provides an inspection system or the like that balances inspection speed and inspection accuracy according to the nature of an inspection object, inspection items to be performed, and the like. It is a thing.

The inspection system according to the first aspect of the present invention is an inspection system that executes an inspection of at least one inspection item among a plurality of set inspection items on an object, and is based on the set conditions. Perform an object inspection using a comprehensively trained model that has each node as an output layer that shows the inspection results corresponding to each of the multiple inspection items, or individual learning that shows the inspection results for any of the multiple inspection items. It includes a selection unit that selects one or more of the completed models and selects whether to execute the inspection of the object, and an inspection execution unit that executes the inspection of the object by the inspection method selected by the selection unit.

The inspection program according to the second aspect of the present invention is an inspection program that executes an inspection of at least one inspection item among a plurality of set inspection items on an object, and is based on the set conditions. Perform an object inspection using a comprehensively trained model that has each node as an output layer that shows the inspection results corresponding to each of the multiple inspection items, or individual learning that shows the inspection results for any of the multiple inspection items. Have the computer perform a selection step that selects one or more of the completed models and selects whether to perform an inspection of the object, and an inspection execution step that executes the inspection of the object by the inspection method selected in the selection step. ..

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an inspection system or the like that balances inspection speed and inspection accuracy according to the nature of an inspection target, inspection items to be performed, and the like.

It is a conceptual diagram which shows the whole structure including the inspection system which concerns on this embodiment. It is a block diagram which shows the whole structure of an inspection system. It is a conceptual diagram of a comprehensively trained model. It is a conceptual diagram of an individually trained model. It is a flow chart which shows the inspection process of the inspection object.

Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Moreover, not all of the configurations described in the embodiments are indispensable as means for solving the problem.

FIG. 1 is a conceptual diagram showing an overall configuration including the inspection system 10 according to the present embodiment. The inspection system 10 inspects the cloth 300, which is the object to be inspected, for a plurality of inspection items related to the appearance abnormality. The inspection system 10 mainly includes a system server 100, a storage unit 110, a monitor 120, a camera unit 130, and a take-up motor 140.

The cloth 300 is transferred by the rotation of the take-up motor 140, and is rewound from one take-up shaft to the other take-up shaft. The take-up motor 140 is provided on each take-up shaft and rotates according to a command from the system server 100. In the figure, as shown by the arrow, the state of rewinding from the left winding shaft to the right winding shaft is shown, but when the winding motor 140 is rotated in the reverse direction, it winds from the right winding shaft to the left winding shaft. Can be replaced.

The camera unit 130 images a part of the surface of the transferred cloth 300 as shown by the dotted line. The camera unit 130 transmits the captured image signal to the system server 100. In addition, in order to obtain a clear image, the cloth 300 may be irradiated with irradiation light. Depending on the nature of the cloth 300 and the like, the irradiation device may be installed so as to irradiate the cloth 300 from the camera unit 130 side, or may be installed so as to irradiate the cloth 300 from the opposite side. For example, when it is desired to reduce the influence of the pattern drawn on the cloth 300, it is advisable to irradiate from the opposite side.

The system server 100 sequentially processes the continuously acquired images and executes an inspection for an abnormality on the surface of the cloth 300. In the present embodiment, the presence or absence of an abnormality is inspected by using a trained model which is a neural network stored in the storage unit 110. The system server 100 displays the inspection result on the monitor 120. By observing the monitor 120, the inspection worker can confirm the presence or absence of an appearance abnormality that may exist in the cloth 300.

FIG. 2 is a configuration diagram showing the overall configuration of the inspection system 10. The system server 100 includes a processor (CPU) 101 that performs control and program execution processing, and the CPU 101 functions as a selection unit 102 as a functional calculation unit that executes various operations according to a process instructed by an inspection program. Or, it functions as an inspection execution unit 103.

The storage unit 110 is a non-volatile storage medium, and is composed of, for example, a large-capacity HDD. The storage unit 110 stores a program that executes control and processing of the system server 100, and also stores various look-up tables related to inspection objects, image data acquired by the camera unit 130, and the like. In the present embodiment, the storage unit 110 stores the integrated trained model 111, which is a trained neural network, and a plurality of individually trained models 112, which are also trained neural networks.

The comprehensively trained model 111 has an input layer for inputting image data from the camera unit 130 and an output layer composed of a group of nodes showing inspection results corresponding to each of a plurality of inspection items. That is, the integrated trained model 111 is a single trained model that outputs the results of each of the set inspection items in parallel. Each of the individually trained models 112 has an input layer for inputting image data from the camera unit 130 and an output layer composed of nodes showing inspection results corresponding to any of a plurality of inspection items. That is, the individually trained model 112 is a set of separate individually trained models 112a, 112b, 112c ... That output results for each inspection item. The storage unit 110 does not have to be composed of a single piece of hardware. For example, a storage medium for storing a program or the like and a storage medium for storing a learned model may be composed of separate hardware.

The monitor 120 includes, for example, a liquid crystal panel, and displays an image in which an abnormality is detected or displays an interface screen. The input device 121 includes, for example, a keyboard, and is used by an inspection worker to set conditions related to an inspection target and give instructions to an inspection system. The camera unit 130 starts imaging in response to an instruction from the CPU 101, converts an image of the cloth 300 to be inspected into an image signal, and periodically transmits the image to the system server 100. The take-up motor 140 rewinds the cloth 300 from one take-up shaft to the other take-up shaft by rotating forward or reverse in response to an instruction from the system server 100.

When the CPU 101 functions as the selection unit 102, the inspection of the cloth 300 is performed using the comprehensively trained model 111 based on the conditions set via the input device or the like, or one of the individually trained models 112. Select the above to select whether to perform the inspection of the cloth 300. When the CPU 101 functions as the inspection execution unit 103, the inspection of the cloth 300 is executed by the inspection method selected by the selection unit 102. The specific processing will be described later.

FIG. 3 is a conceptual diagram of the comprehensively trained model 111. The integrated trained model 111 is, for example, a convolutional neural network (CNN) trained by supervised learning. The frame image data obtained by pre-processing the image signal sent from the camera unit 130 for the input image is input to the input layer. The output layer is provided with nodes corresponding to each of the set inspection items. Here, “with holes”, “with scratches”, and “with stains”, which are abnormal appearances, are set as inspection items, and each of them is associated with the node a, the node b, and the node c.

The comprehensively trained model 111 outputs the probability that a region having a hole exists in the image of the input image data to the node a. Similarly, the probability that a scratched area exists is output to the node b, and the probability that a dirty area exists is output to the node c. The inspection execution unit 103 acquires these output values, and if each of them exceeds the threshold value, determines that there are "holes", "scratches", and "dirt". For example, if 0.8 is set as the threshold value and the output of node a is 0.87, it is determined that there is a hole. The threshold value for each node is adjusted according to the degree of learning and the like.

The comprehensively trained model 111 is created by learning a large amount of teacher image data to which the correct answer is associated in advance. Specifically, as for the teacher image data, 1 is associated with node a if the image is an image of a piece of cloth with a hole, and if the image is an image of a piece of cloth without a hole, it is attached to node a. 0 is associated. Similarly, in the case of image data obtained by capturing an image of a piece of cloth with scratches, 1 is associated with node b, and in the case of an image of an image of a piece of cloth without scratches, 0 is associated with node b. Further, in the case of image data obtained by capturing an image of a dirty cloth, 1 is associated with the node c, and in the case of an image obtained by capturing an image of a clean cloth, 0 is associated with the node c. For example, the correct answers of node a = 1, node b = 0, and node c = 1 are linked in advance to the image data obtained by imaging a piece of cloth with holes, no scratches, and dirt. The edge weights are adjusted by, for example, a back propagation method so that the error between the output and the associated correct answer is reduced by inputting such teacher image data. The comprehensively trained model 111 stored in the storage unit 110 is created by learning a large amount of such teacher image data.

FIG. 4 is a conceptual diagram of the individually trained model 112. Each of the individually trained models 112, like the integrated trained model 111, is, for example, a convolutional neural network (CNN) trained by supervised learning.

The individually trained model A112a is a trained model that determines the inspection item “with holes”. In the individually trained model A112a, when the frame image data obtained by pre-processing the image signal sent from the camera unit 130 for the input image is input to the input layer, a hole is formed in the image of the input image data. The probability that the existing area exists is output to the output node. The output layer of the individually trained model A112a has a single output node. The inspection execution unit 103 acquires the output value of the output node, and if it exceeds the threshold value, determines that there is a “hole”. For example, if 0.8 is set as the threshold value and the output of the output node is 0.92, it is determined that there is a hole.

The individually trained model A112a is created by learning a large amount of teacher image data to which the correct answer is associated in advance. Specifically, in the teacher image data, 1 is associated as an output node if the image is an image of a piece of cloth with a hole, and 0 is a string if the image is an image of a piece of cloth without a hole. It is attached. The edge weights are adjusted by, for example, a back propagation method so that the error between the output and the associated correct answer is reduced by inputting such teacher image data. The individually trained model A112a stored in the storage unit 110 is created by learning a large amount of such teacher image data.

The individually trained model B112b and the individually trained model C112c are also the same as the individually trained model A112a. That is, the individually trained model B112b is a trained model for determining the inspection item "scratched", and the individually trained model C112c is a trained model for determining the inspection item "dirty". In the individually trained model B112b, when the frame image data obtained by pre-processing the image signal sent from the camera unit 130 for the input image is input to the input layer, there is a scratch in the image of the input image data. Output the probability that the area exists to the output node. Similarly, the individually trained model C112c outputs the probability that a dirty region exists to the output node. The inspection execution unit 103 acquires the output value of the output node, and if it exceeds the threshold value, determines that it is “scratched” or “dirty”.

The individually trained model B112b and the individually trained model C112c are also created by learning a large amount of teacher image data to which the correct answer is associated in advance, similarly to the individually trained model A112a. The individually trained model B112b and the individually trained model C112c stored in the storage unit 110 are created by learning in this way.

The selection unit 102 executes the inspection of the cloth 300 using the comprehensively trained model 111 based on the conditions set for the cloth 300 to be inspected, or selects one or more of the individually trained models 112. Then, select whether to perform the inspection of the cloth 300. Explains how to properly inspect a plurality of inspection items set by using the comprehensively trained model 111 and when inspecting several inspection items separately by using one or more of the individually trained models 112. To do.

As described above, the comprehensively trained model 111 outputs the inspection results of a plurality of set inspection items at once. Therefore, in the learning stage, the cloth is imaged for each of a large number of combination conditions in which there is an appearance abnormality corresponding to each of the inspection items or there is no appearance abnormality, and the correct answer is associated with them to prepare the teacher image data. There is a need to. There is not only one type of cloth, but there are various kinds such as plain, patterned, patterned, lace, and thick cloth, so it is necessary to prepare teacher image data for each cloth. Therefore, the number of teacher image data that can be learned tends to decrease when viewed for each inspection item and each type of cloth. Further, even when the number of teacher image data that can be learned is large, since different types of cloths are collectively learned, the edge weight may not be adjusted appropriately. That is, while the comprehensively trained model 111 is highly versatile, the accuracy of the test result may not be sufficient.

Each of the individually trained models 112 outputs an inspection result of one of a plurality of set inspection items. Therefore, it is only necessary to learn about one of the appearance abnormalities, and it is expected that the accuracy of the inspection result will be improved. Further, for a special type of cloth such as a cloth on which a pattern is drawn, a dedicated individually trained model can be created by specially learning the cloth of that type. For example, an individually trained model for determining the stain on a piece of cloth on which a pattern is drawn may be created. While the individually trained model 112 can be expected to have high inspection accuracy for each inspection item, it is necessary to switch and inspect the individually trained model for each inspection item, so that is the case when many inspection items are executed. It takes time.

Therefore, the selection unit 102 confirms the conditions set for the cloth 300 to be inspected, and selects an inspection method suitable for the conditions. For example, when the cloth 300 is set as a general cloth that is plain, or the inspection speed is emphasized, the selection unit 102 uses the comprehensively trained model 111 to inspect the cloth 300. Choose to do. Further, when the cloth 300 is set as a special cloth having a pattern or the inspection accuracy is emphasized, the selection unit 102 uses one or more of the individually trained models 112 to be used as the cloth. Choose to perform 300 tests.

If a special fabric is set and an individually trained model trained for such a special fabric exists in the storage unit 110, the selection unit 102 reads out the individually trained model. You can also specify to use it. Further, when the inspection items to be inspected are specifically specified, the selection unit 102 selects to inspect the cloth 300 using each of the individually trained models 112 corresponding to them. .. In other words, do not select the individually trained model that corresponds to the unspecified inspection item. Further, when the cloth 300 is set as the lace fabric, the selection unit 102 is an individually trained model for hole inspection in order to prevent the gap between the threads of the lace fabric from being determined as a hole. May be excluded and the individually trained model corresponding to other inspection items may be selected.

Next, a series of inspection processes of the inspection system 10 will be described. FIG. 5 is a flow chart showing an inspection process of the cloth 300, which is an inspection target. The flow starts from the time when the inspection worker gives an instruction to start the inspection after inputting the information about the cloth 300 and the information about the inspection instruction through the input device 121 or the like.

In step S101, the selection unit 102 acquires preset setting conditions for the cloth 300 to be inspected. The setting conditions include, for example, as described above, conditions relating to the properties of the cloth 300 and the inspection policy. In step S102, the selection unit 102 determines whether to execute the inspection using the comprehensively trained model 111 or to select one or more of the individually trained models 112 and execute the inspection according to the acquired setting conditions. select. When the selection unit 102 selects any of the inspection methods, the selection unit 102 delivers the selection result to the inspection execution unit 103. The inspection execution unit 103 reads out the comprehensively trained model 111 or the individually trained model 112 from the storage unit 110 according to the received selection result. When reading out the individually trained model 112, one of the target inspection items corresponding to the inspection item to be inspected is read out.

The inspection execution unit 103 starts the inspection of the cloth 300 in step S103. Specifically, the winding motor 140 is started to be driven, and the image pickup by the camera unit 130 is started. In step S104, the inspection execution unit 103 acquires the image data generated from the image signal sent from the camera unit 130. Then, the acquired image data is input to the trained model read in step S103, and the inspection result is determined from the output (step S106). If the determination is "no abnormality", the process proceeds to step S107, and if the determination is "abnormal", the process proceeds to step S112.

First, the processing when it is determined that there is no abnormality will be described. When the process proceeds to step S107, the inspection execution unit 103 determines whether or not all the cloth 300 to be inspected has been wound up. If the winding is not completed, the process returns to step S104, and the inspection of the cloth 300 is continued using the image signal further sent from the camera unit 130. If the winding is completed, the process proceeds to step S108 to stop the rotational drive of the winding motor 140.

After stopping the rotary drive, in step S109, it is confirmed whether or not the inspections of all the scheduled inspection items have been completed. Specifically, if it is selected in step S102 to execute the inspection using the comprehensively trained model 111, the inspection of all the inspection items is completed by one winding operation, so the process proceeds to step S110. On the other hand, if it is selected in step S102 to select one or more of the individually trained models 112 and execute the inspection, it is confirmed whether or not there are still inspection items for which the inspection has not been executed. If it does not remain, the process proceeds to step S110, and if it remains, the process proceeds to step S111.

If the process proceeds to step S111, the inspection execution unit 103 sets the rotation direction of the take-up motor 140 in the opposite direction, and returns to step S102. After returning to step S102, the inspection execution unit 103 reads out one individually learned model corresponding to the inspection item to be inspected next from the storage unit 110, and starts the inspection of the inspection item.

When the process proceeds to step S110, the inspection execution unit 103 displays on the monitor 120 that all the inspection items are normal, and ends a series of inspection processes.

Next, the process when it is determined that “there is an abnormality” in step S106 will be described. Proceeding to step S112, the inspection execution unit 103 stops the rotational drive of the take-up motor 140. Then, in step S113, the inspection item in which the abnormality is detected and the image of the image data used for the determination are displayed on the monitor 120, and a series of inspection processes are completed. Since the winding of the cloth 300 is stopped immediately after the abnormality is detected, the inspection worker can check not only the image displayed on the monitor 120 but also the actual state of the cloth 300. The image to be displayed on the monitor 120 may be subjected to image processing such as highlighting an abnormal portion or masking a normal portion. Further, when the inspection worker gives an instruction to restart the inspection, a processing flow for restarting the inspection may be adopted.

In the present embodiment described above, there is an option of executing the inspection using the comprehensively trained model 111 or selecting one or more of the individually trained models 112 and executing the inspection. Depending on the conditions given, after performing the test using the comprehensively trained model 111, one or more of the individually trained models 112 may be selected and the test may be performed. For example, when it is desired to know the inspection result accurately for a specific item, the inspection may be performed using the comprehensively trained model 111 and then re-inspected using the individually trained model corresponding to the one item.

Further, in the present embodiment described above, the inspection target is the cloth 300, but the inspection target is not limited to this. The inspection target is preferably a sheet-like or film-like material such as a cloth, which can be imaged while being wound up, but is not limited to this, and may be a material such as a building material. In this way, when the material is to be inspected, the image data on the surface of the material may be acquired while moving the camera unit.

Further, in the present embodiment described above, the presence / absence of holes, the presence / absence of scratches, and the presence / absence of stains are examples of inspection items, but in addition to or in place of these, another inspection item shall be adopted. You can also. Further, as an example of the condition for selecting the individually trained model, the case where the pattern is drawn on the surface of the cloth 300 which is the object is described, but it seems that the pattern is drawn not limited to the pattern. The case may be a condition. The same applies when the object is a film-like material.

10 inspection system, 100 system server, 101 CPU, 102 selection unit, 103 inspection execution unit, 110 storage unit, 111 comprehensively trained model, 112 individually trained model, 120 monitor, 121 input device, 130 camera unit, 140 winding Motor, 300 pieces

Claims (5)

It is an inspection system that executes inspection of at least one inspection item out of a plurality of set inspection items on an object.
Based on the set conditions, the inspection of the object is executed using the comprehensively trained model having each node showing the inspection result corresponding to each of the plurality of inspection items as an output layer, or the plurality of inspections are performed. A selection unit that selects one or more of the individually trained models showing the inspection results for any of the items and selects whether to perform the inspection of the object.
An inspection system including an inspection execution unit that executes an inspection of the object by an inspection method selected by the selection unit. The object is a film or sheet
The inspection system according to claim 1, wherein the comprehensively trained model and the individually trained model are input with image data of an image captured by capturing the film or the sheet. The inspection system according to claim 2, wherein the plurality of inspection items include the presence / absence of holes, the presence / absence of scratches, and the presence / absence of stains. The inspection system according to claim 2 or 3, wherein the condition includes selecting an individually trained model when a pattern or pattern is present on the surface of the film or sheet which is the object. It is an inspection program that executes the inspection of at least one inspection item out of a plurality of set inspection items on the object.
Based on the set conditions, the inspection of the object is executed using the comprehensively trained model having each node showing the inspection result corresponding to each of the plurality of inspection items as an output layer, or the plurality of inspections are performed. A selection step that selects one or more of the individually trained models that show the inspection results for any of the items and selects whether to perform the inspection of the object.
An inspection program that causes a computer to execute an inspection execution step that executes an inspection of the object by the inspection method selected in the selection step.

Images