JP2023100280A - Inspection system and inspection method detecting presence or absence of defect on external surface of object subjected to inspection - Google Patents

Abstract

To provide a technology that can enhance the inspection capability of an inspection system that detects a defect on an external surface of an object subjected to inspection using an image analysis technology by an artificial intelligence.SOLUTION: An inspection system includes: a neural network which accepts the input of inspection image data representing an image obtained by imaging an object subjected to inspection, and which outputs the presence or absence of the defect on the object subjected to inspection on the basis of an already-learnt model prepared by machine learning in advance; and an adjusting unit that adjusts the neural network in such a way that the effect of a certain feature quantity on an output result increases using at least one certain feature quantity identified among a plurality of feature quantities on the basis of the inspection image data, and a feature quantity table created using the output by the intermediate layer of a learning neural network utilized when the already-learnt model is created.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to an inspection system and inspection method for detecting the presence or absence of defects on the outer surface of an object to be inspected.

In recent years, for example, as disclosed in Patent Document 1 below, image analysis technology by artificial intelligence (AI) is used to detect defects on the outer surface of an inspection target such as a product. Sometimes. If such AI-based image analysis technology is used, it is possible to obtain inspection results with high accuracy exceeding 98% in less than a few seconds. As a method to improve the detection accuracy of defects in image analysis technology using AI, for example, there is a method of learning the characteristics of many defects by machine learning such as deep learning, and a method of detecting more feature amounts from the image data of the inspection target. There is a method of designing a neural network as follows.

Japanese Patent Application Laid-Open No. 2019-2788

However, in the above-described conventional methods, the defect detection accuracy reaches a certain level or the defect detection time becomes long. An object of the present disclosure is to provide a technology that can improve the detection performance of an inspection system that detects defects on the outer surface of an inspection target using AI-based image analysis technology by a method different from the conventional method.

A first form for solving the above-described problems is provided as an inspection system for detecting the presence or absence of defects on the outer surface of an inspection object. The inspection system of this form receives input of inspection image data representing an image of the inspection object, and uses a learned model prepared in advance by machine learning to output the presence or absence of the defect in the inspection object. Created using a network, at least one specific feature quantity specified from among a plurality of feature quantities based on the inspection image data, and the output of the intermediate layer of the learning neural network that created the trained model. and an adjusting unit that adjusts the neural network using a feature amount table so that the specific feature amount has a greater influence on an output result.

In the inspection system of the second aspect, the adjustment unit may include a feature amount neural network that specifies the specific feature amount from among the plurality of feature amounts for the input inspection image data.

The inspection system of the third embodiment further includes an image processing unit that generates a plurality of divided image data representing a plurality of divided images obtained by dividing an image of the inspection target, wherein the neural network generates the inspection image data , the parallel input of the plurality of divided image data may be received, and the presence or absence of the defect in the inspection object may be output.

A fourth form is a neural network that receives input of inspection image data representing an image of an object to be inspected and outputs whether or not there is a defect on the outer surface of the object to be inspected using a learned model prepared in advance by machine learning. It is provided as a method of inspecting the inspection target using a network. In this method, a computer identifies at least one specific feature quantity from among a plurality of feature quantities based on the inspection image data, and a learning neural network that creates the specific feature quantity and the trained model. a feature table created using the output of the intermediate layer of and adjusting the neural network so that the specific feature has a greater influence on the output result; and inputting the inspection image data to the neural network to detect the presence or absence of scratches on the surface of the inspection object.

According to the inspection system of the first aspect, defects on the outer surface of the inspection object can be detected by the neural network dynamically adjusted in advance according to the input inspection image data. Therefore, it is possible to improve the detection accuracy of defects on the outer surface of the inspection object, shorten the detection time, and improve the inspection performance of the inspection system.

According to the inspection system of the second aspect, the adjusting section can extract a more appropriate specific feature amount according to the feature of the inspection image data by using the feature amount neural network. Therefore, the inspection performance of the inspection system can be further enhanced.

According to the inspection system of the third embodiment, even high-resolution inspection image data can be processed by the neural network in a short time.

According to the method of the fourth aspect, the weighting and layer arrangement of the neural network are dynamically adjusted according to the input inspection image data, and then the inspection image data is input to the neural network. As a result, it is possible to improve the defect detection accuracy of the inspection system that executes the method of this embodiment, shorten the time required to detect the defect, and improve the inspection performance.

1 is a schematic block diagram showing the configuration of an inspection system; FIG. A flowchart of inspection processing. FIG. 4 is an explanatory diagram schematically showing the processing contents of an inspection unit in inspection processing; FIG. 4 is an explanatory diagram schematically showing a method of generating a feature quantity table and a trained model in a trained data generation system;

Hereinafter, embodiments of an inspection system and an inspection method for detecting the presence or absence of defects on the outer surface of an object to be inspected according to the present disclosure will be described in detail with reference to the drawings.

1. Embodiment:
FIG. 1 is a schematic block diagram showing the configuration of an inspection system 100 according to this embodiment. The inspection system 100 of the present embodiment uses image analysis technology based on artificial intelligence (AI) to perform inspection processing for detecting the presence or absence of defects on the outer surface of the inspection target PI. In this embodiment, the inspection target PI of the inspection system 100 is a metal part, and the defects on the outer surface are scratches formed on the metal surface. The inspection system 100 may be able to detect flaws on the order of 10-100 μm in size, for example.

The inspection system 100 includes an image acquisition unit 10 that captures an image of an inspection target PI, generates and outputs inspection image data ID representing the captured image, and an inspection unit 20 that inspects the inspection image data ID using image analysis technology using AI. and

The image acquisition unit 10 includes an imaging unit 12 that captures an image of the inspection object PI, and an imaging execution unit 15 that controls the imaging unit 12 . The imaging unit 12 is configured by, for example, a digital camera or the like having a CCD sensor or a CMOS sensor as an imaging element. The resolution of the imaging unit 12 is preferably in the range of 10 million to 30 million pixels, and may be about 20 million pixels, for example.

The imaging execution unit 15 is configured by a computer having a central processing unit (CPU) and a main memory device (RAM). The image capturing execution unit 15 causes the image capturing unit 12 to capture an image of the inspection object PI according to a predetermined program or user's operation. In addition, the imaging execution unit 15 generates inspection image data ID representing a captured image of the inspection target PI captured by the imaging unit 12 and outputs the inspection image data ID to the inspection unit 20 .

The inspection unit 20 is configured by a computer having a CPU and a RAM. In the inspection unit 20, the functions of the functional units 21, 23, 25, and 28, which will be described below, are realized by the CPU executing the instructions and programs read into the RAM. The inspection unit 20 includes an image processing unit 21 , a neural network 23 , an adjustment unit 25 and an output unit 28 .

The image processing unit 21 acquires the inspection image data ID from the image acquisition unit 10, and inputs it to the neural network 23 and the adjustment unit 25, respectively. The image processing unit 21 inputs inspection image data ID to the neural network 23 in the form of a plurality of divided image data IDd, which will be described later.

As used herein, "neural network" is a term used in AI technology, and means a mathematical model that expresses connections between nerve cells in the human brain. The neural network 23 of this embodiment is represented by a network structure composed of multiple layers of interconnected nodes, as shown in FIG. 3, which will be referred to later. The neural network 23 is composed of an input layer to which data is input, a layer of a plurality of nodes arranged in parallel, an intermediate layer that defines the analysis process of the input data, and an output layer that outputs the analysis results of the input data. and

In this embodiment, the neural network 23 is implemented by a convolutional neural network (CNN) suitable for image analysis.

The neural network 23 receives input of the inspection image data ID from the image processing unit 21, and outputs whether or not there is a defect in the inspection object PI using a learned model TM prepared in advance by machine learning.

Based on the inspection image data ID input from the image processing unit 21 , the adjustment unit 25 identifies at least one specific feature amount from among a plurality of feature amounts that define the image analysis process in the neural network 23 . The adjustment unit 25 includes a feature amount neural network 26 for specifying specific feature amounts. The feature amount neural network 26 will be described later.

Before the inspection image data ID is input to the neural network 23, the adjustment unit 25 adjusts the neural network 23 using the above-described specific feature amount and a later-described feature amount table FVT generated by the learning data generation system 200. adjust. “Adjusting the neural network 23” means adjusting the connections between nodes in the intermediate layer of the neural network 23 and their weightings. Details of the adjustment of the neural network 23 by the adjustment unit 25 and the generation of the feature amount table FVT by the learning data generation system 200 will be described later.

The output unit 28 performs a process of outputting the defect detection result of the inspection unit 20 to the outside. In this embodiment, the output unit 28 outputs the detection result to a device such as a monitor or printer (not shown) that notifies the user of information. The output unit 28 also outputs the detection result to an external storage device (not shown) for storage.

The inspection process performed in the inspection system 100 will be described with reference to FIGS. 2 and 3. FIG. FIG. 2 is a flowchart of inspection processing. FIG. 3 is an explanatory diagram schematically showing the processing contents of the inspection unit 20 in inspection processing.

In step S10, the image acquisition unit 10 acquires inspection image data ID representing an inspection image of the inspection target PI. The image acquisition unit 10 acquires an inspection image by imaging the inspection target PI with the imaging unit 12 , and generates and acquires inspection image data ID representing the inspection image with the imaging execution unit 15 . The image acquisition section 10 outputs the inspection image data ID to the image processing section 21 of the inspection section 20 .

In step S20, the inspection unit 20 identifies the specific feature quantity based on the inspection image data ID as follows. The image processing section 21 of the inspection section 20 inputs the inspection image data ID to the adjustment section 25 . The inspection image data ID input to the adjustment unit 25 is input to the feature amount neural network 26 as it is. The feature amount neural network 26 is designed to be able to analyze the features of the input image by machine learning. The feature quantity neural network 26 analyzes the input inspection image data ID, identifies which of the plurality of feature quantities used in the neural network 23 is suitable for defect detection, and identifies the feature quantity suitable for defect detection. Print the result. In this specification, this identified feature amount is called a "specific feature amount".

At least one of a plurality of feature amounts used in the neural network 23 is specified as the specific feature amount. When there are N multiple feature amounts used in the neural network 23, n specific feature amounts are specified. N is any natural number equal to or greater than 2, and n is any natural number smaller than N (N>n). In this embodiment, N may be a natural number within the range of 50-100, and n may be a natural number within the range of 1-5.

In step S30, the adjustment unit 25 adjusts the neural network 23 using the specific feature amount and the feature amount table FVT. As described above, the feature quantity table FVT is generated in advance in the learning data generation system 200, which will be described later. The feature quantity table FVT is associated with a plurality of feature quantities used in the neural network 23 and functions corresponding to each of the plurality of feature quantities.

The adjustment unit 25 adjusts the neural network 23 so that the specific feature amount has a greater influence on the output result. As a result, in the neural network 23, the specific feature amount is weighted more heavily than the other feature amounts, and the analysis is performed.

In step S40, the image processing section 21 of the inspection section 20 generates a plurality of divided image data IDd from the inspection image data ID. The plurality of divided image data IDd represents each of the plurality of divided images obtained by dividing the image of the inspection target PI, and the plurality of divided image data IDd represents the inspection image data ID representing the image of the inspection target PI. It is one aspect of

In step S<b>50 , the image processing section 21 of the inspection section 20 inputs inspection image data ID to the neural network 23 . In this embodiment, the image processing unit 21 inputs the plurality of divided image data IDd generated in step S40 to the neural network 23 in parallel. As a result, even if the inspection image before division has a high resolution, the parallel processing in the neural network 23 can efficiently obtain the detection result in a short time.

Please refer to FIG. The input layer of the neural network 23 has the same number of nodes as the input divided image data IDd, and the divided image data IDd corresponding to each node is input. Further, the intermediate layer of the neural network 23 is composed of a plurality of layers corresponding to each of the plurality of feature quantities. Note that the coupling of each node in the intermediate layer is adjusted by the adjustment unit 25 in step S30 as described above. The output layer of the neural network 23 consists of two nodes that output a value indicating the possibility that the image represented by the input inspection image data ID contains a defect to be detected and a value indicating the possibility that the defect does not exist. It is configured.

In step S60, the output unit 28 of the inspection unit 20 outputs an inspection result as to whether or not there is a defect on the outer surface of the inspection target PI based on the output result output by the neural network 23 for the input inspection image data ID. do.

As described above, in the inspection process, the neural network 23 is adjusted in the inspection unit 20 so that the specific feature amount is weighted before the inspection image data ID is input to the neural network 23 . Therefore, for example, even if the inspection target PI is a component with metallic luster, a specific feature amount suitable for the image of such a glossy component is specified, and even if it is difficult to recognize a flaw due to the gloss, a high It becomes possible to detect flaws with high detection accuracy. In this way, if the connections between nodes in the intermediate layer of the neural network 23 are dynamically adjusted to a state suitable for the inspection image data ID, the defect detection by the neural network 23 can be made efficient, and the defect detection Accuracy can be improved. Therefore, the detection performance of defects on the outer surface of the inspection object PI by the inspection system 100 is enhanced.

FIG. 4 is an explanatory diagram schematically showing a method of generating the feature quantity table FVT and the trained model TM in the learning data generation system 200. As shown in FIG.

The learning data generation system 200 generates the learned model TM used by the inspection unit 20 and the feature amount table FVT before the inspection system 100 executes the inspection process described above. When generating the trained model TM and the feature quantity table FVT, first, a learning data set DST is prepared.

The learning data set DST includes a plurality of learning image data IDt representing images of defects to be detected by the inspection system 100, and annotation data AD that is information corresponding to the defects represented by each learning image data IDt. be Each of the plurality of learning image data IDt represents images of various patterns of defects having different shapes, positions, appearances, and the like. The annotation data AD includes, as information about defects, label information for labeling defects, coordinates of defects on an image, and the like.

Each piece of data in the learning data set DST is input to a learning neural network 201 for machine learning. In this embodiment, deep learning is applied as machine learning. As with the divided image data IDd input to the neural network 23 of the inspection unit 20, the learning image data IDt is divided into a plurality of pieces and input to the learning neural network 201 in parallel. By inputting the learning data set DST to the learning neural network 201, the trained model TM used in the inspection unit 20 is generated.

Also, the feature amount table FVT is generated using the intermediate layer output of the learning neural network 201 that generated the trained model TM. The feature quantity table FVT is composed of functions corresponding to each defined feature quantity output from the intermediate layer of the learning neural network 201 . When there are N feature quantities that define the intermediate layer of the neural network 23 of the inspection unit 20, the feature quantity table FVT represents the correspondence relationship between the N feature quantities and the N functions F ₁ to F _N. Generated as information. By using this feature amount table FVT, the inspection unit 20 can adjust the neural network 23 based on the specific feature amount as described above.

As described above, according to the inspection system 100 of the present embodiment, the connection and weighting of the nodes in the intermediate layer of the neural network 23 are performed by the specific feature amount specified according to the feature of the inspection image data ID. It is dynamically adjusted to the appropriate state for the ID. Therefore, defect detection by the neural network 23 is made more efficient, and defect detection accuracy is enhanced. Therefore, the detection performance of defects on the outer surface of the inspection object PI by the inspection system 100 is enhanced.

2. Other embodiments:
The technology of the present disclosure is not limited to the configurations of the above-described embodiments. For example, it is possible to modify as follows.

(1) In the inspection system 100 of the above-described embodiment, the presence or absence of defects on the outer surface of a product other than metal parts may be detected as the inspection object PI. For example, the PI to be inspected may be a part made of plastic or a part made of rubber, or may be a finished product instead of a part. The inspection system 100 of the above embodiment may detect defects other than scratches on the outer surface. For example, the inspection system 100 may detect distortion, dimensional errors, color unevenness, stains, etc. on the outer surface of the inspection object PI.

(2) In the inspection system 100 of the above embodiment, the image acquisition section 10 may be omitted. In this case, inspection image data ID prepared in advance outside the system may be input to the inspection unit 20 .

(3) The inspection system 100 of the above embodiment may be configured such that the inspection section 20 does not have the image processing section 21 . The inspection image data ID may be input to the neural network 23 without being divided.

(4) In the inspection system 100 of the above embodiment, the specific feature quantity may be specified by means other than the feature quantity neural network 26 . For example, the specific feature amount may be specified by a prepared map or table that outputs a specific feature amount corresponding to a parameter such as brightness derived from the inspection image data ID, or by parameter setting.

(5) In the above embodiment, the neural network 23 of the inspection unit 20 may be compatible with machine learning other than deep learning. In the inspection system 100 of the above embodiment, the number of nodes in the output layer of the neural network 23 of the inspection unit 20 may be two or more. The neural network 23 may be configured to output not only the presence/absence of a defect but also the type of defect as an output result.

(6) The configuration in which the adjustment unit 25 dynamically adjusts the neural network 23 in the above embodiment may be applied to a system other than the inspection system 100 that detects the presence or absence of defects on the outer surface of the inspection object. For example, it may be applied to a system for detecting characteristic buildings from image data of aerial photographs.

The technology of the present disclosure can be implemented in various forms other than inspection systems and inspection methods for detecting the presence or absence of defects on the outer surface of an inspection target. For example, a product production system incorporating an inspection system, an inspection device, a control unit that executes an inspection method, a program for realizing a function for executing the inspection method, a storage medium in which the program is recorded, etc. It is also possible to implement in the form of

DESCRIPTION OF SYMBOLS 10... Image acquisition part 12... Imaging part 15... Imaging execution part 20... Inspection part 21... Image processing part 23... Neural network 25... Adjustment part 26... Feature amount neural network 100... Inspection system, 200... learning data generation system, 201... learning neural network, AD... annotation data, DST... learning data set, FVT... feature amount table, ID... inspection image data, IDd... divided image data, IDt... learning image data, PI: inspection target, TM: trained model

Claims (4)

An inspection system for detecting the presence or absence of defects on the outer surface of an inspection object,
a neural network that receives an input of inspection image data representing an image of the inspection object and outputs the presence or absence of the defect in the inspection object using a trained model prepared in advance by machine learning;
A feature quantity table created using at least one specified feature quantity specified from among a plurality of feature quantities based on the inspection image data and an output of an intermediate layer of a learning neural network that created the trained model. and an adjustment unit that adjusts the neural network so that the specific feature amount has a greater influence on the output result,
An inspection system comprising: An inspection system according to claim 1,
The inspection system, wherein the adjustment unit includes a feature amount neural network that identifies the specific feature amount from among the plurality of feature amounts for the input inspection image data. The inspection system according to claim 1 or claim 2, further comprising:
An image processing unit that generates a plurality of divided image data representing a plurality of divided images obtained by dividing an image obtained by imaging the inspection object,
The inspection system, wherein the neural network receives parallel input of the plurality of divided image data as the inspection image data, and outputs whether or not the defect exists in the inspection target. The inspection is performed using a neural network that receives input of inspection image data representing an image of an inspection object and outputs whether or not there is a defect on the outer surface of the inspection object using a learned model prepared in advance by machine learning. A method of inspecting an object, comprising:
A computer identifies at least one specific feature quantity from among a plurality of feature quantities based on the inspection image data, and outputs the specific feature quantity and an intermediate layer of the learning neural network that created the trained model. A step of adjusting the neural network by using a feature amount table created using, so that the specific feature amount has a greater influence on the output result;
a computer inputting the inspection image data to the adjusted neural network to detect the presence or absence of scratches on the surface of the inspection target;
A method.

Images