JP6742037B1 - Learning model generation method, learning model, inspection device, abnormality detection method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning model generation method, a learning model, an inspection device, an abnormality detection method, and a computer program capable of accurately detecting an abnormality of an inspection object. SOLUTION: A plurality of inspection objects 4 are irradiated with electromagnetic waves, and an inspection image corresponding to one inspection object 4 extracted from an image acquired according to the electromagnetic waves transmitted through each inspection object 4 is acquired. And teacher data including abnormality presence/absence information regarding the presence/absence of abnormality of the inspection object 4, irradiating electromagnetic waves to a plurality of inspection objects 4 based on the teacher data, and in accordance with the electromagnetic waves transmitted through each inspection object 4. When the inspection image corresponding to one inspection object 4 extracted from the acquired image is input, the learning model 152 that outputs the abnormality presence/absence information of the inspection object 4 is generated. [Selection diagram] Figure 2

Description

The present invention relates to a learning model generation method, a learning model, an inspection device, an abnormality detection method, and a computer program for detecting an abnormality of an inspection object such as food or a part.

X-rays, terahertz waves, infrared rays, visible light, etc. for inspection of foods such as meat and beans, inspection of packages containing contents in packaging materials, inspection of industrial products such as screws, golf balls and electronic parts The inspection device using the inspection wave of is used. In this inspection apparatus, the inspection wave is irradiated to the inspection object, the inspection wave passing through the inspection object is detected, and the inspection image of the inspection object is acquired.

When the inspection object has a foreign substance, the inspection image has a dark portion exceeding the threshold value. Conventionally, the presence or absence of foreign matter has been determined based on the area and shape of this dark portion.
The presence or absence of abnormality in the shape of the inspection object was determined based on the area and perimeter of the blob (lump) in the inspection image. By counting the pixels forming the blob, the area of the blob is calculated, and the perimeter of the blob is calculated based on the arrangement of the background pixels forming the periphery of the blob (for example, Patent Document 1).

JP, 2005-31069, A

In the conventional abnormality detection method, the presence or absence of abnormality was determined by comparing with a threshold value, but depending on the type of the inspection object and the foreign matter, the difference in pixel value between the normal portion and the foreign matter may be small, There was a problem that the detection accuracy was not good.
Similarly, when detecting an abnormality in the shape of the inspection object, the difference between the pixel values of the blob pixel and the background pixel is small, the edge is not clear, and it may not be accurately detected.

The present invention has been made in view of such circumstances, and provides a learning model generation method, a learning model, an inspection device, an abnormality detection method, and a computer program that can accurately detect an abnormality of an inspection object. With the goal.

A method for generating a learning model according to an aspect of the present invention is to irradiate electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and of images acquired according to the electromagnetic waves that have passed through each inspection object . Among them, based on the pixel value of each pixel , identify the blob corresponding to one inspection object , cut out so that the identified blob is within a rectangle of a predetermined size, to obtain an inspection image, the inspection image And teacher data including abnormality presence/absence information regarding the presence/absence of abnormality of the inspected object are obtained, and based on the instructor data, a plurality of inspected objects are irradiated with electromagnetic waves, and the electromagnetic waves transmitted through each inspected object are acquired. If an inspection image corresponding to one inspection object , which is obtained by identifying the blob from the image and cutting the blob into a rectangle of a predetermined size, is input, the inspection object is abnormal. A learning model that outputs presence information is generated.

The learning model according to one aspect of the present invention radiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of the pixel, the blob corresponding to one inspection object is specified, and the inspection image corresponding to the one inspection object that is obtained by cutting out so that the blob falls within a rectangle of a predetermined size is input. An input layer, an output layer that outputs abnormality presence/absence information indicating the presence or absence of abnormality of the inspection object, and irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and each inspection object The blob is specified from among the images acquired according to the electromagnetic waves transmitted through , and the blob is cut out and acquired so as to be within a rectangle of a predetermined size, and the inspection image and the inspection object corresponding to one inspection object Equipped with an intermediate layer whose parameters are learned based on abnormality information, irradiate electromagnetic waves to a plurality of inspection objects distributed and placed on a conveyor and conveyed , and acquired according to the electromagnetic waves transmitted through each inspection object. When the inspection image corresponding to one inspection object is input, which is obtained by identifying the blob among the images and cutting out so that the blob falls within a rectangle of a predetermined size, the calculation by the intermediate layer The computer is caused to output the abnormality presence/absence information of the inspection object from the output layer.

An inspection apparatus according to one aspect of the present invention, an irradiation unit that irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed, and an image acquired according to the electromagnetic waves that have passed through each inspection object . Among them, based on the pixel value of each pixel, identify the blob corresponding to one inspection object, cut out so that the blob is within a rectangle of a predetermined size , the inspection image corresponding to one inspection object An acquisition unit to acquire and identify the blob among the images acquired according to the electromagnetic waves that have passed through the inspection object , and acquire the one inspection object by cutting out so that the blob falls within a rectangle of a predetermined size. Learning inspection unit that inputs the inspection image acquired by the acquisition unit to a learning model that outputs abnormality presence/absence information indicating the presence/absence of abnormality of the inspection object when a corresponding inspection image is input, and acquires the abnormality presence/absence information And an output unit that outputs the abnormality presence/absence information.

The abnormality detection method according to an aspect of the present invention, irradiating electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of each pixel, a blob corresponding to one inspection object is specified, and the blob is cut out so as to be within a rectangle of a predetermined size, and an inspection image corresponding to the one inspection object is acquired. , Identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object , and cutting and acquiring the blob so that it falls within a rectangle of a predetermined size , an inspection image corresponding to one inspection object A computer is made to perform the process which inputs the acquired said inspection image into the learning model which outputs the abnormality presence/absence information which shows the presence or absence of abnormality of a test|inspection when it inputs.

A computer program according to an aspect of the present invention irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of the pixel, identify the blob corresponding to one inspection object, cut out so that the blob is within a rectangle of a predetermined size , to obtain an inspection image corresponding to one inspection object, Input the inspection image corresponding to one inspection object acquired by identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object and cutting out so that the blob falls within a rectangle of a predetermined size In this case, the computer is made to execute the process of inputting the acquired inspection image to a learning model that outputs abnormality presence/absence information indicating the presence/absence of abnormality of the inspection object and outputting the abnormality presence/absence information.

According to the present invention, even when the difference in pixel value between the normal portion and the abnormal portion is small in the inspection image of the inspection object, the normal portion and the abnormal portion can be accurately discriminated.
Therefore, it is possible to accurately detect abnormality presence/absence information such as a shape abnormality and the presence/absence of foreign matter.

FIG. 3 is a perspective view showing the configuration of the abnormality detection system according to the first embodiment. It is a block diagram which shows the structure of an abnormality detection system. It is explanatory drawing which shows the state which enclosed the blob by the bounding box. It is explanatory drawing of the process of resizing. It is explanatory drawing which shows an example of the record layout of inspection image DB. It is explanatory drawing regarding the production|generation process of an abnormality detection model. 6 is a flowchart showing an example of a processing procedure of an abnormality detection model generation processing by a control unit. 6 is a flowchart illustrating an example of a processing procedure of abnormality detection processing by a control unit. It is explanatory drawing which shows an example of the display screen in a display part. It is a flow chart which shows an example of the processing procedure of the above-mentioned re-learning processing by the control part of PC. FIG. 9 is an explanatory diagram related to an abnormality detection model generation process according to the second embodiment. It is explanatory drawing which shows an example of the display screen in a display part. 7 is a flowchart illustrating an example of a processing procedure of switching processing of an abnormality type by a control unit. It is explanatory drawing which shows an example of the display screen which displays the button which shows the state of a test object. FIG. 9 is a block diagram showing a configuration of an abnormality detection system according to a third embodiment. 6 is a flowchart illustrating an example of a processing procedure of abnormality detection processing by a control unit. It is a perspective view which shows the structure of the abnormality detection system which concerns on Embodiment 4. It is a block diagram which shows the structure of an abnormality detection system. It is explanatory drawing regarding an abnormality detection process.

Hereinafter, the present invention will be described in detail with reference to the drawings showing an embodiment thereof.
(Embodiment 1)
1 is a perspective view showing a configuration of an abnormality detection system 10 according to the first embodiment, and FIG. 2 is a block diagram showing a configuration of the abnormality detection system 10.
The abnormality detection system 10 includes an information processing device 1 as an inspection device and an X-ray inspection machine 2. In the present embodiment, the inspection object 4 is placed in a dispersed state on the conveyor belt 61 of the conveyor section (conveyor) 6 of the X-ray inspection machine 2, and an image of the inspection object 4 is captured using X-rays. An abnormality detection system 10 for detecting an abnormality based on will be described. Examples of the inspection object 4 include foods such as seeds and soybeans. The inspection object 4 is not limited to food, and may be a screw, a golf ball or the like as long as the shape can be detected by an image. The inspection object 4 is not limited to being placed in a dispersed state on the conveyor belt 61, and may be stored in a packaging material or placed on a tray. The information processing device 1 and the X-ray inspection machine 2 may be integrated.

The X-ray inspector 2 includes a horizontally-long rectangular parallelepiped lower housing 23 and an upper housing 21 provided on the lower housing 23 and smaller than the lower housing 23.
The transport unit 6 is provided in the lower housing 23. The transport unit 6 is provided with an upstream roller 63 on the outer side of one end in the longitudinal direction of the lower housing 23 and a downstream roller 62 on the outer side of the other end. Between the upstream roller 63 and the downstream roller 62, two lower rollers 64 and 65 are provided below the upstream roller 63 and the downstream roller 62, and the conveyor belt 61 is provided between these rollers. Has been bridged. A driving force from a conveyance motor (not shown) is applied to one of the upstream roller 63 and the downstream roller 62, and when the inspection object is inspected, the conveyance belt 61 rotates clockwise at a constant speed V. Orbit.

An X-ray irradiation unit (irradiation unit) 3 is housed in the upper housing 21. The X-ray irradiation unit 3 has an X-ray tube 31. A display 22 is provided on the front surface of the upper housing 21. The display 22 is a display device such as an LCD (Liquid Crystal Display).
The X-ray detection unit 7 is provided below the transport belt 61 that moves between the upstream roller 63 and the downstream roller 62. The X-ray detection unit 7 is called a TDI (Time Delay Integration) camera or a TDI sensor, and includes a scintillator that emits fluorescence according to the intensity of X-rays, and many photodiodes that detect the fluorescence emitted by the scintillator. have.

In the TDI camera, the photodiodes are linearly arranged in the X direction orthogonal to the Y direction, which is the moving direction of the object to be inspected, to form one detection line 71, and a plurality of detection lines are provided. The plurality of detection lines 71 are arranged in parallel in the Y direction, which is the moving direction of the inspection object. In FIG. 1, each detection line 71 is shown as an independent line sensor element, but in reality, a plurality of photodiodes are regularly arranged in the X direction and the Y direction in the housing of one TDI camera. One line of the detection line 71 is configured by the photodiodes arranged side by side and arranged in the X direction.

The information processing device 1 is an information processing device capable of various information processing and information transmission/reception, and is, for example, a personal computer, a server device, or the like. The X-ray inspection machine 2 is communicatively connected to the information processing apparatus 1 via a network N such as a LAN. When the information processing device 1 is a server device, the network N may be the Internet. Alternatively, a cloud computer communicatively connected via the network N such as the Internet may execute the process of the information processing device 1.
In the present embodiment, the information processing device 1 is assumed to be a personal computer, and will be read as PC1 for simplicity in the following description.

The PC 1 acquires an image of the inspection object 4 and performs a process of detecting the presence or absence of abnormality of the inspection object 4 based on the image. In the present embodiment, the PC 1 detects the presence or absence of an abnormality using an abnormality detection model 152, which will be described later and has been learned to detect (identify) the abnormality of the inspection object 4 from the image by machine learning. The abnormality detection model 152 is expected to be used as a program module that is a part of artificial intelligence software.
The PC 1 inputs an image (inspection image) based on an image obtained by imaging the inspection object 4 using the X-ray inspection machine 2 into the abnormality detection model 152, and acquires an identification result indicating whether or not there is an abnormality in the inspection object 4 as an output. .. The control unit 11 of the PC 1 operates so as to perform an operation on the inspection image input to the input layer according to a command from the abnormality detection model 152 and output the identification result.
Hereinafter, the type of abnormality of the inspection object 4 is a shape abnormality, the PC 1 inputs the inspection image to the abnormality detection model 152, and outputs the probability value of normality and the probability value of abnormality by the softmax function. The case of acquiring as an abnormal value will be described.
The PC 1 sorts out whether or not to discard the inspection object 4 according to the acquired abnormal value.

The PC 1 includes a control unit 11, a main storage unit 12, a communication unit 13, a display unit 14, and an auxiliary storage unit 15.
The control unit 11 has an arithmetic processing unit such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), and GPUs (Graphics Processing Units), and stores the programs P stored in the auxiliary storage unit 15. By reading and executing, various information processing, control processing, and the like related to the PC 1 are performed. Each functional unit in FIG. 2 is executed by the control unit 11 operating based on the program P.

The control unit 11 has, as functional units, an image generating unit (generating unit) 111, an image processing unit (generating unit) 112, a target specifying unit (extracting unit) 113, a cutout unit (enclosing unit) 114, an image processing unit 115, and learning. The inspection unit 116, the display unit (output unit) 117, the selection unit (output unit) 118, and the re-learning unit 119 are included.
The image generation unit 111 transmits an inspection object 4 and generates an X-ray image based on the X-ray detected by the X-ray detection unit 7.
The image processing unit 112 performs image processing on the X-ray image using a smoothing filter, a feature extraction filter, etc., and generates a processed image.
The target specifying unit 113 performs blob analysis of the inspection object 4 based on the processed image and specifies a blob.
The cutout unit 114 cuts out the blob so as to fit inside a rectangle of a predetermined size, and generates a cutout image.
The image processing unit 115 performs processing such as normalization on the cutout image to generate an inspection image.

The learning inspection unit 116 inputs the inspection image to the abnormality detection model 152 and acquires the abnormal value.

The display unit 117 outputs the abnormal value to the display 22 and displays the presence/absence of abnormality of the inspection image on the display 22.
The sorting unit 118 compares the abnormal value of the inspection object 4 with a threshold value, performs sorting of whether or not to discard, stops the transport unit 6, or marks the inspection object 4. For example, when the abnormal value of the inspection object 4 is 0.5 or more, the selection unit 118 regards the inspection object 4 as an abnormal product and selects it when it is discarded. The threshold value of the abnormal value is not limited to 0.5.
The re-learning unit 119 re-learns the abnormality detection model 152 by using the teacher data in which the inspection image and the determination of the quality of the abnormality detection model 152 input by the operator are associated with each other.

The main storage unit 12 is a temporary storage area such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc., and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. Remember. The communication unit 13 is a communication module for performing processing relating to communication, and transmits/receives information to/from the outside.

The auxiliary storage unit 15 is a large-capacity memory, a hard disk, or the like, and stores the program P and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 15 also stores an inspection image DB 151 and the abnormality detection model 152. The inspection image DB 151 is a database that stores information about the inspection object 4 that is an abnormality detection target.

The program P stored in the auxiliary storage unit 15 may be provided by the recording medium 153 in which the program P is readably recorded. The recording medium 153 is, for example, a portable memory such as a USB (Universal Serial Bus) memory, an SD (Secure Digital) card, a micro SD card, or a compact flash (registered trademark). The program P recorded in the recording medium 153 is read from the recording medium 153 using a reading device (not shown) and installed in the auxiliary storage unit 15. When the information processing device 1 includes a communication unit that can communicate with an external communication device, the program P stored in the auxiliary storage unit 15 may be provided by communication via the communication unit.

The auxiliary storage unit 15 may be an external storage device connected to the PC 1. Further, the PC 1 may be a multi-computer including a plurality of computers, or may be a virtual machine virtually constructed by software.

FIG. 3 is an explanatory diagram showing a state in which the blob specified by the target specifying unit 113 is cut out by the cutout unit 114, the blob is surrounded by the bounding box 41, and the cutout image (inspection image) 40 is generated. The target specifying unit 113 of the control unit 11 detects a blob by pattern matching, edge detection, R-CNN (Regions with CNN features), or the like. In each inspection image 40, each inspection object 4 is surrounded by a bounding box 41 having the same size.
FIG. 4 is an explanatory diagram of the resizing process performed by the clipping unit 114.
When the inspection object 4 protrudes from the bounding box 41, the cutout portion 114 of the control unit 11 resizes the inspection object 4 so that the inspection object 4 fits inside the bounding box 41.

FIG. 5 is an explanatory diagram showing an example of the record layout of the inspection image DB 151. The inspection image DB 151 includes an inspection image ID sequence, a coordinate information sequence, and an abnormal value sequence. The inspection image ID column stores inspection image IDs for identifying each inspection image. The coordinate information sequence stores coordinate information indicating the placement position of the inspection object 4 in the X-ray irradiation area on the conveyor belt 61 in association with the inspection image ID. The coordinate information is the X coordinate and the Y coordinate at the four corners of the bounding box 41 when the center in the X direction and the Y direction between the upstream roller 63 and the downstream roller 62 in FIG. 1 is the origin. The abnormal value is the probability value of abnormality as described above.

FIG. 6 is an explanatory diagram related to generation processing of the abnormality detection model 152. FIG. 6 conceptually illustrates a process of performing machine learning to generate the abnormality detection model 152.

The PC 1 uses the inspection image 40 as an input by learning the image feature amount of the abnormality in the inspection image of the inspection object 4 as the abnormality detection model 152, and outputs the information indicating whether or not the inspection object 4 is abnormal. Build (generate) a network. For example, the neural network is a CNN (Convolution Neural Network), and an input layer that receives an input of the inspection image 40, an output layer that outputs a result of identifying whether there is an abnormality, and an intermediate layer that extracts the image feature amount of the inspection image 40. Have.

The input layer has a plurality of neurons that receive the input of the pixel value of each pixel included in the inspection image 40, and transfers the input pixels to the intermediate layer. The intermediate layer has a plurality of neurons for extracting the image feature amount of the inspection image, and transfers the extracted image feature amount to the output layer. In FIG. 6, the number of intermediate layers is three, but the number is not limited to this. For example, when the abnormality detection model 152 is CNN, the convolution layer that convolves the pixel value of each pixel input from the input layer and the pooling layer that maps the pixel value that is convoluted by the convolution layer alternate in the intermediate layer. The feature amount of the image is finally extracted while compressing the pixel information of the inspection image 40. In FIG. 6, the description of the convolution layer and the pooling layer is omitted. The output layer has two neurons that output a discrimination result for discriminating the abnormality of the inspection object 4, and discriminates the presence or absence of abnormality of the inspection object 4 based on the image feature amount output from the intermediate layer. One neuron outputs the probability value of the normal (no foreign substance) of the inspection object 4, and the other neuron outputs the probability value of the abnormal (external substance) of the inspection object 4.

In the present embodiment, the abnormality detection model 152 is described as being CNN, but the abnormality detection model 152 is not limited to CNN, and a neural network other than CNN, SVM (Support Vector Machine), Bayesian network, regression tree It may be a trained model constructed by another learning algorithm.

The PC 1 includes teacher data (abnormality) in which a plurality of inspection images 40 are associated with information indicating whether or not the inspection object 4 is abnormal in each inspection image 40 (whether or not the foreign matter is normal or the foreign matter is normal). Learning is performed using data and normal data. For example, as shown in FIG. 6, with respect to the inspection image 40 of the inspection object 4, the teacher data includes the ID of the inspection image 40, the type of abnormality (abnormal shape), and the rank (1: abnormal, 0: normal). It is labeled data.

The PC 1 inputs the inspection image 40, which is the teacher data, to the input layer, performs the arithmetic processing on the intermediate layer, and obtains the identification result indicating the presence or absence of abnormality of the inspection object 4 from the output layer. The identification result output from the output layer is a continuous probability value (for example, a value in the range from “0” to “1”) when the softmax function is used. The identification result may be a value (for example, a value of “0” or “1”) that discretely indicates the presence or absence of abnormality.

The PC 1 compares the identification result output from the output layer with the information labeled on the inspection image 40 in the teacher data, that is, the correct value, and the intermediate layer so that the output value from the output layer approaches the correct value. Optimize the parameters used in the calculation process in. The parameters are, for example, weights between neurons (coupling coefficient), coefficients of an activation function used in each neuron, and the like. The method of optimizing the parameters is not particularly limited, but for example, the PC 1 optimizes various parameters using the error back propagation method.

The PC 1 performs the above processing on each inspection image 40 included in the teacher data to generate the abnormality detection model 152. When the inspection image 40 of the inspection object 4 is acquired, the PC 1 uses the abnormality detection model 152 to detect the presence/absence of foreign matter in the inspection object 4.

FIG. 7 is a flowchart showing an example of a processing procedure of generation processing of the abnormality detection model 152 by the control unit 11.
The control unit 11 acquires the teacher data in which the inspection images 40 of the plurality of inspection objects 4 are associated with the information indicating the abnormality of the inspection object 4 in each inspection image 40 (S11).

The control unit 11 uses the teacher data to generate the abnormality detection model 152 (learned model) that outputs the abnormality presence/absence information of the inspection object 4 when the inspection image 40 of the inspection object 4 is input (step S12). Specifically, the control unit 11 inputs the inspection image 40, which is the teacher data, to the input layer of the neural network, and acquires the identification result for identifying the presence/absence of a morphology in the inspection image 40 from the output layer. The control unit 11 compares the acquired identification result with the correct value of the teacher data (information labeled on the inspection image 40), and in the middle layer, the identification result output from the output layer approaches the correct value. The parameters (weight, etc.) used in the calculation processing of are optimized. The control unit 11 stores the generated abnormality detection model 152 in the auxiliary storage unit 15 and ends the series of processes.

FIG. 8 is a flowchart showing an example of the processing procedure of the abnormality detection processing by the control unit 11.
The control unit 11 of the PC 1 transmits an inspection object 4 and generates an X-ray image based on the X-ray detected by the X-ray detection unit 7 (S21).
The control unit 11 performs image processing on the X-ray image using a smoothing filter, a feature extraction filter, etc., and generates a processed image (S22).
The control unit 11 performs blob analysis of the inspection object 4 based on the processed image and identifies the blob (S23).
The control unit 11 cuts out the blob so as to fit in a rectangle of a predetermined size, and generates a cutout image (S24).
The control unit 11 performs processing such as normalization on the cutout image to generate an inspection image (S25).
The control unit 11 inputs the inspection image to the abnormality detection model 152, outputs the normal probability value and the abnormal probability value, and acquires the abnormal value (S26).
The control unit 11 outputs the abnormal value to the display 22 and displays the inspection image 40 of each inspection object 4 on the display 22 (S27). On the display 22, the control unit 11 displays the abnormal inspection object 4 having an abnormal value equal to or larger than the threshold value in a state of being surrounded by the bounding box 42.
The control unit 11 selects whether or not to discard based on the abnormal value of the inspection object 4 and the inspection image ID (S28), and ends the process.

As described above, according to the present embodiment, in the inspection image 40 of the inspection object 4, even when the difference in pixel value between the normal portion and the abnormal portion is small, the normal portion and the abnormal portion can be accurately discriminated. it can.
Therefore, it is possible to accurately detect abnormality presence/absence information such as a shape abnormality and the presence/absence of foreign matter.
It is not necessary for the operator to visually detect the abnormality, and the efficiency of the abnormality detection work can be improved.

FIG. 9 is an explanatory diagram showing an example of the display screen 5 on the display 22.
The display screen has a first screen 51 on the left side and a second screen 52 on the right side in FIG. 9.
In the first screen 51, the control unit 11 associates the positions (the coordinate information) of the plurality of inspection objects 4 dispersedly placed on the conveyor belt 61 with each of the plurality of blue bounding boxes 41. The inspection image 40 of is displayed. On the second screen 52, the control unit 11 displays the inspection images 40 side by side in the order of the inspection image IDs, and the abnormal inspection image 40 is surrounded by the red bounding box 42. The operator can select one of the inspection images 40 on the second screen 52 by the selection button 56 composed of “>” and “<” for moving in the row direction of the inspection image 40. The inspection image 40 may be selected by a pointing device such as a mouse. When the operator selects one inspection image 40, the control unit 11 displays the corresponding inspection image 40 on the first screen 51 in a state of being surrounded by a yellow bounding box.
The second screen 52 is also pressed by an OK button (operation button) 54 that is pressed when the operator determines that the estimation result (judgment) of the abnormality detection model 152 is correct, and is pressed when it is determined that the judgment is not correct. It has an NG button (operation button) 53 and an Other button 55 for inputting other instructions such as brightness adjustment.

For the inspection image 40 selected by the selection button 56, when the operator accepts the input of the judgment of the control unit 11 by the OK button 53 or the NG button 54 and collects a predetermined number or more of data, the control unit 11 It is possible to re-learn the abnormality detection model 152 using the inspection image 40 and the quality of the judgment input by the operator as teacher data.

FIG. 10 is a flowchart showing an example of a processing procedure of the above-mentioned re-learning processing by the control unit 11 of the PC 1.

The control unit 11 accepts the determination of whether the determination of the control unit 11 is good or not based on the operator pressing the OK button 53 or the NG button 54 of the second screen 52 for the selected inspection image 40 (S31). When the control unit 11 determines that the quality determination is not accepted (S31: NO), the determination process is repeated.
When the control unit 11 determines that the acceptance/rejection determination is accepted based on the predetermined number of inspection images 40 (S31: YES), the inspection image 40 of the inspection object 4 in which the abnormality is detected, and the control unit 11 input by the operator. The teacher data that is associated with the quality of the determination is acquired (S32).
The control unit 11 uses the teacher data to generate the abnormality detection model 152 that outputs the abnormality presence/absence information of the inspection object 4 when the inspection image of the inspection object 4 is input (S33).
By the above, re-learning is performed. As a result, the abnormality detection model 152 is updated as the abnormality detection processing is continued, and more accurate abnormality detection can be performed.

(Embodiment 2)
The PC 1 according to the second embodiment performs the same processing as the PC according to the first embodiment, except that the abnormality detection model 152 detects the presence or absence of a foreign matter in addition to the abnormality in the shape of the inspection object 4.
FIG. 11 is an explanatory diagram regarding the generation processing of the abnormality detection model 152. The same parts as those in FIG. 6 are designated by the same reference numerals and detailed description thereof will be omitted.

The PC 1 receives the inspection image 40 as an input by learning the image feature amount of the abnormality in the inspection image 40 of the inspection object 4 as the abnormality detection model 152, and indicates the abnormality of the shape of the inspection object 4 and the presence/absence of foreign matter. A neural network that outputs information is generated. The neural network is a CNN, and has an input layer that receives the input of the inspection image 40, an output layer that outputs the identification result of the presence or absence of abnormality, and an intermediate layer that extracts the image feature amount of the inspection image 40.

The input layer has a plurality of neurons that receive the input of the pixel value of each pixel included in the inspection image 40, and transfers the input pixels to the intermediate layer. The intermediate layer has a plurality of neurons for extracting the image feature amount of the inspection image, and transfers the extracted image feature amount to the output layer. In FIG. 11, the number of intermediate layers is three, but the number is not limited to this. For example, when the abnormality detection model 152 is CNN, the convolution layer that convolves the pixel value of each pixel input from the input layer and the pooling layer that maps the pixel value that is convoluted by the convolution layer alternate in the intermediate layer. The feature amount of the image is finally extracted while compressing the pixel information of the inspection image 40. In FIG. 11, the description of the convolution layer and the pooling layer is omitted. The output layer has three neurons that output the discrimination result for discriminating the abnormality of the inspection object 4, and determines whether the shape of the inspection object 4 is abnormal or not based on the image feature amount output from the intermediate layer. Identify. The first neuron outputs the probability value that the test object 4 is normal, the second neuron outputs the probability value that the shape of the test object 4 is abnormal, and the third neuron outputs the probability value that the test object 4 has a foreign substance. Is output. The number of abnormalities may be three or more.

The PC 1 stores teacher data in which a plurality of inspection images 40 are associated with information indicating whether the inspection object 4 is abnormal in each inspection image 40 (whether the shape is abnormal, has foreign matter, or is normal). Use to learn. For example, as shown in FIG. 11, with respect to the inspection image 40 of the inspection object 4, the teacher data includes the ID of the inspection image 40, the type of abnormality (abnormal shape), and the rank (1: abnormal, 0: normal). It is labeled data.
The PC 1 inputs the inspection image 40, which is the teacher data, to the input layer, performs the arithmetic processing on the intermediate layer, and obtains the identification result indicating the presence or absence of abnormality of the inspection object 4 from the output layer. The identification result output from the output layer is a continuous probability value (for example, a value in the range from “0” to “1”) when the softmax function is used.

The PC 1 compares the identification result output from the output layer with the information labeled on the inspection image 40 in the teacher data, that is, the correct value, and the intermediate layer so that the output value from the output layer approaches the correct value. Optimize the parameters used in the calculation process in. The parameters are, for example, weights between neurons (coupling coefficient), coefficients of an activation function used in each neuron, and the like. The method of optimizing the parameters is not particularly limited, but for example, the PC 1 optimizes various parameters using the error back propagation method.

The PC 1 performs the above processing on each inspection image 40 included in the teacher data to generate the abnormality detection model 152. When the inspection image 40 of the inspection object 4 is acquired, the PC 1 uses the abnormality detection model 152 to detect the presence/absence of abnormality of the inspection object 4.

The learning inspection unit 116 inputs the inspection image 40 into the abnormality detection model 152, outputs the above three probability values, and outputs the probability value of abnormal shape (abnormal value A) and the probability value of having foreign matter (abnormal value B). ) To get.

FIG. 12 is an explanatory diagram showing an example of the display screen 5 on the display 22 according to the second embodiment. The same parts as those in FIG. 9 are designated by the same reference numerals, and detailed description thereof will be omitted.
In the second embodiment, the second screen 52 has a Change button (second operation button) 57 instead of the Other button 55.
When the operator presses the Change button 57, the type of abnormality is switched between “abnormal shape” and “presence/absence of foreign matter”. When the type of abnormality is “shape abnormality”, the inspection image 40 having the shape abnormality is surrounded by the red bounding box 42 on the first screen 51 and the second screen 52. When the type of abnormality is switched to “presence or absence of foreign matter”, the inspection image 40 having foreign matter is surrounded by the green bounding box 42 on the first screen 51 and the second screen 52.

FIG. 13 is a flowchart showing an example of a processing procedure of the abnormality type switching processing by the control unit 11.
The control unit 11 determines whether or not the switching of the type of abnormality has been accepted (S41). First, it is assumed that the abnormality type is “shape abnormality”, the inspection image 40 based on the shape abnormality is displayed on the display 22, and the abnormality type is switched to “presence/absence of foreign matter”. When the control unit 11 determines that the switching of the abnormality type is not accepted (S41: NO), the determination process is repeated.

When the control unit 11 determines that the switching of the type of abnormality has been received (S41: YES), the control unit 11 switches and displays the inspection image 40 indicating the presence or absence of the foreign matter on the display 22 (S42). On the second screen of the display 22, the control unit 11 displays the abnormal inspection object 4 in which the abnormal value B relating to the presence or absence of a foreign substance is a threshold value or more, for example, 0.5 or more, surrounded by the green bounding box 42. To do.

The control unit 11 selects the abnormal inspection object 4 based on the abnormal value B and the inspection image ID (S43).

As described above, the abnormality can be detected by switching to the required abnormality type.
When the type of abnormality is selected from “abnormality of shape” or “presence/absence of foreign matter”, the operator selects the OK image 53 on the second screen 52 for the inspection image 40 selected by the operator in the same manner as above. Based on the pressing of the NG button 54, it is possible to accept the input of the quality of the judgment of the control unit 11.
When the NG button 54 is pressed, buttons 58, 59, 60 indicating "normal", "abnormal shape", and "foreign matter" are displayed on the first screen 51 as shown in FIG. The operator selects the button corresponding to the actual content of the inspection image 40.
The PC 1 can re-learn the abnormality detection model 152 using the inspection image 40 and the quality of the judgment input by the operator as teacher data.

(Embodiment 3)
FIG. 15 is a block diagram showing the configuration of the abnormality detection system 20 according to the third embodiment.
The abnormality detection system 20 according to the third embodiment has the same configuration as the abnormality detection system 10 according to the first embodiment, except that the control unit 11 has an abnormality determination unit (second acquisition unit) 120 as a functional unit. Have. The abnormality determination unit 120 calculates an abnormal value of the shape of the inspection object 4 based on the inspection image 40 generated by the image processing unit 115.
In the abnormality detection system 20 according to the present embodiment, the abnormality is detected using the abnormality detection model 152 according to the abnormality value calculated by the abnormality determination unit 120.

FIG. 16 is a flowchart showing an example of a processing procedure of abnormality detection processing by the control unit 11.
The control unit 11 of the PC 1 transmits the inspection object 4 and generates an X-ray image based on the X-ray detected by the X-ray detection unit 7 (S51).
The control unit 11 performs image processing on the X-ray image using a smoothing filter, a feature extraction filter, etc., and generates a processed image (S52).
The control unit 11 performs blob analysis of the inspection object 4 based on the processed image and identifies the blob (S53).
The control unit 11 cuts out the blob so as to fit in a rectangle of a predetermined size, and generates a cutout image (S54).
The control unit 11 performs processing such as normalization on the cutout image to generate an inspection image (S55).

The control unit 11 calculates an abnormal value for the shape of the inspection object 4 based on the pixel value and the like without using the abnormality detection model 152. The closer the abnormal value is to 1, the more abnormal, and the closer to 0, the more normal (S56). For example, it is assumed that the pixel values of the pixels forming the blob (lump) of the inspection image 40 are larger than the reference value and the pixel values of the pixels forming the background of the blob are smaller than the reference value. The area of the blob is calculated by counting the pixels that make up the blob. The perimeter of the blob is calculated based on the array of background pixels that form the perimeter of the blob. The area and the perimeter are compared with a threshold value, and the control unit 11 calculates an abnormal value of the shape by setting 0 when it is completely normal and 1 when it is completely abnormal.

The control unit 11 determines whether the calculated abnormal value is 0 or 1 (S57).
When determining that the calculated abnormal value is 0 or 1 (S57: YES), the control unit 11 determines whether the abnormal value is 0 (S58). When the control unit 11 determines that the abnormal value is 0, that is, when there is no abnormality (S58: YES), the learning test is skipped and the process ends.
When the control unit 11 determines that the abnormal value is not 0 (S58: NO), that is, when the abnormal value is 1 and it is determined that there is an abnormality, the learning test is skipped and the process proceeds to S60.

When determining that the calculated abnormal value is not 0 or 1 (S57: NO), the control unit 11 uses the abnormality detection model 152 to acquire the abnormal value of the inspection object 4 from the acquired inspection image 40 (S59). ..

The control unit 11 selects the abnormal inspection object 4 based on the abnormal value and the inspection image ID (S60).

In the present embodiment, an abnormality is detected in the inspection image 40 after the image processing based on the pixel value or the like, and when the obtained result is uncertain, the abnormality is further detected using the abnormality detection model 152. Therefore, the abnormality can be detected more accurately.
The PC 1 may be configured to generate the inspection image 40 in the image processing unit 115 based on a plurality of image processing algorithms, and may generate the abnormality detection model 152 corresponding to each image processing algorithm. Then, the abnormality determination unit 120 may detect an abnormal value based on the inspection image 40 that has been subjected to each process, and the abnormality may be detected by the abnormality detection model 152 corresponding to the image processing algorithm.

(Embodiment 4)
The X-ray inspection machine 2 according to the fourth embodiment has the same configuration as the X-ray inspection machine 2 according to the second embodiment except that the X-ray inspection machine 2 according to the fourth embodiment has a structure for discarding the inspection object 4 that is determined to be an abnormal product. Have.
17 is a perspective view showing the configuration of the abnormality detection system 30 according to the fourth embodiment, and FIG. 18 is a block diagram showing the configuration of the abnormality detection system 30. 17 and 18, those parts which are the same as those corresponding parts in FIGS. 1 and 2 are designated by the same reference numerals, and a detailed description thereof will be omitted.
In the X-ray inspecting machine 2 according to the present embodiment, a discarding unit 9 having a plurality of air valves 91 in the X direction is provided at the downstream end of the lower housing 23 so as to face the conveyor belt 61. There is. The air valves 91 are sequentially numbered from the end in the X direction.
The control unit 11 includes an opening/closing unit 121 that opens/closes the air valve 91.
The selection unit (second output unit) 118 selects an abnormal product having an abnormal shape or foreign matter, and opens/closes the number of the air valve 91 corresponding to the mounting position of the inspection product 4 as coordinate information of the inspection product 4. It is output to the section 121. The opening/closing part 121 opens the air valve 91 of the corresponding number. As a result, air is ejected toward the inspection object 4 from the corresponding air valve 91 with respect to the inspection object 4 selected as an abnormal product, and the inspection object 4 is repelled from the transport section 6.

FIG. 19 is an explanatory diagram of the abnormality detection process according to the fourth embodiment. FIG. 19 conceptually shows a case where the presence or absence of an abnormality in the shape of the inspection object 4 is detected from the inspection image.
The control unit 11 acquires the inspection image 40 based on the image of the inspection object 4 captured by using the X-ray irradiation unit 3, and inputs the inspection image 40 to the abnormality detection model 152. For example, the control unit 11 acquires the inspection image 40 and also acquires the number of the air valve 91 corresponding to the mounting position of the inspection object 4 as the coordinate information of the inspection object 4.

The control unit 11 performs arithmetic processing to extract the feature amount of the inspection image 40 in the intermediate layer of the abnormality detection model 152, inputs the extracted feature amount to the abnormality detection model 152, and outputs the normal probability value of the inspection object 4. , The probability value of the shape abnormality and the probability value of the presence/absence of foreign matter are output, and the abnormal value A and the abnormal value B are acquired.

As described above, the control unit 11 uses the abnormality detection model 152 to detect an abnormality that requires the inspection object 4 to be discarded. The control unit 11 performs learning by using the inspection image 40 as teacher data and performs abnormality detection using the abnormality detection model 152 that has learned the abnormality of the inspection object 4. The PC 1 may have a different abnormality detection model 152 (neural network) built for each type of abnormality, or a single abnormality detection model 152 may learn all the abnormalities.

The control unit 11 outputs the abnormal value A and the abnormal value B to the display 22. When detecting various abnormalities from the inspection image 40, the PC 1 may notify the worker or notify a predetermined mobile terminal carried by the worker.

For example, the control unit 11 transmits, as the information to be notified, the number of the air valve 91 of the inspection object 4 in which the abnormality is detected, in addition to the detection result indicating the type and degree of the detected abnormality in the inspection object 4.

The abnormal value A or the abnormal value B is compared with the threshold value, the abnormal value A or the abnormal value B is equal to or larger than the threshold value, and the inspection object 4 determined to have the abnormality corresponds to the lower side in FIG. Air is ejected from the air valve 91 and discharged from the conveyor belt 61.
Instead of acquiring the number of the air valve 91 as the coordinate information, the X coordinate and the Y coordinate of the conveyor belt 61 may be acquired and the abnormal inspection object 4 may be discarded. For example, in the case of acquiring the X coordinate and the Y coordinate corresponding to the pixel, the inspection items 4 to be discarded can be narrowed down and the product yield can be increased as compared with the case of acquiring the number of the air valve 91.

As described above, the learning model generation method according to one aspect of the present invention irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed, and responds to the electromagnetic waves that have passed through each inspection object. Based on the pixel value of each pixel in the acquired image , identify the blob corresponding to one inspected object , cut out the identified blob so that it falls within a rectangle of a predetermined size, and acquire the inspected image. Then, the teacher data including the inspection image and abnormality presence/absence information regarding the presence/absence of abnormality of the inspection object is acquired, and based on the teacher data, the plurality of inspection objects are irradiated with electromagnetic waves, and the electromagnetic waves transmitted through each inspection object. Among the images acquired according to , the blob is specified, the blob is cut out and acquired so as to be within a rectangle of a predetermined size, when an inspection image corresponding to one inspection object is input, A learning model that outputs abnormality information about the inspection object is generated.

By using the learning model, the normal portion and the abnormal portion can be accurately discriminated even when the difference in pixel value between the normal portion and the abnormal portion is small in the inspection image of the inspection object.
Therefore, it is possible to accurately detect abnormality presence/absence information such as a shape abnormality and the presence/absence of foreign matter.

In the learning model generation method described above, the electromagnetic waves are X-rays, the blobs are specified in an image acquired according to the X-rays , and the blobs are cut out so as to be within a rectangle of a predetermined size. generates an inspection image corresponding to one object of inspection, to display the bounding box containing the inspection image generated may display abnormal presence information of the inspection object included in the bounding box.

According to the above configuration, the feature amount of the image is easily extracted, and the inspection image and the abnormality presence/absence information are displayed, so that it is easy to acquire the correctness information of the detection result by the operator.

The above-described learning model generation method may acquire an inspection image including an inspection object and abnormality information of the inspection object, and re-learn the learning model based on the inspection image and the abnormality information.

According to the above configuration, the learning model is updated, and more accurate abnormality detection can be performed.

The learning model according to one aspect of the present invention radiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of the pixel, the blob corresponding to one inspection object is specified, and the inspection image corresponding to the one inspection object that is obtained by cutting out so that the blob falls within a rectangle of a predetermined size is input. An input layer, an output layer that outputs abnormality presence/absence information indicating the presence or absence of abnormality of the inspection object, and irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and each inspection object The blob is specified from among the images acquired according to the electromagnetic waves transmitted through , and the blob is cut out and acquired so as to be within a rectangle of a predetermined size, and the inspection image and the inspection object corresponding to one inspection object Equipped with an intermediate layer whose parameters are learned based on abnormality information, irradiate electromagnetic waves to a plurality of inspection objects distributed and placed on a conveyor and conveyed , and acquired according to the electromagnetic waves transmitted through each inspection object. When the inspection image corresponding to one inspection object is input, which is obtained by identifying the blob among the images and cutting out so that the blob falls within a rectangle of a predetermined size, the calculation by the intermediate layer The computer is caused to output the abnormality presence/absence information of the inspection object from the output layer.

By using the learning model, the normal portion and the abnormal portion can be accurately discriminated even when the difference in pixel value between the normal portion and the abnormal portion is small in the inspection image of the inspection object.
Therefore, it is possible to accurately detect abnormality presence/absence information such as a shape abnormality and the presence/absence of foreign matter.

An inspection apparatus according to one aspect of the present invention, an irradiation unit that irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed, and an image acquired according to the electromagnetic waves that have passed through each inspection object . Among them, based on the pixel value of each pixel, identify the blob corresponding to one inspection object, cut out so that the blob is within a rectangle of a predetermined size , the inspection image corresponding to one inspection object An acquisition unit to acquire and identify the blob among the images acquired according to the electromagnetic waves that have passed through the inspection object , and acquire the one inspection object by cutting out so that the blob falls within a rectangle of a predetermined size. Learning inspection unit that inputs the inspection image acquired by the acquisition unit to a learning model that outputs abnormality presence/absence information indicating the presence/absence of abnormality of the inspection object when a corresponding inspection image is input, and acquires the abnormality presence/absence information And an output unit that outputs the abnormality presence/absence information.

According to the above configuration, in the inspection image of the inspection object, the normal portion and the abnormal portion can be accurately discriminated even when the difference in pixel value between the normal portion and the abnormal portion is small.
Therefore, it is possible to accurately detect abnormality presence/absence information such as a shape abnormality and the presence/absence of foreign matter.
It is not necessary for the operator to visually detect the abnormality, and the efficiency of the abnormality detection work can be improved.

In the above-described inspection apparatus, the electromagnetic waves are X-rays, and the acquisition unit includes a generation unit that generates an image based on the X-rays, an extraction unit that extracts the blob from the generated image, and a bounding box for the blob. It is also possible to acquire the inspection image surrounded by the bounding box.

According to the above configuration, it becomes easy to extract the feature amount of the image.

The above-described inspection apparatus includes a display, the output unit displays a plurality of inspection images each surrounded by a bounding box in association with the positions of the plurality of inspection objects, and an abnormal image including an abnormal inspection object is displayed. The first screen further surrounded by a second bounding box different from the bounding box and each image are displayed side by side, and the abnormal image is surrounded by the second bounding box, and each inspection by the operator is performed. It may be a display unit that displays the second screen having an operation button for receiving the input of the abnormality information of the object side by side on the display.

According to the above configuration, by displaying the inspection image and the abnormality presence/absence information, it is possible to acquire correctness/incorrectness information of the detection result by the operator.

The inspection device may include a re-learning unit that re-learns the learning model based on the inspection image and the abnormality presence/absence information received by the operation button.

According to the above configuration, the learning model is updated, and more accurate abnormality detection can be performed.

In the inspection apparatus described above, the second screen includes a second operation button that receives an input of the type of abnormality of the inspection object, and when the second operation button receives the input, the display unit displays the input. The abnormal image may be surrounded by the second bounding box and displayed based on the type.

According to the above configuration, the type of abnormality can be switched to detect the abnormality.

The inspection device described above is a discarding unit that discards an inspection object for which the abnormality inspection information acquired by the learning inspection unit is a threshold value or more, and an inspection image for an inspection object for which the abnormality inspection information acquired by the learning inspection unit is a threshold value or more. And a second output unit that outputs coordinate information corresponding to.

According to the above configuration, it is possible to satisfactorily discard only the inspection item having an abnormality.

The above-mentioned inspection device has a plurality of valves arranged in parallel in a direction intersecting with the conveying direction of the conveyor, and a discarding unit that blows out gas from the valve corresponding to the abnormal inspection object and blows off the inspection object. A second output unit that outputs information on a valve corresponding to the inspection object for which the abnormality inspection information acquired by the learning inspection unit is a threshold value or more; and an opening/closing unit that opens/closes the valve corresponding to the inspection object. Good.

According to the above configuration, only the inspection item having an abnormality can be easily discarded.

The inspection apparatus described above applies the image processing algorithm to the image acquired according to the electromagnetic wave to perform the image processing to identify the blob, and cuts out the blob so that it falls within a rectangle of a predetermined size. And a second acquisition unit that acquires information indicating whether there is an abnormality in the inspection object based on the acquired inspection image corresponding to the inspection object, and the learning inspection unit includes the second acquisition unit. You may input the said inspection image according to the said information which the part acquired.

According to the above configuration, after the abnormality is detected on the image after the image processing based on the pixel value and the like, the abnormality is further detected by using the learning model, so that the abnormality can be detected more accurately.

The abnormality detection method according to an aspect of the present invention, irradiating electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of each pixel, a blob corresponding to one inspection object is specified, and the blob is cut out so as to be within a rectangle of a predetermined size, and an inspection image corresponding to the one inspection object is acquired. , Identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object , and cutting and acquiring the blob so that it falls within a rectangle of a predetermined size , an inspection image corresponding to one inspection object A computer is made to perform the process which inputs the acquired said inspection image into the learning model which outputs the abnormality presence/absence information which shows the presence or absence of abnormality of a test|inspection when it inputs.

A computer program according to an aspect of the present invention irradiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed , and among the images acquired according to the electromagnetic waves that have passed through each inspection object , Based on the pixel value of the pixel, identify the blob corresponding to one inspection object, cut out so that the blob is within a rectangle of a predetermined size , to obtain an inspection image corresponding to one inspection object, Input the inspection image corresponding to one inspection object acquired by identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object and cutting out so that the blob falls within a rectangle of a predetermined size In this case, the computer is made to execute the processing of inputting the acquired inspection image to a learning model that outputs abnormality presence/absence information indicating the presence/absence of abnormality of the inspection object and outputting the abnormality presence/absence information.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above meaning but by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.
For example, the inspection item is not limited to food, and may be a package, an industrial product, or the like.
Further, the discarding unit is not limited to the one including the air valve 66.
The electromagnetic waves with which the inspection object 4 is irradiated are not limited to X-rays, and may be terahertz waves, infrared rays, visible light, or the like.

1 PC (information processing device)
11 Control Section 12 Main Storage Section 13 Communication Section 14 Display Section 15 Auxiliary Storage Section P Program 151 Inspection Image DB
152 Abnormality detection model 2 X-ray inspection machine 22 Display 3 X-ray irradiation unit 4 Inspection object 40 Inspection image 41, 42 Bounding box 5 Display screen 6 Conveying unit 61 Conveying belt 66 Air valve 7 X-ray detecting unit 71 Detection line

Claims (14)

Irradiate electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and transported ,
Among the images acquired according to the electromagnetic waves transmitted through each inspection object , based on the pixel value of each pixel , identify the blob corresponding to one inspection object ,
Cut out the specified blob so that it falls within a rectangle of a predetermined size , acquire an inspection image,
Acquiring teacher data including the inspection image and abnormality presence/absence information regarding presence/absence of abnormality of the inspection object,
Based on the teacher data, a plurality of inspection objects are irradiated with electromagnetic waves, and among the images acquired according to the electromagnetic waves that have passed through each inspection object , the blob is specified, and the blob is within a rectangle of a predetermined size. A learning model generation method for generating a learning model that outputs abnormality information of the inspection object when an inspection image corresponding to one inspection object that is cut out so as to be acquired is input. The electromagnetic waves are X-rays,
The blob is specified from among the images acquired according to the X-rays, the blob is cut out so as to be within a rectangle of a predetermined size, and an inspection image corresponding to one inspection object is generated,
Display the bounding box containing the generated inspection image,
The method for generating a learning model according to claim 1, wherein information on presence/absence of abnormality of an inspection object included in the bounding box is displayed. Acquires the inspection image including the inspection object and the abnormality information of the inspection object,
The learning model generation method according to claim 1, wherein the learning model is re-learned based on the inspection image and the abnormality presence/absence information. Irradiate electromagnetic waves to a plurality of inspected objects that are distributed and placed on a conveyor and conveyed, and in the image acquired according to the electromagnetic waves transmitted through each inspected object , based on the pixel value of each pixel, one inspection An input layer to which an inspection image corresponding to one inspection object is input, which is obtained by specifying a blob corresponding to the object and cutting out so that the blob falls within a rectangle of a predetermined size ,
An output layer for outputting abnormality presence/absence information indicating presence/absence of abnormality of the inspection object,
Irradiate electromagnetic waves to a plurality of inspected objects that are distributed and placed on a conveyor and specify the blobs in the image acquired according to the electromagnetic waves that have passed through each inspected object , and the blobs have a predetermined size. And an intermediate layer in which the parameters are learned based on the inspection image corresponding to one inspection object and the abnormality information of the inspection object , which is obtained by cutting out so as to fall within the rectangle of
Irradiate electromagnetic waves to a plurality of inspected objects that are distributed and placed on a conveyor and specify the blobs in the image acquired according to the electromagnetic waves that have passed through each inspected object , and the blobs have a predetermined size. When an inspection image corresponding to one inspection object , which is obtained by being cut out so as to be included in the rectangle, is input, abnormality information of the inspection object is output from the output layer through calculation by the intermediate layer. A learning model that makes a computer function. An irradiation unit that radiates electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and conveyed ,
Among the images acquired according to the electromagnetic waves transmitted through each inspection object, the blob corresponding to one inspection object is specified based on the pixel value of each pixel so that the blob falls within a rectangle of a predetermined size. And an acquisition unit that acquires an inspection image corresponding to one inspection object,
Input the inspection image corresponding to one inspection object acquired by identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object and cutting out so that the blob falls within a rectangle of a predetermined size In the learning model that outputs the abnormality presence/absence information indicating the presence or absence of abnormality of the inspection object, the learning inspection unit that inputs the inspection image acquired by the acquisition unit and acquires the abnormality presence/absence information,
And an output unit that outputs the abnormality presence/absence information. The electromagnetic waves are X-rays,
The acquisition unit is
A generation unit that generates an image based on the X-rays;
An extraction unit that extracts the blob from the generated image,
An enclosing unit enclosing the blob with a bounding box, and the inspection apparatus according to claim 5, which acquires an inspection image surrounded by the bounding box. Equipped with a display,
The output unit is
A plurality of inspection images each surrounded by a bounding box are displayed in association with positions of the plurality of inspection objects, and the abnormal image including the abnormal inspection object is further surrounded by a second bounding box different from the bounding box. And a second screen having the operation screen for displaying the abnormal image surrounded by the second bounding box and having an operation button for accepting an operator's input of abnormality information of each inspection object. The inspection device according to claim 6, which is a display unit that displays the screen on the display side by side. The inspection apparatus according to claim 7, further comprising a re-learning unit that re-learns the learning model based on an inspection image and the abnormality presence/absence information received by the operation button. The second screen includes a second operation button that receives an input of the type of abnormality of the inspection object,
The inspection apparatus according to claim 7 or 8, wherein when the second operation button receives the input, the display unit surrounds and displays an abnormal image with the second bounding box based on the type. .. A discarding unit that discards an inspection item in which the abnormality presence/absence information acquired by the learning inspection unit is a threshold value or more,
A second output unit that outputs coordinate information corresponding to the inspection image of the inspection object for which the abnormality inspection information acquired by the learning inspection unit is greater than or equal to a threshold value. Inspection equipment. A discarding unit having a plurality of valves arranged in a direction intersecting with the conveyance direction of the inspection object, ejecting gas from the valve corresponding to the abnormal inspection object to blow off the inspection object,
A second output unit that outputs valve information corresponding to the inspection object for which the abnormality inspection information acquired by the learning inspection unit is equal to or greater than a threshold value;
The opening/closing part which opens/closes the valve corresponding to the said to-be-inspected object, The inspection apparatus of any one of Claim 5 to 9. An image obtained by applying an image processing algorithm to the image obtained according to the electromagnetic wave is subjected to image processing to identify the blob, and the blob is cut out so as to be within a rectangle of a predetermined size, and the obtained blob is obtained. A second acquisition unit that acquires information indicating whether there is an abnormality in the inspection object based on an inspection image corresponding to the inspection object ,
The inspection apparatus according to any one of claims 5 to 11, wherein the learning inspection unit inputs the inspection image according to the information acquired by the second acquisition unit. Irradiate electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and transported ,
Among the images acquired according to the electromagnetic waves transmitted through each inspection object, the blob corresponding to one inspection object is specified based on the pixel value of each pixel so that the blob falls within a rectangle of a predetermined size. Cut out to obtain the inspection image corresponding to one inspection object,
Input the inspection image corresponding to one inspection object acquired by identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object and cutting out so that the blob falls within a rectangle of a predetermined size An abnormality detection method that causes a computer to execute a process of inputting the acquired inspection image and outputting the abnormality presence/absence information to a learning model that outputs abnormality presence/absence information indicating presence/absence of abnormality of the inspection object. Irradiate electromagnetic waves to a plurality of inspection objects that are distributed and placed on a conveyor and transported ,
Among the images acquired according to the electromagnetic waves transmitted through each inspection object, the blob corresponding to one inspection object is specified based on the pixel value of each pixel so that the blob falls within a rectangle of a predetermined size. Cut out to obtain the inspection image corresponding to one inspection object,
Input the inspection image corresponding to one inspection object acquired by identifying the blob among the images acquired according to the electromagnetic waves transmitted through the inspection object and cutting out so that the blob falls within a rectangle of a predetermined size A computer program that causes a computer to execute a process of inputting the acquired inspection image to a learning model that outputs abnormality presence/absence information indicating the presence or absence of abnormality of the inspection object and outputting the abnormality presence/absence information.