JP2021018082A - Detection device of foreign matter - Google Patents

Abstract

To provide a detection device of a foreign matter which can determine a portion where a foreign matter adheres to an object.SOLUTION: A detection device of a foreign matter comprises: an upper surface irradiation unit 111 and a lower surface irradiation unit 112 which irradiate an inspection object with light from an upper surface or a lower surface of a transparent or translucent conveyor 41 conveying the inspection object W and whose irradiation angle with respect to the conveyor can be changed; an image data acquisition unit 21 which images the inspection object from the upper surface of the conveyor, and can move so as to approach to or separate from the conveyor. The detection device of a foreign matter is an image determination device which extracts an extraction object from the image data, analyzes an image and determines a foreign matter from the image data obtained from an image data acquisition unit and comprises: a section division unit which divides the image data into sections; and an image identification unit which has a machine learning result of a feature amount of the extraction object with a method of deep learning, and calculates a possibility that the section is the extraction object for each section of the image data divided by the section division unit on the basis of the machine learning result.SELECTED DRAWING: Figure 35

Description

The present invention relates to a foreign matter detecting device. More specifically, the present invention relates to a foreign matter detecting device provided with a foreign matter discriminating portion using AI. For example, it relates to a detection device for foreign substances such as insects in green vegetables as an object to be inspected.

Various types of detection devices for foreign substances (extraction objects, insects) in the object to be inspected (photographed object) are known. Further, there are various known devices for detecting foreign substances in an inspected object by irradiating the inspected object with visible light and infrared light to detect foreign substances.

JP-A-2007-033273 Special Table 2008-541007 JP-A-2002-48727 Japanese Unexamined Patent Publication No. 2013-3129 Japanese Unexamined Patent Publication No. 2006-3134

The following devices are known as devices for detecting foreign substances by irradiating visible light and infrared light.
Patent Document 1 "Foreign matter detection device for white food" includes "a transport conveyor 10 for transporting the white food in a foreign substance detection device for white food that detects colored foreign matter mixed in the white food 20" and the same. A device including a camera 14 installed above a transport conveyor to capture an optical image of the food, and an image processing means for performing image processing including binarization processing on the image to detect foreign matter. The belt surface 10a on which the food 20 is placed is formed of a white light-transmitting material, and is located above the imaging region A on the transport conveyor and is white toward the imaging region. The configuration includes first white illuminations 16 and 17 that irradiate light, and second white illumination 18 that irradiates the imaging region with white light from below the belt surface. " Then, it is stated that "a foreign matter detecting device for white foods capable of detecting foreign matter with high accuracy even if the detection target has unevenness is provided."

Patent Document 2 "Foreign matter detection device for food" is a "foreign matter detection device 10 for detecting foreign matter such as hair and insects mixed in food 20, a transport conveyor 11 for transporting the food 20 to the detection region A, and the detection. Illumination means 14 and 15 that irradiate food with illumination light from at least two directions across the area A, an imaging camera 12 that captures a front image of the food from one side of the detection area A, and the captured table image. An image processing means 13 for detecting foreign matter by performing image processing including binarization processing on the image processing means 13 is provided, and the image processing means 13 binarizes the captured table image to obtain 2 After detecting an abnormal one pixel exceeding a preset blackness threshold in the value image, the presence or absence of an abnormality in a pixel adjacent to the one pixel is detected, and the abnormal pixel appears continuously based on the one pixel. At this time, if the abnormal pixels appear in a continuous linear shape, it is determined to be a fibrous foreign matter such as hair, and if they appear in a continuous area, the area of an insect or the like. It is determined to be a foreign substance. "

Patent Document 3 "Seaweed appearance inspection device" is a "light receiving and receiving light device that includes a visible light and an infrared light above the inspection position and a camera device that can separately receive the visible light and the infrared light. Arranged at a constant angle with respect to the surface, visible light, which is the reflected light of the reflected foreign matter on the surface of the seaweed, which is the subject, and the direct reflected light of the reflecting plate, and infrared light, which is the transmitted light, are separated into the camera device. A seaweed appearance inspection device that inspects part or all of the outer shape, chips, holes, blurring, surface reflection foreign matter, surface foreign matter, inner surface foreign matter, backside foreign matter, etc. of seaweed by receiving light and performing judgment processing. " Become. Therefore, "a compact and high-performance visual inspection device can be obtained without separating the conveyor at the inspection position.
Regardless of the difference in production area, the subject is accurately carried to the inspection position even if the size of the seaweed to be inspected is slightly different. ".

Patent Document 4 describes, "A transport device 2 for placing and moving an article on a translucent transport belt 16, a support frame 3 for supporting the transport device 2, and a semiconductor light emitting element 59, below the transport belt 16. 63, 64, 65 are placed on the printed substrates 57, 61, fixed LED mounting bodies 53, 54 are arranged, and the irradiation device 4 for irradiating the transport belt 16 from below and the transport belt 16 are operated and controlled. At the same time, the article inspection device 1 is composed of the control device 5 that controls the light emission of the semiconductor light emitting elements 59, 63, 64, 65. " The purpose is to provide a light-transmitting article inspection device that makes it difficult for workers to get tired, maintains work efficiency, does not impair eyesight, has no health problems, and enables efficient visual inspection. To do.

Patent Document 5 states, "Agricultural products are imaged by using an imaging means in a state where the agricultural products at the inspection position on the transporting means are irradiated with light from the lighting means, and the outline of the agricultural products is cut out from the imaged data by the IR value. It is an appearance measuring device for agricultural products that measures the appearance of agricultural products from at least R and G values in the cut out contour, and a plurality of lighting means are a plurality of infrared LEDs, red LEDs, and green LEDs in a predetermined arrangement as a set. The LED of each color of the lighting means is arranged in a matrix or a staggered pattern, and all of the LEDs of each color are turned on and off at the inspection position. " "By using a light source in which a plurality of infrared LEDs, red LEDs, and green LEDs are arranged in a predetermined pattern, it is possible to reduce the frequency of brightness adjustment and replacement, and perform maintenance work easily and at low cost. The purpose is to provide the device.

In the inventions described in Patent Documents 1, 2 and 3, the object to be inspected is irradiated with visible light and infrared light, "in a foreign matter detecting device for white food", and "white light toward an imaging region". There is a description about "irradiating."
However, in the inventions described in Patent Documents 1 to 3, it is not easy to find a foreign substance.
The inventions described in Patent Documents 4 and 5 have a problem that it is not easy to irradiate an object to be inspected with an LED and take a picture with a camera.

On the other hand, the inventor found the following.
1. 1. An object to be photographed (lettuce in this case) with a foreign substance (insect in this case) is irradiated with light, and images are taken in a plurality of states including a state in which the exposure is higher than the appropriate exposure to obtain images with different exposures.
2. 2. Increasing the exposure makes the background, which is the object to be photographed, whitish, making it easier to see foreign objects.
3. 3. Obtain multiple image data with different exposures.
4. Foreign matter can be easily found by detecting foreign matter by a deep learning method based on a plurality of image data of different exposures.
5. Irradiate the object to be photographed (lettuce in this case) with a foreign substance (insect in this case) with light of the same color as or similar to the color of the object to be photographed. For example, in the case of lettuce, it is irradiated with light in the green light region (wavelength 495-570 nm), and the white vegetables to be photographed are irradiated with purple light, blue light, or ultraviolet light, and at least the appropriate exposure is achieved. It is even more effective to take pictures in a plurality of states including a state in which the exposure is increased to obtain images with different exposures.

Further, the inventor continuously changes the light of an LED illumination having a different wavelength from ultraviolet light to infrared light (sweeping the frequency of the light) to the object (subject) to be photographed containing a foreign substance, and at the same time. It was found that the frequency of the light used for the whitest subject in the image can be said to be the frequency of light suitable for detecting foreign matter when the image is taken with the exposure higher than the appropriate exposure.
In order to obtain the frequency of this light, the object (subject) to be photographed is photographed by continuously changing the light of, for example, LED illumination having different wavelengths from ultraviolet light to infrared light, and the area photographed in white is calculated by a computer. It is possible to specify the frequency of light required for detecting foreign matter.
Then, by irradiating the object to be photographed (subject) with light having a frequency that specifies the frequency of the light required for detecting the foreign matter, the foreign matter can be identified more efficiently.

This invention
An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
While irradiating the object to be photographed with light and taking pictures in a plurality of states including a state in which the exposure is raised above the proper exposure, images with different exposures are obtained, and foreign matter is discriminated from the image data obtained from the image data acquisition unit. It is a foreign matter detection device
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device, which comprises an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit.
Consists of.

This invention
An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
From the image data obtained from the image data acquisition unit, while irradiating light of the same color or similar to the object to be photographed and shooting in a plurality of states including a state in which the exposure is raised above the appropriate exposure, images with different exposures are obtained. It is a foreign matter detection device that discriminates foreign matter.
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device, which comprises an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit.
Consists of.

This invention
An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
Images of different exposures can be obtained by irradiating white vegetables, which are the objects to be photographed, with purple light, blue light, or ultraviolet light and in multiple states including a state in which the exposure is raised above the proper exposure, and image data. A foreign matter detection device that discriminates foreign matter from image data obtained from the acquisition unit.
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device, which comprises an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit.
Consists of.

The present invention further
The image processing unit performs image processing on the image data divided by the division division unit, and then performs image processing on the image data.
The image identification unit is a device for detecting a foreign substance, which calculates the probability of being the extraction target for each of the categories of the image data with respect to the image data image-processed by the image processing unit. ,
Consists of.

The present invention further
For green vegetables
While irradiating with green light, take a picture with the exposure raised above the proper exposure.
Take in the reflected light of green light and obtain image data,
Foreign matter discrimination device that discriminates foreign matter from image data,
Consists of.

The present invention further
For red vegetables
Irradiate red light or near infrared light, and raise the exposure above the proper exposure to shoot.
Take in red light or near infrared light to obtain image data,
Foreign matter discrimination device that discriminates foreign matter from image data,
Consists of.

This invention
An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
The object to be photographed containing foreign matter is irradiated with light of different wavelengths from ultraviolet light to infrared light by continuously changing the light, and the exposure is raised above the appropriate exposure, and each photographed image is photographed in white. By calculating the area covered and comparing each image, it is determined that the frequency of light used for the whitest subject in the image is the frequency of light suitable for detecting foreign matter.
Foreign matter detector, which identifies the frequency of light required for foreign matter detection
Consists of.

This invention
An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
While irradiating the object to be photographed with light and taking pictures in a plurality of states including a state in which the exposure is raised above the proper exposure, images with different exposures are obtained, and foreign matter is discriminated from the image data obtained from the image data acquisition unit. It is a foreign matter detection device
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
An image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit is provided.
The light that illuminates the object to be photographed is
The object to be photographed containing foreign matter is irradiated with light of different wavelengths from ultraviolet light to infrared light by continuously changing the light, and the exposure is raised above the appropriate exposure, and each photographed image is photographed in white. By calculating the area covered and comparing each image, it is determined that the frequency of light used for the whitest subject in the image is the frequency of light suitable for detecting foreign matter.
A foreign matter detector, characterized by identifying the frequency of light required for foreign matter detection.
Consists of.

Provided is a foreign matter detection device provided with a foreign matter discriminating unit using AI, which enables discrimination of a portion where a foreign matter (extraction target, insect) is attached to a photographed object (object to be inspected) such as vegetables and fruits.

It is a front view of the use state of the image discrimination apparatus which is the foreign matter discriminating part of one Embodiment used for the foreign matter detecting apparatus which is an embodiment of this invention. It is a functional block diagram for demonstrating the function which the image analysis apparatus which is the foreign matter discriminating part of one Embodiment used for the foreign matter detecting apparatus which is an embodiment of this invention has. It is a flowchart for demonstrating the image analysis processing which is the foreign matter discriminating part of one Embodiment used for the foreign matter detecting apparatus which is an embodiment of this invention. It is a flowchart for demonstrating the image identification processing which is the foreign matter discriminating part of one Embodiment used for the foreign matter detecting apparatus which is an embodiment of this invention. Learning about lettuce to which insects, which are the objects to be extracted, are attached by using the deep learning method of the image discrimination device, which is the foreign body discrimination unit of one embodiment used for the foreign matter detection device according to the embodiment of the present invention. It is a figure for demonstrating the processing by the classifier which has a result. Lettuce photographed with appropriate exposure by irradiating lettuce installed in an image discrimination device, which is a foreign matter discrimination unit of one embodiment used in the foreign matter detection device according to the embodiment of the present invention, with green light which is visible light. It is a diagram showing how the insects look. The lettuce of the first embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the second embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by exposure. The lettuce of the second embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the second embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the third embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by exposure. The lettuce of the third embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used for the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the third embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used for the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the fourth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by exposure. The lettuce of the fourth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the fourth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the fifth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by exposure. The lettuce of the fifth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the fifth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discriminating device, which is the foreign matter discriminating unit of one embodiment used for the foreign matter detecting device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. The lettuce of the sixth embodiment installed in the image discrimination device, which is the foreign matter discrimination unit of one embodiment used in the foreign matter detection device of the embodiment of the present invention, is appropriately irradiated with green light which is visible light. It is an image diagram taken by raising the exposure more than the exposure. It is a front view of the foreign matter detection device which is an embodiment of this invention. It is a left side view of the foreign matter detection device which is an embodiment of this invention. It is a right side view of the foreign matter detection device which is an embodiment of this invention. It is a top view of the foreign matter detection apparatus which is an embodiment of this invention. It is a bottom view of the foreign matter detection device which is an embodiment of this invention. It is a rear view of the foreign matter detection device which is an embodiment of this invention. It is a perspective view of the foreign matter detection apparatus which is an embodiment of this invention. It is a perspective view of the state of installation on the conveyor of the foreign matter detection apparatus which is an embodiment of this invention.

Reference numeral 31 is a foreign matter detecting device. The foreign matter detecting device 31 detects the extraction target, the insect A which is a foreign matter. In the embodiment of the present invention, it is mainly a camera for an object to be inspected (object to be photographed) W such as vegetables. The foreign matter detecting device 31 can be moved by the caster 32 attached to the lower part.
41 is a conveyor. The conveyor 41 is made of plastic, urethane rubber, ethylene / propylene rubber, or the like, and is transparent or translucent. The conveyor 41 conveys the object W to be inspected. The conveyor 41 moves the object W to be inspected by driving a motor.
The foreign matter detecting device 31 has an upper arm 33, a lower arm 34, and a support column 35.
The upper arm 33 and the lower arm 34 are attached to the support column 35. The upper arm 33 can be moved in the vertical direction in FIGS. 28 to 35 along the support column 35, and the angle can be changed in the axial direction.

The image discrimination device 1, which is a foreign matter discrimination unit used in the foreign matter detection device 31, has an irradiation unit 11 and an image data acquisition unit 21. The image data acquisition unit 21 includes a camera.
The irradiation unit 11 includes a top surface irradiation unit 111 and a bottom surface irradiation unit 112. The irradiation unit 11 is composed of an LED light.
The irradiation unit 11 includes green light (495-570 nm), red light (620-750 nm), near infrared rays (0.7-2.5 μm), orange light, orange light (590-620 nm), and yellow light (570-590 nm). It is capable of emitting nm), purple light (380-450 nm), blue light (450-495 nm), and infrared light (10-400 nm).
The image data acquisition unit 21 and the irradiation unit 11 including the camera are installed downward on the tip of the upper arm 33.
The irradiation unit 11 and the image data acquisition unit 21 are attached to the lower arm tip 34 upward.

Since the upper arm 33 can be moved in the vertical direction in FIGS. 28 to 35 along the support column 35, the positions of the image data acquisition unit 21 and the irradiation unit 11 can be adjusted. In FIGS. 29 and 30, a conveyor 41 on which the object to be inspected (photographed object) W is placed is installed between the upper arm 33 and the lower arm 34, which are provided in a U shape. The LED attached to the upper arm 33 and the lower arm 34 irradiates the object to be inspected (photographed object) W. The image data acquisition unit 21 attached to the upper arm 33 and the lower arm 34 photographs the object to be inspected (photographed object) W.

The top surface irradiation unit 111 irradiates the object to be inspected (photographed object) W with light from the upper surface of the conveyor 41, and the irradiation angle with respect to the conveyor 41 can be changed.
The bottom surface irradiation unit 112 irradiates the object to be inspected (photographed object) W with light from the bottom surface of the conveyor 41, and the irradiation angle with respect to the conveyor 41 can be changed.

As shown in FIG. 28, the left and right LED lights of the upper surface irradiation unit 111 and the left and right LED lights of the lower surface irradiation unit 112 are installed so as to oppose the cloth in an X shape. The LED light of the upper left irradiation unit 111 and the LED light of the lower right of the lower surface irradiation unit 112 face each other. Face the LED light on the upper right and the LED light on the lower left, and place the object to be inspected (photographed object) W at the focal point.
Since the foreign matter detecting device 31 can be moved by the caster 32 attached to the lower part, the LED light can be moved forward and backward with respect to the conveyor 41.

The object to be inspected (object to be photographed) W is moved by a motor (not shown). The drive of the motor is stopped, the conveyor 41 is stopped, the object to be inspected (object to be photographed) W is stopped by the image data acquisition unit 21, and the image data acquisition unit 21 takes an image.
The foreign matter detecting device 31 drives the conveyor 41 by supplying electric power to drive the motor and the like by electromagnetic induction using a coil. Since there are wet areas in the factory, current is supplied without contact by wireless power supply.
The foreign matter detection device 31 can also be equipped with a robot. Reference numeral 63 illustrated in FIG. 29 is a robot connection terminal. The robot connection terminal 63 serves as a terminal for connecting the robot.

61 and 62 are monitors. It is a touch panel consisting of a TV screen. The monitor 61 is attached to the front surface of the upper surface irradiation unit 111 and the lower surface irradiation unit 112 on the attachment side, and is attached to the upper end side of the upper surface irradiation unit 111. The monitor 62 is attached to the rear portion of the support column 35 on the back side so as to face diagonally upward.
71 is a control panel. The control panel 71 controls the electric system of the foreign matter detecting device 31.

Hereinafter, examples of the image discrimination device 1 which is a foreign matter discrimination unit used in the foreign matter detection device 31 of the present invention will be described with reference to FIGS. 1 to 27 with respect to Examples 1 to 12.
First Example A foreign matter detection device 31, a foreign matter detection method, and an embodiment of a foreign matter detection program will be described.
The image discrimination device 1, which is a foreign matter discrimination unit used in the foreign matter detection device 31, includes an irradiation unit 11, an image data acquisition unit 21, an operation input information acquisition unit 12, and an image data division processing unit 13 which is a division division unit. , An image identification processing unit 14, an identification result conversion processing unit 15, an image processing unit 16, an output processing unit 17, and a storage unit 18.

The irradiation unit 11 is the green vegetable W in this embodiment according to the first embodiment, and in this embodiment, in the lettuce, the irradiation unit 11 has the same color and similar color, and the LED green light (495) which is an approximate color. -570 nm) is used.
The wavelength of visible light is as follows.
Purple 380-450 nm
Blue 450-495 nm
Green 495-570 nm
Yellow 570-590 nm
Orange 590-620 nm
Red 620-750 nm
The wavelength of infrared light is as follows.
Near infrared has a wavelength of about 0.7-2.5 μm
Mid-infrared has a wavelength of approximately 2.5-4 μm
Far infrared rays have a wavelength of approximately 4-1000 μm.
Ultraviolet rays are 10-400 nm

As green vegetables and green fruits,
Moroheiya, leek, green beans, sasage, wisteria beans, bok choy, sansho, edible beans, sora beans, tarzai, kabosu, herbs, avocado, aloe, sudachi, lime, kiwi, squid, Melon, broccoli, vine purple, spring chrysanthemum, spinach, mustard greens, Hiroshima greens, Japanese mustard greens, celery, wiping, artichoke, Japanese mustard spinach, cucumber, lettuce, salad greens, endive, okura, okahijiki, green beans There are Osaka Chinese cabbage, Japanese mustard, asparagus, tomorrow, green beans (large leaves), and pumpkin.

Appropriate exposure and overexposure As shown in FIG. 1, green vegetables, which are the objects to be photographed W In this example, lettuce has a foreign substance A, which is an extraction object, and insects A in this example.
The object W (lettuce in this case) with a foreign substance (insect A in this case) is irradiated with light of the same color as or similar to the color of the object to be photographed. For lettuce, irradiate the green light region (wavelength 495-570 nm).
The lettuce is irradiated with green light, and the exposure is raised above the proper exposure to take a picture. The reflected light of the green light is taken in to obtain image data, and the foreign matter detection device 31 for discriminating foreign matter from the image data, detection of foreign matter It consists of a method and a foreign matter detection program.
When the exposure is increased, the background of the object to be photographed W (lettuce in this embodiment) becomes whitish, and the foreign matter A becomes easy to see.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

FIG. 6 is an image diagram taken by irradiating the lettuce of the first embodiment, which is the embodiment of the present invention, with green light which is visible light and taking an appropriate exposure.
FIG. 7 is an image diagram taken by irradiating the lettuce of the first embodiment, which is the embodiment of the present invention, with green light, which is visible light, to raise the exposure beyond the proper exposure.

2nd Example FIG. 8 is an image diagram taken by irradiating the lettuce of the second embodiment, which is the embodiment of the present invention, with green light which is visible light and taking an appropriate exposure.
9 and 10 are images taken by irradiating the lettuce of the second embodiment, which is the embodiment of the present invention, with green light, which is visible light, to raise the exposure beyond the proper exposure.

Third Example FIG. 11 is an image diagram taken by irradiating the lettuce of the third embodiment, which is the embodiment of the present invention, with green light which is visible light and taking an appropriate exposure.
12 and 13 are images taken by irradiating the lettuce of the third embodiment, which is the embodiment of the present invention, with green light, which is visible light, to raise the exposure beyond the proper exposure.

Fourth Example FIG. 14 is an image diagram taken by irradiating the lettuce of the fourth embodiment, which is an embodiment of the present invention, with green light which is visible light and taking an appropriate exposure.
15 and 16 are images taken by irradiating the lettuce of the fourth embodiment, which is the embodiment of the present invention, with green light, which is visible light, to raise the exposure beyond the proper exposure.

Fifth Example FIG. 17 is an image diagram of lettuce of the fifth embodiment, which is an embodiment of the present invention, taken with green light which is visible light with proper irradiation exposure.
18 and 19 are images taken by irradiating the lettuce of the fifth embodiment, which is the embodiment of the present invention, with green light, which is visible light, to raise the exposure beyond the proper exposure.

6th Example FIG. 20 is an image diagram taken by irradiating the lettuce of the 6th embodiment, which is the embodiment of the present invention, with green light which is visible light and taking an appropriate exposure.
21, FIG. 22, FIG. 23, FIG. 24, FIG. 25, FIG. 26, and FIG. 27 show that the lettuce of the sixth embodiment according to the embodiment of the present invention is irradiated with green light which is visible light and is exposed more than appropriate. It is an image diagram taken with increased exposure.

7th Example In this embodiment according to the 7th embodiment, in the red vegetable and the red fruit W, the irradiation unit 11 has the same color and the similar color, and the LED red light (620-750 nm) or the light of the similar color. , Use near infrared (0.7-2.5 μm).
Red vegetables and red fruits include carrots, tomatoes, strawberries, watermelons, peppers, paprika, Japanese ginger, beets, apples, cherries, peaches, rhubarb, and red peppers.

Appropriate exposure and overexposure As shown in FIG. 1, the imaged object W, the red vegetable and the red fruit W, has a foreign substance A attached to the carrot in this embodiment and an insect A attached to the carrot in this embodiment.
The object W (carrot in this case) with a foreign substance (insect A in this case) is irradiated with light of the same color as or similar to the color of the object to be photographed. In the case of carrots, irradiate with red light (620-750 nm) or near infrared rays (0.7-2.5 μm).
Taking a picture with an exposure higher than the appropriate exposure, taking in the reflected light of red light or near infrared light to obtain image data, a foreign matter detection device 31 that discriminates foreign matter from the image data, a foreign matter detection method, and foreign matter detection Consists of a program.

When the exposure is increased, the background of the object W (carrot in this case) becomes whitish, making it easier to see foreign matter.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

Eighth Example In this embodiment according to the eighth embodiment, in this embodiment, the irradiation unit 11 is an orange vegetable, and in the orange fruit W, the irradiation unit 11 is a light of the same color and a similar color. Use the orange, orange light (590-620 nm) of the LED.
Orange vegetables and orange fruits include pumpkin, carrot, papaya, persimmon, yuzu, mango, anzu, orange, and loquat.

Appropriate exposure and overexposure As shown in FIG. 1, the orange vegetables and orange fruits, which are the objects W to be photographed, have a foreign substance A attached to the pumpkin and an insect A attached to the pumpkin in this embodiment. ..
The object W (pumpkin in this case) with a foreign substance (insect A in this case) is irradiated with light of the same color as or similar to the color of the object to be photographed. In the case of pumpkin, irradiate the area of orange and orange light (590-620 nm).

Taking a picture with the exposure raised upward, the reflected light of orange-orange light (590-620 nm) is taken in to obtain image data, and the foreign matter detection device 31 that discriminates foreign matter from the image data, the foreign matter detection method, and the foreign matter Consists of a detection program.
When the exposure is increased, the background of the object W (pumpkin in this case) becomes whitish, making it easier to see foreign matter.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

9th Example In this embodiment according to the 9th embodiment, in this embodiment, the irradiation unit 11 is a yellow vegetable, and in the yellow fruit W, the irradiation unit 11 is a light of the same color and a similar color. Use LED yellow light (570-590 nm).
Yellow vegetables and yellow fruits include pumpkin, onions, lemons, kumquats, pears, pineapples, grapefruits, bananas, and corn.

Appropriate exposure and overexposure As shown in FIG. 1, the yellow vegetables and yellow fruits, which are the objects W to be photographed, have a foreign substance A attached to the onion and an insect A attached to the onion in this example.
The object W (onion in this case) with a foreign substance (insect A in this case) is irradiated with light of the same color as or similar to the color of the object to be photographed. For onions, illuminate the area of yellow light (570-590 nm).
Irradiate the onion with yellow light (570-590 nm) and raise the exposure above the proper exposure.
It consists of a foreign matter detection device 31 that takes in reflected light of yellow light (570-590 nm), obtains image data, and discriminates foreign matter from the image data, a foreign matter detection method, and a foreign matter detection program.
When the exposure is increased, the background of the object W (onion in this case) becomes whitish, making it easier to see foreign matter.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

10th Example In this embodiment according to the 10th embodiment, the irradiation unit 11 is a purple vegetable in this embodiment, and in the purple fruit W, the irradiation unit 11 is a light of the same color and a similar color. Use the purple light of the LED (380-450 nm).
Purple vegetables and purple fruits include blueberries, eggplants, berries, and grapes.

Appropriate exposure and overexposure As shown in FIG. 1, the purple vegetables and purple fruits, which are the objects to be photographed W, have a foreign substance A attached to the eggplant in this example and an insect A attached to the eggplant in this example.
The imaged object W (eggplant in this case) with a foreign substance (insect A in this case) is irradiated with light of the same color as or similar to the color of the object to be photographed. In the case of eggplant, irradiate the area of purple light (380-450 nm).
In the end, irradiate purple light (380-450 nm) and raise the exposure above the proper exposure to shoot.
It consists of a foreign matter detection device 31 that takes in reflected light of purple light (380-450 nm), obtains image data, and discriminates foreign matter from the image data, a foreign matter detection method, and a foreign matter detection program.
When the exposure is increased, the background of the object to be photographed W (in this case) becomes whitish, and foreign matter becomes easy to see.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

Eleventh Example In this embodiment according to the eleventh embodiment, in this embodiment, the irradiation unit 11 is a white vegetable, and in the white fruit W, the irradiation unit 11 is an LED purple light (380-450 nm) or , Use blue light (450-495 nm) or ultraviolet light (10-400 nm).
White vegetables and white fruits include radish, cabbage, wasabi, green onion, garlic, onion, celeriac (celeriac), turnip, lotus root, lily root, sprouts, radish sprouts, udon, bamboo shoots, Chinese cabbage, and chicory (andive).

Appropriate exposure and overexposure As shown in FIG. 1, the white vegetables and white fruits, which are the objects W to be photographed, have a foreign substance A attached to the radish and an insect A attached to the radish in this example.
In the case of a radish on the object W (radish in this case) with a foreign substance (insect A in this case), purple light (380-450 nm), blue light (450-495 nm), or ultraviolet light (10-). Irradiate a region of light (400 nm).
The radish is irradiated with purple light (380-450 nm), blue light (450-495 nm), or ultraviolet light (10-400 nm), and the exposure is raised above the appropriate exposure. It consists of a foreign matter detection device 31 that takes in reflected light such as 380-450 nm) to obtain image data and discriminates foreign matter from the image data, a foreign matter detection method, and a foreign matter detection program.
When the exposure is increased, the background of the object W (radish in this case) becomes whitish, making it easier to see foreign matter.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

12th Example In the 12th Example, an object (subject) to be photographed containing a foreign substance is irradiated with continuously changing light of, for example, LED illumination having different wavelengths from ultraviolet light to infrared light (sweep of light frequency). In addition, when the image is taken with an exposure higher than the appropriate exposure, the frequency of the light used for the whitest subject in the image can be said to be the frequency of the light suitable for detecting foreign matter.
In order to obtain the frequency of this light, the area of the object (subject) to be photographed is photographed by continuously changing the light of LED illumination having different wavelengths from ultraviolet light to infrared light, and each photographed image is photographed in white. Is calculated and calculated by a computer to compare each image. The frequency of the light used for the whitest subject in the obtained image is determined to be the frequency of the light suitable for detecting foreign matter, the frequency of the light required for detecting the foreign matter is specified, and the frequency of the light is specified. To use.

Appropriate exposure and overexposure As shown in FIG. 1, the green vegetable, which is the object W to be photographed, has a foreign substance A, which is an extraction object, attached to the lettuce in this example, and an insect A, which is an extraction object in this example. ..
When shooting an object W (lettuce in this case) with a foreign object (insect A in this case) with an exposure higher than the appropriate exposure, the light used for the whitest subject in the image. Irradiate light of frequency. For lettuce, irradiate the green light region (wavelength 495-570 nm).
The lettuce is irradiated with green light, and the exposure is raised above the proper exposure to take a picture. The reflected light of the green light is taken in to obtain image data, and the foreign matter detection device 31 for discriminating foreign matter from the image data, detection of foreign matter It consists of a method and a foreign matter detection program.
When the exposure is increased, the background of the object W (lettuce in this case) becomes whitish, and the foreign matter A becomes easy to see.
Therefore, a plurality of images having different exposures, such as an overexposed image in which the exposure is higher than the proper exposure, including an image having an appropriate exposure, are obtained.
By using a deep learning method based on the obtained plurality of images, it becomes easy to find foreign substances.

The image data acquisition unit 21 is an imaging unit and receives the reflected light from the irradiation unit 11.
The image data acquisition unit 21 acquires image data of an inspection object captured in the first to twelfth embodiments by, for example, a digital still camera, a line sensor camera, or the like, and supplies the image data to the image data division processing unit 13. To do.
An image obtained by superimposing the images received by the image data of different exposures taken in the first to twelfth embodiments shown in FIGS. 6 to 27 is obtained.

How to detect insects In the first to twelfth examples, insect A is detected as a black dot on the background, which is a whitish object W, by changing the exposure, so AI learns the color and shape. It can be identified by its shape (see FIGS. 6 to 27).

In the first to twelfth embodiments, binarization is unnecessary.
Unless a method of learning by AI using deep learning is taken, it is difficult to detect insect A, which is a foreign substance using color and shape, so it is necessary to always use AI using deep learning.
Deep learning, which is part of artificial intelligence (AI), allows computers, like humans, to learn new concepts from raw data. In a deep learning system, you can identify the difference by checking thousands of specific images.

The obtained images are overlaid and judged by AI.
Images received at different exposures are obtained, and superimposed images are obtained.

The operation input information acquisition unit 12 acquires the user's operation input input from the input unit 2, and has an image data division processing unit 13, an image identification processing unit 14, an identification result conversion processing unit 15, an image processing unit 16, and an image processing unit 16. , Information corresponding to the user's operation input is supplied to each unit of the output processing unit 17.

The image data division processing unit 13 divides and divides image data of an image received by purple light supplied from the image data acquisition unit 21, an image received by blue light, and an image obtained by superimposing an image received by infrared light. Then, it is supplied to the image identification processing unit 14 or the image processing unit 16 as a square cell having a predetermined size.
The size of the cell to be divided by the image data division processing unit 13 can be specified by the user. The image data division processing unit 13 corresponds to the division division unit. In addition, the cell corresponds to the division divided by the division division portion. The cell is an image area having a predetermined size corresponding to the division divided by the image data division processing unit 13, and even if it is composed of a square area of pixels having the same number of pixels in the vertical and horizontal directions, the cell has a vertical and horizontal size. It may be composed of different rectangular areas.

The image identification processing unit 14 has a learning result of the extraction target object using the technique of deep learning, and discovers the feature amount of the extraction target object from the supplied image data. For example, the image identification processing unit 14, which has learned the feature amount for distinguishing insect A from images in which insect A appears black at different exposures by using a deep learning method, is an insect to be extracted from the supplied image data. It is possible to identify A.

Deep learning imitates the network of nerve cells in the human brain (neural network), and by deepening the layer of the neural network (multi-layer network), the features contained in images and sounds are used as learning results. It is an information processing technology that can be discovered based on it.
In the conventional recognition / identification technology for images, sounds, etc., the user sets the feature amount, and then the extraction target is classified by the classification algorithm based on the feature amount. However, deep learning is used. By causing the image identification processing unit 14 to perform machine learning, it is possible to extract the extraction target without requiring the user to set the feature amount. The image identification processing unit 14 calculates the probability that each cell constituting the image data supplied from the image data division processing unit 13 or the image processing unit 16 is an extraction target, and causes the identification result conversion processing unit 15 to calculate the probability. Supply. The image identification processing unit 14 corresponds to the image identification unit.

The identification result conversion processing unit 15 converts the probability that each cell constituting the image data supplied from the image identification processing unit 14 is an extraction target into a pixel value by a method described later, and converts each cell into a pixel value. Image data corresponding to the pixel value is generated and supplied to the image processing unit 16. The identification result conversion processing unit 15 refers to, for example, a tone curve (or a lookup table), and determines the probability (0% to 100%) that each cell is an object, and a pixel value having a value of 0 to 255. Convert to. For example, when the probability of the cell as the extraction target is 100%, the pixel value is converted to 255 based on the tone curve. The identification result conversion processing unit 15 corresponds to the identification result conversion unit.

The image processing unit 16 is supplied from the identification result conversion processing unit 15 to the image data indicating the probability that the pixel value of each cell is the extraction target, or is supplied from the image data division processing unit 13. For the divided cell image data, for example, image processing that reflects noise removal using a threshold, the knowledge of the processor that distinguishes insect A, expansion processing, closing processing, skeleton line extraction processing, etc. is executed. The processed image data is supplied to the image identification processing unit 14 or the output processing unit 17. The image processing unit 16 corresponds to the image processing unit.

Here, the image data divided into cells by the image data division processing unit 13 may be processed first by either the image identification processing unit 14 or the image processing unit 16. Further, the image data divided into cells by the image data division processing unit 13 may be processed a plurality of times by the image identification processing unit 14 or the image processing unit 16. In the image discrimination device 1, the processing in the image identification processing unit 14 and the image processing unit 16 is performed so as to be suitable for the characteristics of the input image data and the extraction target object based on the user's operation input input by the input unit 2. It is possible to set the order and number of times.

The output processing unit 17 supplies the processed image data supplied from the image processing unit 16 to the output unit 3 for output, or supplies the storage unit 18 for storage. Further, the output processing unit 17 reads out the processed image data stored in the storage unit 18 and supplies the image data to the output unit 3 for output.
The storage unit 18 stores the processed image data supplied from the output processing unit 17 in the built-in storage medium or the removable storage medium mounted on the storage medium.

In the image discrimination apparatus 1, the identification result conversion processing unit 15 converts the probability that each cell constituting the image data identified by the image identification processing unit 14 is an extraction target into image data, and the image thereof. By processing the data, the extraction target can be detected. Therefore, even if the overall brightness and the like of the images taken at different times are different, the analysis result is not affected by the difference, and it is suitable for use in comparing the changes with time of the same portion.

Next, referring to the flowchart of FIG. 3, as an example of the processing executed by the image discrimination apparatus 1, from many images of the insect A attached to the vegetables, the vegetables to which the insect S attached using the deep learning method is attached. The image analysis process 1 executed by the image discrimination apparatus 1 including the image identification processing unit 14 that has learned the feature amount will be described.

In step S1, the image data acquisition unit 21 acquires the image data to be analyzed and supplies it to the image data division processing unit 13.

In step S2, the image data division processing unit 13 divides the image data to be analyzed and supplies it to the image identification processing unit 14. The size of the divided cell may be set by the user depending on the type of image data to be analyzed and the extraction target. The image data to be analyzed may be composed of squares such as 64 pixels vertically and 64 pixels horizontally, or may be composed of rectangles having different vertical and horizontal sizes in one cell. .. Further, one cell may be 1 cm square of insect A as 1 cell or 5 cm square as 1 cell. It is possible to freely set the image resolution per unit pixel.

In step S3, the image identification process described later is executed using FIG.

In step S4, the identification result conversion processing unit 15 determines the identification result by the image identification processing, that is, the probability that each cell is an extraction target calculated by the image identification processing unit 14, based on, for example, a tone curve. , Converted to pixel values and supplied to the image processing unit 16.

In step S5, the conventional image processing 1 described later is executed with reference to FIG.

In step S6, the output processing unit 17 supplies the processed image analysis result supplied from the image processing unit 16 to the storage unit 18. The storage unit 18 stores the supplied image analysis result. Further, the output processing unit 17 supplies the processed image analysis result supplied from the image processing unit 16 to the output unit 3 for output, and the processing is completed.

By such processing, the probability that each cell constituting the image data is an extraction target is converted into image data, and the extraction target is detected using the image data, so that the entire image is displayed. Even if the brightness and the like are different, the analysis result is not affected by it.
Further, in the extraction result of the object thus obtained, for example, a portion confusing with the insect A is not extracted together with the insect A, and it is possible to obtain an analysis result close to visual confirmation. ..

Next, the image identification process executed in step S3 will be described with reference to the flowchart of FIG.

In step S11, the image identification processing unit 14 receives the designation of the target to be identified based on the user's operation input supplied from the operation input information acquisition unit 12. Here, the image identification processing unit 14 receives the designation of insect A as the target to be identified.

In step S12, the image identification processing unit 14 executes the target identification processing based on the learning result.

In 13, the image identification processing unit 14 calculates the probability that the pixel value of the pixel included in the target is the extraction target for each cell based on the identification result obtained by the processing in step S12, and the identification result conversion processing unit 15 The process returns to step S4 of FIG.

By such processing, the image identification processing unit 14 uses a deep learning method, for example, based on the learning result of the feature amount obtained from many images of the vegetables to which the insect A is attached, the feature amount by the user. It is possible to extract the extraction target by excluding the portion corresponding to the portion confusing with the insect A without requiring setting.

Next, the conventional image processing 1 executed in step S5 of FIG. 3 will be described with reference to the flowchart of FIG.

In step S21, the image processing unit 16 refers to the cell image data having the pixel value supplied from the identification result conversion processing unit 15, that is, the image data indicating the probability that the pixel value of the cell image is the extraction target. The pixel value of the cell image having the pixel value equal to or less than a predetermined threshold value is set to 0. In the image data supplied from the identification result conversion processing unit 15, cells having a high probability of presence of insect A are indicated by dots having a high pixel value, and cells having a low probability of presence of insect A are indicated by dots having a low pixel value. It is the image data that has been created.

For cells having a pixel value of 70% of the pixel value as a threshold value and having a pixel value less than that, when the pixel value is set to 0, for example, it has characteristics similar to those of insect A existing in insect A attached to vegetables. Image data can be obtained in which the noise component due to the part confusing with the insect A is removed to some extent.

In step S22, the image processing unit 16 executes an expansion process for expanding the area of the dots having pixel values by the process of step S21. By this processing, in the images of FIGS. 6 to 27 and the like, the portion where the insect A attached to the vegetable is detected is shown.

In step S23, the image processing unit 16 detects the length of continuous pixels from the image data obtained by the process of step S22, and removes a component having a length equal to or less than a predetermined threshold value. By this processing, it is possible to obtain image data in which the portion of the image data shown in FIGS. 6 to 27 that is confusing with the insect A and the noise component that could not be deleted by the processing in step S21 are removed.

In step S24, the image processing unit 16 executes a closing process for smoothing the edge portion of the image data obtained by the process of step S23. By this process, image data can be obtained from the image data shown in FIGS. 6 to 27 and the like.

In step S25, the image processing unit 16 executes the skeleton line extraction process on the image data obtained by the process of step S24, so that the insect A attached to the vegetables expanded by the expansion process of step S22. Image data can be obtained by returning to the shape of.

In step S26, the image processing unit 16 detects the length of continuous pixels of the image data obtained by the process of step S25, and removes a component having a length equal to or less than a predetermined threshold value. By this processing, the portion of the image data that is confused with the insect A and the noise component that could not be deleted by the processing of steps S21 and S23 are removed, and the image data as shown in FIG. 3 can be obtained. After the process of step S26 is completed, the process returns to step S6 of FIG.

By such processing, the image processing unit 16 performs image processing conventionally performed on the image data supplied from the identification result conversion processing unit 15 and indicating the probability that the pixel value of each dot is the extraction target object. Further, by doing so, it becomes possible to extract the object to be detected more accurately than the image identification result by the classifier having the learning result of the extraction object using the deep learning method.

On the other hand, the identification result conversion processing unit 15 determines the probability that each cell is an extraction target, which is detected by using the image identification processing unit 14 having the same ability as the classifier capable of outputting the extraction result. When converted into image data and processed by the image processing unit 16 having an image processing function reflecting the knowledge about the insect A existing in the vegetable W, it is possible to obtain an extraction result in which noise components close to the insect A are removed. It will be possible. In this way, it is possible to obtain an analysis result close to visual confirmation by using the image discrimination device 1.

The series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software can execute various functions by installing a computer embedded in dedicated hardware or various programs. It is installed from a program recording medium on a possible, eg, general purpose personal computer.

1 Image discrimination device (foreign matter discrimination unit)
2 Input unit 3 Output unit 11 Irradiation unit 111 Top surface irradiation unit 112 Bottom surface irradiation unit 12 Operation input information acquisition unit 13 Image data division processing unit (division division means)
14 Image identification processing unit 15 Identification result conversion processing unit 16 Image processing unit 17 Output processing unit 18 Storage unit 21 Image data acquisition unit 31 Foreign matter detection device 41 Conveyor A Extraction target (insects, foreign matter)
W Object to be photographed (object to be inspected)

Claims (8)

An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
While irradiating the object to be photographed with light and taking pictures in a plurality of states including a state in which the exposure is raised above the proper exposure, images with different exposures are obtained, and foreign matter is discriminated from the image data obtained from the image data acquisition unit. It is a foreign matter detection device
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device including an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit. An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
From the image data obtained from the image data acquisition unit, while irradiating light of the same color or similar to the object to be photographed and shooting in a plurality of states including a state in which the exposure is raised above the appropriate exposure, images with different exposures are obtained. It is a foreign matter detection device that discriminates foreign matter.
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device including an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit. An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
Images of different exposures can be obtained by irradiating white vegetables, which are the objects to be photographed, with purple light, blue light, or ultraviolet light and in multiple states including a state in which the exposure is raised above the proper exposure, and image data. A foreign matter detection device that discriminates foreign matter from image data obtained from the acquisition unit.
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
A foreign matter detection device including an image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit. The image processing unit performs image processing on the image data divided by the division division unit, and then performs image processing on the image data.
The image identification unit is characterized in that it calculates the probability of being the extraction target for each of the categories of the image data with respect to the image data image-processed by the image processing unit. The device comprises a foreign matter detection device according to any one of claims 2 and 3. For green vegetables
While irradiating with green light, take a picture with the exposure raised above the proper exposure.
Take in the reflected light of green light and obtain image data,
The foreign matter discriminating device according to any one of claim 1 or 2, claim 3 or claim 4, which discriminates a foreign substance from image data. For red vegetables
Irradiate red light or near infrared light, and raise the exposure above the proper exposure to shoot.
Take in red light or near infrared light to obtain image data,
The foreign matter discriminating device according to any one of claim 1 or 2, claim 3 or claim 4, which discriminates a foreign substance from image data. An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
The object to be photographed containing foreign matter is irradiated with light of different wavelengths from ultraviolet light to infrared light by continuously changing the light, and the exposure is raised above the appropriate exposure, and each photographed image is photographed in white. By calculating the area covered and comparing each image, it is determined that the frequency of light used for the whitest subject in the image is the frequency of light suitable for detecting foreign matter.
A foreign matter detection device that identifies the frequency of light required for foreign matter detection. An upper surface irradiation unit that irradiates the object to be inspected with light from the upper surface of a transparent or translucent conveyor that conveys the object to be inspected and the irradiation angle to the conveyor can be changed.
The bottom surface irradiation part that irradiates the object to be inspected with light from the bottom surface of the conveyor and the irradiation angle to the conveyor can be changed.
It has an image data acquisition unit that captures the object to be inspected from the top surface of the conveyor.
In addition to being able to move close to and away from the conveyor
Extracting the extraction target from the image data and performing image analysis
While irradiating the object to be photographed with light and taking pictures in a plurality of states including a state in which the exposure is raised above the proper exposure, images with different exposures are obtained, and foreign matter is discriminated from the image data obtained from the image data acquisition unit. It is a foreign matter detection device
A division division unit that divides the image data and
It has a machine learning result of the feature amount of the extraction target object by the deep learning method, and based on the machine learning result, the division is extracted for each division of the image data divided by the division division unit. An image identification unit that calculates the probability of being an object,
An identification result conversion unit that converts the probability of being the extraction target for each category into a pixel value, which is calculated by the image identification unit.
An image processing unit that performs image processing on an image having the pixel value converted by the identification result conversion unit is provided.
The light that illuminates the object to be photographed is
The object to be photographed containing foreign matter is irradiated with light of different wavelengths from ultraviolet light to infrared light by continuously changing the light, and the exposure is raised above the appropriate exposure, and each photographed image is photographed in white. By calculating the area covered and comparing each image, it is determined that the frequency of light used for the whitest subject in the image is the frequency of light suitable for detecting foreign matter.
A foreign matter detection device characterized in that the frequency of light required for foreign matter detection is specified.