JP2022160796A - Food product inspection system, food product inspection learning device and food product inspection method - Google Patents

Abstract

To provide a food product inspection system and food product inspection method which can quickly and easily reflect a determination criterion of a non-defective and a defective for a site worker on determination means of a food product inspection device.SOLUTION: A food product inspection system 100 comprises: a food product inspection device 110; and a food product inspection learning device 120. The food product inspection device 110 comprises: a first camera 21 which images a food product; and determination means 30 which determines whether a food product is non-defective or defective by receiving input of an image of the food product. The determination means 30 is constituted by a multi-layer neural network, and learned by using an image of a non-defective and/or an image of a defective in advance. The food product inspection learning device 120 comprises: a second camera 22 which images the hand of the worker; and storage means 50 which stores an image equivalent to the defective that is discarded among the images captured by the first camera 21 when the worker determines that the food product is the defective and discards the defective. The determination means 30 is re-learned by using the stored image of the defective.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to a food inspection system, a food inspection learning device, and a food inspection method for distinguishing good food from defective food such as foreign matter.

The following inventions have been filed with respect to an apparatus and method for inspecting food.
Patent Literature 1 (International Publication No. 2018/038123) discloses a food inspection device capable of detecting foreign matter at high speed.

The food inspection apparatus described in Patent Document 1 includes conveying means, light irradiation means, imaging means for imaging an image of an object to be inspected, a first optical filter and/or a second optical filter, or a wavelength-specific using a light source of 300 nm or more and 1100 nm or less to emphasize a foreign matter specific wavelength characteristic of the foreign matter; are normalized by gradation of 256 steps or less according to the light intensity to obtain weight reduction data; The identifying means identifies foreign matter or non-defective products in-line from weight reduction data obtained by imaging while the inspection object is being conveyed.

In addition, Patent Document 2 (Japanese Patent Application Laid-Open No. 2020-103284) discloses a discriminating device that discriminates a foreign substance or a different type of seaweed from seaweeds in which foreign substances or different types of seaweed are mixed.

The discriminating device described in Patent Document 2 includes a visible light imaging unit that irradiates fishing line, shrimp, fish, shellfish, or seaweed mixed with different types of seaweed with visible light and incorporates the reflected light of the visible light to obtain a visible light image. , an ultraviolet light imaging unit that obtains an ultraviolet light image by irradiating fishing line, shrimp, fish, shellfish, or seaweed mixed with different types of seaweed with ultraviolet light and capturing the reflected light of the ultraviolet light, and extracting from image data obtained from A device for discriminating fishing line, shrimp, fish, shellfish, or different kinds of seaweed from fishing line, shrimp, fish, shellfish, or seaweed mixed with different kinds of seaweed when extracting an object. has segmentation means for segmenting image data and machine learning results of feature amounts of extraction targets by deep learning techniques, and image data segmented by the segmentation means based on the machine learning results image identification means for calculating, for each section, the probability that the section is an extraction target object; identification result conversion means for converting the probability that each section is an extraction target object calculated by the image identification means into a pixel value; and image processing means for performing image processing on an image having pixel values converted by the result conversion means.

Further, Patent Document 3 (Japanese Patent Application Laid-Open No. 2020-125974) discloses a food inspection device capable of detecting foreign matter inside a solid-liquid mixture.

The food inspection apparatus described in Patent Document 3 includes a transport section that transports an inspection object in which a liquid portion and a solid portion are mixed so as to have a thickness, and a plurality of wavelengths directed toward the inspection object on the transport portion. a light irradiating unit that irradiates light containing light, an image acquiring unit that captures an image of the area irradiated with light and acquires a plurality of images based on light according to a plurality of wavelengths, and an inspection in the thickness direction based on the plurality of images and a food inspection processing unit that detects foreign matter Fn that may be contained inside the surface of the object.

In addition, the following invention has been filed for a method of generating a learning model for judging non-defective products and defective products.
Patent Document 4 (Japanese Patent Application Laid-Open No. 2020-181333) discloses a method of generating a learning model that enables inspection with appropriate quality sensitivity.

The method of generating a trained model in Patent Document 4 obtains information on factors that have determined that the delivery destination is defective for a product that has been determined as a non-defective product by a trained model that outputs quality information when a product image is input. , based on the obtained factor information, the trained model is re-learned.

International Publication No. 2018/038123 JP 2020-103284 A JP 2020-125974 A JP 2020-181333 A

In food inspection equipment, when AI such as a multi-layer neural network is used to distinguish between non-defective and non-defective food images, the multi-layer neural network is trained using images of specific non-defective and defective products. Even if food is inspected using a multi-layered neural network, there are cases where foods that should be judged as defective are mixed in with foods that the multi-layered neural network judges as non-defective, or that non-defective products are erroneously judged as defective. .
In this case, if the criteria for judging the inspected foods as non-defective products are stricter, the inclusion of defective products can be prevented to some extent. Therefore, in order to accurately determine good products and defective products, a mechanism is required to reflect the judgment of good products and defective products by a skilled worker in a multi-layer neural network.

In addition, which foods are considered non-defective and which foods are defective differ depending on the supplier of each food. In this case, it is conceivable that the company that manufactures the food inspection equipment receives the image of the good product and the image of the defective product from the company that provides the food as the criteria for judging good products and defective products, and reflects them in the multi-layer neural network. It is actually difficult for the company that provides the products (the user of the food inspection device) to provide suitable images of defective products without omission.
Therefore, there is a need for a system that collects information on images of good and bad products at the actual site of determining good and bad products in food companies and reflects that information in a multi-layer neural network. be.

The food inspection apparatus described in Patent Literature 1 does not incorporate a mechanism for reflecting the determination of non-defective products and defective products by workers on site in a multi-layer neural network.
The same applies to the discriminating device for discriminating foreign matter or different types of seaweed described in Patent Document 2 and the food inspection device described in Patent Document 3.
In the method for generating a trained model described in Patent Document 4, a mechanism is incorporated in which factor information on which the delivery destination is determined to be defective is acquired, and the trained model is re-learned based on the acquired factor information. However, it is difficult to quickly relearn and update criteria for judging non-defective products and defective products because it takes time to communicate with delivery destinations.

The main object of the present invention is to provide a food inspection system and a food inspection method for discriminating good food products from defective products such as foreign substances, in which a food inspection system and a food inspection method can quickly and easily determine criteria for determining good products and defective products by workers on site. An object of the present invention is to provide a food inspection system and a food inspection method that can be reflected in the determination means of an apparatus.
A second object of the present invention is to provide a food inspection system and a food inspection method for distinguishing between non-defective products and defective products such as foreign substances. It is an object of the present invention to provide a food inspection system and a food inspection method capable of quickly and easily reflecting criteria for judging non-defective products and defective products to judging means of a food inspection apparatus.

(1)
A food inspection system according to one aspect is a food inspection system including a food inspection device, a food inspection learning device, and a transport means, wherein the transport means transports food from the food inspection device to the food inspection learning device, the food inspection device comprises a first photographing means for photographing the food on the conveying means, a judgment means for inputting the image of the food photographed by the first photographing means and judging whether it is a non-defective product or a defective product; a removing means for removing the judged food from the conveying means, the judging means is composed of a multi-layer neural network, and is learned in advance using images of non-defective products and/or images of defective products, and the food inspection learning device comprises: A second photographing means for photographing the hand of the worker and an image photographed by the second photographing means are input, and when the worker determines that the food is defective and discards it, the first photographing means photographs the food. storage means for storing an image corresponding to the discarded defective product from among the images obtained, and the image of the defective product stored in the storage means and the image of the good product (prepared in advance) and/or the image of the defective product. The images are used to relearn the judgment means of the food inspection device.

In this case, the determination means of the food inspection apparatus removes clearly defective items such as foreign matter by learning using images of non-defective items and/or images of defective items prepared in advance. Then, from among the foods judged to be non-defective by the food inspection equipment, the food that the operator judged to be defective according to the inspection standards of the company is discarded, and the image of the discarded defective product is saved and saved. The image of the defective product is used to relearn the determination means of the food inspection device. As a result, it is possible to quickly and easily reflect the judgment criteria of the on-site worker in the judgment means of the food inspection apparatus.
In this case, it is not impossible for the operator to directly identify all foods by the food inspection learning device without going through the food inspection device. It is desirable that the food inspection device removes obviously defective products, and the food that has passed through the food inspection device is identified by the food inspection learning device by the operator.
Food inspection not only removes clearly defective products such as foreign substances, but also determines the criteria for good/defective products at the inspection stage, depending on the purpose of the food, the purpose of processing, and the processing stage (for example, whether or not it is fried afterward). is different for each company. Therefore, first, the food inspection equipment removes obvious defective products such as foreign matter, and the operator selects foods that do not meet the inspection standards. And it is possible to accurately judge the food inspection.

In addition, in the judgment criteria of the initial judgment means that learns by inputting images of good products and/or images of defective products in advance, images of good products are learned as good products (and/or images of clearly defective products such as foreign substances are learned). Let it learn that it is defective). In this way, foods near the boundary between good and defective products are judged to be good products. It is desirable to re-learn so that foods that should be judged as defective.
Then, when the accuracy of judgment of non-defective products by the food inspection device is improved to a predetermined level by relearning the judgment means of the food inspection device, it is possible to ship the non-defective products of the food inspection device as non-defective products. .

(2)
A food inspection system according to a second aspect of the present invention is a food inspection system according to one aspect, wherein an image to be determined as a defective product by a food inspection apparatus is further selected from images of defective products stored in a storage means, It is used for re-learning of the determination means of the food inspection device, and the re-learning of the determination means may be performed for each predetermined inspection time or predetermined number of inspected foods.

In this case, an image of a defective product to be re-learned is selected from a group of images of defective products judged to be defective at the work site (inspection line worker), and the judgment means re-learns this. Let
In the food inspection learning device, the operator needs to instantly identify defective products. Therefore, among the foods that the operator judged to be defective and whose images were saved, food that should have been judged to be non-defective was included. there may be. In addition, since the inspection of food depends on the senses of the operator, the determination of good/defective products is not always unequivocally determined, and the inspection results may vary depending on the operator. Therefore, it is desirable to select images to be judged as defective by the food inspection apparatus from among the images of defective products stored in the storage means, and to use those images for re-learning of the judgment means of the food inspection device. .
In addition, in order to obtain the effect of re-learning, it is necessary to collect a certain number of additional images to be judged as defective products. It is desirable to do this for each number of
By selecting an image of a defective product to be re-learned and/or an image of a non-defective product erroneously determined as a foreign object from a group of images determined to be defective on the inspection line, the judgment criteria for the inspection line are determined. can be reflected more accurately.

(3)
A food inspection system according to a third aspect of the invention is the food inspection system according to one aspect or the second aspect of the invention, wherein the food inspection apparatus further comprises light irradiation means for irradiating the food photographed by the first photographing means, The illumination means may emphasize light of a particular wavelength.

Food that is determined to be defective by inspection may have foreign substances such as insects and metals adhering to it. Since such foreign matter strongly reflects light of a specific wavelength, by using a light source that emphasizes light of a specific wavelength as a light irradiation means and photographing images of food, defective products with foreign matter attached and non-defective products with foreign matter attached can be identified. can make the difference clearer.
Instead of using a light source that emphasizes light of a specific wavelength, a filter that emphasizes light of a specific wavelength may be arranged in front of the photographing means.
Alternatively, a circuit having a filter function for emphasizing light of a specific wavelength may be inserted as a pre-processing of the input to the multi-layer neural network of the judging means.

(4)
A food inspection system according to a fourth invention is the food inspection system according to the first aspect to the third invention, wherein the training data of the multi-layer neural network has different labels for each image of good products and different kinds of defective products. May be specified.

As the multi-layer neural network, a network capable of inputting image data and extracting its features may be used.

Also, in training the multi-layer neural network, different labels may be specified for each image of a non-defective product and/or different types of defective products, and the multi-layer neural network may be trained.
In this case, when images of foods to be judged are input to the multi-layer neural network, the labels of those foods are extracted, making it possible not only to judge good and bad products but also to detect the types of defective products.

In addition, in the multi-layer neural network, even if a part of the food contains an abnormal part (crushed, discolored, spoiled, etc.), the food containing the abnormal part is not judged as defective. The whole can be judged as defective.
Therefore, by using a multi-layer neural network, it is possible to more accurately sort out defective products including abnormal parts, and to efficiently and accurately inspect foods that do not meet the inspection standards.

(5)
A food inspection learning device according to another aspect is an inspection learning device for learning judgment means for judging whether a food product is a non-defective product or a non-defective product, which comprises a multi-layer neural network, and comprises a transporter for transporting the food subject to inspection learning. means, a first photographing means for photographing the food on the conveying means, a second photographing means for photographing the hand of the worker, and an image photographed by the second photographing means is input, and the worker detects a defective product. storage means for storing an image corresponding to the discarded defective product among the images taken by the first imaging means when the defective product is discarded after determining that the defective product is discarded, and the defective product image stored in the storage means is used for re-learning of the decision means.

In this case, by collecting images of defective products using the food inspection learning device, it is possible to quickly and easily reflect the judgment criteria of the on-site worker in the judgment means of the food inspection device. In addition, each company has different standards for determining good/defective products at the inspection stage. By re-learning, the determination means of the food inspection apparatus can be quickly and easily adapted to the determination criteria of the company.

(6)
A food inspection method according to still another aspect is a food inspection method for inputting an image of food and determining whether it is a non-defective product or not. A first learning step of learning to determine whether a product is good or defective, a first determination step of photographing an image of food and inputting the image of food into a multi-layer neural network to determine whether it is a good product or a defective product; A second judgment step in which the operator removes defective products from the food determined to be non-defective in the first judgment step, an image of the defective product removed in the second judgment step, and an image of the non-defective product (prepared in advance) and / or and a second training step of retraining the multilayer neural network using the images of the defective products.

In this case, after having the multi-layer neural network learn the basic criteria for judging non-defective and defective products in the first learning process, the company uses the judgment of the on-site worker or the food inspection method in the second learning process. It is possible to re-learn so that the food that should be judged to be defective is judged to be defective according to each determination. By re-learning the multi-layered neural network, the judgment criteria in the first judgment step can correspond to the judgment of the worker on site or the judgment of each company that uses the food inspection method.
Then, when the accuracy of judgment in the first judging process is improved to a predetermined level by re-learning in the second learning process, it is also possible to ship the non-defective product in the first judging process as a non-defective product.
In this case, it is not impossible for the worker to directly determine all the foods in the second determination step without going through the first determination step, but from the viewpoint of reducing the burden on the worker and ensuring concentration, It is desirable that the first determination step is performed first, and the non-defective product is determined by the operator in the second determination step.

It is a mimetic diagram showing the whole food inspection system composition of a 1st embodiment. It is a schematic diagram which shows an example of the foodstuffs mounted in the conveyance means of a food inspection system. It is a schematic diagram which shows the flowchart of a food inspection system. It is a schematic diagram which shows the structure of the food inspection learning apparatus of 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Moreover, in the case of the same reference numerals, their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[First Embodiment]
FIG. 1 is a schematic diagram showing the overall configuration of a food inspection system 100 of the first embodiment.
A food inspection system 100 comprises a belt conveyor 11 , a food inspection device 110 and a food inspection learning device 120 . The belt conveyor 11 is arranged across the food inspection device 110 and the food inspection learning device 120 and conveys the food 60 from the food inspection device 110 to the food inspection learning device 120 .

(Food inspection device 110)
The food inspection device 110 includes a belt conveyor 11, a first camera 21, an illumination device 24, a determination means 30, and a removal means 40. The first camera 21 and the illumination device 24 are arranged above the belt conveyor 11 , and the removing means 40 is arranged on the side of the belt conveyor 11 .
The food 60 placed on the belt conveyor 11 is imaged by the first camera 21 while being illuminated by the illumination device 24 . The photographed image of the food 60 is input to the determination means 30 to determine whether it is a good product or a defective product. The judging means 30 is composed of a multi-layered neural network and is trained in advance using images of non-defective products and/or images of defective products.

In the preliminary learning stage, a plurality of images of only non-defective products may be used to learn non-defective products. Moreover, in the preliminary learning stage, in addition to learning only non-defective products, only images of clearly defective products such as the foreign substances 62 and 63 may be learned as defective products.
By doing so, only obvious defective products such as the foreign substances 62 and 63 are determined to be defective before the re-learning is started. Then, each company's own inspection standard (food 60 near the boundary between non-defective and defective products) provided for each inspection process is that the operator 70 sorts the non-defective product and the defective product, and the selected food 60 is regarded as a defective product. Re-learning enables efficient and accurate food inspection judgments.

Information that the food 60 is defective is sent from the determination means 30 to the removal means 40 , and the food 60 determined as defective is removed from the belt conveyor 11 by the removal means 40 . The removing means 40 is not particularly limited. For example, an actuator is arranged on one side of the belt conveyor 11, and a piston rod pushes out defective products to the other side, or an air gun blows out defective products. Configurations, etc. can be used.

When the food 60 is photographed by the first camera 21, the lighting device 24 emits light to photograph the food 60. - 特許庁The illumination device 24 may use a light source that emphasizes light of a specific wavelength as the light source itself, or may have a filter 24a that emphasizes light of a specific wavelength in front of the light source as shown in FIG. desirable. This is because if foreign matter such as insects or metal adheres to the food 60, such foreign matter strongly reflects light of a specific wavelength.
Instead of using the lighting device 24 that emphasizes light of a specific wavelength, a filter that emphasizes light of a specific wavelength may be arranged in front of the first camera 21 .
Alternatively, as preprocessing of the input to the multi-layer neural network of the determination means 30, a circuit having a filter function that emphasizes light of a specific wavelength may be inserted.

(Food inspection learning device 120)
The food inspection learning device 120 is composed of the belt conveyor 11 , the second camera 22 , the storage means 50 and the disposal box 71 . The second camera 22 is arranged on the rear upper part of the belt conveyor 11 . A worker 70 is positioned near the second camera 22 on the side of the belt conveyor 11 .
The food 60 determined as non-defective by the food inspection device 110 is conveyed to the food inspection learning device 120 .

The food 60 conveyed to the food inspection learning device 120 is conveyed to the front of the worker 70, and the worker 70 judges whether the food 60 on the front is good or defective. to discard. The second camera 22 captures the movement of the hand of the worker 70, the storage means 50 inputs the image captured by the second camera 22, and when the worker 70 discards the food 60, the discarded time Therefore, which food 60 is the food 60 photographed by the first camera 21 is detected, and the image of the food 60 is stored as the image of the defective product.
In this embodiment, the first camera 21 and the second camera 22 are two independent cameras, but a single camera such as a wide-angle camera may be used. In this case, the camera photographs the food 60 on the belt conveyor 11 and the hand of the worker 70 at once.

(Determination means 30)
The multi-layer neural network of the determination means 30 of the present embodiment may be any one that can input image data and extract its features. Net) or ResNet (Residual Network) can be used. This multi-layer neural network is trained in advance using basic images of good products, or images of good products and images of bad products.
Yolo models object detection as a regression problem, and has high accuracy in object detection and is capable of real-time processing.
Efficient nets have a smaller number of parameters (the product of the size of one layer, the number of layers, and the resolution) compared to conventional multi-layer neural networks, making it possible to speed up the determination of non-defective products. It is preferable in that it can be done.
ResNet can deepen layers by learning residual features.

FIG. 2 shows an example of food 60 placed on the belt conveyor 11 of the food inspection system 100. As shown in FIG. In FIG. 2, as an example, a non-defective product 61, foreign substances (defective products) 62 and 63, and a partially discolored defective product 64 are placed on the belt conveyor 11 .
Each food 60 is sequentially photographed by the first camera 21 . Reference numeral 25 in FIG. 2 indicates the range of the image captured by the first camera 21 . The image of this range 25 is input to the determination means 30 to determine whether the product is good or defective. The food 60 other than non-defective products is removed from the belt conveyor 11 by the removal means 40. - 特許庁

Teacher data for the multi-layer neural network is created by assigning a label to each food 60 image. As a label, it is natural to classify non-defective products and defective products. It is desirable to specify different labels for each of them and use them as teacher data.
For example, even if a portion of the food 60 is crushed, there may be no problem in terms of inspection standards, such as mincing in the next step. On the other hand, if the food 60 contains even a small amount of rotten parts, the food 60 as a whole may be determined to be defective. In addition, the threshold value of the determination probability may be changed according to the classification of the defect, such as determining the product as defective even when the determination probability is low in the case of spoilage compared to the case of discoloration. In the multi-layer neural network, defects in the food 60 can be classified by label, so food inspection can be performed more accurately.
Furthermore, the multi-layer neural network can grasp the occurrence status of classified defective products in real time. Therefore, it is possible to effectively grasp process abnormalities such as problems in temperature control in the upstream process due to an increase in the frequency of discoloration of the food 60, for example.

In addition, when the inspection of the food 60 by the food inspection system 100 has passed a predetermined inspection time, or when the inspection of the predetermined number of items has been completed, the image of the good product and the defective product used in the learning of the judgment means 30 in advance is stored in the storage means 50. It is desirable to relearn the determination means 30 by adding the stored images of defective products.
For example, even if the food 60 is partially discolored, whether the food 60 is a good product or a defective product depends on the degree of discoloration, color, size, etc., depending on the purpose of the food 60 and the purpose of processing. , differ from company to company depending on the stage of processing. Therefore, by selecting images of defective products to be re-learned from a group of images determined to be defective products on the inspection line, it is possible to more accurately and effectively reflect the judgment criteria of the inspection line. can.
Then, by re-learning the determination means 30, the determination means 30 can make a determination that matches the judgment of the on-site worker 70 or the judgment of each company that uses the food inspection system 100. Become.

In addition, since the worker 70 in the food inspection learning device 120 must instantly determine whether the product is good or bad, there is a possibility that the worker 70 may judge a good product as a bad product. Therefore, when using the image of the defective product stored in the storage means 50 for re-learning of the judgment means 30, the image to be judged as the defective product by the food inspection apparatus 110 is selected from the image of the defective product, and the food inspection is performed. It is desirable to use it for re-learning of the determination means 30 of the device 110 . In this case, it is desirable to designate an appropriate label for each image to be determined as defective in order to improve the accuracy of the determination criteria.
Further, when the determination accuracy of the determination means 30 is sufficiently increased by repeating the relearning of the determination means 30 of the food inspection device 110, the non-defective products of the food inspection device 110 are not passed through the food inspection learning device 120. It may be shipped as a non-defective product.

In this embodiment, an efficient net is used as the multilayer neural network of the determination means 30, but the multilayer neural network is not limited to the efficient net. A multilayer neural network other than an efficient network may be used as the determination means 30 as long as it can input image data and extract its features (labels).

(Flow chart of food inspection system 100)
In FIG. 3, the schematic diagram of the flowchart of the food inspection system 100 is shown.
Each step in FIG. 3 will be described below.
Step S1: Images of non-defective products and defective products used for learning of the multi-layer neural network of the judging means 30 are prepared. In this case, in the initial learning, only good images may be prepared for learning.
Step S2: Add a label to the prepared image and let the multi-layer neural network of the determination means 30 learn. The number of images at this time is preferably at least 10, preferably 100 or more. Also, at this stage, it is desirable to prepare training data so that the food 60 near the boundary between the non-defective product and the defective product is determined to be non-defective.
Step S3: Place the food 60 to be inspected on the belt conveyor 11. FIG.
Step S4: The image of the food 60 is photographed by the first camera 21, and the judging means 30 judges whether it is a non-defective product or not.
Step S5: If the determination result of the determination means 30 is that the defective product is defective, the defective product is removed from the belt conveyor 11 by the removing means 40 .
Step S6: If the determination means 30 has already been re-learned and the determination accuracy of the determination means 30 of the food inspection device 110 is sufficiently high, the product is shipped as a non-defective product (step S12). In this case, steps S3 to S6 and non-defective product shipment (step S12) are repeated thereafter.

Step S7: If the determination accuracy of the determination means 30 of the food inspection device 110 is not sufficient, or if it is desired to reflect the determination criteria of the company's good products and defective products in the determination means 30 of the food inspection device 110, the food The non-defective product of the inspection device 110 is conveyed to the food inspection learning device 120 , and the operator 70 judges whether the product is non-defective or defective, and the second camera 22 photographs the hand of the operator 70 . The food 60 determined by the operator 70 to be non-defective is shipped (step S12).
Step S8: When the worker 70 discards the food 60, detect which food 60 is the food 60 photographed by the first camera 21 from the time when the food 60 was discarded, The image of the food 60 is saved as an image of the defective product.
Step S9: Steps S3 (food transportation) to step S8 (defective product image storage) are repeated until the inspection time exceeds a predetermined time or until the number of inspected items exceeds a predetermined number.
Step S10: If the inspection time exceeds a predetermined time, or if the number of inspections exceeds a predetermined number, steps S3 to S8 are temporarily stopped, and the image stored in the storage means 50 is used for determination. Means 30 selects images of defective products to be used for re-learning.
Step S11: Add a label to the image of the defective product stored in the storage means 50, combine it with the images of the good product and the defective product prepared in step S1, and relearn the multi-layer neural network of the judgment means 30. FIG. After the re-learning is completed, the inspection of the food 60 and the storage of the defective product image for re-learning are repeated along steps S3 to S8.

[Second embodiment]
The food inspection learning device 120 a of the second embodiment is a device that collects teacher data for re-learning the determination means 30 of the food inspection device 110 . Therefore, the food inspection device 110 is not included.
FIG. 4 is a schematic diagram showing the configuration of a food inspection learning device 120a of the second embodiment. A food inspection learning device 120a of the second embodiment comprises a belt conveyor 11a, a first camera 21a, a second camera 22, storage means 50, and a disposal box 71. FIG. The second camera 22 is arranged in the rear upper part of the belt conveyor 11a. A worker 70 is positioned near the second camera 22 on the side of the belt conveyor 11 .

The image of the food 60 placed on the food inspection learning device 120a is captured by the first camera 21a. The imaged food 60 is conveyed to the front of the worker 70, and the worker 70 judges whether the food 60 on the front is good or defective, and discards the food 60 in a disposal box 71 if it is defective. . The second camera 22 captures the movement of the hand of the worker 70, the storage means 50 inputs the image captured by the second camera 22, and when the worker 70 discards the food 60, the discarded time Therefore, which food 60 is the food 60 photographed by the first camera 21a is detected, and the image of the food 60 is stored as the image of the defective product.

Images of defective products collected by the food inspection learning device 120a are provided to the food inspection device 110 as re-learning data.
Since the worker 70 in the food inspection learning device 120a must instantly judge whether the product is good or bad, there is a possibility that the worker 70 will judge a good product as a bad product. Therefore, when using the image of the defective product stored in the storage means 50 for re-learning of the judgment means 30, the image to be judged as the defective product by the food inspection apparatus 110 is selected from the image of the defective product, and the food inspection is performed. It is desirable to use it for re-learning of the determination means 30 of the device 110 . In this case, it is desirable to designate an appropriate label for each image to be determined as defective in order to improve the accuracy of the determination criteria.

In the present invention, the food inspection device 110 corresponds to the "food inspection device", the food inspection learning devices 120 and 120a correspond to the "food inspection learning device", the food inspection system 100 corresponds to the "food inspection system", The food 60 corresponds to "food", the belt conveyors 11 and 11a correspond to "conveyance means", the first cameras 21 and 21a correspond to "first photographing means", and the determination means 30 corresponds to "determination means , the removing means 40 corresponds to the ``removing means'', the second camera 22 corresponds to the ``second photographing means'', the worker 70 corresponds to the ``worker'', and the storage means 50 corresponds to It corresponds to the "storage means", and the illumination device 24 corresponds to the "light irradiation means".

Although one preferred embodiment of the invention is described above, the invention is not so limited. It is understood that various other embodiments can be made without departing from the spirit and scope of the invention. Furthermore, in this embodiment, the actions and effects of the configuration of the present invention are described, but these actions and effects are examples and do not limit the present invention.

11, 11a Belt conveyor 21, 21a First camera 22 Second camera 24 Lighting device 30 Determination means 40 Removal means 50 Preservation means 60 Food 70 Worker 100 Food inspection system 110 Food inspection device 120, 120a Food inspection learning device

Claims (6)

A food inspection system including a food inspection device, a food inspection learning device, and a conveying means,
the conveying means conveys the food from the food inspection device to the food inspection learning device;
The food inspection device is
a first photographing means for photographing the food on the conveying means;
Judgment means for inputting an image of food photographed by the first photographing means and judging whether it is a non-defective product or a defective product;
a removing means for removing the food determined as defective by the determining means from the conveying means,
The determination means is composed of a multi-layer neural network and is learned in advance using images of good products and/or images of defective products,
The food inspection learning device is
a second photographing means for photographing the hand of the worker;
When an image photographed by the second photographing means is input, and the worker determines that the food is defective and discards the food, among the images photographed by the first photographing means, the discarded defective product is selected. and storage means for storing the relevant image,
A food inspection system, wherein the determination means of the food inspection device is re-learned using the image of the defective product stored in the storage means and the image of the good product and/or the image of the defective product. An image to be determined as a defective product by the food inspection device is further selected from the images of defective products stored in the storage means, and used for re-learning of the determination means of the food inspection device,
2. The food inspection system according to claim 1, wherein the re-learning of said judgment means is performed every predetermined inspection time or every predetermined number of inspected foods. The food inspection device further comprises light irradiation means for irradiating the food photographed by the first photographing means,
3. The food inspection system according to claim 1, wherein said light irradiation means emphasizes light of a specific wavelength. 4. The food inspection system according to any one of claims 1 to 3, wherein the training data of said multi-layer neural network designates different labels for images of good products and images of different kinds of defective products. An inspection learning device for learning judgment means for judging whether food is good or bad, comprising a multi-layer neural network,
a conveying means for conveying food to be inspected and learned;
a first photographing means for photographing the food on the conveying means;
a second photographing means for photographing the hand of the worker;
When the image photographed by the second photographing means is inputted and the worker determines that the product is defective and discards it, the image photographed by the first photographing means corresponds to the discarded defective product. a storage means for storing the image;
A food inspection learning device, wherein images of defective products stored in the storage means are used for re-learning of the judgment means. A food inspection method for inputting an image of food and determining whether it is a good product or a defective product,
a first learning step of making a multi-layered neural network learn to determine whether a product is good or bad, using images of non-defective products and/or images of defective products as training data;
a first determination step of capturing an image of the food, inputting the image of the food into the multi-layer neural network, and determining whether it is a good product or a defective product;
A second judgment step in which an operator removes defective products from the foods determined to be non-defective in the first judgment step;
A food inspection method, comprising a second learning step of re-learning the multilayer neural network using the image of the defective product removed in the second determination step and the image of the good product and/or the image of the defective product. .

Images