WO2022191091A1

WO2022191091A1 - Food inspection system, food inspection method, and computer program

Info

Publication number: WO2022191091A1
Application number: PCT/JP2022/009569
Authority: WO
Inventors: 一樹内田; 真也塚本
Original assignee: 株式会社ニチレイフーズ
Priority date: 2021-03-08
Filing date: 2022-03-04
Publication date: 2022-09-15
Also published as: JPWO2022191091A1

Abstract

[Problem] To provide a food inspection system and a food inspection method with which it is possible to classify processed food into a type other than non-defective items and defective items. [Solution] The food inspection system according to the present disclosure comprises a sensor device that senses processed food being conveyed, and a determination unit that determines whether the processed food is a correctable item that can be corrected into a non-defective item by a correcting work on the basis of information detected by the sensor device.

Description

FOOD INSPECTION SYSTEM, FOOD INSPECTION METHOD AND COMPUTER PROGRAM

This disclosure relates to a food inspection system, food inspection method, and computer program.

Within production factories, the number of cases of automating and mechanizing various tasks is increasing. However, visual inspections of finished products and in-process products are often visually inspected by humans. It is desirable to save manpower and automate this work.

In visual inspection, a technology is known that automatically distinguishes defective products from non-defective products. For example, using two thresholds, the items to be sorted are classified into three categories: good products, defective products, and intermediate products. Optical sorting devices are known.

However, when the object of inspection is processed food, it is often insufficient to classify it into two types, good and bad. That is, it may be necessary to classify processed foods into categories other than good and bad. In this case, the above-described technique for judging non-defective products and defective products cannot be used.

Japanese Patent No. 5673109

The present disclosure provides a food inspection system, food inspection method, and computer program capable of classifying processed foods into types other than good products and defective products.

The food inspection system of the present disclosure includes a sensor device that senses processed food being conveyed, and based on information detected by the sensor device, whether the processed food corresponds to a correctable product that can be corrected to a non-defective product by correcting work. and a determination unit that determines whether or not.

The food inspection method of the present disclosure senses the conveyed processed food with a sensor device, and based on the information detected by the sensor device, determines whether the processed food corresponds to a correctable product that can be corrected to a non-defective product. judge.

A computer program of the present disclosure acquires information detected by sensing processed food being transported;
Based on the information, determining whether the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work;
run on the computer.

1 is a block diagram of a food inspection system according to this embodiment; FIG. The figure which shows the example which conveys the processed food formed with the winding apparatus with a conveying apparatus. The figure which shows the example which a food inspection system communicates with a terminal device and a processing apparatus. The figure which shows the example of a non-defective product. The figure which shows the example of a modifiable article. The figure which shows the example of a defective article. The figure which shows the example which moves the processed food classified into the modifiable goods to another lane. A diagram showing an example of moving processed foods classified as defective products to other lanes. The figure which shows the example which a different robot each performs the movement between several lanes. FIG. 10 is a diagram showing an example of distributing repairable products to a plurality of lanes according to defect types; FIG. 4 is a diagram showing an example of distributing defective products to a plurality of lanes according to defect types; 4 is a flow chart showing an example of the operation of the food inspection system according to the embodiment; FIG. 11 is a diagram for explaining a specific example of modification 2; FIG. 11 is a diagram for explaining a specific example of Modification 3;

Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding elements are denoted by the same reference numerals, and detailed description thereof will be omitted as appropriate.

FIG. 1 is a block diagram of a food inspection system 1 according to this embodiment. FIG. 2 is a plan view showing an example of conveying spring rolls, which are processed foods F formed by the winding device 200, by the conveying device 201 in the processed food factory. The processed food F is, for example, a food in the process of being manufactured. The food inspection system 1 of FIG. 1 is placed in a processed food factory and performs inspection (for example, appearance inspection) of processed food transported by a transport device 201 . Appearance inspection is an inspection regarding the appearance or shape of processed foods. A part or all of the processed food transported by the transport device 201 becomes an inspection target (inspection target food) of the food inspection system 1 . The food inspection system 1 of FIG. 1 may be divided into a plurality and arranged via a wide area network such as the Internet or a local network such as a wireless LAN (Wireless Local Area Network).

The winding device 200 in FIG. 2 wraps spring roll ingredients with spring roll wrappers, forms them into a rectangular shape, and drops them onto the conveyor belt 202 of the conveyor device 201 . In the illustrated example, the processed foods F are conveyed in three rows. The conveying device 201 in FIG. 2 conveys the processed food fed by the winding device 200 to the downstream side by means of a belt. In the example of FIG. 2, the conveying device 201 conveys the processed food F at a constant speed in the downward direction indicated by the arrowed line. The transport source side (winding device 200 side) is called the upstream side, and the transport direction side is called the downstream side. The belt may be temporarily stopped for inspection at regular intervals, or the belt may be driven all the time. Various conveyor devices such as a belt conveyor, a chain conveyor, a roller conveyor, a mesh conveyor, and a gravity conveyor can be used as the conveying device 201 . The winding device 200 is an example of a food processing device. Examples of other processing equipment include molding equipment that shapes and weighs food, powder coating equipment that attaches coating materials, butterers, breaders, stuffers that perform casing, filling equipment that wraps ingredients, and food products. Examples include a topping machine for topping ingredients.

In this embodiment, spring rolls are treated as processed foods, but processed foods to be inspected are not limited to specific foods. For example, the processed food may be fried rice, croquette, mince cutlet, fried chicken, fried chicken, hamburger steak, rice ball, dumplings, Chinese steamed buns, steamed dumplings, pizza, gratin, and the like.

The food inspection system 1 of FIG. 1 includes a food inspection device 10, an imaging device 20, a sorting device 30, a display device 40 and an alarm device 50. The elements of the food inspection system 1 of FIG. 1 are not limited to those shown in FIG. 1, and elements other than those shown in FIG. 1 may be provided. For example, the food inspection system 1 may include an input device operated by a user. Examples of input devices include touch panels, mice, keyboards, wearable devices, and the like. The users are, for example, workers, supervisors, managers, etc. in a processed food factory.

The food inspection device 10 uses the imaging device 20 to image the processed food F transported by the transport device 201, and inspects the processed food F (for example, visual inspection). For example, it is determined whether the processed food F is a non-defective product, a defective product, or a correctable product. Modifiable products are processed foods that can be modified into good products by modification work. The food inspection device 10 controls the sorting device 30 and sorts the processed food F transported by the transporting device 201 to processes according to the determination result. In addition, the food inspection apparatus 10 transmits information according to the determination result, for example, notification information that a defective product or a correctable product has been detected to the user's terminal device, or when a defective product is detected, an alarm output a signal. The food inspection device 10 also transmits an operation parameter instruction signal to a processing device (for example, a winding device, etc.) that processed the food when a correctable product or a defective product is detected.

Such a food inspection device 10 includes a storage unit 11, an image processing unit 12, a control unit 13, a determination unit 14, an information generation unit 15, an alarm unit 16 and a communication unit 17.

The communication unit 17 communicates with a terminal device operated by a user such as a worker or an observer, and a device (processing device such as the winding device 200, etc.) that performs each process. The communication unit 17 may perform either wired or wireless communication.

FIG. 3 shows an example in which the food inspection system 1 communicates with the terminal device 2 and the processing device 3. Examples of the terminal device 2 include a notebook PC (Personal Computer), a smart phone, a desktop PC, a tablet terminal, and the like. FIG. 3 shows an example of communicating with the processing device 3 as a device that performs processing of each step. The processing device 3 may be the winding device 200 in FIG. 3, or other devices (devices for cooking ingredients, devices for producing skins, devices for heating with oil, devices for freezing processed foods after heating, etc.). good. Apparatuses that perform each step may include, in addition to the processing apparatus, inspection apparatuses that perform inspections other than the inspections targeted by the food inspection system 1 (for example, weight inspections), the sorting apparatus 30, and the like.

The control unit 13 controls the entire food inspection device 10, the imaging device 20, the sorting device 30, the display device 40, and the alarm device 50. The control unit 13 may be configured by a processor such as a CPU, or may be configured by dedicated hardware.

The imaging device 20 is a sensor device that senses the processed food conveyed by the conveying device 201 and detects information. The imaging device 20 is arranged at a position capable of imaging the processed food F conveyed by the conveying device 201 . The imaging device 20 captures an image of the processed food F conveyed by the conveying device 201 and acquires image data. One image data includes one or more processed foods F. As an example, images may be taken at regular time intervals, and all processed foods to be transported may be included in at least one of the image data. Alternatively, a portion of all processed food delivered may be sampled and imaged. The imaging device 20 is a camera or sensor capable of acquiring a captured image of processed food. The imaging device 20 may be an RGB camera capable of detecting RGB color luminance information, a monochrome camera capable of detecting monochrome luminance information, or a depth camera capable of detecting distance (depth). A camera capable of detecting both information and distance information (for example, a stereo camera) may be used. The imaging device 20 may include Lidar (Light detecting and ranging). The sensor device that senses the conveyed processed food is not limited to the imaging device. Details will be described in modification 1 described later. Also, the number of sensor devices may be one or two or more.

The imaging device 20 stores the captured image data of the processed food F in the storage unit 11 . Attribute information such as time (time stamp), identifier, shooting location, and file type is added to the image data.

The storage unit 11 is a storage medium capable of storing data or information. The storage unit 11 stores image data acquired by the imaging device 20 . The storage unit 11 may store data or information necessary for the operation of the food inspection device 10 in addition to the image data. For example, when all or part of the operations of the food inspection apparatus 10 are executed by a processor such as a CPU, a program to be executed by the processor and operation parameters necessary for the operation of each unit may be stored. The storage unit 11 may store information on the determination result by the determination unit 14 and information generated by the information generation unit 15 . The storage unit 11 is an arbitrary recording medium such as memory, hard disk, and optical device. The storage unit 11 may be provided outside the food inspection device 10 . For example, the storage unit 11 may be a storage device on the network.

The image processing unit 12 performs preprocessing on the image data to make the data suitable for processing by the determination unit 14 . Examples of pre-processing include at least one of noise removal processing, contrast/brightness/color correction processing, edge enhancement, and enlargement/reduction. The image processing unit 12 stores the processed image data in the storage unit 11 .

The determination unit 14 performs an inspection (for example, an appearance inspection) of the processed food F based on the image data processed by the image processing unit 12 stored in the storage unit 11, and determines whether the processed food F corresponds to a correctable product. determine whether or not Here, the determination unit 14 determines whether the processed food F corresponds to any of a good product, a defective product, and a correctable product. Modifiable products are processed foods that can be modified into good products by modification work. Examples of modification work include work that modifies the shape or appearance of the processed food product.

When the processed food F is classified as a defective product or a repairable product, the determination unit 14 further determines the type of defect.

A learned classification model can be used as a method for the determination unit 14 to perform determination. The trained classification model prepares sample data for training, including image data of processed foods and category values indicating whether the processed foods represented by the image data are good, defective, or modifiable. do. Using sample data for learning, a neural network is generated that outputs any of good, defective, or modifiable products as an estimation result from image data of processed foods to be inspected. The estimation results may be probabilities of non-defective products, defective products, and modifiable products. In this case, the one with the highest probability among non-defective products, defective products, and correctable products is determined. A trained model is not limited to a neural network, and any model such as a multiple regression model, a decision tree, or a logistic regression model can be used. Image segmentation such as semantic segmentation may also be used.

As a method for the determination unit 14 to determine the type of defect, a learned model can be used in the same manner as described above. For example, learning sample data including image data of correctable products and defect type values is prepared. Using sample data for learning, a neural network is generated that outputs the type of defect as an estimation result from the image data of the processed food to be judged. The estimation result may be a probability for each type of candidate defect. In this case, the one with the highest probability among the plurality of candidates is used as the estimation result of the defect type. An example of generating a trained model for a correctable product has been shown, but a trained model for a defective product may be generated in the same manner. A trained model is not limited to a neural network, and any model such as a multiple regression model, a decision tree, or a logistic regression model can be used. Image segmentation such as semantic segmentation may also be used.

The determination unit 14 may simultaneously use the same model to classify non-defective products, defective products, and repairable products, and to determine the type of defect.

Examples of non-defective products, repairable products, and defective products, and examples of defect types are shown using FIGS.

Fig. 4 shows an example of a good product. The spring roll wrapper wraps the ingredients in a rectangular shape, and there is no tearing of the wrapper, poor gluing, deformation of the shape, adhesion of dirt, or the like.

FIG. 5A shows an example of a modifiable product. Part of the spring roll skin is not adhered and is peeling off from the main body. The stripped portion 101 has a triangular shape. By gluing the triangular portion 101 and winding it around the main body, it can be easily corrected to a non-defective product. A defect shown in FIG. 5A is identified as, for example, a "triangle".
FIG. 5B shows another example of a modifiable product. Ingredients (such as bamboo shoots) are attached to the skin of spring rolls. By removing the adhering ingredients, it can be easily corrected to a non-defective product. The defect shown in FIG. 5B is identified as, for example, "adhesion of ingredients".

In the case of FIG. 5(A), the winding shape can be easily corrected by gluing the triangular portion 101 and winding it around the main body. Examples of shape correction work can be defined variously according to the target processed food besides rolling. For example, there may be shape modifications such as splitting or unraveling. For example, if the egg is not cracked and remains with the shell, the shell of the egg may be cracked to remove the contents. In addition, when the noodles are hardened, it may be loosened. In the case of FIG. 5B, the appearance of the spring roll can be easily modified by removing the adhering ingredients. Appearance modification may include removal of adhering matter and attachment of foodstuffs. For example, if a piece of cheese has fallen off the surface of the processed food, there may be an operation to attach the piece of cheese. Further, as other examples of correction work, peeling of the coating material may be reapplied, and defective molding may be corrected in shape.

FIG. 6A shows an example of defective products. Dirt 111 such as coloring adheres to the skin of the spring roll. In this case, it is difficult to make a non-defective product by repair work. The defect shown in FIG. 6A is identified as, for example, "dirt".
FIG. 6B shows another example of defective products. The skin of the spring roll was torn and the ingredients inside were leaking. It is difficult to make a non-defective product by repair work. The defect shown in FIG. 6B is identified as, for example, "skin tear". Ingredients inside may be collected and reused.
FIG. 6C shows still another example of defective products. The skin of the spring roll has collapsed greatly from the square shape. It is difficult to make a non-defective product by repair work. The defect shown in FIG. 6C is identified as, for example, "shape collapse".

The determination unit 14 calculates the position of the processed food F detected as a correctable product or defective product. For example, using image data including the processed food F, the position of the processed food F in a three-dimensional space (real space) is calculated. For example, when the image data includes distance information, the position is calculated based on the distance information indicated by the pixels included in the image of the processed food F and the position and orientation of the imaging device 20 . For example, the position of the processed food F may be calculated by calculating the average of the distance information and adding the average to the position of the imaging device 20 in the direction of the orientation of the imaging device 20 .

Alternatively, when a reference mark is placed on the belt of the conveying device 201, the position of the processed food F may be calculated as a relative position with reference to the mark.

The control unit 13 transmits an instruction signal to the sorting device 30 so as to sort the processed food F conveyed by the conveying device 201 into the corresponding lane (process) according to the determination result (classification result) of the determination unit 14. do. The instruction signal includes, for example, information capable of identifying a non-defective product, a correctable product, or a defective product among the processed foods F transported by the transport device 201, and information indicating the position of the processed food F to be sorted. Non-defective products are sent to the downstream side where they are conveyed by the conveying device 201 as they are on the same lane. However, good products may be distributed to another lane, and correctable products or defective products may be sent on the same lane.

Based on the instruction signal received from the control unit 13, the sorting device 30 sorts the processed food F conveyed by the conveying device 201 to the lane of the corresponding process. As an example, the processed food F classified as a correctable product is sorted to the correction process, and the processed food F classified as the defective product is sorted to the disposal process. The processed food F judged to be non-defective is distributed to the next process for non-defective products.

Examples of the processes after the next process for non-defective products include the measurement process (weight, temperature, composition, etc.), the investigation process for the presence or absence of signs of the occurrence of defective products or correctable products, the alignment process (adjustment of angle or position, overlap removal, turning inside out, etc.), cooking process (for example, frying process), and freezing process. The processed food F that has undergone the correction process may be returned to the next process and subsequent processes for non-defective products, or the same process as at least part of the subsequent processes for non-defective products may be corrected in the same lane as the correction process. You may carry out continuously after a process. The presence or absence of a sign of occurrence of defective products or correctable products can be determined by learning using a model such as a neural network. For example, model learning is performed using an image of a processed food that is within the range of good products but inferior to the standard level of good products as teacher data.

The above process is an example, and other processes may be used. For example, if the defect type of the defective product is contamination, the defective product may be sorted into the process of analyzing the components of the contamination instead of the disposal process. Also, if the defect type of the defective product is "shape collapse", it may be sorted to the step of collecting the ingredients inside the spring roll.

The sorting device 30 is, for example, a robot (mechanical operator) having an extendable arm. Robots are placed, for example, in aisles between lanes to move processed food between lanes. For example, the robot moves the processed food picked up from the transport device 201 to, for example, a lane for the correction process. Other robots may pick up the processed food that has been modified in the modification process and return it to the good product lane.

The sorting device 30 is not limited to robots. For example, the sorting device 30 may be a moving body that moves on rails provided on the ceiling in the facility of the factory. The moving body lifts the processed food conveyed by the conveying device 201 by a crane. The moving body moves the lifted processed food to the lane where the corresponding process is performed, and unloads the processed food. Alternatively, the sorting device 30 may be a mechanism that pushes out the conveyed processed food to the outside of the conveying path by an arm. In this case, the sorting device 30 is attached along the conveying path of the conveying device 201, and the processed food can be pushed out by protruding the arm in the direction orthogonal to the conveying direction. The extruded processed food is collected, for example, by a collecting device (corresponding to a collection box). The contents of the collected processed food may be reused.

In addition, the sorting device 30 may be any device as long as it is a device capable of moving the processed food conveyed by the conveying device 201 to another process (lane).

7 to 11 show a specific example of dividing the group of processed foods conveyed by the conveying device 201 into a plurality of processes.
FIG. 7 shows an example of moving the processed food F1 classified as a modifiable product to the conveying device 211 in lane 2 where the modification process is performed. Lane 1 is provided with a conveying device 201 for conveying the processed food dropped from the winding device. Lane 2 is provided with a conveying device 211 that conveys a correctable product to a place (correctable process) where corrective work is performed. The transport device 211 transports the modifiable item by driving the transport belt 212 with rollers. Lane 3 is provided with a conveying device 221 that conveys processed food determined to be defective to a discarding device 220 . The disposal device 220 is a device that collects defective products. A conveying device 221 conveys defective products by driving a conveying belt 222 with rollers. A sorting device 30, which is a robot arranged between

lanes

1 and 2, conveys processed foods F1 classified as correctable products on a conveying belt 202 based on an instruction signal from the food inspection device 10. Pick up from the crowd. The position of processed food F1 is included in the instruction signal. In the illustrated example, the defect type of the processed food F1 is "Triangle", which means that the triangular part of the skin of the spring roll is not wrapped around the body part. The sorting device 30 unloads the picked-up processed food F1 to the upstream side of the conveying device 211 on the lane 2 . The processed food F1 is conveyed by the conveyor belt 212 in the direction of the worker 241 who performs the correction work.

The worker 241 performs correction work on the processed food F1 to change the processed food F1 from a correctable product to a non-defective product. For example, paste is applied to the triangular portion of the spring roll wrapper and then adhered to the main body. As a result, the processed food F1 is made non-defective. The non-defective processed food F1A may proceed to the next step in the same lane 2 as it is, or may be returned to the same lane 1 as the non-defective product by an operator or a robot.

The main body that performs the correction work may be a device such as a robot (machine operator) instead of a worker (human operator). In this case, for example, a device such as a robot detects the processed food F1 transported by the transport device 211, and performs correction work on the processed food F1. This makes the processed food F1 a non-defective product. A device such as a robot may return the non-defective processed food F1 to lane 1 . A device such as a robot may receive defect type information from the control unit 13 of the food inspection device 10 . In this case, a device such as a robot may determine the details of the repair work to be performed on the processed food F1 based on the defect type information (it is assumed that the device such as a robot can perform multiple repair work). .

FIG. 8 shows an example of picking up processed food F2 classified as defective from the group of processed foods transported by the transporting device 201 of lane 1 and moving it to the transporting device 221 of lane 3, which performs the disposal process. The arm of the sorting device 30 is extendable and can reach up to the lane 3. Based on the instruction signal received from the food inspection device 10, the sorting device 30 identifies and picks up the processed food F2 classified as defective. In the illustrated example, the defect type of the processed food F2 is "dirt" with the dirt 111 attached. The sorting device 30 moves the picked up processed food F2 to lane 2 and unloads it on the upstream side of the conveying device 211 . The processed food F1 is conveyed to the discarding device 220 by the conveying belt 212 and collected by the discarding device 220 .

In the examples shown in FIGS. 7 and 8, the same robot performs both lane-to-lane movement of correctable items and lane-to-lane movement of defective items, but separate robots may perform each lane-to-lane movement. .

FIG. 9 shows an example in which separate robots move correctable products between lanes and defective products between lanes. A robot (sorting device) 30 moves the processed food F1, which is a correctable product, between lanes, and a robot (sorting device) 31 moves the processed food F2, which is a defective product, between the lanes. The food inspection device 10 is connected to both the robot 30 and the robot 31 by wire or wirelessly.

For processed foods classified as correctable products, processed foods may be sorted into different processes (lanes) according to the type of defect. For example, the lanes to be sorted may be different between the repairable products classified into the “triangle” defect type and the repairable products classified into the “material attachment” defect type. Further, it may be determined whether the repair work is performed by a human operator or by a robot (machine operator) according to the type of defect. When it is decided that the correction work is to be performed by a human, the processed food is distributed to the steps (lanes) in which the human is to perform the correction. When it is decided that the robot will perform the correction work, the processed food is distributed to the steps (lanes) in which the robot corrects.

FIG. 10 shows an example in which the sorting device 30 sorts correctable products into lanes 2-1 and 2-2 according to the type of defect. In this example, in lane 2-1, the process of correcting work is performed by a human, and in lane 2-2, the process of correcting work by the robot 242 is performed. The sorting device 30 can move processed foods (modifiable products) from lane 1 to lanes 2-1 and 2-2 by extending and retracting arms. Lane 2-1 is the same as lane 2 in FIG. 9, etc., and the worker performs correction work for the "triangle".

A transport device 223 is provided in lane 2-2. The conveying device 223 drives the conveying belt 224 to move the processed food F2 to the process (operation process 2) of correcting the "adhesion of ingredients". In the work process 2, the robot 242 for correcting work removes the ingredients 102 adhering to the surface from the processed food F2, thereby making the processed food F2 non-defective. Processed food that is deemed non-defective may be returned to Lane 1 by another robot or worker as described above.

In the example of FIG. 10, the contents of the correction work were different between lane 2-1 and lane 2-2, but they may be the same. For example, there may be cases where the correction work for the "triangle" is performed by a human or by the robot 242 . For example, in the case of "adhesion of ingredients", it may be possible to switch between a human performing the correction work and a robot performing the correction work according to the type of the adhering ingredients. In this case, the determination unit 14 may determine the details of the defect (in this example, the type of ingredients attached), and transmit information indicating the details of the defect to the sorting device 30 .

The control unit 13 may perform the process of determining the process (lane) to which the processed food is to be distributed according to the type of defect. In this case, the control unit 13 transmits to the sorting device 30 information indicating the process (lane) to which the processed food is to be sorted. The sorting device 30 sorts the processed foods according to the instructed information.

For processed foods classified as defective products, processed foods may be sorted into different lanes according to the type of defect, as in the case of correctable products. For example, correctable products classified into the defect type of "dirt" are sorted to lane 3-1, and correctable products classified into the defect type of "ripped skin" or "out of shape" are sorted to lane 3-2. may In the lane 3-2, for example, the ingredients inside spring rolls are collected.

FIG. 11 shows an example in which the sorting device 31 and the sorting device 33 distribute defective products to the lane 3 and the recovery device 230 (the recovery device 230 is also considered as one form of lane) according to the type of defect. The sorting device 31 and lane 3 are the same as in FIG. The sorting device 33 is provided in the conveying device 221, and has a mechanism for pushing out the conveyed processed food (defective product) F3 to the outside of the conveying path by means of an arm 33A. The sorting device 33 pushes the processed food F3 out of the transport path by projecting the arm 33A in a direction orthogonal to the transport direction. The extruded processed food F3 is recovered by the recovery device 230. FIG.

Processed foods F classified as non-defective products go through various processes such as arranging process, heating process (frying, grilling, boiling, etc.), and freezing process.

When the control unit 13 detects a defective product or a correctable product, it generates an alarm signal and provides the alarm signal to the alarm device 50 . The alarm device 50 outputs an alarm based on the alarm signal. The alarm device 50 includes, for example, at least one of a sound output device that outputs sound (an alarm sound or voice), a light emitting device that emits light, a vibration device that generates vibration, and the like. Examples of sound output devices include speakers, sirens, or the like. Examples of light emitting devices include warning lamps or warning lights (eg stack lights). If the alarm device is a light emitting lamp, the light emitting part may be arranged at a height where the light is noticeable to the operator or the supervisor. The vibration generator generates vibration in a specified pattern using a vibrator or the like.

One or more alarm devices 50 are provided. The installation location of the alarm device may be arbitrary. For example, the installation location may be in the vicinity of the location where the inspection process is performed, or in the vicinity of a processing device that performs processing that may cause defective products or repairable products. It can also be an office where workers or supervisors do their office work. In addition, the installation location may be a location that is easily noticed by workers or observers.

The control unit 13 may generate an alarm signal according to the type of defective product or repairable product. For example, the degree of urgency may be determined according to the type of defect, and if the degree of urgency is low, it may be blue, if it is medium, it should be yellow, and if it is high, it should be red. For example, in the case of a defective product, if the type of defect is neither a foreign substance nor a predetermined type, it is determined that the degree of urgency is high. Examples of foreign substances include harmful substances, non-ingredients, and ingredients that should not be contained in processed foods. If the type of defect is tear, deformation, crushing, etc., it is determined that the degree of urgency is low. Even if the type of defect is tear, deformation, crushing, etc., if it occurs chronically, it is judged to be of low urgency. case, etc.), it may be judged that the degree of urgency is high.

The alarm device 50 issues an alarm based on the alarm signal received from the control unit 13. When the alarm device 50 is a light-emitting device, the alarm signal includes a signal specifying the color of light emission, the light emission pattern, etc., and the light-emitting device emits light according to the instruction signal to issue an alarm. When the alarm device 50 is a sound output device, the alarm signal includes a signal specifying the type of sound or the pattern of the sound, and the sound output device outputs the sound according to the instruction signal to issue an alarm. When the alarm device 50 is a vibration device, the warning signal includes an instruction signal specifying a vibration pattern, and the vibration device vibrates according to the instruction signal to issue an alarm.

When a defective product or a correctable product is detected, the control unit 13 generates a notification signal indicating that the defective product or the correctable product has been detected, and transmits the generated notification signal from the communication unit 17 to the terminal device 2 (FIG. 3). The communication signal may include, for example, the type of defect of the defective product or the correctable product, and information specifying the location where the defective product or the correctable product is detected (occurrence location information). The notification signal may also include image data including detected defective or correctable items. The notification signal may also include information on the degree of urgency when defective products are detected. The place of occurrence information includes information identifying the lane where the defective product or correctable product was generated, information identifying the section in the factory, or information identifying the inspection process where the defective product or correctable product was detected. It's okay.

The terminal device 2 receives the notification signal and displays the information or data contained in the notification signal on the screen, thereby making it possible for the user (worker, supervisor, etc.) to recognize the occurrence of defective products or correctable products. . The user can, for example, rush to the processing equipment that has generated the defective product or the repairable product, and inspect the processing equipment or adjust the operating parameters according to the type of defect. For example, if the type of defect is "ripped skin", an inspection of the equipment that produces the skin, or an inspection of the winding mechanism of the winding equipment may be performed. Also, operating parameters of the winding device (such as winding strength) may be adjusted. The destination of the notification signal is not limited to the inside of the production site of the factory, but may be the room of a worker or manager, or another place of business (head office or control center). As a result, it is possible to remotely check the screen of the terminal device and adjust the operation parameters of the processing device remotely from the terminal device.

When a defective product or a repairable product is detected, the control unit 13 generates operation parameters according to the type of defect of the defective product or the repairable product, and transmits the generated operation parameters to the defective product or the repairable product from the communication unit 17. It may be sent to the processing device that performs the processing that caused the generation of the product. The communication unit 17 includes a first transmission unit that transmits operation parameters according to the defect type of the repairable product. The communication unit 17 includes a second transmission unit that transmits operation parameters according to the defect type of the defective product. For example, if the type of defect is "triangle", it may be determined that the gluing is insufficient, and an operation parameter for correcting the gluing amount or gluing position may be sent to the device that glues the leather. The gluing may be done at the winding device or sent to another device if gluing the leather is done by another device. As a result, the probability of producing defective products or correctable products can be reduced, that is, the probability of producing non-defective products can be increased. The second transmitter may be physically the same circuit or device as the first transmitter, or may be a circuit or device different from the first transmitter. Another example of adjustment of operating parameters in a processing apparatus other than a winding apparatus is adjustment of the size of a molding frame, filling pressure, etc. in a forming apparatus. In the case of flouring machines, batterers, and breaders, there is adjustment of the adhesion amount, in the case of stuffers, the filling pressure is adjusted, and in the case of filling machines, the amount and position of ingredients and packaging materials are adjusted. Also, if it is a topping machine, there are adjustments such as the discharge amount and position.

The information generation unit 15 generates monitoring information for the observer based on the determination result of the determination unit 14 . As an example, the information generation unit 15 generates statistical information regarding the determination results of non-defective products, modifiable products, and defective products. As another example, the information generation unit 15 generates statistical information about defect types. The information generation unit 15 transmits the monitoring information to the terminal device 2 of the observer via the communication unit 17 . By viewing the monitoring information displayed on the screen of the terminal device 2, the monitoring staff can grasp the status of the determination result.

As an example, the monitoring information may include the total frequency (total number of occurrences) of the determination results of correctable products and defective products for each lane of the spring roll machine. The warning by the warning device 50 may be issued only when the total number of occurrences exceeds the threshold. The monitoring information may include the rate of defective items, the rate of correctable items, or the rate of good items. The monitoring information may also include the incidence rate of each defect type for correctable or defective products. The monitoring information may also include the probability of a defective product or a sign of correctable occurrence for a non-defective product. An application function for generating and displaying monitoring information may be included in the food inspection device 10 or may be installed in the terminal device 2 .

Observers can easily notice the occurrence of defective products and correctable products by checking the monitoring information. In particular, it can also be noticed when the number of occurrences of defective products and correctable products increases slowly. In addition, even when there are many winding devices, the number of occurrences for each lane (total number of occurrences, number of defects by type, etc.) can be quantitatively grasped. In addition, a sudden increase in defective products and correctable products can be noticed immediately, and the processing equipment (winding equipment, etc.) to be adjusted can be easily grasped. Observers can also manage trends in the occurrence of defective or correctable items.

FIG. 12 is a flow chart showing an example of the operation of the food inspection system 1 according to this embodiment. The order of steps in FIG. 12 is an example, and a part of the order may be changed. For example, the order of steps S104 to S106 may be changed. The processed food conveyed by the conveying device 201 is sensed by the sensor device to detect sensing information. For example, an image of processed food is captured by the imaging device 20 to obtain image data (S101). Also, the acquired image data may be displayed as a video in real time on the display device 40 so that the user (worker, supervisor, etc.) can visually confirm the transportation status of the processed food.

After image processing is performed on the image data by the image processing unit 12 to make the image data suitable for determination processing, the determination unit 14 determines whether or not each processed food included in the image data is an item that can be modified (S102). ). Here, it is determined whether the processed food is a non-defective product, a correctable product, or a defective product. That is, it classifies processed foods as good, modifiable, or defective. A defect type may be further determined for the processed food determined as a correctable product or a defective product by the determination unit 14 .

The sorting device 30 sorts the processed food to the corresponding process according to the determination result (S103). Processed foods that are determined to be correctable are sorted to the correction process, processed foods that are determined to be defective are sorted to the disposal process, and processed foods that are determined to be non-defective are sorted to the next process. The process of distributing processed foods (correctable products or defective products) may be further classified according to the type of defect.

When a correctable product or a defective product is detected, the control unit 13 generates an alarm signal and outputs the alarm from the alarm device 50 (S104). The content of the alarm signal may be changed according to the type of defect. In addition, when a correctable product or a defective product is detected, the control unit 13 generates notification information regarding the correctable product or the defective product to the user (worker or supervisor, etc.), and sends it to the terminal device 2 via the communication unit 17. and may be transmitted (S105). In addition, the control unit 13 determines the operation parameters for the processing apparatus that performed the processing that caused the defect, according to the defect type of the repairable product or the defective product. The control unit 13 transmits an instruction signal including operation parameters to the processing device via the communication unit 17 (S106). This can reduce the likelihood of correctable or defective products.

As described above, according to the present embodiment, it is not necessary to assign workers to detect correctable products by detecting correctable products in the inspection of transported processed foods. As a result, it is possible to eliminate the problem of individual differences in determining whether or not an item is a modifiable item, and to detect a modifiable item with stable accuracy.

In addition, in this embodiment, not only non-defective products and defective products but also correctable products are detected. can be reduced.

In addition, according to this embodiment, the sorting device sorts the processed food to the subsequent steps according to whether the processed food is classified as a non-defective product, a defective product, or a correctable product. This eliminates the problem of individual differences and reduces the possibility that processed foods other than those classified in each process will be included, compared to the case where workers classify.

In addition, according to this embodiment, the type of defect is determined in the case of defective products and repairable products, and the sorting device sorts the processed foods to the subsequent processes according to the determined type. As a result, the processed foods classified as defective or correctable can be efficiently sorted to the process according to the defect type.

In addition, according to the present embodiment, by detecting a sign of the occurrence of defective products or correctable products for processed foods classified as non-defective products, the parameters of the processing equipment can be adjusted in advance, and the occurrence rate of defective products or correctable products can reduce the increase in

(Modification 1)
In the above-described embodiment, processed foods are classified into correctable products, non-defective products, and defective products based on image data sensed by an imaging device (RGB camera, depth camera, etc.). Also, the type of defect was determined. A sensor device for sensing processed food is not limited to an RGB camera, a depth camera, or the like, and other sensor devices may be used.

For example, the sensor device may be a temperature sensor. A specific example is a thermal camera or radiation thermometer that detects the heat emitted by processed food. In this case, the determination unit 14 classifies the processed food into a modifiable product, a non-defective product, or a defective product based on the temperature distribution image data of the processed food. The determination unit 14 also determines the types of defects in processed foods classified as correctable products or defective products. The temperature distribution image data may be preprocessed by the image processing unit 12 . The classification may use a model such as the neural network described above, or image segmentation such as semantic segmentation.

The sensor device may also be an X-ray camera or an infrared camera. The X-ray camera detects X-rays emitted from the X-ray generator to the processed food and transmitted through the processed food. The infrared camera detects infrared rays (for example, near-infrared rays) emitted from the infrared generator to the processed food and transmitted through the processed food. The X-ray or infrared transmittance differs depending on the food (ingredients, skin, etc.), and the X-ray image data or infrared image data represents the internal state (food distribution) of the processed food. Based on the X-ray image data or the infrared image data, the determination unit 14 classifies the processed food into a modifiable product, a non-defective product, or a defective product. The determination unit 14 also determines the types of defects in processed foods classified as correctable products or defective products. Preprocessing may be performed on the X-ray image data or the infrared image data by the image processing unit. The classification may use a model such as the neural network described above, or image segmentation such as semantic segmentation.

The sensor device may also be a three-dimensional sensor (3D sensor) that measures the volume of processed food. For example, the 3D sensor acquires image data of distribution of volume (depth, etc.) in units of pixels of the processed food. Based on the volume distribution image data, the determination unit 14 classifies the processed food into a modifiable product, a non-defective product, or a defective product. The determination unit 14 also determines the types of defects in processed foods classified as correctable products or defective products. The classification may use a model such as the neural network described above, or image segmentation such as semantic segmentation.

The sensor device may also be a hyperspectral camera or an infrared sensor. An image representing the component distribution of processed food is generated from spectral image data obtained by a hyperspectral camera. Also, an image representing the component distribution is generated from the image data of the received infrared light emitted from the infrared light emitting device and transmitted through the processed food. The determination unit 14 classifies the processed food into a modifiable product, a non-defective product, or a defective product based on the image data of the component distribution. The determination unit 14 also determines the types of defects in processed foods classified as correctable products or defective products. The image data of component distribution may be preprocessed by the image processing unit. The classification may use a model such as the neural network described above, or image segmentation such as semantic segmentation.

Different sensor devices may be used for determining correctable products, non-defective products, and defective products, and for determining the type of defect. For example, an RGB camera may be used for classification into correctable, good, or bad products, and an X-ray camera may be used for defect type determination.

In the above-described embodiment, in the measurement process for non-defective processed food, the sensor device described above may be used to measure the weight, temperature, or components.

(Modification 2)
Instead of the sorting device 30, a worker may sort the processed food conveyed by the conveying device 201. FIG. In this case, the display device 40 is arranged at a position visible to the worker. The display device 40 displays an image (video) of the group of processed foods conveyed by the conveying device 201, and also displays information identifying the processed foods determined to be correctable or defective. For example, information indicating a correctable or defective product is displayed superimposed on or adjacent to the image of the processed food determined to be correctable or defective. The information indicating the correctable or defective product may be the words "correctable" or "defective product", a specific mark, or the color conversion of the image of the processed food. Based on the information displayed on the display device 40, the operator can easily identify correctable products or defective products. The worker refers to the image, picks up the correctable product or defective product specified, and distributes it to the corresponding process.

FIG. 13 is a diagram illustrating a specific example of modification 2. FIG. FIG. 13A is a front view showing the display device 40 provided in front of the worker and the processed food F6 (modifiable product) conveyed by the conveying device 201. FIG. The display device 40 displays an image of the processed food F6 and information indicating that the processed food F6 is a repairable item with a "triangle" defect type. Although only one processed food is shown in the figure, a plurality of processed foods may actually be transported. Illustration of workers is omitted in FIG.

FIG. 13(B) is a plan view corresponding to FIG. 13(A). The worker 251 looks at the display device 40 and determines that the processed food F6 is a correctable product with the type of "triangle" defect. The worker 251 picks up the processed food F6 and moves it to the conveying device 211 of lane 2 .

(Modification 3)
A projector (projection device) for projecting an image may be arranged, and an image including information identifying correctable or defective products may be directly projected onto the processed food F from the projector. Alternatively, a light-emitting device such as a laser may be arranged to irradiate the processed food F with light from the light-emitting device. The direction of the projector may be controlled in synchronization with the driving of the belt of the conveying device 201 so that the image is projected onto the processed food F for a long time. A worker or robot can directly view the processed food F to identify correctable or defective items.

FIG. 14 is a diagram explaining a specific example of Modification 3. FIG. FIG. 14A shows that the projector 41 connected to the food inspection apparatus 10 projects an image including information identifying correctable or defective products onto the processed food F6 (correctable product) conveyed by the conveying device 201. there is An image is projected on the processed food F while it exists within the projection area 42 of the projector 41 . The image contains the characters "triangle". FIG. 14B is a plan view showing the "triangle" image projected on the processed food F6. A character "triangle" is displayed directly on the processed food F6, and the worker 251 (or robot) can easily identify the modified processed food F6 by visually recognizing this character.

(Hardware configuration)
The food inspection system 1 or food inspection apparatus 10 of FIG. 1 includes, for example, one or more CPUs (Central Processing Units), a storage unit, a communication unit, and information processing such as a computer on which an OS (Operating System) and applications operate. It may be configured as a device. The OS and applications are examples of computer programs for realizing the operations according to this embodiment. A part or all of the functions of the food inspection system 1 or the food inspection device 10 may be implemented by a semiconductor circuit such as FPGA or ASIC, or a GPU (Graphics Processing Unit). The food inspection system 1 or the food inspection device 10 may be a physical computer, or may be realized by a virtual machine (VM), a container, or a combination thereof. One or more physical computers, virtual computers, or containers may share the functions of the food inspection system 1 or food inspection device 10 . It is not excluded to use a configuration in which the number of food inspection systems 1 or food inspection apparatuses 10 is increased in order to improve availability or distribute the load.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention, which may be used in various other forms. Various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

Also, the effects of the present disclosure described in this specification are merely examples, and other effects may be obtained.

In addition, this disclosure can also take the following configurations.
[Item 1]
a sensor device that senses the processed food being conveyed;
a determination unit that determines whether or not the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work, based on the information detected by the sensor device;
Food inspection system with
[Item 2]
The food inspection system according to item 1, wherein the modification work includes modification of at least one of shape and appearance of the processed food.
[Item 3]
3. The food inspection system according to

item

1 or 2, wherein the determination unit determines a defect type of the processed food determined to correspond to the correctable product.
[Item 4]
4. The food inspection system according to item 3, further comprising: a first transmission unit configured to transmit, to the processing device, operation parameters for adjusting the operation of the processing device that processed the processed food according to the type of the defect.
[Item 5]
any one of items 1 to 4, comprising a control unit that determines whether the modification work is to be performed by a human operator or a machine operator for the processed food that has been determined to be a modifiable product. A food inspection system as described.
[Item 6]
6. The food inspection system according to any one of items 1 to 5, wherein the determination unit classifies the processed food into one of a non-defective product, a defective product, and the correctable product.
[Item 7]
Item 7. The food inspection system according to Item 6, wherein the determination unit determines a defect type of the processed food classified as the defective product.
[Item 8]
a second transmission unit configured to transmit, to the processing device, an operation parameter for adjusting the operation of the processing device that processed the processed food according to the defect type of the processed food classified as the defective product; The food inspection system according to item 7.
[Item 9]
9. The food inspection system according to any one of items 6 to 8, wherein the determination unit determines whether or not the processed food classified as the non-defective product has a sign of a defective product or a correctable product.
[Item 10]
Allocating the processed food classified as the modifiable product to a step of performing the modification work on the processed food,
Allocating the processed food classified as non-defective products to a step of performing a first process on the processed food,
10. The food inspection system according to any one of items 6 to 9, further comprising a sorting device that sorts the processed food classified as defective to a step of subjecting the processed food to a second process.
[Item 11]
a communication unit that transmits information indicating that a defective product or a correctable product has been detected when the processed food is classified as the defective product or the correctable product;
a terminal device that receives the information and displays the information on a display unit;
The food inspection system according to any one of items 6 to 10, comprising:
[Item 12]
The food inspection system according to any one of items 1 to 11, wherein the processed food is a food in the process of being manufactured.
[Item 13]
The sensor device is an imaging device,
The determination unit is based on a trained model in which image data of processed food and a value indicating whether or not the processed food indicated by the image data is a modifiable product are associated with each other, and image data acquired by the imaging device. 13. The food inspection system according to any one of items 1 to 12, wherein the food inspection system determines whether or not the transported processed food corresponds to the correctable product.
[Item 14]
Sensing the conveyed processed food with a sensor device,
A food inspection method for determining, based on information detected by the sensor device, whether or not the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work.
[Item 15]
a step of acquiring information detected by sensing of the processed food being conveyed;
Based on the information, determining whether the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work;
A computer program that causes a computer to execute
[Item 16]
Determining the type of defect of the processed food determined to correspond to the correctable product;
16. The computer program according to item 15, further causing the computer to execute a step of transmitting, to the processing device, an operation parameter for adjusting the operation of the processing device that processed the processed food according to the type of the defect. .
[Item 17]
classifying the processed food as one of good, bad, and modifiable;
17. The computer program according to item 15 or 16, further causing the computer to execute: a step of determining whether or not the processed food classified as non-defective has a sign of a defective product or a correctable product.

1 food inspection system 2 terminal device 3 processing device 10 food inspection device 11 storage unit 12 image processing unit 13 control unit 14 determination unit 15 information generation unit 16 alarm unit 17 communication unit 20

imaging devices

30, 31, 33 sorting device 40 display device 41 Projector 42 Projection area 50 Alarm device 101 Part 102 Ingredient 200

Winding device

201, 211, 221, 223

Transport device

202, 212, 222, 224 Transport belt 220 Waste device 230

Recovery device

241, 251, 252 Worker 242 Robot

Claims

a sensor device that senses the processed food being conveyed;
a determination unit that determines whether or not the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work, based on the information detected by the sensor device;
Food inspection system with
The food inspection system according to claim 1, wherein the modification work includes modification of at least one of the shape and appearance of the processed food.
The food inspection system according to claim 1 or 2, wherein the determination unit determines a type of defect of the processed food determined to correspond to the correctable product.
4. The food inspection system according to claim 3, further comprising: a first transmission unit that transmits, to the processing device, operation parameters for adjusting the operation of the processing device that processed the processed food according to the type of the defect.
5. Any one of claims 1 to 4, further comprising a control unit that determines whether the correction work is to be performed by a human operator or a machine operator for the processed food that is determined to be the correctable product. The food inspection system described in .
6. The food inspection system according to any one of claims 1 to 5, wherein the determination unit classifies the processed food into one of a good product, a defective product, and the correctable product.
7. The food inspection system according to claim 6, wherein the determination unit determines a defect type of the processed food classified as the defective product.
a second transmission unit configured to transmit, to the processing device, an operation parameter for adjusting the operation of the processing device that processed the processed food according to the defect type of the processed food classified as the defective product; The food inspection system according to claim 7.
The food inspection system according to any one of claims 6 to 8, wherein the determination unit determines whether or not the processed food classified as the non-defective product has a sign of a defective product or a correctable product. .
Allocating the processed food classified as the modifiable product to a step of performing the modification work on the processed food,
Allocating the processed food classified as non-defective products to a step of performing a first process on the processed food,
10. The food inspection system according to any one of claims 6 to 9, further comprising a sorting device that sorts the processed food classified as defective to a process of subjecting the processed food to a second process.
a communication unit that transmits information indicating that a defective product or a correctable product has been detected when the processed food is classified as the defective product or the correctable product;
a terminal device that receives the information and displays the information on a display unit;
The food inspection system according to any one of claims 6 to 10, comprising:
The food inspection system according to any one of claims 1 to 11, wherein the processed food is a food in the process of being manufactured.
The sensor device is an imaging device,
The determination unit is based on a trained model in which image data of processed food and a value indicating whether or not the processed food indicated by the image data is a modifiable product are associated, and image data acquired by the imaging device. 13. The food inspection system according to any one of claims 1 to 12, wherein the food inspection system determines whether or not the transported processed food corresponds to the correctable product.
Sensing the conveyed processed food with a sensor device,
A food inspection method for determining, based on information detected by the sensor device, whether or not the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work.
a step of acquiring information detected by sensing of the processed food being conveyed;
Based on the information, determining whether the processed food corresponds to a modifiable product that can be modified into a non-defective product by modification work;
A computer program that causes a computer to execute
Determining the type of defect of the processed food determined to correspond to the correctable product;
16. The computer according to claim 15, further causing said computer to execute a step of transmitting, to said processing apparatus, an operation parameter for adjusting the operation of said processing apparatus that processed said processed food according to said defect type. program.
classifying the processed food as one of good, bad, and modifiable;
17. The computer program according to claim 15 or 16, further causing the computer to execute a step of determining whether or not the processed food classified as non-defective has a sign of defective or correctable.