RU2790793C1 - Method for quality control and determination of parameters of defects in food products during their sorting on a conveyor - Google Patents

Abstract

FIELD: quality control of food products.

SUBSTANCE: invention relates to methods for quality control of food products, such as vegetables and fruits, and in particular to the use of colour image analysis to determine the parameters of defects in food products based on the support vector machine and neuron algorithms. To implement the technical result in the claimed method, the following are provided: a conveyor with unsorted food products, a conveyor for defect-free products, a conveyor with products suitable for removing defects, a conveyor for products for disposal, a camera capable of recognizing the type of defect within a certain type of classified product, a specific area of location this defect and predict the depth of its occurrence, a robotic arm, a control unit with tracking systems.

EFFECT: determining the full condition of the surface of the product, the type and depth of defects to improve the quality of sorting and reduce the proportion of losses and rejects during their subsequent cleaning.

6 cl, 1 dwg

Description

The invention relates to methods for quality control of food products, such as vegetables and fruits, namely, the use of color image analysis to determine the types and parameters of defects, such as the area and depth of food products, based on the support vector machine and convolutional neural network algorithms, and can be used when sorting them on the conveyor.

A known method for assessing and controlling the quality of food products on a dynamic production line [patent RU No. 2613319]. This method is based on the assessment of the color of a food product moving in a dynamic production system. Each product receives a plurality of individual scores for color intensities and tints of at least one color responsible for the defect. A plurality of individual product ratings are then compared to the desired color characteristic of the product.

The disadvantages of this method include the rigid dependence of the quality index on the percentage of coloring of the food product with a certain color, which can lead to erroneous identification of a defect-free product, since the product may not have a defect, but will be classified as defective if, for example, in the case of fruits, it is more mature than the others.

Also known is a method of optical quality control of agricultural products of spherical shape when sorting on a conveyor [patent RU No. 2737607]. Sorted objects of agricultural products of a spherical shape make translational movement on a roller conveyor and rotate at the same time. Optical radiation sources are used for optical impact on objects. With the help of a rotary mirror, the light reflected from the surface of the test object is focused on the sensitive elements of the hyperspectral camera with a line of optical sensors. The obtained hyperspectral image of the object is compared with a database containing a plurality of images. The rotation of the mirror is coordinated with the coordinates of the object on the conveyor during the time it takes for the object to make a complete revolution.

The disadvantages of the known method include the inability to determine the parameters and the specific location of defects in food products, as well as the ability to work only with products that have a spherical shape.

The closest method of the same purpose to the claimed invention in terms of a set of features is a method for measuring quality characteristics or defects on products or in products such as vegetables and fruits [RU No. 2738327], including: providing a camera system on the measuring device frame, and the camera system is equipped with at least one light source and at least one image recording camera; moving the product for measurement using conveying means; controlling the camera system with a controller, including issuing a first measurement command and recording at least one product image at at least two frequencies or frequency spectra at the first measurement point and/or along the first measurement path; sending by means of the controller at least a second measurement command for recording at least one product image at at least two frequencies or frequency spectra at the second measurement point and/or on the second measurement path, the second measurement point and/or the second measurement path at least partially different from the first measurement point and/or the first measurement path; compiling an overview image of the product for each frequency or each frequency spectrum using the data processing system; and analyzing the overview image of the product and detecting quality characteristics or defects on or in the product with the detection system. This method is taken as a prototype.

The disadvantages of the known method, taken as a prototype, include the need to use expensive elements, such as cameras with different wavelengths and light sources having a certain frequency and a certain frequency spectrum. In addition, these elements are exposed to the aggressive environment in which they are located, which leads to premature failure. Also, this invention has a limited range of detection of defects and is not able to identify the specific location of the defect on the food product. The two measurement points do not cover the whole product, which can lead to missing a defect.

The technical task of the invention consists in eliminating these shortcomings, namely in creating such a method for controlling the quality of food products, in which the complete surface condition, type and depth of defects are determined to improve the quality of sorting and reduce the proportion of losses and rejects during their subsequent cleaning.

The problem was solved due to the fact that in the known method of quality control and determination of the parameters of defects in food products, such as vegetables or fruits, when they are sorted on a conveyor using cameras and a controller, including the supply of unsorted food products along the conveyor, control with controller, the formation of images obtained from cameras, and their analysis by means of a data processing system, according to the invention, a conveyor for supplying unsorted food products is fixed on a frame on which a camera, a robotic arm, a control unit with tracking systems, control and a programmable logic controller are installed , conveyors for defect-free food products, for food products in which it is possible to remove a defect without affecting the production process and for food products to be disposed of, motors and compartments for placing a food product are installed on all conveyors, while using a robotic arm, supplying attached to a gripping device, a camera and connected to a control unit, and a personal computer is connected to the control unit, which has built-in algorithms that allow classifying the type of product and recognizing the type of defect within a certain type of classified product, the specific area of the location of this defect and predicting the depth of its occurrence, at the same time, the camera of the robot-manipulator is placed at a right angle with respect to the compartments of the conveyor, and after the tracking system embedded in the control unit determines the presence of the product in the compartment of the conveyor, the conveyor is stopped by sending signals to the control unit, after the conveyor is stopped by the robot camera - the manipulator once again photographs the contents of the compartment, but in a higher resolution than when the conveyor is moving, and transmits the photographic image to a personal computer, whose algorithms classify it as belonging to one of the types of food products and determine the presence of a defect, and when o detection, its boundaries, coordinates and depth of occurrence are determined, and if the area of the defect or the depth of occurrence do not correspond to the parameters entered in the software settings on the personal computer, then a signal is sent to the control unit to move the food product by the gripping device of the robotic arm, on the conveyor intended for the disposal of products, and the robot-manipulator executes this command, and if the defect is not detected or is detected, but its area and depth are less than the limit values set in the software settings on the personal computer, then the coordinates of food capture are calculated. of the product, taking into account its classified type, specific shape and minimizing its surface overlap with the dimensions of the gripper when holding, and send a signal to capture it, the gripper of the robotic arm captures the food product and moves it to the camera located on frame, and rotates it with the frequency and angle of movement such that are incorporated in the settings of the algorithms on a personal computer, at each moment of movement, the food product is photographed, and if the food product has a round or oval shape, then its upper hemisphere is photographed first, and then, for one more turn, of the lower hemisphere, while the camera is placed perpendicular to the center of the parallel of 45 degrees, with a pear-shaped form, the upper part of the food product is first photographed with the camera located perpendicular to the center of the elongated vertical part, and then, for one more turn, the lower part, with the camera positioned perpendicular to the center of the convex lower part, with a cone-shaped food product and other than round, oval and pear-shaped forms, photofixation is carried out immediately for the entire product in one revolution, and the product is rotated until the starting point is reached, after which his manager they turn 90 degrees with the side open for the camera on the frame and again perform the photographic fixation operation, then the resulting photographs are sent to the control unit, and from it to a personal computer, in which they analyze for defects and, if detected, determine the boundaries, coordinates and the depth of the defect, moreover, if a defect is detected and its parameters do not meet the criteria laid down in the software of a personal computer, then such a food product is transferred to a conveyor for product disposal, if a defect is detected, but its parameters are within acceptable limits, then such a product is transferred to a conveyor designed for those food products in which it is possible to remove a defect without affecting the production process, and if no defects are found, then such a product is transferred to a conveyor intended for those food products that are completely defect-free, and the movement to the conveyors is carried out by issuing signals by a personal computer to the control unit, and from it to the robotic arm.

In addition, photofixation of a round, oval and pear-shaped food product is carried out with a movement at an angle of not more than 20 degrees.

In addition, an algorithm that allows you to determine the presence of a defect in a particular type of product, its boundaries, coordinates and depth, is implemented on the basis of convolutional neural networks.

In addition, the classification of the image for belonging to a food product to one of the types is carried out by an algorithm that implements the support vector machine.

In addition, the coloring of the compartments of the food conveyor has a different color from those foodstuffs that are planned to be used in sorting.

In addition, the presence of a food product in the compartment for placing food products is determined by the tracking system embedded in the control unit by means of distinguishing two colors between the color of the conveyor compartment and the food product.

The proposed method is illustrated in the drawing, which shows a functional diagram.

The diagram shows:

1 - Industrial robotic arm

2 - The gripper of the robotic arm

3 - Robotic arm camera

4 - Conveyor (No. 1) for supplying unsorted food products

5 - Compartments for placing food products of conveyor No. 1

6 - Conveyor motor No. 1

7 - Conveyor (No. 2) for defect-free food products

8 - Compartments for placing food products of conveyor No. 2

9 - Conveyor motor No. 2

10 - Conveyor (No. 3) for food products from which it is possible to remove the defect without affecting the production process

11 - Compartments for placing food products of conveyor No. 3

12 - Conveyor motor No. 3

13 - Conveyor (No. 4) for food products to be disposed of

14 - Compartments for placing food products of the conveyor No. 4

15 - Conveyor motor No. 4

16 - Control unit with controller

17 - Personal computer

18 - Frame

19 - Camera on frame

The proposed method is carried out in the following sequence. The food products arrive on a conveyor (4) which is held on a frame (18). A camera (19), a robotic arm (1), a conveyor for supplying unsorted food products (4) and at least three conveyors (7, 10, 13) are installed on the frame (18), designed to further move the product, depending on the availability and the nature of the defects found. All conveyors are made with compartments (5, 8, 11, 14) for placing food products in them. Moreover, the color of these compartments has a different color from those food products that are planned to be used. A control unit (16) is connected to the robot-manipulator (1), which has a built-in programmable logic controller that sends control signals to the robot-manipulator (1). The control unit (16) contains tracking and control systems.

The robotic arm (1) is equipped with a gripper (2) and a camera (3). A personal computer (17) is connected to the control unit (16), which outputs signals to the control unit (16). Software is installed on a personal computer (17) with built-in algorithms that allow classifying the type of product and recognizing the type of defect within a certain type of classified product, the specific area where this defect is located and predicting its depth.

After food products appear in the compartment (5) for placing food products on the conveyor (4), the tracking system embedded in the control unit with the controller (16) will establish this by means of two colors difference between the color of the conveyor compartment and the food product. If such a difference is established, then the tracking system will issue signals to stop the conveyor (4). Moreover, the camera of the robot-manipulator (3) is positioned at the moment of movement of the conveyor with unsorted food products in such a way as not to impede the movement of the conveyor (4) and view the contents in the compartments (5) at a right angle.

After the conveyor (4) stops, the camera of the robot-manipulator (3) once again photographs the contents of the cell, but in a higher resolution than when the conveyor (4) moves. After that, the image is transferred to a personal computer (17), where an algorithm, for example, implementing the support vector machine, classifies this image as belonging to one of the product types.

After the type of product is determined, the presence of a defect in a particular type of product is determined using an algorithm built into a personal computer (17). This algorithm, for example, can be implemented on the basis of convolutional neural networks. When a defect is detected, the boundaries and coordinates of this defect are determined, as well as the depth of occurrence. If the defect area or depth do not correspond to the parameters that were entered in the software settings on a personal computer (17), then this food product is captured by the gripper (2) and moved to a conveyor designed for product disposal.

If a defect is not detected or detected, but its area and depth are less than the limit values set in the software settings on a personal computer (17), then the food product capture coordinates are calculated taking into account its classified type, specific shape and minimizing the overlap of its surface with dimensions gripper (2) when held. Then this food product is captured by the gripper (2) and moves to the chamber on the frame (19). Then the food product is rotated through a certain angle, depending on the classified species, and the photographic fixation operation is carried out again. If the food product has a round or oval shape, then its upper hemisphere is photographed first, and then, in one more revolution, the lower hemisphere, while the camera (19) is placed perpendicular to the center of the 45 degree parallel. In the case of a pear-shaped form, first, the upper part of the food product is photographed with the location relative to the chamber (19) perpendicular to the center of the elongated vertical part, and then, in one more turn, the lower part, with the location relative to the chamber (19) perpendicular to the center of the convex lower part. With a cone-shaped food product and other forms, photofixation is carried out immediately for the entire product in one revolution. Moreover, the product is rotated until the starting point is reached. Then the food product is rotated 90 degrees with the side open to the camera (19) on the frame (18), and the photofixation operation is carried out again.

The obtained photographic images are sent to the control unit (16), and from it to the personal computer (17), in which they are analyzed for the presence of defects and, if detected, the boundaries and coordinates of this defect are determined, as well as the depth of occurrence. Moreover, if defects are detected and their parameters do not meet the criteria laid down in the software of a personal computer (17), then such a product is transferred to a conveyor for product disposal. If defects are found, but their parameters are within acceptable limits, then such a product is transferred to a conveyor designed for those food products in which it is possible to remove the defect without compromising the production process. If no defects are found, then such a product is transferred to a conveyor intended for those food products that are completely defect-free.

Thus, the claimed method makes it possible to determine the state of the surface, the type and depth of defects in food products. The method can be used in industries where pre-treatment of food products such as vegetables and fruits is required for their subsequent processing.

Claims (6)

1. A method for quality control and determination of parameters of defects in food products, such as vegetables or fruits, when they are sorted on a conveyor, including the supply of unsorted food products along the conveyor, control with a controller, the formation of images obtained from cameras, and their analysis by means data processing system, characterized in that the conveyor for supplying unsorted food products is fixed on a frame on which a camera, a robotic arm, a control unit with tracking and control systems and a programmable logic controller are installed, conveyors for defect-free food products, for food products, which it is possible to remove a defect without damaging the production process and for food products to be disposed of, motors and compartments for placing a food product are installed on all conveyors, while using a robotic arm equipped with a gripper, a camera and connected to the control unit, and to the control unita personal computer is connected in which algorithms are built in that allow classifying the type of product and recognizing the type of defect within a certain type of classified product, the specific area of the location of this defect and predicting the depth of its occurrence, while the camera of the robotic arm is positioned at the time of movement of the conveyor with unsorted food products at a right angle with respect to the compartments of this conveyor and after the tracking system determines the presence of a product in the conveyor compartment, the conveyor is stopped by sending signals to the control unit, after the conveyor is stopped by the camera of the robot-manipulator, the contents of the compartment are photographed again, but in a larger resolution than when the conveyor is moving, and transfer the photo image to a personal computer, the algorithms of which classify it as belonging to one of the types of food products and determine the presence of a defect, and upon detection, determine its boundaries, which dinat and depth of occurrence, and, if the area of the defect or the depth of occurrence do not correspond to the parameters entered in the software settings on the personal computer, then send a signal to the control unit to move the food product by the gripping device of the robotic arm, to the conveyor intended for disposal products, and the robot-manipulator executes this command, and if the defect is not detected or detected, but its area and depth are less than the limit values set in the software settings on the personal computer, then the food product capture coordinates are calculated taking into account its classified type , a specific shape and minimizing the overlap of its surface with the dimensions of the gripper when holding, and send a signal to capture it, the gripper of the robotic arm captures the food product and moves it to the camera located on the frame, and rotates it with periodicity and angle m of movement such that are incorporated in the settings of the algorithms on a personal computer, at each moment of movement, a photographic fixation of the food product is carried out, and if the food product has a round or oval shape, then its upper hemisphere is photographed first, and then, for one more turn, the lower of the hemisphere, while the camera is placed perpendicular to the center of the 45° parallel, with a pear-shaped form, the upper part of the food product is first photographed with the camera positioned perpendicular to the center of the elongated vertical part, and then, in one more turn, the lower part, with the camera positioned perpendicular to the center of the convex lower part , with a cone-shaped food product and other than round, oval and pear-shaped shapes, photographic fixation is carried out immediately for the entire product in one revolution, and the product is rotated until the starting point is reached, after which it is rotated 90 ° with the side open for the camera on the frame, and again a photographic fixation operation is performed, then the obtained photographic images are sent to the control unit, and from it to a personal computer, in which they are analyzed for the presence of defects and, if detected, the boundaries, coordinates and depth of the defect are determined, moreover, if the defect is detected and its parameters do not meet the criteria laid down in the software of a personal computer, then such a food product is transferred to a conveyor for the disposal of products, if a defect is detected, but its parameters are within acceptable limits, then such a product is transferred to a conveyor designed for those food products that have remove the defect without damaging the production process, and if no defects are found, then such a product is transferred to a conveyor designed for those food products that are completely defect-free, and the movements to the conveyors are carried out by issuing signals by a personal computer to the control unit, and with n it to the robotic arm. 2. The method according to claim 1, characterized in that the photofixation of a food product of round, oval and pear-shaped shapes is carried out with a movement at an angle of not more than 20°. 3. The method according to claim 1, characterized in that the algorithm that allows you to determine the presence of a defect in a particular type of product, its boundaries, coordinates and depth, is implemented on the basis of convolutional neural networks. 4. The method according to claim 1, characterized in that the classification of the image for belonging to a food product to one of the types is carried out by an algorithm that implements the support vector machine. 5. The method according to claim 1, characterized in that the color of the compartments of the food conveyor has a different color from those food products that are planned to be used in sorting. 6. The method according to claim 1, characterized in that the tracking system embedded in the control unit determines the presence of a food product in the compartment for placing food products by means of distinguishing two colors between the color of the conveyor compartment and the food product.

Images