JP2022129666A - Legume selection system and legume selection device - Google Patents

Abstract

To provide a legume selection system and a legume selection device that have higher selection accuracy for legumes by using an artificial intelligence (AI).SOLUTION: A legume selection system is for testing or selecting legumes with shells by analyzing an image of the legumes, and includes: image acquisition means for acquiring an image taken by imaging means; and image analysis means for analyzing the image acquired by the image acquisition means. The image analysis means includes: a legume image extraction means for extracting an image of each legume in the acquired image; and a contraction processing extraction unit for executing contraction processing of reducing an outline region of the images of the extracted legumes.SELECTED DRAWING: Figure 13

Description

TECHNICAL FIELD The present invention relates to a pod sorting system and a pod sorting apparatus for sorting out edamame and other pods, particularly pods with pods still attached. The present invention also relates to a pod sorting system and a pod sorting apparatus that can accurately sort pods with pods using artificial intelligence (AI).

Capsule is a fruit found in many leguminous plants, and includes kidney beans, peas, silk trees, soybeans, red buds, lupines, clovers, vetch peas, and the like. These pods are often harvested with the pods attached, and green soybeans harvested from immature soybeans are often served with the pods attached. When the pods are distributed to the market, they are sorted in order to manage their quality and grade. Several techniques for selecting pods with pods have also been proposed.

The applicant of the present application has proposed Patent Document 1 (International Publication No. 2016/133175) regarding such a pod sorting system and a pod sorting apparatus. In this document, when observing the seeds inside the pods, it is possible to inspect the condition of the seeds inside the pods (particularly the number and size of the grains) by eliminating the need for photographing with transmitted light and photographing the seeds with reflected light. Desirably, a pod sorting system and a pod sorting apparatus are realized in which the pods are inspected or sorted based on the value of the vegetation index calculated from the photographed image.

Such a pod sorting system inspects or sorts the pods by analyzing an image photographed of the pods with pods. and image analysis means for analyzing the captured image, and the image analysis means includes a size inspection for measuring the length of each capsule and the width of each capsule based on the photographed image, and the length of the measured capsule. or, alternatively, a pod sorting system that inspects or sorts pods according to the value of the vegetation index. In this document, it is possible to inspect the condition of the seeds in the pods (especially the number and size of the grains) and the defects such as black spots, discoloration, cracks and whiskers.

Moreover, Patent Document 2 (Japanese Unexamined Patent Application Publication No. 2008-20347) proposes a structure for judging the quality of a pod. In this document, an imaging means is provided for capturing an image of a pod with transmitted light transmitted through the pod containing beans in the pod. There have been proposed structures for judging the size of a pod, the state of tearing at the ends of the pod in the longitudinal direction, the shape of the pod, and the like.

International Publication No. 2016/133175 Japanese Unexamined Patent Application Publication No. 2008-20347

Various techniques for selecting pods with pods using a camera have been proposed so far. However, conventionally proposed pod sorting apparatuses still have room for improvement in terms of accuracy in judging non-defective products or defective products. Therefore, the first object of the present invention is to provide a pod sorting system and a pod sorting apparatus that improve the sorting accuracy of pods by using artificial intelligence (AI) or the like.

In addition, although it was possible to determine black spots and discoloration in the conventional pod sorting device, due to the characteristics of image recognition, it is difficult to accurately determine black spots or discoloration by image inspection due to the influence of the contour area of the pod. was difficult. Therefore, another object of the present invention is to provide a pod sorting system and a pod sorting apparatus that can accurately determine black spots or discoloration existing in areas other than the outline of the pod.

In addition, although the number of bean grains present in the pods is determined in the conventional pod sorting device, it is difficult to improve the accuracy of the appearance inspection because it targets natural pods. Met. Therefore, another object of the present invention is to provide a pod sorting system and a pod sorting apparatus that can accurately determine the number of grains in a pod from the appearance of pods of various shapes.

In addition, although it is possible to sort good pods from defective pods in the previously provided pod sorting apparatus, it was not possible to check the judgment criteria at the time of the sorting afterwards. Therefore, the present invention provides a pod sorting system and a pod sorting apparatus that can verify the past judgment results later, and make it possible to perform the validity of the judgment criteria and the correction of the judgment criteria after that. is a different issue.

In addition, in the conventionally provided pod sorting apparatus, a camera using an image sensor such as a CCD element or a CMOS element is used as an imaging means for picking up an image of the pod. image will be distorted. Such distortion causes an error in judging the size of the pods, etc., and is an obstacle to accurate pod selection. Therefore, it is an object of the present invention to provide a pod sorting system and a pod sorting apparatus that eliminates the distortion of the photographed image and enhances the sorting accuracy of the pods, especially considering that the inspection object is the pod. is another subject.

Also, when an area sensor is used as the imaging means, it is conceivable to determine the same object to be inspected a plurality of times by moving the pod, which is the object to be inspected. If the same object to be inspected is judged and sorted multiple times, the credibility of the ratio of non-defective products or the ratio of defective pods of the harvested pods will decrease. Therefore, another object of the present invention is to provide a pod sorting system and a pod sorting apparatus capable of calculating an accurate value for the ratio of good products or defective products of the sorted pods.

In addition, even in the conventional fruit sorting device, chipping etc. were determined based on the shape of the fruit, but it was difficult to accurately determine the presence or absence of chipping in various shapes of the fruit. . Therefore, another object of the present invention is to provide a pod sorting system and a pod sorting apparatus that can accurately determine the presence or absence of chipping in consideration of the unique characteristics of the shape of the pods.

In the conventionally provided pod sorting apparatus, the judgment criteria must be set individually at the time of installation, and the setting work at the initial stage of introduction is complicated. This is because the pods to be inspected differ greatly depending on the region where they were harvested and the environment in which they were grown, so it was not possible to establish a unified judgment standard. On the other hand, the same user may use a plurality of pod sorting apparatuses, and even in such a case, it is a waste of work to set the criteria for each installed pod sorting apparatus. Therefore, in the present invention, when the pods to be judged are similar, the pod sorting system and the Another object is to provide a pod sorting device.

In order to solve the above problems, the present inventor has further improved the previously proposed pod sorting system and pod sorting apparatus, reviewed the criteria and conditions, and enhanced or introduced a system for judging good or defective products. To provide a pod sorting system and a pod sorting device that facilitate setting work at times. Then, a pod sorting system and a pod sorting apparatus are provided that are preferably capable of using artificial intelligence (AI) when selecting the criterion.

That is, the present invention provides a pod sorting system and a pod sorting apparatus for reliably determining the presence or absence of black spots in a pod. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; An image analysis means for analyzing the image acquired by the acquisition means is provided, and the image analysis means includes a capsule image extraction unit for extracting an image of each capsule in the acquired image, and an image of each extracted capsule. Provided are a pod sorting system characterized by comprising a contraction processing execution unit that executes contraction processing for reducing contour areas, and a pod sorting device using the said seed fruit sorting system.

In addition, in the present invention, a pod sorting system and a pod sorting apparatus that can accurately determine a single pod by setting a pod length measurement standard in consideration of the characteristics of pods, especially green soybeans. offer. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; and an image analysis means for analyzing the image acquired by the acquisition means. A pod length calculation unit for calculating the length of the pod is provided, and the pod length calculation unit calculates the length of the pod in the region where the pod width is a predetermined value, which is the width of the pod calculated from the image of each extracted pod. Provided are a pod sorting system characterized by calculating the thickness of the pods, and a pod sorting apparatus using the said pod sorting system.

In addition, the present invention provides a pod sorting system and a pod sorting apparatus that can review the measurement criteria and the like by storing the measurement data used for judging the pods as log information. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; An image analysis means for analyzing an image acquired by the acquisition means, and a log information holding unit for holding the analysis result of the image analysis means as log information; A pod sorting device using the system is provided. In particular, by using the log information, an artificial intelligence device can be constructed as a learned model, and by using the artificial intelligence device, more accurate judgment authority can be set. In addition, by enabling the log information to be viewed from an external terminal via the Internet, the current inspection status can be monitored in real time from a remote location.

In addition, the present invention provides a pod sorting system and a pod sorting apparatus that increase the accuracy of judging the pods by verifying the analysis result of the pods and the image to be analyzed. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; A pod sorting system characterized by comprising an image analysis means for analyzing an image acquired by the acquisition means, and further comprising an image holding section for holding an image used for analysis by the image analysis means, and the pod sorting system. To provide a pod sorting device using

In addition, the present invention provides a pod sorting system and a pod sorting apparatus that can solve the problem of misjudgment caused by a camera that photographs the pods to be sorted. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; an image analysis means for analyzing an image acquired by the acquisition means, wherein the image analysis means comprises a distortion correction unit for performing distortion correction on the image acquired by the image acquisition means; and A pod sorting device using the pod sorting system is provided.

In addition, the present invention provides a pod sorting system and a pod sorting apparatus that can prevent the deterioration of the credibility of the ratio of non-defective products or the ratio of defective products caused by discriminating the same pods multiple times. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; and image analysis means for analyzing the image acquired by the acquisition means, wherein the image analysis means calculates the movement position of the image acquired by the image acquisition means, and photographs the pods with pods present at the movement position. Provided are a pod sorting system characterized by excluding the image obtained from analysis, and a pod sorting apparatus using the said pod sorting system.

In addition, in the present invention, when installing a plurality of pod sorting systems or pod sorting devices, initial settings, that is, operations such as adjustment of criteria and various devices (including cameras and lighting) can be easily performed. Kind Code: A1 A sorting system and a pod sorting device are provided. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; and an image analysis means for analyzing the image acquired by the acquisition means, wherein the image analysis means sets analysis criteria based on analysis information setting information acquired from another plant selection system. A system and a pod sorting device using the pod sorting system are provided.

Further, the present invention provides a pod sorting system and a pod sorting apparatus capable of accurately determining the presence or absence of defects in pods, particularly edamame. Specifically, a pod sorting system for inspecting or sorting the pods by analyzing an image of the pods with pods photographed, comprising an image acquiring means for acquiring an image photographed by a photographing means; An image analysis means for analyzing the image acquired by the acquisition means is provided, and the image analysis means includes a capsule image extraction unit for extracting an image of each individual capsule in the acquired image, and a plurality of extracted images of each individual capsule. A pod sorting system characterized by dividing the pods for a while and calculating the area ratio between the base end and the tip in the length direction to determine the presence or absence of chipping in the pods, and a pod sorting device using the said pod sorting system. I will provide a.

The pod sorting system according to the present invention can be formed using a computer. Specifically, it can be embodied as a computer in which processing contents and processing procedures are controlled by computer software. The image analysis means can be composed of a CPU and a memory that perform processing by executing computer software.

In addition, the pod sorting system includes transport means such as a belt conveyor and a vibration feeder for transporting the pods, and photographing means such as a digital camera and a digital video camera for photographing the transported pods. Furthermore, it can be accompanied by a removing means for removing the judgment results that do not meet the criteria.

In addition, by combining the pod sorting system configured as described above with a pod conveying means or the like, a pod sorting apparatus capable of continuously sorting pods can be obtained. That is, transporting means for transporting the inspection object, imaging means for photographing the inspection object transported by the transporting means, and capturing the image taken by the imaging means, and the pod sorting means for analyzing the image. and provides a pod sorting device using the pod sorting system as the pod sorting means.

According to this pod sorting apparatus, the pods to be inspected are continuously transported by the transport means, photographed by the photographing means, and analyzed by the pod sorting system according to the present invention to detect non-standard pods. can be found and removed if necessary.

In addition, in the capsule sorting apparatus according to the present invention, among the inspection objects conveyed by the conveying means, air is blown to the inspection object specified by the capsule sorting means, and the inspection object that is out of the standard It is desirable to have a discharge means for blowing off with air. At that time, the image analysis means in the said capsule sorting means can specify the centers of the length and width from the result of the size inspection for measuring the length and width of each individual capsule from the photographed image. Therefore, it is desirable that the discharging means is configured to blow air toward the center of the length and width of the object to be inspected designated by the pod sorting means. The pods (non-standard pods) that have been blown with air to the center are reliably extracted, and even if a large amount of sorting is required, the harvested pods are inspected and sorted accurately and reliably. can do things

In the edamame sorting system of the present invention and the pod sorting apparatus using the same, the image analysis means for analyzing the image acquired by the image acquisition means includes a pod image extraction unit for extracting the image of each pod in the acquired image. and an erosion processing execution unit that executes erosion processing for reducing the outline area of the extracted individual pod images, the black dots present in areas other than the pod outline areas are removed without being affected by the pod outline areas. Alternatively, it is possible to provide a pod sorting system and a pod sorting apparatus capable of accurately determining discoloration.

Further, the image analysis means for analyzing the image acquired by the image acquisition means includes a pod image extraction section for extracting an image of each pod in the acquired image, and calculating the length of the pod from the extracted individual pod images. A pod length calculation unit is provided, and the pod length calculation unit calculates the length of the pod in the region where the pod width is a predetermined value, which is the width of the pod calculated from the image of each extracted pod. Especially for green soybeans, it is possible to provide a pod sorting system and a pod sorting apparatus that can clearly determine the difference in the number of grains in a pod even if the pod length is the same.

In addition, when a log information holding unit is provided to hold the analysis results of the image analysis means as log information, the rate of non-defective products (standard products), defective product ratio (non-standard products), or defective products of the sorting object is determined. It is possible to provide a fruit sorting system and a pod sorting device that can confirm in detail the cause of the problem. and an image acquisition means for acquiring an image photographed by the photographing means, and an image analysis means for analyzing the image acquired by the image acquisition means. When provided, it is possible to provide a pod sorting system and a pod sorting apparatus that can verify the judgment criteria at the time of sorting later, and make it possible to correct the validity of the judgment criteria and then correct the judgment criteria.

Further, when the image analysis means includes a distortion correction section that corrects distortion of the image acquired by the image acquisition means, the distortion of the image in the area away from the center due to the characteristics of the camera is eliminated, It is possible to provide a pod sorting system and a pod sorting device with improved sorting accuracy.

In addition, when the image analysis means calculates the movement position for the image acquired by the image acquisition means and excludes the image of the pod with the pod present at the movement position from the analysis target, the same inspection object It is possible to provide a pod sorting system and a pod sorting device capable of calculating an accurate value for the ratio of good products or the ratio of defective products of the sorted pods by eliminating the need to judge and sort the above multiple times.

In addition, the image analysis means includes a capsule image extraction unit that extracts an image of each capsule in the acquired image, and divides each extracted image of each capsule into a plurality of equal parts, the base end and the tip in the length direction When calculating the area ratio in and determining the presence or absence of chipping in the capsule, a capsule sorting system that can accurately determine the presence or absence of chipping in consideration of the characteristics of the shape unique to the capsule, especially edamame, and A pod sorter can be provided.

When the image analysis means sets the analysis criteria based on the analysis information setting information acquired from other seed seed sorting systems, setting the determination standards for each installed seed seed sorting device is a waste of work. A pod sorting system and a pod sorting system and a pod sorting system in which the determination conditions of the initially set pod sorting device can be expanded to other pod sorting devices when the pods to be judged are similar to each other. Equipment can be provided.

FIG. 1 is an overall configuration diagram showing an edamame sorting apparatus configured using the edamame sorting system according to the present embodiment. Block diagram showing hardware configuration Flowchart showing details of basic processing Processing content diagram showing the content of basic processing Flowchart showing details of processing for grain count inspection Process content diagram for particle number inspection Diagram of process for judging pod length in grain number inspection Flowchart showing the details of the single-grain inspection process Flowchart showing processing details in chipping inspection Processing details for chipping inspection Flowchart showing details of processing for black spot inspection Front view showing sunspots Setting screen showing reduction processing in black point inspection Screen showing inspection log Setting screen for prohibiting multiple inspections of the same edamame Inspection image save setting screen Camera correction setting screen Calibration setting screen

One embodiment of the pod sorting system according to the present invention will be specifically described below with reference to the drawings. In particular, in the present embodiment, a specific description will be given of a pod sorting system for sorting green soybeans out of pods. However, the pod sorting system and pod sorting apparatus according to the present embodiment can also be used to sort pods other than green soybeans, such as soybeans, kidney beans, chickpeas, adzuki beans, and peanuts. Moreover, the pod sorting system and the pod sorting apparatus according to the present embodiment are for inspecting the pods with pods, so the pods indicate the pods with pods.

FIG. 1 shows the overall configuration of a pod sorting device (hereinafter referred to as “edamame sorting device”) configured using the pod sorting system (hereinafter referred to as “edamame sorting system 40”) according to the present embodiment. It is a diagram. This green soybean sorting apparatus includes a hopper section 10 that stores green soybeans W to be sorted and continuously supplies them to a sorting line, a first conveying means 20 that conveys the green soybeans W supplied from the hopper section 10, and the A second conveying means 30 for inverting and conveying green soybeans W conveyed by the first conveying means 20, photographing means 21 and 31 provided in each conveying means, and analysis of images photographed by the photographing means 21 and 31. edamame sorting system 40, conveyance detecting means 22, 32 provided in each conveying means for detecting the conveying speed or conveying amount, and discharge means 23, 33 provided at the drop portion of edamame W on the end side of each conveying means. It consists of

The hopper portion 10 has a space portion for accommodating edamame W (pods) to be sorted. It also has a discharge structure for supplying green soybeans W stored inside the hopper portion 10 to a conveying means consisting of a first conveying means 20 and a second conveying means 30 . Such a discharge structure is a belt conveyor or the like provided on the bottom surface of the hopper section 10, and moves the green soybeans W stored in the hopper section 10 forward. However, the hopper section 10 may have another configuration as long as the green soybeans W stored therein can be delivered to the conveying means for inspection.

The green soybeans W discharged from the hopper portion 10 are supplied to a conveying means consisting of a first conveying means 20 and a second conveying means 30 . At that time, in order to prevent the green soybeans W dropped from the hopper portion 10 from overlapping each other, a vibrating means (not shown) for vibrating and conveying the green soybeans W discharged from the hopper may be provided at the dropping portion. desirable.

The first conveying means 20 and the second conveying means 30 can each be formed by a belt conveyor, a vibrating feeder, or the like. The green soybeans W supplied from the hopper section 10 are first transported by the first transport means 20 . The first conveying means 20 may desirably be provided with an aligning section for aligning the direction of the green soybeans W to be conveyed. Such an aligning portion can be formed by arranging elongated aligning plates or wires (not shown) at predetermined intervals in the direction in which the green soybeans W are transported, for example. The conveyed green soybeans W collide with the aligning plate so as to be oriented vertically in the conveying direction.

Photographing means 21 for photographing green soybeans W being conveyed by the first conveying means 20 is provided above the first conveying means 20 . A camera or a video camera can be used as the photographing means 21 . It is preferable that the photographing means 21 photograph the transported edamame W as a still image or a moving image, and acquire the photographed image as electronic data so that the photographed image can be directly imported into the edamame sorting system 40 for analysis. It is also desirable to provide lighting for stably illuminating the green soybeans W present in the photographing range of the photographing means 21 . This is because the brightness of the photographed image is stabilized by making the brightness of the light illuminating the edamame W constant.

The image photographed by the photographing means 21 is sent to an edamame sorting system 40 comprising a computer or the like. The green soybean sorting system 40 inspects the green soybeans W to be sorted based on the acquired photographed image, and inspects at least the size and the number of grains of the green soybeans W, the presence or absence of chipping, the presence or absence of black spots, or discoloration. Then, the green soybeans W that deviate from the standard are identified. The configuration and processing of this green soybean sorting system 40 will be described later.

The first transport means 20 is provided with transport detection means 22 for acquiring the transport speed and transport distance. By associating the transportation information of the green soybeans W with the photographed image, it is possible to identify where the green soybeans W selected by the green soybean sorting system 40 (non-standard green soybeans W) are transported. By specifying the non-standard edamame W selected by the edamame sorting system 40 in the transportation process, the non-standard edamame W can be discharged out of the transportation line. A rotary encoder, a linear encoder, or the like can be used as the transport detection means 22 .

In this embodiment, the non-standard green soybeans W are removed when they drop from the end of the first conveying means 20 . That is, at the end of the first conveying means 20, there is provided a drop section for dropping the edamame W being conveyed to the second conveying means 30. When dropping from the dropping section, the edamame sorting system 40 identifies The green soybeans W (non-standard green soybeans W) are discharged from the conveying line by the discharging means 23 . The discharging means 23 can be configured to blow air against the specified green soybeans W to blow them out of the conveying line, or can be formed in various configurations capable of taking out the specified green soybeans W.

As described above, the green soybeans W transported by the first transport means 20 and judged to be nonstandard by the green soybean sorting system 40 are discharged by the discharge means 23 , and the remaining green soybeans W are transported by the second transport means 30 . When conveyed by the second conveying means 30, the green soybeans W are desirably conveyed in a state inverted from the state conveyed by the first conveying means 20. As shown in FIG. Therefore, it is desirable to provide reversing means for reversing the green soybeans W to be transported after the first transport means 20 or at the start end side of the second transport means 30 (first transport means 20 side). Such reversing means can reverse the green soybeans W to be transported, for example, by utilizing the drop when the green soybeans W are dropped, or by applying a rotational force from a belt or roller to either surface of the transported pods. Further, by causing the second conveying means 30 to convey in the opposite direction to the first conveying means 20, the green soybeans W dropped by the second conveying means 30 can be reversed.

The green soybeans W transported by the second transport means 30 are also sorted in the same manner as transported by the first transport means 20 . That is, the green soybeans W transported by the second transport means 30 are photographed by the photographing means 31, the photographed image is analyzed by the green soybean sorting system 40, and non-standard green soybeans W are specified. At that time, the conveying speed and distance of the green soybeans are acquired from the conveying detection means 32 consisting of a rotary encoder. Then, the nonstandard edamame W is discharged by blowing air from the discharge means 33 at the end of the conveying means. Since the processing in the second conveying means 30 is the same as the processing in the first conveying means 20, in the drawings, the same configuration as the first conveying line is indicated by adding the value of "10" to the reference numeral, and the details detailed description is omitted.

FIG. 2 shows an example of the hardware configuration of a computer that constitutes the pod sorting system according to this embodiment. However, the computer 500 in FIG. 2 merely exemplifies a typical configuration of the pod sorting system, and the edamame sorting system is a dedicated computer as long as it executes the arithmetic unit, memory and program for image analysis. It may be configured as a device.

The computer 500 shown in FIG. 2 includes a CPU 501, a memory 502, an audio output device 503, a network interface 504, a display controller 505, a display 506, an input device interface 507, a keyboard 508, a mouse 509, an external storage device 510, and an external recording medium drive. It comprises a device 511 and a bus 512 connecting these components together.

The CPU 501 controls the operation of each component of the computer 500, controls the execution of each process of the green soybean sorting system, and controls the operation under the control of the OS. The memory 502 is normally composed of ROM (Read Only Memory), which is non-volatile memory, and RAM (Random Access Memory), which is volatile memory, and corresponds to a storage unit. The ROM stores programs and the like that are executed by the green soybean sorting system when the computer 500 is started. In the RAM, it is executed by the CPU 501, analyzes the image acquired from the photographing means, calculates the size and number of edamame beans, and determines the presence or absence of chipping, whiskers, black spots, discoloration, cracks, activity, etc. programs and data used by those programs while they are running.

The audio output device 503 is a device such as a speaker that outputs audio, and the network interface 504 is an interface for connecting to a network 520 for exchanging information with various devices. A display controller 505 is a dedicated controller for processing drawing commands issued by the CPU 501, and outputs drawing data to a display 506 as a display unit. Such a display 506 is a display device configured by an LCD or the like.

The input device interface 507 receives a signal input from an input/output device such as the keyboard 508, mouse 509, or touch pad, and provides a predetermined command to the CPU 501 according to the signal pattern. A keyboard 508 and a mouse 509 are required when performing operations such as program execution and setting. The external storage device 510 is also included in the category of storage means in this specification. Such an external storage device 510 can be composed of a storage device such as a hard disk drive (HDD). The above-described programs and data are recorded in this device, and loaded from there into the RAM of the memory 502 as required during execution. The external recording medium drive device 511 accesses the recording surface of a portable external recording medium 530 such as a CD (Compact Disc), MO (Magnet-Optical Disc), DVD (Digital Versatile Disc), etc. It is a device that reads data stored in

The green soybean sorting system according to the present embodiment is formed using a computer configured as described above, and executes the processes shown in FIGS. 3 to 18 below.

≪Basic processing≫
FIG. 3 is a flow chart showing the contents of basic processing to be executed on the photographed image acquired from the photographing means, and FIG. 4 is a processing content diagram showing the contents of this basic processing. When the basic processing is executed, the green soybean sorting system first acquires a photographed image showing a plurality of green soybeans from the photographing means and records it in memory (S11). Then, as shown in FIG. 4(B), an image showing a plurality of green soybeans (captured image acquired by the photographing means) is subjected to labeling processing for each green soybean (S12), and an image for each green soybean is extracted. . When analyzing the photographed image, it is desirable that a plurality of pods contained in the photographed image be extracted for each green soybean, and the contour shape of the pods can be specified by performing labeling processing.

Then, angle correction processing is performed on the extracted image for each green soybean (S13). The length along the major axis (hereinafter also referred to as “pod length”) and the width along the minor axis (hereinafter also referred to as “pod width”) are measured for the corrected image that has undergone angle correction. This is because angle correction enables more accurate measurement of the length and width of the pod in the pod. In this angle correction processing, image processing is performed so that the elongated direction (longitudinal direction) of each green soybean image faces the same direction, either vertically or horizontally. Then, as shown in FIG. 4C, the angle-corrected image is binarized (S15). I have to.

In the case of automating the sorting of the pods, it is necessary to convey them by a conveying means such as a belt conveyor, and it is necessary to photograph and inspect the pods being conveyed. However, it is practically difficult to precisely align the orientation of the pods conveyed by the conveying means. Therefore, as described above, by photographing the casually transported pods and correcting the orientation, the length and width of the pods can be accurately measured while meeting the actual inspection conditions. system.

≪Grain count inspection≫
FIG. 5 is a flow chart showing the contents of processing for grain number inspection, FIG. 6 is a diagram showing the details of processing for this grain number inspection, and FIG. 7 is a diagram showing the details of processing for determining pod length in grain number inspection. In this particle number inspection, an image (image after labeling processing and angle correction processing) subjected to the basic processing is obtained as an inspection image. Since this inspection image is created for each green soybean conveyed by the conveying means, this grain count inspection is executed for all green soybeans.

The image analysis means performs a "size check" to obtain information on the length and width of individual pods from the captured images. Information about the length and width can be obtained as the number of dots (or the number of pixels) based on the pixels in the captured image, and can also be obtained as the actual length and width of the inspection object. When calculating the actual length and width of the inspection object from the captured image, the length specified on the captured image can also be multiplied by a fixed enlargement or scaling constant.

That is, when the inspection image is obtained, the CPU of the green soybean sorting system, which is composed of a computer, reads the number of pixels in the longitudinal direction of the area where the green soybeans are projected in the inspection image, as shown in FIG. 6(A). Then, based on this, the length of the edamame is calculated (S52). This calculation can be performed by multiplying the photographed image by a coefficient for making it an actual measurement value.

Then, when the length of green soybeans calculated by calculation is acquired, the number of grains is calculated based on this length (S54). For the calculation of the number of grains, as shown in FIG. 6, a length/number table 41 that associates the number of seed grains in relation to the length of the capsule is prepared, and the length of the read capsule is searched by image analysis. By searching as a value, the number of seeds corresponding to the length can be calculated and extracted. The program can also be designed to output a constant value (the number of seeds) according to the length of the obtained pods without searching the file system. That is, by measuring the length of the pod and using this value, it is possible to calculate the approximate number of seeds contained within the pod.

When calculating the pod length, it is also possible to set a reference value for the pod width, and to calculate the pod length for a range equal to or greater than this value. That is, as described above, it is possible to roughly estimate the number of grains from the length of the pods in green soybeans. However, even with the same pod length, depending on the size of the seeds in the pod, there may be one or two pods. Therefore, by excluding the length of the region where seeds do not exist, the length of the region where seeds actually exist is calculated, and the number of grains is accurately determined.

FIG. 7 is a processing content diagram for judging pod length in grain number inspection. As shown in FIG. 7A, even if the pod length is the same, the number of seeds contained in the pod may be one or two. Therefore, by setting a reference value (LB) for the pod width and calculating the pod length for values larger than the reference value, it is possible to calculate the effective length for the region where the seeds are present. In this embodiment, the reference value (LB) for the pod width is set to 10 mm, and as a result, as shown in FIG. , the pod length is shorter than the pod length of 2 seeds, and it is judged to be 1 seed. On the other hand, in the case of a pod containing two seeds, the pod length in the area larger than the standard pod width is within the range of the pod length that is judged to be two seeds, so it is possible to accurately determine the number of seeds. .

In addition, when calculating the pod length, it is possible to determine the pod fruit with one seed in the pod by calculating the length of the area below the pod width, which is the reference, and also serves as the reference. The number of seed grains in a pod can also be calculated by calculating the number of protruding portions (convex portions) that are greater than the width of the pod.

In the grain number inspection, it is possible to calculate the number of seeds in the pod and the width of the pod that changes depending on the presence of the seeds. That is, in the grain number inspection in the image analysis means, the position where the pod width is maximum is defined as the convex portion, and the position between the convex portions where the width is narrower than the concave portion and is minimum is defined as the concave portion. Such protrusions and recesses can also be calculated for each seed housed in the pod, thereby enabling detailed understanding and determination of the overall shape of the pod.

In addition, since the contour of the place where the seed exists in the pod is swollen, a convex portion can be set, and since there is a concave portion between the seeds, by confirming the existence of the concave portion, the convex portion can be set. It can be confirmed that there are seeds in the part. When the number of protrusions is calculated to be two or more, the number of grains in the pod can be determined by checking the amount of change in width between the protrusions. When the number of grains is specified only by the length of the pods, even with the same length of pods, depending on the size of the seeds inside, there may be one or two grains. It is for Therefore, it is desirable to confirm that the width change between the convex portions is changed so that one valley portion (concave portion) appears.

As described above, it is also desirable to identify pods containing only one seed contained in the pods in the grain count inspection. Such single-seed pods can first be identified by the total length of the pod. That is, it can be assumed that a capsule having a pod length shorter than the reference length in a region where the pod width is equal to or larger than the predetermined pod width has only one seed. By estimating the number of grains from the length of the pod, it is possible to improve the processing speed in the grain number inspection. A pod having a length equal to or longer than the standard value is judged to have two or more seeds in the pod. However, even if it is determined that there are two or more seeds, only one seed may actually be contained if the seed is large. Therefore, it is also possible to read the width from the photographed image, extract and confirm the convex portion and the concave portion, and calculate the actual number of grains. That is, when the length is longer than the reference value and the number of grains is set to 2, the pod is divided into two regions in the length direction, and the widest protrusion is extracted in each region. As a result, if the number of seeds contained in the pod is two or more, there should be two convex portions and a concave portion between the convex portions. However, when there is only one seed in the pod, even if two protrusions are provided, there is no width between the protrusions that is narrower than both protrusions. There will be no recess. Therefore, it is possible to determine whether or not a pod having a length such that one seed can be contained in the pod is one by confirming whether or not there is a concave portion between the convex portions.

In this way, the process of identifying a pod with one seed inside the pod is particularly effective when sorting green soybeans. In the selection of edamame, the number of seeds in a pod also affects the commercial value. Therefore, the commercial value can be increased by identifying pods with one seed and sorting them.

The pods can be sorted by determining whether the number of grains obtained as a result of the calculation is one (S56). This is because, in the case of edamame, one seed may have a low commercial value, so edamame with one seed is selected and removed in order to increase the commercial value. In judging whether or not the number of seeds contained in the pod is one, first, those judged to be one in the above-mentioned judgment of the number of seeds are excluded. Therefore, green soybeans determined to have one seed in the pod are determined to be non-standard products, and are specified as objects to be discharged by the discharging means.

Then, in the determination of the number of grains, the edamame for which the number of seeds housed in the pod is determined to be two or more is divided into areas according to the number of grains (S58). FIG. 6 shows an example in which two grains are determined. For this reason, the inspection image is divided into two areas (A1 and A2) in the longitudinal direction, as shown in FIG. 6B. Next, in each region, a point with the largest width value is set as a convex portion. In FIG. 6C, the widest point in the first region A1 is set as the first convex portion C1, and the widest point in the second region A2 is set as the second convex portion C2. ing. Then, the value (width) of each convex portion is calculated (S59).

Next, as shown in FIG. 6(D), the width of the SA between the respective convex portions (C1 and C2) is calculated, and as shown in FIG. 6(E), a value smaller than the width of each convex portion and the point where the width is the minimum value is set as the recess, and the width is calculated (S60). Then, the presence or absence of recesses is determined (S61), and if the recesses can be calculated, it is determined that two or more seeds are contained in the pod and that the edamame conforms to the standard, and that the grain number inspection is within the standard ( S62). On the other hand, if the concave portion cannot be calculated, the seed in the pod is determined to be one green soybean, is determined to be out of specification, and is specified as an object to be discharged by the discharge means. That is, the green soybean sorting system determines that the number of seeds in the pod is two from the length of the pod (Fig. 8(A)), and executes the process of identifying the convex portion and the concave portion in the same manner as described above. Specifically, the edamame image is divided into two regions (A1 and A2) (FIG. 8B), and the widest region in each region is assigned to each convex portion (C1 and C2). Set (FIG. 8(C)). Then, a recess is set at a position between the protrusions that is narrower than the protrusions and has the narrowest width (FIG. 8(D)). However, when the number of seeds in the pod is one, the width of this concave portion is the same as that of any convex portion, so that concave portion cannot be set (FIG. 8(E)). Therefore, in this case, it is determined that there is one seed in the pod, and the edamame is identified as non-standard edamame.

By executing the particle number inspection by the above processing, it is possible to achieve high speed processing. That is, since the seeds in the pod temporarily exclude one green soybean depending on the length of the pod, the objects to be inspected can be reduced, thereby speeding up the processing. Furthermore, for green soybeans within the standard length, by further inspecting the convex and concave portions, it is possible to extract green soybeans that actually have one seed in the pod while satisfying the length standard. The accuracy of sorting can be greatly improved. Therefore, edamame (green soybeans with one seed in the pod) can be quickly and accurately sorted by executing the grain count inspection by the above process.

≪Crack inspection≫
Further, the green soybean sorting system can further perform chipping inspection by acquiring an inspection image created by performing basic processing. In this chipping inspection, it is inspected whether or not there is chipping in the pods of green soybeans. In this embodiment, as shown in FIGS. It is divided, and the presence or absence of chipping is determined based on the area ratio of both ends. This is based on the fact that in pods, especially green soybeans, pod chipping often occurs at the ends in the length direction.

In this process, the contour shape is specified by the labeling process, and an inspection image with the long axis direction aligned by the angle correction process is acquired (S91). The obtained inspection image is divided into a plurality of parts with equal lengths in the long axis direction (S92), and the areas of the divided areas are compared at both ends in the length direction (S93). Then, it is determined whether the result of this comparison is within a predetermined standard (S94), and if the difference between the two is small and the ratio is almost the same, it is determined that the pods are missing and out of the standard (S96). ). On the other hand, if the difference between the two is large and the ratio is different, it is determined that the pod is within the standard because there is no pod chipping (S95). This is because, as shown in FIG. 10(A), when there is no chipping at the ends in the length direction of the pod, one end (the tip of the pod) is tapered, and both areas are different. On the other hand, as shown in FIG. 10(B), if there is chipping at one of the ends (particularly, the tip end of the pod), the areas at both ends are almost the same. The area ratio between the ends in the length direction can be set arbitrarily. can judge. Also, the comparison of areas can be determined by the difference in area instead of the ratio. When the pod is divided in the longitudinal direction, it can be divided by the length of the long axis. Furthermore, it may be divided into four or more equal parts. It is also possible to extract a predetermined length of both ends of the pod length without dividing the pod into equal parts and compare the areas.

By comparing the areas at both ends in the length direction in this way, it is possible to accurately determine the presence or absence of chipping even in edamame whose pod width changes in a complicated manner, and to improve the accuracy of edamame sorting. .

≪Black spot inspection≫
FIG. 11 is a flow chart showing the details of processing for performing a black dot inspection, FIG. 12 is a front view showing how black dots appear, and FIGS. . In this black point inspection, first, an image captured by an imaging means such as a camera is subjected to labeling processing and tilt correction processing, and the resulting image is acquired as an inspection image (S111). Then, the obtained inspection image is subjected to reduction processing for cutting the outline (periphery) (S112). This is because, as shown in FIG. 12, what is judged to be a black point in normal sorting of green soybeans is a discolored area (black point) existing in an area other than the outline of the pod, and a discolored area existing in the outline area is judged to be a black point. due to not being done. In addition, it is undeniable that a shadow or the like may be reflected in the inspection image at the time of photographing, and this is to prevent the shadow portion from being recognized as a discolored region.

Such reduction processing creates an image in which the contour area in the inspection image is removed as shown in FIG. 13(B). The amount of trimming the outline can be arbitrarily set on the setting screen in FIG. By doing so, it is possible to set a large number of pixels to be reduced. That is, the number of pixels to be removed in one reduction process is set, and the number of pixels to be removed can be adjusted by performing this process multiple times. However, it is also possible to adjust the degree of reduction processing by arbitrarily adjusting the number of pixels to be removed in one operation.

Then, the area (the number of pixels) of the area discolored as a black spot is calculated (S113), and it is determined whether or not it is within the standard range (S114). If the calculated discolored area is within the standard range, it can be determined to be within the standard (S115), and if it exceeds the standard range, it can be determined to be out of standard (S116).

When recognizing this discolored area, as shown in FIG. 13, at least one of R (red), G (green), and B (blue) in the inspection image has a luminance required to determine that it is a black point. By setting the upper and lower limits of the values, it is possible to make objects within these ranges the target of the black point determination. For example, in FIG. 10, a range of minimum 0 and maximum 60 is set for the luminance value of G (green). As a result, it is possible to accurately determine discoloration such as black spots by excluding discoloration and shadows in the contour area of the pod.

≪Inspection log management≫
In the pod sorting system and pod sorting apparatus according to the present embodiment, by storing the above inspection results, it is possible to determine the ratio of non-defective and defective edamame to be inspected, the cause of defective products, etc. can be confirmed. FIG. 14 shows a pod sorting system and a pod sorting apparatus that hold data used for judging green soybeans as a log. Such an inspection log can be held for each inspection day, each inspection time, or for a certain period of time or for each lot to be inspected (FIG. 14(A)). Can be kept in text format. However, the log may of course be held in another file format.

In the present embodiment, the log information can hold numerical values obtained as a result of judging the inspection image, such as pod length, pod width, area, or RGB luminance value. data such as numerical values can be stored. By retaining such log information, the status of non-defective products (within the standard) and defective products (outside the vaporizer), their ratio, or the breakdown of defective products can be obtained from the description contents (raw data) of a specific log file. can also be confirmed (FIG. 14(C)). Therefore, by storing all the data used for sorting green soybeans as the log information, it is possible to check the status of the green soybeans for each lot classified by the harvesting farm and harvesting area of the green soybeans. .

In this way, when identifying the ratio of non-defective products and defective products from the inspection log and the cause of the defective product, in order to improve the accuracy, it is necessary not to re-assess the edamame that has been judged once during the inspection of edamame. It is desirable to configure

Therefore, as shown in FIG. 15, it is possible to predict the movement position of the edamame once determined, and exclude the edamame existing at the movement position from the determination target. That is, on the setting screen shown in FIG. 15A, the amount of movement in the transfer direction (movement amount Y) and its allowable range are set, and the allowable range of the amount of movement (movement amount X) in the direction intersecting the movement direction is set. do. Then, using this value, as shown in FIG. 15B, when the once detected inspection target moves to the set position, the inspection target existing at that position is excluded from the target. In this embodiment, since the green soybeans to be inspected are moved by conveying means such as a conveyor, in principle, the displacement in the moving direction of the conveyor is predicted. Therefore, for the edamame determined in the photographed inspection image, the movement position in the transport direction after a predetermined time (timing of photographing the inspection image) is predicted, and the edamame existing at that position in the next photographed image is excluded from the inspection object. It is. At this time, the movement amount in the transfer direction (movement amount Y) can be set based on the movement speed of the conveying means such as a conveyor and the conveying detection means 22 and 23, and then the error range of the movement distance is set. be able to. Further, in the present embodiment, an error range can be set for the amount of movement (movement amount X) in the direction intersecting the moving direction of the conveying means. In principle, this assumes that the green soybeans do not move in the direction that intersects the direction of movement. Can also be set.

In this way, by configuring so that the edamame that has been judged once is not judged again, it is possible to accurately judge the ratio of non-defective and defective edamame to be inspected, the cause of being judged as defective, etc. In addition, it is possible to accurately grasp the number of green soybeans to be inspected. The log data can also be transferred to an external server device via a network. In this case, the log information can be checked at any time from an external terminal. Also, machine learning (especially deep learning) can be performed based on the log information to construct an artificial intelligence (AI) of the trained model. In other words, with this log information as an input layer, a computer algorithm derives feature values from the data in this input layer, constructs a network structure that shows the connections between the feature values in the intermediate layer, and finally outputs it. Layered output information (criteria) can be constructed. Furthermore, by configuring the log information so that it can be accessed from an external terminal via the Internet, it is possible to check the sorting status of green soybeans based on the latest log information from a remote location.

<<Confirmation of Judgment Conditions>>
In addition, the pod sorting system and pod sorting apparatus according to the present embodiment can be configured to perform more accurate sorting by adjusting the criteria for judging green soybeans in detail. That is, by holding the image captured by the image capturing means, the image is judged later, and it is confirmed whether or not the reference is appropriate. Therefore, it is possible to move the setting screen of FIG. 16A so that it is not overlaid, and save the captured image of the object (green soybeans) that has been inspected once, as shown in the saved image of FIG. 16B. desirable. Although it is conceivable that such photographed images are stored without limit, the number of images to be stored and the storage capacity (file size) may be set in advance so that the latest image is overwritten and stored.

By applying the algorithm for judgment to the saved photographed image later, the actual data (numerical data, etc.) at the time of judgment can be obtained, and by confirming the validity of the value, Accuracy of determination can be confirmed. By adjusting criteria, camera orientation, and lighting settings, you can improve the accuracy of edamame sorting.

In particular, when judging good/defective green soybeans, it is desirable to set camera and lighting settings and judgment criteria each time, because harvested green soybeans come in various shapes. At this time, it is possible to set the optimum sorting condition according to the situation by setting the judgment criteria from the result log based on the already judged image. Therefore, the storage of photographed images can function particularly effectively in the sorting of edamame, which is a natural product.

≪Device Settings≫
The pod sorting system and the pod sorting apparatus according to the present embodiment are preferably configured so as to facilitate the setting work at the time of installation. For this reason, the pod sorting system and pod sorting apparatus according to the present embodiment preferably have a camera correction function and a device setting function based on the already set pod sorting system and pod sorting apparatus.

FIG. 17A shows an actual inspection image before camera correction, and FIG. 17B shows an inspection image after camera correction. Especially in this embodiment, since a camera using a wide-angle lens is used, the captured image is distorted as the distance from the center increases. Therefore, by using camera correction, the distortion of the image can be eliminated and an accurate inspection image can be obtained. In the present embodiment, calculation can be performed as follows.

Correction amount = 1 + |(green soybean center - camera center)| ÷ correction factor Width after correction = width ÷ correction amount By performing the camera correction, an accurate inspection image can be obtained. By applying the above determination criteria to the accurate inspection image, the accuracy of edamame sorting can be improved.

Moreover, FIG. 18 shows a setting screen for setting other pod sorting systems and pod sorting devices based on the already set pod sorting system and pod sorting device. FIG. 18A shows the values before the settings are adjusted, and FIG. 18B shows the values after the calibration settings are adjusted. In particular, in FIG. 18, the values of the pod sorting device (standard machine) that is already in operation or are properly set are used, and based on this, the setting items of the newly installed pod sorting device are calibrated, calibrated, It is configured to perform adjustments, etc. This figure shows a screen for adjusting the focus in order to optimally set the focus of the photographed image. R (red), G (green), and B (blue) values are acquired from the standard machine as the standard "focus value (reference value)", and the current R (red), G (green), and B (blue) values are acquired. ), the camera is adjusted so that the difference (focus value (Δ)) from the “focus value (current value)” is within the range of the reference value. If the current value is within the range of the reference value due to camera adjustment, "OK" is displayed as the adjustment result.

The setting of the pod sorting system and the pod sorting device can be performed in the calibration of the brightness of the captured image, the calibration of the tilt of the captured image, and the calibration of the dimension of the captured image, in addition to the focus calibration.

Calibration using such a standard machine can greatly simplify the setting work at the time of setting the pod sorting system and the pod sorting apparatus.

10 Hopper part 20 First conveying means 21, 31 Photographing means 22, 32 Conveyance detecting means 23, 33 Ejecting means 30 Second conveying means 31 Photographing means 40 Edamame sorting system W Edamame

Claims (5)

A pod sorting system for inspecting or sorting pods by analyzing an image of a pod with a pod attached,
An image acquiring means for acquiring an image captured by the imaging means, and an image analyzing means for analyzing the image acquired by the image acquiring means,
The image analysis means is
A pod image extracting unit for extracting an image of each pod in the acquired image;
A pod sorting system, comprising: a erosion processing execution unit that performs erosion processing for reducing contour regions of extracted individual pod images.
Furthermore, the image analysis means
A pod image extracting unit for extracting an image of each pod in the acquired image;
Equipped with a pod length calculation unit that calculates the length of the pod from the extracted individual pod images,
The pod selection system according to claim 1, wherein the pod length calculation unit calculates the length of the pod in an area where the pod width is a predetermined value, which is the width of the pod calculated from the extracted individual pod images. .
3. The pod sorting system according to claim 1, further comprising a log information holding unit that holds the analysis results of said image analysis means as log information.
4. Any one of claims 1 to 3, wherein the image analysis means calculates a movement position for the image acquired by the image acquisition means, and excludes an image obtained by photographing a pod with pods existing at the movement position from the analysis target. The pod sorting system according to item 1.
Further, the image analysis means is
A pod image extracting unit for extracting an image of each pod in the acquired image;
According to any one of claims 1 to 4, wherein the image of each extracted capsule is divided into a plurality of equal parts, and the area ratio between the base end and the tip in the length direction is calculated to determine the presence or absence of chipping in the capsule. The described pod sorting system.

Images