JPH0456933B2 - - Google Patents

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to an apparatus for automatically determining the ripeness of fruit vegetables such as mandarin oranges, apples, grapes, and kiwifruit, and more particularly, to an apparatus for automatically determining the ripeness of fruit vegetables such as mandarin oranges, apples, grapes, and kiwis. The present invention relates to a ripeness determination device for fruits and vegetables that determines the ripeness of fruits and vegetables to be measured based on the signal level of a green component of a color image signal.

[Conventional technology]

It is known that in fruit vegetables such as mandarin oranges, apples, grapes, and kiwifruit, the green component of the skin color changes markedly depending on the degree of ripeness.
In other words, it is known that the green component level is inversely proportional to the ripeness, such that the lower the green component, the higher the ripeness. Therefore,
By keeping the imaging conditions constant, that is, by determining the magnitude of the image signal level of the green component normalized by brightness, it is possible to quantitatively determine the ripeness of fruits and vegetables.

Therefore, in the past, fruits and vegetables that are the target for determining ripeness are imaged using an imaging means such as a color video camera, and in order to keep the imaging conditions constant,
One idea is to normalize the green component of the captured image signal with brightness, such as a luminance signal, create a histogram of the level changes, and measure, for example, the median value of the created histogram as the degree of coloration that represents the degree of ripeness. It is being (Refer to Japanese Unexamined Patent Publication No. 59-87081.) However, with the above means, it is necessary to create an accurate histogram for each measurement object, and to do so, it is necessary to create a green component signal measured at multiple points for one object. It is necessary to store the image data at a relatively high resolution corresponding to each measurement point, which increases the capacity of the image data memory, etc. Moreover, processing for creating a histogram is also required, which poses the problem of complicating the device configuration.

Therefore, in order to simplify the device configuration, a method has been devised in which the analog image signal obtained by imaging the object is directly binarized based on its color range, and only this binarized data is stored and processed. There is.

[Problems to be Solved by the Invention] However, the above conventional means has the following disadvantages.

In other words, in a method for pre-binarizing image data (usually an analog signal), in order to identify the object to be measured, all imaged signals are binarized using a threshold value corresponding to a preset color range. , At that time, in order to prevent malfunction due to noise etc., 2.
It is necessary to perform processing to automatically remove values that have a small area. Therefore, in the case of measurement objects such as grapes that are irregular in size and have subtle color changes due to differences in ripeness, when binarized, the area difference becomes smaller and the grapes Due to the difference in size of the bunches themselves, image data corresponding to small bunches may be missing, making it impossible to accurately determine the degree of ripeness.

The present invention has been made in view of the above circumstances, and its purpose is to easily determine the overall ripeness level of fruits and vegetables to be measured, without being influenced by the size or shape of the fruit or vegetables. There is a particular thing.

[Means for solving problems]

The characteristics of the ripeness determination device for fruits and vegetables according to the present invention are as follows:
A binarization means for binarizing the image signal of the green component and a means for variably setting the threshold of the binarization means are provided, and the two before and after the threshold is variably set.
The present invention is provided with a means for determining the ripeness of the fruits and vegetables to be measured based on the area change rate of the valued image, and its functions and effects are as follows.

[Effect]

That is, the green component of the image component of the fruit and vegetable to be measured is determined by a predetermined threshold, such as the upper and lower limits of the specified ripeness, or the green component range for specifying the fruit and vegetable to be measured. The image is binarized based on a predetermined threshold, the size or area of the outline of the binarized image is determined, and the threshold is variably set, for example, to the side where the green component decreases, which is the direction of higher ripeness. When the same object is imaged, the smaller the area reduction rate of the binarized image before and after the variable setting of this threshold value is, the more mature the image is.
In this way, you can determine its ripeness.
Furthermore, in order to determine the degree of ripeness, the threshold value may be variably set to the side where the green component increases, and a binarized image with a small area increase rate may be determined to have a high degree of ripeness. The degree of ripeness can be determined based on the area change rate of the binarized image before and after the threshold value is variably set.

〔Effect of the invention〕

Therefore, the green component, which is the color component corresponding to the ripeness of the image signal of the measurement target, is binarized using a predetermined threshold, and the area change rate of the binarized image before and after the threshold is variably set. Since the degree of ripeness is determined based on this, it is possible to accurately determine the overall degree of ripeness of the target fruits and vegetables, regardless of the size or shape of the target, while simplifying the device configuration.

〔Example〕

Embodiments of the present invention will be described below based on the drawings.

As shown in FIG. 6, a pair of left and right propulsion wheels 2L, 2R can be independently driven by a hydraulic motor m, and a caster 2 located behind the wheels 2L, 2R.
A manipulator B support post 1 is erected at the rear of the vehicle body V, which is equipped with a driving part E, and a telescopic manipulator B is attached to the tip of the support post 1.
The manipulator B is attached so as to be swingable up and down and left and right, and a working hand H serving as a working part is attached to the tip of the manipulator B, thereby forming a working machine for terraced fruit vegetables such as grapes.

The manipulator B includes a proximal first arm B ₁ ,
It is composed of three arms: a second arm B ₂ which is telescopically fitted and supported in the first arm B ₁ , and a _{third arm B 3 which} is telescopically fitted and supported in the second arm B 2 . , the proximal first arm B ₁
The entire body is configured to expand and contract by forward and reverse rotation of an expansion/contraction operation motor _M1 provided at the rear end.

Further, the first arm on the base end side of the manipulator B
_B1 is pivotally supported on a bifurcated connecting frame 4 so as to be vertically swingable around the horizontal axis X, and the connecting frame 4 is fitted inside the support post 1 so as to be rotatable around the vertical axis Y. And a motor that operates the manipulator B to swing up and down.
A motor M ₂ and a motor M ₃ for left-right swing operation are provided.

Various types of working hands H are prepared for carrying out various operations such as harvesting, pruning, flower thinning, and spraying chemicals on fruit trees, and these hands are changed and used depending on the task. To explain the hand H for harvesting work as an example, as shown in _FIG . The cylindrical case 6 for taking in fruit is rotatably attached to the tip of the bifurcated support frame 5 about the horizontal axis Q, and the cylindrical case 6 is held in a predetermined upward position by its own weight. It is designed to be maintained. A plurality of pedestal pressing members 7 press and support the peduncle, which is a handle-like portion that connects the fruit to be worked on and the branch introduced into the cylindrical case 6, in a predetermined position as the peduncle is projected. is attached to the opening of the cylindrical case 6 so as to be retractable and retractable, and an arc-shaped sliding cutter 8 for cutting the fruit stalk supported by the stalk pressing member 7 is pressed and supported at the predetermined position. so that it can be operated to protrude into the cutting position against the fruit stem.
It is swingably attached to the cylindrical case 6 side.

A plurality of finger-like members 9 are provided upright on the periphery of the opening of the cylindrical case 6. The tip of the finger-like member 9 is also configured to serve as a contact sensor, so that it can collect information for adjusting the guiding direction of the hand H when the fruit F to be harvested is taken into the cylindrical case 6. be.

In addition, a color video camera 10 is installed at the center of the front part of the vehicle body V so that the viewing direction thereof faces upward.
A proximity sensor 11 using an infrared spot distance sensor is provided at the top of the tip of _B3 to detect when the hand H approaches the target fruit F within a set distance. As shown in FIG. 1, by binarizing the image information obtained by the camera 10 based on a predetermined color range set in advance to identify the target fruit F, the fruit F and other objects such as branches and leaves can be detected. A binarizing means 12 is provided for separating the contour Fa from the fruit to extract an outline Fa corresponding only to the fruit to be worked on, and a monitor 13 for displaying the binarized image. Then, the control device 14 is configured to perform the following operations. In other words, the central position Fc of the extracted contour Fa on the screen of the monitor 13 is determined, and the direction of the position that is a predetermined amount below the height of the shelf on which the fruit F is grown is determined based on this central position Fc. ,
Calculating based on the height of the shelf, the position of the camera 10, the position of the manipulator B, etc., protruding the manipulator B in the calculated direction, and directing the protruding operation to the target fruit F of the proximity sensor 11. The harvesting hand H is configured to be stopped in response to the detection operation, and then automatically moved to a position directly below the center of a lump of one target fruit F.

In order to prevent the imaging information obtained by the camera 10 from being affected by surrounding brightness fluctuations, a strobe device 15 is used as a means for illuminating the target fruit F at a predetermined brightness in synchronization with the imaging operation. A reflector 16 is provided so that its illumination direction is directed toward the vehicle body V, which is the opposite direction from the fruit F to be worked, and which diffuses and diffuses the strobe light upward. In other words, if a fruit with white powder attached to its surface, such as a grape, is directly illuminated, so-called halation will occur, and the skin color will be imaged as whitish, which is different from the original skin color. This results in an inconvenient image in which the target fruit F is incorrectly recognized, but such an inconvenience can be prevented. Therefore, even for fruits and vegetables whose skin color is unclear, it is possible to obtain image information of a color similar to the color sensation obtained by human visual observation.

Hereinafter, the configuration of the control system that determines the position and ripeness of the fruit F to be worked on and guides the hand H in the direction of the fruit F to be worked on will be described as follows.
, the block diagram shown in FIG. 2, the explanatory diagram of threshold setting of the binarization level shown in FIG.
The details will be explained based on the flowchart shown in the figure.

That is, the image taken by the camera 10 is
As shown in Figure 1, the image signal output as an NTSC signal is processed by the NTSC decoder 17 into a luminance signal Y corresponding to brightness and a change in skin color among the three primary colors of light, red, green, and blue. The signal is separated into three signals Y, R, and G, a red component R and a green component G, whose output levels change accordingly, and are input to the binarization means 12.

The binarization means 12 normalizes each of the red component R and the green component G using the luminance signal Y, and then binarizes the color range corresponding to the target fruit F as a threshold value. The image information in which both the red component R and the green component G are at the “H” level is the target fruit F.
The configuration is such that the image is processed as a binary image.

The red component R and green component G are converted into a luminance signal Y.
To explain the principle of normalization, the skin color range of the target fruit F corresponds to the signal levels of the red component R and the green component G under constant brightness conditions. Therefore, the ratios R/Y and G/Y of each level of the red component R and the level of the luminance signal Y to the level of the green component G are directly equal to the absolute skin color range in which the influence of brightness fluctuations has been removed. There is a proportional relationship.

By the way, to find the ratios R/Y and G/Y,
If the signals of the red component R and the green component G are directly divided by the luminance signal Y, the signal frequency band is high, and it is difficult to directly divide the signal. Therefore, in this embodiment, multiplication processing is used. There is.

In other words, the color range to be binarized is the component of a certain color.
V _N can be defined as shown in the following equation (). By transforming this equation () as shown in equation (), it can be replaced with an equation that multiplies the luminance signal Y by a predetermined ratio.

Kn ₁ <V _N /Y<Kn ₂ ... () (However, N is the R or G level, Kn ₁ is the lower limit of R or G, and similarly Kn ₂ is the upper limit.) The above formula () When transformed, Kn ₁・Y<V _N <Kn ₂・Y ... () (However, since Y is a composite signal of R, G, and B, V _N, that is, the level of R or G is lower than the level of Y. Therefore, Kn ₁ ≦1, Kn ₂ ≦1.) Therefore, the lower limit of the color range to be binarized into the luminance signal Y is
By extracting the red component R and the green component G within the signal level range multiplied by Kn ₁ and the upper limit Kn ₂ , a binarized signal that is an image corresponding only to the skin color of the target fruit F is obtained. It will be.

Therefore, the color range to be binarized is determined by the luminance signal Y
By setting the lower limit value Kn ₁ and upper limit value Kn ₂ of the multiplication ratio of , and comparing the level of the brightness signal Y multiplied by the respective multiplication ratios with the respective levels of the red component R and the green component G, the image is captured. It is possible to extract objects within a specific color range from the image information obtained without being affected by changes in surrounding brightness. And the lower limit value Kn ₁ and upper limit value of the color range
After setting Kn ₂ and binarizing, the lower limit value Kn ₁ or upper limit Kn ₂ of the color range is again set variably to binarize the image signal obtained by imaging the same object again, and the image signal before and after the variable setting is By determining the area change rate of the binarized image, it is possible to determine the change in color of the same object, that is, the degree of ripeness.

That is, a pair of potentiometers Rr 1 and Rr ₂ are used to set the respective multiplication ratios Kr ₁ and Kr ₂ as thresholds of the lower binary level corresponding to the lower limit value Vr ₁ and upper limit value Vr ₂ of _the range of the red component R. The multiplication ratio Kr ₁ , Kr ₂ set by this potentiometer Rr ₁ , Rr ₂
A pair of multipliers A ₁ and A ₁ are provided for multiplying the luminance signal Y, respectively. Similarly, a pair of potentiometers Rg 1 and Rg ₂ are provided to set multiplication ratios Kg ₁ and Kg ₂ corresponding to the lower limit value Vg _{1 and} upper limit value Vg ₂ of the range of the green component G, and these potentiometers Rg ₁ ,
_a pair of multipliers A 1 , which respectively multiply the luminance signal Y by multiplication ratios Kg ₁ and Kg ₂ set by Rg ₂ ;
A ₁ is provided. and each pair of multipliers
A pair of comparators A ₂ and A ₂ serve as means for binarizing the luminance signal Knm·Y, n=r, g, m= 1, ₂ , which is multiplied by A ₁ and A ₁ by a predetermined ratio Knm. Each comparison reference voltage of the window comparator is used as a reference voltage, and each window comparator determines the red color within the range of the lower limit value Vr ₁ and upper limit value Vr ₂ of red color set by each potentiometer Rr ₁ , Rr ₂ , Rg 1 , Rg _{2 .} The component R and the green component G in the range of green lower limit value Vg ₁ and upper limit value Vg ₂ are respectively binarized and extracted, and each binarized red component R and green component G are both " Only signals that are at H” level
Extract the target fruit F using AND gate _G1 .
It is converted into a binary image signal corresponding to the contour Fa.

The binarized image signal corresponding to the binarized target fruit F is displayed on the monitor 13 and is temporarily stored in a pair of image memories 18a and 18b. Then, by variably setting the upper limit value Vg ₂ of the green component G, the binarization range of the green component G is changed in accordance with the ripeness level, and the binarization range stored in both image memories 18a and 18b is changed. The degree of ripeness of the target fruit F is determined based on the area change rate a of each contour Fa, Fb before and after the threshold value is variably set.

That is, as shown in FIG. 4A, the image signal Fa containing both the red component R and the green component G set by the four potentiometers Rr ₁ , Rr ₂ , Rg ₁ , and Rg ₂ is separated. By extracting, the fruit F to be measured and its background branches, leaves, sky, etc. are separated and stored in the one image memory 18a. Next, as shown in FIG. 4B, the green component G
The upper limit threshold Vg ₂ is lowered to a predetermined level corresponding to the ripeness of the target fruit F, the image is captured again at the same position, the binarized image Fb is stored in the other image memory 18b, and the control device The area of each of the contours Fa and Fb is calculated by the ripeness determination means 14a provided in the 14, and the rate of change a thereof is determined. and,
It is determined whether the fruit is ripe or not based on whether the calculated area change rate a is less than or equal to a predetermined value k. Incidentally, if the area change rate a is not less than a predetermined value k, the hand H may be prevented from performing the harvesting operation, and this may be displayed on the monitor 13.

Further, when the binarized images are stored in both the image memories 18a and 18b, the operation of the strobe device 15 is controlled by the control device 14 so that it operates in synchronization with the imaging by the camera 10. Become.

Also, stored in the one image memory 18a,
By indicating the position of the outline Fa of the binarized image of the fruit F to be harvested displayed on the monitor 13 with a light pen or the like, the fruit F to be harvested is designated, and the center of the outline Fa is determined based on the instruction information. The position Fc is calculated, and the target direction calculating means 14b calculates a position below the position Fc by offsetting a set amount, that is, the fruit sensing position of the proximity sensor 10, as the guidance target direction of the hand H. demand.

Then, the drive signal calculating means 14c is activated so that the hand H protrudes toward the target direction, and the direction of the manipulator B is changed and adjusted by driving each of the motors M ₂ and M ₃ . By operating the motor _M1 in forward rotation until the sensor 10 detects the fruit F, the fruit F to be harvested is
This guides the hand H to be automatically moved to just below the lowest position. After that, by moving the hand H upward without changing its direction, the fruit F is taken into the hand H,
The stalk pressing member 7 is operated to protrude to press and support the fruit stem at a predetermined position, and the cutter 8 is operated to protrude to a cutting position with respect to the fruit stem that is pressed and supported at the predetermined position to cut and perform a harvesting operation, The motor _M1 is operated in reverse rotation to retract the hand H, thereby completing the harvesting operation for one fruit F whose ripeness is above a predetermined level.

[Another example]

In the above embodiment, not only the green component G but also the red component R is used to binarize the captured image.
This is for the reasons shown below.

In other words, since the ripeness of the object is measured while it is still growing on the fruit tree, the captured image contains not only the target fruit F, but also unnecessary data such as branches, leaves, and the sky in the background. will be included. Therefore, unless this unnecessary data is removed in advance, it is not possible to extract image data that corresponds only to the object F to be measured, but it is not possible to extract the image data that corresponds only to the object F to be measured. Due to the fact that it cannot be specified.

Therefore, when the present invention is applied to a sorting device or the like that sorts fruits according to their ripeness, as will be described later, the target fruits and vegetables whose ripeness is to be determined are identified in advance, and the captured images are Since the degree of ripeness can be determined in a state where objects other than the object to be measured are not included, it is sufficient to use only the green component G and change its threshold value.

Also, the upper limit threshold of the green component G that is variably set.
In order to set Vg ₂ variably, the configuration may be changed so that the potentiometer Rg ₂ is manually operated, and the amount of change in the setting is automatically changed by the control device 14. Furthermore, not only the upper limit threshold Vg ₂ but also the lower limit threshold Vg ₁ may be variably set. Furthermore, the threshold value Vg ₁ as the binarization standard,
Vg ₂ may be variably set in a plurality of stages to simultaneously determine in which range of ripeness the fruits and vegetables to be measured fall.

According to the present invention, the area change rate of the binarized image before and after the setting change is determined separately while setting the threshold value in multiple stages of three or more stages.
It can also be used to determine the ripeness level based on the required area change rate. In other words, it can also be used to sort fruits and vegetables to be measured according to their ripeness.

[Brief explanation of the drawing]

The drawings show an embodiment of the working machine for fruits and vegetables according to the present invention. FIG. 1 is a block diagram showing the overall configuration of the control system, FIG. 2 is a block diagram showing the configuration of the binarization means, and FIG. 3 is a binary diagram. An explanatory diagram of the threshold setting of the conversion level,
Figures 4A and 4B are explanatory diagrams of the binarized image, Figure 5 is a flowchart showing the operation of the control device, Figure 6 is an overall side view of the working machine, and Figure 7 is a plan view showing the schematic configuration of the hand. It is a diagram. 12... Binarization means, F... Fruits and vegetables to be measured,
G...Green component, _Vg2 ...Threshold value, _Rg2 ...Threshold value setting means, Fa...Binarized image, a...Area change rate.

Claims (1)

[Claims] 1. A ripeness determination device for fruits and vegetables that determines the ripeness of fruits and vegetables to be measured based on the signal level of a green component G of a color image signal obtained by capturing an image of fruits and vegetables to be measured, A binarization means 12 for binarizing the image signal of the green component G, and a threshold setting means Rg ₂ for variably setting the threshold Vg ₂ of the binarization means 12 are provided, and the threshold Vg ₂ is variably set. Based on the area change rate a of the front and rear binarized images Fa,
A ripeness determination device for fruits and vegetables, comprising means for determining the ripeness of the fruits and vegetables F to be measured.