ES2505330B2 - Procedure for the automatic estimation of the components of the production of a vine cluster by artificial vision - Google Patents

Abstract

Procedure for the automatic estimation of the components of the production of a vine cluster by artificial vision (1), which comprises the following stages: #a) Capture an original RGB image (100) of a vine cluster (200) with a camera (301) in an acquisition hood (300); # b - e) Process an original RGB image (100) to obtain an image with definitive circles (106) superimposed on the original RGB image (100); # f) Perform a numerical model of the number of berries (201) per cluster (200) based on the number of definitive circles in the image to estimate the number of berries (201) per cluster (200); # g) Perform a numerical model of the weight of the cluster (200) based on the number of final circles in the image to estimate the weight of the cluster (200); # h) Calculate the average weight of the berries (201) of the cluster (200).

Description

Procedure for the automatic estimation of the components of the production of a vine cluster by artificial vision. 5 Object of the invention

The present invention relates to a method that allows the automatic production of a cluster of grapes by artificial vision to be estimated automatically and without destruction of the clusters.

The present invention is of great interest for the wine sector in general, and especially for the estimation of grape quantity and quality production,

15 General and closest prior state of the art

The number of berries (grape grains), cluster weight, and average weight of the berries, called components of the production, are key parameters for estimating vineyard yield. These parameters have an impact not only on the 2 O yield of the vineyard, but also on the cluster architecture and its compactness. Thus, compact clusters are more susceptible to fungal diseases, especially Botrytis and Oidio. On the other hand, performance components influence grape and wine quality. The loss of grape quality is caused both by the incidence of pathogens, and by the heterogeneous ripening of the berries that it has

2 5 place in compact clusters. At the oenological level, small clusters with few berries and small are preferred.

The ability to predict the performance of the vifledo has been identified in recent years as one of the most profitable issues for the wine sector. The estimation of the 30 yield is a critical factor to optimize the management of the vineyard and balance between vegetative and productive growth. Until now several methods have been used to estimate the grape yield of the vineyard, but in general they are destructive, not very robust and subjective. Typically, performance predictions are made using knowledge of the vineyard's historical yields, along with the 35 measurements made manually in the field. These field measurements in general, consist of the cluster count by strain and the weight of the grapes in a few vineyard strains. According to Blom and Tarara (2009), the precision in the grape yield estimation can vary up to 20% or more in some cases. Ferrer et al., (2004) 12) attempted to estimate the yield in several vineyards in Uruguay based on the

4 O dissection of yolks, but their accuracy was very poor. All these current methods are destructive, and require a lot of time and work. In addition, in general they do not perform a representative and sufficient number of measures to be able to obtain an accurate and robust performance estimate.

The traditional procedure for estimating the components of grape production of vii'ieclo is done by weighing the clusters and counting berries in the laboratory. This requires that the bunches have to be manually shelled for later, weigh the berries or place them on a tray and be photographed in conditions

5 controlled light for subsequent image processing. Therefore, this method is manual and destructive. requiring the shelling of the cluster to be sampled. In addition, due to their industriousness, these manual methods require a high workforce and execution time, so the sampling is usually small, insufficient and generally not representative.

10 Other systems allow the field estimation of the number of berries by image analysis, but require the taking of images at night (total darkness), to avoid the influence of sunlight; They also do not estimate the weight of the ration or the average weight of the berries.

15 Image analysis is widely used in the agri-food industry for the classification of fruit. This technology allows the creation of systems capable of estimating or predicting some of the characteristics of the inspected objects without the need for contact, in a fast, reliable and precise way. Recently, Herzog

20 et al. (2014) 131 have shown very interesting results on the use of image analysis for vine phenotyping. Computer vision has been used in viticulture to assess the vegetative and productive state of vii'iedo. Recently, (Diago et al., 2014) 141 developed a new algorithm to evaluate the number of flowers per inflorescence using image analysis in field conditions.

25 Wycislo et al. (2008) 151 have used different parameters or ratios to estimate the shape of table grape berries such as the major axis / minor axis ratio, shape and the value of compactness. Recently, a new method for the detection of the grape cluster peduncle has been developed using image analysis (Cubero et al., 2014) 16].

30 To carry out the detection of the berries in a cluster, the following two steps must be performed: The extraction of the contours, and the detection of the circulars in the image, since this is the supposed form of a berry. Several contour extraction methods have been developed but the most widely used are those based on 35 SobeJ and Canny operators. Both are based on the gradient of intensity in the images; Canny is more advanced, since it includes Sobel's operator as an intentional step. The detection of circulars in the images is another key problem that researchers have tried to solve from a number of different approaches. Hough's transfonnada is probably the most widely spread, but with a

40 very high calculation cost. Grossetéte et al. (2012) 171 used the flash light reflection of a digital camera to count the number of berries per green cluster. This approach cannot be applied after the envero (beginning of ripening of the grape) because the pruina affects the reflection of the light. There are no methods available for assessing berry weight or grape cluster weight using the analysis of

45 image.

The technical advantage of the present invention is that of a non-destructive procedure, which automatically determines or estimates the components of the production of a cluster by artificial vision: number of berries, weight of the cluster, and average weight of the berries .

Bibliographic references

111 Blom PE and Tarara JM, Trellis tensicn monitoring improves yield estimation in vineyards. HortScience 44: 678-685 (2009).

121 Ferrer M, Abella J, Sibille 1, Camussi G and Gonzalez-Neves G, Detennination of bud fertility as a simple method foc the arresnination of harvesting volume in Vitis vinifera L. CV tannat, using two pnming syslems. J Int Sci Vigne Vio 38: 49-53 (2004).

15 13) Herzog K, Roscher R, Wieland M, Kicherer A, Labe T, Forstner W, Kuhlmann H and Topfer R, Initial steps for high-throughput phenotyping in vineyards. Vitis 53: 1-8 (2014).

141 Diago MP, Sanz-Garcia A, Millan B, Blasco J and Tardaguila J, Assessment of 20 flower number per inflorescence in grapevine by image analysis under field conditioos. J Sci Food Agric DOI: 10.1 002 / jsfa.6512 (2014).

151 Wycislo AP, Clark JR and Karcher DE, Fruit shape analysis of vitis using digital photography. HortScience 43: 677-680 (2008). 25

161 Cubero S, Oíago MP, Blasco J, TardaguilaJ, Millán B and Aleixos N, A new method for pediceVpeduncle detection and size assessment of grapevine berries and other fruils by image analysis. Biosyst Eng 117: 62-72 (20 14).

30 I7J Grossetete M, Berthoumieu Y, Da Costa JP, Gennain C, Lavialle O and Grenier G, Early estimation of vineyard yield: site specific counting of berries by using a smartphone. CIGR-AgEng2012 (2012).

Detailed description of the invention

The slow procedure for the automatic estimation of the components of the production of a vine cluster by artificial vision according to the present invention comprises the following steps or steps:

40 Stage "a". Capture an original RGS image (lOO) of a cluster (200) with a camera (30)) in an acquisition hood (300).

A plurality of images are captured, three being taken at 120 '\ the minimum number that guarantees the sweeping of the entire convex envelope of a cluster (200) in three finite distance dimensions. 4

A preferred model and focal length of the camera (30 1) is as follows: a Canon EOS 5500 camera with Canon EFS 18-55 lens (Canon Inc., Japan) with the focaJ distance set at 55 mm.

5 A preferred setting of camera capture parameters (301) is: shutter speed 200 ms, ISO 800 sensitivity, manual focus and white balance 'Shadow', and a resolution of the original images (100) of 0 , 38 mm / pixel.

10 A preferred configuration of the acquisition bell (300) is as follows: interior covered with diffuser fabric, Iwninaria (302) oriented at 450 with respect to the cluster (200) and located at a distance of 30 cm from the cluster (200) to photograph The lighting system, that is to say the luminaires (302), is composed of four lamps located at the ends of the acquisition bell (300) composed of

15 two fluorescent tubes (BioJux LI8W / 965, 6500 K, Osram AG, Germany) powered by high frequency reactances to prevent flickering (this solution does offer improvements against LEO, by CRJ, or against incandescent by the minor heat emission). This configuration achieves spatially more uniform illumination. To avoid unwanted brightness, polarizing filters (303) are placed between the luminaires

20 (302) And the cluster (200) and between the cluster (200) and the chamber (30 1).

and with those that correspond to the fund (Ion.

25 In the case of clusters (200) of grape grapes produces an optimal unexpected effect when choosing an orange background (01) of RGB components:

R: 255 30 G: 175 ± 15

B: 85 ± 15

In the case of grape varieties of white varieties, it produces an optimal unexpected effect when choosing a cyan color background (101) of RGB components: 35

R: 95 ± 15

G: 130 ± 15

B: 180 ± 15

40 Stage "e". Detect the edges using the Canny method in the binary image with a segmented cluster (102) for the red and green channels. in parallel to later add the results to obtain a binary image of edges (104).

The binary edge image (1 04), the result of this procedure, is a binary map of patterns in which a limited amount represents the contour of the berries.

Stage "d". Detect in the binary edge image (104) the circular patterns 5 to obtain a list of candidate circles to represent berries in the image to obtain a binary image with candidate circle centers (105).

A limited amount of these patterns will correspond to beech contours present in the image and others not; In the next step, the 10 wrong profiles will be filtered.

The Hough Circular Transform is preferably used to detect circular patterns (Radial Symmetry Transformation is also applicable, for example). The Hough transform allows you to find geometric patterns in

15 discrete domains such as digital images. Hough Circular Transfonnada is the concrete implementation of such transformed to find circles. The implementation of the transform used requires the establishment of the value of two parameters:

2 O Radio (R): radius of the circles to look for in the image. Perimeter Coincidence (CP): this is a value normalized to the interval [O, 1] that is calculated as follows:

CP ~ ~ 1I2'pi'R

25 where INI is the number of pixels belonging to the perimeter of the circle of radius R under evaluation, which coincide spatially with pixels in the image under analysis.

30 In order to facilitate the subsequent filtering of erroneous berry candidates, their detection and storage is carried out in a list as follows:

Candidates are detected starting from CP = 1 to CP = 0.4 (decrements of 0. 1). Candidates with lower CP values are not calculated because they do not offer the slightest guarantee of representing a berry contour.

For each CP value, candidates with radius from R = 15 (mm) to 3 (mm) are searched. Candidates with R values outside this range are not 4 OR calculated since they do not have a representative size of being berry.

In this way, a list of circles has been obtained in which these are ordered in decreasing order of CP value and, for equal CP values, in descending order of radius.

Stage "e". Filter in the binary image with candidate circle centers (JOS) the candidate berries (201). to obtain an image with definitive circles (106) superimposed on the original RGB image (lOo>.

5 As a result of circle detection, a large number of candidates are obtained. It is characteristic that for each berry contour in the image many candidate circles have been generated with centers very close to each other and different radii. In this step, the best "representatives" of the berry contours are selected eliminating their neighbors. To do this, the following procedure is performed:

From CP = I to CP = 0.4: for each CP value, candidates with a larger radius are selected and those with a smaller radius are eliminated.

In this way, the selected candidates are those who comply with the highest

15 spatial correspondence with berry profiles in the image. At the same time, candidates with a higher radius are also prioritized, since being constituted by a greater number of pixels, the probability that they are not representing a berry contour and if any other pattern, for example noisy, decreases.

20 Stage "r ', Perform a numerical model of the number of berries (201) Dor cluster

(200) based on the number of definitive circles in the image Dara estimate the number of berries (201) per cluster (200).

A numerical model is made to estimate the number of berries per cluster based on

25 of the number of final circles in the image. The linear correlation between the number of berries per cluster, obtained by destructive method in the laboratory, and the number of berries per cluster estimated in the image, obtaining y = Ax + B, and its R2 is determined; where A and B are the coefficients of the linear regression line. By way of non-limiting example of the invention, in FigA, it is shown for Bodal varieties and

30 Mazuelo, a regression graph with y = O.974 x-24.3, R] = O, 885.

Stage "g". Perform a numerical model of cluster weight (200) based on the number of definitive circles in the image. It will estimate cluster weight (200).

35 A numerical model is made to estimate! cluster weight (g) based on the number of final circles in the image. The linear correlation between cluster weight (g), obtained by laboratory weighing, and e! number of beech trees per cluster estimated in the image, obtaining y = C-x + D, Y its R2; where e and D are the coefficients of the linear regression line. By way of non-limiting example of the invention, in the

4 O fig. 5, for Bodal and Mazuelo varieties, a regression graph with y = 1,7178x + 77,181, R '= O, 8854, is shown

"H" stage. Calculate the average weight of the berries (201) of the cluster (200).

4 5 The average weight of the berries is calculated as follows: 7

Average weight of the berries (g) = Cluster weight (g) / No. of berries per cluster Brief description of the figures

Glossary of references

W

Procedure for the automatic estimation of the components of the

cluster production through artificial vision;

10

(100) Original RGB image;

(101)

Background;

(102)

Binary image with segmented cluster;

(104)

Binary Edge Image;

(105)

Binary image with centers of candidate circles;

fifteen

(106) Binary image with definitive circles;

(200)

Cluster;

(201)

Berry;

(300)

Acquisition bell;

(301)

Camera;

twenty

(302) Luminary;

(303)

Polarizing filter;

Figure I (Fig.l) .- shows a configuration of an acquisition bell (300) to capture an original RGB image (100) of a cluster (200), under laboratory conditions with controlled lighting; The cluster (200) is preferably of a vine, both red and white varieties.

Figure 2 (Fig. 2) .- shows a block diagram with the movement flows according to the state of the art and according to the present invention. The state of the

The current technique is reflected by dashed lines, while the flows according to the present invention are shown by solid lines.

Figure 3 (Fig.3) .- shows a set of images of the segmentation process of 35 berries in digital images:

Fig.3A.-shows an original RGB image (1 (0); Fig.3B.-shows a binary image of edges (104);

4 O Fig.3C.-shows a binary image with candidate circle centers (105); Fig.3D.- shows a binary image with definitive circles (106); Fig.3E.-shows a binary image with definitive circles (106),

represented on an original RGB image (100); Figure 4 (Fig. 4) .- shows a graph, which is shown by way of example, where different values of no. Of berries per cluster are measured against their corresponding values of no. Of estimated berries / cluster, as well as the linear correlation between these values.

Figure 5 (Fig.S) .- shows a graph, shown by way of example, showing different values of cluster weight (g) measured against their corresponding estimated number of estimated bayas, as well as linear correlation between these values.

Claims (4)

1. Procedure for the automatic estimation of the components of the production of a vine cluster by artificial vision (1), characterized 5 because it comprises the following stages: a) Capture an original RGB image (100) of a vine cluster (200) with a camera (30 1) in an acquisition hood (300); 10 b) Segment the original RGB image (100) to obtain a binary image with a segmented cluster (102) with the pixels that confront the cluster (200) and those that correspond to the background (1 01); e) Detect in the binary image with segmented cluster (102) the 15 edges by the Canny method, for the red and green channels, of parallel shape for later adding the results to obtain a binary image of edges (I04); d) Detect circular patterns in the binary edge image (104) 20 to obtain a list of candidate circles to represent berries in the image to obtain a binary image with centers of candidate circles (105); e) Filter in the binary image with candidate circle centers (105) 25 the candidate berries (201) to obtain an image with definitive circles (106) superimposed on the original RGB image (100); f) Make a numerical model of the number of berries (201) per cluster (200) based on the number of definitive circles in the image to estimate the number of berries (201) per cluster (200); g) Perform a numerical model of cluster weight (200) based on of the number of definitive circles in the image to estimate the weight of the cluster (200); 35 h) Calculate the average weight of the berries (201) of the cluster (200). 2. Method according to claim 1, characterized in that the cluster (200) is of a vine, both of red and white varieties. 3. Method according to any of the preceding claims, characterized in that the capture of the original RGB image (100) of the cluster (200) is carried out in an acquisition hood (300) in laboratory conditions with controlled lighting.

Four.

Process according anyone from the claims previous.

characterized because

the background color (l 01) for capturing an image

Original RGB (100), in the case of:

5

-uva ink is orange with the following RGB components;

R: 255;

G: 175 ± 15;

B: 85 ± 15.

1 0

- White grape is cyan with the following RGB components:

R: 95 ± 15;

G: 130 ± 15;

B: 180 ± 15.

fifteen