WO2007088225A1

WO2007088225A1 - Computer vision system for picking small row-cultivated fruits

Info

Publication number: WO2007088225A1
Application number: PCT/ES2007/000035
Authority: WO
Inventors: José Ramón Casar Corredera; Paula Tarrio Alonso; Ana María BERNARDOS BARBOLLA; Juan Alberto Besada Portas; Javier Ignacio PORTILLO GARCÍA; José María Duran Altisent
Original assignee: Universidad Politécnica de Madrid
Priority date: 2006-01-31
Filing date: 2007-01-25
Publication date: 2007-08-09
Also published as: ES2253135B2; ES2253135A1

Abstract

The invention relates to a computer vision system for picking small row-cultivated fruits. The invention includes a vision system for precisely guiding a robotic device (6) used to grip, cut and deposit the fruit in the desired location. The system accurately designates the exact 3-D point to which the terminal mechanism (7) is to be directed in order to sieze the peduncle of a ripe fruit and to cut said peduncle from above at the predetermined distance, differentiating same from the peduncles of other adjacent fruits that are not to be cut. The basic vision system comrpises: two colour cameras (1); an array of low-cost spot-type optical laser diodes (2), mounted to an azimuth- and elevation-adjustable platform (3) for orienting the array at the desired angle; and an additional laser diode (4) capable of projecting a flat or fan-shaped optical beam.

Description

Title

Artificial vision system to collect small fruits grown in rows.

Technical sector

The invention is framed in the sector of automated systems for use in agriculture, and specifically in that of robotic systems, based on artificial vision, for tasks of handling and harvesting in horticulture.

State of the art

The interest of the agricultural production sector is known for having automated, semi-automated or robotic systems, for harvesting fruits for different types of cultivation, including tree fruits, fruits grown in soil or fruits grown high, as in hydroponic crops. This invention relates to the collection of small delicate fruits grown high, such as strawberries in hydroponic greenhouse crops. In these types of crops, fruit plants grow in rows, planted on a physical support, such as plastic gutters, which are arranged lengthwise, at a variable height above the greenhouse floor. The fruits grow and ripen with the peduncle hanging out from the culture support. In some facilities, the rows can be raised or lowered to different heights, individually or in groups, to facilitate access to plants in cultivation and harvesting tasks.

The general technological problem to be solved is that of the conception of an autonomous vehicle, equipped with the vision and handling systems necessary to make a careful collection of delicate fruits, with minimal human assistance. To be viable, the system must meet at least the following three requirements:

First, it must be able to collect the fruits with extreme delicacy, to ensure maximum durability and quality. In particular, you should collect without apprehending the fruit, and deposit it in the storage container without subjecting it to any process that involves knocks or friction.

Second, to ensure the efficiency and profitability of the work, the system must be able to collect all ripe fruits and not only those that are easily identifiable by being isolated. Therefore, it must be able to recognize and isolate the fruits that grow in clusters, being understood by cluster, for these purposes, the spatial concentration of several fruits. Likewise, it must be able to solve the problem of occlusions, especially those produced by the partial concealment of ripe fruits that can be collected by other immature ones.

Third, for the same reason of efficiency it must be able to collect the fruits at a reasonably high speed. In this context, the reasonably high speed should be understood to be that which is limited only by the movement capacity of the selected robotic device and not by the limitations of acquisition, operation and processing time of the fruit detection and location systems. This circumstance imposes explicit requirements on the detection and location system.

Well, there are no practical devices capable of satisfying these three requirements simultaneously to the desired degree. The main limitation is, so far, in the vision system itself, understanding as such the set of means and algorithms whose mission is to detect and locate, with the required precision, the position of the ripe fruit and, singularly, of its peduncle in cluster fruit arrangements. Solutions for fruits or vegetables that always grow in isolation (cucumber, eggplant, etc.) are unsuitable for the collection of delicate fruits that can grow together.

The vision systems proposed for harvesting applications are preferably based on the use of a camera, whose output is processed to detect by color, curvature or shape the presence of the fruit and make a first location in the plane of the 2-D image. To determine the third coordinate (depth or distance of the fruit to the vision system), the camera's 2-D information is complements with the information of the third coordinate of rank (distance), that supplies a device of measure of distance. This can be an ultrasonic emitter-receiver device or a pair of laser-type devices (emitter and receiver) operating in the infrared or visible. This is the foundation, for example of the systems proposed in patents JP8112021, JP2004180554, JP5174131, JP5168334, JP5168333. In most cases, the determination of the distance or the calculation of the distance map is done by triangulation, estimating the delay or the phase difference between the signal emitted by the sending device and the one received at the receiver. Thus, in the JP5168334 patent the radiation emitted by the emitter is deflected with a mechanically controlled mirror, which focuses the radiation towards the area to be explored, where it is reflected to be captured by the receiving device or electronic eye. Once the geometry of the arrangement is known, for each angle of deflection, a simple trigonometric triangulation allows the distance of the reflection point to be calculated. Unfortunately, these systems have a limited spatial resolution, probably sufficient for some applications but insufficient to discriminate with the required precision the position of a small fruit in a cluster and especially the 3-D position of the peduncle. Any possible resolution increase requires a slower deflection process and consequently a lower operating speed in the collection.

In these systems, the camera and / or the distance meter can be fixed on the vehicle or mounted on the end of the robotic pickup arm.

In other proposals specifically designed for the collection of medium or large sized fruits that grow in isolation (see for example JP2001095348), the vision system consists of two cameras that operate sequentially, a fixed one that analyzes the entire scene globally and locates the fruit in the 2-D plane and another, mounted on the end of the manipulator, which guides the arm as it approaches its target.

Another theoretically possible solution to solve the problem could be thought to be the long known stereoscopic vision, consisting of having two cameras with overlapping vision and slightly different perspectives of the scene. The unknown 3- D position of a given point in the scene can be determined from their 2-D positions in the images captured by the cameras (projections). These systems do not seem to have practical use at the moment in the type of collection of our interest, despite theoretically allowing a precise location in the space through a simple triangulation. This is due to the practical difficulty of identifying in the two images corresponding (associated) points, that is, two 2-D points that really correspond to the same 3-D point of the scene.

It is insisted that the critical stage of a delicate fruit collection system, such as strawberry, is the precise three-dimensional determination of the point at which the terminal cutting element of the manipulator arm is to be guided, so that it can operate with the maximum cleaning only at the designated point without touching the fruits and at the maximum possible speed. This implies the precise detection of a given point of the peduncle of each of the ripe fruits in the row collected. In the patent application P200501586 / 5, a method based on an optical line laser is proposed for the precise designation and location of a point of an isolated peduncle, once the fruit is located. In this invention, the entire problem is solved by combining the use of spot lasers to determine the position of the fruit in a cluster with that of a line laser, which is pointed at successive elevations to identify and position the desired peduncle.

The terminal element necessary for harvesting is one of the pre-cutter type, consisting of a clamp-shaped element that grabs the peduncle at the designated point and another scissor-shaped one that cuts it over the predetermined distance (as mentioned , for example, in JP2004180554). It is noted that, for this type of delicate collection, other mechanisms whose operation involves an aggression to the fruit in any of the phases (such as suction or traction) are not appropriate.

Finally, harvesting in a greenhouse where plants are grown in parallel rows arranged high requires installing the vision and collection elements on a vehicle controlled by the vision system itself, capable of moving longitudinally along the crop, of so that the vision system successively access the collection scenes. The system could move with traction independent on rails arranged on the ground or mounted on a sliding platform, which in turn allows access to the set of rows in the greenhouse.

Detailed description

The core of the invention is based on a vision system that allows the precise guidance of a robotic arm for the apprehension, cutting and subsequent transfer and delicate deposition of each small ripe fruit in the designated position of a tray or container. The problem that can be solved is that of precisely designating the exact 3-D point to which to direct the pre-cut-end terminal mechanism that will grab the peduncle of a ripe fruit and cut it over the predetermined distance. The two practical difficulties that this invention solves are: first, that of the detection and location of the fruit to be collected, when it is presented in a spatial grouping with other fruits (cluster), and, therefore, is hardly distinguishable by traditional methods based only on color or shape; and, second, that of determining an exact point of your peduncle, distinguishing it from the peduncles of other nearby fruits that should not be cut. The control and processing system can operate on a commercial PC and allows you to manage the robotic arm taking advantage of its maximum movement capacity. Three advantages over any other solution devised to date for this type of accurate collection are:

1) It is able to designate with precision of the order of 1 mm. the position of the cut-off point of the peduncle of a collectible fruit, even and especially in the case of fruits that ripen in clusters (for example, strawberries), distinguishing it from the peduncles of other fruits, ripe or not, in the same cluster. Therefore, the system has an optimum collection efficiency, to the extent that it is able to collect most of the fruits of the rows (and not only those that grow isolated).

2) On the other hand, this precision allows the use of a scissor cutting device; the fruit, therefore, can be grabbed and cut by the peduncle, and transported without rubbing it to its destination in the collection tray, which ensures a unique quality of the fruit collected because it is not touched or subjected to any aggression in the process . 3) Finally, the operating speed of the vision and processing system is such that the robotic arm has no dead timeout, which means the maximum collection speed (which is not possible with other vision systems).

The basic vision system (Fig. 1) is composed of: two color CCD cameras (1) preferably arranged with parallel axes pointing to the region where the fruits are (5), a matrix (2) of laser diodes low-cost spot optics (spot type) (for example, commercial laser pencils or pointers in a wavelength close to 650 nm.) mounted on an adjustable platform (3) of the pan-tilt type (i.e., azimuth orientable and elevation), which will be used to orient the matrix at the desired angle; and an additional laser diode (4), also on the platform, which projects a flat optical beam at the designated time and place. This commercial diode is nothing more than a spot diode, such as those that constitute the matrix, with a lens that spreads the light horizontally to project a fan-shaped beam instead of a spot or spot.

The invention, as shown in Fig. 2, together with the robotic manipulator (6) equipped with a pre-cut-end terminal element (7), is installed on a vehicle (8), which has a trailer with trays attached (9) ) in which the harvested fruits will be deposited. The vehicle, as shown in Fig. 3, slides on two rails (10) on the ground or mounted on a platform (11) arranged in the same direction as the rows of plants (5a) and (5b) of the that hang the fruits that you want to collect.

With the platform fixed in a certain position centered between two rows, the vehicle moves, stepping on the rails, from one side of the greenhouse to the other, picking up the fruits of one of the rows (5a) and on its way back those of the symmetrical row (5b), or alternatively in the two rows only in its way of going.

The auxiliary systems for feeding, traction and control of devices and the processing and control computer are housed inside the vehicle, as well as the parameter selection panel from which the parameters related to the degree of maturity required for cutting, grading sizes, etc., according to the type and variety of fruit, preferences according to the time, etc.

In one embodiment, the system has four cameras arranged in two stereoscopic pairs, the left pair independently being used in the vision of the corresponding left row and the right pair in the symmetrical row on the right. In another embodiment, a single pair of cameras performs the vision of both rows, rotating 180 ° in solidarity at the required time, commanded by the control system.

The operations that are carried out in each collection position (static vehicle) are those described below:

First, the pair of cameras takes pictures of the scene; These are digitized and stored in real time. The images are then converted from the RGB format, in which the cameras are captured, to the HSV format, which is appropriate for efficient segmentation of the scene.

The segmentation operation consists of identifying and isolating in the images the pixels that correspond to the characteristics of the fruit, distinguishing them from those others that correspond to the rest (bottom, leaves, stems, planting bags, etc.). In general, it is appropriate to perform a pre-segmentation based on color and hue or saturation, working on the H and S coordinates. In each pixel it is determined whether the values of the H and S coordinates correspond to those of the ripe fruit (for example, determined red in the case of the strawberry), checking if they are in the margins of specified values. The values of these margins, which can be entered and varied through the control panel, depend on the variety of the fruit and the degree of maturity specified for the collection. This first processing eliminates pixels whose color and saturation do not correspond to those of the ripe fruit.

Once this pre-segmentation is carried out, the result of which is a binary classification of the pixels of the scene in pixels of the color of the fruit and pixels of another color, a post-processing is carried out, consisting of eliminating the pixels that appear isolated (noise ), to stay only with those pixels grouped in spots or "blobs", that is, groups of pixels connected (neighbors) to each other, which correspond to the fruit or a cluster of fruits in clusters.

The next step is to calculate the 3-D coordinates of certain singular points of the scene, with the ultimate goal of determining whether the "blobs" correspond to an isolated fruit or a group. To do this, first single points are marked on each of the 2-D "blobs" of the two segmented stereoscopic images, in order to carry out the correspondence process (process of identifying, in each of the two images, the points 2-D that correspond to the same and unique spatial point of the 3-D scene). In the vision system of this invention, as shown in Fig. 4, for each "blob" (12) of each of the two images, the 2-D position of its centroid (13) is calculated (geometric center at 2 -D) and, drawing virtually horizontal and vertical lines that pass through the coordinates of the centroid, the four extreme points of the "blob" are also marked: upper (14), lower (15), right (16) and left (17 ), which will be used to determine the size of the "blob" and whether it corresponds to an isolated fruit (and, in this case, determine its size) or a cluster.

The correspondence between the 2-D centroids of all the "blobs" of the two images is then made, that is, the pairs of centroids corresponding to the same (approximate) spatial point are determined. To perform correspondence of the centroids, any known method can be used. In our invention, the centroid point of one image is matched in the other one that is at the same height (Y coordinate) and whose disparity in X (difference in its X coordinates) is compatible with the geometry of the cameras for distance of medium vision Once the correspondence between the centroids is established, and therefore between the "blobs", the actual coordinates in the 3-D space of the apparent center of the "blob" (centroid) are calculated. Also, in the 3-D space, the points corresponding to the "blob" end points marked above (upper, lower, etc.) are calculated. The 3-D reconstruction of a point from its stereoscopic projections is a trivial problem with a known solution, which in the real case only requires knowing the transformation matrix corresponding to the stereo system cameras, which is calculated in a previous process of calibration. Obviously, the calculated 3-D positions must refer to a fixed coordinate origin, which can be taken anywhere in the robotic system or the vehicle. In a prototype of this invention the central midpoint between the cameras has been taken as the origin of coordinates, in the plane in which they are mechanically fixed (on the vehicle). This point must be set in the calibration process and will also be the origin of coordinates to which the movement of the robotic arm and the pan-tilt platform will refer.

Once the 3-D positions of the characteristic points of each "blob" of the scene (centroid and extremes) are determined, each of the "blobs" is analyzed sequentially! from left to right (or from right to left), as described below.

First, it is determined whether the "blob" corresponds to a single isolated fruit or several grouped. For this, the distances in 3-D between the upper point and the lower point and between the right point and the left point of the "blob" are compared, respectively with the diametric reference values of the type of fruit that is collected (previously they have entered in the control panel and they depend on the type of variety of the fruit).

If the comparison is consistent with the hypothesis that the "blob" corresponds to a single isolated fruit, the measures taken are used directly to estimate the size of the fruit for later classification, and it is determined whether its degree of maturity is the desired for collection; for this, the percentage of pixels for which the H and S coordinates take values between the preset ones for the ripe fruit is calculated.

If the percentage is lower than the one specified (selectable in the control panel), the fruit is classified as unripe and another "blob" is analyzed.

If the percentage is higher than the specified one, the fruit is classified as ripe and the arm has to be ordered to direct its prehensile-cutter device to a point of the fruit stalk. The clamp will grab the peduncle, the scissor device will cut it over and It will transfer the fruit to the designated tray where it will be placed in the appropriate receptacle, according to its size.

For this, in general, it is essential to detect the position of the peduncle with precision. This is true even when the ripe fruit is isolated because there may be peduncles in the vicinity of other fruits (immature and therefore not detected) that are not to be cut. Precise peduncle detection is described below.

If it is determined that the "blob" does not correspond to a single isolated ripe fruit, it is treated as a cluster of several in cluster. In this case, the procedure to individualize the fruits one by one in succession begins, starting with the one that is closest to the vision system.

To do this, commanding the pan-tilt platform, the laser diode array is oriented towards the grouping of fruits, using the 3-D coordinates of the "blob" centroid previously calculated as the orientation of the axis of the matrix. The logic of on / off control of lasers causes them to generate a grid of laser light points (marks or spots) on the "blob" of fruits, as shown in Fig. 5, for a short time, during that the scene is captured by the stereoscopic camera system. The next step is to accurately determine the 3-D position of each of the light points (18) of the reticle in the fruit group (19). Light marks have a typical collection distance diameter of between one millimeter and one and a half millimeters. In each of the two two-dimensional images taken by the cameras, the marks are isolated, using for example a segmentation algorithm similar to that described for the pre-segmentation of "blobs". In that case, the coordinates H and S were compared with the desired thresholds. In this case, the pixels corresponding to the mark in each projection can be detected by comparing their V values with a minimum threshold, taking advantage of the fact that the mark has a luminosity considerably higher than the rest of the scene. Another option, even simpler, is to perform a subtraction operation on the V coordinate of the image illuminated by the lasers and the unlit image. The result is an image in which only the laser marks remain, directly segmented. Segmented the marks, their 2-D centroids are calculated in both images by the procedure described above, the correspondence between them in the two images is performed by the procedure also described and the 3-D reconstruction of the mark on the group is performed of fruits. There is thus a map of three-dimensional points on the surface of the fruit grouping.

The more laser marks in a group, the better the 3-D representation of it will be. However, the greater the number of brands, the more difficult the correspondence process between the two images will be. To facilitate the process of correspondence between the two images without sacrificing a good 3-D representation of the grouping, interlaced lasers of different color (wavelength) can be used: in this way, using, in addition to the V coordinate, the color coordinate H, you only have to make the correspondence of those brands that have the same color.

On the map of three-dimensional points obtained, the following processing is performed to estimate the position of each of the fruits to be collected: first, the one closest to the vision system is taken. Next, it is determined which of the 3-D points adjacent to that in the grid correspond to the same fruit, without more than calculating the three-dimensional distances between them and discarding those whose distance is greater than a number related to the caliber of the fruit.

Determined the 3-D points of the grid that belong to the same fruit, the position of the fruit in the cluster is located precisely with the order of cm. Specifically, the approximate 3-D position of the upper point of the fruit can be determined, averaging the x (lateral) coordinate of all the points belonging to that fruit and taking the y (height) yz (depth) coordinates of the point of the fruit. fruit that has a coordinate and (height) greater. This point will correspond approximately to the place where the fruit stalk is born. Then proceed to locate your peduncle with precision of the order of the millimeter.

To accurately detect the peduncle, this invention extends the method described in P200501586 / 5 for detection of isolated peduncles. To do this, again controlling The pan-tilt platform on which the line laser (4) is also located, is directed to project the laser line (20) approximately 1 cm. above the top of the fruit, using as reference the 3-D position of the top point of the fruit calculated as described above, so that the line intercepts a point (21) of the peduncle (22), as shown by the Fig. 6. Once oriented along the desired axis, the control diode of the line diode feeds it, it projects the beam (23) and a spot (21) explicitly appears at the illuminated point of the peduncle. If the peduncle were isolated (there were no nearby peduncles) the scene would be acquired with the cameras, this unique spot would be segmented, it would correspond in the two images and its 3-D position would be reconstructed, thus having the almost exact coordinates of the point of the peduncle to which the press-cutter mechanism should be directed. In practice, the target peduncle is or may be close to others of other fruits (from the cluster or from non-mature non-ripe fruits) and, therefore, the illumination with the line laser generates several marks (one per peduncle). To determine the target peduncle, the illumination is repeated at successive elevation angles, as illustrated in Fig. 7, so that a trace of marks (24) is generated for each of the peduncles present. The peduncle to be cut is one for which a simple extrapolation of the trace marks passes closer to the top point of the fruit to be harvested, whose 3-D position was previously calculated.

At the desired point of the peduncle, precisely determined by the previous procedure, the clamp is directed, which gently grabs the peduncle and separates the fruit from the cluster to isolate it from the rest of the fruits. It remains only to treat it as an isolated fruit in the manner described above for isolated fruits: its degree of maturity and its size are determined and, if it is necessary to cut it, it is cut and deposited in the corresponding receiving tray. Otherwise, it is released gently and the entire operation is repeated in another region of the "blob".

After each cut of fruit, the operation is repeated on the "blob" from the beginning, identifying, locating, classifying and cutting, where appropriate, the other fruits of the cluster, possibly occluded in the images previously taken. The process is continued until there are no fruits left in the cluster or until those that remain are not collected because they have not reached the desired maturity point. Once all the fruits have been collected in all the "blobs" of the scene, the vehicle travels longitudinally on the platform the distance necessary for the cameras to capture an adjacent scene on the row in collection.

The way to realize the pattern of illumination of the lasers of the matrix admits diverse variants, all of them viable. By way of example, this part of the invention can be realized:

-Using a single laser pen that, installed on the adjustable platform, is mechanically deflected successively, and illuminates synchronously with the vision-capture system. In each position the corresponding mark is illuminated and the camera couple takes the projections of the scene, allowing a unique correspondence of the points of each image.

Despite the simplification in the correspondence of the spots, this mode of operation is not recommended, because it unnecessarily slows down the cross-linking process by requiring a mechanical deflection, assuming a slight economic advantage instead.

-Using a matrix of lasers that illuminate sequentially at the same speed as that of capturing the cameras that take the successive images of the spots projected on the cluster. The correspondence between the points of the two projections is also direct because in each pair of images only one mark appears at a time.

-using, as described, an array of lasers with the same arrangement, but illuminating all of them simultaneously and making a multipoint correspondence. In this case, if you want to facilitate the correspondence algorithms and quickly identify each pair, the lasers can alternate color (red, green, blue, etc.), or be of the same color but of different power, so that the different brightness of each spot allows discrimination by intensity.

Finally, the line laser can be installed in the hand of the robotic arm instead of integrated in the pan-tilt platform. In that case the aiming is controlled properly orienting the arm The ignition can be carried out at the desired time (for example, you can first guide the arm towards the fruits and turn on the laser when the hand is close).

Brief description of the drawings

Fig. 1 Main elements of the vision system. The two CCD cameras (1) point to the area where the fruits are (5) to capture the scene and the matrix of point lasers (2) and the line laser (4), mounted on the adjustable platform (3), are They will point to the area indicated in each case.

Fig. 2 Schematic view of the vehicle (8) on which the vision system (consisting of the cameras (1), the adjustable platform (3), the array of point lasers (2) and the line laser (4) will be placed )) and the robotic arm (6) equipped with a prehensile-cutter terminal (7), which will be responsible for collecting the fruits indicated by the vision system. The vehicle has attached a trailer with trays (9) where the collected fruits will be deposited.

Fig. 3 Schematic perspective view of the movement of the vehicle (8) on the rails (10) of the platform (11) in relation to the arrangement of the fruits to be collected (5a) and (5b).

Fig. 4 Example image obtained after pre-segmentation and post-processing in which several "blobs" (12) of different sizes are observed. The centroid (13), upper (14), lower (15), right (16) and left (17) points of each of them are indicated, from which their size is estimated.

Fig. 5 Example of the grid of points (18) that projects the matrix of lasers on a cluster (19) of cluster fruits.

Fig. 6 Schematic perspective view of the operating foundation of the line laser. The fan light (23) emitted by the line laser (4) that is intercepted by the peduncle (22) generates an isolated point (21) on it. Fig. 7 Profile (a) and elevation (b) of the lighting at successive elevation angles performed by the line laser (4). The trace of points (24) that this procedure produces in each peduncle is indicated.

Fig. 8 Diagram of the box (25) in which the laser matrix (2) and the line laser (4) are placed in one embodiment.

Fig. 9 Schematic of the robotic arm (6) in which the gripping (26) and cutting terminal accessories (27) are shown.

Fig. 10 Example of possible arrangement inside the vehicle (8). In it are located the arm control unit (28), the feeders of the cameras (29), the platform controllers (30), the control PC (31) with the parameter selection panel, composed of the own Computer keyboard (32) and a conventional flip-up flat TFT display (33), and wiring (34).

Fig.ll Scheme of interconnections and exchange of signals between the different blocks of the system.

Exhibition of an embodiment

All the necessary elements to realize this invention are available, commercially ready to be integrated; The algorithms needed to perform the vision operations can be encoded following the instructions detailed in the previous section. The algorithms to control the movements of the robotic arm are available in the open literature as reverse kinematics methods to control robots of various degrees of freedom and it is also common to be supplied by the manufacturer himself.

As an example of a specific embodiment of the invention, the case of a vision system consisting of only two cameras arranged to realize the lateral vision with respect to the vehicle's line of movement and mounted on a vehicle will be taken. turntable capable of rotating on itself 180 ° so that the cameras can realize the lateral vision in the opposite line. In this embodiment, the collection is first carried out in a crop line from one side of the greenhouse (beginning) to the other (end). Then, turning the cameras, on the way back (from end to beginning), the collection is done on the opposite line.

Lasers (spot and line) can be chosen from the abundant commercially available ones. A recommended option is to select them at a wavelength close to 650 nm, due to their low cost. Lasers, encapsulated with their lens in a cylindrical device, are arranged in matrix form as indicated by way of illustration in Fig. 8, in which they appear arranged in a geometry of 3 rows and 7 columns (21 spot lasers ). It is noted that the laser configuration must be chosen according to the cultivation characteristics and especially according to the maximum size of the finite clusters. The matrix of lasers (2) is arranged in a box (25), in which the line laser (4) can also be housed. Lasers are controlled individually with the on / off logic, which is nothing more than a logic that feeds each of the lasers or not, as indicated by the central control of the PC.

The box with the laser matrix is fixed on a pan-tilt platform controlled by RS232 interface from the control PC synchronously with the rest of the vision system.

The vehicle, with the attached trailer that transports the tray system, is arranged on the rails of the platform. The vehicle travels longitudinally, gathering the fruits on the platform, as indicated above.

The movement is carried out by the action of a servomotor that acts in a conventional manner on the vehicle's traction system and whose displacement in each step is determined by the field of vision of the chosen cameras, so that in the next position the vision of The cameras are adjacent to the previous one. The movement order is given by the control PC, through the servo motor controller, at the moment when the collection of each scene is finished. At the end of the collection hall, there is a simple switch that is activated when it is reached by one of the wheels of the vehicle, indicating that it has reached the end of the aisle and reversing the direction of movement of the vehicle and rotating the camera platform 180 °. The harvesting operation is then carried out on the twin crop line in the opposite direction until the vehicle reaches the beginning of the greenhouse.

It is clear that the philosophy of movement between the rows can be executed in several ways depending on the geometry of the greenhouse.

In the upper part of the vehicle the turntable is arranged with the parallel cameras, and with the angle of elevation necessary to capture the collection scenes. The pan-tilt platform on which the laser matrix is fixed and the line laser is arranged so that it is capable of being directed to one side or the other.

Since commercial pan-tilt platforms usually have a limited azimuth turning capacity (for example, + -160 °), it must be arranged so that the blind angular sector, to which lasers cannot be directed, does not match with the collection areas.

In the same way, the robotic arm or device must be arranged so that the region accessible by the end (hand with terminal elements) includes the harvesting areas on both sides of the vehicle and the area of the fruit receiving tray in the tray carrier vehicle .

The arm (6), as shown in Fig. 9, carries at its end the grip (26) and cut (27) accessories, which are independently controlled. The prehensile device first grabs the peduncle of the fruit at the point indicated by the vision system, approximately 10-15 mm from the fruit. The cutting device immediately cuts the peduncle to 10 mm. above. The fruit is transported by the arm prehensile device and deposited in the tray or receiving basket.

Inside the vehicle they are arranged in the way that best suits their geometry (see a possible arrangement in Fig. 10): the arm control unit (28), the camera feeders (29), the platform controllers ( 30), the PC of control (31) with the parameter selection panel, which in this embodiment is chosen to be the computer's own keyboard (32) and a conventional flip-up flat TFT screen (33), and the wiring (34).

The interconnections and exchange of signals between the blocks of the embodiment are carried out as indicated below (see Fig. 11):

Each of the cameras is connected, via an RG-58-U coaxial cable to the corresponding commercial image capture card (frame-grabber type), in charge of digitizing the images and transferring them to the PC memory through the bus PCI, commanded by the main program.

The platform responsible for orienting lasers is a commercial pan-tilt (azimuth-lift) with precision better than a tenth of a degree, turning capacity in. elevation between -45 and + 30 ° and in azimuth of + -160 °, which allows it to be oriented on both sides of vision. Motion control is performed from the PC via RS232 interface, communicating the coordinates of the desired movement to the unit controller.

The manipulator can be a vertical articulated arm of five degrees of freedom, with repeatability of movements better than + -0.02 mm. Its movements are determined by the controller unit, to access the coordinates indicated by the PC via TCP / IP ethernet or through the serial port. The arm is mounted with a base of standard ISO 9409 terminal elements, on which the two terminal elements are fixed, which can be operated electrically or pneumatically through the arm itself, by the controller that executes the orders given from the control program in the computer. The terminal elements are: a clamp with parallel fingers and a commercial cutting device compatible with ISO 9409.

Lasers are controlled from the main program through the parallel port or a signal generating PCI card (as many as lasers), which is responsible for feeding the lasers individually at the right time. Industrial application

The invention has immediate industrial application used as a collection system for small fruits grown in rows. It is especially suitable for automated harvesting in fruit greenhouses grown in regular structures, in which a vision system guides the robotic system, which performs the tasks of apprehension, cutting and depositing of the fruit in the desired place. The invention makes it possible to effectively designate the precise 3-D position of the point at which to guide the terminal mechanism, which will grab the peduncle of a ripe fruit and cut it over at the predetermined distance, distinguishing it from the peduncles of other nearby fruits that are not due cut.

Claims

1) Stereoscopic artificial vision system for precise guidance of robotic manipulator to detect, locate and collect small fruits, which can grow clustered in clusters, with the ability to isolate and determine the position of each fruit, and a point of its peduncle towards the that the robotic manipulator will be guided, characterized by: • differentiating and locating the individual fruits in the clusters or groups from the three-dimensional reconstruction of the spots or marks projected in the cluster by a set of specific optical lasers arranged in matrix, and • accurately detect the peduncles from the marks generated on them by the fan beams projected by an optical line laser oriented at successive elevation angles.

2) System according to claim 1, characterized in that the laser projections are achieved by deflecting a laser pointer, according to the predetermined pattern, to generate the spot matrix.

3) System according to claim 1, characterized in that the spots are generated with lasers of different wavelengths or with lasers of different power.

4) System according to claims 1, 2 or 3, for use in the collection of strawberries or other delicate delicate fruits grown in high rows, such as in hydroponic crops in greenhouses, equipped with a robotic device with grip and cut pliers, installed on a sliding vehicle, with container attached with trays to deposit the fruit, which performs, in each aisle, the collection in both directions of longitudinal movement, using either two pairs of cameras or only one pair with the ability to rotate 180 ° azimuthal