EP4713894A1

EP4713894A1 - Method and system for autonomous driving an agricultural vehicle between rows

Info

Publication number: EP4713894A1
Application number: EP24728135.5A
Authority: EP
Inventors: Luca Ferrari; Fabio LISURICI; Andrea Rizzi; Andrew Wilson
Original assignee: CNH Industrial Italia SpA
Current assignee: CNH Industrial Italia SpA
Priority date: 2023-05-15
Filing date: 2024-05-14
Publication date: 2026-03-25
Also published as: WO2024235962A1; IT202300009774A1

Abstract

Method of assisting driving of an agricultural vehicle (AV) between rows (LR, RR), in which a pair of rows identifies a lane (P) to be travelled by the agricultural vehicle, the agricultural vehicle including at least one sensor (PCS) suitable for generating a pointcloud of a scenario in front of the vehicle and a processing unit (CPU2) configured to generate a left trajectory (LT) and a right trajectory (RT), based on the pointcloud, approximating a development respectively of a row left (LR) and a right row (RR), the method comprising the calculation of a set of left points and a set of right points of said pointcloud falling respectively inside left and right closed volumes and the detection of an end of the lane (P) when both said set of left and right points have a number lower than a preordained threshold.

Description

DESCRIPTION

"Method and system for autonomous driving an agricultural vehicle between rows"

★ ★ ★

Field of the invention

The present invention relates to the field of autonomous driving systems and in particular to the field of methods and systems for assisting the driving of agricultural vehicles between rows and in particular to a method for identi fying the end of the lane travelled between two adj acent rows .

State of the art

US2021000006A1 describes a scenario in which an agricultural vehicle , in the absence of GPS guidance or in any case unreliable , is faced with the end of a lane between two adj acent rows and plans a reversal maneuver to take another lane adj acent or in any case paral lel to the previous one . In this scenario , the geospatial relationship between the vehicle and environmental features can be determined based on distance and/or camera sensor feedback and analysis . Once the vehicle reaches and detects an end o f row condition, the vehicle can execute a turn and identi fy the optimal traj ectory to enter the next agricultural lane . Using this method, the vehicle can cover an entire parcel of agricultural rows without the use of GPS . In some implementations , a mapping and tracking method called SLAM can be used based on lidar and/or camera data that creates a local map of the environment as the vehicle drives through the environment which can be referenced for later use .

Once the autonomous driving system has identi fied the end of the lane , it can control the vehicle to make a U-turn into a new lane . Figure l a shows a reversal o f a vehicle which from the f irst lane enters the second lane between rows parallel to each other .

The configuration of the row shown in figure la is absolutely theoretical .

In reality, the length of the rows can be conditioned by the topology of the plot of land on which they are planted .

For example , Figure lb shows a situation where the topology of the land parcel has af fected the length of the rows .

Figure 2a schemati zes a top view of a vehicle moving within a lane , in which the left row is continuous , whi le the right row has a locali zed shortage of plants due to the death of some plants .

Considering all the poss ible situations , it is clear that it is di f ficult for an autonomous driving system to correctly decide the actions to be taken to drive the vehicle , especially when GPS guidance is not available .

Figure 2b schemati zes a top view of a vehicle moving within a lane , in which the rows have di f ferent lengths , for example due to an irregular topology of the piece of land, similar to what is shown in figure lb .

Looking at figure la, the autonomous driving system could decide that the lane is finished when both the right and left rows are finished at the same time .

However, this solution is not applicable to the situation shown in figure lb or 2b . The autonomous driving system may decide that it has reached the end of the lane due to a local i zed lack of plants like the one shown in figure 2a . In such circumstances , the autonomous driving system could induce the vehicle to reverse without completing the lane , with the risk of damaging some plants during the reversal maneuver .

Unless speci fically excluded in the detailed description that follows , what is described in this chapter is to be considered as an integral part of the detailed description .

Summary of the invention

The obj ect of the present invention is to indicate a robust algorithm to assist in identi fying the end of the lane identi fied by two adj acent rows of plants .

The basic idea of the present invention is to identi fy a first left traj ectory, relative to the left row, and a second right traj ectory, relative to the right row; then a right closed volume with a long linear shape is ideally coaxial with the right traj ectory and a left closed volume with a long linear shape is ideally coaxial with the left traj ectory, where both the right and left volumes are at the same distance from a frontal part of the vehicle and have the same shape and dimensions , according to the invention the end of the lane is identi fied when it results that both the two trains do not contain points belonging to a plan .

The closed volumes preferably have the shape of a parallelepiped .

The closed volumes each comprise a train of closed subvolumes each having the shape of a spheroid or more preferably the shape of a parallelepiped .

The volumes can have a s i ze that can be set according to the characteristics of the plants that form the rows .

Furthermore , the number of volumes that form a train can be variable in relation

The dependent claims describe preferred variants of the invention, forming an integral part of the present description .

Brief description of the figures

Further obj ects and advantages of the present invention will become clear from the detailed description that follows of an embodiment of the same ( and of its variants ) and from the annexed drawings given for purely explanatory and nonlimiting purposes , in which : figures la and lb show two di f ferent examples of land topologies on which rows of plants are arranged; in figure 2a and 2b two situations are shown which can be mistaken for each other ; figure 3 shows an agricultural vehicle equipped with an autonomous/assisted driving system according to the present invention; figures 4a - 4c ' graphically show a sequence of operations functional to the execution of the method obj ect of the present invention;

Figures 5a and 5b show two situations mirroring those of

Figures 2a and 2b resolved by the method obj ect of the present invention; figure 6 shows a further preferred variant of the invention; figure 7 shows a flowchart representative of a possible implementation of the method obj ect of the present invention, adhering to the sequence o f operations of figures 4a - 4c ' .

The same reference numbers and letters in the figures identi fy the same elements or components or functions .

It should also be noted that the terms " first" , " second" , " third" , " superior" , " inferior" and the like may be used herein to distinguish various elements . These terms do not imply a spatial , sequential , or hierarchical order for the modi fied items unless speci fically indicated or inferred from the text .

The elements and characteristics illustrated in the various preferred embodiments , including the drawings , can be combined with each other without however departing from the scope of protection of the present application as described below .

Detailed description

According to the present invention the AV agricultural vehicle is equipped with a PCS sensor capable of generating a so-called point cloud relating to the scenario in front of the vehicle. For the generation of the pointcloud various devices can be used including LiDAR, cameras, stereocameras, radar, etc...

For this purpose the PCS sensor is preferably fixed to a front part of the vehicle, for example, on the roof of the vehicle control cabin as shown in figure 3.

According to the present invention, by means of any algorithm per se known, for example as indicated in IT201900024685, two trajectories are generated and then acquired: a left trajectory LT, relating to the left row LR, and a right trajectory RT, relating to the right row RR, as shown in figure 4b. These right and left rows identify the lane P in which the agricultural vehicle moves.

Generally, the right and left trajectories are used to correct the trajectory of the vehicle moving inside lane P delimited to the right and left by the right and left rows. Therefore, they are available because they are calculated by the autonomous driving system, indicated as CPU2 in figure 3.

The method object of the present invention aimed at identifying the end of the lane P engaged by the vehicle is now described in detail.

With reference to figure 4c, two closed volumes are generated, left LV and right RV, of longitudinal shape, respectively superimposed and coaxial with the left LT and right RT trajectories. It is an overlay within a virtual space in which the pointcloud is contained. This is a parametric space per se known .

Therefore, the pointcloud points that fall within the virtual volumes are counted and when these are lower in number than a predetermined threshold, simultaneously for both the right and left volumes, then the method identifies that the end of the lane has been reached.

The term "when" should be interpreted as meaning that the end of the lane is identified "in response to" the occurrence of a situation in which the points inside the two volumes right and left are simultaneously lower than this predetermined threshold.

With reference to figure 4c' , the right and left volumes can be divided into sub-volumes LV1 , LV2, LV3, etc. and RV1 , RV2, RV3, etc.

They can be, for example, a sequence of spheroids having a relative centroid on the respective trajectory.

More preferably these sub-volumes are parallelepipeds adjacent to each other without interruption, forming a right train of sub-volumes and a left train of sub-volumes.

When the closed volumes LV and RV are subdivided into subvolumes, then the counting of the points which fall within them is made for each sub-volume. Advantageously, it is easier to eliminate the noise in this way. In fact, if a significant number of points are included in a sub-volume it is highly probable that it is a plant. However, the same number of points when compared to an entire volume may not indicate the presence of a plant. Therefore, the greater the subdivision of closed volumes into sub-volumes, the greater the accuracy of plant identification. Obviously, an excessive subdivision of the plants can make the present method excessively heavy, in terms of computation. Preferably, each sub-volume has a si ze comparable with the average volume of the plants which define the rows .

With reference to figures 5a and 5b it can be noted that the sub-volumes RV3 - RV5 are empty, however, according to the present algorithm, only when all the right sub-volumes RV1 - RV5 and the left ones LV - LV5 are simultaneously empty, then it is identi fied the end of the lane .

More particularly, since it is substantially impossible for there no reflections from the environment are present , even in the absence of plants , then, from the point of view of the balance of probabilities , it can be considered that in each of the right and left volumes or right and left subvolumes , the number of points is lower than a predetermined threshold . When this occurs then the present algorithm detects the end of the lane as a result of the fact that the right and left volumes or the right and left sub-volumes have a number of points lower than the aforementioned predetermined threshold .

Preferably, the point cloud obtained is filtered so as to eliminate the reflection points relating to the ground and preferably also to the grass , so that it is possible to reduce the aforementioned threshold .

Preferably, the fi ltered pointcloud i s decimated in order to eliminate the points with lower intensity . Advantageously, this makes it quite probable that the points within a volume or sub-volume actually belong to a plan .

It is clear that the more ef fective the identi fication of a plant using the pointcloud is , the lower the number of points needed to recogni ze the presence of a plant in a volume or sub-volume .

Figure 7 shows a flow chart useful to facilitate understanding of the cyclic method obj ect of the present invention : Step 1: Acquisition of said left (LT) and right (RT) trajectories and of said pointcloud,

- Step 2: Generation of a left virtual closed volume (LV; LV1 - LV5) and a right virtual closed volume (RV; RV1 - RV5) , both longitudinal in shape, and positioning of said closed volumes so as to be coaxial respectively with said left and right trajectories;

- Step 3: Calculation of a set of left points and a set of right points of said pointcloud falling respectively within said closed left and right volumes;

- Step 4: Detection of an end of the lane (P) when both said set of left and right points have a number lower than a preordained threshold.

It is evident that the closed virtual volumes, being longitudinal shaped, define corresponding development axes. Therefore the expression "closed volumes . . . coaxial" is clear, meaning that the respective development axes are coaxial with the trajectories.

Figure 6 shows a further variant of the invention, in which two right trajectories RR, 2RR and two left trajectories LR, 2LR are reconstructed. In particular, the external rows immediately adjacent to those which define the lane in which the AV vehicle moves are considered.

The same method applied to the first right and left rows can therefore be applied to the second right and left rows in order to have greater robustness in determining the end of the aisle.

Therefore, when the processing unit is configured to calculate also the more external trajectories 2LT and 2RT, the following further steps are performed

- Step Ibis: Acquisition of said second left trajectory 2LT and said second right trajectory 2RT,

Step 2bis: Generation of a second left virtual closed volume 2LV; 2LV1 - LV5 and a second right closed virtual volume 2RV; 2RV1 - RV5 , both longitudinal in shape , and positioning of said second closed volumes so as to be coaxial respectively with said second left traj ectory 2LT and said second right traj ectory 2RT ;

- Step 3bis : Calculation of a second set of left points and a second set of right points of said pointcloud falling respectively within said second closed left and right volumes ;

- Step 4bis : Detection of an end of lane P when said first and second sets of le ft and right points have a number lower than a preordained number threshold .

The number threshold can be variable as a function of a distance from the front portion of the vehicle , and in particular can be inversely proportional with the distance . The blocks shown in dotted lines are completely optional . Steps Ibis - 4bis are optional and can be performed, for example , at the same time or after the respective steps 1 - 4 .

They represent respectively

- Step 12 : Filtering the soil , described above

- Step 13 : decimation of the pointcloud described above .

They can be performed in parallel and according to any order with respect to step 2 relative to the construction of the volumes , in fact , the ef fects of steps 12 and 13 are detectable only when, in step 3 , the calculation of the points that fal l within the volumes or sub-volumes generated in step 1 .

It is also important to note that steps 12 and 13 are independent from each other and therefore they can be executed in any order and one of them cannot be executed when the generated pointcloud has intrinsic characteristics , for example such as to lead to a reduced number of pointcloud points or when ground reflections are automatically eliminated by the sensor itsel f .

The present invention can advantageously be implemented through a computer program compri sing coding means for carrying out one or more steps of the method, when this program is executed on a computer . Therefore it is understood that the scope of protection extends to said computer program and also to computer-readable means comprising a recorded message , said computer-readable means compris ing program coding means for carrying out one or more steps of the method, when said program is run on a computer . Variants of the non-limiting example described are possible , without however departing from the scope of protection of the present invention, including all equivalent embodiments for a person skilled in the art , to the contents of the claims .

From the description given above , the person skil led in the art is capable of reali zing the obj ect of the invention without introducing further constructive details .

Claims

1. Method of assisting of an agricultural vehicle (AV) driving between rows (LR, RR) , wherein a pair of rows identifies a lane (P) to be travelled by the agricultural vehicle, the agricultural vehicle including at least one sensor (PCS) suitable for generating a pointcloud of a scenario in front of the vehicle and a processing unit (CPU2) configured to generate a first left trajectory (LT) and a first right trajectory (RT) corresponding to said pair of rows one the basis on the pointcloud, approximating a development respectively of a left row (LR) and a right row (RR) , the method comprising the following steps in cyclical succession :

(Step 1) : Acquisition of said first left trajectory (LT) and said first right trajectory (RT) and of said pointcloud,

- (Step 2) : Generation of a first left virtual closed volume (LV; LV1 - LV5) and a first right virtual closed volume (RV; RV1 - RV5) , both longitudinal shaped, and positioning of said first virtual closed volumes in so as to be coaxial respectively with said first left trajectory and said first right trajectory;

- (Step 3) : Calculation of a first set of left points and a first set of right points of said pointcloud falling respectively within said first left and right virtual closed volumes ;

(Step 4) : Detection of an end of the lane (P) when both said first sets of left and right points have a number lower than a predefined threshold number.

2. The method according to claim 1, wherein a length of each of said left and right virtual closed volumes is settable.

3. Method according to one of claims 1 or 2, wherein each of said right and left closed virtual volumes is partitioned into left (LV1 - LV5) and right (RV1 - RV5) trains of subvolumes and wherein said detection (Step 4) of the end of lane (P) is caused when both sets of left and right points, respectively falling within the left and right sub-volumes, all have a number lower than said predetermined threshold.

4. Method according to claim 3, wherein said predetermined number threshold is variable as a function of a distance from a front portion of the vehicle.

5. Method according to claim 4, wherein said function is inversely proportional to the distance from the front portion of the vehicle.

6. Method according to any one of the preceding claims 1 - 5, wherein said predetermined threshold is fixed.

7. Method according to any one of the preceding claims, wherein said left and right closed virtual volumes have the shape of a parallelepiped.

8. Method according to any one of claims 3 - 6 when depending on claim 3, wherein said sub-volumes have the shape of a spheroid or parallelepiped and/or wherein said closed virtual volumes are given by the union of said subvolumes .

9. Method according to any one of the preceding claims, wherein said processing unit (CPU2) is configured to generate a second left-hand trajectory (LT1) and a second right-hand trajectory (RT2) corresponding to further rows immediately adjacent to external ones with respect to said pair of rows, and wherein the method comprises

(Step Ibis) : Acquisition of said second left trajectory (2LT) and said second right trajectory (2RT) ,

(Step 2bis) : Generation of a second left virtual closed volume (2LV; 2LV1 - LV5) and a second right virtual closed volume (2RV; 2RV1 - RV5) , both longitudinal, and positioning of said second closed volumes so to be coaxial respectively with said second left trajectory (2LT) and said second right trajectory (2RT) ;

(Step 3bis) : Calculation of a second set of left points and a second set of right points of said pointcloud falling respectively within said second closed left and right volumes ;

- (Step 4bis) : Detection of an end of the lane (P) when said first and second sets of left and right points have a number lower than a predefined number threshold.

10. Method according to any one of the preceding claims, further comprising the following preliminary steps between said acquisition step (Step 1) and said calculation step (Step 3) of said sets of points:

- (Step 12) : filtering the pointcloud in order to eliminate the soil;

(Step 13) : decimation of the pointcloud in order to keep the points having an intensity that exceeds a predetermined intensity threshold.

11. The method according to any preceding claim further comprising a step of activating a reversing manoeuvre in response to detecting said end of the lane.

12. First processing unit (CPU1) operationally connected with a second processing unit (CPU2) configured for autonomous driving control of an agricultural vehicle (AV) , wherein the second processing unit is configured to acquire a pointcloud from at least one sensor (PCS) capable of generating a pointcloud of a scenario in front of the vehicle configured to generate a left trajectory (LT) and a right trajectory (RT) , based on the pointcloud, approximating a development respectively of a left row (LR) and a right row (RR) and wherein said first processing unit is configured for

- Acquiring said left (LT) and right (RT) trajectories and of said pointcloud from said second processing unit,

- Generating a left virtual closed volume (LV; LV1 - LV5) and a right virtual closed volume (RV; RV1 - RV5) , both long-limbed, and position them respectively so that they are coaxial respectively with said left and right trajectories;

- Calculating a set of left points and a set of right points of said pointcloud falling respectively within said closed left and right volumes;

- Detecting an end of the lane (P) when both said set of left and right points have a number lower than a predefined threshold .

13. A computer program comprising instructions for causing the first processing unit (CPU1) of claim 12 to implement the method according to claim 1.

14. A computer readable medium having stored the program of claim 13.

15. Agricultural vehicle (AV) comprising a sensor (PCS) suitable to generate a pointcloud of a scenario in front of the vehicle, a second processing unit (CPU2) configured to generate a left trajectory (LT) and a right trajectory (RT) on the basis of the pointcloud, approximating a development respectively of a left row (LR) and a right row (RR) and to control a trajectory of the vehicle inside an intermediate lane (P) between said right and left trajectories, the vehicle being characterized in that it comprises said first processing unit according to claim 12.