DE102018220835A1 - Method and sensor system for autonomous operation of a group of vehicles - Google Patents

Abstract

The invention relates to a method for autonomously operating a vehicle network away from roads, with detecting information with sensors present in the vehicle network to enable autonomous operation of the vehicle network and communicating the detected information within the vehicle network to maintain the autonomous operation of the vehicle network. The method is carried out with a group of vehicles which have vehicles which are designed for work away from roads. The invention further relates to a sensor system for autonomous operation of a vehicle network away from roads, with a sensor system that can be arranged on vehicles of the vehicle network for recording information, communication means for communicating the recorded information within the vehicle network and a control system for performing the method.

Description

Technical field

The invention relates to a method and a sensor system for operating a group of vehicles. The present invention also relates to a control system for performing such a method.

State of the art

For the operation of a vehicle, it is known to provide sensors with which autonomous operation of the vehicle can be made possible. In the DE 10 2014 216 257 A1 is described for this purpose to adapt the driving strategy of a single vehicle in road traffic to movements of neighboring vehicles.

Presentation of the invention

The invention relates to a method for autonomously operating a group of vehicles. The method is carried out off-road with a vehicle network which has vehicles which are designed for work operation off-road. Working away from roads can include operating or working the vehicle network on an agricultural area and / or construction site.

The vehicle group can be designed as an autonomous vehicle group. The vehicle network can have at least two vehicles, which can be designed as autonomous vehicles. The vehicles can be work machines. In one example, the vehicles can be agricultural machines that can be operated autonomously on the agricultural area. In a further example, the vehicles can be agricultural machines or construction machines that can be operated autonomously on the construction site. At least one vehicle of the vehicle network can be equipped with a loading area or a tool for carrying out work tasks in work operations away from roads.

The vehicle network can have a leading vehicle, which specifies the operation of at least one following vehicle. The vehicle network can also be operated as a vehicle convoy with a vehicle in front and at least one vehicle behind.

The autonomous operation of the vehicle group can have an autonomous operation of one or more vehicles of the vehicle group according to one of the levels from level one to five of the classification of autonomous driving in Europe. As the lowest level of autonomy, autonomous operation can also have driver assistance which supports a driver in operating a vehicle of the vehicle network. Autonomous operation can also include partially or fully automated operation of one or more vehicles in the vehicle network. In the autonomous operation of the vehicle group, at least the following vehicles or the following vehicles can be operated without a driver. In addition, the lead vehicle or the vehicle in front can also be operated without a driver.

The autonomous operation of the vehicle network can include at least one autonomous operating function. Examples of an autonomous operating function can be an autonomous traversing of a trajectory, an autonomous detection and circumvention of obstacles in an autonomously driving route or the autonomous use of tools.

Operating the vehicle network away from roads can include so-called off-highway applications for a vehicle network. Operating the vehicle network away from roads excludes the exclusive operation of the vehicle network on public roads. In contrast, operating the vehicle network on private roads may include operating the vehicle network away from roads.

The invention encompasses the acquisition of information with sensors in the vehicle network to enable autonomous operation of the vehicle network. This is done away from roads. The information can in principle be environment-based or vehicle-specific information, which can be used for the autonomous operation of the vehicle network. The information can therefore be sensor information. The environment-based information can be information about the environment as such and / or information about stationary or dynamic objects in the environment. The vehicle-specific information can be operating parameters of at least one vehicle of the group of vehicles or information about the location or the movement behavior of the at least one vehicle of the group of vehicles.

The sensors in the vehicle network can include an environment detection system, which can have at least one sensor for detecting a vehicle environment. Such an environment detection system can be present on each of the vehicles in the vehicle network. Alternatively or In addition, the sensors in the vehicle network can comprise a navigation system, which can have at least one sensor for navigating a vehicle of the vehicle network. Such a navigation system can be present on at least one of the vehicles in the vehicle network.

The invention further comprises communicating the acquired information within the vehicle group in order to maintain the autonomous operation of the vehicle group. This is done away from roads. The communication of the acquired information can include an exchange of information between at least two vehicles of the vehicle group. The communication can therefore have a bidirectional communication of the acquired information between at least two vehicles. As an alternative or in addition, the communication can include the transmission of information from one vehicle to another vehicle in the vehicle network. The communication can therefore have a unidirectional communication of the acquired information between at least two vehicles.

In order to maintain autonomous operation, it may be necessary to combine, process and / or evaluate the information recorded and / or communicated.

Within the scope of the invention, information acquired by the vehicle network for the autonomous operation of a vehicle network outside of roads is made available or shared across the vehicle network within the vehicle network. A necessary intervention by a driver or operator of the vehicle group to maintain the autonomous operation of the vehicle group due to a lack of information can thereby be avoided. Failure of a sensor on a vehicle can thus be compensated for by the presence and information acquisition of another sensor on another vehicle in the vehicle network. In addition, sensors in the vehicle network can be saved by sharing recorded information. A basic idea of the invention can thus be seen in the fact that sensor data provided by vehicles of an autonomous vehicle group are used across vehicles.

In one embodiment, the method has a redundant detection of information with the sensor system present in the vehicle network and an at least partial communication of the redundantly recorded information within the vehicle network. Redundantly recorded information can have a redundantly recorded environment of the vehicle group and / or redundantly recorded object information. Alternatively or additionally, information recorded redundantly can have vehicle parameters or vehicle positions of a vehicle of the vehicle group that are recorded redundantly. Such redundancy concepts can advantageously increase the reliability and safety of the autonomous operation of the vehicle network. In addition, information available on a vehicle can be checked or improved in terms of quality with information that has been recorded by or is present on another. A malfunction of a sensor that may remain undetected without redundancy in the information acquisition can thus be recognized in good time before the autonomous operation of the vehicle network can be disturbed or can lead to damage.

In a further embodiment, the method comprises acquiring information about the location of an object in the environment of the vehicle group with the environment detection sensor system present in the vehicle group. In this embodiment, the method also comprises communicating the acquired information in order to avoid a collision of the vehicle group with the object. The detection of the vehicle environment can include point cloud-based detection and / or image-based detection of a vehicle environment with corresponding sensors on at least one vehicle of the vehicle group. The sensor system can be, for example, at least one radar sensor, laser scanner and / or ultrasound sensor in order to detect a two- or three-dimensional point cloud of the surroundings. An advantage of such an environment detection can be the detection of an obstacle in the route of the vehicle group in order to avoid a collision with the obstacle by an autonomous evasive maneuver of at least one vehicle of the vehicle group.

In a further embodiment, the capturing of information comprises capturing camera images of an environment of the vehicle group with cameras present in the vehicle group. In this embodiment, communicating the captured information comprises communicating the captured camera images within the vehicle network. For this purpose, at least one camera can be arranged on each vehicle of the vehicle group, which camera is designed to capture the respective vehicle's own surroundings. Obstacles for collision avoidance can be reliably identified in camera images by means of image processing. In addition to true color images, infrared images can also be used to recognize a living object.

In a further embodiment, the acquisition of information comprises an acquisition of position information of the vehicle group with position detection sensor systems present in the vehicle group. In this embodiment, communicating the acquired information comprises communicating the acquired information for navigating the vehicle group. Position information recorded can have absolute vehicle positions in a cross-vehicle coordinate system and / or relative vehicle positions in a vehicle-specific coordinate system.

Acquisition of absolute position information can include satellite-based acquisition of vehicle positions with corresponding sensors on at least one vehicle in the vehicle network. The sensor system can be, for example, a receiver for GNSS navigation signals in order to detect an absolute position or trajectory of a vehicle in the vehicle network. Such a receiver can be, for example, a GPS antenna. Detection of relative position information can include detection of a direction and / or a distance between two vehicles with corresponding sensors on at least one vehicle of the vehicle group. The sensor system can be, for example, a laser scanner or an ultrasound sensor in order to detect a relative position of a vehicle in the vehicle network with respect to a vehicle equipped with the sensor system in the vehicle network.

Based on a detected absolute position of a vehicle and a detected relative position of another vehicle with respect to the vehicle, the absolute position of the further vehicle can be calculated by means of polar appending. The calculated absolute position can be communicated in the vehicle network. This means that an absolute position of a vehicle can be recorded and communicated in the vehicle network even without the vehicle's own sensors.

In a further embodiment of the method, the communication of the recorded information within the group of vehicles comprises a transfer of recorded information across vehicles for the merging of information from different vehicles within the group of vehicles. Merging information can include combining information. The transmission of captured information can include supplementing information that can be captured by a vehicle with information that can be captured by another vehicle in the vehicle network. As an alternative or in addition, the transmission of recorded information can include replacing information that cannot be recorded by a vehicle with information that can be recorded by another vehicle in the vehicle network. The non-detectability of information can be due to a failure or the absence of a sensor on a vehicle. An advantage of such a merging of information can make it possible to provide information regarding the surroundings or the position of a vehicle thereon. This can be the case, although appropriate sensors for generating this information cannot be present or have failed on this vehicle.

The cross-vehicle transmission of recorded information for the merging of information can include access to a secondary data acquisition with a secondary sensor system within the vehicle network. This can be carried out due to the failure of a primary data acquisition with a primary sensor system within the vehicle network. The primary and secondary data acquisition can represent redundant data acquisition. A termination of the autonomous operation of the vehicle network due to a sensor failure can thus be avoided in an advantageous manner.

Another embodiment of the method comprises transforming the communicated information into a vehicle-specific coordinate system. The transforming may include transforming information from one sensor coordinate system to another sensor coordinate system or a vehicle coordinate system. A transformation can have a translation and / or a rotation. Transforming information captured in different coordinate systems into a common coordinate system can allow the information to be merged.

A further embodiment of the method includes taking into account a quality parameter of the communicated information when making a decision about maintaining the autonomous operation of the vehicle network. The quality parameter can be an accuracy or reliability of the information recorded. For example, a detection accuracy of an object in the environment of the vehicle network or a position accuracy of a vehicle of the vehicle network can be taken into account for autonomous operation. Information recorded within the vehicle network can thus be taken into account with a different weighting when deciding whether to maintain or terminate autonomous operation.

In principle, each subset of the information recorded in the vehicle group can be used as the basis for a decision about maintaining the autonomous operation of the vehicle group. An advantageous embodiment of the method has a decision about maintaining the autonomous operation of the vehicle network based on a neural network which is based on the acquired and / or communicated information as input values. In addition, the quality parameters of the recorded and / or communicated information can be used as weightings within the neural network. An output value of the neural network can, for example, be a collision probability of at least one vehicle of the vehicle network with an object in the vicinity of the same. The neural network can thus be used, for example, to make a decision about driving on, driving around an obstacle or stopping the vehicle network.

The invention further relates to a control system for vehicles of a group of vehicles for performing the method for autonomously operating a group of vehicles away from roads or one of the embodiments of the method. For this purpose, the control system has control devices on the vehicles of the vehicle network. The control devices have at least one interface for reading in information that has been recorded with sensors in the vehicle network to enable autonomous operation of the vehicle network. The control devices also have interfaces for communicating the read-in information within the group of vehicles in order to maintain the autonomous operation of the group of vehicles. In addition, the control devices can have corresponding inputs or outputs or interfaces for carrying out further described method steps.

The invention also relates to a sensor system for the autonomous operation of a vehicle network off-road, which has vehicles that are designed for work operation off-road. The sensor system has a sensor system that can be arranged on vehicles of the group of vehicles for acquiring information to enable autonomous operation of the group of vehicles. The sensor system also has communication means for communicating the acquired information within the vehicle network for maintaining the autonomous operation of the vehicle network and the control system. The means of communication can be wireless means of communication.

Figure list

• 1 shows a plan view of a vehicle group to explain the method and the sensor system for autonomous operation of the vehicle group.
• 2nd shows a further top view of the vehicle group to further explain the method and the sensor system for autonomous operation of the vehicle group.
• 3rd shows a top view of vehicles of the vehicle group for explaining a fusion of acquired information.
• 4th shows a neural network, which is based on recorded information of a vehicle of the vehicle group.
• 5 shows schematically a neural network, which is based on acquired and communicated information of several vehicles of the vehicle group.
• 6 shows a flowchart with method steps for performing the method for autonomous operation of a vehicle group.

Description of embodiments

In 1 is a group of vehicles 10th shown which three vehicles 11 includes. With the vehicles 11 are tractors that run side by side in a group on an agricultural area. In the environment 20th of the vehicle network 10th there is an object 22 , which is in the route of the middle vehicle 11 and in the direction of travel.

As in 1 Shown below threatens to maintain autonomous operation of the vehicle network 10th , that means here when the vehicle group continues 10th , a collision between the middle vehicle 11 and the object 22 .

On the vehicles 11 is one camera at a time 12 arranged, each a camera image 13 of the environment 20th detected. As in 1 Shown in the middle is the object 22 from the camera 12 of the middle vehicle 11 in its camera image 13 not recorded. In the others with the cameras 12 of the other vehicles 11 captured camera images 13 is the object 22 on the other hand shown. The image coordinate systems of these camera images 13 are different, so the object 22 in the respective camera image 13 located in different positions.

In 2nd is shown that the camera image 13 of the middle vehicle 11 from 1 through a combination or fusion of the camera images 13 of the side vehicles 11 with a combined or fused camera image 15 is replaced. For this, the camera images 13 of the side vehicles 11 by means of communication means arranged thereon 16 on the middle vehicle 11 transmitted and from a control device 30th on this vehicle 11 read. One in 1 shown missing environment detection on the middle vehicle 11 is then determined by the surroundings on the neighboring vehicles 11 compensated. For combining the camera images 13 in which the object 22 is shown, it is necessary to take the camera pictures 13 to transform in their respective image coordinate system into a common image coordinate system. A control device 30th on the middle vehicle 11 performs such a combination of the camera images 13 to the merged camera image 15 by.

In 3rd are necessary steps for such a transformation of camera images 13 shown schematically. In a first step, object coordinates 23 of the object 22 which exist in a first image coordinate system with respect to a camera or in a vehicle coordinate system of a first vehicle (denoted by A) to a second vehicle (denoted by B referred to). In a second step, the second vehicle calculates 11 transformed object coordinates 24th in his vehicle coordinate system 14 based on the transmitted object coordinates 23 and a measured distance between the two vehicles 11 . In a third step, the transformed object coordinates 24th with object coordinates 25th of the object 22 compared which exist in a second image coordinate system with respect to a camera or in a vehicle coordinate system of the second vehicle.

In 4th is a neural network 40 shown. Information that is on a vehicle is used as input values 11 have been recorded. For this are position information 41 , image-based environment information 42 and short-range information 43 to the environment 20th intended. In neural networking with divergence, this information is shared across multiple processing nodes 44 of the neural network 40 passed on. In further neural networking with convergence, those in the processing nodes 44 processed information to an output node 45 passed on. The output node 45 contains data with which a collision between the vehicle 11 of the vehicle network 10th with the object 22 is predicted. This output value is then used to decide whether the vehicle is to be operated autonomously 11 or the vehicle network 10th maintained or interrupted.

In 5 is another neural network 40 shown. This neural network 40 is based on input values from different vehicles 11 . How to 4th position information is described 41 , Environment information 42 and short-range information 43 of the environment networked internally and across vehicles with divergence. We are also responsible for this 4th described respective in-vehicle processing nodes 44 in the neural network 40 intended. The vehicle's own processing nodes 44 are converging to an output node 45 neuronically networked. The output node 45 provides a cross-vehicle output value with which a collision of a vehicle 11 of the vehicle network 10th with the object 22 Is evaluated.

In 6 are procedural steps S1 , S2 , S3 , S4 in a time sequence for performing the method for autonomous operation of the vehicle group 10th shown.

In a first step S1 information is collected. As in 1 shown, the information acquisition of the environment takes place 20th with on the vehicles 11 arranged cameras 12 . In a second step S2 there is information communication. As in 2nd camera images are shown 13 across vehicles using means of communication 16 transfer. In a third step S3 there is an information transformation. As in the 2nd and 3rd camera images are shown 13 to a merged camera image 15 by means of transformation between camera coordinate systems or vehicle coordinate systems 14 to a merged camera image 15 combined. In a fourth step S4 a decision is made to maintain the autonomous operation of the vehicle network 10th . As in the 4th and 5 shown, this is done by means of a neural network 40 which provides a decision as to whether a collision between a vehicle 11 of the vehicle network 10th and an object 22 in the environment 20th of the vehicle network 10th threatens.

By providing such redundant information acquisition with cameras present in the vehicle network 12 and the use of artificial intelligence using the neural network 40 becomes an autonomous operation of the vehicle network 10th improved with the invention.

Reference symbol list

10th

Vehicle network

11

vehicle

12

camera

13

Camera image

14

Vehicle coordinate system

15

merged camera image

16

Means of communication

20th

environment

22

object

23

Object coordinates

24th

transformed object coordinates

25th

Object coordinates

30th

Control device

40

neural network

41

Position information

42

Environment information

43

Short-range information

44

Processing node

45

Output node

S1

Information gathering

S2

Information communication

S3

Information transformation

S4

Business continuity

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102014216257 A1 [0002]

Claims (11)

Method for the autonomous operation of a group of vehicles (10), with the steps to be carried out off-road: Detection (S1) of information with sensors in the vehicle group (10) to enable autonomous operation of the vehicle group (10) and Communicating (S2) the acquired information within the group of vehicles (10) in order to maintain the autonomous operation of the group of vehicles (10), the steps being carried out with a vehicle network (10) which has vehicles (11) which are designed for work away from roads. Procedure according to Claim 1 , with the steps of redundant detection (S1) of information with the sensors in the vehicle group (10) and at least partial communication (S2) of the redundantly recorded information within the vehicle group (10). Procedure according to Claim 1 or 2nd , with the steps of acquiring (S1) information about the location of an object (22) in the environment (20) of the vehicle group (10) with environmental detection sensors present in the vehicle group (10) and communicating (S2) the acquired information to avoid a collision of the vehicle group ( 10) with the object (22). Method according to one of the preceding claims, in which the step of capturing (S1) information comprises capturing camera images (13) of an environment (20) of the vehicle group (10) with cameras (12) present in the vehicle group (10) and the step of communicating (S2) the acquired information comprises communicating the captured camera images (13) within the vehicle group (10). Method according to one of the preceding claims, in which the step of capturing (S1) information comprises capturing position information of the vehicle group (10) with position detection sensor systems present in the vehicle group (10) and the step of communicating (S2) the acquired information comprises communicating the acquired information for navigating the group of vehicles (10). Method according to one of the preceding claims, in which the step of communicating (S2) recorded information within the group of vehicles (10) comprises cross-vehicle transmission of recorded information for merging information from different vehicles (11) within the group of vehicles (10). Method according to one of the preceding claims, with the step of transforming (S3) the communicated information into a vehicle-specific coordinate system (14). Method according to one of the preceding claims, with the step taking into account a quality parameter of the communicated information when making a decision about maintaining the autonomous operation of the vehicle group (10). Method according to one of the preceding claims, with the step of deciding (S4) on maintaining the autonomous operation of the vehicle network (10) based on a neural network (40) which is based on the detected and / or communicated information as input values. Control system with control devices (30) for vehicles (11) of a vehicle group (10), which is set up to carry out one of the methods according to one of the preceding claims, wherein the control devices (30) have at least one interface for reading information which is included in the vehicle group 10) existing sensors for enabling autonomous operation of the vehicle group (10) have been detected, and have interfaces for communicating the read information within the vehicle group (10) for maintaining the autonomous operation of the vehicle group (10). Sensor system for autonomous operation of a vehicle network (10) away from roads, which has vehicles (11) that are designed for work operations away from roads, with a sensor system that can be arranged on the vehicles (11) of the vehicle network (10) for detecting Information for enabling autonomous operation of the vehicle group (10), communication means (16) for communicating the acquired information within the vehicle group (10) for maintaining the autonomous operation of the vehicle group (10) and a control system according to Claim 10 .

Images