WO2019048206A1

WO2019048206A1 - Interactive object detection during autonomous driving

Info

Publication number: WO2019048206A1
Application number: PCT/EP2018/072210
Authority: WO
Inventors: Navid Nourani-Vatani; Andrew Palmer
Original assignee: Siemens Mobility GmbH
Priority date: 2017-09-07
Filing date: 2018-08-16
Publication date: 2019-03-14
Also published as: EP3652722A1

Abstract

The description relates to a method for interactive object detection by a vehicle (20). The method involves a surrounding area of a vehicle (20) being captured by sensor using a sensor unit (11) of the vehicle (20). The sensor signals (BD) of the sensor unit (11) are taken as a basis for ascertaining and classifying the object (0). The vehicle (20) then specifically physically influences the object (0) from the distance from the object (0) and a response of the object (0) is detected by the vehicle (20). Next, the classification of the object (0) is adjusted on the basis of the detected response, with mechanical properties of the object, concerning the question of whether or not the object can be pushed aside, being ascertained. Moreover, an object detection apparatus (10) is described. Additionally, a vehicle (20) is described.

Description

description

Interactive object recognition during autonomous driving

The invention relates to a method for interactive object recognition for a vehicle. Furthermore, the invention relates to an object recognition device. Moreover, the invention relates to a vehicle.

An important challenge in automated or even autonomous driving is avoiding immobile and mobile obstacles in the environment or in the area of the route. Autonomous vehicles and some advanced driver assistance systems (ADAS = Advanced Driver Assistance

Systems) perform classifications on the type of a detected object. For example, it is determined whether a detected object is a stationary or a moving object. On this basis, the behavior of the object can be predicted in the future, thus possibly giving a warning and / or avoiding possible collisions with these objects.

Conventionally used classification methods can be divided into passive and active classification methods ^¬ to. In passive classification, objects are classified based on a single or multiple observations, and the choice of viewing perspectives of the objects is not affected by the classification algorithm. In active classification future viewing perspectives are selected based on a classification ^¬ profit to improve the classification of the best possible way. That is, the sensor is moved so that the information gain is optimized for the purpose of classification. However, there are still object types and classifi ^¬ zierungsaufgaben are not sufficiently suitable for active and passive CLASSIFICA ^¬ insurance types. An example of this is a pedestrian who crosses the path of an automated vehicle. The vehicle must determine what type of object is approaching the driveway and whether the object is likely to move in front of the vehicle. To clarify these questions, one can benefit from other observations. With the active classification it is possible to adapt or improve the type classification. That is, in this case, it can be more easily determined that the detected object is a pedestrian. However, neither active nor passive classifications can be used to determine if the pedestrian has recognized the vehicle. The pedestrian moves example ei ^¬ nem walkway toward driveway, so perhaps there is little time to react when the pedestrian does not stop in time at the edge of the walkway.

DE 10 2014 111 023 A1 describes a method for controlling an automated vehicle, wherein a planned interaction with a road user is selected in an ambiguous traffic situation, the interaction is signaled to the road user, a reaction of the road user to the interaction is detected and a maneuver is initiated in response to the reaction of the road user.

US 2015/0 336 502 A1 describes a method for an autonomous vehicle, with which it is possible to communicate with external observers. In the process, a planned maneuver is displayed in advance.

DE 10 2014 226 187 A1 describes a control unit for a vehicle which is set up to

Receive environment data from one or more environmental sensors of the vehicle, and on the basis of the environment data at least one road users in an environment of the vehicle to detect. The control unit is configured to cause an air ejection device of the vehicle, a To generate a puff of air to alert the road user to the vehicle.

In DE 10 2013 216 490 AI, its environment is sensor-monitored by an automated controlled vehicle. Objects in the environment are detected. Based on the object recognition is determined whether a contact with the erkann ^¬ th object is possible. If successful, a signal with information about how the object should behave is then output.

EP 1 842 732 A2 describes a movable bumper of a vehicle which, with the aid of an actuator, can change from a retracted position in parking mode to a retracted position while driving.

There is therefore the task of object recognition to reali ^¬ Sieren, with the behavior of objects can be estimated more accurately.

This object is achieved by a method for interactive object recognition according to claim 1, an object recognition device according to claim 9 and a vehicle according to claim 10.

In the method according to the invention for the interactive object recognition by a vehicle, a surrounding area of a vehicle is detected senso ^¬ riell using a sensor unit of the vehicle. A vehicle may be, for example, a motor vehicle or a rail vehicle. The sensor unit may, for example, an image pickup unit, preferably a Ka ^¬ ra. On the basis of the sensor signals of Sensorein ^¬ standardize the object is determined and classified. The sensor ^¬ signals include information about the environment of the vehicle. For example, the information may have image data on ^¬. Thereupon the vehicle acts mechanically on the detected object from the distance and detects a response of the object. Furthermore, an adaptation of the class identification of the object based on the detected response, whereby mechanical properties of the object relating to the question of whether the object can be pushed aside or not are determined. Preferably, the described method steps carried out automatically, so that the classification of the object and to adapt in real time and without Perso ^¬ nalaufwand can be performed. Advantageously, the classification can be refined on the basis of the response of the object or extended by additional classifying parameters or parameters. In this way can be adapted specifically to a detected object to an individual situation, the response of the vehicle, such as an automobile or a shift ^¬ nenfahrzeug, so that can be avoided unnecessary disturbances and interruptions and evasive maneuvers. A refined classi ^¬ fication is advantageously accomplished interactively, which includes information about the future Ver ^¬ hold the object, in particular regarding a possible collision with the vehicle is. Advantageously, properties of objects can be taken into account, which are not recognizable solely by observation or corresponding image recordings.

The object recognition device according to the invention comprises a sensor unit for imaging a surrounding area of a vehicle. The Objekterkennungsvor ^¬ device according to the invention also has a Vorklassifizierungseinheit for au ^¬ tomatisierten classifying an object on the basis of sensor signals of the sensor unit. Part of the object recognition device according to the invention is also an actuator unit for the targeted mechanical action of the vehicle on the object from the distance to the object. The Objekter ^¬ recognition device according to the invention also comprises a detection unit for de- tektieren a response of the object by the vehicle. A Nachklassifizierungseinheit, which is also part of the ER inventive object detection device is used to adjust the classification of the object based on the detected response, mechanical properties of the Ob ^¬ jekts, concerning the question of whether the object beiseite- be pushed or not, are determined. The object recognition device ^according to the invention shares the advantages of the method ^according to the invention for the interactive object recognition by a vehicle.

The vehicle of the invention has the inventive Whether ^¬ jekterkennungsvorrichtung. The vehicle according to the invention shares the advantages of the object recognition device according to the invention.

Some components of the object recognition device according to the invention may possibly be designed after supplementing additional hardware components, such as the actuator unit, the Sen ^¬ soreinheit and the detection unit, for the most part in the form of software components. This applies in particular to parts of the pre-classification unit and the post-classification unit. In principle, however, these components can also be partly realized, in particular in the case of particularly fast calculations, in the form of software-supported hardware, for example FPGAs or the like. Likewise, the required interfaces, for example, if it is only about a transfer of data from other software components, be designed as software interfaces. However, they can also be designed as hardware-based interfaces that are controlled by suitable software.

A largely software implementation has the advantage that existing computing systems may have been in a vehicle easily nachge by a software update will be ^¬ upgraded to work in the inventive manner. By vehicle additional hard ^¬ ware elements, such as sensors, actuators or the like may need to be supplemented, so that the vehicle can operate in the erfindungsge- Permitted way. In this respect, the object is also achieved by a corresponding computer program product with a computer program which can be loaded directly into a memory device of such a computer system, with a program. program sections to perform all steps of the inventive method when the computer program is executed in the computer system. Such a computer program ^product , in addition to the computer ^¬ program optionally additional components, such as documentation and / or additional components, including hardware components, such as hardware keys (dongles, etc.) for using the software.

For transport to the storage device of the computer system and / or for storage on the computer system, a computer-readable medium, for example a memory stick, a hard disk or other transportable or permanently installed data carrier can be used, on which the program sections of the computer program which can be read and executed by a computer unit are stored are. The computer unit may be for example a purpose or more cooperating micropro ^¬ processors or the like.

The dependent claims and the following description each contain particularly advantageous embodiments and further developments of the invention. In this case, in particular, the claims of one category of claims can also be developed analogously to the dependent claims of another category of claims and their descriptive parts. In addition, in the context of the invention, the various features of different embodiments and claims can also be combined to form new embodiments.

In one embodiment of the method according to the invention for interactive object recognition by a vehicle, the targeted physical action has one of the following modes of action:

- emitting light rays,

Sending out acoustic signals

mechanical action on the object by direct contact of the vehicle with the object. The light beams may include, for example, light signals. By means of the signals mentioned, a reaction of the objects to be classified can be brought about remotely, without them coming into direct contact with the vehicle, which increases the safety for the objects as well as the vehicle. The mechanical action can be done by direct contact of the vehicle with the object. Here, mechanical egg ^¬ properties of the object, for example on the issue of whether the object can push aside or not, ermit ^¬ telt.

In a variant of the method according to the invention for interactive object recognition by a vehicle, the emission of light beams comprises one of the following operations:

Using warning lights of the vehicle,

Using front lights of the vehicle.

Advantageously, existing illumination devices of the vehicle can be used for signaling anyway, so that additional expenditure for the assembly of additional signal units can be avoided.

In one embodiment of the method according to the invention for interactive object recognition by a vehicle, the emission of acoustic signals comprises one of the following operations:

send out an acoustic warning signal,

send out acoustic reverse signal.

Advantageously, even in the transmission of acoustic signals to existing signal generator units zurückge ^{¬ be} attacked, whereby the Umrüstungsaufwand for the realization of the method according to the invention can be reduced.

In a variant of the method according to the invention for interactive object recognition by a vehicle, in the event ₀

O

that the answer of the object includes one of the following actions:

a change in the movement of the object or the object changes its movement behavior or its direction of movement, changing the viewing direction of an object associated with the object

Person,

a person associated with the object classifi ^¬ sheet as inattentive, and, in the case that the object associated with the person looks up and / or to look up that person associated with the object classified as attention. Advantageously, the behavior of people on the road in response to the stimulus emitted by the vehicle can be used directly to estimate the future behavior of the detected object.

In one embodiment of the inventive method for interactive object recognition by a vehicle, in the event that the object is a vehicle, in the event that the object remains, the object classified as parking,

and in the event that the object starts and / or a driver assigned to the object looks towards the vehicle and / or into a rear-view mirror, the object is classified as being moving.

Advantageously, the immediately following behavior of vehicles can be determined and included in the classification of the vehicle. In a variant of the method for interactive object recognition by a vehicle is moved up to the vehicle to the object to determine whether the Whether ^¬ ject can be pushed to the side or can be safely run over to fol- a predetermined route further to the . Are objects in the path of the vehicle, it may be useful to test whether the object by means of the driving ^¬ zeugs from the track is removable. Advantageously, the journey of a vehicle can be continued in case of success when an obstacle initially stands directly on the track of the vehicle.

The invention will be explained in more detail below with reference to the beige ^¬ added figures using ^exemplary embodiments. There show: a flow chart illustrating a method for interac tive object recognition by a vehicle according to an embodiment of the invention, a schematic representation of a Obj ekterkennungs apparatus according to an exemplary embodiment of the invention ^¬,

a schematic representation of a vehicle according to an embodiment of the invention.

1 shows a flow chart 100, with which a method for interactive object recognition by a vehicle according to an exemplary embodiment of the invention is illustrated. In step 1.1, first image data BD is taken from the surroundings of the vehicle with the aid of an on-board camera of the vehicle. Subsequently, in the case of step l.II, objects 0 are detected in the image data BD and image sections BAO with the detected objects 0 are determined. Furthermore, carried out at the step l.III an automatic ^¬ catalyzed classification of the detected objects 0, with first classification data is generated KD1 which 0 assign each of the detected objects in a class classification. For example, a detected object 0 is assigned to the class "pedestrian".

The mixture is then stimulates the detected object, for example by a light ^¬ signal ST is emitted toward the object 0 in step 1. IV. The Whether ^¬ ject 0, in this case, a pedestrian, replies with a response A to the light signal ST, for example, by looking in the direction of the vehicle or stops on a curb on the path of the vehicle. The reaction A is at the Step IV by means of sensors, such as a Ka ^¬ mera, detected by the vehicle and there are response data AD, in this case image data, based on the reaction A determined. For example, the response data AD includes the information that the pedestrian is looking toward the vehicle. Occurs Nachfol ^¬ quietly at step l.VI on the basis of the answer data AD, a Nachklassifizierung 0 of the detected object to generate the second classification data KD2. For example, the object is classified into the class "attentive pedestrian", so that a control decision can be made by the vehicle on the basis of this second CLASSIFICA ^¬ approximate data KD2, namely that the pedestrian is not compromised, and a collision of the pedestrian with the driving ^¬ convincing can be excluded.

In Figure 2, an object detection device 10 according to an embodiment of the invention is shown schematically. The object recognition device 10 may be part of a vehicle 20, for example, as shown in FIG. The object detection apparatus 10 shown in FIG 2 includes a sensor unit 11 which is adapted to detect a Conversely ^¬ exercise area of the vehicle 20 pictorially. The sensor data acquired by the sensor unit 11, image data BD in this exemplary embodiment, are transmitted to an image evaluation unit 12. The image evaluation unit 12 detek- advantage in the image data BD objects 0 which are located in the region of the vehicle environment Conversely ^¬ 20th Image sections BAO with detected objects 0 are transmitted to a Vorklassifizierungseinheit 13 that divides based on these selective oriented image sections BAO the detected objects in Klas ^¬ sen and first classification data KD1 which the image data BAO to the detected objects 0 and which comprise these classes assigned , generated. The classification data KD1 are transmitted to a control unit 14, which activates an actuator unit 15, which then emits a stimulus ST in the direction of the object 0 (see FIG. 3). As mentioned earlier, it may be to act as a light signal. The object 0, beispielswei ^¬ se a pedestrian, the responses to the stimulus ST with a response A. For example, the pedestrian directs his gaze 0 in the direction of the vehicle 20 and to indicate that he has noticed the vehicle 20th The response A of the object 0 is detected by a detection unit 16 and transmitted to a post-classification unit 17. The Nachklassifi ^¬ zierungseinheit 17 adjusts the classification of the detected object to the detected 0 response data AD. For example, the classification changes from "pedestrian" to "attentive pedestrian". The adapted classification data KD2 are then transmitted to a control device 21 (see FIG. 3) of the vehicle 20, which can automatically issue control commands on the basis of this more precise classification.

In FIG 3, a scenario 30 with a vehicle 20 according to egg ^¬ nem embodiment of the invention is shown schematically. The vehicle 20 includes an obj ekterkennungsvorrich- device 10 according to an embodiment of the invention, as has already been illustrated in FIG 2. The vehicle 20 travels on a path W at the periphery of which an object 0, in this case a pedestrian, is located. With a camera 11, the vehicle 20 detects the pedestrian 0 and sends by means of an actuator 15, in this case a signal unit, a light signal ST in the direction of the pedestrian 0. The pedestrian 0 then looks attentively in the direction of the vehicle 20, thereby signaling that he noticed the vehicle 20. This response A is detected with the aid of a detection unit 16 of the vehicle 20 and then in the

Object detection device evaluated in the manner described in connection with FIG 1 and FIG 2 way. The adapted classification data KD2 generated in this case are subsequently sent to a control device 21 of the vehicle 20, which carries out a control process on the basis of the adapted classification. For example, the Be ^¬ fails given that the vehicle 20 continues at undiminished speed, since the pedestrian 0, the vehicle 20th recognized in time. In the event that the pedestrian 0 has been classified as inattentive, the control ^¬ device 21 initiates a braking maneuver to avoid endangering the pedestrian 0.

It is finally pointed out once again that this han ^¬ punched in the above-described methods and devices by only preferred embodiments of the invention and that the invention can be varied by the skilled person without departing from the scope of the invention in so far as determined by the claims is. It is the Fully ^¬ ness sake also noted that the use of the indefinite article "a" and does not exclude "a" that the features in question can also be present more than once. Similarly, the term "unit" does not exclude that it consists of several components, which may also be spatially distributed.

Claims

claims

A method of interactive object recognition by a vehicle (20), comprising the steps of:

- sensorial detecting a surrounding region of a driving ^¬ tool (20) by means of a sensor unit (11) of the driving ^¬ zeugs (20)

Determining an object (0) on the basis of sensor signals (BD) of the sensor unit (11), whereby the object (0) is classified,

targeted mechanical action of the vehicle (20) on the object (0) from the distance to the object,

Detecting a response (A) of the object (0) by the vehicle (20),

- Adapting the classification of the object (0) on the basis of the detected response (A), wherein mechanical properties of the object, concerning the question of whether the object can be pushed sideways ^¬ or not, are determined.

2. The method of claim 1, wherein the targeted physi cal ^¬ action comprises one of the following modes of action:

Emitting light rays,

Emitting acoustic signals,

mechanical action on the object (0).

3. The method of claim 2, wherein the emitting of light rays comprises one of the following operations:

Using warning lights,

Using front lights.

4. The method of claim 2, wherein the emitting of acoustic signals comprises one of the following operations:

send out an acoustic warning signal,

send out acoustic reverse signal.

5. The method of claim 2, wherein the mechanical action on the object (0) comprises one of the following operations:

Controlling an on-board mechanical manipulator, Interacting with the entire vehicle (20) on the object (0).

6. The method according to any one of claims 1 to 5, wherein in the event that the response of the object (0) comprises one of the following Ak ^¬ functions:

Changing the movement of the object (0),

Change the viewing direction of a person assigned to the object (0),

an object (0) associated with the person as inattentive is sified classified ^¬, and for the case that the object (0) looks up to ^¬ parent person and / or by looking up which classifies the object (0) associated with the person as attentive becomes .

7. The method according to any one of claims 1 to 6, wherein in the event that it is a vehicle delt with the object (0) han ^¬,

in the event that the object (0) stops, the object (0) is classified as parked,

and associated driver looks for the case that the object (0) to start and / or the object (0) in the direction of the vehicle (20) and / or in a rear view mirror, the Whether ^¬ ject (0) as will be classified moving ,

8. The method according to any one of claims 5 to 7, wherein the vehicle (20) is driven onto the object (0) in order to ^¬ hey, whether the object (0) can be pushed to the side or can safely be driven over, to continue following a predetermined route.

An object recognition device (10) comprising:

a sensor unit (11) for detecting a visual order ^¬ gebungsbereichs of a vehicle (20),

- a Vorklassifizierungseinheit (13) for automatically classifying an object (0) on the basis of Sensorsig ^¬ dimensional (BD) of the sensor unit (11), an actuator unit (15) for targeted mechanical action of the vehicle (20) from the distance to the object on the obj ect (0),

a detection unit (16) for detecting a response of the object (0) by the vehicle (20),

a Nachklassifizierungseinheit (17) to adjust the classification of the object (0) based on the detected response, wherein mechanical properties of the object, be ^¬ aptly be determined whether the object can be set aside or not.

10. Vehicle (20), comprising an object recognition ^device (20) according to one of the preceding claims.

11. A computer program product with a computer program, wel ^¬ ches directly in a storage unit of a computer unit ei ^¬ nes vehicle (20) is loadable, with program sections to perform all steps of a method according to any one of claims 1 to 8 when the computer program in the Rechnerein- is executed.

12. A computer-readable medium on which are stored by a computer unit executable program sections to perform all the steps of the method according to one of claims 1 to 8, when the program sections are executed by the computer unit.