DE102018002521A1 - Method for recognizing persons - Google Patents

Abstract

The invention relates to a method for detecting persons (P1 to Pn) on the basis of sensor data (D) detected by means of at least one radar sensor (2 to 5), an environment being detected by means of at least one radar sensor (2 to 5). According to the invention, the sensor data (D) acquired by means of the radar sensor (2 to 5) are transformed into a spatio-temporal coordinate system (K). In the coordinate system (K), a spatiotemporal identification of persons (P1 to Pn) are performed and the persons (P1 to Pn) are assigned associated radar reflections (R1 to Rx) and the radar reflections (R1 to Rx) are segmented. Spatiotemporal features (M1 to Mz) are extracted, and based on the extracted features (M1 to Mz), at least biometric features and a state of movement of persons (P1 to Pn) are determined as person characteristics (E).

Description

The invention relates to a method for detecting persons on the basis of sensor data acquired by means of at least one radar sensor according to the preamble of claim 1.

• - Aussenden und Empfangen einer Radarwelle;
• - Ermitteln einer Relativgeschwindigkeit vom Körper und mindestens einer Extremität des Fußgängers aus der empfangenen Radarwelle;
• - Ermitteln einer Ausdehnung vom Körper und mindestens einer Extremität des Fußgängers aus der empfangenen Radarwelle;
• - Erkennen des Fußgängers aus den ermittelten Relativgeschwindigkeiten und den Ausdehnungen.

From the DE 10 2016 215 102 A1 For example, a method for recognizing a pedestrian is known. The method comprises the steps:

• Sending and receiving a radar wave;
• Determining a relative speed of the body and at least one extremity of the pedestrian from the received radar wave;
• Determining an extent of the body and at least one extremity of the pedestrian from the received radar wave;
• - Detecting the pedestrian from the determined relative speeds and expansions.

Furthermore, from the DE 44 04 775 C1 discloses a method of operating a hybrid neural network formed of a series connection of a segmenter, a feature extractor, and a classifier for automatic recognition of classed objects. Input images of the method are generated by means of a radar sensor.

The object of the invention is to provide a method for recognizing persons that is improved over the prior art by means of sensor data acquired by means of at least one radar sensor.

The object is achieved by a method having the features specified in claim 1.

Advantageous embodiments of the invention are the subject of the dependent claims.

In the method for detecting persons based on sensor data acquired by means of at least one radar sensor, an environment is detected by means of at least one radar sensor.

According to the invention, the sensor data detected by means of the radar sensor are transformed into a space-time coordinate system. In the coordinate system, a spatio-temporal identification of persons is carried out and the persons are associated with associated radar reflections and the radar reflections are segmented. Furthermore, spatio-temporal features are extracted, and based on the extracted features, at least biometric features and a state of motion of persons are determined as personal characteristics.

The method allows a particularly robust detection of persons, for example pedestrians. Due to the knowledge of biometric characteristics of the persons an improved estimation of a state of motion is possible. Other features may also be provided for recognizing or recognizing individual persons and detecting other objects and subjects that a person carries with him or who are in the vicinity of a target person.

A determination of a person's size, a distinction between children and adults, a distinction as to how far a recognized person for a current driving task a vehicle can be dangerous, as well as a classification of the person carried along items, such as a walker or a shopping cart, possible.

Furthermore, the method enables an improvement of classification algorithms as well as a detection of subjects and objects, a recognition of subjects, a support of other sensors in classification tasks and a motion prediction, a determination of an intention of a subject.

Also, a particularly robust classification of pedestrians by means of new expressive features, a recognition of children and thus a supply of semantic information for assistance systems of a vehicle is possible.

Further, the method allows assistance in the recognition and recognition of particular persons by biometric characteristics, such as a gait, recognition of a vehicle owner already from a great distance. For example, after recognition of persons, comfort functions of a vehicle, for example automatic setting of a seat position, starting of an infotainment system, activation of a heating, ventilation and / or vehicle lighting, can be activated, the vehicle can be opened and / or a trunk can be be opened, z. B. for additional detection of a shopping cart.

Furthermore, the method is suitable as a support for camera systems, since an independent and complementary sensor, i. H. the at least one radar sensor is used. Due to the use of the radar sensor is still given a high level of robustness by weather independence.

Embodiments of the invention are explained in more detail below with reference to drawings.

• 1A bis 1C schematisch einen Verfahrensablauf zum Erkennen von Personen und
• 2 schematisch räumliche Positionen gemessener Radarreflexionen eines Fußgängers.

Showing:

• 1A to 1C schematically a procedure for recognizing persons and
• 2 schematically spatial positions of measured radar reflections of a pedestrian.

Corresponding parts are provided in all figures with the same reference numerals.

In the 1A to 1C is a flow of a possible embodiment of a method for recognizing persons P1 to Pn in an environment of a vehicle 1 shown.

The vehicle 1 includes a plurality of radar sensors 2 to 5 , by means of which the environment of the vehicle 1 is detected. By means of such high-resolution radar sensors 2 to 5 speeds, positions and radar cross sections, also known as RCS value (radar cross section) of objects O1 to Om , including subjects, are measured.

For recognizing objects O1 to Om , especially people P1 to Pn , in a first method step S1 ( 1A above) the environment of the vehicle 1 by means of radar sensors 2 to 5 detected.

Subsequently, in a second method step S2 ( 1A below) a transformation of all sensor data acquired in the first method step S1 D in a spatio-temporal coordinate system K with space coordinates x . y and a time coordinate t ,

In a third method step S3 ( 1B ) becomes a spatiotemporal identification of all relevant objects O1 to Om , in particular of persons P1 to Pn , such as pedestrians or groups of pedestrians, performed in a captured scene and there is an association and segmentation of the objects O1 to Om associated radar reflections R1 to Rx ,

The lower picture in 1B shows a view from the vehicle 1 into the environment with the objects O1 to Om , wherein the object O1 is a person P1, in particular a pedestrian, the object O2 is a group of persons Pn- 1 , Pn, especially pedestrians, and the object Om a vehicle 1 is.

The middle picture in 1B shows the vehicle 1 in a plan view and all by means of the radar sensors 2 to 5 radar reflections detected in the environment R1 to Rx as a point cloud W as well as the assignment of these to the objects O1 to Om ,

The spatio-temporal identification of all relevant objects O1 to Om takes place in a first embodiment by generating high-dimensional feature vectors and / or probability vectors for each measured radar reflection R1 to Rx , The probability vector for each individual radar reflection R1 to Rx For example, you can specify the probability that the measured radar reflection R1 to Rx to an object class, such as pedestrians, cyclists, vehicle, etc., belongs. The generation of the feature vectors and / or probability vectors takes place in the illustrated embodiment according to the upper illustration in FIG 1B using deep neural networks N for example, recurrent networks N , Alternatively or additionally, other approaches to machine learning may be used.

Subsequently, the grouping of the radar reflections takes place based on the previously generated feature vectors and / or probability vectors R1 to Rx to objects O1 to Om , This results in a semantic segmentation of the entire space-time point cloud W or parts thereof. The segmentation and / or grouping can take place by means of so-called unsupervised, semi-supervised or supervised algorithms, for example so-called clustering algorithms.

Subsequently, according to a possible embodiment, a refinement step is performed which improves the generated segmentation. This step is done, for example, by means of deep neural networks N for example, recurrent networks N , alternatively or additionally implemented by other approaches of machine learning or classical algorithms.

The spatio-temporal identification of all relevant objects O1 to Om takes place in a second embodiment in that initially a grouping of the radar reflections R1 to Rx to object proposals, in particular by means of the so-called unsupervised, semi-supervised or supervised algorithms, for example, so-called clustering algorithms, and then the generation of feature vectors and / or probability vectors from the grouped reflections R1 to Rx using deep neural networks N for example, recurrent networks N , alternatively or additionally also by means of other approaches of machine learning. Here, too, a semantic segmentation of the entire spatiotemporal point cloud W or parts thereof. In particular, object proposals are assigned to an object class.

Subsequently, according to a possible embodiment, a refinement step is performed which improves the generated segmentation. This step is done, for example, by means of deep neural networks N for example, recurrent networks N , alternatively or additionally implemented by other approaches of machine learning or classical algorithms.

In a fourth method step S4 ( 1C top figure), an extraction of relevant spatiotemporal features takes place M1 to Mz for all objects predicted in the third method step S3 O1 to Om ,

• - RCS-Werte,
• - Histogramme,
• - Statistische Kenngrößen von Abständen, Winkeln, RCS-Werten und Geschwindigkeiten, wie beispielsweise ein Mittelwert und eine Standardabweichung,
• - Eigenwerte und Eigenvektoren von Kovarianzmatrizen, wie beispielsweise eine räumliche Kovarianz,
• - Merkmale von Faltungsoperationen,
• - Form und Größe von einhüllenden Kurven,
• - Anzahl von gemessenen Radarreflexionen R1 bis Rx,
• - raum-zeitliche Korrelationen von Abstand, Winkel, RCS-Werten und Geschwindigkeiten,
• - Abstand zwischen minimalem und maximalem Wert, beispielsweise RCS-Werten, Winkeln, Geschwindigkeiten, Abständen.

Such space-time characteristics M1 to Mz are for example

• - RCS values,
• - histograms,
• Statistical characteristics of distances, angles, RCS values and speeds, such as an average and a standard deviation,
• Eigenvalues and eigenvectors of covariance matrices, such as a spatial covariance,
• Features of convolution operations,
• - shape and size of enveloping curves,
• - Number of radar reflections measured R1 to Rx .
• - spatio-temporal correlations of distance, angle, RCS values and velocities,
• - Distance between minimum and maximum value, for example RCS values, angles, speeds, distances.

The extraction of the relevant spatiotemporal features M1 to Mz takes place on the basis of expert knowledge or about the learning of characteristics M1 to Mz by a used algorithm, for example by deep neural networks N ,

Under the extraction of features M1 to Mz In the present case, it is understood by expert knowledge that experienced scientists determine by their experience and experiments which features M1 to Mz important for a distinction in the respective task. For example, for a classification of an object proposal statistical variables such. B. a mean distance of all measured radar reflections R1 to Rx of the respective object O1 to Om , a standard deviation of self-motion compensated Doppler velocities or higher moments of measured quantities can be used. The scientist has determined in advance which quantities are important for the task.

In a fifth method step S5 ( 1C lower figure) takes place on the basis of the features extracted in the fourth method step S4 M1 to Mz a determination of personal characteristics e of registered persons P1 to Pn , where the personal characteristics e at least biometric features and a state of movement of the persons P1 to Pn include.

• - eine Schrittweite,
• - eine Körpergröße,
• - eine Gangart,
• - einen Altersbereich (Kind vs. Erwachsener),
• - eine Bekleidung.

The biometric features include, for example

• a step size,
• - a height,
• - a gait,
• - one age range (child vs. adult),
• - a clothing.

The state of motion is determined, for example, on the basis of a movement pattern and / or an average walking speed.

Furthermore, as a person properties e also entrained objects and / or subjects, such as bags, luggage, additional objects such as shopping and strollers, children and animals are recorded.

The method allows use of the obtained information for classification or tracking applications. Also, can be measured from the sensor data D even more information, especially the personal characteristics e to extract a more accurate description of persons P1 to Pn for example, by pedestrians. This information can be found in vehicle assistance systems 1 used, for example, a more accurate prediction of the movement of persons P1 to Pn to enable. Also, it is possible by a robust classification of gait as a biometric personality property e an owner of the vehicle 1 to recognize or the gait as an additional feature M1 to Mz to use for biometric recognition. Also may be a movement intention of a person P1 to Pn , in particular a pedestrian, determined and / or after the (re) recognition of a person P1 to Pn different actions are triggered.

2 shows spatial positions of measured radar reflections R1 to Rx a person trained as a pedestrian P1 to Pn with a associated, in particular color coded speed v the respective radar reflection R1 to Rx ,

LIST OF REFERENCE NUMBERS

1

vehicle

2 to 5

radar sensor

D

sensor data

e

personal property

K

coordinate system

M1 to Mz

feature

N

network

O1 to Om

object

P1 to Pn

person

R1 to Rx

radar reflectivity

S1 to S5

step

t

time coordinate

v

speed

W

point cloud

x, y

spatial coordinate

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102016215102 A1 [0002]
• DE 4404775 C1 [0003]

Claims (3)

Method for detecting persons (P1 to Pn) on the basis of sensor data (D) detected by at least one radar sensor (2 to 5), wherein an environment is detected by at least one radar sensor (2 to 5), characterized in that - by means of Radar sensor (2 to 5) detected sensor data (D) are transformed into a spatiotemporal coordinate system (K), - in the coordinate system (K) a spatiotemporal identification of persons (P1 to Pn) performed and the persons (P1 to Pn associated radar reflections (R1 to Rx) are assigned and the radar reflections (R1 to Rx) are segmented, - spatiotemporal features (M1 to Mz) are extracted and - based on the extracted features (M1 to Mz) as person characteristics (E) at least biometric Characteristics and a state of movement of persons (P1 to Pn) are determined. Method according to Claim 1 , characterized in that in the spatiotemporal identification of persons (P1 to Pn) - high-dimensional feature vectors and / or probability vectors for each measured radar reflection (R1 to Rx) are generated and - then in the segmentation of the radar reflections (R1 to Rx) based the feature vectors and / or probability vectors are grouped into persons (P1 to Pn). Method according to Claim 1 , characterized in that in the space-time identification of persons (P1 to Pn) - the radar reflections (R1 to Rx) are grouped by means of at least one unsupervised, semisupervised or supervised algorithm to object proposals and - then feature vectors and / or probability vectors from the grouped Radar reflections (R1 to Rx) are generated by means of deep neural networks (N) or by means of machine learning approaches.

Images