DE102022112332A1 - Method and detection device for detecting a person in an observation area and vehicle - Google Patents

Abstract

The invention relates to a method for detecting a person in an observation area using a detection device. The following is provided here: provision of a detection device (2) which has a first measuring device (3) with a thermal image sensor device, a second measuring device (4) with a lidar measuring device and an evaluation device which is connected to the first and second measuring devices (3, 4) is connected and set up to process measurement signals detected by the first and second measuring devices (3, 4) for detecting a person (1); Detecting thermal image signals for an observation area (6) by means of the thermal image sensor device, the thermal image signals representing two-dimensional thermal images; Detecting lidar measurement signals for the observation area (6) using the lidar measuring device; Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals with the aid of the evaluation device (5); Determining the presence of the person (1) in the observation area (6) by evaluating the fused measurement signals with the aid of the evaluation device (5) and providing signal data which indicate the presence of the person (1) in the observation area (6) at a data output of the evaluation device (5). Furthermore, a detection device (2) for detecting a person (1) in an observation area (6) and a vehicle (8) are created.

Description

The invention relates to a method and a detection device for detecting a person in an observation area and a vehicle.

background

Detecting or detecting a person in an observation area can be provided in connection with different applications, for example to avoid dangerous situations for the person. It can thus be provided that warning signals are issued to the person if their presence is detected in an observation area, i.e. a monitored spatial area. Alternatively or additionally, it can be provided that upon detecting the person in the observation area, a vehicle that moves into or within the observation area is controlled in such a way that a possible collision with the detected person is avoided.

Different sensor or measuring devices have been proposed to capture measurement signals that can provide information about an object in an area. There is a need to improve the accuracy and reliability of person detection.

Summary

The object of the invention is to provide a method and a detection device for improved detection of a person in an observation area and a vehicle with which the detection of one or more people in the observation area is made possible with increased accuracy and improved reliability.

To solve this, a method and a detection device for detecting a person in an observation area according to independent claims 1 and 14 are created. Furthermore, a vehicle according to independent claim 15 is provided. Refinements are the subject of dependent subclaims.

According to one aspect, a method for detecting a person in an observation area by means of a detection device is created, which has the following: providing a detection device which has a first measuring device with a thermal image sensor device, a second measuring device with a lidar measuring device and an evaluation device which is connected to the connected to the first and second measuring devices and set up to process measurement signals detected by the first and second measuring devices for detecting a person; Detecting thermal image signals for an observation area using the thermal image sensor device, the thermal image signals representing two-dimensional thermal images; Acquiring lidar measurement signals for the observation area using the lidar measuring device; Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals using the evaluation device; Determining the presence of a person in the observation area by evaluating the fused measurement signals with the aid of the evaluation device and providing signal data which indicate the presence of the person in the observation area at a data output of the evaluation device.

According to a further aspect, a detection device for detecting a person in an observation area is created, which has the following: a first measuring device with a thermal image sensor device, a second measuring device with a lidar measuring device and an evaluation device, which is connected and set up with the first and second measuring devices is to process measurement signals detected by the first and second measuring devices to detect a person. The detection device is set up to detect a person in the observation area for the following: detecting thermal image signals for an observation area by means of the thermal image sensor device, the thermal image signals representing two-dimensional thermal images; Acquiring lidar measurement signals for the observation area using the lidar measuring device; Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals using the evaluation device; Determining the presence of a person in the observation area by evaluating the fused measurement signals with the aid of the evaluation device and providing signal data which indicate the presence of the person in the observation area at a data output of the evaluation device.

Furthermore, a vehicle designed as a land, sea and/or aircraft is created with the detection device.

It is provided, on the one hand, to record thermal image signals using the first measuring device and, on the other hand, to record lidar measurement signals using the second measuring device for the spatial observation area, which is to be monitored with regard to the possible presence of one or more people. With the help of the evaluation device, the thermal image signals, which represent two-dimensional thermal image recordings for the observation area, and the lidar measurement signals are merged with one another, in particular in such a way that an association between the lidar measurement signal and a respective thermal image signal is carried out for at least some of the lidar measurement signals. By evaluating the measurement signals merged in this way, it is then determined whether at least one person is present in the observation area or not. When determining the presence of a person, signal data is generated and provided which indicates the presence of the people in the observation area. The signal data are provided by the evaluation device at a data output, so that they can either be called up or transmitted by the evaluation device, for example to a control device or another evaluation device. The signal data can then be evaluated there in order to optionally trigger one or more actions, which can include, for example, the issuance of a warning or information notice, in particular to the person.

The use of both the thermal image signals and the lidar measurement signals in the fused measurement signals supports determining the person's presence in the observation area with high accuracy and reliability.

When determining the fused measurement signals and the presence of the person in the observation area, the following can be provided: determining a three-dimensional point cloud for the lidar measurement signals, in which one or more lidar measurement signals are assigned to the points; Projecting the thermal image signals onto the three-dimensional point cloud for the lidar measurement signals to determine the merged measurement signals; and determining the presence of the person in the observation area by evaluating the thermal image signals projected onto the three-dimensional point cloud for the lidar measurement signals. As a result, the three-dimensional measurement signals from the lidar measurement are combined (fused) with the two-dimensional thermal images, which supports improved evaluation of the recorded measured values with greater reliability. By projecting or mapping the (two-dimensional) thermal image signals onto the three-dimensional point cloud, an association is made between the thermal image signals and the lidar measurement signals. For example, one or more points in the three-dimensional space of the point cloud are assigned to one or more pixels of the thermal images.

When determining the person's presence, a position in the observation area can be determined for the person. Based on the fused measurement signals, the position of the person in the observation area is determined, whereby a positioning of the person in the observation area can be determined.

When determining the position, a relative position of the person to the first and/or second measuring device can be determined. If the first and/or the second measuring device is recorded or arranged on an object, for example in a vehicle, the relative position is determined relative to the object.

When determining the position, a distance between the first and/or the second measuring device and the person can be determined. When the first and/or the second measuring device is arranged in a vehicle, the distance between the vehicle and the detected person can be determined.

When determining the position, a spatial position for the person in the observation area can be determined.

A movement trajectory in the observation area can be determined for the person. Determining a movement trajectory for the person in the observation area optionally enables an analysis of the dynamic behavior (movement) of the person in the observation area. From a movement trajectory for the person in the past, a determination of a movement trajectory for a possible current or future movement of the person in the observation area can optionally be carried out. Thus, the future movement trajectory can be determined as a continuation of the movement trajectory for the past, be it, for example, along a straight or a curved movement path.

When determining the merged measurement signals, the points of the three-dimensional point cloud can each be assigned a brightness value, which is derived from a thermal image signal assigned to the point by means of the projection. The points of the three-dimensional point cloud are assigned a brightness value from the thermal images, for example depending on the brightness of a pixel assigned to the respective point.

When determining the presence of the person, the following can also be provided in the observation area: carrying out a cluster formation, in which several adjacent points of the three-dimensional point cloud are merged into one or more clusters, for which it was determined that the several adjacent points belong to a single one with a predetermined probability Assign object nen are; and performing a cluster analysis in which the one or more clusters are examined for respective cluster properties. By forming the clusters and analyzing them, parts or areas of the three-dimensional point cloud can be analyzed with regard to their cluster properties, for example to determine the extent to which a cluster indicates the presence of the person or not. In cluster analysis, one or more of the following cluster properties can be examined or analyzed: average of the brightness values of points of the cluster; Distribution of brightness (temperature) across the cluster; Cluster size and highest/lowest point of a cluster (height above reference plane, e.g. ground). Based on the cluster analysis, one or more additional properties can be determined for the person, in particular whether the person is sitting, standing or walking and / or a distinction between the person and other living beings.

The detection of the person in the observation area can be carried out as a (purely) deterministic object recognition without artificial intelligence methods. Determining the presence of the person in the observation area by means of the measurement signal evaluation is carried out here solely using deterministic object recognition, which includes measurement signal fusion, but without the use of artificial intelligence methods.

The first measuring device and the second measuring device can be provided in a vehicle which is designed as a land, sea and/or aircraft, and a movement of the vehicle can be controlled by means of a control device of the vehicle, taking into account the detection of the person in the observation area . The control device of the vehicle can receive the signal data, which indicate the presence of the person in the observation area, via the data output of the evaluation device of the detection device, in order to then control the movement of the vehicle based on this. In the case of the presence of a person in the observation area and a possible collision along a planned movement trajectory of the vehicle, the vehicle can, for example, be braked or stopped. Alternatively, the control device can cause the detected person to bypass.

The vehicle can be controlled autonomously, and the autonomous control can be carried out by means of the control device taking into account the detection of the person in the observation area.

By means of the control device of the vehicle, a planned movement trajectory of the vehicle in the observation area and upon detecting the person in the observation area, a safety measure for moving the vehicle along the planned movement trajectory can be determined. The safety measure can, for example, indicate a probability of a possible collision between the vehicle and the person, for example when parking the vehicle if the person is detected as being in the parking area. For example, if the probability exceeds a predetermined threshold value, it can be provided that the control device of the vehicle initiates measures to avoid the collision, for example braking the vehicle or bringing it to a standstill. A warning signal can also be issued to the vehicle driver.

In the various embodiments, when evaluating the fused measurement signals, a filtering step can be provided in which measurement signals that are determined to (definitely) not belong to a person are deleted or removed. For example, in this way, points of the three-dimensional point cloud can be removed or deleted from the further evaluation process for which it was determined that the points cannot indicate a person with a predetermined certainty, for example because the assigned brightness values of the points lie outside a range for brightness values can potentially indicate the presence of a person.

In one example, a method for detecting a person in an observation area by means of an evaluation device can be provided, in which an evaluation device with one or more processors is provided, each of which is set up to process data. The following is provided in the evaluation device: receiving thermal image signals for an observation area, the thermal image signals being detected by means of a thermal image sensor device of a first measuring device for detecting a person in the observation area and representing two-dimensional thermal image recordings; Receiving lidar measurement signals for the observation area, the lidar measurement signals being detected by means of a lidar measurement device of a second measurement device for detecting the person in the observation area; Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals using the evaluation device; Determining the presence of a person in the observation area by evaluating the fused measurement signals and providing signal data that indicates the person's presence in the observation area Display observation area at a data output of the evaluation device. An evaluation device can be provided, which is set up to carry out the above-mentioned method for detecting the person in the observation area using software applications that run on the one or more processors.

The method for detecting the person can be carried out solely and finally on the basis of the measurement signals from the first and second measuring devices, without the need for further measurement information.

The configurations explained above in connection with the various configurations of the method for detecting the person in the observation area can be provided accordingly in connection with the detection device and / or the vehicle with the detection device.

Description of exemplary embodiments

• 1 eine schematische Darstellung einer Anordnung mit einer Detektionsvorrichtung zum Detektieren einer Person in einem Beobachtungsbereich;
• 2 eine schematische Darstellung für ein Verfahren zum Detektieren einer Person in einem Beobachtungbereich mittels der Detektionsvorrichtung und
• 3 eine schematische Darstellung für ein Verfahren zum Detektieren einer Person in einem Beobachtungsbereich mittels der Detektionsvorrichtung, wobei eine Clusterbildung und -analyse bei der Messsignalauswertung angewendet wird.

Further exemplary embodiments are explained below with reference to figures in a drawing. Show here:

• 1 a schematic representation of an arrangement with a detection device for detecting a person in an observation area;
• 2 a schematic representation of a method for detecting a person in an observation area using the detection device and
• 3 a schematic representation of a method for detecting a person in an observation area using the detection device, wherein cluster formation and analysis is used in the measurement signal evaluation.

1 shows a schematic representation of a method for detecting a person 1 by means of a detection device 2, which in the exemplary embodiment shown has a first measuring device 3 with a thermal image sensor device, a second measuring device 4 with a lidar measuring device and an evaluation device 5. The evaluation device 5 is functionally connected to the first and second measuring devices 3, 4, in particular in such a way that the evaluation device 5 can receive signal data from the first and second measuring devices 3, 4, which are recorded by the first and second measuring devices 3, 4 Display measurement signals for detecting person 1.

The first measuring device 3 is designed with a thermal image sensor device, which is set up to capture thermal images for an observation area 6. The thermal imaging sensor device, in particular a thermal imaging camera, receives infrared radiation and thereby captures a thermal image of objects, in this case the person 1 in the observation area 6. Typical thermal imaging cameras use a spectral range between approximately 3.5 to approximately 15 μm wavelength of the infrared radiation. Thermal image sensor devices are known as such in various embodiments.

The second measuring device 4 is formed with a lidar measuring device. Lidar measurement is an application of three-dimensional laser scanning. The lidar measuring device enables optical distance and speed measurement. Lidar measuring devices are known as such in various embodiments.

2 shows a schematic representation of a method for detecting the person 1 in the observation area 6 using the detection device 2.

While the first measuring device 3 captures two-dimensional thermal images for the observation area 6 (step 10), the second measuring device 4 makes it possible to capture three-dimensional distance or distance information using the lidar measuring device (step 20). The thermal image signals and lidar measurement signals obtained in this way are merged into fused measurement signals using the evaluation device 5 (step 30). It is provided here to map or project the two-dimensional thermal image recordings onto the three-dimensional point cloud of the lidar measurement signals in such a way that an association occurs between the pixels of the thermal image recordings and the lidar measurement signals. By evaluating the merged measurement signals in step 40, person 1 is determined to be present in observation area 6. Then, in step 50, signal data is provided in the evaluation device 5, which indicates the presence of the person 1 in the observation area 6.

The signal data can be output via a data output of the evaluation device 5, according to 1 for example to a control device 7, for example a control device of a vehicle 8, which can be designed as a land, sea and / or aircraft. The control device 7 can then control the movement of the vehicle 8 into the observation area or within it depending on the signal data (step 60).

It can also be provided to determine the person 1's position in the observation area 6. Such a position determination can be carried out, for example, with the evaluation device 5. As part of the position determination, for example, a relative position of the person 1 to the first and / or second measuring device 3, 4 can be determined. If the first and second measuring devices 3, 4 are arranged in the vehicle 8, a relative position of the person 1 to the vehicle 8 can be determined. Alternatively or additionally, a spatial position of the person 1 and/or a distance of the person 1 to the first and/or the second measuring device 3, 4 can be determined.

3 shows a schematic representation of a method for detecting the person 1 in the observation area 6, with cluster formation and analysis being applied after the measurement signal fusion in step 70. Incidentally, in 3 for the same features, the same reference numbers as in 2 used.

The cluster formation can provide for combining several adjacent points of the three-dimensional point cloud of the lidar measurement signals (with associated pixels of the thermal images) into one or more clusters, for which it was determined that the several adjacent points can be assigned to a single object with a predetermined probability. In a subsequent cluster analysis, one or more clusters are examined with regard to their respective cluster properties. By forming the clusters and analyzing them, parts or areas of the three-dimensional point cloud can be analyzed with regard to their cluster properties, for example to determine a measure or a probability to what extent a cluster indicates the presence of person 1 or not.

In cluster analysis, one or more of the following cluster properties can be examined or analyzed: average of the brightness values of points of the cluster; Distribution of brightness (temperature) across the cluster; Cluster size and highest/lowest point of a cluster (height above reference plane, e.g. ground). Based on the cluster analysis, one or more additional properties can be determined for the person 1, in particular whether the person is sitting, standing or walking and/or a distinction between the person and other living beings.

In one example, the evaluation device 5 can be designed as a real-time transfer system, which sends the lidar measurement signals and thermal image signals together to a data processing device 8, which is formed separately from the evaluation device 5 and has one or more processors for data processing. In this way, one or more steps of measurement signal evaluation and processing in the person recognition method can be carried out remotely from the evaluation device 5. Cloud computing can be used here.

The features disclosed in the above description, the claims and the drawing can be important both individually and in any combination for the realization of the various designs.

Claims (15)

Method for detecting a person in an observation area using a detection device, with: - Providing a detection device (2) which has a first measuring device (3) with a thermal image sensor device, a second measuring device (4) with a lidar measuring device and an evaluation device which is connected to the first and second measuring devices (3, 4) and is set up to process measurement signals detected by the first and second measuring devices (3, 4) to detect a person (1); - Acquiring thermal image signals for an observation area (6) by means of the thermal image sensor device, the thermal image signals representing two-dimensional thermal images; - Acquiring lidar measurement signals for the observation area (6) using the lidar measuring device; - Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals with the aid of the evaluation device (5); - Determining the presence of the person (1) in the observation area (6) by evaluating the merged measurement signals with the aid of the evaluation device (5) and - Providing signal data which indicates the presence of the person (1) in the observation area (6) at a data output of the evaluation device (5). Procedure according to Claim 1 , characterized in that when determining the fused measurement signals and the presence of the person (1) in the observation area (6), the following is provided: - Determination of a three-dimensional point cloud for the lidar measurement signals, in which one or more lidar measurement signals are assigned to the points are; - Projecting the thermal image signals onto the three-dimensional point cloud for the lidar measurement signals, to determine the merged measurement signals; and - determining the presence of the person (1) in the observation area (6) by evaluating the thermal image signals projected onto the three-dimensional point cloud for the lidar measurement signals. Procedure according to Claim 1 or 2 , characterized in that when determining the presence of the person (1), a position in the observation area (6) is determined for the person (1). Procedure according to Claim 3 , characterized in that when determining the position, a relative position of the person (1) to the first and / or second measuring device (3; 4) is determined. Procedure according to Claim 3 or 4 , characterized in that when determining the position, a distance between the first and / or the second measuring device (3; 4) and the person (1) is determined. Method according to at least one of the Claims 3 until 5 , characterized in that when determining the position, a spatial position for the person (1) in the observation area (6) is determined. Method according to at least one of the preceding claims, characterized in that a movement trajectory in the observation area (6) is determined for the person (1). Method according to at least one of the preceding claims, characterized in that when determining the fused measurement signals, the points of the three-dimensional point cloud are each assigned a brightness value which is derived from a thermal image signal assigned to the point by means of the projection. Method according to at least one of the preceding claims, characterized in that when determining the presence of the person (1) in the observation area (6), the following is further provided: - Execution of a cluster formation, in which several adjacent points of the three-dimensional point cloud are brought together to form one or more clusters for which it was determined that the several adjacent points can be assigned to a single object with a predetermined probability; and - Executing a cluster analysis in which the one or more clusters are examined with regard to respective cluster properties. Method according to at least one of the preceding claims, characterized in that the detection of the person (1) in the observation area (6) is carried out as a deterministic object recognition free of artificial intelligence methods. Method according to at least one of the preceding claims, characterized in that the first measuring device (3) and the second measuring device (4) are provided in a vehicle which is designed as a land, sea and / or aircraft, and a movement of the Vehicle is controlled by means of a control device of the vehicle, taking into account the detection of the person (1) in the observation area (6). Procedure according to Claim 11 , characterized in that the vehicle is controlled autonomously and the autonomous control is carried out by means of the control device taking into account the detection of the person (1) in the observation area (6). Procedure according to Claim 11 or 12 , characterized in that a planned movement trajectory of the vehicle in the observation area (6) is determined by means of the control device of the vehicle and, upon detecting the person (1) in the observation area (6), a safety measure for moving the vehicle along the planned movement trajectory is determined. Detection device (2) for detecting a person (1) in an observation area (6), with a first measuring device with a thermal image sensor device, a second measuring device (4) with a lidar measuring device and an evaluation device (5), which with the first and the second measuring device (3, 4) and is set up to process measurement signals detected by the first and second measuring devices (3, 4) for detecting a person (1), the detection device (2) for carrying out a method of at least one of the preceding Claims is set up for the following: - detecting thermal image signals for an observation area (6) by means of the thermal image sensor device, the thermal image signals representing two-dimensional thermal images; - Acquiring lidar measurement signals for the observation area (6) using the lidar measuring device; - Determining fused measurement signals by means of a measurement signal fusion for the thermal image signals and the lidar measurement signals with the aid of the evaluation device (5); - Determining the presence of the person (1) in the observation area (6) by evaluating the merged measurement signals with the aid of the evaluation device (5) and - Providing signal data which indicates the presence of the person (1) in the observation area (6) at a data output of the evaluation device. Vehicle (8), which is designed as a land, sea and / or aircraft, with a detection device (2). Claim 14 .

Images