DE102019210933A1 - Environmental detection device and system for automated mobility - Google Patents

Abstract

Environmental detection device (10) for automated mobility comprising a housing (11), sensors (12, 13) arranged in the housing (11), the housing (11) comprising openings (14) for the sensors (12, 13) for detecting the Environment, an evaluation device (15) arranged in the housing (11) and connected to transmit signals to the sensors (12, 13), the evaluation device (15) comprising a working memory (16) for executing a computer program for classification, localization and / or tracking of objects (16) in the environment, the computer program comprising commands which cause the evaluation device (15) to receive the objects (16) and their respective state of motion from the signals from the sensors (12, 13) when the computer program is based on the Evaluation device (15) is running, and an interface (17) to provide the objects (16) and their state of motion for automated mobility. The invention also relates to a system comprising the environmental detection devices (10).

Description

The invention relates to an environmental detection device for automated mobility.

The EP 2 818 922 B1 discloses a traffic monitoring device. Such traffic monitoring devices, also known as PoliScan Speed, are in particular systems for speed monitoring.

• • nicht einsichtige Kreuzung muss mit autonomem Fahrzeug befahren werden (Stadtfahrten, Universitätsbereiche auf Campus);
• • nicht einzuschätzende Risiken müssen abgedeckt werden: Personenbeförderung in Situationen mit erhöhter Anzahl an Personen / alkoholisierten Personen, z.B. an Stadtfesten;
• • Sicherung der autonomen Fahrzeuge in Wohngebieten mit riskanten Spielstraßen;
• • nicht rational handelnde Fußgänger und Fahrradfahrer, z.B. Fußgängerüberweg Tokio, Fahrradkuriere in New York;
• • Baustellen: Verkehrsführung ist verändert und es befinden sich spezielle Hindernisse in der Fahrspur. Es können sich nicht bekannte Fahrzeuge, wie Bagger, und sonstige Baustellenfahrzeuge in der Fahrspur befinden. Bauarbeiter können nahe oder auf der Fahrspur stehen, in Bereichen, in denen normal 50-100km/h zulässig sind.

In automated mobility, obscurations and / or complex traffic situations can create risky danger areas in which not all endangered road users can be detected with 100% certainty by the sensor set of an autonomous vehicle. The underlying problem is that the environmental detection, especially of objects (static and dynamic), in the environment of an autonomous vehicle cannot guarantee a security of 100% in certain situations. These situations can be:

• • A non-transparent intersection must be driven through with an autonomous vehicle (city trips, university areas on campus);
• • Risks that cannot be assessed must be covered: Passenger transport in situations with an increased number of people / intoxicated people, eg at city festivals;
• • Securing autonomous vehicles in residential areas with risky play streets;
• • Pedestrians and cyclists who do not act rationally, eg Tokyo pedestrian crossing, bicycle couriers in New York;
• • Roadworks: Traffic routing has been changed and there are special obstacles in the lane. Unknown vehicles such as excavators and other construction site vehicles can be in the lane. Construction workers can stand near or in the lane, in areas where 50-100km / h is normally allowed.

Another problem is that for the full introduction of Level 5 vehicles, all other road users must first be integrated into the automated mobility / autonomous world.

Furthermore, the localization of vehicles is more difficult in difficult situations, including driving in urban canyons.

One object of the present invention was to provide a solution to the above-mentioned problems.

The environmental detection device according to the invention solves this task by means of sensor fusion and detection algorithms and the transmission of detected objects in the risky hazard areas to autonomous vehicles. The environmental detection device comprises a housing. Sensors are arranged in the housing. The housing includes openings for the sensors for detecting the environment. Furthermore, an evaluation device connected to transmit signals to the sensors is arranged in the housing. The evaluation device comprises a main memory for executing a computer program for the classification, localization and / or tracking of objects in the environment. The computer program comprises commands which have the effect that the evaluation device receives the objects and their respective state of movement from the signals from the sensors when the computer program is running on the evaluation device. In addition, the evaluation device comprises an interface in order to provide the objects and their state of motion for automated mobility.

In one aspect, the invention provides a system for automated mobility. The system comprises several environmental detection devices according to the invention and vehicles that can be operated automatically. The environmental detection devices are arranged on university, trade fair, airport, urban sites, at intersections, at crossings on roads, for example pedestrian crossings, in residential areas and / or on construction sites in order to provide the automatically operated vehicles with a list of objects including their state of movement, before the automatically operated vehicles reach the environmental detection devices. The environmental detection devices are arranged stationary or mobile. The vehicles that can be operated automatically are, for example, autonomous people movers or robot taxis.

Advantageous configurations of the invention emerge from the subclaims, the drawing and the description of preferred exemplary embodiments.

Automated mobility comprises the ecosystem of automated driving from partial automation, for example driving with driver assistance systems, to full automation, i.e. driverless driving. Participants in automated mobility are vehicles of all types, in particular road vehicles, passenger transport systems, for example robot taxis, passenger drones, automatic people movers, automatic transit networks, for example personal / group rapid transit systems, drivers, passengers, non-mobilized road users and elements of the infrastructure.

The housing of the environmental device protects the sensors, the evaluation device, the interfaces and other electronic components, for example for voltage supply, against the ingress of foreign bodies, dust and / or liquids. For example, the housing is encapsulated and sealed from the outside.

The evaluation device includes, for example, processors (CPUs, GPUs), ICs, ASCIs, FGPAs and other logic modules to execute the computer program. The hardware / micro-architecture of the evaluation device is designed for a computing power of several 100 Tera-OPS. For example, the evaluation device includes multi-core processors and hardware accelerators. The evaluation device also includes interfaces to the sensors, for example FPD-Link or GMSL-Link interfaces. The main memory is, for example, a RAM, SRAM, or DRAM memory.

The commands of the computer program are software code sections that are written, for example, in the programming language C ++, Java or Python.

For the classification, localization and / or tracking of objects, the computer program executes, for example, an algorithm from the field of machine learning, for example a trained artificial neural network. For example, object features are generated from raw data or individual recordings from the sensors. Different features are merged and tracked over time, that is, tracking. Furthermore, the commands include instructions for the evaluation device to transform the objects recognized by the sensors from the coordinate system of the respective sensor into the world coordinate system. As a result, the computer program understands and perceives a scene in the environment.

The objects and their state of movement are provided, for example, by means of object lists. For example, the Cartesian coordinates x, y, z, speed components v_x, v_y, v_z, components of the rotational speed ω_x, ω_y, ω_z, azimuth angle φ and elevation angle Θ are provided for each object in relation to a world coordinate system. The objects are divided into dynamic and static objects by means of the motion state provided.

According to a further aspect of the invention, the evaluation device comprises a first interface in order to provide the dynamic objects and their state of motion for automated mobility, and a second interface in order to accordingly provide the static objects.

According to one aspect of the invention, the interface is wired. According to a further aspect of the invention, the interface is cordless / wireless.

According to a further aspect of the invention, the environmental detection device comprises a combination of optical sensors, sound sensors and / or radar sensors. Image recordings of the environment are obtained by means of the optical sensors. Additionally or alternatively, noises from the environment, for example sirens from emergency vehicles, children's screams, bicycle bells or driving noises from other vehicles, are recorded by means of the sound sensors. This is particularly advantageous at urban intersections and / or in residential areas with play streets. The detection of noise is also advantageous in electrified automated mobility, which is quieter than automated mobility based on internal combustion engines. With the help of radar sensors, the objects in the environment can be detected even with poor visual visibility, for example when it is raining or snowing.

According to a further aspect of the invention, the environmental detection device comprises at least one camera sensor, lidar sensor, ultrasonic sensor and / or noise sensor. The camera sensor is, for example, a sensor of a mono or stereo camera or a light transit time sensor (time of flight). The lidar sensor is, for example, a 360 ° semiconductor laser scanner. The noise sensor is, for example, a microphone. With the camera sensor, the environment and the objects in the environment are recorded photographically. A stereo camera or a time-of-flight sensor provide depth information. The lidar sensor is characterized by its relatively good resolution.

According to a further aspect of the invention, the evaluation device is designed to merge the signals from the sensors and / or to check plausibility with one another. The fusion of the individual sensor data results in a more detailed and higher-quality recording of the environment compared to individual sensor data. During the development of the environmental detection device, it has surprisingly been found that a fusion of data from a 360 ° lidar scanner with camera-object detections delivers the best results. A more cost-effective variant is a sensor set consisting of ultrasonic sensors, radar sensors and camera sensors. As a result of the plausibility check, the environmental detection device has multiple redundancies.

According to a further aspect of the invention, the commands of the computer program cause the evaluation device to bundle the objects from images and / or point clouds of the sensors, to associate them, to track them over time and / or to track the images and / or process point clouds. For example, the images and / or point clouds are processed using sensor fusion methods, for example track-to-track fusion based on normalized covariance matrices. Tracks are tracked separately for each sensor. These tracks are merged into system tracks. This improves object recognition. A point cloud comprises spatial coordinates of points. 3D radar or lidar data are usually 3D point clouds.

According to a further aspect of the invention, the interface is a radio interface by means of which the environmental detection device communicates with vehicles that can be operated automatically and / or with an automated mobility infrastructure. For example, the interface is a V2X interface. According to a further aspect of the invention, the environmental device comprises a V2X control unit and the radio interface. The radio interface is designed for 5G radio technology, for example. For example, the environmental detection device locates an autonomous vehicle. The autonomous vehicle receives its own absolute position determined by the environmental detection device via the radio interface, and thus has ground truth information. The autonomous vehicle also has a V2X communication module to communicate with the environmental sensing device. Depending on the range of the V2X communication, the autonomous vehicle already has an object list of the objects present in risky danger areas (intersections, play streets, construction sites, etc.). The vehicles receive an object list of all moving and static objects in the environment from the environmental detection device. The V2X communication can take place, for example, via Bluetooth or WLAN.

According to a further aspect of the invention, the interface is a radio interface to a cloud-based memory. The range of communication between the environmental detection device and automated mobility is therefore unlimited.

According to a further aspect of the invention, the environmental detection device comprises a cooling device for guiding a cooling medium in the housing. The cooling medium is liquid, for example water, or a gas, for example air. The cooling device is, for example, a pump or a fan. According to one aspect of the invention, the cooling takes place with active and / or passive cooling methods. This increases the reliability of the sensors in hot weather, for example.

According to a further aspect of the invention, the environmental detection device comprises cleaning devices integrated in the environmental detection device for the sensors, the evaluation device and / or the interface. This allows dirt to be removed. Alternatively or additionally, soiling can be removed by service staff.

According to a further aspect of the invention, the environmental detection device comprises means for online calibration. For example, the sensors, sensor parameters, the evaluation device and / or the interface are calibrated online. Online calibration means that components and / or parameters of the environmental detection device are checked continuously, in particular during the operation of the environmental detection device. According to one aspect of the invention, the environmental detection device comprises an interface, for example a radio interface, to a calibration device which carries out the calibration.

According to a further aspect of the invention, the environmental detection device has the shape of a cylinder. For example, the environmental detection device is an environmental detection column in shape, geometry and structure similar to or corresponding to the traffic monitoring device as in FIG EP 2 818 922 B1 disclosed or similar or corresponding speed camera columns.

According to a further aspect of the invention, the environmental detection device is designed for stationary or mobile operation. For example, the environmental detection device can be arranged on a trailer.

According to a further aspect of the invention, the recordings of the respective environment are merged by several environmental recording devices in a cloud-based memory.

• 1: Ausführungsbeispiel einer erfindungsgemäßen Umwelterfassungsvorrichtung,
• 2: Ausführungsbeispiel eines erfindungsgemäßen Systems für automatisierte Mobilität

The invention is based on the in the figures

• 1 : Embodiment of an environmental detection device according to the invention,
• 2 : Embodiment of a system according to the invention for automated mobility

illustrated embodiments explained.

In the figures, the same reference symbols denote the same or functionally similar reference parts. In the figures, only the reference parts relevant to the respective understanding are given.

In the 1 environmental detection device shown 10 is an environmental column. In one housing 11 the environmental detection device 10 a sensor set is arranged to record the environment. The sensor set includes, for example, camera sensors 12th and lidar sensors 13 . The sensors 12th and 13 are behind openings 14th of the housing 11 arranged to capture the environment. An evaluation device 15th receives the signals from the sensors as input 12th and 13 . The evaluation device provides the output 15th according to the invention lists of detected objects 16 with their respective movement states ready. This provision takes place via a V2X interface 17th to autonomous vehicles 18th , please refer 2

The in 2 The illustrated system for automated mobility are, for example, four environmental detection devices 10 located at an urban intersection with several pedestrian crossings. The data of the environmental recording devices 10 are merged in a cloud based application. Via the interfaces 17th communicate the environmental recording devices 10 with the autonomous vehicles 18th . The autonomous vehicles 18th include V2X control units and corresponding interfaces to communicate with the environmental monitoring devices 10 to communicate and to receive the object lists from them. This gives the autonomous vehicles 18th of all moving and static objects in the environment 16 Information about the respective movement states of these objects 16 .

List of reference symbols

10

Environmental detection device

11

casing

12

sensor

13

sensor

14th

openings

15th

Evaluation device

16

object

17th

interface

18th

vehicle

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2818922 B1 [0002, 0027]

Claims (13)

Environmental detection device (10) for automated mobility comprising • a housing (11), • sensors (12, 13) arranged in the housing (11), the housing (11) comprising openings (14) for the sensors (12, 13) to detect the environment, • an evaluation device (15) arranged in the housing (11) and connected to transmit signals to the sensors (12, 13), • the evaluation device (15) comprising a working memory (16) for execution • a computer program for the classification, localization and / or tracking of objects (16) in the environment, • the computer program comprising commands which cause the evaluation device (15) to receive the objects (16) and their respective state of motion from the signals from the sensors (12, 13) when the computer program is running on the evaluation device (15), and • an interface (17) in order to provide the objects (16) and their state of motion for automated mobility. Environmental detection device (10) according to Claim 1 comprising a combination of optical sensors (12, 13), sound sensors and / or radar sensors. Environmental detection device (10) according to Claim 1 or 2 comprising at least one camera sensor (12), lidar sensor (13), ultrasonic sensor and / or noise sensor. Environmental detection device (10) according to one of the preceding claims, wherein the evaluation device (15) is designed to merge the signals of the sensors (12, 13) and / or to check plausibility with one another. Environmental detection device (10) according to one of the preceding claims, wherein the commands of the computer program cause the evaluation device (15) to bundle the objects (16) from images and / or point clouds of the sensors (12, 13), to associate them, to track them over time and / or to process the images and / or point clouds. Environmental detection device (10) according to one of the preceding claims, wherein the interface (17) is a radio interface by means of which the environmental detection device (10) communicates with automated vehicles (18) and / or an infrastructure for automated mobility. Environmental detection device (10) according to one of the preceding claims, wherein the interface (17) is a radio interface to a cloud-based storage device. Environmental detection device (10) according to one of the preceding claims, comprising a cooling device for guiding a cooling medium in the housing (11). Environmental detection device (10) according to one of the preceding claims, comprising cleaning devices integrated in the environmental detection device (10) for the sensors (12, 13), the evaluation device (15) and / or the interface (17). Environmental recording device (10) according to one of the preceding claims, wherein the environmental recording device (10) comprises means for online calibration. Environmental detection device (10) according to one of the preceding claims, wherein the environmental detection device (10) is in the form of a cylinder. Environmental detection device (10) according to one of the preceding claims, designed for stationary or mobile operation. System for automated mobility comprising a plurality of environmental detection devices (10) according to one of the preceding claims and vehicles (18) which can be operated automatically, the environmental detection devices (10) on university, trade fair, airport, urban sites, at intersections, at crossings on roads, are arranged in residential areas and / or at construction sites in order to provide the automatically operated vehicles (18) with a list of objects (16) including their state of motion before the automatically operated vehicles (18) reach the environmental detection devices (10).

Images