DE102024000597A1 - Obtaining environmental data from a vehicle - Google Patents

Abstract

Environmental data of a vehicle (1) should be easier to obtain. For this purpose, a system and a method for obtaining environmental data of an environment of a vehicle (1) by letting a flying object (3) fly in the environment of the vehicle and recording the environmental data by the flying object (3) are proposed. The flying object (3) has no propulsion. It is launched by a launch device (2) of the vehicle (1) and caught again by a collecting device (5) of the vehicle (1).

Description

The present invention relates to a method for obtaining environmental data of a vehicle's surroundings by flying a flying object in the surroundings of the vehicle and capturing the environmental data by the flying object. In addition, the present invention relates to controlling a vehicle using such environmental data and to a system for obtaining the environmental data.

Complex traffic situations pose a particular challenge, especially when it comes to autonomous or automated vehicle control. Such traffic situations, such as congested intersections, congested roundabouts and the like, require a robust and high-tech system for environmental perception. Depending on the situation, it is sometimes even impossible to detect all relevant vehicles in the vicinity of the automated vehicle, as these are typically obscured by other vehicles, vegetation or house facades.

In a complex traffic situation, it is possible that a so-called "deadlock" may even occur, i.e. a situation in which the automated vehicle can no longer resolve the situation and further navigation is prevented. In such a case, the automated vehicle can even completely prevent further traffic flow.

There are many current approaches that use a drone (UAV: Unmanned Area Vehicles) to record a traffic situation from above. The camera data is then used to resolve the traffic situation. These drones can even be mounted in or on the vehicle and only be started when needed.

For example, the publication EN 10 2022 002 107 A1 a device that includes a vehicle and a drone associated with the vehicle. The device is configured to position at least one warning device in the form of a warning triangle at a predetermined distance from the vehicle in the event of an unplanned breakdown of the vehicle. The drone also has cameras for taking pictures and/or videos. By segmenting the pictures and/or videos, suggestions for detours can be calculated.

The publication also describes EN 10 2017 122 261 A1 a method for monitoring an area surrounding a motor vehicle, in which image data from the area surrounding the vehicle captured by at least one camera of a camera system are received by an evaluation device of the motor vehicle. A blocking object in the area surrounding the vehicle can be detected using the image data.

The object of the present invention is to obtain additional data from the environment of a vehicle in the simplest possible way.

According to the invention, this object is achieved by a method and a system according to the independent claims. Advantageous developments of the invention emerge from the subclaims.

According to the present invention, a method is therefore provided for obtaining environmental data from the environment of a vehicle. The environment of the vehicle largely determines its future navigation. The environmental data from this environment reflects, for example, the current traffic situation. The environmental data can be obtained in different physical ways. For example, they can be based on optical or acoustic signals. For example, they can be based on camera data and/or sound data. Alternatively or additionally, other electromagnetic rays can also be evaluated for the environmental data, such as radar signals.

In one step of the method, a flying object is allowed to fly in the vicinity of the vehicle. The flying object is located outside the vehicle in its immediate vicinity. For example, the flying object is at a distance of up to 20 meters or up to 50 meters from the vehicle during a flight. However, these distance data are to be understood purely as examples. For example, the flying object flies unattached or tethered to the vehicle in the vehicle's vicinity.

The method also involves the flying object capturing the environmental data. In particular, the flying object captures the environmental data during a flight in the vicinity of the vehicle. If necessary, the environmental data can be captured by the flying object before and/or after the flight.

The flying object is without propulsion. This means that it is passive in terms of its own propulsion. The flying object therefore receives the energy necessary for flight from outside, in particular from the vehicle. The flying object therefore has no own Propulsion unit to enable flight. It differs from a drone, for example, which has an electric motor and propeller for self-propulsion.

The flying object is fired from a launching device on the vehicle. It therefore receives the energy required for flight from the vehicle's launching device. For example, the launching device comprises a spring that is initially tensioned and then relaxed for launching. The spring can be tensioned using an electric motor, for example. The launching device can also have any other elastic elements. Alternatively or additionally, the launching device can also generate the necessary launching energy using compressed air, electrical forces, magnetic forces or explosions.

The flying object is finally caught again by a catching device on the vehicle. At the end of a flight, the catching device catches the flying object again so that it is available to the vehicle again. The catching device can be designed in different ways. For example, it can be purely mechanical, such as a landing net. It may have actively movable components, such as a gripper or an opening mechanism like an umbrella. In some circumstances, electrical or magnetic forces as well as suction forces can also be used to catch the flying object with the catching device.

The unpowered flying object makes it easy to obtain environmental data from the vehicle's surroundings. The flying object itself can therefore be designed very simply. Simple and well-known techniques can be used to shoot down and capture the flying object.

In one embodiment, it is provided that the environmental data is transmitted from the flying object to the vehicle, in particular during a flight. The flying object and the vehicle therefore have corresponding data transmission interfaces. The data transmission is preferably wireless. If necessary, the flying object is also guided to a galvanic interface (e.g. plug) after landing in order to transmit the environmental data to the vehicle. However, it is particularly advantageous for data to be transmitted during the flight, so that the data is available more quickly and in a more up-to-date manner. The data transmission can take place via suitable radio interfaces. For example, a WLAN or Bluetooth protocol can be used for the data transmission.

In a further embodiment, the environmental data is analyzed in relation to a traffic situation. For example, the environmental data reflects one or more vehicles, for example from a bird's eye view. These vehicles are in a certain relationship to one another or to any traffic routes that corresponds to the traffic situation. The current traffic situation can therefore be deduced from the environmental data. This traffic situation can in turn be used advantageously to control the vehicle or other vehicles.

According to the invention, a method for controlling a vehicle can also be provided, wherein the environmental data of the vehicle is obtained using a method described above and used for automatic or semi-automatic control of the vehicle. Additional environmental data of a vehicle can therefore be obtained using a simple, unpowered flying object in order to control it more reliably, for example. This can be used advantageously, for example, to clear a traffic jam. In addition, the additional use of the environmental data of the vehicle can also be used advantageously to provide driving lanes for rescue workers, for example.

• - das Fahrzeug und
• - ein Flugobjekt, mit dem die Umgebungsdaten erfassbar sind, wobei
• - das Flugobjekt antriebslos ist,
• - das Fahrzeug eine Abschussvorrichtung aufweist, die ausgebildet ist, das Flugobjekt abzuschießen, und
• - das Fahrzeug eine Auffangvorrichtung aufweist, die ausgebildet ist, das Flugobjekt aufzufangen.

According to the invention, the above-mentioned object is also achieved by a system for obtaining environmental data of an environment of an (automated) vehicle, the system comprising:

• - the vehicle and
• - a flying object with which the environmental data can be recorded, whereby
• - the flying object has no propulsion,
• - the vehicle has a launching device designed to launch the flying object, and
• - the vehicle has a collecting device which is designed to collect the flying object.

The system therefore includes a vehicle, which could be an automated vehicle, for example. But the driver of a conventional vehicle can also gain advantages when controlling the vehicle with the additional environmental data obtained, for example from a bird's eye view.

The advantages and further development possibilities mentioned above in connection with the method according to the invention apply mutatis mutandis also for the system according to the invention. Accordingly, the above-mentioned method steps can also be considered functional features of the system according to the invention.

In one embodiment, the flying object has a tail unit. The tail unit can be one or more control elements that direct the air flow. The tail unit can be constructed in a similar way to the so-called "flight" of a dart. For example, the tail unit can have two or more wings. Overall, the flying object with the tail unit can take on the shape of a dart (without the tip). In a specific embodiment, the flying object can therefore be referred to as a "camera dart".

In a further embodiment of the system according to the invention, the flying object has one or more cameras with which the environmental data can be recorded. The one or more cameras can be arranged, for example, at the tip of the flying object. In the case of the dart shape, the one or more cameras can be arranged in the head of the arrow. In particular, the flying object can have a cavity that can preferably be opened without tools in order to place the respective camera or cameras there. If necessary, the one or more cameras are also assigned a data storage device in order to save or temporarily store the captured images and/or videos. The camera or cameras can thus advantageously be used to obtain two- or three-dimensional images from a bird's eye view for the vehicle.

According to a further embodiment, it is provided that the flying object has a position sensor and/or inertial sensor in order to record position data, pose data, acceleration data and/or rotation data from the flying object during a flight of the flying object. The environmental data obtained from the flying object should preferably be able to be assigned to a coordinate system of the vehicle. This can be done, for example, by determining the position data or pose data (position and orientation) of the flying object relative or absolutely to the vehicle. Speed data, acceleration data and rotation data can also be helpful for this, for example in order to indirectly obtain the pose data. The position sensor can be a GPS sensor, for example. The inertial sensor can be an acceleration sensor or yaw rate sensor, for example. They can be the sensory components of an inertial navigation system.

In a further embodiment, the flying object has a data transmission unit with which data acquired by the flying object can be transmitted to the vehicle. The data transmission unit can be the data interface mentioned above. The data transmission unit can be contact-based or contactless. In addition, the data transmission unit can have a data processing unit, for example to subject the data to be transmitted to preprocessing (e.g. coding). The data to be transmitted can be the environmental data that represents the environment of the vehicle, but possibly also self-motion data, such as the above-mentioned position data, acceleration data and the like.

In yet another embodiment, it is provided that the vehicle records movement data relating to its own movement and/or a wind speed, and a control device of the vehicle uses this movement data to control the launch device, the capture device and/or the vehicle. This means that the vehicle registers, for example, its own movement or the wind speed (and possibly wind direction) absolutely or in relation to the flying object. This movement data can then be used, for example, to calculate a flight path of the flying object in order to control the launch device accordingly. In addition, the capture device can also be controlled using this movement data, for example by controlling its deployment mechanism in accordance with the expected flight path. The vehicle can also be controlled depending on the flight path estimated on the basis of the movement data for capturing the flying object. In this case, appropriate safety conditions must generally be observed.

• 1 eine Prinzipskizze eines Fahrzeugs mit abgeschossenem Flugobjekt.

The present invention will now be explained in more detail with reference to the accompanying drawing. It shows:

• 1 a schematic diagram of a vehicle with a launched flying object.

The present invention is based on the idea of using a type of camera dart to generate camera images from a bird's eye view. A launching device shoots the camera dart and a collecting device catches it again. The camera images obtained from a bird's eye view show little occlusion. The interpretation of the camera images by the (automated) vehicle enables complex traffic scenarios (including deadlocks) to be resolved.

1 shows a motor vehicle 1, not to scale, which can be designed as a car, truck and the like. The vehicle 1 has a launching device 2. With the launching device 2 it is possible to shoot a flying object 3 onto a flight path 4.

In 1 the vehicle 1 moves further, which is symbolized by a dashed silhouette 1'. In this further moving position, the vehicle 1 catches the flying object 3 again. For this purpose, the vehicle 1 has a catching device 5.

The flying object 3 has a sensor and in particular a camera 6 for recording environmental data of the surroundings of the vehicle 1. Alternatively or additionally, the flying object 3 can also have one or more other sensors for recording environmental data, such as a radar sensor, an ultrasonic sensor and the like. In the case of the camera 6, the flying object can record images or videos of the surroundings of the vehicle 1.

Optionally, the flying object 3 also has a data interface 7 with which it can transmit images and other environmental data to a receiver. The receiver is preferably the vehicle 1, which in turn has a data interface 8. The two transmission interfaces 7 and 8 can contain radio modules with which wireless (bidirectional) data communication is possible. The transmission of environmental data from the flying object 3 is also possible to other vehicles or to stationary facilities.

A specific embodiment is presented in more detail below. In this example, a conventional unmanned aerial vehicle (UAV) or drone is not used to create a bird's eye view image, but rather a non-powered flying object that can be referred to as a "sensor dart" or "camera dart". The "camera dart" is a large dart with one or more cameras, for example, at its tip. For example, video cameras can be used, as are known from the sports sector.

The (automated) vehicle 1 can receive and evaluate image or video recordings of the flying object 3 (here camera dart). The reception and evaluation of the image or video data preferably takes place during a flight of the camera dart. Based on the evaluation, the vehicle 1 can recognize, for example, that the current traffic situation is complex and could possibly even lead to a "deadlock".

For firing the flying object 3 or the camera dart, a “dart launching device” (launching device 2) is located on the vehicle 1. This can have a spring that can be pre-tensioned by an electric motor.

The (automated) vehicle 1 can optionally calculate its current own movement. If necessary, it can also determine the current wind speed using a specific external sensor. Based on these input variables, the vehicle 1 can calculate the required tension of the spring and the angle of the "dart launching device" so that the fired camera dart first reaches a certain height and then lands or is caught in a "dart catching device" (catching device 5). This catching device 5 is preferably attached to the vehicle 1, 1' itself. It can, for example, have a safety net and/or a gripper.

To calculate the trajectory, we can use the equation for throwing a ball at an angle, which is as follows: $$y\left(x\right)=-\frac{1}{2}\frac{G}{{\left(v_0\text{cos}\left({\alpha }_0\right)\right)}^2}{x}^2+\text{tan}\left({\alpha }_0\right)x+H$$

Here, v ₀ corresponds to an initial speed and α ₀ to a launch angle, as well as g to the gravitational constant (9.81 m/s ² ) and h to the launch height. This approximation does not take into account that the dart is not a ball. In addition, this simplification does not take into account the vehicle's own motion and the wind speed mentioned above. However, these two quantities should be taken into account in a correspondingly more complex equation.

If necessary, the camera dart or the flying object 3 has a GPS sensor and/or an inertial measurement unit (IMU). With such a sensor or sensors, a small controller in the camera dart can estimate the movement relative to the vehicle 1. Optionally, the movement of the camera dart can be sent to the vehicle 1 at predetermined time steps.

The (automated) vehicle 1 can, if necessary, make small corrections to its own position so that it can be guaranteed that the camera dart also hits the catch device 5. Care must be taken to ensure that this movement of the vehicle 1 is only carried out if it is safe (no collisions) and does not violate any traffic regulations (e.g. red traffic light).

Depending on the desired resolution, the camera dart (or generally the flying object 3) sends its camera images to the vehicle 1 during the flight or after the capture. With a near-field connection (such as can be established after a capture, for example), higher data transfer rates are generally possible. However, the transfer is then delayed accordingly.

The (automated) vehicle 1 interprets the camera data. The bird's eye view makes it possible to detect even hidden road users. On this basis, the vehicle 1 can plan a new path and, in the best case, the deadlock can be prevented. In another embodiment, the camera dart is transported from the collecting device 5 into a magazine. The magazine supplies the launching device 2 so that several measurements can be carried out one after the other using the camera darts.

As the above examples show, "camera darts" can be used to generate camera images from a bird's eye view. A launching device fires the camera dart and a collecting device catches it again. Due to the camera images from a bird's eye view, occlusions are significantly reduced. The interpretation of the camera images by the (automated) vehicle enables complex traffic scenarios (including deadlocks) to be resolved.

Advantageously, the use of unpowered flying objects such as camera darts according to the invention makes it easier to avoid occlusions than when taking pictures from the (automated) vehicle. This helps enormously with navigation in complex situations (especially within a city).

A particular advantage is that a powered flying object or a camera dart is much less dependent on a power supply than drones. A powered flying object does not need any power for the actual flight process. This also means that the charging process for charging a camera dart's battery can be much shorter.

Another advantage of camera darts is that they have less technology (mainly the camera/s), which makes them cheap and less prone to errors.

In addition, a camera dart can be made mainly from foam with a rounded tip. This makes a camera dart an essentially harmless flying object. Compared to a drone, it is less dangerous because it does not have fast-rotating propellers.

Another advantage of a camera dart over a drone is that firing can be done faster than taking off and landing a drone.

List of reference symbols

1

vehicle

1'

silhouette

2

Launcher

3

Flying object

4

Trajectory

5

Catch device

6

camera

7, 8

Transmission interfaces

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 102022002107 A1 [0005]
• DE 102017122261 A1 [0006]

Claims (10)

Method for obtaining environmental data of an environment of a vehicle (1) by - letting a flying object (3) fly in the environment of the vehicle and - capturing the environmental data by the flying object (3), characterized in that - the flying object (3) is without propulsion, - the flying object (3) is fired from a launching device (2) of the vehicle (1) and - the flying object (3) is caught again by a collecting device (5) of the vehicle (1). Procedure according to Claim 1 , characterized in that the environmental data are transmitted from the flying object (3) to the vehicle (1), in particular during a flight. Procedure according to Claim 1 or 2 , characterized in that the environmental data are analyzed in relation to a traffic situation. Method for controlling a vehicle, characterized in that environmental data of the vehicle (1) are processed using a method according to one of the Claims 1 until 3 and used for automatic or semi-automatic control of the vehicle (1). System for obtaining environmental data of an environment of a vehicle (1), the system comprising: - the vehicle (1) and - a flying object (3) with which the environmental data can be acquired, characterized in that - the flying object (3) is without propulsion, - the vehicle (1) has a launching device (2) which is designed to launch the flying object (3), and - the vehicle (1) has a collecting device (5) which is designed to collect the flying object (3). System according to Claim 4 , characterized in that the flying object (3) has a tail unit. System according to Claim 4 or 5 , characterized in that the flying object (3) has one or more cameras (6) with which the environmental data can be recorded. System according to one of the Claims 4 until 6 , characterized in that the flying object (3) has a position sensor and/or inertial sensor in order to detect position data, pose data, acceleration data and/or rotation data from the flying object during a flight of the flying object (3). System according to one of the Claims 4 until 7 , characterized in that the flying object (3) has a data transmission unit (7) with which data detected by the flying object can be transmitted to the vehicle (1). System according to one of the Claims 5 until 9 , characterized in that the vehicle (1) records movement data relating to its own movement and/or a wind speed, and a control device of the vehicle (1) uses these movement data to control the launching device (2), the collecting device (5) and/or the vehicle (1).

Images