DE102019217297A1 - Securing a land vehicle - Google Patents

Abstract

A method (200) for detecting a person (120) or animal (120) in the environment (125) of a land vehicle (105) comprises steps of scanning (205, 210, 230) the environment (125) of the land vehicle (105), the is located on an agricultural area; providing an image of the environment (125) based on the scan; determining (240) a movement in the surroundings (125) of the land vehicle (105) on the basis of the image; and determining that the movement is associated with a human (120) or animal (120).

Description

The present invention relates to securing a land vehicle. In particular, the invention relates to securing a land vehicle against a collision with a person or an animal.

A motor vehicle on a road network can be partially controlled automatically. For this purpose, the surroundings of the motor vehicle can be scanned by means of an imaging sensor. If an object is recognized in the image, the motor vehicle can be controlled in order to avoid a collision with the object. The object can also include a person.

For an agricultural vehicle on an agricultural area, there are different difficulties with regard to recognizing a person or animal in the vicinity than for the motor vehicle on the road network. For example, the frequency with which an animal is found in the environment can be significantly increased. A range of different animals can also be larger than in normal road traffic, which can make it difficult to recognize all possible animals. In the agricultural environment, an animal often tries to hide from the vehicle. Not every animal that can be encountered is able to evade the vehicle on its own. In addition, there is usually vegetation on the usable area, which makes it difficult to visually identify humans or animals.

Out DE 10 2017 113 245 A1 a method for recognizing animals in the tramline of agricultural field cultivation machines, in particular for grassland machines such as mowing machines, is known.

One object of the present invention is to provide an improved technique for recognizing a person or animal in the area of an agricultural area. The invention solves this problem by means of the subjects of the independent claims. Subclaims reproduce preferred embodiments.

According to a first aspect of the present invention, a method for detecting a person or animal in the vicinity of a land vehicle that is located on an agricultural area comprises steps of scanning the environment of the land vehicle; providing an image of the surroundings based on the scan; determining a movement in the vicinity of the land vehicle on the basis of the image; and determining that the movement is associated with a human or animal.

The method can improve recognition of a person or animal, in particular in the area of an agricultural area. An animal can be recognized from a certain, predetermined size. An impending collision with a recognized person or animal can be better recognized or avoided.

The image can comprise a conventional image in the sense of a graphic representation. However, the image can also be the result of an image, for example if the scanning device can detect radiation in a range of invisible wavelengths. These include, for example, infrared or ultraviolet radiation, but also, for example, radar or X-ray radiation. The scanning device can work passively or actively in that it illuminates or excites the environment to be imaged by means of electromagnetic radiation. For example, a radar or LiDAR sensor can be used to provide the image. A false color image can be provided in which pixels can represent an aspect other than an optical aspect. For example, a false-color image can be provided in which a gray level or color of an image point depicts a movement speed of an object being imaged.

The land vehicle is preferably set up to work on the usable area. It can for example comprise a tractor, a setting or harvesting machine or some other agricultural device. The land vehicle preferably comprises a drive motor and can be viewed as a motor vehicle. The land vehicle can be controlled by a person, with the support of a driver assistant, or autonomously.

The usable area can include practically any type of agricultural land. This can include a field, a field, a vineyard, an orchard, a forest or a special cultivation such as a hop garden. The vegetation includes one or more plants that grow or are cultivated on the usable area. Usually the vegetation is a monoculture of plants of only one variety. Individual plants of the vegetation are usually very similar to one another; in particular, they can have corresponding sizes, characteristics or stages of development. Spaces between plants can be regular. Depending on a type of plant, the vegetation can in particular be laid out as a row culture, the agricultural machine preferably traveling along such a row.

The vegetation can at least partially obscure humans or animals. As a result, humans or animals can only be partially or not at all recorded directly from the land vehicle. For example, the vegetation can cover a section of the human or animal close to the ground so that its legs are not visible. Depending on the size ratio of the human or animal to the plants, half or more can be covered. For example, an average sized person can be smaller than fully grown maize plants, so that this person in a maize field can hardly be detected directly from the land vehicle.

In a first variant of the present invention, a movement of the vegetation can be determined. For example, if a person walks through a maize field, they can move the individual maize plants in a way that is different from the way the wind can move the plants. The movement of the vegetation can be further analyzed to determine what path the human or animal is moving, how fast the movement is, what location he or she is in, or approximately how big he or it is. In a second variant, a movement of the person or animal can also be determined directly.

In a further embodiment, it is determined that the movement corresponds to a pattern that indicates a person or an animal. This technique can be used with both of the variants described. The pattern can, for example, reflect a movement sequence, a movement speed, a size, mass or force of the person or animal. Properties of a moving plant, such as its size, flexibility or mass, can also be taken into account. The movement can be compared with a previously trained pattern using a neural network. The pattern does not have to be specified parametrically, but can be determined from a large number of examples through machine-based learning.

In one embodiment, the pattern relates to a vital function of the human or animal. The vital function can in particular include breathing or a heartbeat. A rhythmic movement of a body part or of a plant located on the human or animal body can be compared in amplitude and / or frequency with a predetermined reference value or reference range. Another vital function can relate to a body temperature of the person or animal, which can also be determined.

An imminent collision of the land vehicle with the human or animal is optionally determined. For this purpose, a location or a trajectory of the person or animal can be determined and compared with a location or a trajectory of the land vehicle. In another embodiment, a danger area can be defined around the land vehicle and entry of the human or animal into the danger area can be determined. A shape and size of the danger area can be dependent on movement parameters of the land vehicle and of the person or animal. In one embodiment, a signal is output in the event of an impending collision or a specific risk of collision. The signal can be directed to a driver of the land vehicle, for example. In another embodiment, the land vehicle can also be controlled, for example by providing the signal to a corresponding actuator for longitudinal or lateral control of the land vehicle. The control is preferably carried out in such a way that an impending collision is avoided or the risk of collision is reduced. For this purpose, the land vehicle can in particular be braked or stopped.

According to a further aspect of the present invention, a device for detecting a person or animal in the vicinity of a land vehicle that is located on an agricultural area comprises a scanning device for scanning the environment of the land vehicle and for providing an image on the basis of the scanning; and a processing device. The processing device is set up to determine a movement in the surroundings of the land vehicle on the basis of the image and to determine that the movement is assigned to a person or animal.

The processing device can be set up to carry out a method described herein in whole or in part. For this purpose, the processing device can comprise a programmable microcomputer or microcontroller and the method can be in the form of a computer program product with program code means. The computer program product can also be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the device or vice versa.

The scanning device is preferably imaging and more preferably works without contact. It can include a camera, a LiDAR sensor, a radar sensor or a thermal camera. Further embodiments are possible, for example a camera in different wavelength ranges, a depth camera or a combined scanning device with an optical camera and a laser-based range finder, for example. As a passive sensor, the scanning device can evaluate a physical signal that is emitted by the environment without any further action. As an active sensor, the environment can be stimulated in a predetermined manner to emit a physical signal, for example by directing light or high-frequency electromagnetic radiation onto the environment.

The scanning device can be mounted on board a drone that flies in the area of the land vehicle. The land vehicle can form a system together with the drone. More than one scanning device can be attached both to the land vehicle and to the drone, with the determination of the person or animal being able to take into account signals from several scanning devices. In one embodiment, the human or the animal is determined by the drone and an indication of the human or the animal is transmitted to the land vehicle. In another embodiment, the scanning results of a scanning device attached to the drone are transmitted to the land vehicle and further processed there. The further processing can take into account further scanning results of a scanning device attached to the land vehicle. A scanning result can correspond to an image.

According to yet another aspect of the present invention, a land vehicle includes an apparatus as described herein.

• 1 ein System;
• 2 ein Ablaufdiagramm eines Verfahrens;
• 3 Darstellungen eines Menschen und eines Tiers auf einer landwirtschaftlichen Nutzfläche; und
• 4 einen beispielhaften Gefahrenbereich um ein Landfahrzeug darstellt.

The invention will now be described in more detail with reference to the accompanying figures, in which:

• 1 a system;
• 2 a flow chart of a method;
• 3rd Representations of a person and an animal on an agricultural area; and
• 4th represents an exemplary danger area around a land vehicle.

1 shows a system 100 who have a land vehicle 105 and an optional drone 110 includes. On board the land vehicle 105 is a device 115 attached, which is set up to a human 120 or an animal in an environment 125 of the land vehicle 105 to determine. The surrounding 125 usually includes an agricultural area, and is usually located in the vicinity 125 vegetation that may include one or more plants.

The device is optional 115 set up an impending collision between the land vehicle 105 and man 120 or to determine the animal. It can also be used to control the land vehicle 105 be set up to prevent an impending collision. Although the present invention is applicable to both the determination of a human 120 as well as an animal, in the following it will mainly refer to a human being for easier readability 120 switched off. Where an animal is to be identified, it is usually meant one that exceeds a predetermined size, which size can be expressed in terms of mass or body dimension. The predetermined size can for example correspond approximately to that of an adult cat or a newly born fawn.

The device 115 preferably comprises a processing device 130 having one or more on board the land vehicle 105 attached scanning devices 135 connected is. A scanning device 135 is set up to do this, the environment 125 to be scanned contactlessly as possible and to provide an image that includes a number of pixels that reproduce predetermined information. The pixels form an object in the environment 125 especially humans 120 . The image provided does not have to have an optically detectable impression of the surroundings 125 but can also express other information that is intended, for example, as colors or gray levels for the individual pixels. The scanning device 135 can for example comprise a depth camera, with a gray level of a pixel in each case being a distance to the scanning device 135 reflected. Other possible scanning devices 135 include a radar sensor, a LiDAR sensor, an optical camera, a video camera, or an infrared camera.

The scanning device preferably comprises 135 on the land vehicle 105 at least one radar sensor 135 with motion detection. Two radar sensors can also be preferred 135 right and left of a direction of movement of the land vehicle 105 to be appropriate. The motion detection can be based on the Doppler principle, so that only a single scan can be required for a motion determination. In another variant, a motion blur of an image or a difference between the contents of two images provided with a time delay can be evaluated in order to determine a movement. The processing facility 130 on board the land vehicle 105 can be set up based on a detected movement in the environment 125 the people 120 to determine. Movement can affect people 120 itself or an object in the human realm 120 concern, for example, a plant. In a further embodiment, the processing device is 130 set up to be a vital sign of man 120 to determine. For this purpose, in particular a periodic or rhythmic movement can be determined by the person 120 goes out.

The processing facility 130 is set up for this at an interface 140 provide a signal that points to the particular person 120 indicates. The signal can be sent to a person, in particular in the area of the land vehicle 105 , be directed and output in an optical, acoustic and / or haptic manner. The signal can also be directed to a technical device, in particular a control device of the land vehicle 105 , and can be output electrically, for example. The signal can be used to control the land vehicle 105 be set up and determined in such a way that the control intervention leads to an impending collision with the person 120 is prevented.

To communicate with the drone 110 is preferably a wireless interface 145 intended. The drone 110 can be used as an extension of the land vehicle 105 can be considered to be one or more scanning devices 135 to bring to a position from which the human being 120 can be recorded better. This position is usually in a range of approx. 5 - 50 m above the land vehicle 105 other heights are also possible. It is preferred that the drone 110 a fixed horizontal position with respect to the land vehicle 105 occupies. For example, the drone can 110 are always kept above the land vehicle 105 or at a predetermined position in front of the land vehicle 105 on a planned trajectory. The drone 110 can also be in a fixed relative horizontal position to the land vehicle 105 being held. In a further embodiment, the relative horizontal position is predetermined, while the vertical position can be changed in order to realize different perspectives. The drone 110 can be implemented as an aircraft in any way, for example as a fixed-wing aircraft, rotary-wing aircraft, airship or kite. In a preferred embodiment, the drone is 110 designed as rotary wing aircraft with four, six or eight rotors.

A scan range or scan angle of the drone-based scanner 135 can at a perspective of the drone 110 , in particular an altitude, be adapted. The drone 110 further preferably comprises a processing device 130 that are similar to the processing facility 130 of the land vehicle 105 can be set up to do so on the basis of one of the scanning device 135 provided image to the people 120 to determine. A determination result can be made by means of a wireless interface 145 to the device 115 be transmitted. In a further embodiment, the scanning device 135 provided scanning results, in particular an image or object data derived from the image, by means of the wireless interface 140 to the device 115 be transmitted.

In a particularly preferred embodiment, they are on the drone 110 a thermal camera 135 and / or a radar sensor 135 intended. The thermal camera 135 can cover an infrared range that is invisible to a person and possibly also a visible wavelength range. The radar sensor 135 can be designed in particular broadband or ultra-broadband (UWB, “ultra wide band”) in order to be able to penetrate plant objects to a predetermined depth.

Based on at least one scan of one on the drone 110 attached scanning device 135 can by means of the processing device 130 the human being 120 to be determined. In a further embodiment, the processing device is 130 set up a vital sign of the person on the basis of a detected movement 120 to determine. A determination result or from the scanner 135 Information provided can be sent to the device 115 be transmitted. A human movement 120 can be determined or predicted. A risk of collision with humans 120 can especially be determined when man 120 in a predetermined danger area 150 around the land vehicle 105 is located around or penetrates into it. The danger area 150 will be referred to below with reference to 4th explained in more detail.

2 shows a flow chart of a method 200 . Steps of the process 200 can each be carried out by means of appropriate devices so that the flow chart can also be interpreted in the manner of a block diagram.

In the illustrated embodiment, the environment 125 of the land vehicle 105 in one step 205 by means of a thermal camera 135 and / or in one step 210 by means of a UWB radar sensor 135 both scanned on board the drone 110 are located. In this case, scanning results are preferably provided in image form in each case. In one step 215 becomes man 120 recognized on the basis of the scanning results.

In one embodiment, the step comprises 215 a determination of a vital sign based on the scan of the surroundings 125 using the UWB radar sensor 135 . To this end, it can be determined whether a predetermined movement that indicates a vital sign can be determined in the imaged area. In particular, the vital sign can relate to a heartbeat or breathing. The movement of the imaged area can be analyzed using range or Doppler FFT (Fast Fourier Transformation). The vital sign is preferably recognized by the fact that it is rhythmic, the rhythm lies in a predetermined frequency range, an amplitude of the movement in one predetermined area, and / or an extension of a moving area of the environment 125 lies in a predetermined range. The human being 120 can be determined based on the determined vital sign. In the case of an animal 120 a size, an animal species or another characteristic can also be determined.

Object data about the recognized person 120 , and possibly via another recognized object, can be done in one step 220 to the device 115 on board the land vehicle 105 be transmitted. In the same step 220 can also be a waypoint in the opposite direction from the device 115 on board the land vehicle 105 to the drone 110 be transmitted. In one step 225 can the drone 110 be steered to the transmitted waypoint. Instead of the waypoint, it is also possible, for example, to transmit a trajectory along which the drone is moving 110 should move. The waypoint or the trajectory preferably include three-dimensional information. They can also include two-dimensional information, with altitude information being transmitted separately or by the drone 110 can be determined by yourself. The information can be on the land vehicle 105 or a higher-level coordinate system.

On the part of the device 115 can be done in one step 230 the surrounding 125 in particular by means of one or more radar sensors 135 are scanned. A used radar sensor 135 is preferably set up to allow movement in the environment 125 capture. From the radar sensor 135 Information provided can be done in one step 235 be evaluated for an inhomogeneous movement in the environment 125 to determine. Such movement can involve a spatially limited object roughly the size of a human 120 Has. The movement can also include plants in the environment 125 concern that are or have been moved by a person. A detected movement can be compared to a movement pattern to determine whether the movement is from a human 120 can come from. In general, it can be said that a movement is the more inhomogeneous, the faster the person moves 120 emotional. Usually there is also an area in which by humans 120 Moving plants stand, the bigger the faster a person moves 120 emotional. On the basis of the particular movement, a person can 120 to be determined. Man stands 120 quiet so its position can be determined. Man moves 120 so its trajectory can be determined. The trajectory can be predicted for a predetermined time.

Object data about the recognized person 120 , and possibly via another recognized object, can be done in one step 240 merged with object data received from the drone 110 were received. The merged object data can be evaluated to identify an impending collision of the land vehicle 105 with the particular person 120 to determine. In particular, it can be checked whether the object data of the drone 110 with those of the land vehicle 105 are compatible. If a collision is imminent, a path for the land vehicle can be created 105 be planned in such a way that the collision is avoided. The path can be implemented in a step 245 in that longitudinal and / or lateral control of the land vehicle is carried out.

3rd shows representations of a person 120 and an animal 120 on an agricultural area. 3a shows a picture of a deer 120 in a corn field. The representation is based on an optical recording in the visible wavelength range. The deer 120 is relatively difficult to see, especially if it is surrounded by large areas of maize plants or partially covered by maize plants.

3b shows a picture of the same deer 120 like 3a , but recorded in the near infrared range. Light areas of the displayed image stand for certain high temperatures, dark areas for certain low temperatures. The deer 120 stands out clearly in this representation from its cooler surroundings.

A fully clothed person 120 however, may not be clearly visible on an infrared image. A better detection can be achieved by means of a UWB radar 135 take place, which may include relatively low frequency components. Radar signals used can penetrate clothing or parts of plants to a predetermined depth.

3c shows a person moving through a cornfield 120 . Along a path 305 along which man 120 has moved, a movement of maize plants can be determined. It can also be a movement of humans 120 be determined by yourself. Its movements can be locally restricted in a predetermined manner. Looking at a larger section of the environment 125 , one of the people appears 120 caused movement inhomogeneous. A natural movement of plants, for example by wind, can be much more homogeneous. The human being 120 can thus be determined based on its own movement or the movement of surrounding plants.

4th shows an exemplary danger area 150 around a land vehicle 105 around. One in the danger area 150 located person 120 can from the moving land vehicle 105 detected so that there is a risk of an accident. Shape and size of the danger area 150 are based on the assumed speed of movement of the land vehicle 105 and of man 120 . At a predetermined speed of movement of the land vehicle 105 must be a person crossing at a certain distance 120 Reach a speed in a predetermined range to use the land vehicle 105 to collide. Is man 120 the land vehicle is moving too slowly 105 past him before he reaches it, if he is too fast he runs in front of the land vehicle 105 past. For different distances between the land vehicle 105 and the trajectory of the man crossing 120 assigned speed ranges can be determined in which a collision is imminent. The individual areas can be overlaid to form an entire area, which in a realistic example extends between 7.6 and 24 km / h. The specified shape and size of the hazardous area 150 in 4th are to be regarded as exemplary. In a refinement, contrary to what is shown, a trapezoidal danger area can also be used 150 be assumed, its narrow side to the agricultural machine 105 adjoins.

In order to prevent being caught or a collision, it should be ensured that the human 120 if possible in the entire danger area 120 can be determined. The danger area 120 can be scanned in one or more sections. A first, in front of the land vehicle 105 The area 405 lying down can, for example, by means of a on the land vehicle 105 attached camera 135 are scanned. Another, further ahead of the land vehicle 105 The lying area 405 can, for example, by means of a camera 135 on board the drone 110 are scanned.

To prevent yourself from being a human 120 sideways on the land vehicle 105 too moved and creates a risk of collision, lateral areas 415 and / or 420, in particular by means of radar sensors 135 are scanned. The scanning can allow motion detection. For example, short-range radar sensors 135 (SRR: "short range radar") can be used, which have a scanning distance in the range of a few meters. A lateral width of a scanning area 415, 420 can be determined as a function of an assumed maximum lateral approach speed of a person 120 to be determined. The faster the person 120 can be, the wider the scan area 415, 420 should be. A depth of a scanning area 415, 420 can be selected as a function of a maximum movement speed of the land vehicle, a high maximum speed correlating to a deep scanning area 415, 420 and vice versa.

List of reference symbols

100

system

105

Land vehicle

110

drone

115

contraption

120

Human or animal

125

environment

130

Processing facility

135

Scanning device

140

interface

145

wireless interface

150

Danger area

200

Procedure

205

Scanning thermal camera

210

Scanning UWB radar sensor

215

Determine human or animal

220

Exchange of information

225

Control drone

230

Scanning radar sensor

235

determine inhomogeneous movement

240

Determining an associated movement

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102017113245 A1 [0004]

Claims (12)

Method (200) for detecting a person (120) or animal (120) in the vicinity (125) of a land vehicle (105) which is located on an agricultural area, the method (200) comprising the following steps: - Scanning (205, 210, 230) of the surroundings (125) of the land vehicle (105); - Providing an image of the surroundings (125) on the basis of the scan; - determining (215, 235) a movement in the surroundings (125) of the land vehicle (105) on the basis of the image; and - Determining (240) that the movement is associated with a human (120) or an animal (120). Method (200) according to Claim 1 , wherein the land vehicle (105) is set up to process the usable area. Method (200) according to Claim 1 or 2 whereby vegetation, the person (120) or the animal (120) at least partially obscures. Method (200) according to Claim 3 , whereby a movement of the vegetation is determined. The method (200) according to any one of the preceding claims, wherein it is determined that the movement corresponds to a pattern that indicates a human (120) or an animal (120). Method (200) according to Claim 5 , wherein the pattern relates to a vital function of the human (120) or animal (120). Method (200) according to one of the preceding claims, wherein an imminent collision with the human (120) or animal (120) is determined. Device (115) for detecting a person (120) or animal (120) in the surroundings (125) of a land vehicle (105) which is located on an agricultural area, the device comprising: a scanning device (135) for scanning the surroundings (125) the land vehicle (105) and providing an image based on the scan; a processing device (130) which is set up to determine a movement in the surroundings (125) of the land vehicle (105) on the basis of the image and to determine that the movement is associated with a person (120) or an animal (120) . Device (115) after Claim 8 wherein the scanning device (135) comprises one of a camera, a LiDAR sensor, a radar sensor, and a thermal camera. Device according to Claim 8 or 9 , wherein the scanning device (135) is mounted on board a drone (110) which flies in the area of the land vehicle (105). Device (115) according to one of the Claims 8 to 10 , wherein a plurality of scanning devices (135) are used for scanning the surroundings (125). Land vehicle (105) comprising a device (115) according to one of Claims 8 to 11 .

Images