EP3970116A1

EP3970116A1 - Method for determining an operating angle between a tractor and a trailer of said tractor

Info

Publication number: EP3970116A1
Application number: EP20723023.6A
Authority: EP
Inventors: Uwe MAEHNERT; Jerg Pfeil; Martin GOTH
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2019-05-14
Filing date: 2020-04-17
Publication date: 2022-03-23
Also published as: US20220196395A1; DE102019206985A1; WO2020229094A1; CN113874914A

Abstract

The invention relates to a method for determining an operating angle (α) between a tractor (10) and a trailer (20) of the tractor (10), comprising the steps: d) capturing camera images in chronological succession by means of at least one camera (11, 12) which is arranged on the tractor (10) and directed backward; e) identifying structures of the trailer (20) in the camera images; and a) determining the operating angle (α) by evaluating the detected structures in the camera images.

Description

description

Title:

Method for determining an operating angle between a

Tractor and a trailer of the tractor

The present invention relates to a method for determining an operating angle between a tractor and a trailer or semitrailer of the tractor. The invention also relates to a device for determining an operating angle between a tractor and a trailer of the Zugma machine. The invention also relates to a computer program product with program code means for carrying out the method.

State of the art

In order to describe a trajectory of a truck including one or more trailers or semitrailers, a movement model for truck trains with one or more trailers or semitrailers is required. This is conventionally approximated by estimating the angle between the tractor and the trailer and estimated on the basis of a mathematical model. However, this estimate gives rise to uncertainties that can lead to an error in the movement model.

Disclosure of the invention

One object of the invention is to provide an improved method for determining an angle between a towing vehicle and a trailer of the

Provide towing vehicle.

According to a first aspect, the object is achieved with a method for determining an operating angle between a tractor and a

Trailer of the tractor, with the steps:

Determination of camera images carried out one after the other by means of at least one rearward-facing camera arranged on the tractor; Recognition of structures of the trailer in the camera images; and determining the operating angle by evaluating the recorded structures in the camera images.

After the operating angle is strongly influenced by the vehicle's own motion model, e.g. of the truck is received, correct determination of the operating angle is of great importance. If the operating angle is incorrectly determined, uncertainties in the operating angle can disadvantageously increase even further as a result of error squares. As a result, the proposed determination of the operating angle supports an improved creation of the current eigenbe movement model of the vehicle.

According to a second aspect, the object is achieved with a device for determining an operating angle between a tractor and a trailer of the tractor, comprising:

a capture device for capturing at least two temporally successively determined camera images by means of at least one rearward-facing camera arranged on the tractor; and a determination device, functionally connected to the detection device, for determining the operating angle by evaluating recorded structures in the camera images.

According to a third aspect, the object is achieved with a computer program product with program code means which are executed on a proposed system or are stored on a computer-readable storage medium. In this way, the method can advantageously be designed as software and thereby modified and adapted in a simple and efficient manner.

The dependent claims relate to advantageous developments of the method.

An advantageous development of the method provides that in step b) flow vectors of an optical flow are formed from the camera images, similar structures being determined in temporally successive camera images. Corresponding elements are determined in camera images, from which the operating angle is determined. Another advantageous development of the method provides that an image of a trailing edge of the trailer is recorded for the optical flow. This is advantageously possible in a simple manner using so-called “edge algorithms”. Alternatively, other matching structures in camera images, such as surfaces of the trailer, can also be determined via the optical flow.

A further advantageous development of the method provides that in step b) a disparity of image elements of the camera images is determined in order to recognize the structures of the trailer in the camera images. In this variant, a camera in the form of a stereo camera is used, for which a Basisab was the two lenses is known. As a result of the system, there is a slight shift in the two camera images recorded with the different lenses, with a distance to the object being recorded image being measured via triangulation. In this way, the operating angle can be inferred by determining a changed distance.

A further advantageous development of the method provides that image data segmentation of the camera images is carried out in step b) to identify the structures of the trailer in the camera images, with a size and / or length of the trailer being determined by counting image pixels. In this way, an alternative method for recognizing structures in the camera images is advantageously provided.

Another advantageous development of the method provides that the recognition of structures of the trailer in the camera images, a classification of the recognized structures and / or a neural network is trained. In this way, too, an alternative method for recognizing structures in the camera images is provided.

Another advantageous development of the method provides that when determining the operating angle by evaluating the recorded structures in the camera images, a mean value calculation is carried out. In this way, a type of confidence estimate is realized, the accuracy of the operating angle determined being higher, the more of the proposed methods can be used to recognize the structures in the camera images. It makes sense to use at least two different methods to determine the structures.

Another advantageous development of the method provides that at least one of the following vehicle parameters is used to determine the operating angle: trailer length, trailer height, height of the loading area of the trailer, speed of the tractor, yaw rate of the tractor. In this way, the operating angle can be determined even more precisely by taking additional parameters of the vehicle into account.

Another advantageous development of the method provides that at least one of the following operating angles is determined: roll angle, pull angle, roll angle, kink angle. With the proposed method, different types of operating angles between the tractor and trailer can advantageously be determined, all of which represent different operating states or geometrical orientations of the tractor and trailer.

Another advantageous development of the method provides that in step b) the camera images are recorded using one or two cameras. The operating angle can advantageously be determined even more precisely using two cameras.

Another advantageous development of the method provides that the determined operating angle is used to create a vehicle's own movement model of the load. The own movement model represents an important input variable for various driver assistance systems and thus benefits from the precise determination of the operating angle. As a result, an improved operating characteristic of the driver assistance systems is supported as a function of the operating angle determined.

Further measures improving the invention are shown in more detail below together with the description of the preferred exemplary embodiments of the invention with reference to figures. The figures are not necessarily true to scale.

Brief description of the figures In the accompanying figures shows:

Fig. 1 is a basic plan view of a truck with a train

machine and a trailer;

Fig. 2 is a system diagram showing a proposed one

Method for detecting an operating angle between a tractor and a trailer of the truck;

3 shows a basic block diagram of a device for determining an operating angle between a tractor and a trailer of a truck; and

4 shows a basic sequence of a proposed method for

Determine an operating angle between a tractor and a trailer of a truck.

Embodiments of the invention

Conventionally, hardly any sensors are attached to the trailers or semitrailers of trucks to record the trailer's own movements. As a tractor in normal daily operation with many different

Trailers or semitrailers, retrofitting with the mentioned sensors is expensive and in particular not economically sensible. Even in the tractor itself, the existing sensors (e.g. sensors for ESP) cannot record the driving dynamics of the trailer or semitrailer.

It is proposed to use at least one, preferably two or more rearward-facing cameras on the tractor, whereby the rear environment of the tractor including the trailer is captured visually or with sequential image sequences (“video image”). Depending on the operating angle between the tractor and the trailer or the semi-trailer or the trailer combination, part of the trailer or the semi-trailer or the trailer combination is in the camera image, which makes it possible to determine the articulation angle (i.e. the angle between the tractor and the trailer) based on the number of pixels. By means of the An intrinsic movement model of the vehicle, for example the truck, can advantageously be determined much more precisely at the articulation angle.

In addition, by detecting the displacement of the rear edge of the trailer, a different operating angle can be estimated in the form of a roll angle, whereby a swaying of the trailer can be recognized early and suitable countermeasures (e.g. assistance-controlled braking and / or steering maneuvers) can be initiated. This is also required to model the vehicle's own movement.

With the proposed method, at least one of the following operating angles a can advantageously be determined: roll angle, pull angle, roll angle, kink angle.

Furthermore, with the proposed method it is advantageously also possible to estimate the angular rates (i.e. temporal changes in the different operating angles mentioned) of the trailer's own motion by detecting the displacement of objects in the image.

As a result, the possibility is provided by using a rearward-facing camera to measure the kink and roll angle that occurs during turning, maneuvering and maneuvering processes, when changing lanes, opening and reeving processes as well as in the event of possible rolling or tipping hazards Plays a role in predicting the proper movement of the entire vehicle.

Fig. 1 shows a basic top view of a vehicle 100 with a tractor 10 and a trailer 20 connected to the tractor 10. The vehicle 100 can be used, for example, as a truck or as another vehicle with a trailer (e.g. a passenger vehicle with a camping trailer, etc.) .) be trained. It can be seen that a rearward-facing camera 11, 12 is attached to both sides of the tractor 10. The cameras 11, 12 can be arranged, for example, in the vicinity of the rearview mirror (not shown) or in the front or rear part of the tractor 10, so that the cameras 11, 12 always have a pivoting range of the trailer 20 in view when the vehicle 100 is in operation. If the trailer 20 is not in a camera 11, 12 because the vehicle 100 is cornering visible, the respective other camera 11, 12 is used accordingly. The camera 11, 12 detects, in particular, a rear edge 21, 22 of the trailer 20 as 2D coordinates in the single image based on the options specified below.

By parameterizing technical trailer data (e.g. length, height of the loading space, height of the loading space above the ground, etc.), the position of the rear edge 21, 22 is calculated in a vehicle coordinate system. The length and height of the hold can also be determined automatically using existing surface estimation methods.

In order to improve the robustness of the position estimation of the trailing edge 21, 22 and to avoid the detection of errors on similar structures (e.g. lamppost, traffic sign, or the like), a filter (e.g. extended Kalman filter or the like) can optionally be used. Based on the vehicle dynamics and previous measurements, this predicts how the trailer 20 is moving and uses this to measure the operating angle a.

The filtered measurements from the left and right cameras 11, 12 are combined in a central control device 13 or in one of the cameras 11, 12. Knuckle and roll angles are used to determine the trajectory of the trailer 20 to be. Based on this, the entire trajectory of the entire vehicle 100, e.g. the drag curve can also be modeled. This can be used for the lateral control of the vehicle 100 within the framework of existing systems, such as, for example, lane departure warning systems or in the area of automated driving.

The above-mentioned pictorial detection of the rear edges 21, 22 of the trailer can be implemented in different ways, which are explained in more detail below.

(i) First possibility: optical flow

In this case, an optical flow or a flow vector is determined from the camera images from temporally successive camera images (image sequences or video images) from the cameras 11, 12 using common methods. Since the trailer 20 hardly moves in contrast to the surroundings while driving, separated the flow field of the optical flow the trailer 20 from the environment, where can be visually recognized through the rear edge 21, 22 of the trailer.

(ii) Second possibility: disparity estimation

By using a camera 11, 12 designed as a stereo camera, depth information can be determined for each pixel by means of a disparity estimate. As a result, the surface of the trailer 20 or other structures of the trailer 20 can be separated from the background by means of triangulation.

(iii) Third possibility: image data segmentation

In the event that the image determined by means of the camera 11, 12 itself is used, the trailing edge 21, 22 or an area of the trailer 20 can be determined by detecting the trailing edge 21, 22 using classifications or a neural network. This is possible because the trailer area can be separated from the rest of the environment as a homogeneous area. Since the trailer area is located in a predictable space, it is possible to determine this in a way that conserves resources.

All three variants mentioned use different basic assumptions for edge detection, which is why they can be used individually or also in combination in order to increase the robustness of the visual detection of the rear edge 21, 22. With a trailer combination (not shown in the figures), several trailing edges can also be detected.

Structures in the camera image sequences, in particular edges, can advantageously be recognized by means of an extended Kalman filter.

At dusk or at night, instead of the detection of the trailing edge 21, 22, the detection of trailer or trailer lights (not shown) of the trailer 20 can be used in order to visually detect the outlines of the trailer 20 and the trailing edge 21, 22. FIG. 2 shows a basic system diagram of the proposed method for determining an operating angle a between trailer 20 and tractor 10 of a vehicle 100.

In a step 200, a video or a camera image or a camera image sequence is determined by means of at least one rearward-facing camera 11, 12 on the tractor 10. In a step 210, the edge detection is carried out in one or more of the ways explained above. In a step 220, a mathematical algorithm is carried out which determines the operating angle α between the tractor 10 and the trailer 20. Optionally, a trailer height, trailer length, height of the loading area of the trailer 20, etc. can also be used as additional parameters in step 230 to determine the operating angle a. Optionally, in a step 240, a proper movement of the vehicle 100, e.g. a speed, yaw rate, etc. can be used for the calculation of the operating angle α.

As a result of step 220, the operating angle α is fed to a lateral control of the vehicle 100, which is used for a lateral control of the vehicle 100. In this case, the operating angle α is preferably used to determine an intrinsic movement model of the vehicle 100.

3 shows a greatly simplified block diagram of a device 400 for determining an angle a between the tractor 10 and a trailer 20 of the tractor 10. A detection device 310 can be seen for detecting a rear environment of the tractor 10 by means of at least one rearward camera 11, 12 Functionally connected to the detection device 310 is a determination device 320 which determines the operating angle α in one and / or more of the above-mentioned ways.

The determination device 320 is preferably designed as a computer program product that runs on a control device 13 of the tractor 10. Optionally, the control device can also be designed in a decentralized and cloud-based manner separately from the vehicle 100, the operating angle a being determined using the decentralized control device and the transmission of the operating angle a to the vehicle 100 being implemented using a high-performance wireless radio link (not shown). 4 shows a basic sequence of a method method for determining an operating angle a between a tractor 10 and a trailer 20 of the tractor 10. In a step 500, a determination of

Camera images carried out by means of at least one rearward-facing camera 11, 12 arranged on the tractor 10.

In a step 510, structures of the trailer 20 are recognized in the camera images.

In a step 520, the operating angle α is determined by evaluating the recorded structures in the camera images. When implementing the invention, the person skilled in the art will also implement embodiments not explained above.

Claims

Expectations

1. Method for determining an operating angle (a) between a

Tractor (10) and a trailer (20) of the tractor (10), with the steps:

a) ascertaining camera images carried out one after the other by means of at least one rearward-facing camera (11, 12) arranged on the tractor (10);

b) recognizing structures of the trailer (20) in the camera images; and c) determining the operating angle (a) by evaluating the detected

Structures in the camera images.

2. The method according to claim 1, wherein in step b) from the camera images

Flow vectors of an optical flow are formed, with similar structures being determined in temporally successive camera images.

3. The method according to claim 2, wherein an image of a rear edge (21, 22) of the trailer (20) is recorded for the optical flow.

4. The method according to any one of the preceding claims, wherein in step b) a disparity of picture elements of the camera images is determined in order to recognize the structures of the trailer (20) in the camera images.

5. The method according to any one of the preceding claims, wherein in step b) an image data segmentation of the camera images is carried out in order to identify the structures of the trailer (20) in the camera images, a size and / or length of the trailer (20) being determined by counting image pixels. is determined.

6. The method according to any one of the preceding claims, wherein the recognition of structures of the trailer (20) in the camera images, a classification of the recognized structures and / or a neural network is trained.

7. The method according to any one of the preceding claims, wherein when determining the operating angle (a) by evaluating the recorded structures in the camera images, a mean value calculation is carried out.

8. The method according to any one of the preceding claims, wherein for the

Determination of the operating angle (a) at least one of the following

Parameters of a vehicle (100) is used: trailer length,

Trailer height, height of the bed of the trailer, speed of the tractor, yaw rate of the tractor.

9. The method according to any one of the preceding claims, wherein at least one of the following operating angles (a) is determined: roll angle, pull angle, roll angle, kink angle.

10. The method according to any one of the preceding claims, wherein in step b) the camera images are recorded by means of one or by means of two cameras (11, 12).

11. The method according to any one of the preceding claims, wherein the determined operating angle (a) for creating a proper movement model of the

Vehicle (100) is used.

12. Device (400) for determining an operating angle (a) between a tractor (10) and a trailer (20) of the tractor (10), comprising:

an acquisition device (310) for acquiring at least two camera images determined one after the other by means of at least one rearward-facing camera (11, 12) arranged on the tractor (10); and

one functionally connected to the detection device (310)

Determination device (320) for determining the operating angle (a) by evaluating recorded structures in the camera images.

13. Computer program product with program code means for performing the method according to one of claims 1 to 11, if it is on a

Device (400) for determining an operating angle (a) between a tractor (10) and a trailer (20) of a vehicle (100) is executed or is stored on a computer-readable storage medium.