DE102011113197B4 - Method and device for determining an angle between a towing vehicle and a trailer coupled to it, and the vehicle - Google Patents

Abstract

Method for determining an angle between a towing vehicle (1) and a trailer (2) coupled therewith, comprising the steps
Acquiring an image of the trailer (2) with the aid of an image acquisition device (3), characterized by
Evaluation of the captured image by an image evaluation, the image evaluation being based on a three-dimensional movement model of a trailer drawbar (5) of the trailer (2) and including three rotation angles of the trailer drawbar (5) as output variables, the rotation angle being a rotation of the trailer drawbar (5) in one Describe the trailer coupling (4) of the towing vehicle (1) about three different spatial axes, and
Determining the angle between the towing vehicle (1) and the trailer (2) on the basis of the image evaluation (16).

Description

The invention relates to a method and a device for determining an angle between a towing vehicle and a trailer coupled to it, and the vehicle.

Various assistance systems are available to provide support when driving with a trailer. A distinction is made between passive systems in which, for example, auxiliary lines are displayed in a rear view camera image, and active systems in which, for example, supporting control interventions take place. Active systems require a sensor to measure tiller movement. This can be a direction of rotation sensor or magnetic sensor integrated in a trailer coupling. The drawbar of the trailer must therefore include a magnet on a counterpart to the trailer coupling. When the angle between the towing vehicle and the trailer changes, the orientation of the magnet to the magnetic sensor changes. This is detected by the magnetic sensor and can be used to determine the change in angle. Such an angle determination requires that the trailer coupling is equipped with such a magnet, otherwise an angle determination is not possible. Furthermore, the magnet-based type of angle determination is susceptible to magnetic fields or magnetic field distortions that can occur due to loading of the towing vehicle or the trailer.

DE 10 2004 022 113 A1 describes a method for determining an orientation angle of the trailer to the towing vehicle, with current image data from a video camera being sent to an evaluation sub-unit and these data being checked for correlation in a computing device with templates stored in a data memory, i.e. with images stored in reference driving situations ( Template matching).

DE 10 2011 101 990 B3 discloses a method for determining the relative drawbar angle, wherein a first intensity and / or color gradient along at least one first trajectory can be determined by means of an evaluation device at a first point in time in a first image, the first trajectory depicting an arc around a pivot point of the drawbar, wherein A further intensity and / or color gradient along the first trajectory can be determined at at least one further point in time in a further image, a degree of similarity between the first intensity and / or color gradient along the at least first trajectory in the first image and a number of shifted others Intensity and / or color gradients can be determined along the at least first trajectory in the further image, the shifted further intensity and color gradients being shifted along the at least first trajectory in the further image, the relative tiller angle depending on the Similarity measures is determined.

The object of the present invention is therefore to provide a method and a device for determining an angle between a towing vehicle and a trailer coupled to it, whereby a very precise determination of the trailer movement and thus active driving assistance is made possible. Furthermore, it should be possible to determine the angle in a cost-effective and robust manner with regard to external influences.

This object is achieved by a method according to claim 1, a device according to claim 11 and a vehicle according to claim 13.

Further advantageous refinements of the invention are specified in the dependent claims.

According to the invention, a method and a device for determining an angle between a towing vehicle and a trailer coupled to it are provided, an image of the trailer being captured with the aid of an image capturing device, the image capturing device being a rear camera with known internal and external camera parameters which is already included in a standardized way in many vehicles. The method and the device are characterized in that the captured image is evaluated by image processing, the image processing being based on a three-dimensional movement model of a trailer drawbar. As a result, the angle, for example an initial articulation angle, between the vehicle and the trailer can be determined.

The angle between the vehicle and the trailer coupled to it is determined by evaluating image data, so that no special devices are required on the trailer, so that the method and the device can be used with any trailer. Many vehicles have an image capture device, for example a rear camera, which supports the driver of the towing vehicle when parking or maneuvering. The method and the device can thus be implemented cost-effectively, for example in a control unit of the vehicle.

The method and the device for determining the angle can be based on an evaluation of corresponding feature points of chronologically successive images. An image k and a temporally preceding image k-1 are evaluated as input variables for a point in time.

Furthermore, for example, a calibrated rear camera with known internal and external camera parameters and a known position of a ball of a trailer coupling are used.

In order to achieve a high degree of accuracy in the estimation of the angle between the towing vehicle and the associated trailer, a movement model is used which describes a rotation of the trailer drawbar around the coupling ball with three rotation angles.

The output variables of the movement model are the three angles of rotation r _x , r _y and r _{z of} the trailer drawbar, which describe a rotation of the drawbar around a head of the trailer coupling, for example a ball head of the trailer coupling, as the pivot point of the trailer drawbar.

To estimate the three angles of rotation, an image area containing a tiller head and the coupling ball can be used, with only feature points in this image area being to be detected and followed by images that follow one another in time. Such a masking can reduce the computational effort and the disruptive influence of, for example, drawbars with a longitudinal suspension.

The coordinates of the feature points in the coordinate system of the image sensor of the camera can continue to be used. By selecting correspondences, a correspondence being two feature points in two consecutive images, only those feature points are provided which are located on the trailer drawbar, and these are used for the further calculation of the angle. The selection can be made, for example, by a random sampling consensus method (RANSAC method).

The rotation angle can be estimated from the image coordinates of the feature points and the points of the trailer drawbar in three-dimensional space. The desired change in the horizontal angle of rotation can be derived from the continuously estimated rotation angle for each image.

The method and the device for determining the horizontal angle of rotation between a towing vehicle and a trailer on the basis of a three-dimensional image evaluation make it possible to achieve high levels of accuracy, especially when the trailer is tilted or on uneven ground, since the movement model is rotated the trailer drawbar around the coupling ball is described with the three angles of rotation r _x , r _y and r _z . Furthermore, in addition to an image display, a highly precise measurement of trailer movement and thus active driving support can be made possible. The invention can be implemented inexpensively by means of already integrated rear cameras.

• 1 zeigt ein Zugfahrzeug mit einem daran gekoppelten Anhänger und eine Vorrichtung zur Bestimmung des Winkels zwischen dem Zugfahrzeug und dem Anhänger gemäß einem Ausführungsbeispiel der Erfindung.
• 2 zeigt eine schematische Darstellung eines Verfahrens zur Bestimmung des Winkels zwischen einem Fahrzeug und einem daran gekoppelten Anhänger gemäß einem weiteren Ausführungsbeispiel der Erfindung.
• 3 zeigt das Zugfahrzeug mit dem daran gekoppelten Anhänger aus 1 mit drei Rotationswinkeln r_x, r_y und r_z der Anhängerdeichsel, die eine Rotation der Deichsel um den Kugelkopf der Anhängerdeichsel als Drehpunkt der Anhängerdeichsel beschreiben, sowie die Position der Kamera und das Fahrzeugkoordinatensystem.

The invention is described in more detail below on the basis of preferred exemplary embodiments with reference to the drawings.

• 1 shows a towing vehicle with a trailer coupled to it and a device for determining the angle between the towing vehicle and the trailer according to an embodiment of the invention.
• 2 shows a schematic representation of a method for determining the angle between a vehicle and a trailer coupled to it according to a further exemplary embodiment of the invention.
• 3 shows the towing vehicle with the trailer attached to it 1 with three angles of rotation r _x , r _y and r _{z of} the trailer drawbar, which describe a rotation of the drawbar around the ball head of the trailer drawbar as the pivot point of the trailer drawbar, as well as the position of the camera and the vehicle coordinate system.

1 shows a towing vehicle 1 , which has a coupling ball 4th with a trailer 2 is coupled and to an image capture device 3 , for example a rear camera, is provided. In many vehicles 1 a rear camera is already installed so that the rear camera can be used to determine a kink or rotation angle between the towing vehicle 1 and the trailer 2 can be implemented inexpensively.

By evaluating an image from the image capture device 3 must be attached to the associated trailer 2 no special fixtures exist, so any trailer 2 can be used.

Furthermore, the vehicle is 1 with an image evaluation device 16 associated with the image capture device 3 is electronically connected, and a trailer maneuvering assistance system 17th , which with the image evaluation device 16 is electronically connected. The one from the image evaluation device 16 certain angles between the vehicle 1 and the trailer 2 becomes the trailer maneuvering assistance system 17th made available for maneuvering the towing vehicle 1 to be able to support.

2 shows a schematic representation of a method for determining an angle, in particular a bending angle, between a towing vehicle and a trailer coupled to it according to an embodiment of the invention, as it is in the in 1 device shown and in particular in the image evaluation device 16 can be provided.

In step 6th is from the image capture device 3 a picture of the trailer 2 recorded. The image evaluation device 16 evaluates the resulting image data by means of an image evaluation in order to be able to determine the kink angle. A current image k and a temporally preceding image k-1 are evaluated as input variables for a point in time. The images can be captured by means of a calibrated camera with known internal and external camera parameters, for example the rear camera 3 of the vehicle 1 , are detected, the position T _{K of} the coupling ball 4th the trailer coupling is assumed to be known. The internal camera parameters include, for example, a focal length, a height aspect ratio of an image sensor of the camera and a lens distortion of the camera. The external camera parameters include, for example, a position and an orientation of the camera relative to a vehicle coordinate system, which as in FIG 3 shown is oriented, the origin of the vehicle coordinate system being at the position of the coupling ball of the trailer drawbar.

The coordinate system of the camera can be transformed by a rotation of -90 degrees around an x-axis, a rotation of +180 degrees around a rotated y-axis and a shift in the vehicle coordinate system. A mapping of a point in a three-dimensional space P _j = (X, Y, Z) ^T in an image point p _{j, k} = (x, y) ^T is obtained with the aid of a projection matrix A _k = K _k R _k [IC _k ] which contains a calibration _{matrix K k} , which contains the internal camera parameters, and a rotation matrix R _k , which describes the orientation of the camera and is composed of three rotation angles r _x , r _y , r _z . Furthermore, I describes a 3x3 identity matrix, and C _k describes the position of the camera, where: ${\text{p}}_{\text{j}\text{, k}}={\text{A.}}_{\text{k}}{\text{P.}}_{\text{j}}.$

In the 3 Rotation angles r _x , r _y , r _{z shown} describe a rotation of the trailer drawbar around the ball head of the trailer coupling as a pivot point of the drawbar around the x, y and z axes of the vehicle coordinate system and relate to a start image k = 0 relative to the drawbar movement , as in 3 shown. With a known absolute orientation of the drawbar for the image k = 0, the articulation angle α _K between the trailer and the towing vehicle can be calculated _{from the rotation angle r z determined continuously for each image.} In contrast to the determination of an absolute articulation angle, the change in articulation angle Δα _{K can be calculated from the rotation angle r z} for at least two images without knowing the absolute orientation of the drawbar for the image k = 0.

To determine the relative change in the articulation angle Δα _K , step 7th masking performed. An image area that contains the tiller head and the coupling ball can be used for the estimation. In this case, only feature points that lie in the image area are detected and further tracked in images that follow one another in time. The masking reduces the computational effort and reduces the disruptive influence of drawbars that have, for example, longitudinal suspension.

The in 2 The embodiment shown is in a further step 8th a correspondence analysis carried out. The correspondence analysis can use corresponding feature points from chronologically consecutive points and from the image acquisition device 3 determine captured images. The image area that can be used for the estimation preferably includes the tiller head and the coupling ball. Only feature points from their image area can be evaluated and determined in images that follow one another over time.

The image coordinates of the feature points can be found in step 9 lens distortion compensation. The image coordinates of the feature points are then combined with the internal camera parameters 10 , which contain the parameters of the lens distortion of the camera, rectified and transformed into a coordinate system of the image sensor of the camera. The coordinate system of the image sensor of the camera can have its origin in a main point of the camera and has a metric unit in mm.

Further procedural steps 11-15 process the coordinates of the feature points in the coordinate system of the image sensor of the camera.

The in 2 shown embodiment are in the further step 11 with a selection of correspondences, with a correspondence being two feature points in two temporally successive images, only feature points that are located on the trailer drawbar are provided for further processing. The selection can be made using a random sampling consensus method (RANSC method). As a movement model of corresponding feature points, it can approximately be assumed that the feature points move tangentially around the coupling ball in the image. With auxiliary vectors v _k = p _{j, k-1} -t _c and v _k-1 = p _{j, k} -t _{c of} a correspondence j, an angle β □ with β = arccos [(v _k-1 * V _k ) * (| v _k-1 | * | v _k |) ^-1 ] can be calculated as model parameters, the calculation of the angle β being carried out from a random sample of a feature point p _{j, k} which belongs to a correspondence. Here, tc describes a projected center of rotation, for example the projected ball of the trailer drawbar, p _{j, k-1} describes a corresponding feature point of image k-1, and p _{j, k} describes a corresponding feature point of image k. With the angle β thus calculated, a check of the remaining feature points can be calculated on the basis of a Euclidean distance. A correspondence supports a calculated angle β if the Euclidean distance to be calculated is smaller than a predefined threshold value. The RANSC method can be terminated if an angle β D has been found which is supported by a minimum number of feature points, for example more than 70% of the feature points, or if a maximum number of random samples has been reached. The largest number of supporting feature points found up to the termination is used for the further process steps 13-15 selected.

In a further step 13th an initialization of points in three-dimensional space is carried out if there is little or no movement of the trailer drawbar or if the trajectories of the feature points are too short, since it is then difficult to determine a three-dimensional position of the points because a parallax of lines of sight of the corresponding feature points Triangulation is too imprecise. Therefore, for each feature point, an intersection of a line of sight with a plane that runs parallel to the roadway at the height of the ball of the trailer coupling can be calculated. The point of intersection can then be an initial position of the point belonging to a feature point in three-dimensional space.

If the parallax of the lines of sight is sufficiently large, the three-dimensional positions of the points in three-dimensional space in step 14th of the in 2 The embodiment shown can be updated by means of triangulation. The triangulation can lead to a linear system of equations, which can be solved, for example, with a singular value decomposition. The solution then provides the three-dimensional coordinates of the point in three-dimensional space.

A back projection error of the point in three-dimensional space can be evaluated for an image k. If this is larger than a pixel, the three-dimensional point can be rejected as faulty.

wobei für die Abbildung eines Raumpunktes P_j in die Bildebene der Kamera gilt: ${\text{p}}_{j,k}=A_k^{\text{'}}{P}_j.$

The in 2 embodiment shown can in step 15th an estimation of the angles of rotation can be carried out, with a movement of the image capturing device instead of the estimation of a rotational movement of the trailer drawbar 3 or the camera relative to the trailer drawbar 5 can be determined and the camera is on a circular path around a center of rotation, which is the position of the coupling ball 4th is moved. The movement model of the camera used here can be restricted, because in the case of a translational and rotational movement of the camera it can be assumed that the camera only moves through a rotational movement. Since a displacement of the camera from the center of rotation T _{k is} known, the movement of the camera can be calculated from the three angles of rotation and the position of the center of rotation T _k. The external camera parameters thus include three angles of rotation r _X , r _Y , and r _Z. A shift of the projection center in temporally successive images is dependent on the three angles of rotation. With the restricted motion model of the camera, a projection matrix of the camera is called ${\mathrm{A.}}_k^{\text{'}}={\text{K}}_{\text{k}}{\text{R.}}_{\text{k}}\left[\text{I.}|\left({\mathrm{R.}}_k^T-\text{I.}\right){\text{T}}_{\text{K}}\right],$

where for the mapping of a spatial point P _j into the image plane of the camera, the following applies: ${\text{p}}_{j,k}={\mathrm{A.}}_k^{\text{'}}{\mathrm{P.}}_j.$

For a motion estimation of the camera of the current image k, all points in three-dimensional space can be used which have a selected corresponding feature point in the current image. Only the movement of the camera can be estimated.

For all points in three-dimensional space, a square distance between the projected point in three-dimensional space and the associated corresponding feature point in the image can also be minimized by the equation $\left\{{\widehat{r}}_x,{\widehat{r}}_y,{\widehat{r}}_z\right\}=\text{bad}\underset{\left\{{\widehat{r}}_x,{\widehat{r}}_y,{\widehat{r}}_z\right\}}{{\displaystyle \text{min}}}{\displaystyle \sum _{j=1}^J{d}_{\wedge }{\left(p_{j,k},{\mathrm{A.}}_k^{\text{'}}{\mathrm{P.}}_j\right)}^2.}$

Here, r̂ _x , r̂ _y , r̂ _z are the estimated values of the angles of rotation, d _∧ is the Mahalanobis distance, which describes a distance measure between points in the multidimensional vector space.

The minimization can take place with an iterative Levenberg-Marquardt method, the position and orientation of the camera being initialized with the already determined camera parameters of the temporally preceding image k-1 and values of the three rotation angles being determined.

Claims (13)

Method for determining an angle between a towing vehicle (1) and a trailer (2) coupled therewith, comprising the steps Acquisition of an image of the trailer (2) with the aid of an image acquisition device (3), characterized by evaluating the acquired image by means of an image evaluation, the image evaluation being based on a three-dimensional movement model of a trailer drawbar (5) of the trailer (2) and three angles of rotation of the trailer drawbar ( 5) as output variables, the rotation angle describing a rotation of the trailer drawbar (5) in a trailer coupling (4) of the towing vehicle (1) around three different spatial axes, and determining the angle between the towing vehicle (1) and the trailer (2) Basis of image evaluation (16). Procedure according to Claim 1 , characterized in that the image capturing device (3) comprises a rear camera of the towing vehicle (1). Procedure according to Claim 1 or Claim 2 , characterized in that the image evaluation is based on an evaluation of corresponding feature points of successive images. Procedure according to Claim 1 , characterized in that, for the determination of the rotation angle, feature points are detected in an image area of the image which contains a head of the trailer drawbar (5) and a head of the trailer coupling (4). Procedure according to Claim 3 and Claim 4 , characterized in that the evaluation of the corresponding feature points is based on the same image area of the successive images. Method according to one of the Claims 3 , 4 or 5, characterized in that image coordinates of the feature points are rectified by internal camera parameters (10) of the image acquisition device (3) and transformed into a sensor coordinate system. Method according to one of the preceding claims, characterized in that during the image evaluation only those feature points are selected in the captured image which are located on the trailer drawbar (5). Method according to one of the preceding claims, characterized in that, during the image evaluation, feature points of the images to be taken into account are selected using a random sampling consensus method (RANSAC method) Method according to one of the Claims 7 or 8th , characterized in that rotation angles between the towing vehicle (1) and the associated trailer (2) are determined after evaluating image coordinates of the feature points and points in a three-dimensional space of the trailer drawbar (5). Procedure according to Claim 1 , 4th or 9 , characterized in that a change in a rotation angle is determined by the rotation angle determined for each captured image. Device for determining an angle between a towing vehicle (1) and a trailer (2) coupled therewith, comprising an image capturing device (3) for capturing an image of the trailer (2), characterized by an image evaluation device (16) which is connected to the image capturing device (3 ) is coupled, wherein the captured images are evaluated by means of a three-dimensional movement model of a trailer drawbar (5) of the trailer (2), the image evaluation comprising three rotation angles of the trailer drawbar (5) as output variables, the rotation angle being a rotation of the trailer drawbar (5) in describe a trailer coupling (4) of the towing vehicle (1) about three different spatial axes, and the angle between the towing vehicle (1) and the trailer (2) is determined on the basis of the image evaluation. Device according to Claim 11 , characterized in that the device for performing the method according to one of the Claims 1 - 10 is designed. Vehicle (1) with a device according to Claim 11 or Claim 12 .

Images