JPWO2021197651A5 - - Google Patents

Description

FIELD OF THE INVENTION The present invention relates generally to the field of vehicle assistance systems. In particular, the present invention relates to a method and system for calculating the yaw angle of a trailer coupled to a towing vehicle based on image information provided by a camera on the vehicle.

Methods are known for calculating the angle of a trailer relative to a towing vehicle based on image information provided by a camera on the vehicle.

In particular, a first type of method is known which provides a reliable approximation of the trailer yaw angle without taking into account the location of the trailer turning point. However , the accuracy of the yaw angle approximation of the first type of method is low when the yaw angle is large.

In addition , a second type of method is known which takes into account the location of the turning point of the trailer. However , in certain situations, the second type of method has high noise.

An object of embodiments of the invention is to provide a method for calculating the yaw angle of a trailer with high robustness and reliability. This problem is solved by the features of the independent claims. Preferred embodiments are described in the dependent claims. Although not explicitly described, embodiments of the invention can be freely combined with each other.

According to one aspect , the invention relates to a method for determining the yaw angle of a trailer relative to the longitudinal axis of a towing vehicle. The method includes the following methods.

First, at least first and second images of the trailer are captured using a camera. The first and second images are taken such that the orientation of the trailer relative to the vehicle is different on at least the two images.

After taking this image, at least a first characteristic of the trailer is determined. This first feature needs to be visible on the first and second images.

Additionally , at least first and second algorithms are provided for calculating an angle estimate based on the at least first feature. This first algorithm may be an algorithm that examines the rotational geometry of the trailer and takes into account in particular the location of the turning point of the trailer. The second algorithm may be an algorithm that approximates the yaw angle of the trailer by taking into account two or more features at different rotational positions of the trailer. This second algorithm does not need to examine the rotational geometry of the trailer.

At least a first angle estimate is provided for the first feature based on at least one of the first algorithm and the second algorithm.

According to a first embodiment , providing a first angle estimate determines whether a first or a second algorithm should be used to calculate this first angle estimate based on one or more criteria for each feature. and calculating the first angle estimate based on the selected algorithm for the first feature. Thus, in other words, the decision as to which algorithm to use is made before calculating the angle estimate.

According to a second embodiment , providing a first angle estimate comprises calculating a first angle estimate for the at least one feature based on the first and second algorithms, and calculating the first angle estimate obtained by the first algorithm. or by determining whether the first angle estimate obtained by the second algorithm is used for further processing based on one or more criteria. Thus, in other words, in the second form, the first angle estimate is calculated based on a plurality of algorithms, and the decision as to which angle estimate should be used takes into account the results of this angle estimate calculation. It will be done. Thus, in contrast to the first embodiment above, the decision as to which algorithm to use is made after the calculation of the angle estimate.

Finally, the trailer's yaw angle is calculated based on this first angle estimate.

This method is advantageous because it determines which method to use from multiple methods for each feature to provide an angle estimate, and the result of yaw angle determination is very accurate and robust. This is because for each feature, a suitable method that provides the most reliable angle estimation result can be selected .

According to one embodiment , for each feature an attempt is made to calculate an angle estimate based on a first algorithm, an algorithm that takes advantage of the rotational geometry of the trailer. If this first algorithm fails because one or more criteria, in particular a predetermined criterion, is not fulfilled, a second algorithm is used .

According to one embodiment , the first algorithm is configured to perform the following steps.
- a first ray between the camera and the determined first feature on the first image is set and projected onto a horizontal plane to obtain a first projected feature position; Similarly, a second ray between the camera and the determined first feature on the second image is set and projected onto this horizontal plane to obtain a second projected feature position.
- a first orthogonal bisector is set between the location of the first projected feature location and the location of the second projected feature location; More specifically, the first orthogonal bisector may be a straight line that passes orthogonally through the midpoint of the line connecting the location of the first projected feature position and the location of the second projected feature position. In this way, the first orthogonal bisector may be set on the horizontal plane.
- After setting the first orthogonal bisector, a first point of intersection of the first orthogonal bisector with the reference axis or a further orthogonal bisector is set .
- a first angle estimate is calculated based on this first orthogonal bisector; The first angle estimation is an angle on a horizontal plane between a first line from the first projected feature position to this first intersection point and a second line from the second projected feature position to this first intersection point. be.

References to "the position of the first feature on the first image or the second image" do not refer to the 2D image coordinates of the image feature or its corresponding ray (e.g., expressed as a 3D unit vector or azimuth or elevation). worth it. 2D image coordinates may be converted to rays using camera calibration information.

This first angle estimate may be open from the towing vehicle toward the trailer.

According to one embodiment , a second feature of the trailer visible on the first and second images is determined, and the second feature is placed at a different location of the trailer than the first feature. The first algorithm uses two or more features of the trailer to provide an angle estimate. In particular, the first algorithm is
- Obtaining a third projected feature position by projecting the ray between the camera and the determined second feature on the first image onto a horizontal plane; Obtaining a fourth projected feature position by projecting a ray between the second feature onto this horizontal plane,
- setting a second orthogonal bisector between the location of the third projected feature position and the location of the fourth projected feature position;
- determining a second point of intersection of a second orthogonal bisector with this reference axis , this first orthogonal bisector or a further orthogonal bisector;
- The second angle estimation is performed between the first line from the third projected feature position to this second intersection point and the second line from the fourth projected feature position to this second intersection point on this horizontal plane. The second angle estimate is configured to calculate an angle.

By using two or more features, multiple angle estimates can be calculated based on different features reducing the effects of noise and mismatch .

According to one embodiment , the second algorithm is configured to calculate the first angle estimate. The first angle estimate characterizes a turning angle in a horizontal plane between a first feature on the first image and a first feature on the second image with respect to a fixed point of the towing vehicle.

According to one embodiment , a second feature of the trailer visible on the first and second images is determined, the second feature is placed at a different position of the trailer than the first feature, and the second algorithm determines a second angle. The second angle estimate is configured to calculate an estimate, the second angle estimate characterizing a turning angle in a horizontal plane between a second feature on the first image and a second feature on the second image with respect to a fixed point of the towing vehicle. By using two or more features in the second algorithm, multiple angle estimates can be calculated based on different features reducing the effects of noise and mismatch .

According to one embodiment , the step of determining based on the one or more criteria includes determining the length of the base line between the feature on the first image and the feature on the second image; and comparing the length to a length threshold. This length of the base line may be determined after transforming the first and second features captured on the first and second images into a common mapping or coordinate system, which coordinate system is one of the vehicles. or information regarding the position of more fixed points, for example the position of a camera. Preferably, if the length of the base line is less than this length threshold, the angle estimate is provided by the second algorithm instead of the first algorithm. This avoids inaccurate angle estimation with high noise.

According to one embodiment , the step of determining based on one or more of the plurality of criteria includes determining at least one of roll and pitch of the trailer, comparing the roll to a roll threshold, and comparing the pitch to a pitch threshold. and at least one of comparing . Preferably, the angle estimate is provided by the second algorithm instead of the first algorithm if the roll exceeds the roll threshold and/or if the pitch exceeds the pitch threshold. This avoids inaccurate angle estimation with high noise.

According to one embodiment , the step of determining based on one or more criteria includes determining a vertical distance of the feature relative to a horizontal reference plane and comparing this vertical distance to a distance threshold. Be prepared . This vertical distance may be determined after transforming at least one feature captured on the first and second images to a common map or coordinate system. This coordinate system may also contain information about the height level of one or more fixed points of the vehicle, for example the height level of the camera, in particular with respect to a horizontal reference plane. If the vertical distance is less than this distance threshold, the angle estimate is provided by the second algorithm instead of the first algorithm. This avoids inaccurate angle estimation with high noise.

According to one embodiment , in the first or second image the yaw angle of the trailer with respect to the vehicle is zero. Thereby, this image can be used as a "zero pose image", that is, as a reference for exact alignment between the longitudinal axis of the vehicle and the longitudinal axis of the trailer. However , as long as the other yaw angle is known, this other yaw angle value can also be used as a reference value.

According to one embodiment , in the second algorithm, calculating the at least one angle estimate comprises determining a ray between the fixed point and the at least one feature in the first and second images. This ray relates to a straight line extending between this fixed point and the respective feature. Based on this ray, the current turning angle can be determined , for example based on a geometric method, with low computational costs.

According to one embodiment , camera calibration information is used to convert the position of the first and /or second feature into a light beam , particularly in the second algorithm. For example, by having information about the camera position using camera calibration information, the position of a certain feature on an image can be converted into location information depending on or correlated to the camera position.

According to one embodiment , the camera calibration information is used to transform the position of the first feature and /or the second feature from a local region of the image to a local region of the vehicle or a fixed point location of the vehicle. be exposed . For example, the camera calibration information is used to obtain the camera position, which converts the position of a certain feature on an image into location information depending on or correlated to the position of a camera included or attached to the vehicle. I can .

According to one embodiment , in addition to this at least one feature, at least one further feature of the trailer is used to calculate the yaw angle. The at least one further feature is located at a different location on the trailer than the first feature. For example, the first feature may be a first distinct feature at a first location on the trailer, and the second feature may be a second distinct feature at a second location on the trailer . By using two or more features, additional angle estimates can be obtained, which further improves the robustness and reliability of the yaw angle determination.

According to one embodiment , the yaw angle is calculated by setting a median value based on at least two angle estimates provided by the first and second algorithms. This results in a very stable yaw angle determination.

According to another embodiment , the yaw angle is calculated by setting an average value of at least two angle estimates provided by the first and second algorithms or by using a statistical method applied to this angle estimate. be done.

According to one embodiment , the method further comprises determining an angular window. The angle window may have an upper boundary and a lower boundary around the yaw angle. Furthermore, a set of features is determined, and this feature within the set of features provides an angle estimate that lies within this angle window. This determined set of features, preferably only the features included in this set of features , are used for subsequent yaw angle calculations. Thus, in other words, by using the information of the previous yaw angle determination, two or more features of the trailer that result in an angle estimate very close to the determined yaw angle (i.e. within the angle window) are determined and determined. Features that result in angle estimates that deviate significantly from the calculated yaw angle (i.e., outside the angle window) are not tracked. This can significantly reduce the complexity and accuracy of angle estimation calculations.

According to one embodiment , the value of the calculated yaw angle is increased by a certain amount or percentage to correct the underestimation. In order to correct for an underestimation of the calculation result, for example, the calculated yaw angle may be scaled by 5% to 15%, in particular by 10%.

According to one embodiment , in the first algorithm , the reference axis is The line is this longitudinal axis of the towing vehicle.

According to another embodiment , in the first algorithm, if at least one of the camera and the towball has a lateral offset with respect to the longitudinal axis of the towing vehicle, the reference axis extends between the camera and the towball . It is a straight line. This makes it possible to compensate for lateral offsets between the camera and the towball .

According to one embodiment , the camera is a rear view camera of the vehicle. Based on a rear-view camera, images of the trailer can be captured with low technical costs.

According to a further aspect , a system for determining a yaw angle of a trailer relative to a longitudinal axis of a towing vehicle is disclosed. The system includes a camera that captures images of the trailer, and a processing entity that processes the captured images. This system is
- capturing at least first and second images of the trailer using a camera, the orientation of the trailer relative to the vehicle being different on the at least two images;
- determining at least a first feature of the trailer visible on the first and second images;
- providing at least first and second algorithms for calculating at least one angle estimate based on at least a first feature;
- providing at least a first angle estimate for the first feature;
- determining whether to use a first or a second algorithm to calculate the first angle estimate based on one or more criteria for each feature; 1 calculating an angle estimate, or
- Calculate a first angle estimate for the first feature based on the first and second algorithms, and the first angle estimate obtained by the first algorithm or the first angle estimate obtained by the second algorithm is subjected to further processing. providing at least a first angle estimate for the first feature by determining whether to use it based on one or more criteria;
- calculating a yaw angle based on the first angle estimate.

Any of the above features described as an embodiment of the method are also applicable as system features in the system of the present disclosure.

According to yet another aspect, a vehicle is disclosed that includes a system according to any one of the above aspects.

The term "vehicle" as used in this disclosure may relate to a passenger car, lorry, bus, train or any other watercraft.

The term "yaw angle" as used in this disclosure may refer to the turning angle between the longitudinal axis of the vehicle and the longitudinal axis of the trailer.

The term "median" as used in this disclosure may refer to the value that separates the upper and lower halves of a data sample or probability distribution.

The term “essentially” or “substantially” as used in the present invention means
means a deviation of +/-10%, preferably +/-5%, from the exact value;
to mean a deviation in the form of an insignificant change in the function and/or the traffic law;
means at least one of the following .

Various aspects of the invention, including its specific features and advantages, will be readily understood from the following detailed description and accompanying drawings.

FIG. 1 shows an exemplary top view of a vehicle towing a trailer. FIG. 2 schematically shows angle estimation based on a first feature captured by a camera image at different turning angles between a trailer and a towing vehicle according to a first algorithm. FIG. 3 schematically shows two angle estimations based on first and second features captured by camera images at different turning angles between the trailer and the towing vehicle according to the first algorithm. FIG. 4 schematically shows angle estimation based on first and second features captured by camera images at different turning angles between the trailer and the towing vehicle, according to a second algorithm. FIG. 5 shows a schematic block diagram illustrating the steps of a method for determining the yaw angle of the trailer with respect to the longitudinal axis of the towing vehicle.

The invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments are shown. While the embodiments in the figures relate to preferred embodiments, all elements and features described in connection with the embodiments may be used herein, insofar as appropriate, particularly in relation to any other embodiments described above. It may also be used in combination with any other embodiments and features described. However , the invention should not be considered limited to the embodiments described herein. Throughout the following description, like reference numerals are used, where applicable, to indicate similar elements, members, items or features.

The features of the invention disclosed in the description, the claims , the exemplary embodiments and /or the drawings, both separately and in any combination thereof, enable the invention to be understood in its various forms, both separately and in any combination thereof. It may be any material that realizes this.

FIG. 1 shows a top view of a vehicle 1 towing a trailer 2. FIG. The vehicle 1 includes a longitudinal axis LAV passing through the center of the vehicle 1. Similarly, the trailer 2 includes a longitudinal axis LAT passing through the center of the trailer 2. The trailer 2 is connected to the vehicle 1 using a trailer hitch including a tow ball 4.

In some driving situations, the longitudinal axis LAV of the vehicle 1 and the longitudinal axis LAT of the trailer 2 may not be arranged parallel to each other and may not coincide with each other, but these axes may vary depending on the yaw angle. It will lock up YA. In other words, the yaw angle YA defines the angular deviation of the longitudinal axis LAT of the trailer 2 from the longitudinal axis LAV of the vehicle 1. The yaw angle YA may be measured on a horizontal plane including the longitudinal axis LAT of the trailer 2 and the longitudinal axis LAV of the vehicle 1.

Obtaining the yaw angle YA is advantageous, inter alia, for example in trailer assistance systems.

In order to determine the yaw angle YA, a plurality of images of at least a part of the trailer 2 are taken using the camera 3. The camera 3 may be, for example, a rear view camera of the vehicle 1, and the rear view camera may also be used to take images of the surroundings of the vehicle 1 when reversing. One of the captured images may show a known angular positioning of the trailer 2 with respect to the towing vehicle 1. This image may be used as a reference for calculating the yaw angle YA. In this known angular arrangement of the trailer 2 with respect to the towing vehicle 1, the yaw angle YA may be 0° or any other angular value.

FIG. 2 shows a schematic diagram showing the angular relationship of the first feature of the trailer 2 at different times when the trailer 2 has different yaw angles YA relative to the towing vehicle 1.

The camera 3 may take two or more images at different times, with different angular positions of the trailer 2 relative to the vehicle 1. For example, a series of images may be taken.

In this example, the second image may show the orientation of the trailer relative to the vehicle at a yaw angle YA=0°.

Due to the angular movement of the trailer 2 over time, certain features detected on the trailer may appear in different positions in the first and second images. In FIG. 2, the first feature is indicated by a square.

The upper representation of the first feature (associated with the solid ray R connecting the feature to camera 3) is identified in the first image, and the lower representation of the first feature (associated with the dashed ray R connecting the feature with camera 3) is different. identified in the second image at the time point. Calibration information of the camera 3 may be used to correlate the location of the first feature in each image with the location of the vehicle 1, in particular with a predetermined fixed point of the vehicle 1. In particular, the calibration information of the camera 3 may be used to determine a ray R connecting the first feature with the camera 3 and converting the location of this first feature in image coordinates into a ray. In other words, the location of the feature on the image is correlated with the location of a fixed point on the vehicle based on the calibration information of the camera 3 in order to associate the camera location and the feature location.

Trailer features are located and matched using feature detection and matching algorithms. For example, Harris Corner Detection Method, Scale Invariant Feature Transform (SIFT) Algorithm, Speed Up Robust Features (SURF) Algorithm, Binary Robust Invariant Scalable Keypoint (BRISK) Algorithm, Binary Robust Independent Fundamental Features (BRIEF), Directed FAST and The Rotated BRIEF (ORB) algorithm or other suitable feature detection and matching algorithm may be used .

Feature detection and matching algorithms may detect image features that are on the trailer or not on the trailer. A number of different methods may be used to separate trailer characteristics from non- trailer characteristics. For example, in the case of straight-line forward travel, trailer features may be separated from non- trailer features by looking for features that remain in the same location over time. Alternatively, the motion of background features may be modeled over time using the known motion of the vehicle. This may be extracted from CAN data regarding speed and steering. In this case, features that do not apply to the epipolar constraint of the fundamental matrix may be considered as trailer features.

Depending on certain situations or conditions, it may be preferable to select an algorithm from a set of algorithms instead of using only a single algorithm for determining at least one angle estimate α1, α2 representing the yaw angle YA. It seemed like there was. In the following, first and second algorithms for determining at least one angle estimate α1, α2 and a framework for determining which algorithm to use to determine this at least one angle estimate α1, α2 are provided. .

The determination of the yaw angle YA based on the first algorithm is shown in more detail in FIG. 2. This change in the location of the at least one feature between the first and second images is used to determine at least a first angle estimate α1.

After identifying features in each image, the first features of the first and second images are projected onto a common horizontal plane. More specifically, the first projected feature position PFP1a is obtained by projecting the light beam between the camera 3 and the determined first feature on the first image onto a horizontal plane. Furthermore, the second projected feature position PFP1b is obtained by projecting the light rays between the camera 3 and the determined first feature on the second image onto the same horizontal plane. It is worth mentioning that this projection is performed in the vertical direction, thereby changing only the elevation angle of the ray, but not the azimuth angle.

After determining the first and second projected feature positions PFP1a and PFP1b, a first orthogonal bisector B1 is set based on the first and second projected feature positions PFP1a and PFP1b. As shown in FIG. 2, the first orthogonal bisector B1 is a line that is perpendicular to the line connecting the first and second projected feature positions PFP1a and PFP1b. Furthermore, the first orthogonal bisector B1 passes through the midpoint of this connection line. This first orthogonal bisector B1 intersects with the reference axis, which is the longitudinal axis LAV of the vehicle in this embodiment. A rotation point is obtained by this intersection of the first orthogonal bisector B1 and the reference axis , and the trailer 2 rotates around this rotation point. More specifically, this intersection indicates the position of the tow ball 4.

A first angle estimate α1 is calculated based on this first orthogonal bisector B1. This first angle estimation α1 is based on a first line L1 connecting the first projected feature position PFP1a, the intersection of the first orthogonal bisector B1 and the reference axis , the second projected feature position PFP1b, and the second projected feature position PFP1b. This is the angle provided between the intersection of the 1 orthogonal bisector B1 and the reference axis , and the second line L2 connecting the two. The intersection may indicate the position of the tow ball 4. More specifically , the first angle estimate α1 is calculated as follows: ( Characterizes the turning angle of the trailer 2 in this horizontal plane around this first intersection point IP1 (which is at least approximately the position of the tow ball 4).

In this embodiment, the reference axis is the longitudinal axis LAV of the towing vehicle 1 because the camera 3 and the tow ball 4 are located on the longitudinal axis LAV of the vehicle 1. In other embodiments, the camera 3 or the towball 4 has a lateral offset with respect to the longitudinal axis LAV of the vehicle 1 , or the camera 3 and the towball 4 are laterally offset with respect to the longitudinal axis LAV of the vehicle 1. If the directional offsets are different, the reference axis may be formed by a straight line connecting the camera 3 and the tow ball 4.

The first angle estimate α1 represents the yaw angle YA of the trailer 2 about its actual rotation point.

FIG. 3 shows the first and second images of the trailer 2 captured at different times (times when the trailer 2 has different yaw angles YA with respect to the towing vehicle 1) using the first algorithm to set the yaw angle YA. 2 shows an embodiment similar to FIG. 2 using features (ie two different features).

A plurality of different features may be distinguishable on the image taken by the camera 3. As shown in FIG. 3, this feature is identified at different angular positions relative to the fixed point of the vehicle 1. The first feature is indicated by a square and the second feature by a triangle. This fixed point may be the location of the camera 3 or the location of the tow ball 4.

In FIG. 3, an upper pair of first and second features (denoted by PFP1a, PFP2a and associated with the solid ray connecting the features with camera 3) is identified in the first image, and the first and second features F1, The lower pair of F2 (denoted by PFP1b, PFP2b and associated with the dashed ray connecting the feature with camera 3) has been identified in the second image at different times.

The determination of the yaw angle YA is performed in the same way as in the embodiment of FIG. The main difference is that two angle estimates α1, α2 are set and the yaw angle YA of the trailer is developed based on these two angle estimates α1, α2. More specifically, setting the first orthogonal bisector B1 and obtaining the first angle estimate α1 are performed as described above.

Furthermore, the second angle estimation α2 sets the third projected feature position PFP2a and the fourth projected feature position PFP2b, sets the second orthogonal bisector B2 to obtain the second intersection point IP2, and sets the third projected feature position PFP2a and the fourth projected feature position PFP2b. It is obtained by connecting the feature position PFP2a and the fourth projected feature position PFP2b with this second intersection point IP2. This third projected feature position PFP2a is obtained by projecting the second feature in the first image onto this horizontal plane, and the fourth projected feature position PFP2b is obtained by projecting the second feature in the second image onto this horizontal plane. It can be obtained by The second intersection point IP2 may be a point where the second orthogonal bisector B2 intersects the reference axis , in this embodiment, the longitudinal axis LAV of the vehicle. The second angle estimate α2 is the angle between the first line connecting the third projected feature position PFP2a and the intersection IP2 and the second line connecting the fourth projected feature position PFP2b and the intersection IP2.

Also in the embodiment of FIG. 3, the reference axis is the longitudinal axis LAV of the towing vehicle 1 because the camera 3 and tow ball 4 are located on this longitudinal axis LAV of the vehicle 1. . In other embodiments, the camera 3 or the towball 4 has a lateral offset with respect to the longitudinal axis LAV of the vehicle 1 , or the camera 3 and the towball 4 are laterally offset with respect to the longitudinal axis LAV of the vehicle 1. If the directional offsets are different, the reference axis may be formed by a straight line connecting the camera 3 and the tow ball 4.

It is worth mentioning that more than two features of trailer 2 can be determined and tracked over multiple images. Furthermore, preferably more than two images are taken at different times in order to improve the results of the yaw angle estimation. This allows more than two angle estimates α1, α2 to be set in order to improve the quality of the yaw angle determination.

A second algorithm for determining at least one angle estimate α1, α2 will be described below with reference to FIG. 4.

FIG. 4 is a schematic diagram illustrating the angular relationship between the first and second features F1, F2 of the trailer 2, in which the features F1, F2 are identified at different times and at different angular positions with respect to the fixed point of the vehicle 1. A schematic diagram is shown.

The camera 3 may take two or more images at different times, with different angular positions of the trailer 2 relative to the vehicle 1. For example, a series of images may be taken. This image sequence may include three or more , in particular more than five images.

In this example, the second image may show the orientation of the trailer 2 relative to the vehicle at a yaw angle YA=0°. However , in other embodiments, yaw angle YA may be any other reference yaw angle that is known and can be used to determine the current yaw angle.

In the image taken by camera 3, a plurality of different features may be distinguishable. In FIG. 4, features F1, F2 are shown identified in different angular positions relative to the position of the camera 3 of the vehicle 1 or the reference axis . The first feature F1 is indicated by a square and the second feature by a triangle. It is worth mentioning that more than two features and more than two images can be used for yaw angle estimation. It is also possible to use only one feature for yaw angle estimation.

In this way, the upper pair of first and second features F1, F2 (associated with the solid ray connecting features F1, F2 with camera 3) may be identified in the first image, and the first and second features The lower pair of F1, F2 (associated with the dashed ray connecting the features F1, F2 with the camera 3) may be identified in the second image at different times.

Also, in the second algorithm, the features of the trailer 2 may be located and matched using a feature detection and matching algorithm. For example, Harris Corner Detection Method, Scale Invariant Feature Transform (SIFT) Algorithm, Speed Up Robust Features (SURF) Algorithm, Binary Robust Invariant Scalable Keypoint (BRISK) Algorithm, Binary Robust Independent Fundamental Features (BRIEF), Directed FAST and The Rotated BRIEF (ORB) algorithm or other suitable feature detection and matching algorithm may be used .

Feature detection and matching algorithms may detect image features that are on the trailer or not on the trailer. A number of different methods may be used to separate trailer features from non- trailer features. For example, in the case of straight-line forward travel, trailer features may be separated from non-trailer features by looking for features that remain in the same location over time. Alternatively, the motion of background features may be modeled over time using the known motion of the vehicle. This may be extracted from CAN data regarding speed and steering. In this case, features that do not apply to the epipolar constraint of the fundamental matrix may be considered as trailer features.

It is worth mentioning that the first and second algorithms may use the same detected features for determining the angle estimates α1, α2. Therefore, in other words, feature detection only needs to be performed once for both algorithms.

To determine the angle estimates α1, α2, the ray R connecting the features F1, F2 with the camera 3 is used . In order to associate the captured image features F1, F2 with the position of the camera 3, the location of the features F1, F2 in the image coordinates may be transformed into the spatial domain of the camera 3 using the calibration information of the camera 3, thereby , it becomes possible to provide a ray R connecting the location of each feature F1, F2 with this camera position. In other words, in order to associate the camera position and the feature positions , the positions of the features F1, F2 on the image are determined based on the calibration information of the camera 3 in the local area of the vehicle 1, respectively of the camera 3 of the vehicle 1. Transformed in local area.

After determining the ray R between the camera position and the one or more features in the first and second images, a rotation angle of at least one of the first feature F1 and the second feature F2 is determined. . In FIG. 4, α1 indicates the angle estimation of the rotation angle of the first feature F1 between two photographed images, and α2 indicates the angle estimation of the rotation angle of the second feature F2 between these images. In embodiments, only one or two or more features of the trailer are determined and tracked across multiple images. Furthermore, preferably more than two images are taken at different times in order to improve the results of the yaw angle estimation.

As described above, one of the captured images may serve as a reference image in which the angular position of the trailer 2 with respect to the vehicle 1 is known. In this known angular configuration of the trailer 2 relative to the tow vehicle 1, the yaw angle YA may be 0° or any other angular value. Therefore, the yaw angle YA may be calculated based on this at least one angle estimate α1, α2. Referring again to FIG. 4, for example, the angular configuration of the light ray R indicated by the dashed line may be known because when an image is taken with reference to this light ray R, the trailer 2 has a known angular configuration with respect to the vehicle 1. This is because it has a reference direction.

The second algorithm described above is very robust, i.e. provides angle estimation even in the case of poor image quality, but the accuracy of the angle estimation is low.

In this disclosure, for each feature F1, F2, one of the presented algorithms (first or second algorithm) may be selected and used to provide at least one angle estimate α1, α2. good. This selection may be made by examining one or more conditions.

In preferred embodiments , a first algorithm using at least one orthogonal bisector B1, B2 (FIG. 2, FIG. 3) may preferably be used to determine at least one angle estimate α1, α2. . More specifically, a number of tests are performed in order to attempt to calculate at least one angle estimate α1, α2 based on this first algorithm. Depending on the results of this test , a first or second algorithm is used to determine an angle estimate for a particular feature.

The first condition checked is the length L of the base line BL used to create the orthogonal bisector (see FIG. 2). This base line BL may be a straight line connecting the locations of the specific features F1 and F2 in the first and second images. If the length L is less than a predetermined length threshold, the second algorithm is used instead of the first algorithm because the first algorithm may have significant noise.

The second condition checked is whether there is a change in the pitch and/ or roll of the trailer 2 due to uneven ground in the series of captured images. If the change in pitch and /or roll exceeds a predetermined pitch or roll threshold, the first algorithm is used instead of the second algorithm because the second algorithm may have significant noise. be.

In addition , a third condition to be checked is the distance of the features F1, F2 in the perpendicular direction to said horizontal plane, which constitutes a reference plane in which the features are transformed before determining the at least one angle estimate α1, α2. If the vertical distance of a feature with respect to this reference plane is less than a predetermined distance threshold, the second algorithm is used instead of the first algorithm because the first algorithm can have significant noise. be.

In this way, if one or more of the above criteria or conditions tested are fulfilled, the first algorithm is skipped and a second algorithm is used, in which the at least one angle estimate α1, α2 It is determined.

A separate determination may be made for each feature F1, F2 based on which algorithm the at least one angle estimate α1, α2 is determined. Therefore, in other words, the angle estimates α1, α2 may be calculated based on the same or different algorithms.

Under ideal conditions, when setting up multiple angle estimates based on an algorithm, the first angle estimate α1 and at least one further angle estimate α2 are equal (α1=α2) and are indicative of the yaw angle YA. It is. However , due to noise and discrepancies , the values of the angle estimates α1, α2 may be different. It is worth mentioning that more than two angle estimates can be configured to improve the quality of the yaw angle determination.

Statistical measurements may be used to determine the yaw angle YA based on multiple angle estimates α1, α2 having different values. In a first embodiment, the median value of two or more angle estimates α1, α2 may be used to determine the yaw angle YA. In other embodiments, statistical methods may be used to determine the yaw angle YA based on two or more angle estimates α1, α2. The statistical method may be, for example, a RANSAC algorithm (RANSAC: random sample consensus) or a least squares algorithm.

It appeared that not all features visible on the captured image were equally suitable for calculating the yaw angle YA. In order to reduce computational complexity and robustness, such features are selected and further used to determine the yaw angle YA, which results in a turning angle that is very close to the actual yaw angle. For feature selection, only these features are tracked in subsequent images, resulting in angle estimates α1, α2 in a fixed window around the actual yaw angle. For example, a window may be defined by an upper boundary and a lower boundary, where the upper and lower boundaries define an angular window around this actual yaw angle. For example, the window may extend over a distance of 2° to 10°, especially 3° to 5°. All features that led to a turning angle within this window in the last two or more yaw angle determination steps are further tracked in the next captured image.

When tracking a particular trailer feature for multiple images, due to the movement of the trailer 2, samples of this feature may be arranged in circular segments. The center of this circular segment indicates the location of the tow ball 4. Thus, by tracking specific trailer features across multiple images, the location of the tow ball 4 can be derived .

To mitigate noise, the determination of the location of the towball 4 may take into account multiple trailer features tracked over a period of time over multiple images. Each trailer feature may correspond to a circular segment with a predetermined center estimate. By applying this statistical method for multiple center estimates, the actual location of the tow ball 4 can be developed . The statistical method may be, for example, a RANSAC algorithm or a least squares algorithm.

FIG. 5 shows a block diagram illustrating the method steps of a method for determining the yaw angle YA of the trailer 2 with respect to the longitudinal axis LAV of the tow vehicle 1.

As a first step, first and second images of the trailer are taken (S10).

After image capture, at least one feature of the trailer visible on the first and second images is determined (S11).

Furthermore, at least first and second algorithms are provided for calculating at least one angle estimate (S12).

At least a first angle estimate is provided for the first feature. In a first option, determining whether to use a first or a second algorithm to calculate the angle estimate based on one or more criteria for the first feature, and selecting for the first feature. The at least one angle estimate is provided by calculating the first angle estimate based on the algorithm determined (S13A).

In a second option, an angle estimate is calculated for this first feature based on this first and second algorithm, and the angle estimate obtained by the first algorithm or the angle estimate obtained by the second algorithm is used for further processing. The at least one angle estimate is provided by determining whether to use it based on one or more criteria (S13B).

Finally, a yaw angle is calculated based on the at least one angle estimate (S14).

It should be noted that the specification and drawings merely illustrate the principles of the proposed invention. Those skilled in the art will be able to implement various configurations that embody the principles of the invention, even if not explicitly described or illustrated in this disclosure.

1 Vehicle 2 Trailer 3 Camera 4 Towball
α1 First angle estimation α2 Second angle estimation B1 First orthogonal bisector B2 Second orthogonal bisector BL Base line F1 First feature F2 Second feature IP1 First intersection IP2 Second intersection L Length LAT Trailer anteroposterior axis of
LAV Vehicle longitudinal axis
R Ray YA Yaw angle

Claims (15)

A method for determining a yaw angle (YA) of a trailer (2) with respect to a longitudinal axis (LAV) of a towing vehicle (1), the method comprising:
- taking at least first and second images of said trailer (2) using a camera (3), the orientation of said trailer (2) relative to said vehicle (1) being different on said at least two images; Step (S10),
- determining (S11) at least a first feature (F1) of said trailer (2) visible on a first and second image;
- providing (S12) at least first and second algorithms for calculating at least one angle estimate based on said at least first feature;
- providing at least a first angle estimate (α1) for said first feature (F1) ,
- determining whether to use the first or the second algorithm for calculating the first angle estimate (α1) based on one or more criteria for each feature (F1); a step (S13A) of calculating said first angle estimate (α1) based on an algorithm selected for F1) ;
- Calculate a first angle estimate (α1) for the first feature (F1) based on the first and second algorithms, and calculate the first angle estimate (α1) obtained by the first algorithm or the Determining based on one or more criteria whether the first angle estimate (α1) obtained by the second algorithm is used for further processing (S13B)
providing at least a first angle estimate (α1) for said first feature (F1) by;
- a step (S14) of calculating said yaw angle (YA) based on said first angle estimate (α1 ); How to determine angle (YA) . The first algorithm is
- obtaining a first projected feature position (PFP1a) by projecting a ray between the camera (3) and the determined first feature on the first image onto a horizontal plane; and the determined first feature on the second image by projecting a light ray onto the horizontal plane, obtaining a second projected feature position (PFP1b),
- setting a first orthogonal bisector (B1) between the location of the first projected feature position (PFP1a) and the location of the second projected feature position (PFP1b);
- determining a first point of intersection (IP1) of the first orthogonal bisector (B1) with the reference axis or a further orthogonal bisector;
- an angle between a first line from the first projected feature position (PFP1a) to the first intersection (IP1) and a second line from the second projected feature position (PFP1b) to the first intersection (IP1); 2. The method according to claim 1, wherein the method is configured to calculate a first angle estimate (α1) , which is an angle on the horizontal plane. A second feature (F2) of the trailer (2) visible on the first and second images is determined, the second feature (F2) being different from the first feature (F1) of the trailer (2). the first algorithm uses two or more features of the trailer (2) to provide an angle estimation;
- obtaining a third projected feature position (PFP2a) by projecting a ray between the camera (3) and the determined second feature (F2) on the first image onto a horizontal plane; (3) and the determined second feature (F2) on the second image by projecting the light ray onto the horizontal plane to obtain a fourth projected feature position (PFP2b),
- setting a second orthogonal bisector (B2) between the location of the third projection feature position (PFP2a) and the location of the fourth projection feature position (PFP2b);
- determining a second point of intersection (IP2) of a second orthogonal bisector (B2) with a reference axis, said first orthogonal bisector (B1) or a further orthogonal bisector;
- the angle between the first line from the third projected feature position (PFP2a) to the second intersection (IP2) and the second line from the fourth projected feature position (PFP2b) to the second intersection (IP2); 3. The method according to claim 2, configured to calculate a second angle estimate (α2), which is an angle on the horizontal plane. Said second algorithm is configured to calculate a first angle estimate (α1), said first angle estimate (α1) being said first angle estimate (α1) on said first image relative to a fixed point of said towing vehicle (1). 4. The method according to claim 1, characterized in that the angle of rotation in a horizontal plane between one feature (F1) and the first feature (F1) on the second image is characterized. A second feature (F2) of the trailer (2) visible on the first and second images is determined, the second feature (F2) being different from the first feature (F1) of the trailer (2). located at a position, said second algorithm is configured to calculate a second angle estimate (α2), said second angle estimate (α2) 5. The method according to claim 4, characterized by a turning angle in a horizontal plane between the second feature (F2) on one image and the second feature (F2) on the second image. The step of determining based on one or more criteria includes determining the base line (BL) between the features (F1, F2) on the first image and the features (F1, F2) on the second image. ) and comparing said length (L ) with a length threshold. The step of determining based on one or more criteria includes determining at least one of roll and pitch of the trailer (2), comparing the roll to a roll threshold and comparing the pitch to a pitch threshold. 7. A method according to any one of claims 1 to 6, comprising at least one of comparing. The step of determining based on one or more criteria includes determining a vertical distance of the features (F1, F2) with respect to a horizontal reference plane and comparing the vertical distance with a distance threshold. 8. A method according to any one of claims 1 to 7, comprising : In said second algorithm, calculating at least one angle estimate (α1, α2) comprises calculating the ray (R) between said fixed point and said at least one feature (F1, F2) in the first and second images. 9. A method according to any one of claims 1 to 8, comprising determining: ). converting the position of at least one of the first feature and second feature (F1, F2) from a local area of the image to a local area of the vehicle (1) in order to determine the light ray (R); 10. The method of claim 9, wherein camera calibration information is used . 11. Any one of claims 1 to 10, wherein, in addition to the first and second features (F1, F2), at least one further feature of the trailer (2) is used to calculate the yaw angle (YA). or the method described in paragraph 1. The yaw angle (YA) is determined by setting a median value based on the at least two angle estimates, by setting an average value of the at least two angle estimates , or by a statistical method applied to the angle estimate. 12. A method according to any one of claims 1 to 11, wherein the method is calculated by using . determining an angular window, the angular window having an upper bound and a lower bound around a yaw angle (YA), determining a set of features that lead to a plurality of angle estimates within the angular window ; 13. A method according to any preceding claim, wherein the determined set of features is used for future yaw angle (YA) calculations. A system for determining a yaw angle (YA) of a trailer (2) with respect to a longitudinal axis (LAV) of a towing vehicle (1), the system comprising:
The system comprises a camera (3) that captures images of the trailer (2) and a processing entity that processes the captured images,
The system is
- taking at least first and second images of said trailer (2) using a camera (3), the orientation of said trailer (2) relative to said vehicle (1) being different on said at least two images; Step (S10),
- determining (S11) at least a first feature (F1) of said trailer (2) visible on a first and second image;
- providing (S12) at least first and second algorithms for calculating at least one angle estimate based on said at least first feature;
- providing at least a first angle estimate (α1) for said first feature (F1),
- determining whether to use the first or the second algorithm for calculating the first angle estimate (α1) based on one or more criteria for each feature (F1); calculating the first angle estimate (α1) based on an algorithm selected for the feature (F1) (S13A); or
- Calculate a first angle estimate (α1) for the first feature (F1) based on the first and second algorithms, and calculate the first angle estimate (α1) obtained by the first algorithm or the Determining based on one or more criteria whether the first angle estimate (α1) obtained by the second algorithm is used for further processing (S13B)
providing at least a first angle estimate (α1) for said first feature (F1) by;
- a longitudinal axis (LAV) of the towing vehicle (1), further configured to perform the step (S14) of calculating said yaw angle (YA) based on said first angle estimate ( α1 ); A system for determining the yaw angle (YA) of the trailer (2) relative to . A vehicle comprising a system according to claim 14.