WO2018202464A1 - Calibration of a vehicle camera system in vehicle longitudinal direction or vehicle trans-verse direction - Google Patents
Calibration of a vehicle camera system in vehicle longitudinal direction or vehicle trans-verse direction Download PDFInfo
- Publication number
- WO2018202464A1 WO2018202464A1 PCT/EP2018/060393 EP2018060393W WO2018202464A1 WO 2018202464 A1 WO2018202464 A1 WO 2018202464A1 EP 2018060393 W EP2018060393 W EP 2018060393W WO 2018202464 A1 WO2018202464 A1 WO 2018202464A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera
- top view
- vehicle
- image data
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
- B60R2300/402—Image calibration
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Definitions
- the invention relates to a method for calibrating a camera device of a vehicle including generating a reference top view to an environment of the vehicle from image data of a reference camera of the camera device and including generating a further top view to the environment from image data of a further camera of the camera device, wherein the image data of the further camera is captured in a preset time interval after the image data of the reference camera.
- the invention also relates to a camera device for a vehicle with a reference camera and a further camera for capturing respective image data and with a computing unit, which is formed to generate a reference top view to an environment of the camera from image data of the reference camera and to generate a further top view to the environment of the camera device from image data of the further camera, wherein the image data of the further camera is captured in a presettable time interval after the image data of the reference camera.
- the interest is directed to calibration methods, in particular to extrinsic self- calibration methods without target objects, for cameras of a motor vehicle.
- the cameras are adapted such that they represent the three-dimensional environmental region of the motor vehicle in a two-dimensional image or picture, for example a top view. Therefore, a camera model is usually provided to image the 3D world coordinates of the environmental region to 2D pixel coordinates of the image.
- the camera model usually depends on intrinsic camera parameters like the focal length, the image sensor format and the image center, and extrinsic camera parameters, like the orientation and the position of the camera in vehicle vertical direction, the height of the camera, which are also referred to as posture (including position and orientation) of the camera, wherein the extrinsic parameters are generally set by specification.
- the respective positions of the cameras in a vehicle longitudinal direction and a transverse direction also play a role as further extrinsic parameters, for example to be able to obtain a coherent picture of the environment from the images of the different cameras.
- extrinsic parameters may not represent the real and current extrinsic parameters of the camera, for example due to mechanical tolerances in the vehicle production process, due to an Airmatic suspension or due to loading variations in the vehicle such as e.g. the number of passengers or the weight in a trunk of the vehicle.
- a camera which is attached or coupled to the vehicle afterwards, for example a camera of a trailer of the vehicle, the position in vehicle longitudinal and transverse direction is also generally unknown.
- the disadvantage is in that these methods do not provide sufficient information for a 6- DOF calibration of cameras, thus for the three rotational and the three translational degrees of freedom.
- the known approaches for calibrating a camera device in a vehicle which are based on top views to an environment, typically ground views of a road, on which the vehicle moves, generally relate to the four extrinsic parameters or calibration parameters of the three rotational axes and the vehicle vertical direction (height).
- Such methods are for example described in the US 2009 290 032 A1 , WO 2012/139 636 A1 and WO 12 139 660 A1 .
- Such approaches are sufficient for cameras, which are installed at a motor vehicle ex factory since the position of respective cameras of the camera device in the vehicle transverse and vehicle longitudinal direction can be accurately ascertained from constructional drawings, for example CAD (Computer Aided Design) drawings.
- the camera Since trailers have different lengths, the camera has to be calibrated with respect to its distance in the vehicle longitudinal direction such that the longitudinal position, thus the distance of the camera from the towing vehicle and thereby at least one further camera of the camera device, is known.
- exact instructions in an installation manual are not sufficient since the cameras are usually installed by the end consumer, whereby great individual fluctuations in the respective positions arise. Therefore, exact CAD drawings either are not a reliable aid.
- the object arises to calibrate a camera device with respect to a position of a camera of the camera device in automated manner, in particular to reduce user inputs required for calibration to the bare necessities.
- the invention relates to a method for calibrating a camera device of a vehicle.
- the vehicle can be a motor vehicle or a vehicle/trailer combination with a motor vehicle as a towing vehicle and a trailer or a trailer.
- calibrating the camera device here, calibrating at least one camera of the camera device can be understood.
- the calibration of the camera can in turn be understood as calibration of the position (and/or the orientation) of the camera, providing information about the position (and/or orientation) of the camera, which allows classifying image data captured by the camera together with the image data of other calibrated cameras in a global reference system internal to camera device to thus be able to arrive at a consistent overall image data set or overall view from the image data sets or views of multiple cameras.
- a view can for example be a top view.
- a first method step is generating at least one reference top view to an environment of the vehicle from image data of a reference camera of the camera device.
- the reference camera can be an already calibrated camera, the six extrinsic parameters of which are already known and recorded in the camera device.
- the reference camera can be disposed at the towing vehicle of the vehicle/trailer combination if the vehicle is a vehicle/trailer combination.
- a further method step is generating at least one further top view to the environment from image data of a further camera of the camera device.
- calibrating or a calibration of the other four extrinsic parameters of the reference camera and/or the further camera can be effected, thus height correction (calibration with respect to the position of the camera in vehicle vertical axis) and rotational correction (calibration with respect to the orientation of the camera in the three rotational axes).
- calibration of the reference camera in the further two extrinsic parameters can also be effected, thus a length correction (calibration with respect to the position of the camera in vehicle longitudinal axis) and transverse correction (calibration with respect to the position of the camera in vehicle transverse axis).
- the calibration can be effected by the known Motion Tracking Calibration (MTC).
- MTC Motion Tracking Calibration
- the MTC approach is based on the motion vectors, which can be calculated in a top view, thus from a bird's eye perspective to an environment of the camera and the vehicle, respectively, to perform a rotational and height correction for a camera.
- the further camera can therein in particular be disposed at a trailer of the vehicle/trailer combination.
- the image data of the further camera is at least partially or completely captured in a preset time interval, which is greater than/equal to zero, preferably greater than zero, after the image data of the reference camera.
- the image data of the further camera is preferably captured with a temporal delay.
- the perspective of the two cameras to the environment is different, in particular disjoint. Thereby, in simultaneous capture of image data by the two cameras respectively with a disjunctive perspective, exclusively areas of the environment are captured or imaged, which are not imaged by the respectively other camera. With only different, but not disjointly different perspectives, partial overlap can here also be present.
- a spatial correlation of the reference top view and the further top view based on an image comparison of reference top view and further top view by a computing unit of the camera device is important.
- finding of matching image parts in the different top views can be understood by a spatial correlation, for example computing a cross-correlation. If the preset time interval is greater than zero, thus, only top views, which are based on image data captured at different points of time, are correlated.
- a further method step is calculating an offset of the further top view relative to the reference top view in a vehicle longitudinal direction and/or a vehicle transverse direction of the vehicle, in particular of the trailer, by the computing unit depending on a result of correlation.
- automatic calibration of the further camera relative to the reference camera is finally effected based on the calculated offset by the computing unit.
- the calculated offset can represent the posture of the perspective of the further camera relative to the reference cam- era.
- the invention is based on the realization that the effect of a deviation of a position of a camera in vehicle longitudinal direction and/or vehicle transverse direction only minimally affects the motion vectors, which usually are used in the known approaches for calibration, for example the motion tracking calibrations, since an error in vehicle longitudinal direction and/or vehicle transverse direction only results in shifting the top view. Therefore, a motion vector is not changed in consecutive individual images, so-called frames, by the shift, the offset of the top view.
- the used cameras should be calibrated in height and rotational degrees of freedom before generating the top views. If the cameras used are not calibrated in their height, in particular not matched to each other, the different top views are differently severely magnified relative to each other, which aggravates the spatial correlation. If the rotational degrees of freedom are not calibrated, in particular matched to each other, thus, this results in distortions in the respective top views, which again aggravate the spatial correlation.
- multiple reference top views and/or multiple further top views are generated and buffered and the correlation of the top views is performed in pairs for the one reference top view and one further top view.
- a pair of top views is respectively evaluated with a quality criterion and the offset is calculated depending on a result of the correlation of the pair of the top views with the best value for the quality criterion.
- the offset can be calculated based on the reference top view and the further top view with the highest correlation, thus based on the top views, which best fit to each other.
- the distance can then also be calculated within the scope of calculating the offset, which the vehicle has traveled between the two top views.
- the spatial distance of the top views and thereby of the cameras can be calculated from the relative position of the image parts in the respective top view and the traveled distance.
- the method steps of generating the reference top view, generating the further top view, spatially correlating and calculating the offset are repeatedly performed.
- the mentioned method steps can be repeatedly performed with one and the same reference camera or else with at least one (thus one or more) further reference camera of the camera device.
- multiple cameras of the towing vehicle can for example be used as the reference cameras and each one camera disposed at the trailer can be used as the further camera.
- multiple cameras of the trailer can thus also be successively calibrated, wherein a calibrated camera of the trailer can then in particular also be used as the reference camera.
- the automatic calibration is performed based on a plurality of calculated offsets, in particular based on an offset averaged from the calculated offsets.
- the time interval is preset depending on a minimum and/or maximum expected distance of the further camera from the reference camera, for example a presettable minimum and/or maximum trailer length, and a traveling speed of the vehicle.
- the time interval can also be preset depending on a sampling rate for the image data.
- the time interval can in particular also be preset depending on odometric data of the vehicle, preferably of the towing vehicle.
- the computing unit can calculate the position of the further camera relative to the reference camera and thus determine the position of the further camera and for example the length of the trailer.
- the image comparison is performed with an area matching algorithm, a so-called block matching algorithm, in certain areas.
- this can be effected with calculation of the sum of the absolute differences and/or calculation of the sum of the squared absolute differences in the respective areas of the top views.
- the differences can for example relate to grey values of pixels or other pixel characteristics of the top views.
- the offset in vehicle transverse direction is calculated as proportional to a pixel to millimeters ratio of the further top view.
- the pixel to millimeters ratio can be calculated from height and resolution of the respective camera.
- the pixel to millimeters ratio of the further top view is preferably identical to the pixel to millimeters ratio of the reference top view.
- the offset can for example be calculated from the difference of the two pixels, thus presently two pixels modified with the pixel to millimeters ratio.
- the offset in vehicle longitudinal direction is calculated as proportional to a pixel to millimeters ratio of the further top view plus the distance traveled by the vehicle in the time interval.
- the traveled distance can be ascertained from information capable of being retrieved via the CAN bus like the vehicle speed and the time interval.
- the time interval is used for the top views used for calculating the offset.
- the offset can for example be calculated from the difference of the two pixels, thus presently two pixels modified with the pixel to millimeters ratio plus the traveled distance of for example 1500 mm.
- This absolute value is particularly well suitable for the calibration of the camera since the offset is thus calculated in a global reference system.
- the vehicle includes a trailer, in particular is a vehicle/trailer combination, and the further camera is disposed at or attached to the trailer.
- the reference camera can in particular be disposed at the towing vehicle of the vehicle/trailer combination.
- the computing unit calculates information about a length of the trailer from a know position of the reference camera and a position of the further camera known after calibration of the further camera and provides this information.
- the invention also relates to a camera device for a vehicle with a reference camera and a further camera for capturing respective image data, a computing unit, which is formed to generate a reference top view to an environment of the camera device and thereby with intended use of the camera device also to an environment of the vehicle from the image data of the reference camera and to generate a further top view to the environment from the image data of the further camera, wherein the image data of the further camera is cap- tured in a presettable time interval after the image data of the reference camera.
- the computing unit is formed to spatially correlate the reference top view and the further top view based on an image comparison of reference top view and further top view, to calculate an offset of the further top view relative to the reference top view depending on a result of correlation and to automatically calibrate the further camera relative to the reference camera based on the calculated offset.
- the calculated offset is therein an offset in a vehicle longitudinal direction and/or in a vehicle transverse direction.
- advantages and advantageous embodiments of the camera devices correspond to advantages and advantageous embodiments of the method for calibrating the camera device.
- the invention also relates to a vehicle with a camera with such a camera device.
- Fig. shows a motor vehicle with an exemplary embodiment of a camera device, based on which the method for calibrating is exemplarily explained.
- a vehicle 1 which is presently formed as a vehicle/trailer combination with a motor vehicle 2 as the towing vehicle and a trailer 3, includes a camera device 4.
- the camera device 4 is equipped with at least one, presently four reference cameras 5a-5d, which are suitable for capturing respective image data.
- respective reference top views 7a-7d to an environment 10 of the vehicle 1 can be generated by a computing device 6.
- the camera device 4 also comprises a further camera 8a for capturing corresponding image data.
- the further camera 8a is presently disposed at a rear of the trailer 3.
- the computing unit 6 is formed to generate a further top view 9a to an environment 10 of the vehicle 1 and the camera device 4, respectively, also from the image data of the at least one further camera 8a.
- the image data of the further camera 8a is captured in a presettable time interval ⁇ after the image data of the reference cameras 5a-5d, which is greater than zero in the shown example.
- the computing unit 6 is formed to spatially correlate at least one of the reference top views 7a- 7d and the further top view 9a based on an image comparison of the corresponding reference top view and the further top view 9a as well as to calculate an offset dx in vehicle longitudinal direction x and/or an offset dy in vehicle transverse direction y depending on a result of correlation and to automatically calibrate the further camera 8a relative to at least one of the reference cameras 5a- 5d, for example to the rear-side reference camera 5c, based on the calculated offset dx, dy.
- the reference camera 5a-5d for example the rear-side camera 5c or else the front-side camera 7a or one of the two lateral cameras 5b, 5d can thus be used for calibrating. Multiple or all of the reference cameras 5a to 5d can also be used to reduce an error in calibration.
- an exemplary embodiment of a method for calibrating the camera device 4 based on the rear-side reference camera 5c as the selected reference camera 5a- 5d is now to be described.
- the reference top view 7c to the environment 10 of the vehicle 1 is now first generated from the image data of the reference camera 5c.
- the preset time interval ⁇ can for example be adapted to the traveling speed v and a minimum or maximum expected distance of the further camera 8a from the reference camera 5c and/or to a sampling rate for the image data and/or multiple reference top views 7c and multiple further top views 9a can be generated, respectively, from which the suitable pair of reference top view 7c and further top view 9a is selected.
- the top views 7c, 9a are correlated in pairs and each pair is evaluated with a quality criterion such that an offset dx, dy of the further top view 9a relative to the reference top view 7c is calculated depending on the correlation of the pair of the top views with the best value for the quality criterion. If only the reference top view 7c and the further top view 9a is used, thus, they are spatially correlated based on an image comparison by the computing unit 6 and then the offset dx, dy of the further top view 9a to the reference top view 7c is calculated depending on a result of this correlation.
- the offset in vehicle transverse direction y can be calculated as proportional to a pixel to millimeters ratio r of the top views 7c and 9a:
- the offset in vehicle longitudinal direction x is calculated as proportional to the pixel to millimeters ratio r plus the distance traveled by the vehicle 1 in the time interval ⁇ .
- a pixel is again associated with the feature 1 1 and 1 1 ', respectively.
- This pixel j and j', respectively, is presently the fiftieth and seventieth pixel, respectively, in the x-direction.
- the distance traveled by the vehicle in the time interval ⁇ is added to a relative shift of the feature 1 1 and 1 1 ', respectively, within the top view 7c and 9a, respectively.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a method for calibrating a camera device (4) of a vehicle (1), including a) generating a reference top view (7a- 7d) to an environment (10) of the vehicle (1) from image data of a reference camera (5a-5d) of the camera device (4); b) generating a further top view (9a) to the environment (10) from image data of a further camera (8a) of the camera device (4), wherein the image data of the further camera (8a) is captured in a presettable time interval (Δt) after the image data of the reference camera (5a-5d); c) spatially correlating the reference top view (7a-7d) and the further top view (9a) based on an image comparison of reference top view (7a-7d) and further top view (9a) by a computing unit (6) of the camera device (4); and d) calculating an offset (dx, dy) of the further top view (9a) relative to the reference top view (7a-7d) in a vehicle longitudinal direction (x) or a vehicle transverse direction (y) by the computing unit (6) depending on a result of the correlation; as well as e) automatically calibrating the further camera (8a) relative to the reference camera (5a- 5d) based on the calculated offset (dx, dy) to calibrate the camera device (4) with respect to a position of a camera (8a) of the camera device (4) in automated manner.
Description
Calibration of a vehicle camera system in vehicle longitudinal direction or vehicle transverse direction
The invention relates to a method for calibrating a camera device of a vehicle including generating a reference top view to an environment of the vehicle from image data of a reference camera of the camera device and including generating a further top view to the environment from image data of a further camera of the camera device, wherein the image data of the further camera is captured in a preset time interval after the image data of the reference camera. The invention also relates to a camera device for a vehicle with a reference camera and a further camera for capturing respective image data and with a computing unit, which is formed to generate a reference top view to an environment of the camera from image data of the reference camera and to generate a further top view to the environment of the camera device from image data of the further camera, wherein the image data of the further camera is captured in a presettable time interval after the image data of the reference camera.
Presently, the interest is directed to calibration methods, in particular to extrinsic self- calibration methods without target objects, for cameras of a motor vehicle. The cameras are adapted such that they represent the three-dimensional environmental region of the motor vehicle in a two-dimensional image or picture, for example a top view. Therefore, a camera model is usually provided to image the 3D world coordinates of the environmental region to 2D pixel coordinates of the image. The camera model usually depends on intrinsic camera parameters like the focal length, the image sensor format and the image center, and extrinsic camera parameters, like the orientation and the position of the camera in vehicle vertical direction, the height of the camera, which are also referred to as posture (including position and orientation) of the camera, wherein the extrinsic parameters are generally set by specification. Especially in use of multiple cameras, the respective positions of the cameras in a vehicle longitudinal direction and a transverse direction also play a role as further extrinsic parameters, for example to be able to obtain a coherent picture of the environment from the images of the different cameras.
These extrinsic parameters may not represent the real and current extrinsic parameters of the camera, for example due to mechanical tolerances in the vehicle production process, due to an Airmatic suspension or due to loading variations in the vehicle such as e.g. the number of passengers or the weight in a trunk of the vehicle. Especially for a camera, which is attached or coupled to the vehicle afterwards, for example a camera of a trailer of
the vehicle, the position in vehicle longitudinal and transverse direction is also generally unknown.
These incorrect extrinsic parameters can have a negative impact on the rendering process or the image synthesis. Accordingly, it is required to calibrate the camera to obtain reliable images, which can for example be provided to a driver assistance system of the motor vehicle. The driver assistance system can then for example reliably recognize objects or obstacles for the motor vehicle in the environmental region of the motor vehicle by means of the images.
Most of the online calibration methods without target objects for cameras mounted on vehicles, which are known from the prior art, rely on the presence of features, which can be tracked on a road surface or in the environment over a sequence of pictures or frames from one or more vehicle-side cameras. Typically, line markings are used as features and/or trajectories of the features are used in a sequence of frames. The known methods usually require turning maneuvers of the vehicle in addition to a straight movement to recover the complete orientation of the camera in three degrees of freedom (DOF) as well as the odometry of the vehicle as an additional input to resolve the calibration.
The disadvantage is in that these methods do not provide sufficient information for a 6- DOF calibration of cameras, thus for the three rotational and the three translational degrees of freedom. The known approaches for calibrating a camera device in a vehicle, which are based on top views to an environment, typically ground views of a road, on which the vehicle moves, generally relate to the four extrinsic parameters or calibration parameters of the three rotational axes and the vehicle vertical direction (height).
Such methods are for example described in the US 2009 290 032 A1 , WO 2012/139 636 A1 and WO 12 139 660 A1 . Such approaches are sufficient for cameras, which are installed at a motor vehicle ex factory since the position of respective cameras of the camera device in the vehicle transverse and vehicle longitudinal direction can be accurately ascertained from constructional drawings, for example CAD (Computer Aided Design) drawings.
However, there are also cameras, which are not yet installed at the vehicle ex factory. For example, the known approaches reach their limits if a camera of the camera device is fixed to a trailer. Typically, such cameras for a trailer are purchased separately from the trailer after delivery thereof and attached to it. Therein, trailers inherently have different
sizes and shapes. Therefore, exact instructions are generally given in an installation manual, in which height and in which lateral position, thus in which distance relative to a central vehicle or trailer longitudinal axis, the respective cameras are to be installed. In any case, a camera therein has to be installed at a rear of the trailer. Since trailers have different lengths, the camera has to be calibrated with respect to its distance in the vehicle longitudinal direction such that the longitudinal position, thus the distance of the camera from the towing vehicle and thereby at least one further camera of the camera device, is known. However, even exact instructions in an installation manual are not sufficient since the cameras are usually installed by the end consumer, whereby great individual fluctuations in the respective positions arise. Therefore, exact CAD drawings either are not a reliable aid.
The object arises to calibrate a camera device with respect to a position of a camera of the camera device in automated manner, in particular to reduce user inputs required for calibration to the bare necessities.
This object is solved by the subject matters of the independent claims. Advantageous embodiments are apparent from the dependent claims, the description and the figure.
The invention relates to a method for calibrating a camera device of a vehicle. Therein, the vehicle can be a motor vehicle or a vehicle/trailer combination with a motor vehicle as a towing vehicle and a trailer or a trailer. By calibrating the camera device, here, calibrating at least one camera of the camera device can be understood. The calibration of the camera can in turn be understood as calibration of the position (and/or the orientation) of the camera, providing information about the position (and/or orientation) of the camera, which allows classifying image data captured by the camera together with the image data of other calibrated cameras in a global reference system internal to camera device to thus be able to arrive at a consistent overall image data set or overall view from the image data sets or views of multiple cameras. Here, a view can for example be a top view.
Therein, a first method step is generating at least one reference top view to an environment of the vehicle from image data of a reference camera of the camera device. Therein, the reference camera can be an already calibrated camera, the six extrinsic parameters of which are already known and recorded in the camera device. In particular, the reference camera can be disposed at the towing vehicle of the vehicle/trailer combination if the vehicle is a vehicle/trailer combination. A further method step is generating at least one further top view to the environment from image data of a further camera of the camera device.
Before generating the top views, calibrating or a calibration of the other four extrinsic parameters of the reference camera and/or the further camera can be effected, thus height correction (calibration with respect to the position of the camera in vehicle vertical axis) and rotational correction (calibration with respect to the orientation of the camera in the three rotational axes). In particular, calibration of the reference camera in the further two extrinsic parameters can also be effected, thus a length correction (calibration with respect to the position of the camera in vehicle longitudinal axis) and transverse correction (calibration with respect to the position of the camera in vehicle transverse axis). For example, the calibration can be effected by the known Motion Tracking Calibration (MTC). The MTC approach is based on the motion vectors, which can be calculated in a top view, thus from a bird's eye perspective to an environment of the camera and the vehicle, respectively, to perform a rotational and height correction for a camera.
If the vehicle is a vehicle/trailer combination, the further camera can therein in particular be disposed at a trailer of the vehicle/trailer combination. Therein, the image data of the further camera is at least partially or completely captured in a preset time interval, which is greater than/equal to zero, preferably greater than zero, after the image data of the reference camera. Thus, the image data of the further camera is preferably captured with a temporal delay. Therein, the perspective of the two cameras to the environment is different, in particular disjoint. Thereby, in simultaneous capture of image data by the two cameras respectively with a disjunctive perspective, exclusively areas of the environment are captured or imaged, which are not imaged by the respectively other camera. With only different, but not disjointly different perspectives, partial overlap can here also be present.
Here, spatial correlation of the reference top view and the further top view based on an image comparison of reference top view and further top view by a computing unit of the camera device is important. Here finding of matching image parts in the different top views can be understood by a spatial correlation, for example computing a cross-correlation. If the preset time interval is greater than zero, thus, only top views, which are based on image data captured at different points of time, are correlated. Therein, a further method step is calculating an offset of the further top view relative to the reference top view in a vehicle longitudinal direction and/or a vehicle transverse direction of the vehicle, in particular of the trailer, by the computing unit depending on a result of correlation. Here, automatic calibration of the further camera relative to the reference camera is finally effected based on the calculated offset by the computing unit. Thus, the calculated offset can represent the posture of the perspective of the further camera relative to the reference cam-
era.
This has the advantage that all of the six extrinsic parameters, the three rotational degrees of freedom and the three translational degrees of freedom, of the further camera, for example a camera attached to a trailer, can now be automatically calibrated by an algorithm. Here, the proposed method for the two translational degrees of freedom not automatically calibrateable up to now can be used with the known method for calibrating the height and the three rotational degrees of freedom without problem. Moreover, here, from the position of the further camera at the trailer relative to a reference camera at the towing vehicle known thanks to the calibration, as explained in more detail below, there arises the possibility of calculating the length of a trailer. This information can then be provided to further devices in the motor vehicle or towing vehicle, for example a driver assistance device.
Therein, the invention is based on the realization that the effect of a deviation of a position of a camera in vehicle longitudinal direction and/or vehicle transverse direction only minimally affects the motion vectors, which usually are used in the known approaches for calibration, for example the motion tracking calibrations, since an error in vehicle longitudinal direction and/or vehicle transverse direction only results in shifting the top view. Therefore, a motion vector is not changed in consecutive individual images, so-called frames, by the shift, the offset of the top view.
In order that the spatial correlation can be performed in a simpler way, the used cameras should be calibrated in height and rotational degrees of freedom before generating the top views. If the cameras used are not calibrated in their height, in particular not matched to each other, the different top views are differently severely magnified relative to each other, which aggravates the spatial correlation. If the rotational degrees of freedom are not calibrated, in particular matched to each other, thus, this results in distortions in the respective top views, which again aggravate the spatial correlation.
In an advantageous embodiment, it is provided that multiple reference top views and/or multiple further top views are generated and buffered and the correlation of the top views is performed in pairs for the one reference top view and one further top view. Therein, a pair of top views is respectively evaluated with a quality criterion and the offset is calculated depending on a result of the correlation of the pair of the top views with the best value for the quality criterion. Thus, the offset can be calculated based on the reference top view and the further top view with the highest correlation, thus based on the top views, which
best fit to each other. Based on the selected top views and the time interval respectively associated with them, the distance can then also be calculated within the scope of calculating the offset, which the vehicle has traveled between the two top views. Thus, if identical image parts or features are recognized in the different top views, thus, the spatial distance of the top views and thereby of the cameras can be calculated from the relative position of the image parts in the respective top view and the traveled distance.
This has the advantage that assumptions on the position of the further camera do not have to be made at all for a successful calibration, in particular a trailer length can be unknown. However, if such information is still present, the accuracy of the method can be increased in the described embodiment.
In a further advantageous embodiment, the method steps of generating the reference top view, generating the further top view, spatially correlating and calculating the offset are repeatedly performed. Therein, the mentioned method steps can be repeatedly performed with one and the same reference camera or else with at least one (thus one or more) further reference camera of the camera device. Thus, multiple cameras of the towing vehicle can for example be used as the reference cameras and each one camera disposed at the trailer can be used as the further camera. Of course, multiple cameras of the trailer can thus also be successively calibrated, wherein a calibrated camera of the trailer can then in particular also be used as the reference camera. Therein, the automatic calibration is performed based on a plurality of calculated offsets, in particular based on an offset averaged from the calculated offsets.
This has the advantage that noise, which occurs in a dynamic environment as it is given with a moving vehicle, can be filtered out. Via the use of different reference cameras, falsifications optionally caused by camera can also be compensated for and the method can be increased in accuracy and speed.
In another advantageous embodiment, it is provided that the time interval is preset depending on a minimum and/or maximum expected distance of the further camera from the reference camera, for example a presettable minimum and/or maximum trailer length, and a traveling speed of the vehicle. In particular, the time interval can also be preset depending on a sampling rate for the image data. The time interval can in particular also be preset depending on odometric data of the vehicle, preferably of the towing vehicle.
This has the advantage that the generated top views are already implicitly preselected
such that after appearance of a specific image feature in the reference top view, it is already known when this image feature approximately appears in the further top view. Based on the traveling speed and/or odometric data and/or the time, which the specific image feature, for example a road marking on the road surface, takes to appear in the further top view after the reference top view, the computing unit can calculate the position of the further camera relative to the reference camera and thus determine the position of the further camera and for example the length of the trailer.
In a further advantageous embodiment, it is provided that the image comparison is performed with an area matching algorithm, a so-called block matching algorithm, in certain areas. In particular, this can be effected with calculation of the sum of the absolute differences and/or calculation of the sum of the squared absolute differences in the respective areas of the top views. Therein, the differences can for example relate to grey values of pixels or other pixel characteristics of the top views.
This has the advantage that matching image areas can be particularly easily identified such that the spatial correlation of the top views can be particularly simply and fast performed.
In a particularly advantageous embodiment, it is provided that the offset in vehicle transverse direction is calculated as proportional to a pixel to millimeters ratio of the further top view. For example, the pixel to millimeters ratio can be calculated from height and resolution of the respective camera. Therein, the pixel to millimeters ratio of the further top view is preferably identical to the pixel to millimeters ratio of the reference top view. Thus, with an identical pixel to millimeters ratio of the two top views and correlation for example of the fifth pixel in vehicle transverse direction starting from a reference pixel determining the position of the reference camera with the seventh pixel in the further top view, counted from a reference pixel representing the position of the further camera, the offset can for example be calculated from the difference of the two pixels, thus presently two pixels modified with the pixel to millimeters ratio. Thereby, one obtains the offset of the further camera in vehicle transverse direction relative to the reference camera in an absolute reference system.
Therein, the offset in vehicle longitudinal direction is calculated as proportional to a pixel to millimeters ratio of the further top view plus the distance traveled by the vehicle in the time interval. For example, the traveled distance can be ascertained from information capable of being retrieved via the CAN bus like the vehicle speed and the time interval. With
multiple top views, here, the time interval is used for the top views used for calculating the offset. Thus, with an identical pixel to millimeters ratio of the two top views and correlation for example of the fifth pixel in vehicle longitudinal direction starting from a reference pixel determining the position of the reference camera with the seventh pixel in the further top view, counted from a reference pixel representing the position of the further camera, the offset can for example be calculated from the difference of the two pixels, thus presently two pixels modified with the pixel to millimeters ratio plus the traveled distance of for example 1500 mm.
This has the advantage that the offset can be fast, simply and reliably calculated as an absolute value. This absolute value is particularly well suitable for the calibration of the camera since the offset is thus calculated in a global reference system.
In a particularly preferred embodiment, it is provided that the vehicle includes a trailer, in particular is a vehicle/trailer combination, and the further camera is disposed at or attached to the trailer. Therein, the reference camera can in particular be disposed at the towing vehicle of the vehicle/trailer combination. Under these circumstances, the described method is particularly advantageous since alternative approaches for configuration of the camera in vehicle longitudinal and vehicle transverse direction, respectively, are here not known.
Therein, it can be provided that the computing unit calculates information about a length of the trailer from a know position of the reference camera and a position of the further camera known after calibration of the further camera and provides this information.
This has the advantage that the length of the trailer can be particularly accurately calculated, namely up to millimeters, such that further devices of the vehicle, which fall back on the provided information, can automatically consider the correct length of the trailer without the assistance of a user for example in maneuvering within the scope of a parking assistance. Therein, additional hardware is advantageously not required.
The invention also relates to a camera device for a vehicle with a reference camera and a further camera for capturing respective image data, a computing unit, which is formed to generate a reference top view to an environment of the camera device and thereby with intended use of the camera device also to an environment of the vehicle from the image data of the reference camera and to generate a further top view to the environment from the image data of the further camera, wherein the image data of the further camera is cap-
tured in a presettable time interval after the image data of the reference camera.
Here, it is important that the computing unit is formed to spatially correlate the reference top view and the further top view based on an image comparison of reference top view and further top view, to calculate an offset of the further top view relative to the reference top view depending on a result of correlation and to automatically calibrate the further camera relative to the reference camera based on the calculated offset. With intended use of the camera device, the calculated offset is therein an offset in a vehicle longitudinal direction and/or in a vehicle transverse direction.
Therein, advantages and advantageous embodiments of the camera devices correspond to advantages and advantageous embodiments of the method for calibrating the camera device. The invention also relates to a vehicle with a camera with such a camera device.
The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim.
Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.
Below, embodiments of the invention are explained in more detail based on a schematic drawing.
Therein, the only Fig. shows a motor vehicle with an exemplary embodiment of a camera device, based on which the method for calibrating is exemplarily explained.
Here, a vehicle 1 , which is presently formed as a vehicle/trailer combination with a motor vehicle 2 as the towing vehicle and a trailer 3, includes a camera device 4. The camera device 4 is equipped with at least one, presently four reference cameras 5a-5d, which are suitable for capturing respective image data. Therein, from the image data of the respec-
tive reference cameras 5a- 5d, respective reference top views 7a-7d to an environment 10 of the vehicle 1 can be generated by a computing device 6. The camera device 4 also comprises a further camera 8a for capturing corresponding image data. Therein, the further camera 8a is presently disposed at a rear of the trailer 3. Therein, the computing unit 6 is formed to generate a further top view 9a to an environment 10 of the vehicle 1 and the camera device 4, respectively, also from the image data of the at least one further camera 8a. Therein, the image data of the further camera 8a is captured in a presettable time interval Δΐ after the image data of the reference cameras 5a-5d, which is greater than zero in the shown example.
The computing unit 6 is formed to spatially correlate at least one of the reference top views 7a- 7d and the further top view 9a based on an image comparison of the corresponding reference top view and the further top view 9a as well as to calculate an offset dx in vehicle longitudinal direction x and/or an offset dy in vehicle transverse direction y depending on a result of correlation and to automatically calibrate the further camera 8a relative to at least one of the reference cameras 5a- 5d, for example to the rear-side reference camera 5c, based on the calculated offset dx, dy.
Here, as the reference camera 5a-5d, for example the rear-side camera 5c or else the front-side camera 7a or one of the two lateral cameras 5b, 5d can thus be used for calibrating. Multiple or all of the reference cameras 5a to 5d can also be used to reduce an error in calibration. In the following, an exemplary embodiment of a method for calibrating the camera device 4 based on the rear-side reference camera 5c as the selected reference camera 5a- 5d is now to be described.
Thus, the reference top view 7c to the environment 10 of the vehicle 1 is now first generated from the image data of the reference camera 5c. In the shown example, in the reference top view, which is generated from image data having been captured at a point of time t1 , a specific feature 1 1 is represented, presently a cross on a ground of the environment 10. Since the vehicle 1 advances in x-direction with a non-zero speed v, this feature 1 1 is represented at a later point of time t2 = t1 + Δΐ as the feature 1 1 ' in the further top view 9a from the image data captured at the point of time t2 in the shown example.
In order that the further top view 9a is here the suitable top view and actually represents the feature 1 1 ', the preset time interval Δΐ can for example be adapted to the traveling
speed v and a minimum or maximum expected distance of the further camera 8a from the reference camera 5c and/or to a sampling rate for the image data and/or multiple reference top views 7c and multiple further top views 9a can be generated, respectively, from which the suitable pair of reference top view 7c and further top view 9a is selected. In this case, the top views 7c, 9a are correlated in pairs and each pair is evaluated with a quality criterion such that an offset dx, dy of the further top view 9a relative to the reference top view 7c is calculated depending on the correlation of the pair of the top views with the best value for the quality criterion. If only the reference top view 7c and the further top view 9a is used, thus, they are spatially correlated based on an image comparison by the computing unit 6 and then the offset dx, dy of the further top view 9a to the reference top view 7c is calculated depending on a result of this correlation.
For example, the offset in vehicle transverse direction y can be calculated as proportional to a pixel to millimeters ratio r of the top views 7c and 9a: This is exemplarily represented in the Fig., in which a pixel i = 70 and i = 50, respectively, thus each starting from a central pixel, the seventh and fifth pixel, respectively are associated with the feature 1 1 and 1 1 ', respectively. Thereby, the offset dy results as dy = (i-i') * r = 20*r.
Correspondingly, the offset in vehicle longitudinal direction x is calculated as proportional to the pixel to millimeters ratio r plus the distance traveled by the vehicle 1 in the time interval Δΐ. Here, in the shown example, a pixel is again associated with the feature 1 1 and 1 1 ', respectively. This pixel j and j', respectively, is presently the fiftieth and seventieth pixel, respectively, in the x-direction. Correspondingly, the offset dx is given by dx = v * Δΐ + (j-j') * r = v * Δΐ + 20 * r in the shown example. Thus, the distance traveled by the vehicle in the time interval Δΐ is added to a relative shift of the feature 1 1 and 1 1 ', respectively, within the top view 7c and 9a, respectively.
Claims
Method for calibrating a camera device (4) of a vehicle (1 ), including the method steps of:
a) generating a reference top view (7a-7d) to an environment (10) of the vehicle (1 ) from image data of a reference camera (5a-5d) of the camera device (4);
b) generating a further top view (9a) to the environment (10) from image data of a further camera (8a) of the camera device (4),
wherein the image data of the further camera (8a) is captured in a preset time interval (Δΐ) after the image data of the reference camera (5a-5d) ;
characterized by
c) spatially correlating the reference top view (7a-7d) and the further top view (9a) based on an image comparison of reference top view (7a-7d) and further top view (9a) by a computing unit (6) of the camera device (4) ; and
d) calculating an offset (dx, dy) of the further top view (9a) relative to the reference top view (7a- 7d) in a vehicle longitudinal direction (x) or a vehicle transverse direction (y) by the computing unit (6) depending on a result of the correlation;
e) automatically calibrating the further camera (8a) relative to the reference camera (5a- 5d) based on the calculated offset (dx, dy).
Method according to claim 1 ,
characterized in that
multiple reference top views (7a- 7d) and/or multiple further top views (9a) are generated, the correlation of the top views (7a- 7d, 9a) is performed in pairs for one reference top view (7a- 7d) and one further top view (9a) each and each respective pair of top views (7a- 7d, 9a) is evaluated with a quality criterion, and the offset (dx, dy) is calculated depending on a result of the correlation of the pair of the top views (7a- 7d, 9a) with the best value for the quality criterion.
Method according to any one of the preceding claims,
characterized in that
the method steps a) to d) are repeatedly performed and the automatic calibration is performed based on a plurality of calculated offsets (dx, dy), in particular based on an offset (dx, dy) averaged from the calculated offsets.
4. Method according to any one of the preceding claims,
characterized in that
the time interval (At) is preset depending on a minimum and/or maximum expected distance of the further camera (8a) from the reference camera (5a-5d) and a traveling speed (v) of the vehicle (1 ), and in particular also preset depending on a sampling rate for the image data.
5. Method according to any one of the preceding claims,
characterized in that
the image comparison is performed with an area matching algorithm in certain areas, in particular calculating the sum of the absolute differences and/or calculating the sum of the squared absolute differences in the respective areas of the top views (7a-7d, 9a).
6. Method according to any one of the preceding claims,
characterized in that
the offset (dx, dy) in vehicle transverse direction (y) is calculated as proportional to a pixel-to-millimeters-ratio of the further top view (9a) and the offset (dx, dy) in vehicle longitudinal direction (x) is calculated as proportional to a pixel-to-millimeters-ratio of the further top view (9a) plus the distance traveled by the vehicle (1 ) in the time interval (At).
7. Method according to any one of the preceding claims,
characterized in that
the vehicle (1 ) includes a trailer (3) and the further camera (8a) is disposed at the trailer (3).
8. Method according to claim 7,
characterized in that
information about a length of the trailer (3) is calculated from a known position of the reference camera (5a-5d) and a position of the further camera (8a) known after calibration of the further camera (8a) and provided by the computing unit (6).
9. Camera device (4) for a vehicle (1 ), including
- a reference camera (5a-5d) and a further camera (8a) for capturing respective image data;
- a computing unit (6), which is formed to generate a reference top view (7a- 7d) to an environment (10) of the camera device (4) from image data of the reference camera (5a-5d) and to generate a further top view (9a) to the environment (10) from image data of the further camera (8a), wherein the image data of the further camera (8a) is captured in a presettable time interval (At) after the image data of the reference camera (5a-5d);
characterized in that
the computing unit (6) is formed to
- spatially correlate the reference top view (7a-7d) and the further top view (9a) based on an image comparison of reference top view (7a-7d) and further top view (9a),
- to calculate an offset (dx, dy) of the further top view (9a) relative to the reference top view (7a- 7d) depending on a result of correlation, and
- to automatically calibrate the further camera (8a) relative to the reference camera (5a-5d) based on the calculated offset (dx, dy).
10. Vehicle (1 ) with a camera device (4) according to claim 9.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102017109445.8 | 2017-05-03 | ||
| DE102017109445.8A DE102017109445A1 (en) | 2017-05-03 | 2017-05-03 | Calibration of a vehicle camera device in the vehicle longitudinal direction or vehicle transverse direction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018202464A1 true WO2018202464A1 (en) | 2018-11-08 |
Family
ID=62044743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/EP2018/060393 Ceased WO2018202464A1 (en) | 2017-05-03 | 2018-04-24 | Calibration of a vehicle camera system in vehicle longitudinal direction or vehicle trans-verse direction |
Country Status (2)
| Country | Link |
|---|---|
| DE (1) | DE102017109445A1 (en) |
| WO (1) | WO2018202464A1 (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111222579A (en) * | 2020-01-09 | 2020-06-02 | 北京百度网讯科技有限公司 | Inter-camera obstacle correlation method, apparatus, device, electronic system and medium |
| JP2020113268A (en) * | 2019-01-07 | 2020-07-27 | コンチネンタル オートモーティヴ ゲゼルシャフト ミット ベシュレンクテル ハフツングContinental Automotive GmbH | How to calculate the position of the tow hitch |
| WO2021055969A1 (en) * | 2019-09-19 | 2021-03-25 | Continental Automotive Systems, Inc. | Automatic trailer camera calibration |
| US20220212600A1 (en) * | 2012-09-26 | 2022-07-07 | Magna Electronics Inc. | Vehicular trailer angle detection system |
| CN114842088A (en) * | 2022-03-28 | 2022-08-02 | 南京市德赛西威汽车电子有限公司 | Vehicle-mounted camera external parameter calibration method applicable to full scene without building picture |
| GB2620561A (en) * | 2022-07-06 | 2024-01-17 | Woven By Toyota Inc | Validating vehicle image sensor timestamps |
| US12299923B2 (en) | 2021-04-27 | 2025-05-13 | Continental Autonomous Mobility US, LLC | System and method for estimating relative trailer angle |
| CN120747250A (en) * | 2025-08-29 | 2025-10-03 | 吉林大学 | Feature-based combined calibration method for calibrating external parameters of collaborative shooting of multiple cameras |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022171794A1 (en) * | 2021-02-12 | 2022-08-18 | Zf Friedrichshafen Ag | Omniview for vehicles having an articulated joint |
| DE102021132334A1 (en) | 2021-12-08 | 2023-06-15 | Bayerische Motoren Werke Aktiengesellschaft | Scanning an environment of a vehicle |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090290032A1 (en) | 2008-05-22 | 2009-11-26 | Gm Global Technology Operations, Inc. | Self calibration of extrinsic camera parameters for a vehicle camera |
| DE102009057996A1 (en) * | 2009-11-12 | 2011-05-19 | Daimler Ag | Method for determining position and/or alignment of camera relative to position of vehicle and semi-trailer, involves determining rotation parameter and/or translation parameter of camera-coordinate system to vehicle-coordinate system |
| WO2012139636A1 (en) | 2011-04-13 | 2012-10-18 | Connaught Electronics Limited | Online vehicle camera calibration based on road surface texture tracking and geometric properties |
| WO2012139660A1 (en) | 2011-04-15 | 2012-10-18 | Connaught Electronics Limited | Online vehicle camera calibration based on road marking extractions |
| US20160048966A1 (en) * | 2014-08-13 | 2016-02-18 | Bendix Commercial Vehicle Systems Llc | Learning the distance between cameras for articulated vehicles |
| EP3035676A1 (en) * | 2014-12-19 | 2016-06-22 | Conti Temic microelectronic GmbH | Surround view system and vehicle including a surround view system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9661319B2 (en) * | 2014-05-21 | 2017-05-23 | GM Global Technology Operations LLC | Method and apparatus for automatic calibration in surrounding view systems |
| JP6458439B2 (en) * | 2014-10-09 | 2019-01-30 | 株式会社デンソー | On-vehicle camera calibration device, image generation device, on-vehicle camera calibration method, and image generation method |
-
2017
- 2017-05-03 DE DE102017109445.8A patent/DE102017109445A1/en active Pending
-
2018
- 2018-04-24 WO PCT/EP2018/060393 patent/WO2018202464A1/en not_active Ceased
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090290032A1 (en) | 2008-05-22 | 2009-11-26 | Gm Global Technology Operations, Inc. | Self calibration of extrinsic camera parameters for a vehicle camera |
| DE102009057996A1 (en) * | 2009-11-12 | 2011-05-19 | Daimler Ag | Method for determining position and/or alignment of camera relative to position of vehicle and semi-trailer, involves determining rotation parameter and/or translation parameter of camera-coordinate system to vehicle-coordinate system |
| WO2012139636A1 (en) | 2011-04-13 | 2012-10-18 | Connaught Electronics Limited | Online vehicle camera calibration based on road surface texture tracking and geometric properties |
| WO2012139660A1 (en) | 2011-04-15 | 2012-10-18 | Connaught Electronics Limited | Online vehicle camera calibration based on road marking extractions |
| US20160048966A1 (en) * | 2014-08-13 | 2016-02-18 | Bendix Commercial Vehicle Systems Llc | Learning the distance between cameras for articulated vehicles |
| EP3035676A1 (en) * | 2014-12-19 | 2016-06-22 | Conti Temic microelectronic GmbH | Surround view system and vehicle including a surround view system |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220212600A1 (en) * | 2012-09-26 | 2022-07-07 | Magna Electronics Inc. | Vehicular trailer angle detection system |
| US11872939B2 (en) * | 2012-09-26 | 2024-01-16 | Magna Electronics Inc. | Vehicular trailer angle detection system |
| JP7444605B2 (en) | 2019-01-07 | 2024-03-06 | コンチネンタル オートモーティヴ ゲゼルシャフト ミット ベシュレンクテル ハフツング | How to calculate the location of the tow hitch |
| JP2020113268A (en) * | 2019-01-07 | 2020-07-27 | コンチネンタル オートモーティヴ ゲゼルシャフト ミット ベシュレンクテル ハフツングContinental Automotive GmbH | How to calculate the position of the tow hitch |
| WO2021055969A1 (en) * | 2019-09-19 | 2021-03-25 | Continental Automotive Systems, Inc. | Automatic trailer camera calibration |
| CN115210764A (en) * | 2019-09-19 | 2022-10-18 | 大陆智行美国有限责任公司 | Automatic trailer camera calibration |
| US12198382B2 (en) | 2019-09-19 | 2025-01-14 | Continental Autonomous Mobility US, LLC | Automatic trailer camera calibration |
| CN115210764B (en) * | 2019-09-19 | 2026-01-23 | 大陆智行美国有限责任公司 | Automated trailer camera calibration |
| CN111222579B (en) * | 2020-01-09 | 2023-10-03 | 北京百度网讯科技有限公司 | Cross-camera obstacle association methods, devices, equipment, electronic systems and media |
| CN111222579A (en) * | 2020-01-09 | 2020-06-02 | 北京百度网讯科技有限公司 | Inter-camera obstacle correlation method, apparatus, device, electronic system and medium |
| US12299923B2 (en) | 2021-04-27 | 2025-05-13 | Continental Autonomous Mobility US, LLC | System and method for estimating relative trailer angle |
| CN114842088A (en) * | 2022-03-28 | 2022-08-02 | 南京市德赛西威汽车电子有限公司 | Vehicle-mounted camera external parameter calibration method applicable to full scene without building picture |
| GB2620561A (en) * | 2022-07-06 | 2024-01-17 | Woven By Toyota Inc | Validating vehicle image sensor timestamps |
| GB2620561B (en) * | 2022-07-06 | 2024-07-10 | Woven By Toyota Inc | Validating vehicle image sensor timestamps |
| CN120747250A (en) * | 2025-08-29 | 2025-10-03 | 吉林大学 | Feature-based combined calibration method for calibrating external parameters of collaborative shooting of multiple cameras |
Also Published As
| Publication number | Publication date |
|---|---|
| DE102017109445A1 (en) | 2018-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018202464A1 (en) | Calibration of a vehicle camera system in vehicle longitudinal direction or vehicle trans-verse direction | |
| US11554717B2 (en) | Vehicular vision system that dynamically calibrates a vehicular camera | |
| CN110859044B (en) | Integrated sensor calibration in natural scenes | |
| US10659677B2 (en) | Camera parameter set calculation apparatus, camera parameter set calculation method, and recording medium | |
| JP5926228B2 (en) | Depth detection method and system for autonomous vehicles | |
| KR102104005B1 (en) | Methods for detecting objects on the roadside of automobiles, computing devices, driver assistance systems and automobiles | |
| EP3367677B1 (en) | Calibration apparatus, calibration method, and calibration program | |
| KR101188588B1 (en) | Monocular Motion Stereo-Based Free Parking Space Detection Apparatus and Method | |
| CN108692719B (en) | Object detection device | |
| CN110717445B (en) | Front vehicle distance tracking system and method for automatic driving | |
| US10839231B2 (en) | Method for detecting a rolling shutter effect in images of an environmental region of a motor vehicle, computing device, driver assistance system as well as motor vehicle | |
| WO2019058582A1 (en) | Distance estimation device and method | |
| US20090297036A1 (en) | Object detection on a pixel plane in a digital image sequence | |
| WO2020118619A1 (en) | Method for detecting and modeling of object on surface of road | |
| JP6959032B2 (en) | Position estimation device, moving device | |
| JP2021193340A (en) | Self-position estimation device | |
| JP7232005B2 (en) | VEHICLE DRIVING ENVIRONMENT DETECTION DEVICE AND DRIVING CONTROL SYSTEM | |
| Ruland et al. | Hand-eye autocalibration of camera positions on vehicles | |
| WO2022133986A1 (en) | Accuracy estimation method and system | |
| JPH09259282A (en) | Moving obstacle detection device and method thereof | |
| Lins et al. | A novel machine vision approach applied for autonomous robotics navigation | |
| Winkens et al. | Optical truck tracking for autonomous platooning | |
| Vu et al. | Catadioptric omnidirectional vision sensor integration for vehicle-based sensing | |
| CN121788619A (en) | A method, apparatus, and electronic equipment for multi-camera online calibration of lane lines. | |
| WO2022269875A1 (en) | Information processing device, information processing method, and information processing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18719563 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 18719563 Country of ref document: EP Kind code of ref document: A1 |