CN111696160A - Automatic calibration method and device for vehicle-mounted camera and readable storage medium - Google Patents
Automatic calibration method and device for vehicle-mounted camera and readable storage medium Download PDFInfo
- Publication number
- CN111696160A CN111696160A CN202010571305.3A CN202010571305A CN111696160A CN 111696160 A CN111696160 A CN 111696160A CN 202010571305 A CN202010571305 A CN 202010571305A CN 111696160 A CN111696160 A CN 111696160A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- target
- mounted camera
- visual parameters
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an automatic calibration method, equipment and a readable storage medium for a vehicle-mounted camera, wherein the automatic calibration method for the vehicle-mounted camera is used for identifying rigid targets in a plurality of road images acquired by the vehicle-mounted camera to be calibrated, so that a large error caused by calibration results by taking variable flexible targets such as human bodies in the images as calibration bases is avoided, and the stability and the precision level of a calibration process are improved; obtaining a plurality of sample data for reference by obtaining initial visual parameters of a plurality of road images; the vehicle-mounted camera is calibrated by determining the target visual parameter with the maximum possibility through the Gaussian distribution characteristic and the initial visual parameters, so that the calibration precision is improved. In addition, compared with the prior art, the method can carry out full-automatic calibration without any external manual assistance and professional technical support, greatly reduces the calibration difficulty, and has wide adaptability and advancement.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an automatic calibration method and device for a vehicle-mounted camera and a readable storage medium.
Background
In the field of automotive autopilot and co-driver, vision is one of the most common perception methods, as well as methods that utilize multi-sensor fusion. Under the condition that vision is directly used as a sensing unit, particularly monocular vision, higher requirements are placed on the installation height and the vanishing point of a camera, and the wrong vanishing point often causes the computed distance to be inaccurate, so that the accuracy and timeliness of forward collision warning are influenced, and a delayed accident is caused; the wrong installation height can affect the width of a calculation target such as a road sign and the width of a pedestrian, so that filtering caused by parameter errors is caused, and a missing detection accident is caused. In addition, under the influence of external factors, the vehicle-mounted camera may be displaced or adjusted in angle, and a method is needed for determining the change of the installation parameters of the camera and then recalibrating and calibrating the camera.
At present, for vanishing point calibration, the existing full-automatic calibration method of the vehicle-mounted camera is realized by directly using the mode of taking the mean value of the intersection points of a plurality of lane line fitting straight lines, and the vanishing point calibration result is often not accurate enough due to the unstable state of a vehicle in the driving process; in order to improve the calibration accuracy, a common solution is to analyze the state of the vehicle and calculate the state of the vehicle when the state of the vehicle is in an ideal state, such as speed, steering angle, and the like, which obviously increases the complexity of calculation, and the accuracy calculated by adopting the method is still not ideal in the actual situation of continuous uphill and downhill. For the calibration of the installation height, the existing full-automatic calibration method of the vehicle-mounted camera usually selects a pedestrian or a ground reference object for calibration. Because the pedestrian target is a flexible target and the posture changes frequently, the calibration result is often caused to have a large error; ground reference objects such as lane lines, guide marks and the like are difficult to unify due to diversity and local difference of the set standards, so that the accuracy of the calibrated installation height is very low. Therefore, the existing automatic calibration methods for the vehicle-mounted camera have the problem of low calibration precision.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide an automatic calibration method for a vehicle-mounted camera, and aims to solve the technical problem that the existing calibration method for the vehicle-mounted camera is low in calibration precision.
In order to achieve the above object, the present invention provides an automatic calibration method for a vehicle-mounted camera, which comprises:
acquiring a plurality of road images acquired by a vehicle-mounted camera to be calibrated, and identifying and extracting a rigid target contained in each road image;
acquiring characteristic information corresponding to each rigid target, and acquiring initial visual parameters of each road image according to the characteristic information;
and determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters.
Optionally, the rigid target comprises a lane line target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
acquiring edge feature points in the lane line target to serve as the feature information, and fitting a lane line equation;
judging whether the edge feature points meet a preset deviation standard or not based on the lane line equation;
if so, acquiring the intersection point position of the lane line target to serve as the initial visual parameter.
Optionally, the step of determining a target visual parameter according to the gaussian distribution characteristic and the initial visual parameter includes:
when the intersection points with the preset first number are obtained, analyzing the intersection points with the preset first number according to Gaussian distribution characteristics to obtain the positions of the target vanishing points, wherein the positions of the target vanishing points are used as the target visual parameters.
Optionally, the rigid targets include a front vehicle target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
determining the vehicle type of the front vehicle target as the characteristic information by using a preset vehicle type recognition algorithm, and determining the actual vehicle width of the front vehicle target by using preset vehicle type width comparison information;
and obtaining the vehicle bottom edge coordinate and the bottom edge width of the front vehicle target, and obtaining the initial installation height as the initial visual parameter according to the vehicle bottom edge coordinate and the bottom edge width, the actual vehicle width, the target vanishing point position and the preset camera focal length of the vehicle-mounted camera to be calibrated.
Optionally, the step of obtaining the vehicle bottom coordinate and the bottom width of the preceding vehicle target, and obtaining an initial installation height as the initial visual parameter according to the vehicle bottom coordinate and the bottom width, the actual vehicle width, the target vanishing point position, and a preset camera focal length of the vehicle-mounted camera to be calibrated includes:
acquiring the vehicle position of the preceding vehicle target in the road image and the image height of the road image, and acquiring a camera pitch angle by utilizing the trigonometric function relationship among the image height, the target vanishing point position and the camera focal length;
and calculating the initial installation height according to the width of the bottom edge of the vehicle, the actual width of the vehicle, the position of a target vanishing point, the focal length of a camera, the position of the vehicle and the pitching angle of the camera by using a small hole imaging principle.
Optionally, the step of determining a target visual parameter according to the gaussian distribution characteristic and the initial visual parameter to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter includes:
when the preset second number of initial installation heights are obtained, the preset second number of initial installation heights are analyzed according to Gaussian distribution characteristics, the target installation height is obtained and serves as the target visual parameter, and the calibration of the vehicle-mounted camera to be calibrated is completed based on the target vanishing point position and the target installation height.
Optionally, after the step of determining a target visual parameter according to the gaussian distribution characteristic and the initial visual parameter to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter, the method further includes:
acquiring a plurality of driving images based on the calibrated vehicle-mounted camera, and acquiring a plurality of current visual parameters corresponding to the driving images;
judging whether a plurality of current visual parameters meet preset calibration conditions or not based on the target visual parameters;
if so, determining calibration visual parameters according to the Gaussian distribution characteristics and the current visual parameters to calibrate the target visual parameters, and outputting calibration success information;
if not, outputting the current calibration-free information.
Optionally, the step of determining whether a plurality of the current visual parameters satisfy preset calibration conditions based on the target visual parameters includes:
acquiring a first visual parameter quantity of a plurality of current visual parameters, wherein the deviation value between the current visual parameters and the target visual parameters exceeds a preset deviation threshold value, and a second visual parameter quantity of the current visual parameters, wherein the second visual parameter quantity does not exceed the preset deviation threshold value;
judging whether the number of the first visual parameters is larger than the number of the second visual parameters;
if yes, judging that the current visual parameters meet preset calibration conditions;
if not, judging that the current visual parameters do not meet the preset calibration conditions.
In addition, in order to achieve the above object, the present invention further provides an automatic calibration device for a vehicle-mounted camera, including:
the rigid target extraction module is used for acquiring a plurality of road images acquired by a vehicle-mounted camera to be calibrated, and identifying and extracting a rigid target contained in each road image;
the initial parameter acquisition module is used for acquiring the characteristic information corresponding to each rigid target and acquiring the initial visual parameters of each road image according to the characteristic information;
and the target parameter determining module is used for determining a target visual parameter according to the Gaussian distribution characteristic and the initial visual parameter so as to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter.
Optionally, the rigid target comprises a lane line target,
the initial parameter acquisition module comprises:
the equation fitting unit is used for acquiring edge feature points in the lane line target to serve as the feature information and fitting a lane line equation;
the deviation judgment unit is used for judging whether the edge feature points meet a preset deviation standard or not based on the lane line equation;
and if so, acquiring the intersection point position of the lane line target as the initial visual parameter.
Optionally, the target parameter determination module includes:
and the target position acquisition unit is used for analyzing the intersection positions of the preset first number according to the Gaussian distribution characteristic to obtain the positions of the target vanishing points as the target visual parameters when the intersection positions of the preset first number are acquired.
Optionally, the rigid targets include a front vehicle target,
the initial parameter acquisition module comprises:
an actual width determination unit configured to determine a vehicle type of the preceding vehicle target as the feature information using a preset vehicle type recognition algorithm, and determine an actual width of the preceding vehicle target using preset vehicle type width comparison information;
and the initial height acquisition unit is used for acquiring the vehicle bottom edge coordinate and the bottom edge width of the front vehicle target, and acquiring the initial installation height as the initial visual parameter according to the vehicle bottom edge coordinate and the bottom edge width, the actual vehicle width, the position of the target vanishing point and the preset camera focal length of the vehicle-mounted camera to be calibrated.
Optionally, the initial height obtaining unit is further configured to:
acquiring the vehicle position of the preceding vehicle target in the road image and the image height of the road image, and acquiring a camera pitch angle by utilizing the trigonometric function relationship among the image height, the target vanishing point position and the camera focal length;
and calculating the initial installation height according to the width of the bottom edge of the vehicle, the actual width of the vehicle, the position of a target vanishing point, the focal length of a camera, the position of the vehicle and the pitching angle of the camera by using a small hole imaging principle.
Optionally, the target parameter determination module includes:
and the target height determining unit is used for analyzing the initial mounting heights of the preset second number according to Gaussian distribution characteristics when the initial mounting heights of the preset second number are obtained, obtaining the target mounting heights as the target visual parameters, and completing the calibration of the vehicle-mounted camera to be calibrated based on the target vanishing point position and the target mounting heights.
Optionally, the automatic calibration device for a vehicle-mounted camera further includes:
the system comprises a current parameter acquisition unit, a driving image acquisition unit and a driving image acquisition unit, wherein the current parameter acquisition unit is used for acquiring a plurality of driving images based on a calibrated vehicle-mounted camera and acquiring a plurality of current visual parameters corresponding to the plurality of driving images;
a calibration condition judging unit, configured to judge whether the plurality of current visual parameters satisfy a preset calibration condition based on the target visual parameter;
a calibration success determining unit, configured to determine, if yes, a calibration visual parameter according to the gaussian distribution characteristic and the plurality of current visual parameters to calibrate the target visual parameter, and output calibration success information;
and the calibration-free judging unit is used for outputting the current calibration-free information if the current calibration-free information is not the same as the calibration-free information.
Optionally, the calibration condition determining unit is further configured to:
acquiring a first visual parameter quantity of a plurality of current visual parameters, wherein the deviation value between the current visual parameters and the target visual parameters exceeds a preset deviation threshold value, and a second visual parameter quantity of the current visual parameters, wherein the second visual parameter quantity does not exceed the preset deviation threshold value;
judging whether the number of the first visual parameters is larger than the number of the second visual parameters;
if yes, judging that the current visual parameters meet preset calibration conditions;
if not, judging that the current visual parameters do not meet the preset calibration conditions.
In addition, in order to achieve the above object, the present invention further provides an automatic calibration apparatus for a vehicle-mounted camera, including: the automatic calibration method comprises a memory, a processor and an automatic calibration program of the vehicle-mounted camera, wherein the automatic calibration program of the vehicle-mounted camera is stored on the memory and can run on the processor, and when being executed by the processor, the automatic calibration program of the vehicle-mounted camera realizes the steps of the automatic calibration method of the vehicle-mounted camera.
In addition, in order to achieve the above object, the present invention further provides a computer-readable storage medium, where a vehicle-mounted camera automatic calibration program is stored on the computer-readable storage medium, and when executed by a processor, the vehicle-mounted camera automatic calibration program implements the steps of the vehicle-mounted camera automatic calibration method as described above.
The invention provides an automatic calibration method and equipment for a vehicle-mounted camera and a computer readable storage medium. The automatic calibration method of the vehicle-mounted camera comprises the steps of identifying and extracting a rigid target contained in each road image by acquiring a plurality of road images acquired by the vehicle-mounted camera to be calibrated; acquiring characteristic information corresponding to each rigid target, and acquiring initial visual parameters of each road image according to the characteristic information; and determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters. By the mode, rigid targets in a plurality of road images acquired by the vehicle-mounted camera to be calibrated are identified, so that large errors caused by calibration results by taking variable flexible targets such as human bodies in the images as calibration bases are avoided, and the stability and the precision level of the calibration process are improved; obtaining a plurality of sample data for reference by obtaining initial visual parameters of a plurality of road images, and avoiding errors caused by single detection; the calibration of the vehicle-mounted camera is carried out by determining the target visual parameter with the maximum possibility through the Gaussian distribution characteristic and the initial visual parameters, so that the calibration precision is improved, and the technical problem of low calibration precision of the conventional calibration method of the vehicle-mounted camera is solved. In addition, compared with the prior art, the method can carry out full-automatic calibration without any external manual assistance and professional technical support, greatly reduces the calibration difficulty, and has wide adaptability and advancement.
Drawings
Fig. 1 is a schematic structural diagram of an automatic calibration device for a vehicle-mounted camera in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of an automatic calibration method for a vehicle-mounted camera according to the present invention;
FIG. 3 is a schematic projection diagram of a vehicle in an image coordinate system according to an embodiment of the automatic calibration method for a vehicle-mounted camera of the present invention;
FIG. 4 is a schematic diagram illustrating a geometric relationship between a vehicle-mounted camera and a vehicle in a road plane coordinate system according to an embodiment of the automatic calibration method for the vehicle-mounted camera of the present invention;
fig. 5 is a schematic diagram of a vanishing point in a specific embodiment of the automatic calibration method for the vehicle-mounted camera.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of an automatic calibration device for a vehicle-mounted camera in a hardware operating environment according to an embodiment of the present invention.
The automatic calibration equipment for the vehicle-mounted camera is terminal equipment with the vehicle-mounted camera.
As shown in fig. 1, the automatic calibration apparatus for an in-vehicle camera may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The optional user interface 1003 may include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the configuration of the in-vehicle camera automatic calibration apparatus shown in fig. 1 does not constitute a limitation of the in-vehicle camera automatic calibration apparatus, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an in-vehicle camera auto-calibration program.
In the automatic calibration device for a vehicle-mounted camera shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to call the in-vehicle camera auto-calibration program stored in the memory 1005, and perform the following operations:
acquiring a plurality of road images acquired by a vehicle-mounted camera to be calibrated, and identifying and extracting a rigid target contained in each road image;
acquiring characteristic information corresponding to each rigid target, and acquiring initial visual parameters of each road image according to the characteristic information;
and determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters.
Further, the rigid target includes a lane line target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
acquiring edge feature points in the lane line target to serve as the feature information, and fitting a lane line equation;
judging whether the edge feature points meet a preset deviation standard or not based on the lane line equation;
if so, acquiring the intersection point position of the lane line target to serve as the initial visual parameter.
Further, the step of determining the target visual parameter according to the gaussian distribution characteristics and the initial visual parameter comprises:
when the intersection points with the preset first number are obtained, analyzing the intersection points with the preset first number according to Gaussian distribution characteristics to obtain the positions of the target vanishing points, wherein the positions of the target vanishing points are used as the target visual parameters.
Further, the rigid targets include a front vehicle target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
determining the vehicle type of the front vehicle target as the characteristic information by using a preset vehicle type recognition algorithm, and determining the actual vehicle width of the front vehicle target by using preset vehicle type width comparison information;
and obtaining the vehicle bottom edge coordinate and the bottom edge width of the front vehicle target, and obtaining the initial installation height as the initial visual parameter according to the vehicle bottom edge coordinate and the bottom edge width, the actual vehicle width, the target vanishing point position and the preset camera focal length of the vehicle-mounted camera to be calibrated.
Further, the step of obtaining the vehicle bottom side coordinate and the bottom side width of the front vehicle target, and obtaining an initial installation height as the initial visual parameter according to the vehicle bottom side coordinate and the bottom side width, the actual vehicle width, the target vanishing point position, and the preset camera focal length of the vehicle-mounted camera to be calibrated includes:
acquiring the vehicle position of the preceding vehicle target in the road image and the image height of the road image, and acquiring a camera pitch angle by utilizing the trigonometric function relationship among the image height, the target vanishing point position and the camera focal length;
and calculating the initial installation height according to the width of the bottom edge of the vehicle, the actual width of the vehicle, the position of a target vanishing point, the focal length of a camera, the position of the vehicle and the pitching angle of the camera by using a small hole imaging principle.
Further, the step of determining a target visual parameter according to the gaussian distribution characteristic and the initial visual parameter to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter includes:
when the preset second number of initial installation heights are obtained, the preset second number of initial installation heights are analyzed according to Gaussian distribution characteristics, the target installation height is obtained and serves as the target visual parameter, and the calibration of the vehicle-mounted camera to be calibrated is completed based on the target vanishing point position and the target installation height.
Further, after the step of determining a target visual parameter according to the gaussian distribution characteristic and the initial visual parameter to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter, the processor 1001 may be further configured to invoke an automatic calibration program of the vehicle-mounted camera stored in the memory 1005, and perform the following operations:
acquiring a plurality of driving images based on the calibrated vehicle-mounted camera, and acquiring a plurality of current visual parameters corresponding to the driving images;
judging whether a plurality of current visual parameters meet preset calibration conditions or not based on the target visual parameters;
if so, determining calibration visual parameters according to the Gaussian distribution characteristics and the current visual parameters to calibrate the target visual parameters, and outputting calibration success information;
if not, outputting the current calibration-free information.
Further, the step of determining whether a plurality of the current visual parameters satisfy preset calibration conditions based on the target visual parameters includes:
acquiring a first visual parameter quantity of a plurality of current visual parameters, wherein the deviation value between the current visual parameters and the target visual parameters exceeds a preset deviation threshold value, and a second visual parameter quantity of the current visual parameters, wherein the second visual parameter quantity does not exceed the preset deviation threshold value;
judging whether the number of the first visual parameters is larger than the number of the second visual parameters;
if yes, judging that the current visual parameters meet preset calibration conditions;
if not, judging that the current visual parameters do not meet the preset calibration conditions.
Based on the hardware structure, the invention provides various embodiments of the automatic calibration method of the vehicle-mounted camera.
In the field of automotive autopilot and co-driver, vision is one of the most common perception methods, as well as methods that utilize multi-sensor fusion. Under the condition that vision is directly used as a sensing unit, particularly monocular vision, higher requirements are placed on the installation height and the vanishing point of a camera, and the wrong vanishing point often causes the computed distance to be inaccurate, so that the accuracy and timeliness of forward collision warning are influenced, and a delayed accident is caused; the wrong installation height can affect the width of a calculation target such as a road sign and the width of a pedestrian, so that filtering caused by parameter errors is caused, and a missing detection accident is caused. In addition, under the influence of external factors, the vehicle-mounted camera may be displaced or adjusted in angle, and a method is needed for determining the change of the installation parameters of the camera and then recalibrating and calibrating the camera.
At present, for vanishing point calibration, the existing full-automatic calibration method of the vehicle-mounted camera is realized by directly using the mode of taking the mean value of the intersection points of a plurality of lane line fitting straight lines, and the vanishing point calibration result is often not accurate enough due to the unstable state of a vehicle in the driving process; in order to improve the calibration accuracy, a common solution is to analyze the state of the vehicle and calculate the state of the vehicle when the state of the vehicle is in an ideal state, such as speed, steering angle, and the like, which obviously increases the complexity of calculation, and the accuracy calculated by adopting the method is still not ideal in the actual situation of continuous uphill and downhill. For the calibration of the installation height, the existing full-automatic calibration method of the vehicle-mounted camera usually selects a pedestrian or a ground reference object for calibration. Because the pedestrian target is a flexible target and the posture changes frequently, the calibration result is often caused to have a large error; ground reference objects such as lane lines, guide marks and the like are difficult to unify due to diversity and local difference of the set standards, so that the accuracy of the calibrated installation height is very low. Therefore, the existing automatic calibration methods for the vehicle-mounted camera have the problem of low calibration precision.
In order to solve the technical problems, the invention provides an automatic calibration method for a vehicle-mounted camera, which is characterized in that rigid targets in a plurality of road images collected by the vehicle-mounted camera to be calibrated are identified, so that large errors caused by calibration results by taking variable flexible targets such as human bodies in the images as calibration bases are avoided, and the stability and the precision level of a calibration process are improved; obtaining a plurality of sample data for reference by obtaining initial visual parameters of a plurality of road images; the calibration of the vehicle-mounted camera is carried out by determining the target visual parameter with the maximum possibility through the Gaussian distribution characteristic and the initial visual parameters, so that the calibration precision is improved, and the technical problem of low calibration precision of the conventional calibration method of the vehicle-mounted camera is solved. In addition, compared with the prior art, the method can carry out full-automatic calibration without any external manual assistance and professional technical support, greatly reduces the calibration difficulty, and has wide adaptability and advancement. The automatic calibration method of the vehicle-mounted camera is applied to automatic calibration equipment of the vehicle-mounted camera.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of an automatic calibration method for a vehicle-mounted camera.
The first embodiment of the invention provides an automatic calibration method for a vehicle-mounted camera, which comprises the following steps:
step S10, acquiring a plurality of road images acquired by a vehicle-mounted camera to be calibrated, and identifying and extracting a rigid target contained in each road image;
the vehicle-mounted camera is the basis for realizing functions of a plurality of Advanced Driving Assistance Systems (ADAS). Among numerous ADAS functions, the vision image processing system is comparatively basic, and the camera is vision image processing system's input again, therefore on-vehicle camera is essential in intelligent driving. The vehicle-mounted camera mainly comprises an interior camera, a rear camera, a front-view camera, a side-view camera and a surrounding-view camera, and the embodiment does not limit the interior camera, the rear-view camera, the front-view camera and the surrounding-view camera.
The road image is an image which is shot by the vehicle-mounted camera and contains the current road condition when a vehicle provided with the vehicle-mounted camera to be calibrated normally runs on the road, and the road image at least comprises a lane line on the road surface and possibly comprises other vehicles which also run on the current road, or pedestrians and the like. The plurality of road images may be images continuously captured by the vehicle-mounted camera in the same time period, or images captured at different times.
The rigid object is a concept as opposed to a flexible object, and in the present embodiment, refers to a fixed-form image object such as a lane line that may appear in a road image, another vehicle that appears within a photographing range, or the like. And flexible targets such as human targets have different postures and heights, and are not suitable for target ranging, blind area detection and the like under the condition of higher precision.
In the present embodiment, a description will be given of a processing procedure for a single road image. The vehicle-mounted camera calibration equipment acquires a road image shot by a vehicle-mounted camera to be calibrated, and identifies and extracts a rigid target in the road image by using a preset image identification algorithm. For example, if the road image contains a lane line, a preset lane line detection algorithm is used for identifying and extracting a lane line target in the image; and if the road image contains other vehicles, identifying and extracting the vehicle target in the image by using a preset vehicle identification algorithm. The vehicle target may be a front vehicle target or a rear vehicle target, which is not limited in this embodiment.
Step S20, acquiring characteristic information corresponding to each rigid target, and obtaining initial visual parameters of each road image according to the characteristic information;
the feature information is related information of all features of the rigid object in the road image. For example, if the rigid object is a lane line object, the feature information of the lane line object is an edge feature point of a lane line in the road image; if the rigid target is a vehicle target, the characteristic information of the vehicle target is an actual specification of the vehicle. The initial visual parameters are obtained according to the characteristic information when the vehicle-mounted camera to be calibrated shoots each road image. For example, if the feature information is an edge feature point of a lane line, the initial visual parameter obtained based on the feature information is an initial vanishing point position; if the characteristic information is the actual specification of the vehicle target, the initial visual parameter obtained based on the characteristic information is the initial installation height of the vehicle-mounted camera.
In this embodiment, when acquiring feature information corresponding to a rigid target extracted from a certain number of road images, the vehicle-mounted camera calibration device obtains a plurality of initial visual parameters corresponding to the road images through a plurality of pieces of feature information.
And step S30, determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters.
The target visual parameter is a visual parameter value with the highest probability calculated according to the Gaussian distribution characteristic and the initial visual parameters. For example, if the initial visual parameter is the initial vanishing point position, since the vehicle body shakes during the driving process, the vanishing point also deviates up and down due to the change of the camera pitch angle of the vehicle-mounted camera caused by the vehicle body shake, and the vertical deviation of the vanishing point conforms to the gaussian distribution, so that the most possible vanishing point position can be calculated according to the characteristics of the gaussian distribution and a plurality of initial visual parameters; if the initial visual parameter is the initial installation height, the calculated installation height error basically accords with Gaussian distribution due to vehicle body vibration and detection errors of the vehicle body, and therefore the most possible installation height can be calculated according to the Gaussian distribution characteristic when a certain number of initial installation heights are obtained.
In this embodiment, when the vehicle-mounted camera calibration device acquires a certain number of initial visual parameters, the value distribution of the initial visual parameters satisfies the gaussian distribution characteristics, so that the target visual parameters with the highest possibility can be determined according to the values of the initial visual parameters and the gaussian distribution characteristics. After the target visual parameters are determined, the calibration of the vehicle-mounted camera can be carried out by using the target visual parameters.
In the embodiment, a plurality of road images acquired by a vehicle-mounted camera to be calibrated are acquired, and a rigid target contained in each road image is identified and extracted; acquiring characteristic information corresponding to each rigid target, and acquiring initial visual parameters of each road image according to the characteristic information; and determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters. By the mode, rigid targets in a plurality of road images acquired by the vehicle-mounted camera to be calibrated are identified, so that large errors caused by calibration results by taking variable flexible targets such as human bodies in the images as calibration bases are avoided, and the stability and the precision level of the calibration process are improved; obtaining a plurality of sample data for reference by obtaining initial visual parameters of a plurality of road images; the calibration of the vehicle-mounted camera is carried out by determining the target visual parameter with the maximum possibility through the Gaussian distribution characteristic and the initial visual parameters, so that the calibration precision is improved, and the technical problem of low calibration precision of the conventional calibration method of the vehicle-mounted camera is solved. In addition, compared with the prior art, the method can carry out full-automatic calibration without any external manual assistance and professional technical support, greatly reduces the calibration difficulty, and has wide adaptability and advancement.
Further, not shown in the drawings, a second embodiment of the automatic calibration method for the vehicle-mounted camera according to the present invention is provided based on the first embodiment shown in fig. 2. In this embodiment, the rigid target includes a lane line target, and step S20 includes:
acquiring edge feature points in the lane line target to serve as the feature information, and fitting a lane line equation;
judging whether the edge feature points meet a preset deviation standard or not based on the lane line equation;
if so, acquiring the intersection point position of the lane line target to serve as the initial visual parameter.
In the present embodiment, the lane line target is a lane line portion in the road image, preferably a lane edge line and a lane center line. The edge feature points are pixel points of the positions of the lane lines in the road image. The lane line mode can be a linear equation, a quadratic curve equation, a cubic curve equation and the like, and is preferably a linear equation. The preset deviation standard is that the lane line is a straight line, and the deviation between the lane line and the edge feature point of which lane line the image is smaller than a preset deviation threshold. The preset deviation threshold may be flexibly set according to actual conditions, which is not limited in this embodiment.
And the vehicle-mounted camera calibration equipment acquires pixel points where the lane lines in the road image are located, namely the edge characteristic points, and fits a lane line equation. Specifically, the step of fitting the lane line equation is a conventional technical means in the art, and is not described herein again. The vehicle-mounted camera calibration equipment acquires a deviation value between the edge characteristic point and a lane line equation, and if the deviation value is smaller than a preset deviation threshold value and a lane line target is a straight line, the current condition that the lane line target meets a preset deviation standard can be judged, and intersection points of mutually parallel lane lines in an image are acquired; if the deviation value is larger than the preset deviation threshold value and/or the lane line target is not a straight line, judging that the current deviation value does not accord with the preset deviation standard, and continuously acquiring the edge feature points for judgment.
Further, in the present embodiment, step S30 includes:
when the intersection points with the preset first number are obtained, analyzing the intersection points with the preset first number according to Gaussian distribution characteristics to obtain the positions of the target vanishing points, wherein the positions of the target vanishing points are used as the target visual parameters.
In this embodiment, the preset first number is a number defining value used for determining whether the target vanishing point position can be obtained from a plurality of intersection positions. Since the vehicle body shakes during the running process of the vehicle, the vanishing point also deviates up and down due to the fact that the vehicle body shakes to cause the change of the camera pitch angle of the vehicle-mounted camera, and the up and down deviation of the vanishing point conforms to Gaussian distribution, the most possible vanishing point position, namely the target vanishing point position, can be calculated according to the characteristics of the Gaussian distribution and a plurality of initial visual parameters.
Further, in this embodiment, the rigid target includes a front vehicle target, and the step S20 includes:
determining the vehicle type of the front vehicle target as the characteristic information by using a preset vehicle type recognition algorithm, and determining the actual vehicle width of the front vehicle target by using preset vehicle type width comparison information;
and obtaining the vehicle bottom edge coordinate and the bottom edge width of the front vehicle target, and obtaining the initial installation height as the initial visual parameter according to the vehicle bottom edge coordinate and the bottom edge width, the actual vehicle width, the target vanishing point position and the preset camera focal length of the vehicle-mounted camera to be calibrated.
In this embodiment, the preset vehicle type recognition algorithm may adopt a vehicle type recognition algorithm based on a deep convolutional neural network, and the specific vehicle type recognition mode is a conventional technical means in the field and is not described herein again. The preset vehicle width comparison information is vehicle actual width information corresponding to each pre-stored vehicle type, for example, if the vehicle type is a household passenger vehicle, the corresponding vehicle actual width is 1.8 meters. Specifically, the vehicle-mounted camera calibration equipment determines that the front vehicle target in the road image is the type of the household passenger vehicle by using a vehicle type recognition algorithm based on a deep convolutional neural network, and the actual width of the front vehicle target is 1.8 m through preset vehicle type width comparison information. The vehicle-mounted camera calibration equipment acquires the vehicle bottom edge width of a front vehicle target in an image, and then the initial installation height of the corresponding vehicle-mounted camera can be calculated according to the vehicle bottom edge width, the actual width of the vehicle, the position of a target vanishing point calibrated before and the inherent camera focal length of the vehicle-mounted camera.
In addition, as another specific embodiment, the actual width of the vehicle of the preceding vehicle target may be determined by locating a rectangular vehicle region of the preceding vehicle target on the image and a rectangular license plate region of the license plate of the preceding vehicle target on the image, and obtaining the rectangular vehicle width of the rectangular vehicle region and the rectangular license plate width of the rectangular license plate region. Because the actual width of the license plate is known, and the proportional relation between the rectangular width of the license plate and the rectangular width of the vehicle is the same as the proportional relation between the actual width of the license plate and the actual width of the vehicle, the actual width of the vehicle of the front vehicle target can be obtained through the proportional relation between the rectangular width of the vehicle and the rectangular width of the license plate and the known actual width of the license plate. After the actual width of the vehicle of the front vehicle target is obtained, the initial installation height can be obtained according to the method through the actual width of the vehicle and the coordinates of the bottom edge of the vehicle.
Further, in this embodiment, the step of obtaining the coordinates and the width of the bottom edge of the vehicle of the preceding vehicle target, and obtaining an initial installation height as the initial visual parameter according to the coordinates and the bottom edge width of the vehicle, the actual width of the vehicle, the position of the target vanishing point, and the preset camera focal length of the vehicle-mounted camera to be calibrated includes:
acquiring the vehicle position of the preceding vehicle target in the road image and the image height of the road image, and acquiring a camera pitch angle by utilizing the trigonometric function relationship among the image height, the target vanishing point position and the camera focal length;
and calculating the initial installation height according to the width of the bottom edge of the vehicle, the actual width of the vehicle, the position of a target vanishing point, the focal length of a camera, the position of the vehicle and the pitching angle of the camera by using a small hole imaging principle.
In this embodiment, the vehicle-mounted camera calibration device obtains a vehicle position of a preceding vehicle target in a road image and an image height of the road image, and obtains a camera pitch angle by using a trigonometric function relationship among the image height, a target vanishing point position and a camera focal length. And designing a calculation formula by using a small hole imaging principle, and calculating the initial installation height according to the actual width of the vehicle, the position of the target vanishing point, the focal length of the camera, the position of the vehicle and the pitching angle of the camera.
Specifically, the calculation formula is:
θ＝arctan(v0/f)，α＝atan(H/2f)
where θ denotes a camera pitch angle, α denotes a vertical field angle, and H denotes an image vertical resolution, it is easy to know that the target vehicle bottom side coordinate v and vanishing point coordinate (u) are on the image0,v0) In known cases, the mounting height h can be determined by the actual width Δ X of the target vehicle, since the focal length and the image sensor position are fixed (f is constant)wAnd the width deltau of the target vehicle on the image.
Further, in the present embodiment, step S30 includes:
when the preset second number of initial installation heights are obtained, the preset second number of initial installation heights are analyzed according to Gaussian distribution characteristics, the target installation height is obtained and serves as the target visual parameter, and the calibration of the vehicle-mounted camera to be calibrated is completed based on the target vanishing point position and the target installation height.
In this embodiment, the preset second number is a number-defining value for determining whether the target mounting height can be obtained from a plurality of initial mounting heights. Due to the fact that the vehicle body shakes and the detection errors of the vehicle body, the calculated installation height errors basically accord with Gaussian distribution, and therefore when a certain number of initial installation heights are obtained, the most possible installation height, namely the target installation height, can be calculated according to the characteristics of the Gaussian distribution. And after the position of the target vanishing point and the target installation height are obtained, a calibration matrix is obtained by combining the calculation of an imaging principle, and the calibration of the vehicle-mounted camera can be completed based on the calibration matrix.
Specifically, the method comprises the following steps:
as shown in fig. 3 and 4, W represents an image width, H represents an image height, f represents a camera focal length, ξ represents a half of a horizontal field angle, γ represents an angle of a target point with respect to an optical axis in a camera X direction, θ represents a camera pitch angle, α represents a half of a vertical field angle, β represents an angle of a base of a target vehicle with respect to the optical axis of the camera, and P (X)W,YW,ZW) World coordinates of the target point. The derivation formula of the angle parameters is as follows:
ζ＝atan(W/2f)
γ＝atan(u/f)
θ＝atan(v0/f)
α＝atan(H/2f)
β＝atan(v/f)
Zw＝h/tan(θ-β)
point V in fig. 3P(u0,v0) Representing the coordinates of the vanishing point on the image plane coordinate system, point P (u, v) representing the coordinates of the target point on the image plane coordinate system, point P (X)W,YW,ZW) Representing the coordinates of the target point on the road plane coordinate system. The final formula can be derived by combining the derivation formula of each angle parameter and the geometric relationship between each line segment and the angle in the graph:
it is easy to know from the final formula that the conversion between the world coordinate width of the target vehicle and the image coordinate width of the target vehicle is dependent only on the installation height h when the vanishing point and the bottom edge of the target vehicle are known.
As a specific embodiment, for the calibration of the vanishing point, when the vehicle normally runs on the road, the road image is collected, the lane line is extracted, and the linear lane line equation is fitted. If the edge lines of the left lane and the right lane in the image are straight lines and the deviation between the edge lines of the lane and the edge feature points of the corresponding lane lines is small enough, the intersection point position V of the two edge lines of the lane is obtainedt(xt,yt). As shown in fig. 5, the horizontal straight Line in the figure is a horizon Line, the lower three straight lines are lane lines, a white Line segment of the Bottom of the trolley running on the lane represents the width Bottom Line of the vehicle, and the position marked with a "+" in the figure is the intersection Point of the horizontal Line and the lane Line, i.e. the Vanishing Point. In view of the change of the pitch angle of the camera caused by the shake of the car body, the vanishing point can shift up and down, and the shift accords with Gaussian distribution, so that the intersection point position V meeting the conditions needs to be collected and calculated for multiple timest(xt,yt). After collecting data of a certain number of intersection positions, the most probable intersection position, namely the position V of the target vanishing point can be calculated according to Gaussian distributionp(x, y) and storing the vanishing point data file to finish the vanishing point calibration.
For the calibration of the installation height, a front vehicle target in a road image needs to be extracted, and the type of the front vehicle needs to be identified. In the case of a household passenger car, the actual width of the car is 1.8 meters. Then according to the relation between the actual width and the coordinate and the position V of the target vanishing pointp(x, y) and the focal length of the camera, and calculating to obtain the initial installation height h, wherein the specific calculation formula can refer to the formula. In view of vehicle shake and detection error per se, the error of the calculated installation height basically accords with Gaussian distribution, so that the initial installation height can be calculated through multiple collection, and when a certain number of initial installation heights are collected, the installation height which best accords with conditions, namely the target installation height, is calculated according to the Gaussian distribution, and is stored in a data file of the installation height to be calibrated.
In this embodiment, the initial visual parameters are further collected based on a preset deviation standard, so that the initial visual parameters have more reference significance; the position of a target vanishing point and the target installation height are determined through the Gaussian distribution characteristics, so that the most possible numerical values of the vanishing point position and the installation height can be obtained without complicated and complicated calculation, and the efficiency of obtaining results is improved; the initial mounting height is calculated through the trigonometric function relationship between the small hole imaging principle and each parameter, and the accuracy of the data of the initial mounting height is improved.
Further, not shown in the drawings, a third embodiment of the automatic calibration method for the vehicle-mounted camera according to the present invention is provided based on the first embodiment shown in fig. 2. In this embodiment, after step S30, the method further includes:
acquiring a plurality of driving images based on the calibrated vehicle-mounted camera, and acquiring a plurality of current visual parameters corresponding to the driving images;
judging whether a plurality of current visual parameters meet preset calibration conditions or not based on the target visual parameters;
if so, determining calibration visual parameters according to the Gaussian distribution characteristics and the current visual parameters to calibrate the target visual parameters, and outputting calibration success information;
if not, outputting the current calibration-free information.
In this embodiment, the driving image is an image captured by the calibrated vehicle-mounted camera during driving. The current visual parameters are real-time values of the visual parameters obtained through driving images shot by the calibrated vehicle-mounted camera. The current visual parameters can be the current vanishing point position and the current installation height, and the calibration visual parameters can be the calibration vanishing point position and the calibration installation height correspondingly.
Specifically, after the vanishing point is calibrated, the vanishing point can be automatically calibrated. The current vanishing point position acquired in real time and the previously calibrated target vanishing point position V need to be acquiredpComparing, if V is compared with VpIf the deviation exceeds a certain value, an error count is added, otherwise, it means that V is in the normal fluctuation range, and a normal count is added. After a long period of time, for example, when the sum of the correct count and the error count reaches 512 times, if the error count is greater than the correct count, it indicates that the vanishing point needs to be recalibrated. The previous method is still used, and according to the gaussian distribution and a plurality of current vanishing point positions, the most probable vanishing point position is calculated as a new target vanishing point position, namely a calibration vanishing point position, and is stored in a vanishing point data file, and calibration success information is output, so that automatic calibration of the vanishing point is completed. And if the preset calibration condition is not met, outputting calibration-free information.
After the installation height is calibrated, the installation height can be automatically calibrated. The current installation height acquired in real time needs to be compared with a target installation height calibrated previously, if the deviation between the current installation height and the target installation height exceeds a certain percentage, an error count is added, otherwise, the current installation height is within a normal deviation range, and a correct count is added. After a long period of statistics, if the error count is greater than the correct count, it indicates that the installation height needs to be recalibrated currently. Still continuing to use the previous method, according to the gaussian distribution and a plurality of current mounting heights, the most probable mounting height is calculated as a new target mounting height, namely, a calibration mounting height, and is stored in a mounting height data file, and calibration success information is output, so as to complete automatic calibration of the mounting height. And if the preset calibration condition is not met, outputting calibration-free information.
Further, in this embodiment, the step of determining whether the plurality of current visual parameters satisfy preset calibration conditions based on the target visual parameters includes:
acquiring a first visual parameter quantity of a plurality of current visual parameters, wherein the deviation value between the current visual parameters and the target visual parameters exceeds a preset deviation threshold value, and a second visual parameter quantity of the current visual parameters, wherein the second visual parameter quantity does not exceed the preset deviation threshold value;
judging whether the number of the first visual parameters is larger than the number of the second visual parameters;
if yes, judging that the current visual parameters meet preset calibration conditions;
if not, judging that the current visual parameters do not meet the preset calibration conditions.
In this embodiment, the preset deviation threshold may be flexibly set according to actual situations, which is not limited in this embodiment. The first visual parameter quantity is the quantity of the current visual parameters with deviation values of the target visual parameters larger than a preset deviation threshold value, and the second visual parameter quantity is the quantity of the current visual parameters with deviation values of the target visual parameters smaller than or equal to the preset deviation threshold value. For example, 521 current visual parameters are now obtained, wherein the number of deviation values from the target visual parameter exceeding the preset deviation threshold is 200, the number of the first visual quantity parameters is 200, and the number of the second visual parameters is 321.
In this embodiment, further, the automatic calibration is performed by detecting the deviation between the current visual parameter and the target visual parameter and determining the reliability of the calibration result, so that the delayed accidents and the missed detection accidents caused by parameter errors and parameter changes can be reduced, and the method has important significance for improving the safety of equipment.
The invention further provides automatic calibration equipment for the vehicle-mounted camera.
The automatic calibration equipment for the vehicle-mounted camera comprises a processor, a memory and an automatic calibration program of the vehicle-mounted camera, wherein the automatic calibration program of the vehicle-mounted camera is stored in the memory and can run on the processor, and when the automatic calibration program of the vehicle-mounted camera is executed by the processor, the steps of the automatic calibration method for the vehicle-mounted camera are realized.
The method for implementing the automatic calibration program of the vehicle-mounted camera when executed can refer to each embodiment of the automatic calibration method of the vehicle-mounted camera of the present invention, and details are not repeated here.
The invention also provides a computer readable storage medium.
The computer readable storage medium of the invention stores the vehicle-mounted camera automatic calibration program, and the vehicle-mounted camera automatic calibration program realizes the steps of the vehicle-mounted camera automatic calibration method when being executed by the processor.
The method for implementing the automatic calibration program of the vehicle-mounted camera when executed can refer to each embodiment of the automatic calibration method of the vehicle-mounted camera, and details are not repeated here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for causing an on-vehicle camera automatic calibration apparatus to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The automatic calibration method for the vehicle-mounted camera is characterized by comprising the following steps:
acquiring a plurality of road images acquired by a vehicle-mounted camera to be calibrated, and identifying and extracting a rigid target contained in each road image;
acquiring characteristic information corresponding to each rigid target, and acquiring initial visual parameters of each road image according to the characteristic information;
and determining target visual parameters according to the Gaussian distribution characteristics and the initial visual parameters, and calibrating the vehicle-mounted camera to be calibrated based on the target visual parameters.
2. The automatic calibration method for vehicle-mounted camera according to claim 1, wherein the rigid target comprises a lane line target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
acquiring edge feature points in the lane line target to serve as the feature information, and fitting a lane line equation;
judging whether the edge feature points meet a preset deviation standard or not based on the lane line equation;
if so, acquiring the intersection point position of the lane line target to serve as the initial visual parameter.
3. The automatic calibration method for the vehicle-mounted camera according to claim 2, wherein the step of determining the target visual parameters according to the gaussian distribution characteristics and the initial visual parameters comprises:
when the intersection points with the preset first number are obtained, analyzing the intersection points with the preset first number according to Gaussian distribution characteristics to obtain the positions of the target vanishing points, wherein the positions of the target vanishing points are used as the target visual parameters.
4. The automatic calibration method for the vehicle-mounted camera according to claim 3, wherein the rigid target comprises a front vehicle target,
the step of obtaining the characteristic information corresponding to each rigid target and obtaining the initial visual parameters of each road image according to the characteristic information comprises the following steps:
determining the vehicle type of the front vehicle target as the characteristic information by using a preset vehicle type recognition algorithm, and determining the actual vehicle width of the front vehicle target by using preset vehicle type width comparison information;
and obtaining the vehicle bottom edge coordinate and the bottom edge width of the front vehicle target, and obtaining the initial installation height as the initial visual parameter according to the vehicle bottom edge coordinate and the bottom edge width, the actual vehicle width, the target vanishing point position and the preset camera focal length of the vehicle-mounted camera to be calibrated.
5. The automatic calibration method for the vehicle-mounted camera according to claim 4, wherein the step of obtaining the vehicle bottom coordinate and the bottom edge width of the front vehicle target, and obtaining an initial installation height as the initial visual parameter according to the vehicle bottom coordinate and the bottom edge width, the actual vehicle width, the target vanishing point position and the preset camera focal length of the vehicle-mounted camera to be calibrated comprises the following steps:
acquiring the vehicle position of the preceding vehicle target in the road image and the image height of the road image, and acquiring a camera pitch angle by utilizing the trigonometric function relationship among the image height, the target vanishing point position and the camera focal length;
and calculating the initial installation height according to the width of the bottom edge of the vehicle, the actual width of the vehicle, the position of a target vanishing point, the focal length of a camera, the position of the vehicle and the pitching angle of the camera by using a small hole imaging principle.
6. The automatic calibration method for the vehicle-mounted camera according to claim 4, wherein the step of determining a target visual parameter according to the Gaussian distribution characteristic and the initial visual parameter so as to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter comprises:
when the preset second number of initial installation heights are obtained, the preset second number of initial installation heights are analyzed according to Gaussian distribution characteristics, the target installation height is obtained and serves as the target visual parameter, and the calibration of the vehicle-mounted camera to be calibrated is completed based on the target vanishing point position and the target installation height.
7. The automatic calibration method for the vehicle-mounted camera according to claim 1, wherein after the step of determining the target visual parameter according to the gaussian distribution characteristic and the initial visual parameter to calibrate the vehicle-mounted camera to be calibrated based on the target visual parameter, the method further comprises:
acquiring a plurality of driving images based on the calibrated vehicle-mounted camera, and acquiring a plurality of current visual parameters corresponding to the driving images;
judging whether a plurality of current visual parameters meet preset calibration conditions or not based on the target visual parameters;
if so, determining calibration visual parameters according to the Gaussian distribution characteristics and the current visual parameters to calibrate the target visual parameters, and outputting calibration success information;
if not, outputting the current calibration-free information.
8. The automatic calibration method for the vehicle-mounted camera according to claim 7, wherein the step of determining whether the plurality of current visual parameters satisfy preset calibration conditions based on the target visual parameters comprises:
acquiring a first visual parameter quantity of a plurality of current visual parameters, wherein the deviation value between the current visual parameters and the target visual parameters exceeds a preset deviation threshold value, and a second visual parameter quantity of the current visual parameters, wherein the second visual parameter quantity does not exceed the preset deviation threshold value;
judging whether the number of the first visual parameters is larger than the number of the second visual parameters;
if yes, judging that the current visual parameters meet preset calibration conditions;
if not, judging that the current visual parameters do not meet the preset calibration conditions.
9. The utility model provides an automatic calibration equipment of vehicle-mounted camera which characterized in that, the automatic calibration equipment of vehicle-mounted camera includes: the automatic calibration method for the vehicle-mounted camera comprises a memory, a processor and an automatic calibration program for the vehicle-mounted camera, wherein the automatic calibration program is stored on the memory and can run on the processor, and when the automatic calibration program for the vehicle-mounted camera is executed by the processor, the steps of the automatic calibration method for the vehicle-mounted camera according to any one of claims 1 to 8 are realized.
10. A computer-readable storage medium, wherein the computer-readable storage medium has stored thereon an onboard camera automatic calibration program, which when executed by a processor implements the steps of the onboard camera automatic calibration method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571305.3A CN111696160B (en) | 2020-06-22 | 2020-06-22 | Automatic calibration method and equipment for vehicle-mounted camera and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571305.3A CN111696160B (en) | 2020-06-22 | 2020-06-22 | Automatic calibration method and equipment for vehicle-mounted camera and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111696160A true CN111696160A (en) | 2020-09-22 |
| CN111696160B CN111696160B (en) | 2023-08-18 |
Family
ID=72482619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010571305.3A Active CN111696160B (en) | 2020-06-22 | 2020-06-22 | Automatic calibration method and equipment for vehicle-mounted camera and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111696160B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380953A (en) * | 2020-11-10 | 2021-02-19 | 支付宝(杭州)信息技术有限公司 | Communication address calibration method and device of sales counter camera equipment and calibration board |
| CN112686964A (en) * | 2021-03-15 | 2021-04-20 | 中科长光精拓智能装备(苏州)有限公司 | Calibration method and system for improving precision image indirect identification |
| CN112785653A (en) * | 2020-12-30 | 2021-05-11 | 中山联合汽车技术有限公司 | Vehicle-mounted camera attitude angle calibration method |
| CN112800986A (en) * | 2021-02-02 | 2021-05-14 | 深圳佑驾创新科技有限公司 | Vehicle-mounted camera external parameter calibration method and device, vehicle-mounted terminal and storage medium |
| CN113227708A (en) * | 2021-03-30 | 2021-08-06 | 深圳市锐明技术股份有限公司 | Method and device for determining pitch angle and terminal equipment |
| CN113348464A (en) * | 2021-04-30 | 2021-09-03 | 华为技术有限公司 | Image processing method and device |
| CN113538597A (en) * | 2021-07-16 | 2021-10-22 | 英博超算(南京)科技有限公司 | Camera parameter calibration system |
| CN113625238A (en) * | 2021-08-11 | 2021-11-09 | 南京隼眼电子科技有限公司 | Vehicle-mounted millimeter wave radar pitch angle error calibration method and device, storage medium and electronic equipment |
| CN113658252A (en) * | 2021-05-17 | 2021-11-16 | 毫末智行科技有限公司 | Method, medium, apparatus for estimating elevation angle of camera, and camera |
| CN113965698A (en) * | 2021-11-12 | 2022-01-21 | 白银银珠电力(集团)有限责任公司 | Monitoring image calibration processing method, device and system for fire-fighting Internet of things |
| CN114820819A (en) * | 2022-05-26 | 2022-07-29 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN117315048A (en) * | 2023-11-22 | 2023-12-29 | 深圳元戎启行科技有限公司 | External parameter self-calibration method of vehicle-mounted camera, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833791A (en) * | 2010-05-11 | 2010-09-15 | 成都索贝数码科技股份有限公司 | Scene modeling method under single camera and system |
| CN102324017A (en) * | 2011-06-09 | 2012-01-18 | 中国人民解放军国防科学技术大学 | FPGA (Field Programmable Gate Array)-based lane line detection method |
| US20150279035A1 (en) * | 2014-03-25 | 2015-10-01 | Ford Global Technologies, Llc | Camera calibration |
| CN106875448A (en) * | 2017-02-16 | 2017-06-20 | 武汉极目智能技术有限公司 | A kind of vehicle-mounted monocular camera external parameter self-calibrating method |
| CN107730559A (en) * | 2017-09-30 | 2018-02-23 | 东风商用车有限公司 | A kind of scaling method of the vehicle-mounted camera based on image procossing |
| CN109685858A (en) * | 2018-12-29 | 2019-04-26 | 北京茵沃汽车科技有限公司 | A kind of monocular cam online calibration method |
| CN110555885A (en) * | 2018-05-31 | 2019-12-10 | 海信集团有限公司 | calibration method and device of vehicle-mounted camera and terminal |
| KR102060113B1 (en) * | 2019-01-30 | 2019-12-27 | 주식회사 몹티콘 | System and method for performing calibration |
-
2020
- 2020-06-22 CN CN202010571305.3A patent/CN111696160B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833791A (en) * | 2010-05-11 | 2010-09-15 | 成都索贝数码科技股份有限公司 | Scene modeling method under single camera and system |
| CN102324017A (en) * | 2011-06-09 | 2012-01-18 | 中国人民解放军国防科学技术大学 | FPGA (Field Programmable Gate Array)-based lane line detection method |
| US20150279035A1 (en) * | 2014-03-25 | 2015-10-01 | Ford Global Technologies, Llc | Camera calibration |
| CN106875448A (en) * | 2017-02-16 | 2017-06-20 | 武汉极目智能技术有限公司 | A kind of vehicle-mounted monocular camera external parameter self-calibrating method |
| CN107730559A (en) * | 2017-09-30 | 2018-02-23 | 东风商用车有限公司 | A kind of scaling method of the vehicle-mounted camera based on image procossing |
| CN110555885A (en) * | 2018-05-31 | 2019-12-10 | 海信集团有限公司 | calibration method and device of vehicle-mounted camera and terminal |
| CN109685858A (en) * | 2018-12-29 | 2019-04-26 | 北京茵沃汽车科技有限公司 | A kind of monocular cam online calibration method |
| KR102060113B1 (en) * | 2019-01-30 | 2019-12-27 | 주식회사 몹티콘 | System and method for performing calibration |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380953B (en) * | 2020-11-10 | 2023-05-09 | 支付宝(杭州)信息技术有限公司 | Communication address calibration method and device for sales counter camera equipment and calibration plate |
| CN112380953A (en) * | 2020-11-10 | 2021-02-19 | 支付宝(杭州)信息技术有限公司 | Communication address calibration method and device of sales counter camera equipment and calibration board |
| CN112785653A (en) * | 2020-12-30 | 2021-05-11 | 中山联合汽车技术有限公司 | Vehicle-mounted camera attitude angle calibration method |
| CN112785653B (en) * | 2020-12-30 | 2024-06-21 | 中山联合汽车技术有限公司 | Vehicle-mounted camera attitude angle calibration method |
| CN112800986A (en) * | 2021-02-02 | 2021-05-14 | 深圳佑驾创新科技有限公司 | Vehicle-mounted camera external parameter calibration method and device, vehicle-mounted terminal and storage medium |
| CN112686964A (en) * | 2021-03-15 | 2021-04-20 | 中科长光精拓智能装备(苏州)有限公司 | Calibration method and system for improving precision image indirect identification |
| CN112686964B (en) * | 2021-03-15 | 2021-06-04 | 中科长光精拓智能装备(苏州)有限公司 | Calibration method and system for improving precision image indirect identification |
| CN113227708A (en) * | 2021-03-30 | 2021-08-06 | 深圳市锐明技术股份有限公司 | Method and device for determining pitch angle and terminal equipment |
| CN113348464A (en) * | 2021-04-30 | 2021-09-03 | 华为技术有限公司 | Image processing method and device |
| CN113658252A (en) * | 2021-05-17 | 2021-11-16 | 毫末智行科技有限公司 | Method, medium, apparatus for estimating elevation angle of camera, and camera |
| CN113538597A (en) * | 2021-07-16 | 2021-10-22 | 英博超算(南京)科技有限公司 | Camera parameter calibration system |
| CN113538597B (en) * | 2021-07-16 | 2023-10-13 | 英博超算(南京)科技有限公司 | Calibration camera parameter system |
| CN113625238A (en) * | 2021-08-11 | 2021-11-09 | 南京隼眼电子科技有限公司 | Vehicle-mounted millimeter wave radar pitch angle error calibration method and device, storage medium and electronic equipment |
| CN113965698B (en) * | 2021-11-12 | 2024-03-08 | 白银银珠电力(集团)有限责任公司 | Monitoring image calibration processing method, device and system for fire-fighting Internet of things |
| CN113965698A (en) * | 2021-11-12 | 2022-01-21 | 白银银珠电力(集团)有限责任公司 | Monitoring image calibration processing method, device and system for fire-fighting Internet of things |
| CN114820819B (en) * | 2022-05-26 | 2023-03-31 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN114820819A (en) * | 2022-05-26 | 2022-07-29 | 广东机电职业技术学院 | Expressway automatic driving method and system |
| CN117315048A (en) * | 2023-11-22 | 2023-12-29 | 深圳元戎启行科技有限公司 | External parameter self-calibration method of vehicle-mounted camera, electronic equipment and storage medium |
| CN117315048B (en) * | 2023-11-22 | 2024-04-12 | 深圳元戎启行科技有限公司 | External parameter self-calibration method of vehicle-mounted camera, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111696160B (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111696160B (en) | Automatic calibration method and equipment for vehicle-mounted camera and readable storage medium | |
| CN104573646B (en) | Chinese herbaceous peony pedestrian detection method and system based on laser radar and binocular camera | |
| CN112362055B (en) | Attitude estimation method and device and electronic equipment | |
| JP5441549B2 (en) | Road shape recognition device | |
| EP1909064A1 (en) | Object detection device | |
| US20140176679A1 (en) | Method for Automatically Classifying Moving Vehicles | |
| US10767994B2 (en) | Sensor output correction apparatus | |
| EP2642364B1 (en) | Method for warning the driver of a motor vehicle about the presence of an object in the surroundings of the motor vehicle, camera system and motor vehicle | |
| CN113256739B (en) | Self-calibration method and device for vehicle-mounted BSD camera and storage medium | |
| US20160207473A1 (en) | Method of calibrating an image detecting device for an automated vehicle | |
| CN110341621B (en) | Obstacle detection method and device | |
| EP3428902A1 (en) | Image processing device, imaging device, mobile apparatus control system, image processing method, and program | |
| JP2018048949A (en) | Object recognition device | |
| CN112124304B (en) | Library position positioning method and device and vehicle-mounted equipment | |
| CN112927309A (en) | Vehicle-mounted camera calibration method and device, vehicle-mounted camera and storage medium | |
| JP2007011994A (en) | Road recognition device | |
| JP6564127B2 (en) | VISUAL SYSTEM FOR AUTOMOBILE AND METHOD FOR CONTROLLING VISUAL SYSTEM | |
| US20200039434A1 (en) | Method and system for determining and displaying a wading situation | |
| CN113256701B (en) | Distance acquisition method, device, equipment and readable storage medium | |
| WO2020064543A1 (en) | Vision system and method for a motor vehicle | |
| CN113903103B (en) | Partial image generation device, partial image generation method, and storage medium | |
| KR20180081966A (en) | Image correction method by vehicle recognition | |
| JPH08329397A (en) | Vehicle recognition device and vehicle approach reporting device using same | |
| EP3330893A1 (en) | Information processing device, information processing method, and carrier means | |
| CN110843786B (en) | Method and system for determining and displaying a wading situation and vehicle having such a system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 14 / F, Beidou building, 6 Huida Road, Jiangbei new district, Nanjing, Jiangsu Province 210000 Applicant after: Jiangsu Zhongtian Anchi Technology Co.,Ltd. Address before: 3 / F and 5 / F, building 2, Changyuan new material port, building B, Changyuan new material port, science and Technology Park community, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN ZHONGTIAN ANCHI Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |