CN115311370A - Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium - Google Patents

Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115311370A
CN115311370A CN202211052204.0A CN202211052204A CN115311370A CN 115311370 A CN115311370 A CN 115311370A CN 202211052204 A CN202211052204 A CN 202211052204A CN 115311370 A CN115311370 A CN 115311370A
Authority
CN
China
Prior art keywords
camera
calibrated
projection
external
external parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211052204.0A
Other languages
Chinese (zh)
Inventor
李耀萍
朱磊
贾双成
单国航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhidao Network Technology Beijing Co Ltd
Original Assignee
Zhidao Network Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhidao Network Technology Beijing Co Ltd filed Critical Zhidao Network Technology Beijing Co Ltd
Priority to CN202211052204.0A priority Critical patent/CN115311370A/en
Publication of CN115311370A publication Critical patent/CN115311370A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The application relates to a camera external reference calibration and evaluation method, a camera external reference calibration and evaluation device, an electronic device and a storage medium. The camera external reference calibration method comprises the following steps: acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same live-action road; projecting the lane lines in the multiple continuous images to a preset projection space to obtain projection line segments; and adjusting the camera external parameters, and determining the corresponding camera external parameters when the straight lines where the projection line segments are located are parallel as the optimal external parameters of the camera to be calibrated. The camera external parameters are calibrated by adopting a plurality of pictures, the camera external parameters corresponding to the situation that when the straight lines of the projection line segments of the lane lines in the projection space in each picture are parallel are selected to be determined as the optimal external parameters of the camera to be calibrated, and the calibration error caused by camera shake can be effectively overcome.

Description

Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of image data processing technologies, and in particular, to a method and an apparatus for calibrating and evaluating external parameters of a camera, an electronic device, and a computer-readable storage medium.
Background
With the development of computer technology, the vehicle auxiliary automatic driving technology and the navigation technology are also gradually improved, the vehicle-mounted camera is required for automatic driving and navigation, and the camera needs to be calibrated before the vehicle-mounted camera is used.
In a related technical scheme, when a camera is calibrated by using one picture, the camera shaking condition cannot be fully considered, so that the camera calibration is not accurate enough, the camera calibration effect cannot be accurately evaluated after the camera calibration is completed, and the navigation effect is poor due to the fact that camera parameters are directly used under the condition that the camera calibration effect is poor, and the problem needs to be solved.
Disclosure of Invention
In order to solve or partially solve the problems in the related art, the application provides a camera external reference calibration and evaluation method, a camera external reference calibration and evaluation device, an electronic device and a computer-readable storage medium, which can effectively reduce calibration errors caused by camera shake, and provide an intuitive camera parameter evaluation method to ensure the accuracy of camera calibration.
The application provides a camera external reference calibration method in a first aspect, which includes:
acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same live-action road; wherein each image comprises a lane line in the real scene road;
projecting the lane lines in the multiple continuous images to a preset projection space by adopting a preset camera external reference calibration model to obtain multiple projection line segments of the lane lines in each image in the projection space;
and adjusting camera external parameters in the camera external parameter calibration model, and determining the optimal external parameters of the camera to be calibrated, wherein when the camera external parameters are the optimal external parameters, straight lines of any lane line in the projection space where the plurality of projection line segments are located are parallel.
As a possible embodiment of the present application, in this embodiment, the acquiring multiple continuous images of the same real road captured by the camera to be calibrated includes:
and acquiring at least three images of the same real scene road shot by the camera to be calibrated at intervals of a preset time period.
The second aspect of the present application provides a camera external reference evaluation method, including:
acquiring a plurality of continuous images which are shot by a camera calibrated based on the camera external parameter calibration method in any one of the embodiments and aim at the same real scene lane line;
projecting the lane lines in each image into a world coordinate system based on the external parameters of the camera and the position of the camera to obtain the projection coordinates of the lane lines in each image in the world coordinate system;
and performing straight line fitting on the projection coordinates, and determining the accuracy of the external parameter of the camera based on a straight line error obtained by fitting.
As a possible implementation manner of this application, in this implementation manner, the acquiring multiple continuous images captured by the camera calibrated based on the camera external reference calibration method according to any of the foregoing embodiments and directed to the same real scene lane line includes:
at least three images of the same real scene lane line shot by the camera calibrated by the camera external parameter calibration method according to any one of the embodiments at preset time intervals are obtained.
As one possible embodiment of the present application, in the embodiment, the determining the accuracy of the camera external parameter based on the line error obtained by fitting includes:
calculating the mean square error of the straight line obtained by fitting;
and determining the accuracy of the camera external parameters based on the magnitude of the mean square deviation, wherein the accuracy of the camera external parameters corresponding to the minimum value of the mean square deviation is highest, and the accuracy of the camera external parameters corresponding to the maximum value of the mean square deviation is lowest.
As a possible embodiment of the present application, in this embodiment, the method further includes:
and determining the camera external parameter corresponding to the minimum value of the mean square error as the optimal external parameter.
The third aspect of the present application provides a camera external reference calibration apparatus, including:
the first image acquisition module is used for acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same real scene road; wherein each image comprises a lane line in the real scene road;
the first projection module is used for projecting the lane lines in the multiple continuous images to a preset projection space by adopting a preset camera external parameter calibration model to obtain multiple projection line segments of the lane lines in each image in the projection space;
and the external parameter determining module is used for adjusting the camera external parameters in the camera external parameter calibration model and determining the optimal external parameters of the camera to be calibrated, wherein when the camera external parameters are the optimal external parameters, straight lines of the plurality of projection line segments of any lane line in the projection space are parallel.
The present application in a fourth aspect provides a camera external reference evaluation device, including:
a second image obtaining module, configured to obtain multiple continuous images of the same real scene lane line captured by the camera calibrated by the camera external reference calibration method according to claim 7;
the second projection module is used for projecting the lane lines in each image into a world coordinate system based on the external parameters of the camera and the position of the camera to obtain the projection coordinates of the lane lines in each image in the world coordinate system;
and the external parameter evaluation module is used for performing straight line fitting on the projection coordinates and determining the accuracy of the external parameter of the camera based on a straight line error obtained by fitting.
A fifth aspect of the present application provides an electronic device, comprising:
a processor; and
a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method as described above.
A sixth aspect of the present application provides a computer-readable storage medium having stored thereon executable code, which, when executed by a processor of an electronic device, causes the processor to perform the method as described above.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The method comprises the steps of calibrating external parameters of a camera by adopting a plurality of pictures, selecting the corresponding external parameters of the camera when straight lines of the projection line sections of the lane lines in the projection space in each image are parallel, determining the external parameters as the optimal external parameters of the camera to be calibrated, effectively overcoming calibration errors caused by camera shake, projecting the lane lines in each picture to a world coordinate system based on the plurality of pictures of the same lane line shot by the calibrated camera after the camera finishes calibration, performing straight line fitting on the obtained projection line sections, and determining the calibration effect of the camera based on the errors of the straight lines obtained by fitting.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.
Fig. 1 is a schematic flowchart of a camera external reference calibration method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a camera external parameter evaluation method according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating an optimal extrinsic parameter determination method according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a camera external reference calibration apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a camera external parameter evaluation device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
With the development of computer technology, the auxiliary automatic driving technology and the navigation technology of vehicles are also gradually improved, vehicle-mounted cameras are needed for automatic driving and navigation, calibration and pose calculation are needed before the vehicle-mounted cameras are used, and the cameras need to be calibrated. In a related technical scheme, when a camera is calibrated by using one picture, the camera shaking condition cannot be fully considered, so that the camera calibration is not accurate enough, the camera calibration effect cannot be accurately evaluated after the camera calibration is completed, and the navigation effect is poor due to the fact that camera parameters are directly used under the condition that the camera calibration effect is poor, and the problem needs to be solved.
In view of the above problems, embodiments of the present application provide a camera external parameter calibration and evaluation method, which can effectively reduce calibration errors caused by camera shake, and provide an intuitive camera parameter evaluation method to ensure accuracy of camera calibration.
The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flowchart of a camera external reference calibration method according to an embodiment of the present application.
Referring to fig. 1, the method for calibrating external reference of a camera provided in the embodiment of the present application includes:
step S101, acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same real scene road; wherein each image includes a lane line in the real road.
In the embodiment of the application, the camera to be calibrated refers to a camera used for shooting a road lane line on an automatic driving vehicle or an automatic navigation vehicle, and can be a monocular camera. When acquiring multiple continuous images of a camera to be calibrated for the same real scene road, the heights of the camera from the road ground are basically consistent, and the shooting angles of the camera are consistent. In the embodiment of the present application, a plurality of consecutive images captured by the camera to be calibrated should all include the same lane line.
As a possible implementation manner of the present application, for convenience of description, taking a specific embodiment as an example, when multiple continuous images of a camera to be calibrated for a same real road are obtained, the camera to be calibrated may be installed on a test vehicle, so that the test vehicle runs on a relatively flat and straight road, and then the camera to be calibrated captures an image of the real road at the same capturing angle at intervals, where the captured image should include a lane line in the real road. In the embodiment of the application, the shooting interval time is related to the running speed of the test vehicle, and in order to ensure the accuracy of data, the running speed of the vehicle can be selected to be 30km/h, the shooting interval time is 1s, and a plurality of images are continuously shot.
And S102, projecting the lane lines in the multiple continuous images to a preset projection space by adopting a preset camera external reference calibration model to obtain multiple projection line segments of the lane lines in each image in the projection space.
In the embodiment of the application, the camera external parameter calibration model is a calculation model used for determining the external parameter of the camera to be calibrated, and the model can be used for projecting the lane lines in the real-scene image shot by the camera to be calibrated to a preset projection space.
As a possible implementation manner of the present application, for convenience of description, taking a specific embodiment as an example, after acquiring multiple continuous images shot by a camera to be calibrated for a same real-scene road, extracting a lane line in the image for each image, fitting the lane line with points in a space coordinate system, determining coordinates of each lane line point, projecting the points on each lane line into a projection space by using a camera external reference calibration model, establishing a corresponding coordinate system in the projection space, and connecting the points on the projection space coordinate system to form a projection line segment. According to the method, the projection line segment of the lane line in the projection space in each live-action road image is obtained. The camera external reference calibration model is a conversion calculation model for converting the coordinates of points on the lane line from the coordinates in the real image coordinate system to the coordinates in the coordinate system in the projection space.
As a possible implementation manner of the present application, for convenience of description, a specific embodiment is taken asFor example, the coordinates of the lane line in the real-scene image of the road to be calibrated and shot by the camera in the camera coordinate system are (x) c ,y c ,z c ) The coordinate of which corresponding to a point in projection space is (x) w ,y w ,z w ) When the coordinate conversion is performed, the calculation is performed by using the formula (1):
[x c ,y c ,z c ]＝(R|T)[x w ,y w ,z w ] (1)
wherein R is a rotation matrix of 3 × 3, and T is a translation matrix of 3 × 1, the following formula (2) is obtained:
Figure BDA0003824106180000061
wherein, RT is the external parameter of the camera to be calibrated.
Step S103, adjusting camera external parameters in the camera external parameter calibration model, and determining the optimal external parameters of the camera to be calibrated, wherein when the camera external parameters are the optimal external parameters, straight lines of the plurality of projection line segments of any lane line in the projection space are parallel.
In the embodiment of the application, when the lane lines of a plurality of images shot by the camera to be calibrated are projected into the projection space, the camera external parameter value in the camera external parameter calibration model is adjusted, so that the straight lines where the projection line segments projected by the lane lines in the projection space in each road live-action image are located are parallel, wherein the superposition can also be in a parallel form, and thus the optimal external parameter of the camera to be calibrated is determined. In the embodiment of the application, the fact that the straight lines where the projection line segments obtained by projecting the lane lines in the projection space in each road live-action image are parallel means that for the same lane line, a plurality of projection line segments exist in the projection space, the plurality of projection line segments are obtained by projecting according to a plurality of images, and when the straight lines where the plurality of projection line segments of the lane line in the projection space are parallel, the best external parameters of the camera to be calibrated are determined.
For convenience of description, a specific embodiment is taken as an example, when the formula (1) is used for calculation, the position of the projection line segment of the lane line in the projection space in each road live-action image is further adjusted by continuously adjusting the external parameter of the camera to be calibrated, and when the straight line where each projection line segment is located is parallel (or coincident), the external parameter of the corresponding camera to be calibrated is determined as the optimal external parameter. Optionally, whether the straight lines of the projection line segments are parallel or not may be determined by calculating the slope of the straight line of each projection line segment, and as a possible implementation manner of the present application, when the slopes of the straight lines of all projection line segments are within a preset slope range, it may be determined that the straight lines of all projection line segments are parallel.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The camera external parameters are calibrated by adopting a plurality of pictures, and the camera external parameters corresponding to the condition that straight lines of the projection line segments of the lane lines in the projection space in each picture are parallel are selected to be determined as the optimal external parameters of the camera to be calibrated, so that the calibration error caused by camera shake can be effectively overcome.
As a possible embodiment of the present application, in this embodiment, the acquiring multiple continuous images of the same real road captured by the camera to be calibrated includes:
at least three images of the same real scene road shot by the camera to be calibrated at intervals of a preset time period are obtained.
In the embodiment of the application, when a plurality of continuous images shot by the camera to be calibrated for the same real road are obtained, at least three images are obtained, wherein the at least three images are images shot by the camera to be calibrated for the same real road at preset time intervals. As a possible implementation manner of the application, in order to ensure the accuracy of camera calibration and simultaneously reduce the data calculation amount, the effect of selecting three continuous images is optimal. Of course, the specific number of the images may be selected according to actual situations, and the application is not limited thereto.
According to the embodiment of the application, the camera is calibrated by acquiring at least three images continuously shot by the camera to be calibrated aiming at the same real-scene road, so that the accuracy of camera calibration is ensured, and the calculation amount of data can be reduced.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The camera external parameters are calibrated by adopting a plurality of pictures, and the camera external parameters corresponding to the condition that straight lines of the projection line segments of the lane lines in the projection space in each picture are parallel are selected to be determined as the optimal external parameters of the camera to be calibrated, so that the calibration error caused by camera shake can be effectively overcome.
The embodiment of the application provides a camera external reference evaluation method, as shown in fig. 2, the camera external reference evaluation method includes:
step S201, acquiring multiple continuous images of the same real scene lane line captured by the camera calibrated by the camera external reference calibration method according to any of the foregoing embodiments.
In this embodiment of the application, after the camera to be calibrated is calibrated according to the camera external reference calibration method described in any one of the foregoing embodiments, multiple continuous images shot by the camera for the same real-world lane line are obtained, and as described in the foregoing embodiments, when multiple continuous images are shot for the same real-world lane line, it should be ensured that the heights of the camera from the road are substantially the same, and the shooting angles of the camera are the same.
Step S202, the lane lines in each image are projected into a world coordinate system based on the external parameters of the camera and the position of the camera, and the projection coordinates of the lane lines in each image in the world coordinate system are obtained.
In the embodiment of the present application, the position of the camera refers to a position of the camera in a world coordinate system when the camera takes an image, and may be represented by coordinates of the camera in the world coordinate system, an external parameter of the camera is determined by the external parameter calibration method provided in any one of the foregoing embodiments, and based on the external parameter of the camera and the position of the camera in the world coordinate system, lane lines in each image are projected into the world coordinate system, where the lane lines in each image may be decomposed into discrete points according to an inverse operation of formula (1), and coordinates of each point in the world coordinate system are calculated according to the coordinates of each point, so as to obtain projected coordinates of the lane lines in each image in the world coordinate system.
And S203, performing straight line fitting on the projection coordinate, and determining the accuracy of the external parameter of the camera based on a straight line error obtained by fitting.
In the embodiment of the application, after the projection coordinates of the lane lines in the world coordinate system in each image are obtained, all points are subjected to straight line fitting to obtain a fitted straight line, the error of the fitted straight line is calculated, and the accuracy of the external parameters of the camera is determined according to the error of the fitted straight line.
As a possible implementation manner of the present application, for convenience of description, a specific embodiment is taken as an example, after calibration is completed, a camera to be calibrated continuously captures a plurality of images for a same lane line, wherein when the plurality of images are captured, it is required to ensure that heights of the camera from the ground are substantially consistent, the lane line is a straight lane line, and a capturing angle of the camera is kept unchanged when the camera captures the lane line. After a plurality of continuous images are shot at intervals of a time period, a lane line in each image is dispersed into a set of points, all the points are projected into the same world coordinate system to obtain a set of projection points in the world coordinate system, straight line fitting is carried out on the set of projection points in the world coordinate system, errors of straight lines obtained through fitting are calculated, and accuracy of camera external parameters is judged according to the errors. Wherein, the smaller the error of the straight line obtained by fitting is, the more accurate the external reference of the camera is.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The camera external parameters are calibrated by adopting a plurality of pictures, the camera external parameters corresponding to the situation that when the straight lines of the lane lines in the projection space are parallel to each other are selected to be determined as the optimal external parameters of the camera to be calibrated, the calibration error caused by camera shake can be effectively overcome, after the camera is calibrated, the lane lines in each picture are projected to a world coordinate system based on the plurality of pictures of the same lane line shot by the calibrated camera, the obtained projection line segments are subjected to straight line fitting, the camera calibration effect is determined based on the error of the straight line obtained by fitting, and the visual camera calibration effect evaluation method is provided, so that the optimal external parameters of the camera can be more accurately found, and the accuracy of later navigation is ensured.
As a possible implementation manner of this application, in this implementation manner, the acquiring multiple continuous images captured by the camera calibrated based on the camera external reference calibration method according to any of the foregoing embodiments and directed to the same real scene lane line includes:
at least three images of the same real scene lane line shot by the camera calibrated by the camera external parameter calibration method according to any one of the embodiments at preset time intervals are obtained.
In the embodiment of the application, when a plurality of continuous images shot by a camera to be calibrated for the same lane line are obtained, at least three images are obtained, wherein the at least three images are images shot by the camera to be calibrated for the same lane line at preset time intervals. As a possible implementation manner of the application, in order to ensure the accuracy of camera calibration and reduce the amount of data calculation, the effect of selecting three continuous images is the best. Of course, the specific number of the images may be selected according to actual situations, and the application is not limited thereto.
As one possible embodiment of the present application, in this embodiment, as shown in fig. 3, the determining the accuracy of the camera external parameter based on the line error obtained by fitting includes:
in step S301, the mean square error of the straight line obtained by fitting is calculated.
In the embodiment of the present application, when calculating the error of the straight line obtained by fitting, the mean square error of the fitted straight line may be calculated.
Step S302, based on the size of the mean square error, determining the accuracy of the camera external parameter, wherein the accuracy of the camera external parameter corresponding to the minimum value of the mean square error is the highest, and the accuracy of the camera external parameter corresponding to the maximum value of the mean square error is the lowest.
In the embodiment of the application, after the mean square error of the fitted straight line is obtained through calculation, the accuracy of the camera external parameter is determined based on the magnitude of the mean square error. Optionally, the smaller the value of the mean square error is, the more accurate the external parameter of the identification camera is. As a possible implementation manner of the present application, a mean square error threshold may be set, and when the mean square error of the calculated fitted straight line is smaller than the mean square error threshold, it indicates that the corresponding camera external parameter is available.
In the embodiment of the present application, after the mean square deviations of the fitting straight lines corresponding to the respective camera external parameters are obtained through calculation, the camera external parameter corresponding to the minimum value of the mean square deviations is determined as the camera external parameter with the highest accuracy, and the camera external parameter corresponding to the maximum value of the mean square deviations is determined as the camera external parameter with the lowest accuracy.
The embodiment of the application determines the accuracy of the corresponding camera external parameter by calculating the mean square error of the fitted straight line, and provides a method for intuitively and accurately judging the accuracy of the camera external parameter.
As a possible embodiment of the present application, in this embodiment, the method further includes:
and determining the camera external parameter corresponding to the minimum value of the mean square error as the optimal external parameter.
In the embodiment of the present application, after the mean square error of the fitted straight line corresponding to each camera external parameter is calculated, the camera external parameter corresponding to the minimum value of the mean square error is determined as the camera external parameter with the highest accuracy, and is determined as the best camera external parameter.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The method comprises the steps of calibrating external parameters of a camera by adopting a plurality of pictures, selecting the corresponding external parameters of the camera when straight lines of the projection line sections of the lane lines in the projection space in each image are parallel, determining the external parameters as the optimal external parameters of the camera to be calibrated, effectively overcoming calibration errors caused by camera shake, projecting the lane lines in each picture to a world coordinate system based on the plurality of pictures of the same lane line shot by the calibrated camera after the camera finishes calibration, performing straight line fitting on the obtained projection line sections, and determining the calibration effect of the camera based on the errors of the straight lines obtained by fitting.
Corresponding to the embodiment of the application function implementation method, the application also provides a camera external reference calibration and evaluation device, electronic equipment and a corresponding embodiment.
Fig. 4 is a camera external reference calibration apparatus 40 according to an embodiment of the present application, where the camera external reference calibration apparatus 40 includes a first image obtaining module 410, a first projection module 420, and an external reference determining module 430, where:
the first image acquisition module 410 is configured to acquire multiple continuous images of the same real road captured by a camera to be calibrated; wherein each image comprises a lane line in the real scene road;
a first projection module 420, configured to project the lane lines in the multiple continuous images to a preset projection space by using a preset camera external parameter calibration model, so as to obtain multiple projection line segments of the lane lines in each image in the projection space;
and an external parameter determining module 430, configured to adjust a camera external parameter in the camera external parameter calibration model, and determine an optimal external parameter of the camera to be calibrated, where when the camera external parameter is the optimal external parameter, straight lines where the plurality of projection line segments of any lane line in the projection space are located are parallel.
As an embodiment of the present application, in a possible implementation manner, when acquiring multiple continuous images of a same real road captured by a camera to be calibrated, the first image acquisition module 410 is configured to:
and acquiring at least three images of the same real scene road shot by the camera to be calibrated at intervals of a preset time period.
Fig. 5 is a camera external reference evaluation apparatus 50 according to an embodiment of the present application, where the camera external reference evaluation apparatus 50 according to the embodiment of the present application includes a second image acquisition module 510, a second projection module 520, and an external reference evaluation module 530, where:
a second image obtaining module 510, configured to obtain multiple continuous images of the same real scene lane line captured by the camera calibrated based on the camera external parameter calibration method;
a second projection module 520, configured to project the lane lines in each of the images into a world coordinate system based on the external parameters of the camera and the position of the camera, so as to obtain the projection coordinates of the lane lines in each of the images in the world coordinate system;
and the external parameter evaluation module 530 is configured to perform straight line fitting on the projection coordinates, and determine accuracy of the camera external parameter based on a straight line error obtained through fitting.
As a possible implementation manner of the present application, in this implementation manner, when acquiring multiple continuous images of the same real-scene lane line captured by the camera calibrated based on the camera external parameter calibration method described in the foregoing embodiment, the second image acquisition module 510 is configured to:
at least three images of the same real scene lane line shot by the camera calibrated by the camera external parameter calibration method according to the embodiment at preset time intervals are obtained.
As one possible embodiment of the present application, in this embodiment, the external parameter evaluation module 530, when determining the accuracy of the camera external parameter based on the line error obtained by fitting, is configured to:
calculating the mean square error of the straight line obtained by fitting;
and determining the accuracy of the camera external parameters based on the magnitude of the mean square deviation, wherein the accuracy of the camera external parameters corresponding to the minimum value of the mean square deviation is highest, and the accuracy of the camera external parameters corresponding to the maximum value of the mean square deviation is lowest.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
The method and the device for calibrating the camera external parameters obtain a plurality of continuous images aiming at the same real scene road, project the lane lines in the plurality of continuous images to a preset projection space by adopting a preset camera external parameter calibration model, obtain the projection line sections of the lane lines in each image in the projection space, adjust the camera external parameters in the camera external parameter calibration model, and determine the corresponding camera external parameters when the straight lines of the projection line sections of the lane lines in the projection space are parallel as the optimal external parameters of the camera to be calibrated. The camera external parameters are calibrated by adopting a plurality of pictures, the camera external parameters corresponding to the situation that when the straight lines of the lane lines in the projection space are parallel to each other are selected to be determined as the optimal external parameters of the camera to be calibrated, the calibration error caused by camera shake can be effectively overcome, after the camera is calibrated, the lane lines in each picture are projected to a world coordinate system based on the plurality of pictures of the same lane line shot by the calibrated camera, the obtained projection line segments are subjected to straight line fitting, the camera calibration effect is determined based on the error of the straight line obtained by fitting, and the visual camera calibration effect evaluation method is provided, so that the optimal external parameters of the camera can be more accurately found, and the accuracy of later navigation is ensured.
Fig. 6 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
Referring to fig. 6, the electronic device 600 includes a memory 610 and a processor 620.
Processor 620 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 610 may include various types of storage units such as system memory, read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions that are required by the processor 620 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, as well. In some embodiments, memory 610 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, min SD card, micro-SD card, etc.), a magnetic floppy disc, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.
The memory 610 has stored thereon executable code that, when processed by the processor 620, may cause the processor 620 to perform some or all of the methods described above.
Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.
Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having executable code (or a computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the various steps of the above-described method according to the present application.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. A camera external reference calibration method is characterized by comprising the following steps:
acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same live-action road; wherein each image comprises a lane line in the real scene road;
projecting the lane lines in the multiple continuous images to a preset projection space by adopting a preset camera external reference calibration model to obtain multiple projection line segments of the lane lines in each image in the projection space;
and adjusting the camera external parameters in the camera external parameter calibration model, and determining the optimal external parameters of the camera to be calibrated, wherein when the camera external parameters are the optimal external parameters, straight lines of the plurality of projection line segments of any lane line in the projection space are parallel.
2. The camera external reference calibration method according to claim 1, wherein the acquiring of the plurality of continuous images of the same real scene road captured by the camera to be calibrated comprises:
at least three images of the same real scene road shot by the camera to be calibrated at intervals of a preset time period are obtained.
3. A camera-external-reference evaluation method, characterized by comprising:
acquiring a plurality of continuous images which are shot by a camera calibrated based on the camera external reference calibration method of any one of claims 1 or 2 and aim at the same real scene lane line;
projecting the lane lines in each image into a world coordinate system based on the external parameters of the camera and the position of the camera to obtain the projection coordinates of the lane lines in each image in the world coordinate system;
and performing straight line fitting on the projection coordinate, and determining the accuracy of the external parameter of the camera based on a straight line error obtained by fitting.
4. The camera extrinsic parameter evaluation method according to claim 3, wherein the acquiring of the plurality of continuous images of the same real scene lane line captured by the camera calibrated by the camera extrinsic parameter calibration method according to any one of claims 1 or 2 comprises:
acquiring at least three images of the same real scene lane line shot at intervals of preset time period by a camera calibrated based on the camera external reference calibration method of any one of claims 1 or 2.
5. The method according to claim 4, wherein the determining the accuracy of the camera external parameter based on the fitted straight line error comprises:
calculating the mean square error of the straight line obtained by fitting;
and determining the accuracy of the camera external parameters based on the size of the mean square error, wherein the accuracy of the camera external parameters corresponding to the minimum value of the mean square error is the highest, and the accuracy of the camera external parameters corresponding to the maximum value of the mean square error is the lowest.
6. The camera external reference evaluation method according to claim 5, further comprising:
and determining the camera external parameter corresponding to the minimum value of the mean square error as the optimal external parameter.
7. The camera external reference calibration device is characterized by comprising the following components:
the first image acquisition module is used for acquiring a plurality of continuous images which are shot by a camera to be calibrated and aim at the same real scene road; wherein each image comprises a lane line in the real scene road;
the first projection module is used for projecting the lane lines in the multiple continuous images to a preset projection space by adopting a preset camera external parameter calibration model to obtain projection line segments of the lane lines in each image in the projection space;
and the external parameter determining module is used for adjusting the camera external parameters in the camera external parameter calibration model and determining the optimal external parameters of the camera to be calibrated, wherein when the camera external parameters are the optimal external parameters, straight lines of the plurality of projection line segments of any lane line in the projection space are parallel.
8. A camera external parameter evaluation device, characterized by comprising:
a second image obtaining module, configured to obtain multiple continuous images of the same real scene lane line captured by the camera calibrated by the camera external reference calibration method according to claim 7;
the second projection module is used for projecting the lane lines in each image to a world coordinate system based on the external parameters of the camera and the position of the camera to obtain the projection coordinates of the lane lines in each image in the world coordinate system;
and the external parameter evaluation module is used for performing straight line fitting on the projection coordinate and determining the accuracy of the external parameter of the camera based on a straight line error obtained by fitting.
9. An electronic device, comprising:
a processor; and
a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any one of claims 1-6.
10. A computer-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-6.
CN202211052204.0A 2022-08-31 2022-08-31 Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium Pending CN115311370A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211052204.0A CN115311370A (en) 2022-08-31 2022-08-31 Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211052204.0A CN115311370A (en) 2022-08-31 2022-08-31 Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115311370A true CN115311370A (en) 2022-11-08

Family

ID=83864056

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211052204.0A Pending CN115311370A (en) 2022-08-31 2022-08-31 Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115311370A (en)

Similar Documents

Publication Publication Date Title
CN110146869B (en) Method and device for determining coordinate system conversion parameters, electronic equipment and storage medium
CN108805934B (en) External parameter calibration method and device for vehicle-mounted camera
CN114034307B (en) Vehicle pose calibration method and device based on lane lines and electronic equipment
CN114241062A (en) Camera external parameter determination method and device for automatic driving and computer readable storage medium
CN111784784B (en) IMU internal reference calibration method and device, electronic equipment and storage medium
JP7343054B2 (en) Location estimation method, location estimation device, and location estimation program
CN114187579A (en) Target detection method, apparatus and computer-readable storage medium for automatic driving
CN111811527A (en) Time synchronization method and device of map data and related equipment
CN112595335B (en) Intelligent traffic driving stop line generation method and related device
CN113284194A (en) Calibration method, device and equipment for multiple RS (remote sensing) equipment
CN112598743A (en) Pose estimation method of monocular visual image and related device
CN116052117A (en) Pose-based traffic element matching method, equipment and computer storage medium
CN114820748B (en) Full-view-angle characteristic intelligent evidence obtaining method based on panoramic data
CN116051812A (en) Target detection method and device, electronic equipment and storage medium
CN115311370A (en) Camera external parameter calibration and evaluation method and device, electronic equipment and storage medium
CN112348903B (en) Method and device for calibrating external parameters of automobile data recorder and electronic equipment
CN114863347A (en) Map checking method, device and equipment
CN112556702A (en) Height correction method for vehicle moving track and related device
CN114677448A (en) External reference correction method and device for vehicle-mounted camera, electronic equipment and storage medium
CN113538546B (en) Target detection method, device and equipment for automatic driving
CN114332130A (en) Monocular camera acquisition method and device for high-precision images
CN115345948A (en) Camera calibration method and device, electronic equipment and computer readable storage medium
CN115523929B (en) SLAM-based vehicle-mounted integrated navigation method, device, equipment and medium
CN113838149B (en) Camera internal parameter calibration method, server and system for automatic driving vehicle
CN114248782B (en) Unmanned vehicle pitch angle determination method, unmanned vehicle pitch angle determination device and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination