CN114049404A - Method and device for calibrating internal phase and external phase of vehicle - Google Patents

Method and device for calibrating internal phase and external phase of vehicle Download PDF

Info

Publication number
CN114049404A
CN114049404A CN202210029319.1A CN202210029319A CN114049404A CN 114049404 A CN114049404 A CN 114049404A CN 202210029319 A CN202210029319 A CN 202210029319A CN 114049404 A CN114049404 A CN 114049404A
Authority
CN
China
Prior art keywords
matching
image
video
matching points
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210029319.1A
Other languages
Chinese (zh)
Other versions
CN114049404B (en
Inventor
全丹辉
刘国清
王启程
杨广
徐涵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ruijian Zhixing Technology Co.,Ltd.
Original Assignee
Shenzhen Minieye Innovation Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Minieye Innovation Technology Co Ltd filed Critical Shenzhen Minieye Innovation Technology Co Ltd
Priority to CN202210029319.1A priority Critical patent/CN114049404B/en
Publication of CN114049404A publication Critical patent/CN114049404A/en
Application granted granted Critical
Publication of CN114049404B publication Critical patent/CN114049404B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and a device for calibrating external parameters of an automobile internal phase. According to the method, the template image is obtained in advance, the image acquired after the position of the camera is changed is matched with the template image, and the rotation matrix and the translation vector of the camera, namely the camera external parameter, are automatically estimated according to the matching result, so that the camera external parameter can be automatically estimated after the position of the camera is changed, and the camera external parameter calibration efficiency is improved.

Description

Method and device for calibrating internal phase and external phase of vehicle
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a device for calibrating external parameters of an automobile interior phase.
Background
At present, a commonly used camera external reference calibration method comprises: firstly fixing a camera at a position, placing a calibration plate at a specific position to ensure that the calibration plate is filled with a camera picture and is in the middle of the camera picture as far as possible, then controlling the camera to take a picture, and calculating the external parameter of the camera at the current position by a designed external parameter calibration program of the camera. When the position of the camera is changed artificially or non-artificially, actively or passively, the external parameters calculated by re-calibration are also changed, so that the calibration accuracy is influenced, and the re-calibration operation is greatly limited due to the complex operation and long time consumption of the camera external parameter calibration method.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides the method and the device for calibrating the external parameters of the internal camera of the vehicle, which can automatically estimate the external parameters of the camera after the position of the camera is changed, and improve the calibration efficiency of the external parameters of the camera.
In order to solve the above technical problem, in a first aspect, an embodiment of the present invention provides a method for calibrating external parameters of an internal vehicle, including;
installing a camera on a pipe column of a steering wheel, installing a calibration plate above a driving position, and enabling the camera to completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position;
when the steering wheel moves to the first position, acquiring a reference image acquired by the camera, and intercepting an image of the matching area from the reference image as a template image;
acquiring a first video acquired by the camera during the movement of the steering wheel from the first position to the second position, acquiring a second video acquired by the camera during the movement of the steering wheel from the third position to the fourth position, and acquiring a third video acquired by the camera during the S-shaped movement of the steering wheel within the first position, the second position, the third position and the fourth position;
and respectively matching each image in the first video, the second video and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points and all the third matching points.
Further, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points specifically includes:
respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points;
calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
Further, the step of capturing the image of the matching region from the reference image as a template image specifically includes:
and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting the image of the matching area from the intermediate image as the template image.
Further, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points specifically is:
respectively carrying out image preprocessing on each image in the first video, the second video and the third video to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and respectively matching each first image to be matched, each second image to be matched and each third image to be matched with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
Further, the image preprocessing includes edge detection, morphological erosion, and dilation.
In a second aspect, an embodiment of the present invention provides an external reference calibration apparatus for an interior vehicle, including;
the matching area confirming module is used for installing a camera on a pipe column of a steering wheel, installing a calibration plate above a driving position and enabling the camera to completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position;
the template image acquisition module is used for acquiring a reference image acquired by the camera when the steering wheel moves to the first position, and intercepting an image of the matching area from the reference image as a template image;
a multi-directional video acquisition module, configured to acquire a first video captured by the camera during movement of the steering wheel from the first position to the second position, acquire a second video captured by the camera during movement of the steering wheel from the third position to the fourth position, and acquire a third video captured by the camera during S-shaped movement of the steering wheel within the first position, the second position, the third position, and the fourth position;
and the camera external parameter calibration module is used for respectively matching each image in the first video, the second video and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points and all the third matching points.
Further, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points specifically includes:
respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points;
calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
Further, the step of capturing the image of the matching region from the reference image as a template image specifically includes:
and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting the image of the matching area from the intermediate image as the template image.
Further, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points specifically is:
respectively carrying out image preprocessing on each image in the first video, the second video and the third video to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and respectively matching each first image to be matched, each second image to be matched and each third image to be matched with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
Further, the image preprocessing includes edge detection, morphological erosion, and dilation.
The embodiment of the invention has the following beneficial effects:
the camera is arranged on a pipe column of a steering wheel, a calibration plate is arranged above a driving position, the camera can completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position, a reference image collected by the camera is obtained when the steering wheel moves to the first position, an image of the matching area is intercepted from the reference image and is used as a template image, a first video collected by the camera is obtained in the process that the steering wheel moves from the first position to the second position, a second video collected by the camera is obtained in the process that the steering wheel moves from the third position to the fourth position, a third video collected by the camera is obtained in the process that the steering wheel moves in an S shape in the first position, the second position, the third position and the fourth position, and each image of the first video, the second video and the third video is matched with the template image respectively, and obtaining a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating translation vectors of the camera according to all the first matching points, all the second matching points and all the third matching points to realize the external reference calibration of the vehicle inside the vehicle. Compared with the prior art, the embodiment of the invention matches the acquired image of the camera after the position of the camera is changed with the template image by acquiring the template image in advance, and automatically estimates the rotation matrix and the translation vector of the camera, namely the camera external parameter according to the matching result, so that the camera external parameter can be automatically estimated after the position of the camera is changed, and the camera external parameter calibration efficiency is improved.
Drawings
Fig. 1 is a schematic flow chart of an external reference calibration method for an interior vehicle according to a first embodiment of the present invention;
fig. 2 is a schematic view of a camera mounting position exemplified in the first embodiment of the present invention;
fig. 3 is a schematic view of an exemplary moving position of a steering wheel in the first embodiment of the present invention;
FIG. 4 is a schematic illustration of the formation of a ChAruco Board as exemplified in the first embodiment of the present invention;
fig. 5 is a schematic structural diagram of an external reference calibration device in a vehicle according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the present invention will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, the step numbers in the text are only for convenience of explanation of the specific embodiments, and do not serve to limit the execution sequence of the steps. The method provided by the embodiment can be executed by the relevant terminal device, and the following description takes a processor as an execution subject as an example.
As shown in FIG. 1, a first embodiment provides a method for calibrating internal phase and external phase references of an automobile, comprising steps S1-S4;
s1, mounting the camera on a column of a steering wheel, and mounting a calibration plate above a driving position, so that the camera can completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position;
s2, when the steering wheel moves to the first position, acquiring a reference image acquired by the camera, and intercepting an image of the matching area from the reference image as a template image;
s3, acquiring a first video acquired by a camera in the process that the steering wheel moves from the first position to the second position, acquiring a second video acquired by the camera in the process that the steering wheel moves from the third position to the fourth position, and acquiring a third video acquired by the camera in the process that the steering wheel moves in an S shape in the first position, the second position, the third position and the fourth position;
s4, matching each image in the first video, the second video and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points and all the third matching points.
It should be noted that the camera external parameters are composed of a rotation matrix R and a translation vector T, and are generally written as
Figure 92459DEST_PATH_IMAGE001
This matrix determines the pose of the camera.
As an example, in step S1, a camera is mounted on the column of the steering wheel according to the mounting position of the camera as shown in fig. 2, and when the position of the steering wheel is adjusted, the position of the camera changes, and when the steering wheel extends back and forth around the point O as a fulcrum and moves between the first position a and the second position B, or moves between the third position C and the fourth position D, the position of the camera changes, and when the steering wheel rotates up and down around the point O as a fulcrum and moves between the first position a and the fourth position D, or moves between the second position B and the third position C, the position of the camera also changes, as shown in fig. 3.
The calibration plate is arranged above the driving position, so that the camera can completely and clearly shoot the calibration plate and a preset matching area when the steering wheel moves to the first position A, the second position B, the third position C and the fourth position D, for example, an area of a window of the driving position is preset as the matching area.
It is understood that the calibration plate is a flat plate with a pattern array with a fixed pitch, and the conversion relation between the physical size and the pixels can be determined by the calibration plate and a calibration algorithm, calculating camera parameters and distortion coefficients, and the like in computer vision. The most common calibration board is the checkerboard calibration board Chessboard with the checkerboard pattern.
In a preferred embodiment of this example, the calibration plate is a checkerboard calibration plate ChArUco Board with ArUco marks. A schematic of the formation of the ChAruco Board is shown in FIG. 4.
In step S2, the steering wheel is adjusted to move to the first position a, the camera is controlled to shoot the calibration board and the matching area, and the reference image collected by the camera is obtained and recorded as IaAnd from a reference picture IaTaking the image of the middle intercepted matching area as a template image, and marking as It。
In step S3, the calibration board is kept stationary, the steering wheel is adjusted to move from the first position a to the second position B, the camera is controlled to record the calibration board and the matching area during the movement of the steering wheel, and a first video recorded as V is acquired by the cameraabAdjusting the steering wheel to move from a third position C to a fourth position D, controlling the camera to record the calibration plate and the matching area in the moving process of the steering wheel, and acquiring a second video collected by the camera and recorded as VcdAdjusting the steering wheel to move in an S-shape in the first position a, the second position B, the third position C and the fourth position D, for example, taking a bisection position between the first position a and the second position B as 0.5AB, similarly, taking a bisection position between the fourth position D and the third position C as 0.5DC, moving in an S-shape according to a path of a → D → 0.5AB → 0.5DC → B → C, in order to obtain a better effect, more equally dividing AB and DC and moving in an S-shape, controlling the camera to record a calibration plate and a matching area during the moving of the steering wheel, and obtaining a third video collected by the camera as V, and marking as Vs。
In step S4, the first video V is put into effectabEach image in (1) and template image ItMatching to obtain each image and template image I in the first videotThe best matching location point, i.e. the first matching points, is denoted as [ M ]ab1(xab1,yab1), Mab2(xab2,yab2),…, Mabm(xabm,yabm)]Second video VcdEach image in (1) and template image ItMatching to obtain each image and template image I in the second videotThe best matching location point, i.e. the second plurality of matching points, is denoted as [ M ]cd1(xcd1,ycd1), Mcd2(xcd2,ycd2),…, Mcdn(xcdn,ycdn)]A third video VsEach image in (1) and template image ItMatching to obtain each image and template image I in the third videotThe most matched location points, i.e. the third plurality of matching points, are denoted as [ M ]s1(xs1,ys1), Ms2(xs2,ys2),…, Msp(xsp,ysp)]。
It is understood that template matching is a method of pattern recognition in image processing, where a template is a known small image and template matching is a search for objects in a large image.
The camera external parameters can be rotated by a 3x3 rotation matrix R and a translation vector T (x)t，yt，zt) And (4) showing. Rotation relationships instead of using a rotation matrix representation, a rotation vector R may be usedvOr quaternion q (w)q，xq，yq，zq) Indicating that the three can be mutually transformed.
The pose of the camera is estimated for the image in the video by using OpenCV, namely the external parameters of the camera are estimated, and the rotation vector R from the camera to a coordinate system of a calibration plate can be obtainedvAnd a translation vector T. For ease of calculation, a quaternion q is used to represent the rotational relationship in the camera external parameters.
Considering that when the steering wheel takes the point O as the pivot pointThen stretching and retracting, moving between a first position A and a second position B, or moving between a third position C and a fourth position D, wherein the rotation relation between the camera and the calibration plate is unchanged, and only translation and rotation relation q are providedtQuaternion q with first position A and second position B0And quaternion q at third position C and fourth position D1Slerp calculation yields:
qt=Slerp(q0，q1，t) （1）；
wherein t is a rational number, and t is more than or equal to 0 and less than or equal to 1.
It will be appreciated that Slerp is a spherical linear interpolation, a linear interpolation operation of quaternions, primarily used to smooth differences between two quaternions representing rotations.
According to all the first matching points [ M ]ab1(xab1,yab1), Mab2(xab2,yab2),…, Mabm(xabm,yabm)]And all second matching points [ M ]cd1(xcd1,ycd1), Mcd2(xcd2,ycd2),…, Mcdn(xcdn,ycdn)]And calibrating a rotation matrix R of the camera.
Specifically, the matching points M (x, y) and t satisfy:
Figure 890650DEST_PATH_IMAGE002
（2）；
where J is a 3x1 matrix.
T =0 during the movement of the steering wheel from the first position a to the second position B, and t =1 during the movement of the steering wheel from the third position C to the fourth position D, then for the first video VabAnd a second video VcdAfter image preprocessing, each image in the image is combined with a template image ItMatching to obtain multiple first matching points [ M ]ab1,Mab2,…,Mabm]And a plurality of second matching points [ M ]cd1,Mcd2,…,Mcdn]And then:
Figure 620709DEST_PATH_IMAGE003
（3）；
the following can be obtained:
Figure 837058DEST_PATH_IMAGE004
（4）；
the t required in formula (1) is determined by substituting formula (4) into formula (2), and q is known0、q1Substituting the obtained t into formula (1), namely Slerp, to obtain a quaternion q representing a rotation relationtThus, the rotation relation q can be obtainedt。
According to all the first matching points [ M ]ab1(xab1,yab1), Mab2(xab2,yab2),…, Mabm(xabm,yabm)]All second matching points [ M ]cd1(xcd1,ycd1), Mcd2(xcd2,ycd2),…, Mcdn(xcdn,ycdn)]And all third matching points [ M ]s1(xs1.ys1), Ms2(xs2.ys2),…, Msp(xsp,ysp)]And calibrating a translation vector T of the camera.
Specifically, the matching point M (x, y) and the translation vector T satisfy:
Figure 361580DEST_PATH_IMAGE005
(5)；
where L is a 3x3 matrix.
First video VabThe second video VcdAnd a third video VsEach image in (1) has a corresponding matching point M and a translation vector T, and a series of equations can be obtained:
Figure 647068DEST_PATH_IMAGE006
（6）；
the following can be obtained:
Figure 118500DEST_PATH_IMAGE007
（7）；
the translation vector T can be obtained by bringing L into formula (5).
According to the calculated rotation relation qtAnd translating vector T, namely external reference of the camera can be marked.
According to the embodiment, the template image is obtained in advance, the image acquired after the position of the camera is changed is matched with the template image, and the rotation matrix and the translation vector of the camera, namely the camera external parameter, are automatically estimated according to the matching result, so that the camera external parameter can be automatically estimated after the position of the camera is changed, and the camera external parameter calibration efficiency is improved.
In a preferred embodiment, the matching each image of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrating the rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points specifically includes: respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points; calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
Illustratively, in practical applications, the template image I is acquired in advancetThe matrix J in equation (2) and the matrix L in equation (5) are calculated by matching the image captured by the camera with the template image I without the aid of a ChAruco Board or other calibration platetMatching to obtain a matching point M (x, y) so as to estimate the phaseThe machine parameters are the rotation matrix R and the translational vector T.
When template matching is performed, the following problems may occur: first, the matching takes longer time; secondly, due to different camera positions, matching regions in the acquired image may have different sizes, which affects the matching accuracy.
Aiming at the first problem, a series of processing and matching can be carried out after the image is cut and reduced, so that the resource consumption is reduced.
For the second problem, the template image may be scaled into a plurality of sizes and then matched to find the best matching scaling k, i.e. the best matching scaling and the matching point M.
Thus, equation (2) may be modified as follows:
Figure 173044DEST_PATH_IMAGE008
（8）；
formula (5) can be modified as follows:
Figure 386244DEST_PATH_IMAGE009
（9）；
in a corresponding manner, the first and second electrodes are,
Figure 424607DEST_PATH_IMAGE010
（10）；
Figure 230889DEST_PATH_IMAGE011
（11）。
in a preferred embodiment, the capturing an image of the matching region from the reference image as a template image specifically includes: and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting an image of the matching area from the intermediate image as a template image.
In a preferred embodiment, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points specifically is: the method comprises the steps of conducting image preprocessing on each image in a first video, a second video and a third video respectively to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and matching each first image to be matched, each second image to be matched and each third image to be matched with a template image respectively to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
In a preferred embodiment, image pre-processing includes edge detection, morphological erosion and dilation.
It will be appreciated that edge detection is a fundamental problem in image processing and computer vision, the purpose of which is to identify points in a digital image where changes in brightness are significant. Morphological processing is a technique of analyzing an image by a computer using basic operations of mathematical morphology, the most basic morphological operations being dilation and erosion.
In the embodiment, before the image of the matching area is intercepted from the reference image as the template image, the edge detection, the morphological erosion and the expansion are carried out on the reference image, and before each image in the video is matched with the template image, the edge detection, the morphological erosion and the expansion are carried out on each image in the video, so that the matching accuracy is improved.
Based on the same inventive concept as the first embodiment, the second embodiment provides an internal vehicle external reference calibration apparatus as shown in fig. 5, including; the matching area confirming module 21 is used for installing the camera on a pipe column of the steering wheel, installing a calibration plate above the driving position and enabling the camera to completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position; the template image acquisition module 22 is configured to acquire a reference image acquired by the camera when the steering wheel moves to the first position, and intercept an image of the matching area from the reference image as a template image; the multi-directional video acquisition module 23 is configured to acquire a first video acquired by the camera in a process that the steering wheel moves from the first position to the second position, acquire a second video acquired by the camera in a process that the steering wheel moves from the third position to the fourth position, and acquire a third video acquired by the camera in a process that the steering wheel moves in an S-shape in the first position, the second position, the third position, and the fourth position; the camera external reference calibration module 24 is configured to match each image of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrate a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrate a translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points.
In a preferred embodiment, the matching each image of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrating the rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points specifically includes: respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points; calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
In a preferred embodiment, the capturing an image of the matching region from the reference image as a template image specifically includes: and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting an image of the matching area from the intermediate image as a template image.
In a preferred embodiment, the matching each image in the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points specifically is: the method comprises the steps of conducting image preprocessing on each image in a first video, a second video and a third video respectively to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and matching each first image to be matched, each second image to be matched and each third image to be matched with a template image respectively to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
In a preferred embodiment, image pre-processing includes edge detection, morphological erosion and dilation.
In summary, the embodiment of the present invention has the following advantages:
the camera is arranged on a pipe column of a steering wheel, a calibration plate is arranged above a driving position, the camera can completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position, a reference image collected by the camera is obtained when the steering wheel moves to the first position, an image of the matching area is intercepted from the reference image and is used as a template image, a first video collected by the camera is obtained in the process that the steering wheel moves from the first position to the second position, a second video collected by the camera is obtained in the process that the steering wheel moves from the third position to the fourth position, a third video collected by the camera is obtained in the process that the steering wheel moves in an S shape in the first position, the second position, the third position and the fourth position, and each image of the first video, the second video and the third video is matched with the template image respectively, and obtaining a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating translation vectors of the camera according to all the first matching points, all the second matching points and all the third matching points to realize the external reference calibration of the vehicle inside the vehicle. According to the embodiment of the invention, the template image is obtained in advance, the image acquired after the position of the camera is changed is matched with the template image, and the rotation matrix and the translation vector of the camera, namely the camera external parameter, are automatically estimated according to the matching result, so that the camera external parameter can be automatically estimated after the position of the camera is changed, and the camera external parameter calibration efficiency is improved.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
It will be understood by those skilled in the art that all or part of the processes of the above embodiments may be implemented by hardware related to instructions of a computer program, and the computer program may be stored in a computer readable storage medium, and when executed, may include the processes of the above embodiments. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

Claims (10)

1. A calibration method for external parameters of an in-vehicle phase is characterized by comprising the following steps of;
installing a camera on a pipe column of a steering wheel, installing a calibration plate above a driving position, and enabling the camera to completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position;
when the steering wheel moves to the first position, acquiring a reference image acquired by the camera, and intercepting an image of the matching area from the reference image as a template image;
acquiring a first video acquired by the camera during the movement of the steering wheel from the first position to the second position, acquiring a second video acquired by the camera during the movement of the steering wheel from the third position to the fourth position, and acquiring a third video acquired by the camera during the S-shaped movement of the steering wheel within the first position, the second position, the third position and the fourth position;
and respectively matching each image in the first video, the second video and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points and all the third matching points.
2. The in-vehicle external reference calibration method according to claim 1, wherein the matching is performed on each of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points, and all the third matching points, specifically:
respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points;
calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
3. The in-vehicle external reference calibration method according to claim 1, wherein the image of the matching area is cut from the reference image as a template image, specifically:
and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting the image of the matching area from the intermediate image as the template image.
4. The in-vehicle external reference calibration method according to claim 1, wherein the step of matching each image of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points comprises:
respectively carrying out image preprocessing on each image in the first video, the second video and the third video to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and respectively matching each first image to be matched, each second image to be matched and each third image to be matched with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
5. The in-vehicle external reference calibration method according to claim 3 or 4, wherein the image preprocessing comprises edge detection, morphological erosion and dilation.
6. An external reference calibration device for an automobile interior phase is characterized by comprising;
the matching area confirming module is used for installing a camera on a pipe column of a steering wheel, installing a calibration plate above a driving position and enabling the camera to completely shoot the calibration plate and a preset matching area when the steering wheel moves to a first position, a second position, a third position and a fourth position;
the template image acquisition module is used for acquiring a reference image acquired by the camera when the steering wheel moves to the first position, and intercepting an image of the matching area from the reference image as a template image;
a multi-directional video acquisition module, configured to acquire a first video captured by the camera during movement of the steering wheel from the first position to the second position, acquire a second video captured by the camera during movement of the steering wheel from the third position to the fourth position, and acquire a third video captured by the camera during S-shaped movement of the steering wheel within the first position, the second position, the third position, and the fourth position;
and the camera external parameter calibration module is used for respectively matching each image in the first video, the second video and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points, calibrating a rotation matrix of the camera according to all the first matching points and all the second matching points, and calibrating a translation vector of the camera according to all the first matching points, all the second matching points and all the third matching points.
7. The in-vehicle external reference calibration device according to claim 6, wherein the matching is performed on each of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, the calibration of the rotation matrix of the camera is performed according to all the first matching points and all the second matching points, and the calibration of the translation vector of the camera is performed according to all the first matching points, all the second matching points, and all the third matching points, specifically:
respectively determining the optimal matching proportion of the template image and each image in the first video, the second video and the third video, and respectively matching each image in the first video, the second video and the third video with the template image which is scaled by the corresponding optimal matching proportion to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points;
calibrating the rotation matrix of the camera according to all the first matching points, all the second matching points and the corresponding optimal matching proportion, and calibrating the translation vector of the camera according to all the first matching points, all the second matching points, all the third matching points and the corresponding optimal matching proportion.
8. The in-vehicle external reference calibration device according to claim 6, wherein the image of the matching area is cut from the reference image as a template image, specifically:
and carrying out image preprocessing on the reference image to obtain an intermediate image, and intercepting the image of the matching area from the intermediate image as the template image.
9. The in-vehicle external reference calibration device according to claim 6, wherein the matching is performed on each of the first video, the second video, and the third video with the template image to obtain a plurality of first matching points, a plurality of second matching points, and a plurality of third matching points, specifically:
respectively carrying out image preprocessing on each image in the first video, the second video and the third video to obtain a plurality of first images to be matched, a plurality of second images to be matched and a plurality of third images to be matched, and respectively matching each first image to be matched, each second image to be matched and each third image to be matched with the template image to obtain a plurality of first matching points, a plurality of second matching points and a plurality of third matching points.
10. The in-vehicle external reference calibration device according to claim 8 or 9, wherein the image preprocessing includes edge detection, morphological erosion and dilation.
CN202210029319.1A 2022-01-12 2022-01-12 Method and device for calibrating internal phase and external phase of vehicle Active CN114049404B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210029319.1A CN114049404B (en) 2022-01-12 2022-01-12 Method and device for calibrating internal phase and external phase of vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210029319.1A CN114049404B (en) 2022-01-12 2022-01-12 Method and device for calibrating internal phase and external phase of vehicle

Publications (2)

Publication Number Publication Date
CN114049404A true CN114049404A (en) 2022-02-15
CN114049404B CN114049404B (en) 2022-04-05

Family

ID=80196345

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210029319.1A Active CN114049404B (en) 2022-01-12 2022-01-12 Method and device for calibrating internal phase and external phase of vehicle

Country Status (1)

Country Link
CN (1) CN114049404B (en)

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100245576A1 (en) * 2009-03-31 2010-09-30 Aisin Seiki Kabushiki Kaisha Calibrating apparatus for on-board camera of vehicle
JP2018177004A (en) * 2017-04-13 2018-11-15 クラリオン株式会社 Calibration device for on-vehicle camera
JP2019050622A (en) * 2018-11-28 2019-03-28 株式会社東芝 Image processing apparatus, image processing method, image processing program, and image processing system
US20190118712A1 (en) * 2016-04-03 2019-04-25 Denso Corporation Mounting angle detection device for in-vehicle camera, mounting angle calibration device, and mounting angle detection method
CN109690623A (en) * 2016-06-29 2019-04-26 醒眸行有限公司 The system and method for the posture of camera in scene for identification
US20200082568A1 (en) * 2018-09-06 2020-03-12 Aisin Seiki Kabushiki Kaisha Camera calibration device
US20200167951A1 (en) * 2018-11-27 2020-05-28 GM Global Technology Operations LLC Movement tracking of operator-facing cameras
US20200211224A1 (en) * 2017-07-28 2020-07-02 Clarion Co., Ltd. Calibration apparatus and method for in-vehicle camera
CN112183512A (en) * 2020-12-02 2021-01-05 深圳佑驾创新科技有限公司 Camera calibration method, device, vehicle-mounted terminal and storage medium
CN112330756A (en) * 2021-01-04 2021-02-05 中智行科技有限公司 Camera calibration method and device, intelligent vehicle and storage medium
WO2021026705A1 (en) * 2019-08-09 2021-02-18 华为技术有限公司 Matching relationship determination method, re-projection error calculation method and related apparatus
CN112489113A (en) * 2020-11-25 2021-03-12 深圳地平线机器人科技有限公司 Camera external parameter calibration method and device and camera external parameter calibration system
CN112509054A (en) * 2020-07-20 2021-03-16 北京智行者科技有限公司 Dynamic calibration method for external parameters of camera
CN112862899A (en) * 2021-02-07 2021-05-28 黑芝麻智能科技(重庆)有限公司 External parameter calibration method, device and system for image acquisition equipment
WO2021233309A1 (en) * 2020-05-21 2021-11-25 杭州海康威视数字技术股份有限公司 Extrinsic parameter change detection method and apparatus, electronic device, and detection system

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100245576A1 (en) * 2009-03-31 2010-09-30 Aisin Seiki Kabushiki Kaisha Calibrating apparatus for on-board camera of vehicle
US20190118712A1 (en) * 2016-04-03 2019-04-25 Denso Corporation Mounting angle detection device for in-vehicle camera, mounting angle calibration device, and mounting angle detection method
CN109690623A (en) * 2016-06-29 2019-04-26 醒眸行有限公司 The system and method for the posture of camera in scene for identification
JP2018177004A (en) * 2017-04-13 2018-11-15 クラリオン株式会社 Calibration device for on-vehicle camera
US20200211224A1 (en) * 2017-07-28 2020-07-02 Clarion Co., Ltd. Calibration apparatus and method for in-vehicle camera
US20200082568A1 (en) * 2018-09-06 2020-03-12 Aisin Seiki Kabushiki Kaisha Camera calibration device
US20200167951A1 (en) * 2018-11-27 2020-05-28 GM Global Technology Operations LLC Movement tracking of operator-facing cameras
JP2019050622A (en) * 2018-11-28 2019-03-28 株式会社東芝 Image processing apparatus, image processing method, image processing program, and image processing system
WO2021026705A1 (en) * 2019-08-09 2021-02-18 华为技术有限公司 Matching relationship determination method, re-projection error calculation method and related apparatus
WO2021233309A1 (en) * 2020-05-21 2021-11-25 杭州海康威视数字技术股份有限公司 Extrinsic parameter change detection method and apparatus, electronic device, and detection system
CN112509054A (en) * 2020-07-20 2021-03-16 北京智行者科技有限公司 Dynamic calibration method for external parameters of camera
CN112489113A (en) * 2020-11-25 2021-03-12 深圳地平线机器人科技有限公司 Camera external parameter calibration method and device and camera external parameter calibration system
CN112183512A (en) * 2020-12-02 2021-01-05 深圳佑驾创新科技有限公司 Camera calibration method, device, vehicle-mounted terminal and storage medium
CN112330756A (en) * 2021-01-04 2021-02-05 中智行科技有限公司 Camera calibration method and device, intelligent vehicle and storage medium
CN112862899A (en) * 2021-02-07 2021-05-28 黑芝麻智能科技(重庆)有限公司 External parameter calibration method, device and system for image acquisition equipment

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JUN TAN ET AL: "Automatic Extrinsic Calibration for an Onboard Camera", 《2013 CHINESE AUTOMATION CONGRESS》 *
刘昱岗等: "基于双目立体视觉的倒车环境障碍物测量方法", 《交通运输系统工程与信息》 *
张娟等: "基于图像处理与计算机视觉的车辆四轮定位仪的设计与实现", 《现代电子技术》 *

Also Published As

Publication number Publication date
CN114049404B (en) 2022-04-05

Similar Documents

Publication Publication Date Title
US10462362B2 (en) Feature based high resolution motion estimation from low resolution images captured using an array source
EP2870585B1 (en) A method and system for correcting a distorted image
CN107749987B (en) Digital video image stabilization method based on block motion estimation
US20110299761A1 (en) Image Processing Apparatus, Image Processing Method, and Program
US20100302402A1 (en) Image processing apparatus, image processing method, and program
CN110458788B (en) Homography correction system, method and computer readable storage medium
US20180174326A1 (en) Method, System and Apparatus for Determining Alignment Data
CN113223066B (en) Multi-source remote sensing image matching method and device based on characteristic point fine tuning
CN110225321B (en) Training sample data acquisition system and method for trapezoidal correction
CN111292278A (en) Image fusion method and device, storage medium and terminal
CN114049404B (en) Method and device for calibrating internal phase and external phase of vehicle
CN113838138A (en) System calibration method, system, device and medium for optimizing feature extraction
CN116524041A (en) Camera calibration method, device, equipment and medium
CN115965697A (en) Projector calibration method, calibration system and device based on Samm's law
US11810266B2 (en) Pattern radius adjustment for keypoint descriptor generation
CN113837949A (en) Image processing method and device
JPH09128550A (en) Method and device for processing image
CN109767390A (en) A kind of digital picture of block parallel disappears image rotation method
CN113570529A (en) Method for fusing images and data processing equipment
CN114419168B (en) Calibration method and device for image feature points
JP2703454B2 (en) Image pattern matching method
CN111914856B (en) Layout method, device and system for plate excess material, electronic equipment and storage medium
Yuan et al. Digital image stabilization based on log-polar transform
CN118053160A (en) Seal image recognition method and device
CN117523167A (en) Industrial instrument image automatic correction method based on Yolov8 and perspective transformation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20221229

Address after: No. 602-165, Complex Building, No. 1099, Qingxi Second Road, Hezhuang Street, Qiantang District, Hangzhou, Zhejiang, 310000

Patentee after: Hangzhou Ruijian Zhixing Technology Co.,Ltd.

Address before: 518051 401, building 1, Shenzhen new generation industrial park, No. 136, Zhongkang Road, Meidu community, Meilin street, Futian District, Shenzhen, Guangdong Province

Patentee before: SHENZHEN MINIEYE INNOVATION TECHNOLOGY Co.,Ltd.