CN114387350A - Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment - Google Patents
Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment Download PDFInfo
- Publication number
- CN114387350A CN114387350A CN202111469484.0A CN202111469484A CN114387350A CN 114387350 A CN114387350 A CN 114387350A CN 202111469484 A CN202111469484 A CN 202111469484A CN 114387350 A CN114387350 A CN 114387350A
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- vehicle
- coordinates
- world
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000006243 chemical reaction Methods 0.000 claims abstract description 46
- 238000004364 calculation method Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 88
- 230000009466 transformation Effects 0.000 claims description 75
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 4
- 238000006073 displacement reaction Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000011217 control strategy Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a vehicle-mounted camera external parameter calibration method, a system, a storage medium and equipment, which comprise the steps of obtaining vehicle position information, searching for characteristic points near the vehicle position information in a high-precision map, obtaining world coordinates of the same characteristic point in a world coordinate system and camera coordinates in a camera coordinate system, inputting the world coordinates and the camera coordinates into a pre-trained conversion model, and obtaining camera external parameters by calculation by combining the vehicle position information. According to the vehicle-mounted camera external reference calibration method, the system, the storage medium and the equipment, the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system are matched, external reference calibration of the camera is not needed in a specific calibration period, the external reference calibration can be performed at any time and any place, the camera external reference calibration convenience is improved, and the external reference calibration cost is reduced.
Description
Technical Field
The invention relates to the technical field of camera calibration, in particular to a method, a device, a storage medium and equipment for calibrating external parameters of a vehicle-mounted camera.
Background
With the development of the automatic driving technology, the vehicle-mounted camera becomes an important component in the automatic driving vehicle. The vehicle-mounted camera is a sensor for automatically driving the automobile, and the automatic driving sensing module identifies information such as obstacles, lane lines and the like in the environment through image information acquired by the camera, provides environment information for the automatically driving automobile and provides basis for subsequent planning and control strategies.
The camera includes internal and external parameters, which are determined by the camera itself and are only relevant to the camera itself. The accuracy of the sensing function of the automatic driving automobile depends on the accuracy of the internal and external parameters of the camera, and after the automobile runs for a period of time, the external parameters of the vehicle-mounted camera change due to the reasons of vehicle vibration, environmental temperature and humidity change and the like, so that the accuracy of the sensing function of the automatic driving automobile is reduced, the running safety is affected, and the external parameters of the camera need to be calibrated.
In the prior art, a general external reference calibration method needs to be realized by depending on a specific calibration object in a specific calibration room, and if the external reference calibration needs to be carried out again in the calibration room every time of calibration, the required time cost and coordination cost are high.
Disclosure of Invention
Based on the above, the invention aims to provide a method, a device, a storage medium and equipment for calibrating external parameters of a vehicle-mounted camera, and solve the problems that in the background art, external parameters need to be calibrated again in a calibration room, and the time cost and the coordination cost are high.
The invention provides a method for calibrating external parameters of a vehicle-mounted camera, which is applied to a vehicle and comprises the following steps:
acquiring vehicle position information, and determining characteristic points near the vehicle position information in a high-precision map;
acquiring world coordinates of the feature points in a world coordinate system based on the high-precision map;
acquiring camera coordinates of the feature points under the camera coordinates;
and inputting the world coordinate and the camera coordinate into the conversion model according to the conversion model from the pre-trained camera coordinate system to the world coordinate system, and obtaining a second conversion matrix between the camera coordinate system and the vehicle body coordinate system by combining the vehicle position information, wherein the second conversion matrix is calibrated camera external parameters.
According to the method, the vehicle position information is acquired, the characteristic points near the vehicle position information are searched in a high-precision map, the world coordinates of the same characteristic point in a world coordinate system and the camera coordinates in a camera coordinate system are acquired, the world coordinates and the camera coordinates are input into a pre-trained conversion model, and the camera external parameters are obtained through calculation by combining the vehicle position information. By matching the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system, external reference calibration of the camera is not needed in a specific calibration period, and can be performed at any time and any place, so that the problems of high calibration time cost and coordination cost in the specific calibration period in the background technology are solved.
Further, the pre-trained conversion model of the camera coordinate system to the world coordinate system includes:
establishing a first transformation matrix from the vehicle body coordinate system to the world coordinate system and a second transformation matrix from the camera coordinate system to the vehicle body coordinate system;
and establishing a third transformation matrix from the camera coordinate system to the world coordinate system according to the first transformation matrix and the second transformation matrix.
Further, the step of acquiring the camera coordinates of the feature points under the camera coordinates includes:
and acquiring pixel coordinates of the feature points in an image coordinate system and camera internal parameters, and calculating the camera coordinates of the feature points in the camera coordinate system through the camera internal parameters by combining the pixel coordinates.
Further, the method further comprises:
and acquiring world coordinates of the vehicle in the world coordinate system according to the vehicle position information, and acquiring a first transformation matrix from the world coordinate system to the vehicle body coordinate system according to the world coordinates of the vehicle.
Further, the step of inputting the world coordinate and the camera coordinate into the conversion model according to the pre-trained conversion model from the camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information includes:
inputting the world coordinates and the camera coordinates into a conversion model of the camera coordinate system into the world coordinate system;
and calculating to obtain the third transformation matrix according to the world coordinate and the camera coordinate, and calculating to obtain the second transformation matrix according to the third transformation matrix and the first transformation matrix obtained based on the vehicle position information.
Further, the step of inputting the world coordinate and the camera coordinate into the conversion model according to the pre-trained conversion model from the camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information further includes:
and acquiring a plurality of characteristic points near the vehicle position information, and fitting a plurality of second transformation matrixes based on a plurality of second transformation matrixes obtained by calculation of the plurality of characteristic points to obtain an optimal second transformation matrix.
In another aspect, the present invention provides an external parameter calibration device for a vehicle-mounted camera, comprising:
the characteristic point acquisition module is used for acquiring vehicle position information and determining characteristic points near the vehicle position information in a high-precision map;
the world coordinate acquisition module is used for acquiring world coordinates of the feature points in a world coordinate system based on the high-precision map;
the camera coordinate acquisition module is used for acquiring camera coordinates of the feature points under the camera coordinates;
and the camera external parameter calculation module is used for inputting the world coordinates and the camera coordinates into the conversion model according to the conversion model from the pre-trained camera coordinate system to the world coordinate system, and obtaining a second conversion matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information, wherein the second conversion matrix is the calibrated camera external parameters.
Another aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the vehicle-mounted camera external parameter calibration method as described above.
The invention also provides a data processing device which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the vehicle-mounted camera external parameter calibration method is realized.
Drawings
Fig. 1 is a flowchart of a vehicle-mounted camera external parameter calibration method in a first embodiment of the present invention;
FIG. 2 is a flowchart of a calibration method for external parameters of a vehicle-mounted camera according to a second embodiment of the present invention;
FIG. 3 is a block diagram of an apparatus according to a third embodiment of the present invention;
the following detailed description will further illustrate the invention in conjunction with the above-described figures.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Several embodiments of the invention are presented in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like are used herein for illustrative purposes only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
When the camera is used for external reference calibration, the following coordinate systems are commonly used:
world coordinate system: the coordinates of an object in the real world, such as the origin of the world coordinate system of a black and white checkerboard, are set at the vertex of the first checkerboard, Xw, Yw, Zw are mutually perpendicular, and the Zw direction is the direction perpendicular to the checkerboard panel. The visible world coordinate system varies with the size and position of the object, and the unit is a length unit. As long as the size of the checkerboard is determined, the coordinates of the corner points of the checkerboard generally do not change (because the position relative to the world coordinate system origin is not changed), and Zw is regarded as 0.
Camera coordinate system: the optical center is taken as the origin of the camera coordinate system, the x and y directions parallel to the image are taken as the Xc axis and the Yc axis, the Zc axis is parallel to the optical axis, and the Xc, Yc and Zc are perpendicular to each other and have the length unit.
Image coordinate system: the u and v directions are parallel to the x and y directions with the apex of the image as the origin of coordinates, and the units are in pixels.
The coordinate system of the vehicle body takes the center of a rear axle of the vehicle as an origin, and takes the driving direction of the vehicle as a reference, and gives the directions of all coordinate axes: x is front (direction of travel), y is left, and z is up.
The camera extrinsic parameters are divided into a rotation matrix and a translation matrix, which together describe how to convert points from the world coordinate system to the camera coordinate system. The rotation matrix describes the orientation of the coordinate axes of the world coordinate system relative to the camera coordinate axes; the translation matrix describes the position of the spatial origin in the camera coordinate system.
The invention mainly aims to solve the transformation matrix of the same characteristic point from an image coordinate system to a world coordinate system by utilizing characteristic information in a high-precision map of an automatic driving automobile, and further solve and obtain the transformation matrix from a camera coordinate system to an automobile body coordinate system, wherein the transformation matrix from the camera coordinate system to the automobile body coordinate system is the camera external reference.
Example one
Referring to fig. 1, a vehicle-mounted camera external parameter calibration method according to a first embodiment of the present invention includes steps S101-S104.
S101, vehicle position information is acquired, and feature points near the vehicle position information are determined in a high-precision map.
According to the vehicle positioning modules GPS and IMU, longitude and latitude information obtained by the GPS is used as an input signal and is transmitted into the IMU, and the IMU is connected with the controller through a serial port line so as to obtain a positioning result with higher frequency and obtain vehicle position information.
The absolute accuracy of a high-accuracy map is generally on a sub-meter level, i.e., an accuracy within 1 meter, and the lateral relative accuracy (e.g., the relative position accuracy of a lane and a lane, and a lane line) is often higher. The high-precision data comprises absolute geographic coordinates of the traffic participants, physical dimensions, characteristic characteristics of the traffic participants and the like, such as pedestrian crossings, boards along the roads, isolation zones, speed limit signs, traffic lights, phone stops at the sides of the roads and the like.
Therefore, the current position of the vehicle can be positioned by utilizing the high-precision map, and available characteristic points such as street lamps, road rods, parking space angular points, vehicle line lines, pedestrian crossings and the like can be searched near the position of the vehicle.
And S102, determining world coordinates of the feature points in a world coordinate system.
Determining the position information of the feature points according to the high-precision map, and recording the world coordinates of the feature points in a world coordinate system as (X)w,Yw,Zw) And the world coordinates are established based on the characteristic information of the high-precision map.
And S103, acquiring camera coordinates of the feature points under the camera coordinates.
The method comprises the steps of obtaining an image near a vehicle according to a vehicle-mounted camera, finding the same feature point in the image, recording the pixel coordinates (u, v) of the feature point in an image coordinate system, and calculating the coordinates (X) of the feature point in a camera system through internal parameters of the camerac,Yc,Zc)。
Wherein (u)0,v0) Is the coordinate of the origin of the image coordinate system in the pixel coordinate system, dxAnd dyRespectively the physical dimensions of each pixel in the x and y directions of the image plane, and f is the camera focal length.
S104, inputting the world coordinate and the camera coordinate into a conversion model according to the conversion model from the camera coordinate system to the world coordinate system which is pre-trained, and obtaining a second conversion matrix from the camera coordinate system to a vehicle body coordinate system by combining the vehicle position information, wherein the second conversion matrix is calibrated camera external parameters.
Establishing a conversion model from a camera coordinate system to a world coordinate system:
let the coordinate of the characteristic point M in the coordinate system of the vehicle body be (X)i,Yi,Zi)，
Establishing a first transformation matrix from the body coordinate system to the world coordinate system
The relative rotation between the world coordinate system and the vehicle body coordinate system is a matrix R1The relative displacement being a vector T1，
Establishing a second transformation matrix between the camera coordinate system and the vehicle body coordinate system
The relative rotation between the coordinate system of the vehicle body and the coordinate system of the camera is a matrix R2The relative displacement being a vector T2，
Wherein the transformation matrix is represented by a homogeneous coordinate matrix formed by combining a rotation matrix and a translation vector:
the coordinate system of the vehicle body is the origin, and a first transformation matrix from the world coordinate system to the coordinate system of the vehicle body is obtained according to the acquired vehicle positioning information
World coordinate (X) of characteristic pointw,Yw,Zw) Camera coordinates (X)c,Yc,Zc) First transformation matrix
Inputting the transformation model to calculate the relative rotation between the coordinate system of the vehicle body and the coordinate system of the camera as a matrix R2The relative displacement being a vector T2To obtain a transformation matrix between the two
Namely the external parameters of the camera.
In summary, in the method for calibrating external parameters of a vehicle-mounted camera in the above embodiment of the present invention, the vehicle position information is obtained, the feature point near the vehicle position information is searched in the high-precision map, the world coordinate of the same feature point in the world coordinate system and the camera coordinate in the camera coordinate system are obtained, the world coordinate and the camera coordinate are input into the pre-trained transformation model, and the external parameters of the camera are obtained by calculation in combination with the vehicle position information. By matching the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system, the camera can be calibrated at any time and any place without calibrating the external reference in a specific calibration period, thereby solving the problems of high calibration time cost and coordination cost in the background technology.
Example two
Referring to fig. 2, a vehicle-mounted camera external parameter calibration method according to a first embodiment of the present invention is shown, including steps S201-S207.
S201, vehicle position information is acquired, and feature points near the vehicle position information are determined in a high-precision map.
According to the vehicle positioning modules GPS and IMU, longitude and latitude information obtained by the GPS is used as an input signal and is transmitted into the IMU, and the IMU is connected with the controller through a serial port line so as to obtain a positioning result with higher frequency and obtain vehicle position information.
The absolute accuracy of a high-accuracy map is generally on a sub-meter level, i.e., an accuracy within 1 meter, and the lateral relative accuracy (e.g., the relative position accuracy of a lane and a lane, and a lane line) is often higher. The high-precision data comprises absolute geographic coordinates of the traffic participants, physical dimensions, characteristic characteristics of the traffic participants and the like, such as pedestrian crossings, boards along the roads, isolation zones, speed limit signs, traffic lights, phone stops at the sides of the roads and the like.
Therefore, the current position of the vehicle can be positioned by utilizing the high-precision map, and available characteristic points such as street lamps, road rods, parking space angular points, vehicle line lines, pedestrian crossings and the like can be searched near the position of the vehicle.
And S202, determining world coordinates of the characteristic points in a world coordinate system.
Determining the position information of the feature points according to the high-precision map, and recording the world coordinates of the feature points in a world coordinate system as (X)w,Yw,Zw) And the world coordinates are established based on the characteristic information of the high-precision map.
S203, finding out corresponding characteristic points in the image, acquiring pixel coordinates of the characteristic points in the image, and calculating camera coordinates of the characteristic points in the camera coordinates through camera internal parameters.
The method comprises the steps of obtaining an image near a vehicle according to a vehicle-mounted camera, finding the same feature point in the image, recording the pixel coordinates (u, v) of the feature point in an image coordinate system, and calculating the coordinates (X) of the feature point in a camera system through internal parameters of the camerac,Yc,Zc)。
Wherein (u)0,v0) Is the coordinate of the origin of the image coordinate system in the pixel coordinate system, dxAnd dyRespectively the physical dimensions of each pixel in the x and y directions of the image plane, and f is the camera focal length.
And S204, establishing a conversion model from the camera coordinate system to the world coordinate system.
Let the coordinate of the characteristic point M in the coordinate system of the vehicle body be (X)i,Yi,Zi)，
Establishing a first transformation matrix from the body coordinate system to the world coordinate system
The relative rotation between the world coordinate system and the vehicle body coordinate system is a matrix R1The relative displacement being a vector T1，
Establishing a second transformation matrix between the camera coordinate system and the vehicle body coordinate system
The relative rotation between the coordinate system of the vehicle body and the coordinate system of the camera is a matrix R2The relative displacement being a vector T2，
Wherein the transformation matrix is represented by a homogeneous coordinate matrix formed by combining a rotation matrix and a translation vector:
and S205, obtaining a first transformation matrix from the world coordinate system to the vehicle body coordinate system according to the vehicle position information.
The coordinate system of the vehicle body is the origin, and a first transformation matrix from the world coordinate system to the coordinate system of the vehicle body is obtained according to the acquired vehicle positioning information
S206, according to the conversion model from the camera coordinate system to the world coordinate system, inputting the world coordinate, the camera coordinate and the first conversion matrix from the world coordinate system to the vehicle body coordinate system into the conversion model to obtain a second conversion matrix from the camera coordinate system to the vehicle body coordinate system.
World coordinate (X) of characteristic pointw,Yw,Zw) Camera coordinates (X)c,Yc,Zc) First transformation matrix
Inputting the transformation model to calculate the relative rotation between the coordinate system of the vehicle body and the coordinate system of the camera as a matrix R2The relative displacement being a vector T2To obtain a transformation matrix between the two
Namely the external parameters of the camera.
S207, obtaining a plurality of feature points, calculating based on the feature points to obtain a plurality of second transformation matrixes, and fitting the second transformation matrixes to obtain an optimal second transformation matrix.
Searching a plurality of characteristic points near the vehicle position information, calculating a second transformation matrix obtained by each characteristic point according to the steps S201-S206, and fitting the plurality of second transformation matrices by a least square method or other fitting methods to obtain an optimal second transformation matrix, namely an optimal camera external parameter.
In summary, in the method for calibrating external parameters of a vehicle-mounted camera in the above embodiment of the present invention, the vehicle position information is obtained, the feature point near the vehicle position information is searched in the high-precision map, the world coordinate of the same feature point in the world coordinate system and the camera coordinate in the camera coordinate system are obtained, the world coordinate and the camera coordinate are input into the pre-trained transformation model, and the external parameters of the camera are obtained by calculation in combination with the vehicle position information. By matching the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system, the camera can be calibrated at any time and any place without calibrating the external reference in a specific calibration period, thereby solving the problems of high calibration time cost and coordination cost in the background technology.
EXAMPLE III
In another aspect, the present invention further provides a calibration apparatus for external reference of a vehicle-mounted camera, referring to fig. 3, which is a block diagram of the calibration apparatus for external reference of a vehicle-mounted camera, and the apparatus includes:
the characteristic point acquisition module is used for acquiring vehicle position information and determining characteristic points near the vehicle position information in a high-precision map;
the world coordinate acquisition module is used for acquiring world coordinates of the feature points in a world coordinate system based on the high-precision map;
the camera coordinate acquisition module is used for acquiring camera coordinates of the feature points under the camera coordinates;
and the camera external parameter calculation module is used for inputting the world coordinate and the camera coordinate into the conversion model according to the conversion model from the pre-trained camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information, wherein the second transformation matrix is calibrated camera external parameters.
Further, in some optional embodiments of the present invention, the camera external parameter calculating module further includes:
the conversion model unit is used for establishing a first transformation matrix from the vehicle body coordinate system to the world coordinate system and a second transformation matrix from the camera coordinate system to the vehicle body coordinate system;
and establishing a third transformation matrix from the camera coordinate system to the world coordinate system according to the first transformation matrix and the second transformation matrix.
Further, in some optional embodiments of the present invention, the camera coordinate acquiring module further includes:
and the camera coordinate conversion unit is used for acquiring the pixel coordinates of the characteristic points in an image coordinate system and camera internal parameters, and obtaining the camera coordinates of the characteristic points in the camera coordinate system through the camera internal parameter calculation by combining the pixel coordinates.
Further, in some optional embodiments of the invention, the apparatus further comprises:
and the first transformation matrix calculation module is used for acquiring the world coordinates of the vehicle in the world coordinate system according to the vehicle position information and obtaining a first transformation matrix from the world coordinate system to the vehicle body coordinate system according to the world coordinates of the vehicle.
Further, in some optional embodiments of the present invention, the camera external parameter calculating module further includes:
a matrix conversion unit for inputting the world coordinates and the camera coordinates into a conversion model of the camera coordinate system into the world coordinate system;
and calculating to obtain the third transformation matrix according to the world coordinate and the camera coordinate, and calculating to obtain the second transformation matrix according to the third transformation matrix and the first transformation matrix obtained based on the vehicle position information, wherein the second transformation matrix is the external parameter of the camera.
Further, in some optional embodiments of the invention, the apparatus further comprises:
and the matrix fitting module is used for acquiring a plurality of characteristic points near the vehicle position information, and fitting a plurality of second transformation matrixes based on a plurality of second transformation matrixes obtained by calculation of the plurality of characteristic points to obtain an optimal second transformation matrix.
The functions or operation steps of the modules and units when executed are substantially the same as those of the method embodiments, and are not described herein again.
In summary, in the vehicle-mounted camera external parameter calibration apparatus in the above embodiment of the present invention, the vehicle position information is obtained, the feature point near the vehicle position information is searched in the high-precision map, the world coordinate of the same feature point in the world coordinate system and the camera coordinate in the camera coordinate system are obtained, the world coordinate and the camera coordinate are input into the pre-trained conversion model, and the camera external parameter is obtained by calculation in combination with the vehicle position information. By matching the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system, the camera can be calibrated at any time and any place without calibrating the external reference in a specific calibration period, thereby solving the problems of high calibration time cost and coordination cost in the background technology.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the external parameter calibration method for a vehicle-mounted camera in the foregoing embodiments.
Example four
The invention also provides vehicle-mounted camera external parameter calibration equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the vehicle-mounted camera external parameter calibration method in the embodiment. In some embodiments, the processor may be an Electronic Control Unit (ECU), a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, and is configured to run program codes stored in the memory or process data, such as executing an access restriction program.
Wherein the memory includes at least one type of readable storage medium including flash memory, hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory may in some embodiments be an internal storage unit of the vehicle, for example a hard disk of the vehicle. The memory may also be an external storage device of the vehicle in other embodiments, such as a plug-in hard drive provided on the vehicle, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card (FlashCard), and the like. Further, the memory may also include both an internal storage unit and an external storage device of the vehicle. The memory may be used not only to store application software installed in the vehicle and various types of data, but also to temporarily store data that has been output or is to be output.
In summary, in the vehicle-mounted camera external parameter calibration apparatus in the above embodiment of the present invention, the vehicle position information is obtained, the feature point near the vehicle position information is searched in the high-precision map, the world coordinate of the same feature point in the world coordinate system and the camera coordinate in the camera coordinate system are obtained, the world coordinate and the camera coordinate are input into the pre-trained conversion model, and the camera external parameter is obtained by calculation in combination with the vehicle position information. By matching the coordinates of the same characteristic point in the camera coordinate system and the world coordinate system, the camera can be calibrated at any time and any place without calibrating the external reference in a specific calibration period, thereby solving the problems of high calibration time cost and coordination cost in the background technology.
The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a sequential list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. A vehicle-mounted camera external parameter calibration method is applied to a vehicle, and is characterized by comprising the following steps:
acquiring vehicle position information, and determining characteristic points near the vehicle position information in a high-precision map;
acquiring world coordinates of the feature points in a world coordinate system based on the high-precision map;
acquiring camera coordinates of the feature points under the camera coordinates;
and inputting the world coordinate and the camera coordinate into the conversion model according to the conversion model from the pre-trained camera coordinate system to the world coordinate system, and obtaining a second conversion matrix between the camera coordinate system and the vehicle body coordinate system by combining the vehicle position information, wherein the second conversion matrix is calibrated external parameters of the camera.
2. The vehicle-mounted camera external reference calibration method according to claim 1, wherein the pre-trained conversion model from the camera coordinate system to the world coordinate system comprises:
establishing a first transformation matrix from the vehicle body coordinate system to the world coordinate system and a second transformation matrix from the camera coordinate system to the vehicle body coordinate system;
and establishing a third transformation matrix from the camera coordinate system to the world coordinate system according to the first transformation matrix and the second transformation matrix.
3. The vehicle-mounted camera external reference calibration method according to claim 1, wherein the step of obtaining the camera coordinates of the feature point under the camera coordinates comprises:
and acquiring pixel coordinates of the feature points in an image coordinate system and camera internal parameters, and calculating the camera coordinates of the feature points in the camera coordinate system through the camera internal parameters by combining the pixel coordinates.
4. The vehicle-mounted camera external parameter calibration method according to claim 2, further comprising:
and acquiring the world coordinate of the vehicle in the world coordinate system according to the vehicle position information, and acquiring a first transformation matrix from the world coordinate system to the vehicle body coordinate system according to the world coordinate of the vehicle.
5. The vehicle-mounted camera external reference calibration method according to claim 4, wherein the step of inputting the world coordinates and the camera coordinates into the conversion model according to the pre-trained conversion model from the camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information comprises:
inputting the world coordinates and the camera coordinates into a conversion model of the camera coordinate system into the world coordinate system;
and calculating to obtain the third transformation matrix according to the world coordinate and the camera coordinate, and calculating to obtain the second transformation matrix according to the third transformation matrix and the first transformation matrix obtained based on the vehicle position information.
6. The vehicle-mounted camera external reference calibration method according to claim 1, wherein the step of inputting the world coordinates and the camera coordinates into the conversion model according to the pre-trained conversion model from the camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information further comprises:
and acquiring a plurality of characteristic points near the vehicle position information, and fitting a plurality of second transformation matrixes based on a plurality of second transformation matrixes obtained by calculation of the plurality of characteristic points to obtain an optimal second transformation matrix.
7. The external parameter calibration device for the vehicle-mounted camera is characterized by comprising the following components:
the characteristic point acquisition module is used for acquiring vehicle position information and determining characteristic points near the vehicle position information in a high-precision map;
the world coordinate acquisition module is used for acquiring world coordinates of the feature points in a world coordinate system based on the high-precision map;
the camera coordinate acquisition module is used for acquiring camera coordinates of the feature points under the camera coordinates;
and the camera external parameter calculation module is used for inputting the world coordinates and the camera coordinates into the conversion model according to the conversion model from the pre-trained camera coordinate system to the world coordinate system, and obtaining a second transformation matrix from the camera coordinate system to the vehicle body coordinate system by combining the vehicle position information, wherein the second transformation matrix is calibrated camera external parameters.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method for external parameter calibration of a vehicle camera according to any one of claims 1-6.
9. A data processing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the vehicle camera external parameter calibration method according to any one of claims 1 to 6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111469484.0A CN114387350A (en) | 2021-12-03 | 2021-12-03 | Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111469484.0A CN114387350A (en) | 2021-12-03 | 2021-12-03 | Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114387350A true CN114387350A (en) | 2022-04-22 |
Family
ID=81195551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111469484.0A Pending CN114387350A (en) | 2021-12-03 | 2021-12-03 | Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114387350A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663458A (en) * | 2022-05-24 | 2022-06-24 | 魔门塔(苏州)科技有限公司 | Dynamic calibration method, dynamic calibration device, driving state detection method, driving state detection medium and driving state detection equipment |
| CN115690194A (en) * | 2022-10-17 | 2023-02-03 | 广州赤兔宸行科技有限公司 | Vehicle-mounted XR equipment positioning method, device, equipment and storage medium |
| CN117274956A (en) * | 2023-11-17 | 2023-12-22 | 深圳市航盛电子股份有限公司 | Vehicle side view generation method, device, terminal equipment and storage medium |
| CN117274402A (en) * | 2023-11-24 | 2023-12-22 | 魔视智能科技(武汉)有限公司 | Calibration method and device for camera external parameters, computer equipment and storage medium |
| CN117274956B (en) * | 2023-11-17 | 2024-05-24 | 深圳市航盛电子股份有限公司 | Vehicle side view generation method, device, terminal equipment and storage medium |
-
2021
- 2021-12-03 CN CN202111469484.0A patent/CN114387350A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663458A (en) * | 2022-05-24 | 2022-06-24 | 魔门塔(苏州)科技有限公司 | Dynamic calibration method, dynamic calibration device, driving state detection method, driving state detection medium and driving state detection equipment |
| CN114663458B (en) * | 2022-05-24 | 2022-10-11 | 魔门塔(苏州)科技有限公司 | Dynamic calibration method, dynamic calibration device, driving state detection method, driving state detection medium and driving state detection equipment |
| CN115690194A (en) * | 2022-10-17 | 2023-02-03 | 广州赤兔宸行科技有限公司 | Vehicle-mounted XR equipment positioning method, device, equipment and storage medium |
| CN115690194B (en) * | 2022-10-17 | 2023-09-19 | 广州赤兔宸行科技有限公司 | Vehicle-mounted XR equipment positioning method, device, equipment and storage medium |
| CN117274956A (en) * | 2023-11-17 | 2023-12-22 | 深圳市航盛电子股份有限公司 | Vehicle side view generation method, device, terminal equipment and storage medium |
| CN117274956B (en) * | 2023-11-17 | 2024-05-24 | 深圳市航盛电子股份有限公司 | Vehicle side view generation method, device, terminal equipment and storage medium |
| CN117274402A (en) * | 2023-11-24 | 2023-12-22 | 魔视智能科技(武汉)有限公司 | Calibration method and device for camera external parameters, computer equipment and storage medium |
| CN117274402B (en) * | 2023-11-24 | 2024-04-19 | 魔视智能科技(武汉)有限公司 | Calibration method and device for camera external parameters, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110264520B (en) | Vehicle-mounted sensor and vehicle pose relation calibration method, device, equipment and medium | |
| CN114387350A (en) | Vehicle-mounted camera external parameter calibration method and device, storage medium and equipment | |
| US20230215187A1 (en) | Target detection method based on monocular image | |
| CN110146910B (en) | Positioning method and device based on data fusion of GPS and laser radar | |
| CN111507130B (en) | Lane-level positioning method and system, computer equipment, vehicle and storage medium | |
| CN109849930B (en) | Method and device for calculating speed of adjacent vehicle of automatic driving automobile | |
| CN110927708A (en) | Calibration method, device and equipment of intelligent road side unit | |
| CN110969055B (en) | Method, apparatus, device and computer readable storage medium for vehicle positioning | |
| CN110388929B (en) | Navigation map updating method, device and system | |
| CN115164918B (en) | Semantic point cloud map construction method and device and electronic equipment | |
| CN113850867A (en) | Camera parameter calibration method, camera parameter calibration device control method, camera parameter calibration device control device, and storage medium | |
| CN111930877B (en) | Map guideboard generation method and electronic equipment | |
| CN112198878B (en) | Instant map construction method and device, robot and storage medium | |
| CN109115232B (en) | Navigation method and device | |
| CN111982132B (en) | Data processing method, device and storage medium | |
| CN116386000A (en) | Method and system for measuring obstacle distance based on high-precision map and monocular camera | |
| CN114119682A (en) | Laser point cloud and image registration method and registration system | |
| CN112781893B (en) | Spatial synchronization method and device for vehicle-mounted sensor performance test data and storage medium | |
| CN113658262B (en) | Camera external parameter calibration method, device, system and storage medium | |
| CN112304305A (en) | Vehicle initial positioning method and system combined with marker image | |
| CN115618602A (en) | Lane-level scene simulation method and system | |
| CN116215520A (en) | Vehicle collision early warning and processing method and device based on ultrasonic waves and 3D looking around | |
| CN114755663A (en) | External reference calibration method and device for vehicle sensor and computer readable storage medium | |
| CN207931598U (en) | A kind of lane recognition system of electric vehicle | |
| CN111383268A (en) | Vehicle distance state acquisition method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |