CN116630436A - Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium - Google Patents
Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN116630436A CN116630436A CN202310558550.4A CN202310558550A CN116630436A CN 116630436 A CN116630436 A CN 116630436A CN 202310558550 A CN202310558550 A CN 202310558550A CN 116630436 A CN116630436 A CN 116630436A
- Authority
- CN
- China
- Prior art keywords
- lane line
- matrix
- sequence
- camera
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012937 correction Methods 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 45
- 239000011159 matrix material Substances 0.000 claims abstract description 117
- 238000001514 detection method Methods 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 21
- 239000013598 vector Substances 0.000 claims description 87
- 238000004590 computer program Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 25
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000006243 chemical reaction Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005457 optimization Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Algebra (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Embodiments of the present disclosure disclose camera exogenous correction methods, apparatuses, electronic devices, and computer-readable media. One embodiment of the method comprises the following steps: carrying out lane line detection on the pre-acquired road image to generate a lane line coordinate matrix; generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix; and carrying out correction processing on a preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix. This embodiment may reduce the consumption of computing resources.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a camera external parameter correction method, apparatus, electronic device, and computer readable medium.
Background
Computer vision algorithms commonly used in autopilot, such as visual localization, environmental awareness, map reconstruction, etc., rely on accurate off-camera parameters. The shaking generated during the running process of the vehicle can continuously change the parameters outside the camera, thereby causing serious faults of an automatic driving system. In the process of camera external parameter correction, the following methods are generally adopted: the coordinates of the lane lines and the fixed markers (such as a guideboard, a lamp post and the like) are identified from the road image, and the camera external parameters are adjusted by combining the prior information of the lane lines.
However, the inventors found that when the camera external parameter correction is performed in the above manner, there are often the following technical problems:
firstly, information extraction is carried out on a fixed mark in an image while a lane line is identified, so that the camera external parameters are corrected by utilizing the lane line, the fixed mark and the priori information of the lane line, and more calculation resources are required to be consumed;
secondly, if the camera external parameters are corrected based on the virtual lane line or the stop line, the method is difficult to be applied to expressway scenes, and the adaptability of the camera external parameters is reduced, so that the camera external parameters are difficult to be corrected in time.
The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose camera exogenous correction methods, apparatus, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a camera exogenous correction method, the method comprising: carrying out lane line detection on the pre-acquired road image to generate a lane line coordinate matrix; generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix; and carrying out correction processing on a preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix.
In a second aspect, some embodiments of the present disclosure provide a camera exogenous correction device, the device comprising: the lane line detection unit is configured to perform lane line detection on the pre-acquired road image so as to generate a lane line coordinate matrix; the generation unit is configured to generate a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix; the correction processing unit is configured to perform correction processing on a preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the implementations of the first aspect above.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
The above embodiments of the present disclosure have the following advantageous effects: by the camera external parameter correction method of some embodiments of the present disclosure, the consumption of computing resources can be reduced. Specifically, the reason for consuming more computing resources is that: and extracting information of the fixed marks in the image while recognizing the lane lines, so that the camera external parameters are corrected by using the lane lines, the fixed marks and the prior information of the lane lines. Based on this, in some embodiments of the present disclosure, first, in response to determining that a road on which a current vehicle is located satisfies a preset road condition, lane line detection is performed on a pre-acquired road image to generate a lane line coordinate matrix. By introducing preset road conditions, a road suitable for camera external parameter correction can be selected. In addition, by generating the lane line coordinate matrix, camera external parameter correction can be conveniently carried out by using lane line coordinates later. And then, generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix. By generating the sequence of lane line equations, camera outlier matrix correction can be made using only lane line equations. And finally, based on the lane line equation sequence, correcting the preset camera external parameter matrix to obtain a corrected camera external parameter matrix. Therefore, the camera external parameter correction can be realized only through the identified lane line without extracting a fixed mark from the road image and prior information of the lane line. Further, the consumption of computing resources is reduced.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a flow chart of some embodiments of a camera exogenous correction method according to the present disclosure;
FIG. 2 is a schematic structural view of some embodiments of a camera exogenous correction device according to the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates a flow 100 of some embodiments of a camera exogenous correction method according to the present disclosure. The camera external parameter correction method comprises the following steps:
and step 101, in response to determining that the road where the current vehicle is located meets the preset road condition, carrying out lane line detection on the pre-acquired road image so as to generate a lane line coordinate matrix.
In some embodiments, the execution subject of the camera exogenous correction method may perform lane line detection on the pre-acquired road image to generate the lane line coordinate matrix in response to determining that the road on which the current vehicle is located satisfies the preset road condition. The preset road condition may be that the road where the current vehicle is located is a straight road, that is, a lane line on the road is approximately a straight line. So as to be convenient for correcting the camera external parameters by utilizing the lane lines. The pre-acquired road image may be a road image photographed by an on-vehicle camera.
In some optional implementations of some embodiments, the performing body performs lane line detection on the pre-acquired road image to generate a lane line coordinate matrix, and may include the following steps:
first, lane line detection is performed on a pre-acquired road image to generate a lane line detection information sequence. The lane line detection can be carried out on the road image through a preset lane line detection algorithm. Each lane line detection information in the lane line detection information sequence may include a lane line detection coordinate sequence. Each lane line detection coordinate sequence may be used to characterize one lane line. Each lane line detection coordinate in the lane line detection coordinate sequence may be a continuous coordinate point.
As an example, the lane line detection algorithm may include, but is not limited to, at least one of: a Residual Network model, a VGG (Visual Geometry Group Network, convolutional neural Network) model, a google net (deep neural Network) model, and the like.
And a second step of determining a lane line detection coordinate sequence included in each lane line detection information in the lane line detection information sequence as a lane line coordinate vector to obtain a lane line coordinate vector sequence. The lane line coordinate vector may include continuous lane line detection coordinates in the lane line detection coordinate sequence.
And thirdly, combining each lane line coordinate vector in the lane line coordinate vector sequence into a lane line coordinate matrix. The first column in the lane line coordinate matrix may correspond to a lane line coordinate vector of a leftmost lane line detected in the road image. The last column in the lane line coordinate matrix corresponds to the lane line coordinate vector of the rightmost lane line detected in the road image. Here, each vector in the lane line coordinate matrix may be used to characterize each lane line. And simultaneously, the left to right can respectively correspond to the sequence of the lane lines on the lane from left to right. Thereby facilitating the correction of camera external parameters.
And 102, generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix.
In some embodiments, the execution subject may generate the lane-line equation sequence in various manners based on the lane-line coordinate matrix and a preset camera internal reference matrix.
In some optional implementations of some embodiments, the generating, by the executing body, a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix may include the following steps:
and firstly, carrying out back projection on each lane line coordinate in the lane line coordinate matrix to generate a back projection coordinate matrix. The coordinates of each lane line can be projected onto a normalized plane from an image coordinate system through a back projection algorithm, so as to obtain back projection coordinates. The normalization plane may be a plane within a preset camera coordinate system. And secondly, combining each back projection coordinate into a back projection coordinate matrix according to the position of the corresponding lane line coordinate. Thus, each backprojection coordinate in each column vector in the backprojection coordinate matrix may also correspond to one lane line.
And secondly, fitting each column of back projection coordinates in the back projection coordinate matrix to generate a lane line equation, and obtaining a lane line equation sequence. Wherein, each back projection coordinate in each column vector can be fit into a lane line equation. Here, the lane line equation may be a standard type of straight line equation. Thus, an abscissa coefficient, an ordinate coefficient, and a constant term may be included in each lane line equation. In addition, each lane-line equation may be used to characterize one lane-line.
And 103, based on the lane line equation sequence, carrying out correction processing on the preset camera external parameter matrix to obtain a corrected camera external parameter matrix.
In some embodiments, the executing body may perform correction processing on the preset camera external parameter matrix in various ways based on the lane line equation sequence to obtain a corrected camera external parameter matrix.
In some optional implementations of some embodiments, the executing body corrects the preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix, and may include the following steps:
firstly, constructing a vanishing point error function based on the camera external parameter matrix, the lane line equation sequence and a preset transverse axis base vector of a ground coordinate system.
Optionally, the executing body constructs the vanishing point error function based on the camera external parameter matrix, the lane line equation sequence and a preset ground coordinate system transverse axis base vector, and may include the following steps:
step one, converting the transverse axis base vector of the ground coordinate system into a camera coordinate system by using the camera external parameter matrix so as to generate a converted transverse axis base vector.
As an example, the ground coordinate system transverse axis basis vector may be (1, 0).
And secondly, constructing a vanishing vector by using coefficients of each lane line equation in the lane line equation sequence to obtain a vanishing vector sequence. The vanishing vector can be constructed by taking the abscissa coefficient, the ordinate coefficient and the constant term included in each lane line equation as data in the vanishing vector.
And thirdly, constructing a vanishing point error function based on the converted horizontal axis base vector and the vanishing vector sequence. First, the product of the converted horizontal axis base vector and each vanishing vector can be determined as a vanishing point error value, and a vanishing point error value sequence is obtained. Next, the sum of squares of absolute values of the respective vanishing point error values in the vanishing point error value sequence may be determined as the vanishing point error function value.
As an example, the vanishing point error function may be expressed by the following formula:
wherein f p Representing the vanishing point error function value. N represents the number of vanishing vectors in the vanishing vector sequence. i represents a sequence number. E represents the vanishing point error value. e, e i Representing the i-th error value in the vanishing point error sequence. q x Representing the transformed lateral basis vector. v denotes the vanishing vector in the vanishing vector sequence. v i Representing the ith vanishing vector in the vanishing vector sequence.
And secondly, constructing an vanishing line error function based on the camera external parameter matrix, the lane line equation sequence and a preset vertical axis base vector of a ground coordinate system.
Optionally, the executing body constructs an vanishing line error function based on the camera external parameter matrix, the lane line equation sequence and a preset vertical axis base vector of the ground coordinate system, and may include the following steps:
step one, converting the vertical axis base vector of the ground coordinate system into a camera coordinate system by using the camera external parameter matrix so as to generate a converted vertical axis base vector.
As an example, the ground coordinate system vertical axis basis vector may be (0, 1).
And step two, screening the vanishing vector sequence to obtain a target vanishing vector sequence. Wherein the screening may be to select a target number (e.g., 3) vanishing vectors from the vanishing vector sequence. In practice, the target number may be set according to the actual algorithm.
And thirdly, generating a vanishing line equation by utilizing each target vanishing vector in the target vanishing vector sequence. Wherein the vanishing line vector may be generated by the following formula: l= ((v) 1 ×v 3 ) T ×(v 2 ×v 3 ))×v 2 +2×((v 1 ×v 2 ) T ×(v 3 ×v 2 ))×v 3 。
Wherein L represents the vanishing line vector described above. v 1 And representing the 1 st target vanishing vector in the target vanishing vector sequence. v 2 And representing the 2 nd target vanishing vector in the target vanishing vector sequence. v 3 And representing the 3 rd target vanishing vector in the target vanishing vector sequence. T represents the transpose matrix.
And step four, constructing a vanishing line error function based on the converted vertical axis base vector and the vanishing line vector. Wherein, first, the product of the converted vertical axis base vector and the vanishing line equation may be determined as a vanishing line error value. Second, the square of the absolute value of the vanishing line error value may be determined as the vanishing line error function value.
And thirdly, adding the vanishing point error function and the vanishing line error function to a preset factor graph to obtain an added factor graph. The added factor graph may include various optimized data with the camera external parameter matrix as an initial value. Second, the factor graph may be a preset optimization model for adjusting the camera extrinsic matrix. The various optimization data may be data required to optimize the camera extrinsic matrix.
And step four, optimizing the camera external parameter matrix in the added factor graph as an initial value to generate a corrected camera external parameter matrix. And performing optimization processing on the camera external parameter matrix in the added factor graph as an initial value through a preset optimization solving algorithm to generate a corrected camera external parameter matrix.
As an example, the optimization solving algorithm may be a gradient descent algorithm or the like.
The above formulas and the related contents serve as an invention point of the embodiments of the present disclosure, and solve the second technical problem mentioned in the background art, that is, if the camera external parameter correction is performed based on the virtual lane line or the stop line, the method is difficult to be applied to the expressway scene, and the adaptability of the camera external parameter correction is reduced, so that the camera external parameter is difficult to be corrected in time. Factors that cause difficulty in timely correcting camera parameters are often as follows: if the camera external parameter correction is carried out based on the virtual lane line or the stop line, the method is difficult to be applied to expressway scenes, and the adaptability of the camera external parameter correction is reduced. If the factors are solved, the adaptability of the camera external parameter correction method can be improved, so that the camera external parameter can be corrected in time. To achieve this effect, first, since the camera external parameter correction can be performed by satisfying the above-described preset road condition, there is no need to limit to a scene based on a virtual lane line or a stop line, and the adaptability of the camera external parameter correction is improved. Thus, it is also applicable to expressway scenes. Specifically, under the condition that the horizontal axis base vector of the ground coordinate system is unchanged, by generating the converted horizontal axis base vector, errors existing in the camera coordinate system external parameter matrix can be guided into the converted horizontal axis base vector. Secondly, a vanishing point error function is constructed by using the vanishing vector sequence and the transformed horizontal axis basis vector. The error generated in the aspect of the coordinates of the camera coordinate system external parameter matrix can be quantized through the vanishing point error function. Similarly, in the case where the ground coordinate system vertical axis base vector is unchanged, by generating the post-conversion vertical axis base vector, an error existing in the camera coordinate system external parameter matrix can be guided in the post-conversion vertical axis base vector. Then, a vanishing line error function is constructed by using the target vanishing vector sequence and the converted vertical axis basis vector. Errors in vanishing lines produced by camera coordinate system outlier matrices can be quantified by vanishing line error functions. Finally, by adding the vanishing point error function and the vanishing line error function to the factor graph, optimization processing can be performed in the factor graph. So as to greatly eliminate errors and correct the camera external parameter matrix. In practice, vanishing lines are directly calculated through three equidistant lane lines, so that constraint items are constructed, compared with a method for calculating a plurality of vanishing points by combining lane lines in pairs, the method has the advantages of small scene dependence, obviously reduced calculated amount and higher stability of calculated results. Furthermore, the camera external parameters can be corrected in time, so that the accuracy of the corrected camera external parameter matrix is improved.
Optionally, the executing body may further execute the following steps:
the first step is to use the corrected camera external parameter matrix to convert each lane line detection information in the lane line detection information sequence to obtain a converted lane line detection information sequence, and to store the corrected camera external parameter matrix. The conversion process may be to convert each lane line detection coordinate in the lane line detection coordinate sequence included in the lane line detection information from the image coordinate system to the camera coordinate system, so as to obtain a converted lane line coordinate sequence. Next, each post-conversion lane line coordinate in the post-conversion lane line coordinate sequence may be fitted to a post-conversion lane line equation as post-conversion lane line detection information. The accuracy of the lane line equation after conversion can be improved through the corrected camera external parameter matrix.
And step two, the converted lane line information sequence is sent to a target display terminal for display. The converted lane line information sequence may be sent to the target terminal for displaying the three-dimensional lane line in the camera coordinate system. Here, the target display terminal may be a display terminal of the current vehicle.
The above embodiments of the present disclosure have the following advantageous effects: by the camera external parameter correction method of some embodiments of the present disclosure, the consumption of computing resources can be reduced. Specifically, the reason for consuming more computing resources is that: and extracting information of the fixed marks in the image while recognizing the lane lines, so that the camera external parameters are corrected by using the lane lines, the fixed marks and the prior information of the lane lines. Based on this, in some embodiments of the present disclosure, first, in response to determining that a road on which a current vehicle is located satisfies a preset road condition, lane line detection is performed on a pre-acquired road image to generate a lane line coordinate matrix. By introducing preset road conditions, a road suitable for camera external parameter correction can be selected. In addition, by generating the lane line coordinate matrix, camera external parameter correction can be conveniently carried out by using lane line coordinates later. And then, generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix. By generating the sequence of lane line equations, camera outlier matrix correction can be made using only lane line equations. And finally, based on the lane line equation sequence, correcting the preset camera external parameter matrix to obtain a corrected camera external parameter matrix. Therefore, the camera external parameter correction can be realized only through the identified lane line without extracting a fixed mark from the road image and prior information of the lane line. Further, the consumption of computing resources is reduced.
With further reference to fig. 2, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a camera exogenous correction device, which correspond to those method embodiments shown in fig. 1, and which are particularly applicable in various electronic apparatuses.
As shown in fig. 2, the camera exogenous correction device 200 of some embodiments includes: a lane line detection unit 201, a generation unit 202, and a correction processing unit 203. Wherein, the lane line detection unit 201 is configured to perform lane line detection on the pre-acquired road image to generate a lane line coordinate matrix; a generating unit 202 configured to generate a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix; the correction processing unit 203 is configured to perform correction processing on the preset camera external parameter matrix based on the lane line equation sequence, so as to obtain a corrected camera external parameter matrix.
It will be appreciated that the elements described in the apparatus 200 correspond to the various steps in the method described with reference to fig. 1. Thus, the operations, features and resulting benefits described above for the method are equally applicable to the apparatus 200 and the units contained therein, and are not described in detail herein.
Referring now to fig. 3, a schematic diagram of an electronic device 300 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 3 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 3, the electronic device 300 may include a processing means 301 (e.g., a central processing unit, a graphics processor, etc.) that may perform various suitable actions and processes in accordance with a program stored in a Read Only Memory (ROM) 302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
In general, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 308 including, for example, magnetic tape, hard disk, etc.; and communication means 309. The communication means 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. While fig. 3 shows an electronic device 300 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 3 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 309, or from storage device 308, or from ROM 302. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing means 301.
It should be noted that, in some embodiments of the present disclosure, the computer readable medium may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be embodied in the apparatus; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to determining that the road where the current vehicle is located meets a preset road condition, carrying out lane line detection on the pre-acquired road image so as to generate a lane line coordinate matrix; generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix; and carrying out correction processing on a preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a lane line detection unit, a generation unit, and a correction processing unit. The names of these units do not constitute limitations on the unit itself in some cases, and for example, the correction processing unit may also be described as "a unit that performs correction processing on a preset camera external matrix".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (10)
1. A camera external parameter correction method comprises the following steps:
in response to determining that the road where the current vehicle is located meets a preset road condition, carrying out lane line detection on the pre-acquired road image so as to generate a lane line coordinate matrix;
generating a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix;
and based on the lane line equation sequence, correcting the preset camera external parameter matrix to obtain a corrected camera external parameter matrix.
2. The method of claim 1, wherein the lane line detection of the pre-acquired road image to generate a lane line coordinate matrix comprises:
carrying out lane line detection on the pre-acquired road image to generate a lane line detection information sequence;
determining a lane line detection coordinate sequence included in each lane line detection information in the lane line detection information sequence as a lane line coordinate vector to obtain a lane line coordinate vector sequence;
and combining all lane line coordinate vectors in the lane line coordinate vector sequence into a lane line coordinate matrix, wherein a first column in the lane line coordinate matrix corresponds to the lane line coordinate vector of the leftmost lane line detected in the road image, and a last column corresponds to the lane line coordinate vector of the rightmost lane line detected in the road image.
3. The method of claim 2, wherein the method further comprises:
converting each lane line detection information in the lane line detection information sequence by using the corrected camera external parameter matrix to obtain a converted lane line detection information sequence, and storing the corrected camera external parameter matrix;
and sending the converted lane line information sequence to a target display terminal for display.
4. The method of claim 1, wherein the generating a sequence of lane-line equations based on the lane-line coordinate matrix and a preset camera internal reference matrix comprises:
back-projecting each lane line coordinate in the lane line coordinate matrix to generate a back-projected coordinate matrix;
and fitting each column of back projection coordinates in the back projection coordinate matrix to generate a lane line equation, so as to obtain a lane line equation sequence.
5. The method of claim 1, wherein the correcting the preset camera extrinsic matrix based on the lane line equation sequence to obtain a corrected camera extrinsic matrix comprises:
constructing a vanishing point error function based on the camera external parameter matrix, the lane line equation sequence and a preset ground coordinate system transverse axis base vector;
constructing an vanishing line error function based on the camera external parameter matrix, the lane line equation sequence and a preset vertical axis base vector of a ground coordinate system;
adding the vanishing point error function and the vanishing line error function to a preset factor graph to obtain an added factor graph, wherein the added factor graph comprises various optimized data taking the camera external parameter matrix as an initial value;
and optimizing the camera external parameter matrix in the added factor graph as an initial value to generate a corrected camera external parameter matrix.
6. The method of claim 5, wherein the constructing a vanishing point error function based on the camera outlier matrix, the sequence of lane-line equations, and a pre-set ground coordinate system cross-axis basis vector comprises:
converting the horizontal axis base vector of the ground coordinate system to a camera coordinate system by using the camera external reference matrix to generate a converted horizontal axis base vector;
constructing a vanishing vector by using coefficients of each lane line equation in the lane line equation sequence to obtain a vanishing vector sequence;
and constructing a vanishing point error function based on the converted horizontal axis base vector and the vanishing vector sequence.
7. The method of claim 6, wherein the constructing a vanishing line error function based on the camera outlier matrix, the sequence of lane line equations, and a pre-set ground coordinate system vertical axis basis vector comprises:
converting the vertical axis base vector of the ground coordinate system to a camera coordinate system by using the camera external reference matrix to generate a converted vertical axis base vector;
screening the vanishing vector sequence to obtain a target vanishing vector sequence;
generating vanishing line vectors by utilizing all target vanishing vectors in the target vanishing vector sequence;
and constructing a vanishing line error function based on the converted vertical axis base vector and the vanishing line vector.
8. A camera exogenous correction device, comprising:
the lane line detection unit is configured to perform lane line detection on the pre-acquired road image so as to generate a lane line coordinate matrix;
the generation unit is configured to generate a lane line equation sequence based on the lane line coordinate matrix and a preset camera internal reference matrix;
and the correction processing unit is configured to perform correction processing on a preset camera external parameter matrix based on the lane line equation sequence to obtain a corrected camera external parameter matrix.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
10. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310558550.4A CN116630436B (en) | 2023-05-17 | 2023-05-17 | Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310558550.4A CN116630436B (en) | 2023-05-17 | 2023-05-17 | Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116630436A true CN116630436A (en) | 2023-08-22 |
| CN116630436B CN116630436B (en) | 2024-01-12 |
Family
ID=87609210
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310558550.4A Active CN116630436B (en) | 2023-05-17 | 2023-05-17 | Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116630436B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
| CN111412856A (en) * | 2019-12-31 | 2020-07-14 | 吉林大学 | Automobile morphology monocular active vision reconstruction system and method based on double-rectangle identification |
| CN112614192A (en) * | 2020-12-24 | 2021-04-06 | 湖北亿咖通科技有限公司 | On-line calibration method of vehicle-mounted camera and vehicle-mounted information entertainment system |
| CN112785655A (en) * | 2021-01-28 | 2021-05-11 | 中汽创智科技有限公司 | Method, device and equipment for automatically calibrating external parameters of all-round camera based on lane line detection and computer storage medium |
| CN113674357A (en) * | 2021-08-04 | 2021-11-19 | 禾多科技（北京）有限公司 | Camera external parameter calibration method and device, electronic equipment and computer readable medium |
| CN113706624A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Camera external parameter correction method and device and vehicle-mounted all-round-looking system |
| CN114022561A (en) * | 2021-10-18 | 2022-02-08 | 武汉中海庭数据技术有限公司 | Urban area monocular mapping method and system based on GPS constraint and dynamic correction |
| CN114663529A (en) * | 2022-03-22 | 2022-06-24 | 阿波罗智能技术(北京)有限公司 | External parameter determining method and device, electronic equipment and storage medium |
| CN114742914A (en) * | 2022-04-15 | 2022-07-12 | 苏州挚途科技有限公司 | Lane line generation method and device and electronic equipment |
| CN114842436A (en) * | 2022-04-24 | 2022-08-02 | 禾多科技（北京）有限公司 | Three-dimensional lane line generation method and device, electronic device and computer readable medium |
| CN115578706A (en) * | 2022-10-25 | 2023-01-06 | 毫末智行科技有限公司 | Lane line correction method and device, vehicle control unit and readable storage medium |
| CN115661299A (en) * | 2022-12-27 | 2023-01-31 | 安徽蔚来智驾科技有限公司 | Method for constructing lane line map, computer device and storage medium |
-
2023
- 2023-05-17 CN CN202310558550.4A patent/CN116630436B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
| CN111412856A (en) * | 2019-12-31 | 2020-07-14 | 吉林大学 | Automobile morphology monocular active vision reconstruction system and method based on double-rectangle identification |
| CN113706624A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Camera external parameter correction method and device and vehicle-mounted all-round-looking system |
| CN112614192A (en) * | 2020-12-24 | 2021-04-06 | 湖北亿咖通科技有限公司 | On-line calibration method of vehicle-mounted camera and vehicle-mounted information entertainment system |
| CN112785655A (en) * | 2021-01-28 | 2021-05-11 | 中汽创智科技有限公司 | Method, device and equipment for automatically calibrating external parameters of all-round camera based on lane line detection and computer storage medium |
| CN113674357A (en) * | 2021-08-04 | 2021-11-19 | 禾多科技（北京）有限公司 | Camera external parameter calibration method and device, electronic equipment and computer readable medium |
| CN114022561A (en) * | 2021-10-18 | 2022-02-08 | 武汉中海庭数据技术有限公司 | Urban area monocular mapping method and system based on GPS constraint and dynamic correction |
| CN114663529A (en) * | 2022-03-22 | 2022-06-24 | 阿波罗智能技术(北京)有限公司 | External parameter determining method and device, electronic equipment and storage medium |
| CN114742914A (en) * | 2022-04-15 | 2022-07-12 | 苏州挚途科技有限公司 | Lane line generation method and device and electronic equipment |
| CN114842436A (en) * | 2022-04-24 | 2022-08-02 | 禾多科技（北京）有限公司 | Three-dimensional lane line generation method and device, electronic device and computer readable medium |
| CN115578706A (en) * | 2022-10-25 | 2023-01-06 | 毫末智行科技有限公司 | Lane line correction method and device, vehicle control unit and readable storage medium |
| CN115661299A (en) * | 2022-12-27 | 2023-01-31 | 安徽蔚来智驾科技有限公司 | Method for constructing lane line map, computer device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 谭博怡等: ""一种车轮轮廓的自动提取方法"", 《自动化学报》, pages 516 - 522 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116630436B (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112598762B (en) | Three-dimensional lane line information generation method, device, electronic device, and medium | |
| CN112348029B (en) | Local map adjusting method, device, equipment and computer readable medium | |
| CN113674357B (en) | Camera external reference calibration method and device, electronic equipment and computer readable medium | |
| CN113607185B (en) | Lane line information display method, lane line information display device, electronic device, and computer-readable medium | |
| CN114993328B (en) | Vehicle positioning evaluation method, device, equipment and computer readable medium | |
| CN114399588B (en) | Three-dimensional lane line generation method and device, electronic device and computer readable medium | |
| CN113327318B (en) | Image display method, image display device, electronic equipment and computer readable medium | |
| CN116182878B (en) | Road curved surface information generation method, device, equipment and computer readable medium | |
| CN112598731B (en) | Vehicle positioning method and device, electronic equipment and computer readable medium | |
| CN114445597B (en) | Three-dimensional lane line generation method and device, electronic device and computer readable medium | |
| CN116740382B (en) | Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium | |
| CN115620264B (en) | Vehicle positioning method and device, electronic equipment and computer readable medium | |
| CN115937046B (en) | Road ground information generation method, device, equipment and computer readable medium | |
| CN114723640B (en) | Obstacle information generation method and device, electronic equipment and computer readable medium | |
| CN116630436B (en) | Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium | |
| CN116311155A (en) | Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium | |
| CN115393826A (en) | Three-dimensional lane line generation method and device, electronic device and computer readable medium | |
| CN115170674B (en) | Camera principal point calibration method, device, equipment and medium based on single image | |
| CN114399555B (en) | Data online calibration method and device, electronic equipment and computer readable medium | |
| CN114494428B (en) | Vehicle pose correction method and device, electronic equipment and computer readable medium | |
| CN115307652B (en) | Vehicle pose determination method, device, equipment and medium | |
| CN117373024B (en) | Method, device, electronic equipment and computer readable medium for generating annotation image | |
| CN114863025B (en) | Three-dimensional lane line generation method and device, electronic device and computer readable medium | |
| CN113542800B (en) | Video picture scaling method, device and terminal equipment | |
| CN114708336A (en) | Multi-camera online calibration method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |