CN112446924B - Camera calibration system of vehicle, vehicle and camera calibration method - Google Patents
Camera calibration system of vehicle, vehicle and camera calibration method Download PDFInfo
- Publication number
- CN112446924B CN112446924B CN201910822881.8A CN201910822881A CN112446924B CN 112446924 B CN112446924 B CN 112446924B CN 201910822881 A CN201910822881 A CN 201910822881A CN 112446924 B CN112446924 B CN 112446924B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- filter
- output result
- information
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims description 63
- 238000005259 measurement Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 14
- 238000007499 fusion processing Methods 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000007613 environmental effect Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000003745 diagnosis Methods 0.000 abstract description 6
- 238000011084 recovery Methods 0.000 abstract description 6
- 230000000694 effects Effects 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 238000013480 data collection Methods 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Abstract
The embodiment of the disclosure provides a camera calibration system of a vehicle, the vehicle and a camera calibration method. The system comprises: the device comprises an acquisition device, a federal filter device and a calibration device, wherein the federal filter device comprises a main filter and at least two local filters, the acquisition device is connected with one end of each local filter, the other end of each local filter is connected with one end of the main filter, and the other end of the main filter is connected with the calibration device; the acquisition device is used for acquiring vehicle operation condition data and inputting corresponding vehicle operation condition data to each local filter respectively; each local filter is used for inputting the output result of the local filter to the main filter; the main filter is used for inputting the output result of the main filter to the calibration device; the calibration device is used for calibrating the camera of the vehicle according to the output result of the main filter. The embodiment of the disclosure can get rid of the limitations of the existing vehicle camera calibration mode, and has certain fault diagnosis and recovery capability and fault tolerance capability.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of vehicle engineering, in particular to a camera calibration system of a vehicle, the vehicle and a camera calibration method.
Background
With the rapid development of the technical field of vehicle engineering, vehicles are becoming more and more popular, and vehicles become important vehicles in daily life.
At present, cameras are arranged on many vehicles, and precisely calibrated camera parameters play an important role in improving the precision and accuracy of the system. Generally, in the prior art, a static calibration mode is adopted to calibrate a camera, and when the calibration mode is adopted, a vehicle must be stopped under a specific environment of a designated place to calibrate, so that the existing calibration mode has strong limitation.
Disclosure of Invention
The embodiment of the disclosure provides a camera calibration system of a vehicle, the vehicle and a camera calibration method, so as to solve the problem of strong limitation of a calibration mode of a vehicle camera in the prior art.
In a first aspect, embodiments of the present disclosure provide a camera calibration system for a vehicle, comprising: collection system, federal filter device and calibration device, federal filter device includes: the acquisition device is connected with one end of each local filter, the other end of each local filter is connected with one end of the main filter, and the other end of the main filter is connected with the calibration device; wherein,
The acquisition device is used for acquiring vehicle operation condition data and inputting corresponding vehicle operation condition data to each local filter respectively;
Each local filter is used for inputting an output result of the local filter to the main filter;
the main filter is used for inputting the output result of the main filter to the calibration device;
The calibration device is used for calibrating the camera of the vehicle according to the output result of the main filter.
In some embodiments, the federal filter arrangement includes: a first local filter, a second local filter, and a third local filter; wherein,
The first vehicle operation condition data corresponding to the first local filter includes: running state information of the vehicle;
the second vehicle operation condition data corresponding to the second local filter includes: the running state information of the vehicle and the position information of the vehicle;
The third vehicle operation condition data corresponding to the third local filter includes: the position information of the vehicle and the environment information of the environment in which the vehicle is located.
In some embodiments, the acquisition device comprises: and the inertia measurement unit is used for acquiring the running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information.
In some embodiments, the calibration device comprises: the conversion unit is electrically connected with the main filter and the calibration unit respectively; wherein,
The conversion unit is used for converting the output result of the main filter into an external parameter matrix of the camera under a vehicle body coordinate system by utilizing a preset gesture transformation matrix;
The calibration unit is used for carrying out difference processing on the external parameter matrix of the camera under the vehicle body coordinate system and the initial calibration external parameter matrix of the camera so as to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
In some embodiments, the federal filter arrangement is a federal kalman filter arrangement.
In a second aspect, embodiments of the present disclosure provide a vehicle including a camera calibration system of the vehicle described above.
In a third aspect, embodiments of the present disclosure provide a camera calibration method, applied to a camera calibration system of a vehicle, the camera calibration system including: collection system, federal filter device and calibration device, federal filter device includes: a main filter and at least two local filters;
the method comprises the following steps:
the acquisition device acquires vehicle operation condition data and inputs corresponding vehicle operation condition data to each local filter respectively;
each local filter inputs an output result of the local filter to the main filter;
the main filter inputs the output result of the main filter to the calibration device;
And the calibration device calibrates the camera of the vehicle according to the output result of the main filter.
In some embodiments, the federal filter arrangement includes: a first local filter, a second local filter, and a third local filter; wherein,
The first vehicle operation condition data corresponding to the first local filter includes: running state information of the vehicle;
the second vehicle operation condition data corresponding to the second local filter includes: the running state information of the vehicle and the position information of the vehicle;
The third vehicle operation condition data corresponding to the third local filter includes: the position information of the vehicle and the environment information of the environment in which the vehicle is located.
In some embodiments, the acquisition device comprises: and the inertia measurement unit is used for acquiring the running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information.
In some embodiments of the present invention, in some embodiments,
The first output result of the first local filter is obtained by performing smoothing filtering processing on the running state information in the first vehicle running condition data;
the second output result of the second local filter is obtained by carrying out information fusion processing on the running state information and the position information in the second vehicle running condition data;
and the third output result of the third local filter is obtained by carrying out information fusion processing on the position information and the environment information in the third vehicle operation condition data.
In some embodiments, the calibration device calibrates the camera according to the output result of the main filter when the redundancy check is performed according to the output result of each local filter and the redundancy check passes.
In some embodiments, the first output result of the first local filter includes: the running state information, the second output result of the second local filter includes: the third output result of the third local filter includes: location information and environmental information;
when the running state information in the first output result is matched with the running state information in the second output result, and the position information in the second output result is matched with the position information in the third output result, the redundancy check is passed; otherwise, the redundancy check does not pass.
In some embodiments, the calibrating device calibrates the camera of the vehicle according to the output result of the main filter, including:
The calibration device converts the output result of the main filter into an external parameter matrix of the camera under a vehicle body coordinate system by using a preset gesture transformation matrix; performing difference processing on an external parameter matrix of the camera and an initial calibration external parameter matrix of the camera under the vehicle body coordinate system to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
In some embodiments, the federal filter arrangement is a federal kalman filter arrangement.
In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the camera calibration method described above.
In an embodiment of the present disclosure, a camera calibration system of a vehicle may include an acquisition device, a federal filter device, and a calibration device; after data acquisition, the acquisition device can provide corresponding vehicle operation condition data for each local filter in the federal filter device; each local filter can input an output result obtained according to the received vehicle operation condition data to a main filter in the federal filter device; the main filter may input an output result obtained according to the output result of each local filter to the calibration device, so that the calibration device performs calibration of the vehicle camera. It can be seen that, in the embodiments of the present disclosure, the calibration of the vehicle camera is not dependent on a specific environment, but on the collected vehicle operation condition data and the federal filter device, so that the embodiments of the present disclosure can get rid of the limitations of the existing calibration mode of the vehicle camera, and further have a certain fault diagnosis and recovery capability and a certain fault tolerance capability.
Drawings
FIG. 1 is a flow chart of a camera calibration method provided by an embodiment of the present disclosure;
FIG. 2 is a schematic structural view of a camera calibration system for a vehicle provided by an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of still another configuration of a camera calibration system for a vehicle provided by an embodiment of the present disclosure.
Detailed Description
The following description of the technical solutions in the embodiments of the present disclosure will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments. All other embodiments, which are derived by a person of ordinary skill in the art from the embodiments in the disclosure without creative efforts, are within the protection scope of the present disclosure.
The camera calibration method provided by the embodiment of the present disclosure is first described below.
It should be noted that, the camera calibration method provided by the embodiment of the present disclosure is applied to a camera calibration system of a vehicle, where the camera calibration system includes: collection system, federal filter device and calibration device, federal filter device includes: a main filter and at least two local filters.
Here, the vehicle may be a pure electric vehicle, a hybrid electric vehicle, or the like; the federal filter arrangement may be a federal kalman filter arrangement, that is, the filters in the federal filter arrangement all take the form of kalman filters; the number of local filters in the federal filter arrangement may be three, four or five. Of course, the type of the vehicle, the type of the federal filter arrangement, and the number of local filters in the federal filter arrangement are not limited thereto, and may be specifically determined according to actual circumstances, and the embodiments of the present disclosure are not limited thereto.
Referring to fig. 1, a flowchart of a camera calibration method provided by an embodiment of the present disclosure is shown. As shown in fig. 1, the method comprises the steps of:
Step 101, the acquisition device acquires vehicle operation condition data and inputs corresponding vehicle operation condition data to each local filter respectively.
Here, the collecting device may include a plurality of working condition data collecting tools, and different working condition data collecting tools are used for collecting different types of data; the number of the working condition data collection tools can be three, four, five or more than five, which are not listed here.
It should be noted that, a correspondence relationship between the local filter and the operating condition data type may be preset. In step 101, a plurality of working condition data collection tools may be called simultaneously to collect vehicle operation working condition data, and all the collected data together form a working condition data set. Then, according to the correspondence, the corresponding operating condition data type of each local filter can be determined, vehicle operating condition data with the determined operating condition data type is extracted from the operating condition data set, and the extracted vehicle operating condition data is input to the vehicle operating condition data.
Step 102, each local filter inputs its own output result to the main filter.
For any local filter, after receiving the vehicle operation condition data from the acquisition device, it may process the received vehicle operation condition data and provide the processing result as an output result to the main filter.
Step 103, the main filter inputs the output result of the main filter to the calibration device.
After receiving the output results from the local filters, the main filter may perform information fusion processing on the output results of the local filters and provide the obtained processing results as output results to the calibration device.
And 104, calibrating the camera of the vehicle by the calibrating device according to the output result of the main filter.
In some embodiments, step 104 comprises:
The calibration device converts the output result of the main filter into an external parameter matrix of the camera under the vehicle body coordinate system by using a preset gesture transformation matrix; performing difference processing on an external parameter matrix of a camera under a vehicle body coordinate system and an initial calibration external parameter matrix of the camera to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
It should be noted that, the calibration device may store an initial calibration external parameter matrix of the camera and a preset gesture transformation matrix, where the preset gesture transformation matrix may be determined according to parameter information such as length, width, height and the like of the vehicle body.
Here, the output result of the main filter may be in a matrix form, and in particular, the output result of the main filter may be an external matrix of the camera in the world coordinate system. After the output result of the main filter is obtained, the calibration device may obtain the stored gesture transformation matrix, and multiply the output result of the main filter with the gesture transformation matrix to obtain another matrix, where the another matrix may be used as an external parameter matrix of the camera in the vehicle body coordinate system. And then, calculating the matrix difference between the external reference matrix of the camera and the initial calibration external reference matrix of the camera under the vehicle body coordinate system, so as to conveniently obtain the dynamic calibration matrix of the camera. It can be seen that in this embodiment, the operation of implementing the dynamic calibration of the camera is very convenient to implement.
In an embodiment of the present disclosure, a camera calibration system of a vehicle may include an acquisition device, a federal filter device, and a calibration device; after data acquisition, the acquisition device can provide corresponding vehicle operation condition data for each local filter in the federal filter device; each local filter can input an output result obtained according to the received vehicle operation condition data to a main filter in the federal filter device; the main filter may input an output result obtained according to the output result of each local filter to the calibration device, so that the calibration device performs calibration of the vehicle camera. It can be seen that, in the embodiments of the present disclosure, the calibration of the vehicle camera is not dependent on a specific environment, but on the collected vehicle operation condition data and the federal filter device, so that the embodiments of the present disclosure can get rid of the limitations of the existing calibration mode of the vehicle camera, and further have a certain fault diagnosis and recovery capability and a certain fault tolerance capability.
It should be noted that the number of local filters in the federal filter apparatus may be three, in which case the federal filter apparatus may include: a first local filter, a second local filter, and a third local filter; wherein,
The first vehicle operating condition data corresponding to the first local filter includes: running state information of the vehicle;
the second vehicle operation condition data corresponding to the second local filter includes: running state information of the vehicle and position information of the vehicle;
The third vehicle operation condition data corresponding to the third local filter includes: the position information of the vehicle and the environment information of the environment in which the vehicle is located.
Here, the collecting device may include three working condition data collecting tools, which are respectively: the first working condition data acquisition tool is used for acquiring the running state information of the vehicle, and the second working condition data acquisition tool is used for acquiring the position information of the vehicle; and the third working condition data acquisition tool is used for acquiring environment information of the environment where the vehicle is located. Specifically, the running state information may include attitude information, speed information, and the like; the location information may include coordinates of the real-time location; the environmental information may include road inclination, vehicle deviation road information (which may be used for a positional offset between the vehicle and the lane line), and the like.
In some embodiments, the acquisition device may comprise: and the inertial measurement unit (Inertial Measurement Unit, IMU) is used for acquiring the running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information. In this case, the first operating mode data acquisition means may be considered as an inertial measurement unit.
It will be appreciated that the inertial measurement unit can be used to measure the three-axis attitude angle (or angular velocity) and acceleration of an object. Specifically, the inertial measurement unit comprises three single-axis accelerometers and three single-axis gyroscopes, wherein the accelerometers detect acceleration signals of the object on three independent axes of a carrier coordinate system, the gyroscopes detect angular velocity signals of the carrier relative to a navigation coordinate system, angular velocity and acceleration of the object in a three-dimensional space are measured, and the posture of the object is calculated according to the angular velocity and the acceleration signals. Thus, the inertial measurement unit can be used to obtain the running state information of the vehicle conveniently and reliably.
In some embodiments, the acquisition device may include a global positioning system (Global Positioning System, GPS) for acquiring location information of the vehicle. In this case, the second operating mode data collection tool may be considered a global positioning system.
It will be appreciated that the global positioning system can provide accurate positioning, speed measurement, and high accuracy standard time for most areas of the earth's surface. Thus, the GPS is utilized to conveniently and reliably obtain the position information of the vehicle.
In some embodiments, the acquisition device may be installed with a map application, for example with a high-precision map application. In this case, the third operating mode data collection tool may be considered a map application.
It will be appreciated that the map application can provide rich information, for example, the map application may prompt whether the vehicle is currently on an uphill road or a downhill road, and if so, the angle of inclination of the current road; the map application may also prompt the location of the lane line of the current road, as well as the amount of positional offset between the current location of the vehicle and the lane line. Thus, by using the map application, the environment information of the environment where the vehicle is located can be obtained conveniently and reliably.
After the first local filter to the third local filter respectively acquire corresponding vehicle operation condition data, the first local filter to the third local filter can respectively acquire corresponding output results.
Specifically, the first output result of the first local filter may be obtained by performing smoothing filtering processing on the running state information in the first vehicle running condition data;
the second output result of the second local filter can be obtained by performing information fusion processing on the running state information and the position information in the second vehicle running condition data;
the third output result of the third local filter may be obtained by performing information fusion processing on the position information and the environmental information in the third vehicle operation condition data.
Here, by performing smoothing filtering processing on the running state information in the first vehicle running condition data, very accurate running state information may exist in the first output result; by carrying out information fusion processing on the running state information and the position information in the second vehicle running condition data, the advantages of the running state information and the position information can be combined, so that a better filtering effect is achieved; the advantages of the position information and the environment information can be combined through information fusion processing of the position information and the environment information in the third vehicle operation condition data, so that a better filtering effect is achieved. The smoothing filter processing and the information fusion processing can bring the effects, so that the calibration effect of the camera in calibration can be well ensured.
In some embodiments, the calibration device calibrates the camera according to the output result of the main filter in the case that the redundancy check is performed according to the output result of each local filter and the redundancy check passes.
Specifically, the first output result of the first local filter comprises running state information, the second output result of the second local filter comprises running state information and position information, and the third output result of the third local filter comprises position information and environment information;
in the case that the running state information in the first output result and the running state information in the second output result are matched, and the position information in the second output result and the position information in the third output result are matched, the redundancy check passes; otherwise, the redundancy check does not pass.
Here, the driving state information may include an angle between the vehicle and the ground, and a driving speed of the vehicle, and when a difference between the angle in the driving state information in the first output result and the angle in the second output result is smaller than a preset angle, and a difference between the driving speed in the driving state information in the first output result and the driving speed in the second output result is smaller than a preset speed difference, the driving state information in the first output result may be considered to be matched with the driving state information in the second output result.
Here, the position information may include coordinates of a current position of the vehicle, and in the case where a distance between the coordinates in the position information in the second output result and the coordinates in the position information in the second output result is smaller than a preset distance, the position information in the second output result and the position information in the third output result may be considered to match.
It should be noted that, when the running state information in the first output result and the running state information in the second output result are matched, and the position information in the second output result and the position information in the third output result are matched, the information input into the main filter by each local filter may be considered to be sufficiently accurate, and at this time, the calibration device may determine that the redundancy check passes, and perform the calibration operation of the camera.
Therefore, in the embodiment of the disclosure, the redundancy check can be used for checking the accuracy of the information input into the main filter, and the calibration device can execute the calibration operation of the camera only under the condition that the redundancy check passes, so that the accuracy of the calibration result can be better ensured.
Of course, the manner of redundancy check is not limited to the above case. For example, the calibration device may store in advance a correspondence between the position coordinates and an included angle range, where the included angle range includes all possible values of the included angle between the vehicle and the ground when the vehicle is at the position corresponding to the position coordinates. When the first output result of the first local filter includes driving state information, the driving state information includes an included angle, and the second output result of the second local filter includes coordinates of the current position of the vehicle, the calibration device may determine, according to a pre-stored correspondence, an included angle range corresponding to the coordinates in the second output result, and determine whether the included angle in the first output result is within the included angle range; in the case where the determination result is located, it may be determined that the first output result and the second output result match, which is also possible.
The following describes in detail the implementation of embodiments of the present disclosure with a specific example.
As shown in fig. 2 and 3, the camera calibration system of the vehicle may include: the acquisition device 21, the federal filter device 23 and the calibration device 25, the federal filter device 23 includes: a main filter 231 and at least two local filters; wherein the acquisition device 21 is connected to one end (right end shown in fig. 2) of each local filter, the other end (left end shown in fig. 2) of each local filter may be connected to one end (right end shown in fig. 2) of the main filter 231, and the other end (left end shown in fig. 2) of the main filter 231 may be connected to the calibration device 25. In some embodiments, the calibration device 25 may include a conversion unit 251 and a calibration unit 253, and the conversion unit 251 is electrically connected to the main filter 231 (specifically, the left end of the main filter 231) and the calibration unit 253, respectively.
Specifically, the federal filter apparatus 23 may be a federal kalman filter apparatus, the number of local filters in the federal filter apparatus 23 may be three, and all the three local filters may operate independently, and then the federal filter apparatus 23 may include: a first local filter 233, a second local filter 235, a third local filter 237; the acquisition device 21 may include an inertial measurement unit for acquiring running state information of the vehicle and a global positioning system for acquiring position information of the vehicle, and the acquisition device 21 may be installed with a high-precision map application for acquiring environmental information of an environment in which the vehicle is located.
In actual operation, the acquisition device 21 may input the running state information acquired by the inertial measurement unit into the first local filter 233; inputting the driving state information acquired by the inertial measurement unit and the position information acquired by the global positioning system into a second local filter 235; the position information acquired by the global positioning system, and the environmental information acquired by the high-precision map application are input to the third local filter 237.
Next, the first local filter 233 may perform a smoothing filter process on the received driving state information to obtain a first output result, and input the first output result to the main filter 231; the second local filter 235 may perform information fusion processing on the received position information of the driving state information to obtain a second output result, and input the second output result to the main filter 231; the third local filter 237 may perform information fusion processing on the received location information and environment information to obtain a third output result, and input the third output result to the main filter 231.
After that, the output results of the first to third local filters 233 to 237 may be summarized to the main filter 231, and the main filter 231 may perform redundancy check according to the received output result, and in the case where the redundancy check passes, perform information fusion processing to obtain an output result including posture information, position information, environmental information, and the like, and input the output result to the calibration device 25.
Finally, the conversion unit 251 in the calibration device 25 may convert the output result of the main filter 231 into an external parameter matrix of the camera under the vehicle body coordinate system by using a preset gesture transformation matrix, and the calibration unit 253 in the calibration device 25 may perform a difference processing on the external parameter matrix of the camera under the vehicle body coordinate system and the initial calibration external parameter matrix of the camera to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera, so that the calibration of the vehicle camera is realized.
In summary, in the embodiments of the present disclosure, information collected by an inertial measurement unit, a global positioning system, and a high-precision map application may be fused by using a federal kalman filter device to implement self-calibration of a vehicle camera, so that the embodiments of the present disclosure may implement calibration of a vehicle camera without depending on a specific environment, and the embodiments of the present disclosure have certain fault diagnosis and recovery capabilities, and certain fault tolerance capabilities.
A camera calibration system for a vehicle provided by embodiments of the present disclosure is described below.
Referring to fig. 2, a schematic structural diagram of a camera calibration system of a vehicle provided by an embodiment of the present disclosure is shown. As shown in fig. 2, the camera calibration system includes: the acquisition device 21, the federal filter device 23 and the calibration device 25, the federal filter device 23 includes: a main filter 231 and at least two partial filters, the acquisition device 21 is connected to one end (right end shown in fig. 2) of each partial filter, the other end (left end shown in fig. 2) of each partial filter is connected to one end (right end shown in fig. 2) of the main filter 231, and the other end (left end shown in fig. 2) of the main filter 231 is connected to the calibration device 25; wherein,
The acquisition device 21 is used for acquiring vehicle operation condition data and inputting corresponding vehicle operation condition data to each local filter respectively;
each local filter inputs its own output result to the main filter 231;
The main filter 231 is used for inputting the output result of the main filter to the calibration device 25;
The calibration device 25 is used for calibrating the camera according to the output result of the main filter 231.
In some embodiments, as shown in fig. 2 and 3, the federal filter arrangement 23 includes: a first local filter 233, a second local filter 235, and a third local filter 237; wherein,
The first vehicle operating condition data corresponding to the first local filter 233 includes: running state information of the vehicle;
The second vehicle operating condition data corresponding to the second local filter 235 includes: running state information of the vehicle and position information of the vehicle;
The third vehicle operating condition data corresponding to the third local filter 237 includes: the position information of the vehicle and the environment information of the environment in which the vehicle is located.
In some embodiments, the acquisition device 21 comprises: and the inertia measurement unit is used for acquiring running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information.
In some embodiments, as shown in fig. 2, the calibration device 25 includes: a conversion unit 251 and a calibration unit 253, the conversion unit 251 being electrically connected to the main filter 231 and the calibration unit 253, respectively; wherein,
A conversion unit 251, configured to convert the output result of the main filter 231 into an external parameter matrix of the camera under the vehicle body coordinate system by using a preset gesture transformation matrix;
The calibration unit 253 is used for performing difference processing on the external parameter matrix of the camera under the vehicle body coordinate system and the initial calibration external parameter matrix of the camera so as to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
In some embodiments, the federal filter arrangement 23 is a federal kalman filter arrangement.
It can be seen that, in the embodiments of the present disclosure, the calibration of the vehicle camera is not dependent on the specific environment, but on the collected vehicle operation condition data, and the federal filter device 23, so that the limitations of the existing vehicle camera calibration mode can be removed, and the embodiments of the present disclosure also have a certain fault diagnosis and recovery capability, and a certain fault tolerance capability.
A vehicle provided by an embodiment of the present disclosure is described below.
Embodiments of the present disclosure also provide a vehicle including the camera calibration system of the vehicle described above. The specific implementation process of the camera calibration system of the vehicle is described above, and will not be described herein.
It can be seen that, in the embodiments of the present disclosure, the calibration of the vehicle camera is not dependent on a specific environment, but on the collected vehicle operation condition data and the federal filter device, so that the embodiments of the present disclosure can get rid of the limitations of the existing calibration mode of the vehicle camera, and further have a certain fault diagnosis and recovery capability and a certain fault tolerance capability.
The embodiments of the present disclosure further provide a computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements each process of the above-mentioned camera calibration method embodiment, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here. The computer readable storage medium is, for example, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk or an optical disk.
The foregoing is merely specific embodiments of the disclosure, but the protection scope of the disclosure is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the disclosure, and it is intended to cover the scope of the disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (13)
1. A camera calibration system for a vehicle, comprising: collection system, federal filter device and calibration device, federal filter device includes: the acquisition device is connected with one end of each local filter, the other end of each local filter is connected with one end of the main filter, and the other end of the main filter is connected with the calibration device; wherein,
The acquisition device is used for acquiring vehicle operation condition data and inputting corresponding vehicle operation condition data to each local filter respectively;
Each local filter is used for inputting an output result of the local filter to the main filter;
the main filter is used for inputting the output result of the main filter to the calibration device;
The calibration device is used for calibrating the camera of the vehicle according to the output result of the main filter;
the federal filter arrangement includes: a first local filter, a second local filter, and a third local filter; wherein,
The first vehicle operation condition data corresponding to the first local filter includes: the running state information of the vehicle comprises attitude information and speed information;
The second vehicle operation condition data corresponding to the second local filter includes: the driving state information of the vehicle and the position information of the vehicle, wherein the position information comprises coordinates of a real-time position;
the third vehicle operation condition data corresponding to the third local filter includes: the vehicle position information and the environment information of the environment where the vehicle is located, wherein the environment information comprises road inclination angle and vehicle deviation road information.
2. The system of claim 1, wherein the acquisition device comprises: and the inertia measurement unit is used for acquiring the running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information.
3. The system of claim 1, wherein the calibration device comprises: the conversion unit is electrically connected with the main filter and the calibration unit respectively; wherein,
The conversion unit is used for converting the output result of the main filter into an external parameter matrix of the camera under a vehicle body coordinate system by utilizing a preset gesture transformation matrix;
The calibration unit is used for carrying out difference processing on the external parameter matrix of the camera under the vehicle body coordinate system and the initial calibration external parameter matrix of the camera so as to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
4. A system according to any one of claims 1 to 3, wherein the federal filter arrangement is a federal kalman filter arrangement.
5. A vehicle comprising a camera calibration system of the vehicle of any one of claims 1 to 4.
6. A camera calibration method, characterized by being applied to a camera calibration system of a vehicle, the camera calibration system comprising: collection system, federal filter device and calibration device, federal filter device includes: a main filter and at least two local filters;
the method comprises the following steps:
the acquisition device acquires vehicle operation condition data and inputs corresponding vehicle operation condition data to each local filter respectively;
each local filter inputs an output result of the local filter to the main filter;
the main filter inputs the output result of the main filter to the calibration device;
the calibration device calibrates a camera of the vehicle according to the output result of the main filter;
the federal filter arrangement includes: a first local filter, a second local filter, and a third local filter; wherein,
The first vehicle operation condition data corresponding to the first local filter includes: the running state information of the vehicle comprises attitude information and speed information;
The second vehicle operation condition data corresponding to the second local filter includes: the driving state information of the vehicle and the position information of the vehicle, wherein the position information comprises coordinates of a real-time position;
the third vehicle operation condition data corresponding to the third local filter includes: the vehicle position information and the environment information of the environment where the vehicle is located, wherein the environment information comprises road inclination angle and vehicle deviation road information.
7. The method of claim 6, wherein the acquisition device comprises: and the inertia measurement unit is used for acquiring the running state information of the vehicle, wherein the running state information of the vehicle comprises attitude information and speed information.
8. The method of claim 6, wherein the step of providing the first layer comprises,
The first output result of the first local filter is obtained by performing smoothing filtering processing on the running state information in the first vehicle running condition data;
the second output result of the second local filter is obtained by carrying out information fusion processing on the running state information and the position information in the second vehicle running condition data;
and the third output result of the third local filter is obtained by carrying out information fusion processing on the position information and the environment information in the third vehicle operation condition data.
9. The method according to claim 8, wherein the calibration means calibrates the camera according to the output result of the main filter in the case where redundancy check is performed according to the output result of each of the local filters and the redundancy check passes.
10. The method of claim 9, wherein the first output result of the first local filter comprises: the running state information, the second output result of the second local filter includes: the third output result of the third local filter includes: location information and environmental information;
when the running state information in the first output result is matched with the running state information in the second output result, and the position information in the second output result is matched with the position information in the third output result, the redundancy check is passed; otherwise, the redundancy check does not pass.
11. The method of claim 6, wherein the calibrating means calibrates the camera of the vehicle based on the output of the main filter, comprising:
The calibration device converts the output result of the main filter into an external parameter matrix of the camera under a vehicle body coordinate system by using a preset gesture transformation matrix; performing difference processing on an external parameter matrix of the camera and an initial calibration external parameter matrix of the camera under the vehicle body coordinate system to obtain an incremental external parameter matrix; and taking the increment external parameter matrix as a dynamic calibration matrix of the camera.
12. The method according to any one of claims 6 to 11, wherein the federal filter arrangement is a federal kalman filter arrangement.
13. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the camera calibration method according to any of claims 6 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910822881.8A CN112446924B (en) | 2019-09-02 | Camera calibration system of vehicle, vehicle and camera calibration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910822881.8A CN112446924B (en) | 2019-09-02 | Camera calibration system of vehicle, vehicle and camera calibration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112446924A CN112446924A (en) | 2021-03-05 |
| CN112446924B true CN112446924B (en) | 2024-06-28 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10176596B1 (en) * | 2017-07-06 | 2019-01-08 | GM Global Technology Operations LLC | Calibration verification methods for autonomous vehicle operations |
| CN110047109A (en) * | 2019-03-11 | 2019-07-23 | 南京航空航天大学 | A kind of camera calibration plate and its recognition detection method based on self-identifying label |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10176596B1 (en) * | 2017-07-06 | 2019-01-08 | GM Global Technology Operations LLC | Calibration verification methods for autonomous vehicle operations |
| CN110047109A (en) * | 2019-03-11 | 2019-07-23 | 南京航空航天大学 | A kind of camera calibration plate and its recognition detection method based on self-identifying label |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110160542B (en) | Method and device for positioning lane line, storage medium and electronic device | |
| CN109931944B (en) | AR navigation method, AR navigation device, vehicle-side equipment, server side and medium | |
| CN105865461B (en) | A kind of car position system and method based on Multi-sensor Fusion algorithm | |
| CN107728175A (en) | The automatic driving vehicle navigation and positioning accuracy antidote merged based on GNSS and VO | |
| CN110411457B (en) | Positioning method, system, terminal and storage medium based on stroke perception and vision fusion | |
| CN105547288A (en) | Self-localization method and system for mobile device in underground coal mine | |
| CN104181573B (en) | Big Dipper inertial navigation deep integrated navigation micro-system | |
| CN112835085B (en) | Method and device for determining vehicle position | |
| CN110458885B (en) | Positioning system and mobile terminal based on stroke perception and vision fusion | |
| CN111947644B (en) | Outdoor mobile robot positioning method and system and electronic equipment thereof | |
| CN114061619B (en) | Inertial navigation system attitude compensation method based on online calibration | |
| CN108613675B (en) | Low-cost unmanned aerial vehicle movement measurement method and system | |
| CN104197958A (en) | Speedometer calibration method based on laser velocimeter dead reckoning system | |
| CN105444764A (en) | Attitude measurement method based on assistance of speedometer of vehicle | |
| CN109916417B (en) | Map establishing method, map establishing device, computer equipment and storage medium thereof | |
| CN104833529A (en) | Vehicle performance test system based on inertial navigation and test method thereof | |
| CN112446924B (en) | Camera calibration system of vehicle, vehicle and camera calibration method | |
| CN114147717B (en) | Robot motion track estimation method, device, controller and storage medium | |
| CN115900732A (en) | Combined navigation method and system based on roadside camera and vehicle-mounted unit | |
| CN112446924A (en) | Camera calibration system of vehicle, vehicle and camera calibration method | |
| Heipke et al. | Integrated sensor orientation-an OEEPE Test | |
| CN115560744A (en) | Robot, multi-sensor-based three-dimensional mapping method and storage medium | |
| Trojnacki et al. | Determination of motion parameters with inertial measurement units–Part 1: mathematical formulation of the algorithm | |
| CN114623834B (en) | Combined navigation positioning measurement method and measurement system | |
| CN108279004A (en) | A kind of single GPS robot navigation methods based on perturbation action |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |