CN103593836A - A Camera parameter calculating method and a method for determining vehicle body posture with cameras - Google Patents
A Camera parameter calculating method and a method for determining vehicle body posture with cameras Download PDFInfo
- Publication number
- CN103593836A CN103593836A CN201210287115.4A CN201210287115A CN103593836A CN 103593836 A CN103593836 A CN 103593836A CN 201210287115 A CN201210287115 A CN 201210287115A CN 103593836 A CN103593836 A CN 103593836A
- Authority
- CN
- China
- Prior art keywords
- camera
- car body
- vehicle
- parameter
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention is to solve the problems of low efficiency and rough precision in a plurality of cameras calibration process in a vehicle-mounted around-looking system. According to the method, in one progress, fewer feature points are utilized to completely calibrate internal parameters and external parameters of a camera. An analytic mathematical algorithm is utilized to find an optimal value of the parameters; and with respect to a parameter space scanning method, an initial value, which is close to a true value, of the internal parameters and external parameters is not needed. In the invention, individual differences produced in the camera design production process are taken into consideration. Therefore, vehicle-mounted wide-angle cameras which are produced by different manufacturers and in different batches are adapted in the method without configuring any internal parameter for the camera separately. According to the method, individual position and angle differences produced in the vehicle-mounted camera assembling process are taken into consideration without presetting any external parameters for any type of vehicles by operators. Position relationship between all cameras and correction markers is utilized to calculate the posture of a vehicle body, thereby maximumly moderating the strict requirement of high-precision disposition of the vehicle and the correction markers, enabling the vehicle to park freely in a certain range, and improving operation convenience and time efficiency.
Description
Technical field
The present invention relates to vehicle-mounted multi-view camera viewing system, relate in particular to the scaling method of vehicle-mounted viewing system multiple cameras inner parameter and external parameter and the computing method of car body attitude.
Background technology
Society, automobile has become a kind of requisite vehicles.The convenience that people bring at enjoyment automobile and efficiently while, motor traffic accidents, the problem that the environmental pollution that motor vehicle exhaust emission brings and traffic jam are brought, has become increasingly serious global social concern.Therefore, utilize various advanced persons' automotive safety technology, equipment and theory minimizing traffic hazard and raising automotive safety to have very large market potential.
The last century is after the nineties, and along with the widespread use in automobile product of electronic technology, control technology, sensor technology and new material, automotive safety technology has obtained swifter and more violent development.Nowadays, the research of automotive safety technology is by the research and development of single safety technique, to various safety techniques, merges mutually collaborative integrated, systematization and intelligent direction development.Intelligentized automobile safety system be take modern Detection Techniques, photoelectric sense technology, computer technology and automatic control technology as core, there is specific identification judgement, can under various complex situations, automatically assist driver or control voluntarily automobile, guarantee traffic safety.
Vehicle environment sensory perceptual system utilizes various sensors to survey information such as vehicle self, surrounding environment and driver status, by comparing with predefined standard, differentiate the whether in the hole and hazard level of vehicle, can to driver, carry out early warning by modes such as sound, light if desired.
The sensor that vehicle environment sensory perceptual system is used at present mainly contains: 1) monocular or multi-lens camera system, by the ambient image that Real-time Collection is come, process information such as obtaining correlation distance, position; 2) laser radar or millimetre-wave radar, by sending and receive infrared laser or electric wave, according to Doppler effect, the information such as the distance of calculating periphery barrier and position; 3) sonar, sends and receives ultrasound wave by orientation, the information such as the distance of calculating periphery barrier and position.
By comparison, it is wide that laser or millimetre-wave radar can be surveyed scope, and the ability of anti-extraneous rugged surroundings is strong, but conventionally only have one or more layers scanning plane, cannot obtain the 3 D stereo information of whole scene, and expensive.Sonar is only applicable to low coverage and measures (such as reversing is used), and only has the dot information on direction of illumination.Vehicle-mounted camera system obtains information by vision, be at present most widely used general be also one of the most promising sensor.Such as backup camera is to apply the most general a kind of vehicle-mounted vision system.It is generally arranged on vehicle rear, and direction is towards side-lower.When driver moves backward, system is connected vehicle-carrying display screen automatically, for driver provides rear wide area scene.
In order to allow driver understand more accurately the dangerous information such as barrier of vehicle-surroundings, system is transformed to virtual video camera image vertically downward by the conversion process at visual angle by original image with certain angle of depression.
Backup camera is used for the reversing environment of more complicated, but the visual field of a video camera is limited, in the situation that vehicle front side barrier is more, conventionally adopt multiple cameras to form vehicle viewing system, for driver provides the complete environment sensing information of vehicle 360 degree without dead angle.
Vehicle viewing system is mainly the image of taking by being arranged on vehicle multiple cameras all around, through viewpoint change, process and to obtain multiple and virtually take vehicle as with reference to the downward vertical view of central vertical, and be spliced into a complete vertical view of looking around by composition algorithm.
Described in this patent, vehicle-mounted multi-view camera viewing system refers to the multi-eye stereo camera shooting machine system of vehicle environment perception.
(1) prior art one related to the present invention: single viewpoint change technology
Prior art one (International Publication patent W000-07373, Jap.P. JP3625059) can utilize and be assemblied in the inner parameter of car body 4 cameras all around and external parameter and the image absorbing of each camera is done to single viewpoint change it is for conversion into take car body as the vertical view with reference to center, and then this 4 width is done to splicing by the resulting vertical view of each camera generate a width by 4 vertical views that camera image is seamless spliced.
(2) prior art two related to the present invention: the demarcation of video camera external parameter posture
Prior art two (Jap.P. JP 2011-151666)
Prior art two is in the left-front corner of vehicle, right front angle, and left rear corner, 4 positive triangle calibration mark things are placed according to both allocations by place, right back angle.In implementation process, to strictly guarantee the position relationship of vehicle and positive triangle calibration mark thing.Such modes of emplacement can guarantee that every camera can absorb two equilateral triangle calibration mark things in its image.Then prior art two utilizes the vertex information of positive triangle calibration mark thing in image through certain calculation, the posture of camera to be demarcated.The posture of 4 cameras all demarcated complete after, just can utilize viewpoint change technology to generate and synthesize vertical view.Prior art two take true vertical view that a width obtained by the aerial camera in vehicle central top as benchmark with through viewpoint change, synthesize vertical view and compare and determine that whether calibration result correct.
(3) prior art three related to the present invention: the demarcation to minority camera inside and outside parameter
Prior art three (Jap.P. JP 2011-185753)
Prior art three is used road white line as calibration mark thing, adopts the mode of parameter space scanning first then the distortion parameter k1 in the inner parameter of camera to be demarcated the angle of depression up and down in external parameter is demarcated.
(4) prior art four related to the present invention: the posture angle in camera external parameter and horizontal level are demarcated
Prior art four (Jap.P. JP 2009-288152)
Prior art four use on surface level, lay 6 both sizing to camera, use tentative wrong parameter spacescan method to carry out corner and the horizontal level demarcation in external parameter with the rectangle calibration mark thing of position relationship.
(5) prior art five related to the present invention: sequentially find camera inner parameter and external parameter by parameter space scan mode
Prior art five (Jap.P. JP 2012-15576)
Prior art five arranges the parallel lines that two groups of distance relations are known on level road, and these two groups of parallel lines are mutual orthogonal again.The straight line that must contain at least two parallel lines and one and their orthogonal in the image of every video camera all around of installing on vehicle.First prior art five proofreaies and correct the inner parameter of every camera from one group of default camera inner parameter, and its evaluation criterion is the linearity of each straight line above the image after distortion correction.Camera inner parameter is proofreaied and correct complete prior art five afterwards and from the external parameter of one group of acquiescence, camera external parameter is proofreaied and correct, its correction target is the parallel relation between the straight line of finding in the vertical view that one group of external parameter makes to obtain by viewpoint change, orthogonal relationship, distance relation is consistent with actual value.
The method that prior art five is used tentative wrong parameter spacescan (is generally near the parameter space inner parameter that the design inner parameter) know according to this group of a certain definite step scan from one group of inner parameter of knowing, for its corresponding distortion correction image of parameter space dot generation one width of each scanning, the evaluation index of the parameter space point that the linearity of the straight line on this width distortion correction image of usining is scanned as this.After the tentative wrong parameter spacescan method of prior art five use has been determined the inner parameter of every camera, also (normally the camera of design is assemblied near the parameter space external parameter that three-dimensional position car body and posture) know according to this group of a certain definite step scan from one group of external parameter of knowing to use tentative wrong parameter spacescan method, its corresponding viewpoint change vertical view of parameter space dot generation one width for each scanning, the depth of parallelism with the straight line on this width vertical view, orthogonal degree, distance and same rectilinearity are as the evaluation index of parameter point.
(6) shortcoming of prior art:
The prerequisite of prior art one is that inner parameter and the external parameter of camera known, car body parks according to both allocations or the attitude of car body is known.
The shortcoming of prior art two: the position relationship between vehicle and calibration mark thing will strictly guarantee, the inner parameter of camera must both know, camera must both be known with respect to the positional information of car body.
The shortcoming of prior art three: the inner parameter that prior art three is demarcated and external parameter very little, can only be for the rough demarcation of single camera, the demarcation of the seamless ring viewing system of inadaptable 4 cameras; Within the parameter space scanning algorithm that prior art three adopts must guarantee that desired true value falls within the parameter space being scanned, this just requires initial value to approach with true value.
The shortcoming of prior art four: the method for prior art four operation parameter spacescans is found the external parameter of camera.This kind of method must have the initial value of one group of external parameter nearer apart from true value.Also different for dissimilar car body or different its external parameter initial value of camera rigging position.This will be realistic border to select according to the car body type of concrete use and camera rigging position attitude (posture and position) initial value of different external parameters when implementing.Prior art four is because the mode of operation parameter scanning is found true value, and when parameter dimensions is huge, its scanning need to expend very large calculating and carrying cost, thereby poses a big pressure to embedded real-time processing.The step-length of scanning also can cause delicate impact to result in addition, sometimes even can not restrain.
The shortcoming of prior art five: the method for the tentative wrong parameter spacescan of prior art five use is found inner parameter and the external parameter of camera.This kind of method must have one group apart from the initial value of the nearer inner parameter of true value and external parameter.Its inner parameter initial value of camera for different batches different manufacturers is different; Also different for dissimilar car body or different its external parameter initial value of camera rigging position.This will be realistic border to select different inner parameters and the initial value of external parameter according to the camera type car body type of concrete use etc. when implementing.Prior art five is because the mode of operation parameter scanning is found true value, when parameter dimensions is huge (according to different camera model internal references in 10 dimensions left and right; The outer ginseng of each camera is 6 dimensions, and 4 Camera extrinsic are 6 * 4=24 dimensions) its scanning need to expend very large calculating and carrying cost, thereby pose a big pressure to embedded real-time processing.The step-length of scanning also can cause delicate impact to result in addition, sometimes even can not restrain.Mode action required space and workload that prior art five uses arrange straight line are larger, and the change work needing while changing straight line configuration for dissimilar vehicle is larger.
Summary of the invention
Position relationship between many camera calibration technical requirement vehicles of vehicle-mounted viewing system and calibration mark thing will be parked or put according to set position relationship high precision in the past.In practical operation, or vehicle is first parked and then take the position of vehicle and put calibration mark thing as benchmark, or first correction mark is first well placed and then take calibration mark thing and park cars as benchmark.This method need to take a long time to vehicle and calibration mark thing according to set position relationship, to place completely by trial and error mode when practical operation.Its shortcoming is inefficiency.
The present invention is in order to solve the extensive problem of inefficiency precision in many camera calibration processes of vehicle-mounted viewing system.
The present invention utilizes less unique point that the inner parameter of camera and external parameter are demarcated complete completely in a process.The present invention adopts parsing mathematical algorithm to find optimal value to parameter, with respect to the present invention of parameter space scan method, does not need one to approach the inner parameter of true value and the initial value of external parameter.The present invention has considered the individual difference producing in camera design production run.Therefore the present invention adapts to for the vehicle-mounted wide-angle camera of looking around of different vendor's different batches, and does not need to configure separately any inner parameter for it.The present invention has considered the individual body position and the angle difference that in in-vehicle camera assembling process, produce, and not needing operator is that the vehicle of each type presets any external parameter.
The present invention utilizes the position relationship between each camera and calibration mark thing, calculate the attitude of car body, thereby at utmost relaxed the strict demand that vehicle and calibration mark object height precision are placed, vehicle can freely be parked within the specific limits, improved operation ease and time efficiency.
Accompanying drawing explanation
Fig. 1: proofread and correct and use mark schematic diagram.
Fig. 2: the set-up mode schematic diagram of mark for 4 block corrections.
Fig. 3: the schematic diagram of parking the vehicle in demarcation region.
Fig. 4: the FOV (Field of View) schematic diagram of each vehicle-mounted vidicon when storing cycle enters to demarcate in region.
Fig. 5: the demarcation that each camera absorbs image example.
Fig. 6: a kind of storing cycle example that meets storing cycle constraint condition.
Fig. 7: selected characteristic point coordinate exemplary plot on image.
Fig. 8: the demarcation flow process of single camera.
Fig. 9: single camera is demarcated complete projective transformation gained unique point afterwards
Figure 10: each coordinate system relation
Embodiment
Below in conjunction with drawings and Examples, the invention will be further described.The vehicle-mounted disposal system of looking around is to be become by the Duo Tai wide-angle imaging mechanism being assemblied on car body all around, and it is certain interval that every video camera covers, and image generates vertical view through viewpoint change, and splices and obtain vehicle and look around aerial view.In order to make the image that above-mentioned many wide-angle imaging machines that are assemblied on car body absorb can generate the not seamless spliced synthetic panoramic view of looking down of distortion through viewpoint change, the above-mentioned inner parameter (camera lens of video camera and the parameter of sensor devices) that is assemblied in the wide-angle imaging machine on car body must be by Accurate Calibration with external parameter (camera coordinate system, the relative position relation of world coordinate system and bodywork reference frame).Whether the inner parameter of video camera accurately directly affects the distortion correction effect of composite diagram.Whether whether the external parameter of video camera accurately can affect each camera image junction in composite diagram misplaces, and also can affect whether in final composite diagram, look down the relative position relation of panoramic view and car body correct.
The present invention proposes by less unique point, the inner parameter of camera and external parameter to be demarcated complete completely in a process.
The present invention utilizes the position relationship between each camera and calibration mark thing, has at utmost relaxed the strict demand that vehicle and calibration mark object height precision are placed, and vehicle can freely be parked within the specific limits.
The present invention puts in the following manner calibration mark thing on level road.Calibration mark thing in the present invention is black and white grid as shown in Figure 1.Big or small dX and the dY of black and white grid both knew, number nX and the nY of whole calibration mark thing medium square both knew (nX * nY=2 * 2 in Fig. 1).In the present invention, used 4 block correction marks, this 4 block mark thing is put by rectangular mode on surface level, requires to be parallel to each other, as shown in Figure 2 between mark.The Width of the rectangle that 4 block mark things in Fig. 2 surround and width and the length that Height is greater than car body, thus vehicle can be parked into being of a size of as shown in Figure 2 in the rectangular area of Width and Height.
Scaling method proposed by the invention requires to be demarcated vehicle and stops shown in Fig. 2, being of a size of in the rectangular area of Width and Height, parks afterwards as schematically shown in Figure 3.Concrete parking is restricted to: vehicle-mounted 4 camera Cam all around
f, Cam
r, Cam
b, Cam
lfOV (Field of View) VF
f, VF
r, VF
b, VF
loverlapping region VF
rF, VF
rB, VF
lF, VF
lBin must have a complete correction mark, in the image that every camera absorbs, have two complete calibration mark things, as shown in Figure 4.Figure 5 shows that vehicle-mounted 4 camera Cam all around
f, Cam
r, Cam
b, Cam
lthe example of image for the demarcation of absorbing according to above-mentioned calibration mark thing disposing way and storing cycle mode.In the present invention, the constraint condition of storing cycle is: parking of vehicle will guarantee that every camera can absorb two blocks of complete mark images.Except this constraint, vehicle can freely be parked, thereby has improved convenience and the efficiency of operation.Fig. 6 is that the another kind that meets storing cycle constraint condition is parked example.
Set forth the inside and outside parameter scaling method of single camera below.
Existing vehicle-mounted its inner parameter of wide-angle camera on market is comprised: horizontal focal distance f x, longitudinal focal distance f y, the non-square degree alpha of sensitive chip pixel, the position cx of camera lens optical axis on sensitive chip, cy, radial distortion and tangential distortion k1, k2, k3, k4, k5, also has other model parameter to add for fish eye lens according to the camera model of selecting.For fish eye lens camera, existed several universal models to select now, according to implementing convenience, can therefrom choose.
The necessity that camera inner parameter is demarcated: one, camera fluctuates near design load because fabrication error makes the Inside Parameter Value of the camera of final finished in production assembling process.If do not consider this fabrication error, only by design load, camera is not carried out to distortion correction, it proofreaies and correct result still can contain distortion composition.Two, different suppliers may adopt different camera model, if use its specific camera model can cause together with interdepending of camera and program code to each supply of material commercial city, thereby the independent design code of the camera that is necessary for each supplier, the versatility that is unfavorable for like this product, is also unfavorable for the maintenance work in product later stage.
The present invention is by using universal wide-angle imaging machine mathematical model, avoided the dependence to the specific mathematical model of the camera of particular vendors particular batch, thereby can when coding code, not need to work out specific calibrating procedure for the specific mathematical model to the camera of particular vendors particular batch.
The external parameter of camera comprises that calibration mark article coordinate is CS
gin camera coordinates, be CS
cin position (X
gc, Y
gc, Z
gc) and the anglec of rotation (rx
gc, ry
gc, rz
gc)
The present invention adjusts camera inner parameter by numerical solution algorithm and external parameter makes to proofread and correct by the unique point of mark in true three-dimension space minimum through the whole residual error of position coordinates between the unique point of the calibration mark thing after projective transformation and in camera institute pickup image.This numerical solution mode need to be asked for the coordinate of unique point measured on image with respect to the Jacobi matrix (Jacobian Matrix) of camera inside and outside parameter, then utilizes the linearities such as Kalman filtering (Kalman Filtering) or levenberg-marquardt algorithm or nonlinear optimization method to find the inside and outside parameter of one group of optimum.
The present invention for the calibration process of single camera is: operator chooses the unique point of two-piece update mark on original image, it is unique point (as shown in Figure 7, wherein 4 little square frames on mark summit are the unique points being selected) that every block mark thing is chosen its 4 summits; It is measurement data that calibration algorithm be take the coordinate of these 8 points, utilizes the inside and outside parameter of above-mentioned method of value solving calibration for cameras; Camera inside and outside parameter utilizes the camera inside and outside parameter of gained to carry out projective transformation to the mark unique point in three dimensions after demarcating and finishing, projected on original image, if the coordinate of the unique point that the coordinate in incident point and operator choose coincide, represent to demarcate correct, can exit calibration process, otherwise continue to demarcate.
Mode by virtual measurement joins in calibration frame by priori and uses by the mode of entering to demarcate and solve for measuring dimension and be equal to or less than the situation of demarcating dimension, is easy to occur by the coupling between calibrating parameters (coupling) problem with over-fitting (over-fitting).
Demarcation for single camera, be measured as 82 dimension points, i.e. 16 data, and the external parameter of required demarcation has 3 rotation parameters of 3 displacement parameters totally 6 parameters, the inner parameter of required demarcation also has 10 left and right according to above-mentioned to the elaboration of camera inner parameter, and the parameter that therefore altogether will demarcate has 16 left and right.For measuring dimension, be equal to or less than the situation of demarcating dimension, be easy to occur by the coupling between calibrating parameters (coupling) and over-fitting (over-fitting).This present invention is proposed to following calibration strategy: increase priori with by entering to demarcate.Increase priori refers to following priori is joined in calibration process: the horizontal focal distance f x of camera approaches with longitudinal focal distance f y numerical value, the position cx of camera lens optical axis on sensitive chip, and cy should be near the center of image.The present invention sees through virtual measurement technique above-mentioned priori is joined in calibration process.So-called virtual measurement refers to increases by three measuring amount fy, cx, and cy, its measured value is respectively fx, ImageWidth/2, ImageHeight/2, the horizontal pixel number that wherein ImageWidth is camera image, longitudinal pixel count that ImageHeight is camera image.So-called by entering to demarcate to refer to that the present invention does in batches and demarcates all parameters, but as shown in Figure 8 calibrating parameters grouping is demarcated every group of parameter by entering, when demarcating a certain group of parameter, other parameters keep original value constant.This carries out by entering calibration process circulation, until the residual error of unique point is less than some threshold values or cycle index exceeds some threshold values on mark.If residual error exceeds a certain threshold value after exiting circulation, the present invention can initiate a message and require it again to choose measurement point again to demarcate to operator.
Figure 9 shows that according to the inventive method single camera is demarcated and finished to utilize afterwards the result of demarcation that mark unique point is projected to the effect on image.The residual error that Fig. 9 demonstrates mark unique point is intuitively very little.
Set forth vehicle axis system, the method for world coordinate system and camera coordinate system position relationship determined below.
Utilize above-mentioned single camera scaling method just can utilize projective transformation technology will proofread and correct mark coordinate system CS after complete to every camera calibration
gany point in middle three dimensions projects the image coordinate system CS of camera
imgin.For above-mentioned vehicle-mounted looking around for supervisory system, its target is to find vehicle axis system CS
vin three dimensions in any point and the image coordinate system CS of camera
imgin the corresponding relation of respective point.The present invention determines this corresponding relation by following method.
Selection is by world coordinate system CS
wthe place, right front angle that is positioned over left back calibration mark thing, its attitude is as shown in figure 10; At single camera timing signal, select camera CAM
lcalibration mark article coordinate be CS
gLthe place, right back angle that is positioned over left back calibration mark thing, its attitude is as shown in figure 10; At single camera timing signal, select camera CAM
fcalibration mark article coordinate be CS
gFthe left rear corner place that is positioned over left front calibration mark thing, its attitude is as shown in figure 10; At single camera timing signal, select camera CAM
rcalibration mark article coordinate be CS
gRthe left-front corner place that is positioned over right front calibration mark thing, its attitude is as shown in figure 10; At single camera timing signal, select camera CAM
bcalibration mark article coordinate be CS
gBthe place, right front angle that is positioned over right back calibration mark thing, its attitude as shown in figure 10.
Place in the manner described above world coordinate system and each calibration mark article coordinate system, because size and the relative position relation of each calibration mark thing were both known, can try to achieve calibration mark article coordinate from these information and tie up to position and the attitude world coordinate system, be i.e. position and the attitude (R of front end camera calibration mark coordinate system in world coordinate system
gFw, T
gFw), position and the attitude (R of right-hand member camera calibration mark coordinate system in world coordinate system
gRw, T
gRw), position and the attitude (R of rear end camera calibration mark coordinate system in world coordinate system
gBw, T
gBw), position and the attitude (R of left end camera calibration mark coordinate system in world coordinate system
gLw, T
gLw).With (R
gFw, T
gFw) be example, wherein R
gFwfor rotation matrix, T
gFwfor motion vector.Suppose certain some front end camera calibration mark coordinate system CS
gFcoordinate be (X
gF, Y
gF, Z
gF), the coordinate in world coordinate system is (X
w, Y
w, Z
w),
Place in the manner described above world coordinate system and each calibration mark article coordinate system, because size and the relative position relation of each calibration mark thing were both known, can from these information also can be in the hope of world coordinate system position and the attitude calibration mark article coordinate system, world coordinates ties up to position and the attitude (R in front end camera calibration mark coordinate system
wgF, T
wgF), world coordinates ties up to position and the attitude (R in right-hand member camera calibration mark coordinate system
wgR, T
wgR), world coordinates ties up to position and the attitude (R in the camera calibration mark coordinate system of rear end
wgB, T
wgB), world coordinates ties up to position and the attitude (R in left end camera calibration mark coordinate system
wgL, T
wgL).
With (R
wgF, T
wgF) be example, wherein R
wgFfor rotation matrix, T
wgFfor motion vector.Suppose certain some front end camera calibration mark coordinate system CS
gFcoordinate be (X
gF, Y
gF, Z
gF), the coordinate in world coordinate system is (X
w, Y
w, Z
w),
After complete to single camera demarcation, calibration mark article coordinate is CS
gin camera coordinates, be CS
camin position and attitude can calculate by external parameter, position and the attitude of front end mark coordinate system in front end camera coordinates system is (R
gFc, T
gFc), position and attitude that right end marker article coordinate ties up in right-hand member camera coordinates system are (R
gRc, T
gRc), position and the attitude of rear end mark coordinate system in rear end camera coordinates system is (R
gBc, T
gBc), position and attitude that left end marker article coordinate ties up in left end camera coordinates system are (R
gLc, T
gLc).With (R
gFc, T
gFc) be example, suppose that certain is a bit at front end camera calibration mark coordinate system CS
gFcoordinate be (X
gF, Y
gF, Z
gF), in front end camera coordinates, be CS
camFin coordinate be (X
cF, Y
cF, Z
cF),
Calculating to obtain (R
gFc, T
gFc), (R
gRc, T
gRc), (R
gBc, T
gBc), (R
gLc, T
gLc) can to try to achieve camera coordinates according to above-mentioned formula be afterwards CS
camat its calibration mark article coordinate, be CS
gin position and attitude, front end camera coordinates ties up to position in front end mark coordinate system and attitude is (R
cgF, T
cgF), position and attitude that right-hand member camera coordinates ties up in right end marker article coordinate system are (R
cgR, T
cgR), position and attitude that rear end camera coordinates ties up in the mark coordinate system of rear end are (R
cgB, T
cgB), position and attitude that left end camera coordinates ties up in left end marker article coordinate system are (R
cgL, T
cgL).With (R
cgF, T
cgF) be example, its computing method are:
With (R
cgF, T
cgF) be example, suppose that certain is a bit at front end camera calibration mark coordinate system CS
gFcoordinate be (X
gF, Y
gF, Z
gF), in front end camera coordinates, be CS
camFin coordinate be (X
cF, Y
cF, Z
cF),
Therefore as shown in figure 10, the coordinate system of each camera self is positioned at the former of camera lens optical axis to be pointed out, and gets the value that initial point (0,0,0) in camera coordinates system converts thereof into world coordinate system, can draw the position of camera in world coordinate system.Take front end camera as example, and the computation process of its position in world coordinate system is:
Be that the position of front end camera in world coordinate system is.It is that the position of left end camera in world coordinate system is that same method can calculate the position of the position Wei， rear end camera of right-hand member camera in world coordinate system in world coordinate system.
Method in line computation car body size.Due to camera be assemblied on car body and also its rigging position design in advance, can be according to 4 cameras the position in world coordinate system determine the size of car body.Suppose that left and right camera is assemblied in car body left and right rearview mirror symmetric position herein, front end camera is assemblied in car body front end centre position, and rear end camera is assemblied in centre position, car body rear end, the width width of car body
vcan use the near distance of left and right camera seemingly, the length l ength of car body
vcan be approximate by the horizontal range of front and back camera,
Because the present invention uses the aforementioned method in line computation car body size, thereby make the present invention can be common to the vehicle (can demarcate with the present invention from the motor bus of the model cars to tens of tens centimetres meter) of various sizes, and needn't in program code, be the special size of vehicle set of each type.
By the car body size that aforementioned online car body size computing method has obtained can calculating according to this after car body size, determine viewing system scope to display region, rather than with fixing numerical value, viewing area is locked.
The size of car body icon while showing with the car body size ring viewing system that aforementioned online car body size computing method has obtained can calculating according to this after car body size, rather than in program code according to the size of the different prior anchor icon of type of vehicle.
The online relation of determining vehicle axis system and world coordinate system.Due to camera be assemblied on car body and also its rigging position design in advance, can determine the orientation of vehicle axis system in world coordinate system in the position in world coordinate system according to 4 cameras.Enumerate an example determining vehicle axis system according to this principle below.
The attitude of vehicle axis system is determined with the following method: vehicle axis system CS
vx-y plane be chosen in the surface level that vehicle is parked, straight up, y-direction of principal axis is parallel with car body longitudinal direction for z-axle.The longitudinal direction of car body can be according to being assemblied in 4 cameras all around on car body location positioning in world coordinate system.The axial angle of y-of car body longitudinal direction and world coordinate system is bodywork reference frame CS
vz-axle with respect to world coordinate system CS
wthe anglec of rotation of z-axle.Example is selected to determine with the position of left and right camera herein:
Vehicle axis system CS
vat world coordinate system CS
win attitude R
vwfor:
Vehicle axis system CS
vat world coordinate system CS
win position T
vw, select to be in this example positioned over front end camera and go out in the projection of surface level,
, vehicle axis system CS
vat world coordinate system CS
wposition and attitude be (R
vw, Y
vw).Suppose that certain is a bit at vehicle axis system CS
vcoordinate be (X
v, Y
v, Z
v), at world coordinate system CS
win coordinate be (X
w, Y
w, Z
w),
Utilize above-mentioned coordinate conversion process can try to achieve point in vehicle axis system and the corresponding relation of the point in camera image coordinate system, then utilize viewpoint change technology by the dot generation panoramic view in vehicle axis system.
Claims (8)
1. a camera calculation method of parameters, is characterized in that, described method comprises: in a process, by less unique point, the inner parameter of camera and external parameter are demarcated complete completely.
2. the present invention is joined in calibration frame by priori and is used by the mode of entering to demarcate and solve for measurement dimension and be equal to or less than in the situation of demarcating dimension and be easy to generation by the problem of the coupling between calibrating parameters (coupling) and over-fitting (over-fitting) by the mode of virtual measurement.
3. method according to claim 1, is characterized in that, described method comprises:
Use universal wide-angle imaging machine mathematical model, avoided the dependence to the specific mathematical model of the camera of particular vendors particular batch, thereby can when coding code, not need to work out specific calibrating procedure for the specific mathematical model to the camera of particular vendors particular batch.
4. in a method for line computation car body size, it is characterized in that, described method comprises:
Due to camera be assemblied on car body and also its rigging position design in advance, can be according to 4 cameras the position in world coordinate system determine the size of car body.
5. suppose that left and right camera is assemblied in car body left and right rearview mirror symmetric position herein, front end camera is assemblied in car body front end centre position, rear end camera is assemblied in centre position, car body rear end, the width widthv of car body can be with the near distance of left and right camera seemingly, and the length l engthv of car body can be approximate by the horizontal range of front and back camera.
6. a method for online ring viewing system indication range, is characterized in that, described method comprises:
By the car body size that aforementioned online car body size computing method has obtained can calculating according to this after car body size, determine viewing system scope to display region, rather than with fixing numerical value, viewing area is locked.
7. online ring viewing system shows by a method for car body icon size, it is characterized in that, described method comprises:
The size of car body icon while showing with the car body size ring viewing system that aforementioned online car body size computing method has obtained can calculating according to this after car body size, rather than in program code according to the size of the different prior anchor icon of type of vehicle.
8. an online method of determining the relation of vehicle axis system and world coordinate system, is characterized in that, described method comprises:
Due to camera be assemblied on car body and also its rigging position design in advance, can determine the orientation of vehicle axis system in world coordinate system in the position in world coordinate system according to 4 cameras, at utmost relaxed the strict demand that vehicle and calibration mark object height precision are placed, vehicle can freely be parked within the specific limits.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210287115.4A CN103593836A (en) | 2012-08-14 | 2012-08-14 | A Camera parameter calculating method and a method for determining vehicle body posture with cameras |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210287115.4A CN103593836A (en) | 2012-08-14 | 2012-08-14 | A Camera parameter calculating method and a method for determining vehicle body posture with cameras |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103593836A true CN103593836A (en) | 2014-02-19 |
Family
ID=50083961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210287115.4A Pending CN103593836A (en) | 2012-08-14 | 2012-08-14 | A Camera parameter calculating method and a method for determining vehicle body posture with cameras |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103593836A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985118A (en) * | 2014-04-28 | 2014-08-13 | 无锡观智视觉科技有限公司 | Parameter calibration method for cameras of vehicle-mounted all-round view system |
| WO2015149781A1 (en) * | 2014-04-01 | 2015-10-08 | Conti Temic Microelectronic Gmbh | Device for calibrating a camera system |
| CN105608693A (en) * | 2015-12-18 | 2016-05-25 | 上海欧菲智能车联科技有限公司 | Vehicle-mounted panoramic around view calibration system and method |
| CN106707287A (en) * | 2016-12-23 | 2017-05-24 | 浙江大学 | Fish school quantity estimation method based on extended Kalman filtering combined with nearest neighbor clustering algorithm |
| CN106780630A (en) * | 2017-01-09 | 2017-05-31 | 上海商泰汽车信息系统有限公司 | Demarcate panel assembly, vehicle-mounted camera scaling method and device, system |
| CN107146255A (en) * | 2017-04-05 | 2017-09-08 | 纵目科技（上海）股份有限公司 | Panoramic picture error calibration method and device |
| CN107784627A (en) * | 2016-08-31 | 2018-03-09 | 车王电子(宁波)有限公司 | The method for building up of vehicle panorama image |
| CN107945136A (en) * | 2017-12-04 | 2018-04-20 | 深圳市路畅科技股份有限公司 | A kind of fish eye images antidote, system, equipment and computer-readable storage medium |
| CN108259879A (en) * | 2016-12-28 | 2018-07-06 | 丰田自动车株式会社 | Video generation device and image generating method |
| CN108564629A (en) * | 2018-03-23 | 2018-09-21 | 广州小鹏汽车科技有限公司 | A kind of scaling method and system of vehicle-mounted camera external parameter |
| CN108665501A (en) * | 2018-07-12 | 2018-10-16 | 合肥晟泰克汽车电子股份有限公司 | Automobile viewing system three-dimensional scaling scene and the scaling method for using the scene |
| CN109801339A (en) * | 2018-12-29 | 2019-05-24 | 百度在线网络技术（北京）有限公司 | Image processing method, device and storage medium |
| WO2019170066A1 (en) * | 2018-03-07 | 2019-09-12 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted camera extrinsic parameter determination method, apparatus, device and system |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN110969664A (en) * | 2018-09-30 | 2020-04-07 | 北京初速度科技有限公司 | Dynamic calibration method for external parameters of camera |
| CN111107309A (en) * | 2018-10-25 | 2020-05-05 | 现代摩比斯株式会社 | Apparatus and method for calibrating driver monitoring camera |
| CN112785653A (en) * | 2020-12-30 | 2021-05-11 | 中山联合汽车技术有限公司 | Vehicle-mounted camera attitude angle calibration method |
| CN116823681A (en) * | 2023-08-31 | 2023-09-29 | 尚特杰电力科技有限公司 | Method, device and system for correcting distortion of infrared image and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005258953A (en) * | 2004-03-12 | 2005-09-22 | Ntt Docomo Inc | Fish eye camera and calibration method in the fish eye camera |
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN101551907A (en) * | 2009-04-28 | 2009-10-07 | 浙江大学 | Method for multi-camera automated high-precision calibration |
| CN101763640A (en) * | 2009-12-31 | 2010-06-30 | 无锡易斯科电子技术有限公司 | Online calibration processing method for vehicle-mounted multi-view camera viewing system |
-
2012
- 2012-08-14 CN CN201210287115.4A patent/CN103593836A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005258953A (en) * | 2004-03-12 | 2005-09-22 | Ntt Docomo Inc | Fish eye camera and calibration method in the fish eye camera |
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN101551907A (en) * | 2009-04-28 | 2009-10-07 | 浙江大学 | Method for multi-camera automated high-precision calibration |
| CN101763640A (en) * | 2009-12-31 | 2010-06-30 | 无锡易斯科电子技术有限公司 | Online calibration processing method for vehicle-mounted multi-view camera viewing system |
Non-Patent Citations (1)
| Title |
|---|
| 丁鑫: "全景视觉泊车辅助系统研究", 《中国优秀硕士学位论文全文数据库（信息科技辑）》, 31 August 2010 (2010-08-31), pages 138 - 626 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015149781A1 (en) * | 2014-04-01 | 2015-10-08 | Conti Temic Microelectronic Gmbh | Device for calibrating a camera system |
| CN103985118A (en) * | 2014-04-28 | 2014-08-13 | 无锡观智视觉科技有限公司 | Parameter calibration method for cameras of vehicle-mounted all-round view system |
| CN105608693A (en) * | 2015-12-18 | 2016-05-25 | 上海欧菲智能车联科技有限公司 | Vehicle-mounted panoramic around view calibration system and method |
| CN105608693B (en) * | 2015-12-18 | 2018-08-28 | 上海欧菲智能车联科技有限公司 | The calibration system and method that vehicle-mounted panoramic is looked around |
| CN107784627A (en) * | 2016-08-31 | 2018-03-09 | 车王电子(宁波)有限公司 | The method for building up of vehicle panorama image |
| CN106707287A (en) * | 2016-12-23 | 2017-05-24 | 浙江大学 | Fish school quantity estimation method based on extended Kalman filtering combined with nearest neighbor clustering algorithm |
| CN108259879A (en) * | 2016-12-28 | 2018-07-06 | 丰田自动车株式会社 | Video generation device and image generating method |
| CN106780630A (en) * | 2017-01-09 | 2017-05-31 | 上海商泰汽车信息系统有限公司 | Demarcate panel assembly, vehicle-mounted camera scaling method and device, system |
| CN107146255A (en) * | 2017-04-05 | 2017-09-08 | 纵目科技（上海）股份有限公司 | Panoramic picture error calibration method and device |
| CN107945136A (en) * | 2017-12-04 | 2018-04-20 | 深圳市路畅科技股份有限公司 | A kind of fish eye images antidote, system, equipment and computer-readable storage medium |
| CN107945136B (en) * | 2017-12-04 | 2021-09-24 | 深圳市路畅科技股份有限公司 | Fisheye image correction method, fisheye image correction system, fisheye image correction equipment and computer storage medium |
| WO2019170066A1 (en) * | 2018-03-07 | 2019-09-12 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted camera extrinsic parameter determination method, apparatus, device and system |
| CN108564629A (en) * | 2018-03-23 | 2018-09-21 | 广州小鹏汽车科技有限公司 | A kind of scaling method and system of vehicle-mounted camera external parameter |
| CN108665501A (en) * | 2018-07-12 | 2018-10-16 | 合肥晟泰克汽车电子股份有限公司 | Automobile viewing system three-dimensional scaling scene and the scaling method for using the scene |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN110969664A (en) * | 2018-09-30 | 2020-04-07 | 北京初速度科技有限公司 | Dynamic calibration method for external parameters of camera |
| CN110969664B (en) * | 2018-09-30 | 2023-10-24 | 北京魔门塔科技有限公司 | Dynamic calibration method for external parameters of camera |
| CN111107309A (en) * | 2018-10-25 | 2020-05-05 | 现代摩比斯株式会社 | Apparatus and method for calibrating driver monitoring camera |
| CN109801339A (en) * | 2018-12-29 | 2019-05-24 | 百度在线网络技术（北京）有限公司 | Image processing method, device and storage medium |
| CN112785653A (en) * | 2020-12-30 | 2021-05-11 | 中山联合汽车技术有限公司 | Vehicle-mounted camera attitude angle calibration method |
| CN116823681A (en) * | 2023-08-31 | 2023-09-29 | 尚特杰电力科技有限公司 | Method, device and system for correcting distortion of infrared image and storage medium |
| CN116823681B (en) * | 2023-08-31 | 2024-01-26 | 尚特杰电力科技有限公司 | Method, device and system for correcting distortion of infrared image and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103593836A (en) | A Camera parameter calculating method and a method for determining vehicle body posture with cameras | |
| US10659677B2 (en) | Camera parameter set calculation apparatus, camera parameter set calculation method, and recording medium | |
| JP7073315B2 (en) | Vehicles, vehicle positioning systems, and vehicle positioning methods | |
| KR101265012B1 (en) | Calibration device, method, and program for onboard camera | |
| JP5588812B2 (en) | Image processing apparatus and imaging apparatus using the same | |
| CN101976460B (en) | Generating method of virtual view image of surveying system of vehicular multi-lens camera | |
| CN101763640B (en) | Online calibration processing method for vehicle-mounted multi-view camera viewing system | |
| CN109360245A (en) | The external parameters calibration method of automatic driving vehicle multicamera system | |
| KR102516326B1 (en) | Camera extrinsic parameters estimation from image lines | |
| CN103985118A (en) | Parameter calibration method for cameras of vehicle-mounted all-round view system | |
| CN108444390A (en) | A kind of pilotless automobile obstacle recognition method and device | |
| WO2010113672A1 (en) | Calibration indicator used for calibration of onboard camera, calibration method of onboard camera using calibration indicator, and program for calibration device of onboard camera using calibration indicator | |
| JP2004086779A (en) | Obstacle detection device and its method | |
| CN102982526A (en) | Method of calibrating a vehicle vision system and vehicle vision system | |
| CN103871071A (en) | Method for camera external reference calibration for panoramic parking system | |
| CN112180373A (en) | Multi-sensor fusion intelligent parking system and method | |
| KR101583663B1 (en) | Method for generating calibration indicator of camera for vehicle | |
| CN110060298B (en) | Image-based vehicle position and posture determining system and corresponding method | |
| CN105551020A (en) | Method and device for detecting dimensions of target object | |
| CN102538766A (en) | Obstacle test method for active intelligent vehicle | |
| CN111435540A (en) | Annular view splicing method of vehicle-mounted annular view system | |
| KR20150051388A (en) | Stereo Camera Rectification System | |
| CN202608712U (en) | Stereoscopic vision based active grating projection sensor system | |
| CN109895697B (en) | Driving auxiliary prompting system and method | |
| US20220132049A1 (en) | Systems and method for image normalization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140219 |