CN104019829A - Vehicle-mounted panorama camera based on POS (position and orientation system) and external parameter calibrating method of linear array laser scanner - Google Patents

Abstract

The invention discloses a vehicle-mounted panorama camera based on a POS (position and orientation system) and an external parameter calibrating method of a linear array laser scanner. The external parameter calibrating method comprises the following steps: (1) establishing an outdoor calibrating field with absolute coordinate points in advance, wherein data acquisition is performed in the field, time for obtaining each group of panorama images and each laser-point cloud is obtained according to time information after acquisition, and the position and attitude information of the POS in each moment is obtained according to the high-density position and attitude information obtained by the POS; (2) resolving external parameters of the panorama camera; (3) resolving external parameters of the linear array laser scanner. The external parameter calibrating method has the beneficial effects that when a linear array laser sensor is calibrated, a large quantity of feature points do not need to be measured in advance, and only repeatable measurement features of the images are utilized, so as to reduce the workload to the minimum. More importantly, the panorama images determined by an algorithm principle and the point cloud scanned by a linear array laser can be accurately rectified after calibration.

Description

A kind of vehicle-mounted panoramic camera based on POS system and the outer ginseng scaling method of linear array laser scanner

Technical field

The present invention relates to a kind of vehicle-mounted panoramic camera based on POS system and the outer ginseng scaling method of linear array laser scanner, can be used for setting up the spatial relationship between three-dimensional laser point cloud and full-view image, belong to transducer calibration method and technology field.

Background technology

The captured full-view image of panorama camera can provide surface level 360 degree, and the visual range of vertical 180 degree, is a kind of image capturing mode that can obtain maximum photographic intelligence in single position.

Linear array laser scanner is due to itself radiation feature, the demarcation with respect to the outer ginseng of other reference frames becomes a very difficult thing to it to make independent static state, in most cases have to demarcate by accurate expensive electronics turntable and huge experiment internal field, economic benefit is not outstanding.

Above-mentioned two kinds of sensors and POS system are carried out to combined calibrating, can obtain the coordinate (conventionally this coordinate reference system by POS system determined, generally under the prerequisite that there is no statement refer to WGS-84 coordinate system) of each data source under unified coordinate reference system.Thereby make various number spaces be integrated into possibility according to the degree of depth between source.

Summary of the invention

Problem to be solved by this invention is the problem in associating POS system timing signal equipment needed thereby instrument costliness for current panorama camera and linear array laser scanner, and a kind of vehicle-mounted panoramic camera based on POS system and the outer ginseng scaling method of linear array laser scanner are provided.

The present invention solves the technical scheme that its technical matters takes:

An outer ginseng scaling method for vehicle-mounted panoramic camera based on POS system and linear array laser scanner, comprises the following steps:

(1) set up in advance the outdoor Calibration Field with absolute coordinates point (WGS-84 coordinate system), in this place, carry out data acquisition, collection finishes rear according to temporal information, time when obtaining each group full-view image and each laser point cloud and obtaining, the high density position and attitude information of obtaining by POS system, obtains above-mentioned each constantly position and attitude information of POS system;

(2) calculate the outer ginseng of panorama camera: by known point position in space, utilize imaging center point, picture point, object space point three point on a straight line principle, by obtaining POS system according to the moment of obtaining this Zhang Quanjing image in position and the attitude in this moment and processing to put together, be called one group of data; By multi-group data (the minimum 6 groups) Simultaneous Equations of putting together A, just can solve panorama camera with respect to the outer ginseng of POS system like this;

(3) resolve the outer ginseng of linear array laser scanner: demarcating on the basis of panorama camera, pick up simultaneously and be no less than same point on two picture points on image and laser point cloud, by these POS system position and attitude records constantly, be designated one group simultaneously; Several groups of data of multiselect (minimum 6 groups) like this, Simultaneous Equations B, solves linear array laser scanner with respect to the outer ginseng of POS system.

Preferably, described absolute coordinates is WGS-84 coordinate system.

Preferably, described system of equations A is:

$F(x,y,z,\mathrm{\φ},\mathrm{\ω},\mathrm{\κ})=\frac{m_1{m}_2+n_1{n}_2+p_1{p}_2}{\sqrt{m_1^2+n_1^2+p_1^2}\sqrt{m_2^2+n_2^2+p_2^2}}-1=0$

In formula, x, y, z representative sensor is with respect to the position coordinates of POS system, and φ, ω, κ distinguish representative sensor with respect to the attitude of POS system.M1, m2, n1, n2, p1, p2 is intermediate variable.

Preferably, described system of equations B is:

In formula, x, y, z representative sensor is with respect to the position coordinates of POS system, and φ, ω, κ distinguish representative sensor with respect to the attitude of POS system.M3, m4, n3, n4, p3, p4 is intermediate variable.

The invention has the beneficial effects as follows and make that we are unnecessary when demarcating linear array laser sensor first measures a large amount of unique points, and the characteristic that only need overlap with laser scanning point by the light path of image, workload is reduced to minimum, full-view image and a cloud that linear array laser is swept that what is more important is determined by algorithm principle can accuracy registrations after demarcation completes.

Additional aspect of the present invention and advantage in the following description part provide, and part will become obviously from the following description, or recognize by practice of the present invention.

Accompanying drawing explanation

Fig. 1 process flow diagram of the present invention;

The schematic diagram of Fig. 2 a full-view image definition of coordinate system in space;

Fig. 2 b is the schematic diagram after the panorama video of Fig. 2 a launches.

The schematic diagram of Fig. 3 linear array laser scanner definition of coordinate system in space.

Embodiment

Embodiment 1:

The implication of outer ginseng: for example, to described sensor (: panorama camera or linear array laser scanner) at a time, position (X with respect to a certain sensor (this patent middle finger POS system), Y, Z) three-dimensional and with respect to its attitude angle (φ, ω, κ), six such parameters, we are referred to as certain sensor with respect to the outer ginseng of another sensor.And due to the effect of POS system be measure this sensor (POS system itself) during operation the position under earth coordinates and attitude (for the ease of calculating, the earth coordinates described in after us refer to WGS-84 coordinate system.Position and the attitude of POS system output, we have been transformed in corresponding Gauss 3 degree band projected coordinate systems.Therefore the available achievement in this patent of the final output of POS system is Time (a certain moment), East (east is to coordinate), North (north is to coordinate), High (elevation), Yaw (course angle), Pitch (angle of pitch), Roll (roll angle)), we simply claim aforesaid " certain sensor is with respect to the outer ginseng of another sensor " for outer ginseng.

Vehicle-mounted panoramic camera according to the present embodiment a kind of based on POS system and the outer ginseng scaling method of linear array laser scanner, as shown in Figure 1, comprise the following steps:

(1) set up in advance the outdoor Calibration Field with absolute coordinates (WGS-84 coordinate system) point, institute's reconnaissance position should meet image and the easy identification part of point cloud chart---institute's reconnaissance position should be on full-view image can clear identification, normally set special sign target position, and this sign target position in this Calibration Field, should be unique effectively.In this place, carry out data acquisition----collection bag equipment: the collection car that has loaded panorama camera and linear array laser scanner and POS system, in car, also should comprise computer system for controlling the parameters of sensor and preserving the data of its collection, also should comprise power supply back-up system, for the said equipment provides electric power support.The packet gathering contains: the data of POS system (are time, position (X, Y, Z) three-dimensional and corresponding attitude angle (Yaw, Pitch, Roll)), full-view image (image data, and the shooting moment corresponding to each image), linear array laser data (original linear array laser data should comprise the moment of each laser spots data acquisition, and each puts corresponding distance, and each point is with respect to the angle of linear array laser sensor).Collection finishes rear according to temporal information, and the time when obtaining each group full-view image and each laser point cloud and obtaining, the high density position and attitude information of obtaining by POS system, obtains above-mentioned each position and attitude information of each sensor constantly.

(2) first calculate the outer ginseng of panorama camera, by known point position in space, utilize three point on a straight line principle (panorama unit ball theoretical center, picture point, object space point) according to the moment of obtaining this Zhang Quanjing image, obtain POS system in position and the attitude in this moment and process to put together being called one group of data.---the processing here can be regarded as varying due to POS system, the form of the achievement thing of exporting has a variety of, we are all converted to the form of its output a kind of process of specific format, be a normalized process, its final achievement thing should be Time (a certain moment), East (east is to coordinate), North (north is to coordinate), High (elevation), Yaw (course angle), Pitch (angle of pitch), Roll (roll angle).Like this multi-group data (minimum 6 groups) is put together and put into system of equations (A), just can solve the outer ginseng of panorama camera.

Described system of equations A is:

$F(x,y,z,\mathrm{\φ},\mathrm{\ω},\mathrm{\κ})=\frac{m_1{m}_2+n_1{n}_2+p_1{p}_2}{\sqrt{m_1^2+n_1^2+p_1^2}\sqrt{m_2^2+n_2^2+p_2^2}}-1=0$

In formula, detail parameters information please refer to the description of system of equations (A).

(3) because laser scanner fetched data is difficult for identification, so choose the atural object that laser and image are all easier to identification in software, demarcating on the basis of panorama camera, pick up simultaneously and be no less than same point on two picture points on image and laser point cloud, by these POS system position and attitude records constantly, be designated one group simultaneously.Several groups of data of multiselect (minimum 6 groups) like this, Simultaneous Equations (B), solves linear array laser scanner with respect to the outer ginseng of POS system.

System of equations B is:

In formula, detail parameters information please refer to the description of system of equations B.

About the description of full-view image, as shown in Fig. 2 a, Fig. 2 b.

Full-view image can be mapped to (this is the characteristic of full-view image, does not meet this characteristic, can not be referred to as full-view image) on a unit ball, and mentioned unit ball is that its radius of sphericity is 1 spheroid here, and unit is 1.Being deployed into plane, is exactly the image (this process is completed by panorama camera producer) that we collect.Its parametric description is: picture, and long X is wide=2d*d.Here stipulating to take the true origin that the upper left corner is this full-view image, is to the right u axle, is downwards v axle, the planar coordinate system that unit is pixel.On full-view image, each point has had unique describable coordinate like this.Continue like this coordinate system of definition unit ball, take the unit ball center of circle as initial point, point to corresponding 3/4 place that expands into picture after plane picture, vertical direction is that the direction of the point (3d/2, d/2) at d/2 place is X-axis; By centre of sphere initial point, point to the rear point (d/2, d/2) of spheroid expansion and locate as Y-axis; According to right-hand rule, direction is straight up Z axis.θ be the line in any point and the center of circle on sphere on X-Y plane with the angle of Y-axis, to the right for just, left for negative.φ be the line in any point and the center of circle on sphere in Y-Z plane with the angle of Y-axis, upwards for just, downwards for bearing.

About the description of linear array laser scanner as shown in Figure 3.Here take linear array laser scanner inner laser emission sensor place is initial point, to become the direction at 1/2 place of emission angle be X-axis, in the plane of departure, perpendicular to X-axis direction straight up, be Z axis, according to right-hand rule, direction is forward Y-axis.The angle (penetrating direction and X-axis institute angle) that is defined in the laser beam that X and Z axis positive dirction penetrate is being for just, otherwise is to bear.

The description of system of equations A:

Utilize impact point, picture point, photo centre's three point on a straight line, by the picture point on individual spherical panorama image, can list following collinearity equation:

${r_i}^m=r_{\mathrm{pos}}^m\left(t\right)+R_{\mathrm{pos}}^m\left(t\right)[R_c^{\mathrm{pos}}{r}_p+r_c]$

$r_p=\mathrm{\λ}\left[\begin{array}{c}\cos \mathrm{\φ}\sin \mathrm{\θ}\\ \cos \mathrm{\φ}\cos \mathrm{\θ}\\ \sin \mathrm{\φ}\end{array}\right]\begin{array}{c}\mathrm{\θ}=[u-\mathrm{INT}\left(\frac{2d-1}{2}\right)]\frac{\mathrm{\π}}{d}\\ \mathrm{\φ}=[\mathrm{INT}\left(\frac{d-1}{2}\right)-v]\frac{\mathrm{\π}}{d}\end{array}$

Wherein:

R _i ^mthe coordinate of target in earth coordinates is known quantity

the coordinate of POS system in earth coordinates is known quantity

pOS system is with respect to the rotation matrix of earth coordinates

panorama camera is with respect to the rotation matrix of POS system

R _ptarget, with respect to the spherical co-ordinate of panorama camera, can only show radiation direction

λ imaging engineer's scale

R _cpanorama camera is with respect to the displacement of GPS

U, v spherical panorama image coordinate

Half of width after d full-view image launches

$\begin{array}{c}{R^m}_{\mathrm{pos}}=\\ \left[\begin{array}{ccc}\cos \left(R\right)\cos \left(Y\right)+\sin \left(P\right)\sin \left(R\right)\sin \left(Y\right)& \cos \left(P\right)\sin \left(Y\right)& \cos \left(Y\right)\sin \left(R\right)-\cos \left(R\right)\sin \left(P\right)\sin \left(Y\right)\\ \cos \left(Y\right)\sin \left(P\right)\sin \left(R\right)-\cos \left(R\right)\sin \left(Y\right)& \cos \left(P\right)\cos \left(Y\right)& -\sin \left(R\right)\sin \left(Y\right)-\cos \left(R\right)\cos \left(Y\right)\sin \left(P\right)\\ -\cos \left(P\right)\sin \left(R\right)& \sin \left(P\right)& \cos \left(P\right)\cos \left(R\right)\end{array}\right]\end{array}$

Here special declaration: Yaw is abbreviated as to Y, and Roll is abbreviated as R, and Pitch is abbreviated as P

Above formula is actual, and what represent is the coordinate of target in earth coordinates and the relation of its image coordinate on spherical panorama image, and definite is position and the direction of taking a light on moment spherical panorama image.

The object of demarcating is to obtain camera shooting center with respect to the coordinate r of POS system _c(x, y, z) and camera coordinates system is with respect to the attitude of POS system they are that unknown number needs the outer ginseng solving, and known number is reference mark terrestrial coordinate r _i ^mpixel coordinate (u, v) with reference mark on spherical panorama image _i.

Its mathematical model is as follows:

1) according to the terrestrial coordinate r at reference mark _i ^m(X, Y, Z), intersection point A compute ray and unit ball ₁(x, y, z) ₁,

$r_{A1}=\frac{R_c^{-1\mathrm{pos}}[R_{\mathrm{pos}}^{-1m}\left(t\right)({r_i}^m-r_{\mathrm{pos}}^m\left(t\right))-r_c)]}{\mathrm{\λ}}$ λ＝|r _i ^m-r _c|

$\begin{array}{c}{R^{-1m}}_{\mathrm{pos}}={R^{\mathrm{Tm}}}_{\mathrm{pso}}=\\ \left[\begin{array}{ccc}\cos \left(R\right)\cos \left(Y\right)+\sin \left(P\right)\sin \left(R\right)\sin \left(Y\right)& \cos \left(Y\right)\sin \left(P\right)\sin \left(R\right)-\cos \left(R\right)\sin \left(Y\right)& -\cos \left(P\right)\sin \left(R\right)\\ \cos \left(P\right)\sin \left(Y\right)& \cos \left(P\right)\cos \left(Y\right)& \sin \left(P\right)\\ \cos \left(Y\right)\sin \left(R\right)-\cos \left(R\right)\sin \left(P\right)\sin \left(Y\right)& -\sin \left(R\right)\sin \left(Y\right)-\cos \left(R\right)\cos \left(Y\right)\sin \left(P\right)& \cos \left(P\right)\cos \left(R\right)\end{array}\right]\end{array}$

2) according to the pixel coordinate (u, v) at reference mark, intersection point A compute ray and unit ball ₂(x, y, z) ₂

$\mathrm{\θ}=[u-\mathrm{INT}\left(\frac{2d-1}{2}\right)]\frac{\mathrm{\π}}{d}$

3) between angle α order

$r_{A1}=\left[\begin{array}{c}{m}_1\\ n_1\\ p_1\end{array}\right],r_{A2}=\left[\begin{array}{c}{m}_2\\ n_2\\ p_2\end{array}\right]$

$\cos \mathrm{\α}=\frac{|m_1{m}_2+n_1{n}_2+p_1{p}_2|}{\sqrt{m_1^2+n_1^2+p_1^2}\sqrt{m_2^2+n_2^2+p_2^2}}$

When α approaches 0, cos α approaches 1, so final objective function, i.e. system of equations (A)

$F(x,y,z,\mathrm{\φ},\mathrm{\ω},\mathrm{\κ})=\frac{m_1{m}_2+n_1{n}_2+p_1{p}_2}{\sqrt{m_1^2+n_1^2+p_1^2}\sqrt{m_2^2+n_2^2+p_2^2}}-1=0$ ----------------------system of equations A

The description of system of equations (B):

Utilize the picture point on full-view image, the principle of the unique point that laser scanner is swept to and camera shooting center three point on a straight line, can obtain following relational expression:

${r_i}^m=r_{\mathrm{pos}}^m\left(t\right)+R_{\mathrm{pos}}^m\left(t\right)[R_c^{\mathrm{pos}}{r}_p+r_c]$ ---------full-view image observation equation

${r_i}^m=r_{\mathrm{pos}}^m\left(T\right)+R_{\mathrm{pos}}^m\left(T\right)[R_L^{\mathrm{pos}}{r}_{p2}+r_L]$ --------linear array laser observation equation

Symbol representative implication in above formula:

Wherein: $r_{p2}=\left[\begin{array}{c}L*\cos \left(\mathrm{\β}\right)\\ 0\\ L*\sin \left(\mathrm{\β}\right)\end{array}\right]$

R _i ^mthe coordinate of target in earth coordinates

the coordinate of POS system in earth coordinates

pOS system is with respect to the rotation matrix of earth coordinates

panorama camera is with respect to the rotation matrix of POS system

R _ptarget, with respect to the spherical co-ordinate of panorama camera, can only show radiation direction

R _cpanorama camera is with respect to the displacement of POS system

linear array laser is with respect to the rotation matrix of POS system, its expression formula of the amount that need to ask with unanimously

R _p2target is with respect to linear array laser coordinate

R _llinear array laser is with respect to the displacement of POS system, the amount that need to ask

β is laser beam transmit direction and X-axis angulation

L is the distance value that laser scanner observes

Wherein the parameter expression of full-view image observation equation illustrated, did not repeat here.

Above-mentioned equation is out of shape, and the system of equations form of distortion of easily being demarcated by panorama camera obtains similar process:

First we will obtain linear array laser analyzing spot and a certain the image of having selected corresponding point position

These two corresponding vectors of point of corresponding camera imaging center

Can be described as $r_{A3}=r_{\mathrm{pos}}^m\left(T\right)+R_{\mathrm{pos}}^m\left(T\right)[R_L^{\mathrm{pos}}{r}_{p2}+r_L]-[r_{\mathrm{pos}}^m\left(t\right)+R_{\mathrm{pos}}^m\left(t\right)r_c]$

Here expression formula with in full accord in form, do not repeat them here

The vector that same full-view image picture point becomes with imaging center

Can be described as

$\mathrm{\θ}=[u-\mathrm{INT}\left(\frac{2d-1}{2}\right)]\frac{\mathrm{\π}}{d}$

$\mathrm{\φ}=[\mathrm{INT}\left(\frac{d-1}{2}\right)-v]\frac{\mathrm{\π}}{d}$

$r_{A4}=\left[\begin{array}{c}\cos \mathrm{\φ}\sin \mathrm{\θ}\\ \cos \mathrm{\φ}\cos \mathrm{\θ}\\ \sin \mathrm{\φ}\end{array}\right]$ Be easy to form consistent conclusion with the demarcation of aforementioned full-view image, overlapping by two vectors on aggregating characteristic, can obtain two relations between vector should be as follows:

Same order $r_{A3}=\left[\begin{array}{c}{m}_3\\ n_3\\ p_3\end{array}\right],r_{A4}=\left[\begin{array}{c}{m}_4\\ n_4\\ p_4\end{array}\right]$

The pass of its vector angle χ is

$\cos \mathrm{\χ}=\frac{|m_3{m}_4+n_3{n}_4+p_3{p}_4|}{\sqrt{m_3^2+n_3^2+p_3^2}\sqrt{m_4^2+n_4^2+p_4^2}}$

When χ approaches 0, cos χ approaches 1, so final objective function, i.e. system of equations (B)

----------------------system of equations B

The demarcation of the panorama camera that Using such method is done and linear array laser scanner, its resultant error sees the following form:

Selecting LadyBug5 is to demarcate under 8192*4096 in resolution

Select SICK511-pro molded line battle array laser scanner to demarcate, its stated accuracy is:

Call the roll	Directions X error dX	Y-direction error dY	Z deflection error dZ	Panorama calibrated error
					VP001	0.012	0.056	0.136	0.147566934
VP002	0.012	0.112	0.005	0.11275194
					VP103	0.002	0.198	0.002	0.198020201
VP104	0.058	0.135	0.008	0.147149584
					VP105	0.069	0.256	0.009	0.265288522
VP106	0.025	0.22	0.005	0.221472346
					VP107	0.025	0.365	0.112	0.382614689
VP108	0.056	0.128	0.025	0.141933083
					VP209	0.179	0.198	0.089	0.281364532
VP210	0.165	0.145	0.025	0.22107691
					VP211	0.135	0.324	0.022	0.351688783
VP212	0.15	0.159	0.001	0.218590942
					VP213	0.11	0.248	0.005	0.271346642
VP214	0.158	0.265	0.008	0.308630847
					VP215	0.129	0.125	0.009	0.179852718
VP216	0.121	0.125	0.005	0.174043098
					VP217	0.1	0.004	0.125	0.160128074
VP218	0.023	0.119	0.018	0.122531629
					VP319	0.009	0.123	0.025	0.125837196
VP320	0.058	0.158	0.068	0.181526858
					VP321	0.052	0.121	0.152	0.201119368
VP322	0.158	0.125	0.123	0.236046606
					VP323	0.026	0.118	0.058	0.134029847
VP324	0.036	0.198	0.258	0.327206357
					VP425	0.125	0.147	0.153	0.246257995
VP426	0.018	0.159	0.121	0.200614057
					VP427	0.016	0.114	0.11	0.159223114
VP428	0.005	0.058	0.015	0.060116553
					VP529	0.056	0.009	0.005	0.056938563
VP530	0.025	0.002	0.002	0.025159491
					VP531	0.002	0.009	0.006	0.011
Average error	0.068225806	0.145903226	0.055	0.189391209
					Middle error	0.088854539	0.169553398	0.08441239	0.209210266

Claims (3)

1. an outer ginseng scaling method for the vehicle-mounted panoramic camera based on POS system and linear array laser scanner, is characterized in that, comprises the following steps:

(1) set up in advance the outdoor Calibration Field with absolute coordinates point, in this place, carry out data acquisition, collection finishes rear according to temporal information, time when obtaining each group full-view image and each laser point cloud and obtaining, the high density position and attitude information of obtaining by POS system, obtains above-mentioned each constantly position and attitude information of POS system;

(2) calculate the outer ginseng of panorama camera: by known point position in space, utilize imaging center point, picture point, object space point three point on a straight line principle Simultaneous Equations A can solve panorama camera with respect to the outer ginseng of POS system;

(3) resolve the outer ginseng of linear array laser scanner: demarcating on the basis of panorama camera, Simultaneous Equations B, solves linear array laser scanner with respect to the outer ginseng of POS system.

2. method according to claim 1, is characterized in that, described absolute coordinates is WGS-84 coordinate system.

3. method according to claim 2, is characterized in that, described system of equations A is:

$F(x,y,z,\mathrm{\φ},\mathrm{\ω},\mathrm{\κ})=\frac{m_1{m}_2+n_1{n}_2+p_1{p}_2}{\sqrt{m_1^2+n_1^2+p_1^2}\sqrt{m_2^2+n_2^2+p_2^2}}-1=0$

In formula, x, y, z representative sensor is with respect to the position coordinates of POS system, and φ, ω, κ distinguish representative sensor with respect to the attitude of POS system.M1, m2, n1, n2, p1, p2 is intermediate variable.

Method according to claim 3, is characterized in that, described system of equations B is:

In formula, x, y, z representative sensor is with respect to the position coordinates of POS system, and φ, ω, κ distinguish representative sensor with respect to the attitude of POS system.M3, m4, n3, n4, p3, p4 is intermediate variable.