CN104748728A - Intelligent machine attitude matrix calculating method and method applied to photogrammetry - Google Patents

Abstract

The invention discloses an intelligent machine attitude matrix calculating method and a method applied to photogrammetry. The intelligent machine attitude matrix calculating method mainly comprises the following steps that an intelligent machine measures the components ax, ay and az of gravitational acceleration at three directions x, y and z of the intelligent machine; the intelligent machine acquires the azimuth angle phi; and the attitude matrix Rg is calculated according to the description. According to the invention, the attitude of a camera on the intelligent machine can be obtained by the attitude matrix Rg of the intelligent machine, and the intelligent machine attitude matrix calculating method and the method applied to photogrammetry can be applied to the fields of photogrammetry, panorama measurement, computer vision, LBS service, virtual reality simulation, attitude measurement reinforcement of wearable equipment and the like.

Description

Intelligent machine attitude matrix computing method and be applied to photogrammetric method

Technical field

The present invention relates to areas of information technology, be specifically related to intelligent machine attitude matrix computing method and camera monocular, multi-vision visual localization method.

Background technology

Intelligent machine and camera attitude measurement significant, computer vision and photogrammetric, and Other Engineering is measured all in the urgent need to a kind of cheap, light its attitude of attitude measurement measuring apparatus.

Because intelligent machine attitude is different, the reality scene that same position is seen is different, the Accuracy navigation accuracy of intelligent machine attitude, photogrammetric accuracy, three-dimensional artificial precision and augmented reality information superposition precision, therefore the Accurate Measurement of intelligent machine attitude is significant.

Traditional intelligent machine attitude determination, mainly utilize the single shaft course angle data message of attitude sensor, system is placed with particular requirement to intelligent machine attitude, as required, intelligent machine is parallel to ground, or put perpendicular to ground, and the attitude of user's hand-held intelligent machine is ever-changing in augmented reality application, might not perfect parallelism or perpendicular to ground, this orientation that will cause navigating is forbidden.In addition the error of intelligent machine attitude determination will bring the error of information superposition, affect the experience of augmented reality user, therefore consider in the urgent need to a kind of the attitude determination algorithm that intelligent machine X-Y-Z tri-axle rotates.

Traditional exact posture detection technique is mainly used in aircraft, the main equipments such as automobile, pursues very high precision, for the field such as military, industrial, not only very heavy, and very valuable.And the present invention mainly considers to realize popular attitude determination by cheap intelligent machine terminal.

Summary of the invention

The technical problem to be solved in the present invention is not to be suitable for intelligent machine terminal for exact posture detection technique traditional in prior art, and the existing attitude determination method measuring error for intelligent machine terminal is large, the defect that Consumer's Experience is not good, one is provided to be applicable to common intelligent machine terminal, and the high attitude matrix computing method of measuring accuracy and camera monocular multi-vision visual localization method.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of intelligent machine attitude matrix computing method are provided, comprise the following steps:

By detecting three value { v of Gravity accelerometer on intelligent machine _ax', v _ay', v _az' calculate a _x, a _yand a _z:

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v_{\mathrm{ax}}}^{\′}\\ {v_{\mathrm{ay}}}^{\′}\\ {v_{\mathrm{az}}}^{\′}\end{array}\right],$

Or to { v _ax', v _ay', v _az' carry out filtering after obtain { v _ax, v _ay, v _az, calculate a _x, a _yand a _z:

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v}_{\mathrm{ax}}\\ v_{\mathrm{ay}}\\ v_{\mathrm{az}}\end{array}\right],$ Wherein R _athe pose transformation matrix of Gravity accelerometer relative to intelligent machine;

Position angle is obtained by intelligent machine magnetometer trigonometric function; Or direction sensor obtains position angle

Calculate attitude matrix

Wherein

$R_{\mathrm{\θ}}=\left[\begin{array}{ccc}\frac{{\mathrm{sa}}_z}{L\sqrt{1-{(a_y/L)}^2}}& -s{a}_x{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_x/L\\ 0& s\sqrt{1-{(a_y/L)}^2}& a_y/L\\ \frac{-{\mathrm{sa}}_x}{L\sqrt{1-{(a_y/L)}^2}}& -{\mathrm{sa}}_z{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_z/L\end{array}\right],L=\sqrt{a_x^2+a_y^2+a_z^2}$

Work as a _zduring <0, s=-1, a _zwhen>=0, s=1;

The attitude matrix R that formula (1) obtains _gfully describe the attitude of intelligent machine, attitude matrix R _gthe 1st row be the unit vector of intelligent machine x-axis under local coordinate system; 2nd row is the unit vector of intelligent machine y-axis under local coordinate system; 3rd row is the unit vector of intelligent machine z-axis under local coordinate system; Wherein local coordinate system be z-axis along gravity tester to outside the earth, x-axis y-axis of being exposed to the north is exposed to the west, or the coordinate system that x-axis is exposed to the north towards eastern y-axis.

The present invention also provides a kind of camera multi-vision visual localization method, comprises the following steps:

Step 1:

In N number of geographic position, point takes same object X, wherein N >=2, surveys the following amount of i-th observation point, wherein i ∈ (1,2 ..., N):

The position that (1) shows in video frame images (ui, vi);

(2) geographic position of shooting point the longitude of i-th shooting point respectively, latitude, highly;

(3) the attitude matrix R of i-th shooting point is recorded as stated above _i;

The long f of inner parameter focal length of (4) i-th shooting points _i, principal point coordinate (c _xi, c _yi);

Step 2:

Calculate the N number of attitude matrix relative to the earth's core, i=1 ~ N, R _0i=R _ir _vi

Wherein

Step 3:

Utilize formula obtain the geocentric coordinate system coordinate T of N number of shooting point camera photocentre _0i={ Txi, Fyi, Fzi} ^ti=1 ~ N; Wherein { Δ Txi, Δ Txi, Δ Tzi} ^t=R _vi ^tΔ T _i; Δ T _iit is the camera optical centre bias vector that method according to claim 6 calculates; by longitude, latitude, height conversion becomes the function of the X-coordinate of geocentric coordinate system; by longitude, latitude, height conversion becomes the function of the Y-coordinate of geocentric coordinate system; by longitude, latitude, height conversion becomes the function of the Z coordinate of geocentric coordinate system;

Step 4

According to the attitude matrix R of shooting point N _0N, position [T _xNt _yNt _zN], the position coordinates (u shown in video frame images _n, v _n), the long f of camera focal length pixel _n, principal point coordinate (c _xN, c _yN) all meet following equation

$u_i=c_{\mathrm{xi}}+f_i\frac{r_{0i11}(X_x-T_{\mathrm{xi}})+r_{0i12}(X_y-T_{\mathrm{yi}})+r_{0i13}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

$v_i=c_{\mathrm{yi}}+f_i\frac{r_{0i21}(X_x-T_{\mathrm{xi}})+r_{0i22}(X_y-T_{\mathrm{yi}})+r_{0i23}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

Wherein r _0iabrepresent attitude matrix R _0ithe element value of a capable b row; I=1 ~ N is row 2N equation altogether, separates (Xx, Xy, Xz) three unknown numbers; Or adopt the photogrammetric front method that crosses to calculate object X and obtain geocentric coordinate system coordinate (Xx, Xy, Xz).

The present invention also provides a kind of monocular single-point terrestrial positioning method, comprises step:

Step 1

The attitude matrix of camera is calculated according to said method $R=\mathrm{\&Delta;R}\&times;R_g=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right];$

Step 2

Camera capture video to be chosen on ground level a bit, (u v), obtains this long f of inner parameter focal length at camera in advance, records principal point coordinate (c to record its position shown in video frame images _x, c _y), this is as follows in the computing formula of the coordinate (X, Y) of local coordinate system:

Wherein h is camera height overhead;

Step 3

Local coordinate system coordinate (X, Y ,-h) is converted to longitude and latitude by Geodetic surveying method,

T _{the earth's core}=T _s+ R _v ^t([X, Y ,-h] ^t+ Δ T)

Wherein T _sintelligent machine geocentric coordinate,

Δ T is by the displacement vector of said method record intelligent machine camera photocentre relative to GPS anchor point.

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}, obtain T by geodesy formula _{the earth's core}corresponding longitude and latitude.

Present invention also offers the position of monocular two point measurement perpendicular to the object on ground and the method for height, comprise the following steps:

Step 1

1 image perpendicular to ground object is taken with intelligent machine camera, and following amount under measurements and calculations:

(1) the attitude matrix R=Δ R × R of camera is calculated according to said method _g

(2) position (u ⊥, v ⊥) that object intersection point on the ground shows in video frame images is measured

(3) position (u shown in video frame images perpendicular to the top of ground object is measured _h, v _h)

(4) the long fx of inner parameter focal length of camera is obtained, principal point coordinate (c _x, c _y);

(5) camera overhead height h is obtained;

Step 2

Calculate following amount

$z_{\&perp;}=\frac{-h}{r_{13}\frac{u_{\&perp;}-c_x}{f}+r_{23}\frac{v_{\&perp;}-c_y}{f}+r_{33}}$

$p_{\&perp;}=r_{11}\frac{u_{\&perp;}-c_x}{f}+r_{21}\frac{v_{\&perp;}-c_y}{f}+r_{31}$

$q_{\&perp;}=r_{12}\frac{u_{\&perp;}-c_x}{f}+r_{22}\frac{v_{\&perp;}-c_y}{f}+r_{32}$

$p_h=r_{11}\frac{u_h-c_x}{f}+r_{21}\frac{v_h-c_y}{f}+r_{31}$

$q_h=r_{12}\frac{u_h-c_x}{f}+r_{22}\frac{v_h-c_y}{f}+r_{32}$

$r_h=r_{13}\frac{u_h-c_x}{f}+r_{23}\frac{v_h-c_y}{f}+r_{33}$

Step 3

Solution is matrix equation below, obtains [X Y H]

$\left[\begin{array}{ccc}1& & \\ & 1& \\ 1& & \frac{-p_h}{r_h}\\ & 1& \frac{-q_h}{r_h}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ H\end{array}\right]=\left[\begin{array}{c}{z}_{\&perp;}{p}_{\&perp;}\\ z_{\&perp;}{q}_{\&perp;}\\ 0\\ 0\end{array}\right]$

Wherein (X, Y) is the coordinate of object under local coordinate system, and H is perpendicular to the height of ground object;

Step 4

The highest for object summit local coordinate system coordinate (X, Y, H) is converted to longitude and latitude by Geodetic surveying method; T _{the earth's core}=T _s+ R _v ^t([X, Y, H] ^t+ Δ T)

Wherein Ts is intelligent machine geocentric coordinate, and Δ T is the intelligent machine shooting that method according to claim 6 calculates

Head photocentre is relative to the displacement vector of GPS anchor point;

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}, obtain T by geodesy formula _{the earth's core}corresponding longitude and latitude.

Present invention also offers a kind of intelligent machine full-view image generation method, comprise the following steps:

Step 1, intelligent machine have taken n at a certain point of fixity by different attitude and open image, adopt the method for claim 6 to obtain attitude matrix and are respectively R _i, i is the integer between 0 ~ n-1, n be more than or equal to 1 integer;

Step 2, open up the full-view image of a wide w height h size;

Each pixel on step 3, scanning full-view image, each pixel (x, y) for panorama sketch carries out (1) (2) process as follows

(1) horizontal ordinate x converts to ordinate y converts θ to;

wherein w, h are the wide of panoramic picture, high;

(2) according to pinhole imaging system formula wherein $K_i=\left[\begin{array}{ccc}{f}_i& & c_{\mathrm{ix}}\\ & f_i& c_{\mathrm{iy}}\\ & & 1\end{array}\right]$ F _ifor camera i focal length length in pixels, (c _ix, c _iy) be the principal point of camera i, utilize R _iwill convert the picture coordinate (u being mapped to each camera i to _i, v _iif) (u _i, v _i) in image range, then by camera i as coordinate (u _i, v _i) taking-up of place's color, weighting is rendered into panorama sketch (x, y) place; Weight and (u _i, v _i) relevant to this image border bee-line, the shorter weight of distance is less; If (the u of none camera _i, v _i) in its image range, rendering transparent look or background colour are to (x, y) place;

Scan on panoramic picture and filled a little corresponding color, namely completed playing up of panoramic picture;

If R _imatrix is inaccurate, then by R _ias initial value, LM adjustment Algorithm model in computer vision subject is adopted to be optimized, minimum to ensure the grey value difference of overlapping region respective pixel;

Or in step 2 first by panoramic picture color value all clear 0, open image in step 3 taken turns by each for the respective area on panoramic picture picture element scan n for n, color value is accumulated in the pixel of panorama sketch by a point n wheel successively;

The three dimensional navigation of step 4 panoramic picture and retrieval location

(A) intelligent machine gesture stability browses three-dimensional panoramic image

Be observation point with the centre of sphere in virtual scene, sphere inner wall plays up a ball using panorama sketch as texture; Setting intelligent machine camera direction of observation { r ₃₁, r ₃₂r ₃₃, self upward direction {-r ₂₁,-r ₂₂,-r ₂₃; r _ijit is the i-th row jth column element in attitude R matrix;

(B) panorama sketch builds storehouse, retrieval and location

For effectively utilizing storage space, except the formula adopted in step 3 (1), go back available inks Kato conversion formula,

Convert panoramic picture to, keep building storehouse consistent with the conversion regime of retrieval;

Build storehouse

Extract the unique point set of panoramic picture, set up the characteristic point data storehouse associated with panoramic picture, in storehouse, each unique point points to the panoramic picture at its place;

Retrieval

Images uniting panoramic picture to be retrieved in c >=1, same place also extracts and plays up district's unique point, to each unique point, with the panoramic picture pointed by its most similar features point in inquiry storehouse, by unique point similarity degree, ballot is weighted to the panoramic picture pointed in storehouse, by the sequence of ballot summation size;

To play up district in panorama sketch to be retrieved and in the storehouse that obtains of voting, the set of pixels of panorama sketch corresponding region carries out relevant matches calculating, related coefficient is more large more similar; Export the result that the forward and related coefficient of ballot sequence is greater than preset value; If c=1, also can according to single shot attitude R, panoramic picture in storehouse is mapped to single shot image area and single shot image carries out relevant matches;

Location

In known GPS situation, the subset feature storehouse of the panoramic picture composition of retrieval GPS near zone;

Panoramic picture and its geographical position coordinates are bound together storage by server database, and result for retrieval is the image being numbered i*, can find out the geographic position corresponding to i* in a database.

The beneficial effect that the present invention produces is: the invention provides intelligent machine and camera attitude measurement method scheme thereof, user puts intelligent machine with any attitude in three dimensions can obtain its attitude matrix R _g, and obtain the attitude of camera on intelligent machine further, can be applicable to photogrammetric, computer vision, LBS serves, virtual reality emulation, augmented reality, the fields such as the attitude sensing of real scale game wearable device.

Further, the present invention not only considers the position angle that the value [0] of direction sensor obtains, there are being other sensors, as under acceleration of gravity magnetometer and gyroscope situation, also contemplate the data that other several attitude sensors capture, make the attitude that detects more accurate, do not having also can obtain rough attitude under these sensor condition.The attitude of intelligent machine superposes important role for full spectrum information.The present invention can at intelligent machine, and apple system realizes such as Andriod easily.

Accompanying drawing explanation

Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:

Fig. 1 is embodiment of the present invention R _gthe polar plot of matrix.

Embodiment

In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.

In the present invention, the implication of intelligent machine comprises the cell phone system of built-in camera, video, the camera system of embedded with network IP, the camera system of embedded with network IP, IPhone, panel computer, and the various system etc. simultaneously implanting video camera and network IP.

The imaging system that appearance determined by need in the present invention comprises the device that all that intelligent machine is equipped with have imaging function, module, and equipment, as camera, face battle array or line-scan digital camera, infrared camera, camera-laser 3D detecting devices, 3D camera, laser detection equipment, kinect equipment etc.

Intelligent machine is at the attitude matrix of local coordinate system $R_g=\left[\begin{array}{ccc}{r}_{g11}& r_{g12}& r_{g13}\\ r_{g21}& r_{g22}& r_{g23}\\ r_{g31}& r_{g32}& r_{g33}\end{array}\right]=\left[\begin{array}{c}{r}_{\mathrm{gx}}\\ r_{\mathrm{gy}}\\ r_{\mathrm{gz}}\end{array}\right],$ Intelligent machine is fixed on a certain device, the attitude matrix R that intelligent machine exports _gjust reflect the attitude of this device.

R _gmatrix is a 3x3 unit orthogonal matrix, is made up of three row, often row trivector, and three vectors are respectively r _gx, r _gy, r _gz, as shown in Figure 1, these three vectors are intelligent machine self three axle three unit vectors under local coordinate system.The first row r _gxthe unit vector of intelligent machine self x-axis under local coordinate system; Second row r _gythe unit vector of intelligent machine self y-axis under local coordinate system; The third line r _gzthe unit vector of intelligent machine self z-axis under local coordinate system.

Intelligent machine attitude matrix R _grelative to local coordinate system, local coordinate system Z axis is consistent with gravity direction, if intelligent machine magnetometer direction indication is north, then north represents local coordinate system X-axis, and west represents local coordinate system Y-axis; If intelligent machine magnetometer direction indication is east, then east represents local coordinate system X-axis, and north represents local coordinate system Y-axis; If intelligent machine magnetometer direction indication is south, then south represents local coordinate system X-axis, and east represents local coordinate system Y-axis; If intelligent machine magnetometer direction indication is west, then west represents local coordinate system X-axis, and south represents local coordinate system Y-axis.

Embodiments provide two kinds of intelligent machine attitude determination methods, method (1) needs accelerometer and magnetometer and gyroscope to coordinate.Method (2) utilizes rotating vector sensor to realize intelligent machine attitude determination; Really like this solve the omnibearing attitude determination of 3 axle, photogrammetric, GIS service, augmented reality emulates, and there is important application in the fields such as game.

Intelligent machine attitude matrix R _gone of following two kinds of methods can be adopted to obtain:

Method (1) adopts Gravity accelerometer and magnetometer

If intelligent machine has Gravity accelerometer, and there are magnetometer or direction sensor, calculate R with following formula (1) _g

Wherein

$R_{\mathrm{\&theta;}}=\left[\begin{array}{ccc}\frac{{\mathrm{sa}}_z}{L\sqrt{1-{(a_y/L)}^2}}& -s{a}_x{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_x/L\\ 0& s\sqrt{1-{(a_y/L)}^2}& a_y/L\\ \frac{-{\mathrm{sa}}_x}{L\sqrt{1-{(a_y/L)}^2}}& -{\mathrm{sa}}_z{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_z/L\end{array}\right]$

A _x, a _y, a _zcalculate by following methods:

By detecting three value { v of Gravity accelerometer on intelligent machine _ax', v _ay', v _az' calculate a _x, a _yand a _z;

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v_{\mathrm{ax}}}^{\&prime;}\\ {v_{\mathrm{ay}}}^{\&prime;}\\ {v_{\mathrm{az}}}^{\&prime;}\end{array}\right]$

Or to { v _ax', v _ay', v _az' carry out filtering after obtain { v _ax, v _ay, v _az, calculate a _x, a _yand a _z;

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v}_{\mathrm{ax}}\\ v_{\mathrm{ay}}\\ v_{\mathrm{az}}\end{array}\right]$

Wherein R _athe pose transformation matrix of Gravity accelerometer relative to intelligent machine;

According to filtering data, detect in filtering and be newly worth v _ax', v _ay', v _az' can adopt:

α v _ax'+(1-α) v _axreplace original v _ax

α v _ay'+(1-α) v _ayreplace original v _ay

α v _az'+(1-α) v _azreplace original v _az

α is a value between [0,1].

R _athe pose transformation matrix of gravity sensor relative to intelligent machine, for most of smart mobile phone,

$R_a=\left[\begin{array}{ccc}-1& & \\ & -1& \\ & & -1\end{array}\right].$

$L=\sqrt{a_x^2+a_y^2+a_z^2}$

Work as a _zduring <0, s=-1, a _zwhen>=0, s=1;

According to direction sensor: the direction sensor position angle output valve in intelligent machine, namely bring formula (1) into and calculate R _g; The 1st value that direction sensor exports is position angle, and the 2nd is the angle of pitch, and the 3rd is roll angle.

According to magnetometer sensor: magnetometric sensor vector value { v detected _mx', v _my', v _mz', calculate

$\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right]=R_m^T\left[\begin{array}{c}{v_{\mathrm{mx}}}^{\&prime;}\\ {v_{\mathrm{my}}}^{\&prime;}\\ {v_{\mathrm{mz}}}^{\&prime;}\end{array}\right];$

Or to { v _mx', v _my', v _mz' carry out filtering after obtain { v _mx, v _my, v _mz, calculate

$\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right]=R_m^T\left[\begin{array}{c}{v}_{\mathrm{mx}}\\ v_{\mathrm{my}}\\ v_{\mathrm{mz}}\end{array}\right]$

Wherein R _mthe pose transformation matrix of magnetometric sensor relative to intelligent machine;

Newly v is worth as detected in process _mx', v _my', v _mz' following methods can be adopted magnetometer signals filtering:

α _mv _mx'+(1-α _m) v _mxreplace original v _mx;

α _mv _my'+(1-α _m) v _myreplace original v _my;

α _mv _mz'+(1-α _m) v _mzreplace original v _mz;

α _mit is a value between [0,1].

And calculate

$\left[\begin{array}{c}{m}_{0x}\\ m_{0y}\\ m_{0z}\end{array}\right]=R_{\mathrm{\&theta;}}^T\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right]$

Utilize magnetometer computer azimuth angle trigonometric function be:

Bring formula (1) into and calculate R _g, recycle above-mentioned formula (1) and calculate attitude matrix R _g.

Above-mentioned intelligent machine attitude matrix R _ggyroscope can also be adopted to carry out iterative computation.First R has been calculated at formula (1) _gafterwards, if there is gyroscope, following methods iteration can be adopted to obtain R _gmatrix:

1st step setting q ₀, q ₁, q ₂, q ₃initial value

Formula (1) is selected to calculate R _ginitial value, then by R _gmatrix conversion becomes 4 element q ₀, q ₁, q ₂, q ₃;

By 4 yuan of number q ₀, q ₁, q ₂, q ₃as the initial value of following iterative computation;

2nd step setting exInt, eyInt, ezInt original value is 0

exInt=0，eyInt=0，ezInt=0；

3rd step is according to the magnetometer vector { m received _x, m _y, m _z, obtain correct magnetic vector { w _x, w _y, w _zfirst by vector { m _x, m _y, m _zreplace to its unitization later vector

And obtain vector { b as follows _x, 0, b _z}

h _x=2×m _x×(0.5-q ₂×q ₂-q ₃×q ₃)+2×my×(q ₁×q ₂-q ₀×q ₃)+2×m _z×(q ₁×q ₃+q ₀×q ₂);

h _y=2×m _x×(q ₁×q ₂+q ₀×q ₃)+2×m _y×(0.5-q ₁×q ₁-q ₃×q ₃)+2×m _z×(q ₂×q ₃-q ₀×q ₁);

h _z=2×m _x×(q ₁×q ₃-q ₀×q ₂)+2×m _y×(q ₂×q ₃+q ₀×q ₁)+2×m _z×(0.5-q ₁×q ₁-q ₂×q ₂);

$b_x=\sqrt{{h_x}^2+{h_y}^2};$

b _z=h _z

Be transformed into correct magnetic vector { w again _x, w _y, w _z}

w _x=2×b _x×(0.5-q ₂×q ₂-q ₃×q ₃)+2×b _z×(q ₁×q ₃-q ₀×q ₂);

w _y=2×b _x×(q ₁×q ₂-q ₀×q ₃)+2×b _z×(q ₀×q ₁+q ₂×q ₃);

w _z=2×b _x×(q ₀×q ₂+q ₁×q ₃)+2×b _z×(0.5-q ₁×q ₁-q ₂×q ₂);

4th step is according to the Gravity accelerometer data a received _x, a _y, a _z, and { w _x, w _y, w _zobtain error vector { e _x, e _y, e _zand calculate e _xint, e _yint, e _zint

First by vector { a _x, a _y, a _zreplace to its unitization later vector

v _x=2×q ₁×q ₃-q ₀×q ₂)；

v _y=2×(q ₀×q ₁+q ₂×q ₃)；

v _z=q ₀×q ₀-q ₁×q ₁-q ₂×q ₂+q ₃×q ₃；

e _x=(a _y×v _z-a _z×v _y)+(m _y×w _z-m _z×w _y)；

e _y=(a _z×v _x-a _x×v _z)+(m _z×w _x-m _x×w _z)；

e _z=(a _x×v _y-a _y×v _x)+(m _x×w _y-m _y×w _x)；

Error of calculation aggregate-value

E _xint replaces with e _xint+e _x× Ki;

E _yint replaces with e _yint+e _y× Ki;

E _zint replaces with e _zint+e _z× Ki;

Wherein Ki is an adjustable positive coefficient, and Ki chooses arbitrarily in 0.00001 to 0.5;

5th step is according to error vector { e _x, e _y, e _zand aggregate-value correction gyro data { g _x0, g _y0, g _z0suppose that intelligent machine reads one group of current gyro data for { g _x0, g _y0, g _z0}

g _x=g _x0+Kp×e _x+e _xInt；

g _y=g _y0+Kp×e _y+e _yInt；

g _z=g _z0+Kp×e _z+e _zInt；

Wherein Kp is an adjustable positive coefficient, and Kp chooses arbitrarily in 0.000001 to 20.0;

6th step is according to gyro data { g _x, g _y, g _zrevise hypercomplex number

Along with constantly receiving gyro data { g _x, g _y, g _z, revise as follows 4 yuan of numbers, halfT is the correction cycle, halfT=0.00001 ~ 10.0,

Q ₀replace with q ₀+ (-q ₁× g _x-q ₂× g _y– q ₃× g _z) × halfT;

Q ₁replace with q ₁+ (q ₀× g _x-q ₃× g _y+ q ₂× g _z) × halfT;

Q ₂replace with q ₂+ (q ₃× g _x+ q ₀× g _y-q ₁× g _z) × halfT;

Q ₃replace with q ₃+ (-q ₂× g _x+ q ₁× g _y+ q ₀× g _z) × halfT;

7th step exports R _gmatrix and hypercomplex number

By hypercomplex number { q ₀, q ₁, q ₂, q ₃unit changes into export

4 yuan of numbers turn R _gmatrix Formula is as follows

$R_g=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right]$

8th step is got back to the 3rd step and is continued to receive Motions of Gyroscope Data Update attitude 4 yuan of number q ₀~ q ₃, in the process of circulation, current R can be exported to the 7th step at every turn _gmatrix.

Method (2) uses rotating vector sensor

When intelligent machine has rotating vector sensor,

1. first adopt any one in (a) (b) (c) to obtain R _g0matrix

If a () detects that the rotating vector sensor on intelligent machine only has 3 data values [0], values [1], values [2]

Following formula is adopted to obtain

q ₁=values[0],q ₂=values[1],q ₃=values[2],

Convert q0, q1, q2, q3 to R as follows again _g0matrix

$R_{g0}=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right]$

If b rotating vector sensor that () is detected on intelligent machine has 4 data, values [0], values [1], values [2], values [3], make q ₀=values [3], q ₁=values [0], q ₂=values [1], q ₃=values [2].Or p ₀=values [3], p ₁=values [0], p ₂=values [1], p ₃=values [2]

Calculate

q ₀=p ₀/L _p

q ₁=p ₁/L _p

q ₂=p ₂/L _p

q ₃=p ₃/L _p

Wherein $L_p=\sqrt{{p_0}^2+{p_1}^2+{p_2}^2+{p_3}^2}$

Again by q ₀, q ₁, q ₂, q ₃convert R to as follows _g0matrix

$R_{g0}=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right]$

C on () intelligent machine, specified sensor type is rotating vector type TYPE_ROTATION_VECTOR, obtain the observed reading R of rotating vector sensor in real time _vecthe rotating vector in intelligent machine system is utilized to turn matrix function; By R _vecconvert drawing transition matrix R to ₄₄, R ₄₄upper left corner 3x3 matrix is exactly R _g0;

2., then according to the difference of intelligent machine rotating vector sensor coordinate system definition, take following several situation to obtain R _g:

If the unit matrix of rotating vector sensor definition is X-axis point to east, Y-axis points to north, then

$R_g=R_{g0}{\left[\begin{array}{ccc}& 1& \\ -1& & \\ & & 1\end{array}\right]}^T$

If the unit matrix of rotating vector sensor definition is X-axis point to north, Y-axis points to west, then

R _g=R _g0

If the unit matrix of rotating vector sensor definition is X-axis energized south, Y-axis points to east, then

$R_g=R_{g0}{\left[\begin{array}{ccc}-1& & \\ & -1& \\ & & 1\end{array}\right]}^T.$

In order to measure attitude matrix R intelligent machine being installed camera further, take following steps:

The attitude matrix R of intelligent machine is defined as $R=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$

Wherein R=Δ R × R _g; Δ R is camera attitude matrix R and the intelligent machine attitude matrix R of intelligent machine _gbetween the conversion in a fixing direction, be camera optical centre bias vector.When camera direction and intelligent machine screen orientation are just contrary $\mathrm{\&Delta;R}=\left[\begin{array}{ccc}1& & \\ & -1& \\ & & -1\end{array}\right],$ Camera direction and intelligent machine screen orientation completely the same time $\mathrm{\&Delta;R}=\left[\begin{array}{ccc}1& & \\ & 1& \\ & & 1\end{array}\right],$ Otherwise adopt ripe photogrammetric rear Convergence method Accurate Calibration Δ R.Method is as follows:

1. set up controlling filed coordinate system, namely initial point is positioned at the GPS acceptance point of intelligent machine or the local coordinate system of Big Dipper anchor point;

2. with the point set in intelligent machine camera shooting controlling filed coordinate system; Measure R during intelligent machine shooting point set as stated above _g;

3. concentrate according to controlling filed point 3 dimension coordinates of the controlling filed coordinate system of each known point and shooting image Point Set as the corresponding relation of coordinate, adopt photogrammetric resection method to measure the attitude matrix R of intelligent machine camera relative to controlling filed coordinate system _awith the coordinate Δ T of camera photocentre relative to controlling filed coordinate system;

4. obtain Δ R=R by following formula _ar _g ^-1.

Δ T is also that camera photocentre offsets relative to the local coordinate system of intelligent machine GPS or Big Dipper anchor point simultaneously.

Above-mentioned attitude matrix R can be used for camera multi-vision visual location.Suppose that geocentric coordinate system is with the earth centre of sphere for initial point, the earth centre of sphere is z-axis to the arctic, and the centre of sphere 0 degree to equator warp point is the right-handed coordinate system of x-axis.Suppose longitude, latitude, height conversion becomes the function of the X-coordinate of geocentric coordinate system; longitude, latitude, height conversion becomes the function of the Y-coordinate of geocentric coordinate system; be longitude, latitude, height conversion becomes the function of the Z coordinate of geocentric coordinate system, utilizes conversion formula classical in geodesy to calculate the geocentric coordinate system coordinate (Xx, Xy, Xz) of object X

$\left[\begin{array}{c}{X}_X\\ X_Y\\ X_Z\end{array}\right]=T_{0i}=\left[\begin{array}{c}{F}_x(h_i,{\mathrm{\&theta;}}_i,{\mathrm{\&phi;}}_i)\\ F_y(h_i,{\mathrm{\&theta;}}_i,{\mathrm{\&phi;}}_i)\\ F_z(h_i,{\mathrm{\&theta;}}_i,{\mathrm{\&phi;}}_i)\end{array}\right]=\left[\begin{array}{c}(N_i+h_i)\cos {\mathrm{\&theta;}}_i\cos {\mathrm{\&phi;}}_i\\ (N_i+h_i)\cos {\mathrm{\&theta;}}_i\sin {\mathrm{\&phi;}}_i\\ (N_i(1-e^2)+h_i)\sin {\mathrm{\&theta;}}_i\end{array}\right]$

Wherein

H _iit is point overhead height

n _iit is the radius of curvature in prime vertical with latitude change.

$e=\sqrt{a^2-b^2}/a$

A, b are the major and minor axis of earth ellipsoid, are the known constants in geodesy;

E is excentricity, is definite value.

The method of being carried out camera multi-vision visual location by attitude matrix R is comprised the following steps:

Step 1

In N(N >=2) individual geographic position point takes same object X, surveys the following amount of i-th (i=1 ~ N) individual observation point:

(1) i-th position coordinates that observation point shows in video frame images (ui, vi);

(2) geographic position of shooting point is , be the longitude of i-th shooting point respectively, latitude, highly;

(3) the attitude matrix R of i-th shooting point is recorded as stated above _i;

The long f of inner parameter focal length of (4) i-th shooting points _i, principal point coordinate (c _xi, c _yi), principal point coordinate computer vision and photogrammetric in all have definition, be not repeated herein.

Step 2

Obtain the N number of attitude matrix i=1 ~ N relative to the earth's core further

R _0i=R _iR _vi

Wherein

Step 3

Utilize formula

Obtain the geocentric coordinate system coordinate T of N number of shooting point camera photocentre _0i={ Txi, Fyi, Fzi} ^ti=1 ~ N is { Δ Txi, Δ Txi, Δ Tzi} wherein ^t=R _vi ^tΔ T _i

Δ T _iit is the camera optical centre bias vector recorded according to said method.

Step 4

According to

The attitude matrix R of shooting point 1 ₀₁position [T _x1t _y1t _z1] object is as coordinate u ₁, v ₁the long f of camera focal length pixel ₁, principal point coordinate (cx ₁, cy ₁)

……

The attitude matrix R of shooting point N _0Nposition [T _xNt _yNt _zN] object is as coordinate (u _n, v _n) the long f of camera focal length pixel _n, principal point coordinate (c _xN, c _yN), all meet following equation:

$u_i=c_{\mathrm{xi}}+f_i\frac{r_{0i11}(X_x-T_{\mathrm{xi}})+r_{0i12}(X_y-T_{\mathrm{yi}})+r_{0i13}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

$v_i=c_{\mathrm{yi}}+f_i\frac{r_{0i21}(X_x-T_{\mathrm{xi}})+r_{0i22}(X_y-T_{\mathrm{yi}})+r_{0i23}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

Wherein r _0iabrepresent attitude matrix R _0ithe element value of a capable b row.

I=1 ~ N is row 2N equation altogether, separates (X _x, X _y, X _z) three unknown numbers,

Or adopt the photogrammetric front method that crosses to calculate object X=(X _x, X _y, X _z)

Obtain geocentric coordinate system coordinate (X _x, X _y, X _z); Finally obtain longitude and latitude corresponding to geocentric coordinate (Xx, Xy, Xz) by geodesy formula.

If spheric approximation ball in ground can be calculated with following methods

Longitude=atan2 (X _y, X _x);

Latitude=asin

If the earth is approximated to ellipsoid, available geodesy has ellipsoid formula and calculates.Above-mentioned attitude matrix R also can be used for camera monocular single-point or monocular two TV feel location.Definable local coordinate system be initial point at intelligent machine photocentre, Z axis points to outside the earth along gravity direction or perpendicular to ellipsoid ground, and X-axis points to north, and Y-axis points to the coordinate system in west.

Location, monocular single-point ground mainly comprises the following steps:

Step 1

The attitude matrix of camera is calculated according to said method $R=\mathrm{\&Delta;R}\&times;R_g=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$

Step 2

Camera capture video to be chosen on ground level a bit, (u v), records this long f of inner parameter focal length at camera, records principal point coordinate (c to record its position coordinates shown in video frame images _x, c _y), principal point coordinate refers to camera sight line central shaft and the picpointed coordinate as Plane intersects.Computer vision and photogrammetric in have definition.(1) main line of sight that principal point mainly refers to and dioramic intersection point; (2) any point on the axis of lens in such 2, this relation of 2 be point to the light of a principal point from object any point will from the parallel injection of lens, and by another principal point; (3) lens axis of aerocamera and the intersection point in film face.It is exactly this principal point coordinate that this principal point on the axis of lens is converted to image slices coordinate.

In the embodiment of the present invention, principal point is camera sight line central shaft and the picpointed coordinate as Plane intersects, and be the picture coordinate that central optical camera axis converts in the point that lens intersect, on ground level, this is as follows in coordinate (X, the Y) computing formula of local coordinate system:

Wherein h is camera height overhead.

Step 3

Local coordinate system coordinate (X, Y ,-h) is converted to longitude and latitude by geodetic surveying maturation method.

T _{the earth's core}=T _s+ R _v ^t([X, Y ,-h] ^t+ Δ T)

Wherein T _sintelligent machine geocentric coordinate,

Δ T is by the displacement vector of said method record intelligent machine camera photocentre relative to GPS anchor point.

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}=[A, B, C], obtains T by geodesy formula _{the earth's core}corresponding longitude and latitude.

The earth is approximately a ball, and specific implementation method is:

Longitude=atan2 (B, A);

Latitude=asin (C/ (earth radius-h));

Highly=-h;

Monocular two point is measured and is comprised the following steps perpendicular to the position of the object on ground and the method for height:

Step 1

Take 1 image perpendicular to ground object with intelligent machine camera, record following amount

(1) with the attitude matrix R=Δ R × R of said method record camera _g

(2) position coordinates (u ⊥, v ⊥) that object intersection point on the ground shows in capture video two field picture is recorded

(3) position coordinates (u that shows in capture video two field picture perpendicular to the top of ground object of record _h, v _h)

(4) the long fx of inner parameter focal length pixel of camera is obtained, principal point coordinate (c _x, c _y);

(5) camera overhead height h is obtained;

Step 2

Calculate following amount

$z_{\&perp;}=\frac{-h}{r_{13}\frac{u_{\&perp;}-c_x}{f}+r_{23}\frac{v_{\&perp;}-c_y}{f}+r_{33}}$

$p_{\&perp;}=r_{11}\frac{u_{\&perp;}-c_x}{f}+r_{21}\frac{v_{\&perp;}-c_y}{f}+r_{31}$

$q_{\&perp;}=r_{12}\frac{u_{\&perp;}-c_x}{f}+r_{22}\frac{v_{\&perp;}-c_y}{f}+r_{32}$

$p_h=r_{11}\frac{u_h-c_x}{f}+r_{21}\frac{v_h-c_y}{f}+r_{31}$

$q_h=r_{12}\frac{u_h-c_x}{f}+r_{22}\frac{v_h-c_y}{f}+r_{32}$

$r_h=r_{13}\frac{u_h-c_x}{f}+r_{23}\frac{v_h-c_y}{f}+r_{33}$

Step 3

Solution is matrix equation below, obtains [X Y H]

$\left[\begin{array}{ccc}1& & \\ & 1& \\ 1& & \frac{-p_h}{r_h}\\ & 1& \frac{-q_h}{r_h}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ H\end{array}\right]=\left[\begin{array}{c}{z}_{\&perp;}{p}_{\&perp;}\\ z_{\&perp;}{q}_{\&perp;}\\ 0\\ 0\end{array}\right]$

Wherein (X, Y) is the coordinate of object under local coordinate system, and H is perpendicular to the height of ground object.

Step 4

By geodetic surveying maturation method, the highest for object summit local coordinate system coordinate (X, Y, H) is changed into longitude and latitude.

T _{the earth's core}=T _s+ R _v ^t([X, Y, H] ^t+ Δ T)

Wherein Ts is intelligent machine geocentric coordinate,

Δ T is camera photocentre relative to the displacement vector of intelligent machine GPS or Big Dipper anchor point.Approximate desirable T=[000]

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}=[A, B, C], obtains T by geodesy formula _{the earth's core}corresponding longitude and latitude.

The earth is approximately a ball, and specific implementation method is:

Longitude=atan2 (B, A);

Latitude=asin (C/ (earth radius+H));

Highly=H.

The intelligent machine full-view image generation method of the embodiment of the present invention comprises the following steps:

Step 1. intelligent machine have taken n at a certain point of fixity by different attitude and opens image, adopts the method for claim 6 to obtain attitude matrix and is respectively R _i, i is the integer between 0 ~ n-1, n be greater than 1 integer;

Step 2. opens up the full-view image of a wide w height h size;

Step 3. scans each pixel on full-view image, and each pixel (x, y) for panorama sketch carries out (1) (2) process as follows

(1) horizontal ordinate x converts to ordinate y converts θ to;

wherein w, h are the wide of panoramic picture, high;

(2) according to pinhole imaging system formula foundation pinhole imaging system formula

Wherein $K_i=\left[\begin{array}{ccc}{f}_i& & c_{\mathrm{ix}}\\ & f_i& c_{\mathrm{iy}}\\ & & 1\end{array}\right]$ F _ifor camera i focal length length in pixels, (c _ix, c _iy) be the principal point of camera i, utilize R _iwill convert the picture coordinate (u being mapped to each camera i to _i, v _iif) (u _i, v _i) in image range, then by camera i as coordinate (u _i, v _i) taking-up of place's color, weighting is rendered into panorama sketch (x, y) place; Weight and (u _i, v _i) relevant to this image border bee-line, the shorter weight of distance is less; If (the u of none camera _i, v _i) in its image range, rendering transparent look or background colour are to (x, y) place; Scan on panoramic picture and filled a little corresponding color, namely completed playing up of panoramic picture;

Be specially according to formula

i=0～n-1

Wherein $K_i=\left[\begin{array}{ccc}{f}_i& & c_{\mathrm{ix}}\\ & f_i& c_{\mathrm{iy}}\\ & & 1\end{array}\right]$ F _ifor camera i focal length length in pixels, (c _ix, c _iy) be the principal point coordinate of camera i

Will convert (u to ₀, v ₀..., u _n-1, v _n-1);

According to (u ₀, v ₀.., u _n-1, v _n-1) this n group is mapped in coordinate on n shooting angle image, can get:

N opens the color C that image takes out ₀~ C _n-1;

N as the distance e of coordinate to photo frame edge ₀~ e _n-1;

Ask each the weight w of image at this pixel place further _i=e _i ²/ e _s ²

Wherein ${e_s}^2=\underset{i=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{e}_i^2$

On panoramic picture, the color value played up at pixel (x, y) place is e _sduring >0, play up otherwise play up a Transparent color or background colour (x, y) place to panorama sketch;

Scan on panoramic picture and filled a little corresponding color, namely completed playing up of panoramic picture;

If step 1 terminates later R _imatrix is inaccurate, then by R _ior convert the result of hypercomplex number to as initial value, adopt minimum 2 multiplication of LM to be optimized, minimum to ensure the grey value difference of overlapping region respective pixel;

Or first by panoramic picture color value all clear 0, then open image for n and taken turns by each for the respective area on panoramic picture picture element scan n, color value is accumulated in the pixel of panorama sketch by a point n wheel successively.So once only load an image, memory consumption can be avoided excessive.

The three dimensional navigation of step 4 panoramic picture and retrieval location

(A) intelligent machine gesture stability browses three-dimensional panoramic image

To roam centered by observer in virtual scene, draw hollow spheroid, sphere inner wall is played up using aforementioned panorama sketch as texture.Method is as follows: any pixel coordinate (x, y) on panorama sketch is corresponding also just corresponding radius is 3 dimension coordinates on the sphere of r therefore the grid of a lot of sub-block composition will be divided in panorama sketch, the panorama sketch coordinate on each grid 4 summit corresponds to the tile that 43 dimension coordinates on sphere surround, and utilizes the graph rendering function on intelligent machine to play up texture tiles all in grid thus and just achieves playing up whole spheroid.

By regulating the above-mentioned intelligent machine attitude matrix R recorded _g, or utilize said method to record the attitude matrix R of camera on intelligent machine, regulate roaming observer to browse the attitude of sphere.Alpha channel panorama sketch can not being photographed part sets to 0.

For ensureing to see the effect of observing sphere inner wall panorama texture from inside on intelligent machine display screen.Set following amount:

(1) observation place and sphere centre coordinate X _d, Y _d, Z _d; Can be exactly 0,0,0

(2) direction of observation { r ₃₁, r ₃₂r ₃₃or user's blinkpunkt coordinate (X _d+ r ₃₁, Y _d+ r ₃₂, Z _d+ r ₃₃)

(3) intelligent machine self direction vector {-r upwards ₂₁,-r ₂₂,-r ₂₃}

R wherein in step (2) and (3) _ijthat the i-th row j in intelligent machine camera attitude R matrix arranges, i=2 or 3, j=1,2 or 3;

(B) panorama sketch builds storehouse, retrieval and location

For effectively utilizing storage space, except the formula adopted in step 3 (1), go back available inks Kato conversion formula,

Convert panoramic picture to, keep building storehouse consistent with the conversion regime of retrieval.

The advantage that the present invention obtains panoramic picture by intelligent machine attitude is to define a kind of unified image standard based on local coordinate system.Two Zhang Quanjing image scalings mate to onesize, and ratio has been unified, and the internal reference differences such as the rotation of X-Y-Z tri-axle, camera focus have all done unitized process.The problem of ratio, rotation, internal reference these serious imaged image couplings inconsistent, retrieval is obtained for solution.Mainly consider illumination in matching process, just, such speed, accuracy will be greatly improved the factors such as intensity profile.

Build storehouse:

Extract unique point set in panoramic picture storehouse; by unique point by its position grid division on panoramic picture; each grid sets up a word bank; the descriptor of unique point is stored in corresponding word bank by the form of hash table or kdtree binary tree or balanced binary tree; each node of binary tree houses the descriptor of a unique point; and the numbering of the panorama sketch index of its correspondence of sensing, set up unique point storehouse; Unique point can adopt ORB, and Harris, Surf, SIFT do not consider the simplification version of rotation, ratio.

Retrieval:

For a c >=1 to be retrieved image, first c is opened Images uniting 1 Zhang Quanjing image, the unique point that extraction panoramic picture has been played up in district is N number of, for each unique point pi, i=1..N, only in feature database, carry out binary tree search with it near panorama sketch position is adjacent in several unique point word bank, find the image ki with the numbering of the immediate Feature point correspondence of pi descriptor di in adjacent word bank, the distance being retrieved unique point descriptor in the sub-di of record description and picture library is ei.Increase to the weights W ki of image ki wherein ε is a very little positive number.

Through the picture number k that the retrieval of N number of feature obtains ₁..., k _nprobably have repetition, suppose to obtain m Zhang Quanjing image after merging, respective weights is W ₁~ W _m.Calculate the degree of correlation M that N number of image k is similar ₁m _mif select maximal value j* M _j*the image being then numbered enough greatly j* is preliminary search result.

Panorama sketch to be retrieved is played up district and carried out relevant matches with the set of pixels of the panorama sketch of preliminary search from storehouse or integral image block of pixel summation, the larger image of related coefficient is more similar, substantially assert to be exactly a panoramic picture as related coefficient >0.9.

Location

Reflect the surrounding enviroment in a certain region because panoramic picture is complete, the location to region can be realized to the coupling of panoramic picture.The image being numbered i* is result for retrieval and then illustrates that positioning result is the position corresponding to image being numbered i*.

Should be understood that, for those of ordinary skills, can be improved according to the above description or convert, and all these improve and convert the protection domain that all should belong to claims of the present invention.

Claims (10)

1. intelligent machine attitude matrix computing method, is characterized in that, comprise the following steps:

By detecting three value { v of Gravity accelerometer on intelligent machine _ax', v _ay', v _az' calculate a _x, a _yand a _z:

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v_{\mathrm{ax}}}^{\&prime;}\\ {v_{\mathrm{ay}}}^{\&prime;}\\ {v_{\mathrm{az}}}^{\&prime;}\end{array}\right],$

Or to { v _ax', v _ay', v _az' carry out filtering after obtain { v _ax, v _ay, v _az, calculate a _x, a _yand a _z:

$\left[\begin{array}{c}{a}_x\\ a_y\\ a_z\end{array}\right]=R_a^T\left[\begin{array}{c}{v}_{\mathrm{ax}}\\ v_{\mathrm{ay}}\\ v_{\mathrm{az}}\end{array}\right],$ Wherein R _athe pose transformation matrix of Gravity accelerometer relative to intelligent machine;

Position angle is obtained by intelligent machine magnetometer trigonometric function; Or direction sensor obtains position angle

Calculate attitude matrix

Wherein

$R_{\mathrm{\&theta;}}=\left[\begin{array}{ccc}\frac{{\mathrm{sa}}_z}{L\sqrt{1-{(a_y/L)}^2}}& -s{a}_x{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_x/L\\ 0& s\sqrt{1-{(a_y/L)}^2}& a_y/L\\ \frac{-{\mathrm{sa}}_x}{L\sqrt{1-{(a_y/L)}^2}}& -{\mathrm{sa}}_z{a}_y/\left[L^2\sqrt{1-{(a_y/L)}^2}\right]& a_z/L\end{array}\right],L=\sqrt{a_x^2+a_y^2+a_z^2}$

Work as a _zduring <0, s=-1, a _zwhen>=0, s=1;

The attitude matrix R that formula (1) obtains _gdescribe the attitude of intelligent machine, attitude matrix R _gthe 1st row be the unit vector of intelligent machine x-axis under local coordinate system; 2nd row is the unit vector of intelligent machine y-axis under local coordinate system; 3rd row is the unit vector of intelligent machine z-axis under local coordinate system; Wherein local coordinate system be z-axis along gravity tester to outside the earth, x-axis y-axis of being exposed to the north is exposed to the west, or the coordinate system that x-axis is exposed to the north towards eastern y-axis.

2. method according to claim 1, is characterized in that, if be provided with direction sensor in intelligent machine:, obtain position angle by this intelligent machine bring formula (1) into and calculate attitude matrix R _g.

3. method according to claim 1, is characterized in that, if be provided with magnetometer sensor in intelligent machine, then three value { v detected by this magnetometer sensor _mx', v _my', v _mz', calculate $\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right]=R_m^T\left[\begin{array}{c}{v_{\mathrm{mx}}}^{\&prime;}\\ {v_{\mathrm{my}}}^{\&prime;}\\ {v_{\mathrm{mz}}}^{\&prime;}\end{array}\right];$ Or to { v _mx', v _my', v _mz' carry out filtering after obtain { v _mx, v _my, v _mz, calculate $\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right]=R_m^T\left[\begin{array}{c}{v}_{\mathrm{mx}}\\ v_{\mathrm{my}}\\ v_{\mathrm{mz}}\end{array}\right],$ Wherein R _mthe pose transformation matrix of magnetometric sensor relative to intelligent machine; And calculate $\left[\begin{array}{c}{m}_{0x}\\ m_{0y}\\ m_{0z}\end{array}\right]=R_{\mathrm{\&theta;}}^T\left[\begin{array}{c}{m}_x\\ m_y\\ m_z\end{array}\right],$ Wherein computer azimuth angle trigonometric function be:

Carry it into formula (1) and calculate R _g.

4. method according to claim 1, is characterized in that, if be provided with rotating vector sensor in intelligent machine, then:

(1) R is obtained by any one in following (a) (b) (c) _g0matrix:

If a the rotating vector sensor on () intelligent machine has 3 data values [0], values [1], values [2], then make q ₁=values [0], q ₂=values [1], q ₃=values [2], again by q ₀, q ₁, q ₂, q ₃convert R to as follows _g0matrix

$R_{g0}=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right]$

If b the rotating vector sensor on () intelligent machine has 4 data, values [0], values [1], values [2], values [3], make q ₀=values [3], q ₁=values [0], q ₂=values [1], q ₃=values [2], by q ₀, q ₁, q ₂, q ₃convert R to as follows _g0matrix

$R_{g0}=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right];$

If c on () intelligent machine, specified sensor type is rotating vector class TYPE_ROTATION_VECTOR, then the observed reading R of Real-time Obtaining rotating vector sensor _vec, utilize the rotating vector in intelligent machine system to turn matrix function by R _vecconvert drawing transition matrix R to ₄₄, R ₄₄upper left corner 3x3 matrix is exactly R _g0;

(2) according to the difference of intelligent machine rotating vector sensor coordinate system definition, in following several situation, Rg is obtained:

If the unit matrix of rotating vector sensor definition is X-axis point to east, Y-axis points to north, then

$R_g=R_{g0}{\left[\begin{array}{ccc}& 1& \\ -1& & \\ & & 1\end{array}\right]}^T;$

If the unit matrix of rotating vector sensor definition is X-axis point to north, Y-axis points to west, then R _g=R _g0;

If the unit matrix of rotating vector sensor definition is X-axis energized south, Y-axis points to east, then

$R_g=R_{g0}{\left[\begin{array}{ccc}-1& & \\ & -1& \\ & & 1\end{array}\right]}^T.$

5. method according to claim 1, is characterized in that, if intelligent machine is provided with gyroscope, then and attitude matrix R _ggyroscope is adopted to carry out iterative computation;

Calculating first R _gafterwards, if there is gyroscope, following methods iteration is adopted to obtain R _gmatrix:

1st step setting q ₀, q ₁, q ₂, q ₃initial value

Formula (1) is selected to calculate R _ginitial value, then by R _gmatrix conversion becomes 4 element q ₀, q ₁, q ₂, q ₃;

By 4 yuan of number q ₀, q ₁, q ₂, q ₃as the initial value of following iterative computation;

2nd step setting exInt, eyInt, ezInt original value is 0

exInt=0，eyInt=0，ezInt=0；

3rd step is according to the magnetometer vector { m received _x, m _y, m _z, obtain correct magnetic vector { w _x, w _y, w _z}

First by vector { m _x, m _y, m _zreplace to its unitization later vector $\{m_x,m_y,m_z\}/\sqrt{m_x^2+m_y^2+m_z^2}$

And obtain vector { b as follows _x, 0, b _z}

h _x=2×m _x×(0.5-q ₂×q ₂-q ₃×q ₃)+2×my×(q ₁×q ₂-q ₀×q ₃)+2×m _z×(q ₁×q ₃+q ₀×q ₂);

h _y=2×m _x×(q ₁×q ₂+q ₀×q ₃)+2×m _y×(0.5-q ₁×q ₁-q ₃×q ₃)+2×m _z×(q ₂×q ₃-q ₀×q ₁);

h _z=2×m _x×(q ₁×q ₃-q ₀×q ₂)+2×m _y×(q ₂×q ₃+q ₀×q ₁)+2×m _z×(0.5-q ₁×q ₁-q ₂×q ₂);

$b_x=\sqrt{{h_x}^2+{h_y}^2};$

b _z=h _z

Be transformed into correct magnetic vector { wx, wy, wz} again

w _x=2×b _x×(0.5-q ₂×q ₂-q ₃×q ₃)+2×b _z×(q ₁×q ₃-q ₀×q ₂);

w _y=2×b _x×(q ₁×q ₂-q ₀×q ₃)+2×b _z×(q ₀×q ₁+q ₂×q ₃);

w _z=2×b _x×(q ₀×q ₂+q ₁×q ₃)+2×b _z×(0.5-q ₁×q ₁-q ₂×q ₂);

4th step is according to the Gravity accelerometer data a received _x, a _y, a _z, and { w _x, w _y, w _zobtain error vector { e _x, e _y, e _zand calculate e _xint, e _yint, e _zint

First by vector { a _x, a _y, a _zreplace to its unitization later vector

v _x=2×q ₁×q ₃-q ₀×q ₂)；

v _y=2×(q ₀×q ₁+q ₂×q ₃)；

v _z=q ₀×q ₀-q ₁×q ₁-q ₂×q ₂+q ₃×q ₃；

e _x=(a _y×v _z-a _z×v _y)+(m _y×w _z-m _z×w _y)；

e _y=(a _z×v _x-a _x×v _z)+(m _z×w _x-m _x×w _z)；

e _z=(a _x×v _y-a _y×v _x)+(m _x×w _y-m _y×w _x)；

Error of calculation aggregate-value

E _xint replaces with e _xint+e _x× Ki;

E _yint replaces with e _yint+e _y× Ki;

E _zint replaces with e _zint+e _z× Ki;

Wherein Ki is an adjustable positive coefficient, and Ki chooses arbitrarily in 0.00001 to 0.5;

5th step is according to error vector { e _x, e _y, e _zand aggregate-value correction gyro data { g _x0, g _y0, g _z0suppose that intelligent machine reads one group of current gyro data for { g _x0, g _y0, g _z0}

g _x=g _x0+Kp×e _x+e _xInt；

g _y=g _y0+Kp×e _y+e _yInt；

g _z=g _z0+Kp×e _z+e _zInt；

Wherein Kp is an adjustable positive coefficient, and Kp chooses arbitrarily in 0.000001 to 20.0;

6th step is according to gyro data { g _x, g _y, g _zrevise hypercomplex number

Along with constantly receiving gyro data { g _x, g _y, g _z, revise as follows 4 yuan of numbers, halfT is the correction cycle, halfT=0.00001 ~ 10.0,

Q ₀replace with q ₀+ (-q ₁× g _x-q ₂× g _y– q ₃× g _z) × halfT;

Q ₁replace with q ₁+ (q ₀× g _x-q ₃× g _y+ q ₂× g _z) × halfT;

Q ₂replace with q ₂+ (q ₃× g _x+ q ₀× g _y-q ₁× g _z) × halfT;

Q ₃replace with q ₃+ (-q ₂× g _x+ q ₁× g _y+ q ₀× g _z) × halfT;

7th step exports R _gmatrix and hypercomplex number

By hypercomplex number { q ₀, q ₁, q ₂, q ₃unit changes into export 4 yuan of numbers and turn R _gmatrix Formula is as follows

$R_g=\left[\begin{array}{ccc}{q_0}^2+{q_1}^2-{q_2}^2-{q_3}^2& 2(q_1{q}_2+q_0{q}_3)& 2(q_1{q}_3-q_0{q}_2)\\ 2(q_1{q}_2-q_0{q}_3)& {q_0}^2-{q_1}^2+{q_2}^2-{q_3}^2& 2(q_2{q}_3+q_0{q}_1)\\ 2(q_1{q}_3+q_0{q}_2)& 2(q_2{q}_3-q_0{q}_1)& {q_0}^2-{q_1}^2-{q_2}^2+{q_3}^2\end{array}\right]$

8th step is got back to the 3rd step and is continued to receive Motions of Gyroscope Data Update attitude 4 yuan of number q ₀~ q ₃, in the process of circulation, current R can be exported to the 7th step at every turn _gmatrix.

6. the method according to any one of claim 1-5, is characterized in that, to attitude matrix R _gcorrect, correct attitude matrix $R=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$

Wherein R=Δ R × R _g;

Δ R is camera attitude matrix R and the intelligent machine attitude matrix R of intelligent machine _gbetween the conversion in a fixing direction, when camera direction and intelligent machine screen orientation are just contrary $\mathrm{\&Delta;R}=\left[\begin{array}{ccc}1& & \\ & -1& \\ & & -1\end{array}\right],$ Camera direction and intelligent machine screen orientation completely the same time $\mathrm{\&Delta;R}=\left[\begin{array}{ccc}1& & \\ & 1& \\ & & 1\end{array}\right],$ Otherwise adopt photogrammetric rear Convergence method to demarcate Δ R, concrete grammar is as follows:

Set up controlling filed coordinate system, namely initial point is positioned at the GPS acceptance point of intelligent machine or the local coordinate system of Big Dipper anchor point;

With the point set in intelligent machine camera shooting controlling filed coordinate system; By according to any one of claim 1-5 method measure intelligent machine shooting point set time R _g;

Concentrate according to controlling filed point the three-dimensional coordinate of the controlling filed coordinate system of each known point and shooting image Point Set as the corresponding relation of coordinate, adopt photogrammetric resection method to measure the attitude matrix R of intelligent machine camera relative to controlling filed coordinate system _aalso be simultaneously camera photocentre relative to the local coordinate system skew of intelligent machine GPS or Big Dipper anchor point with the coordinate Δ T of camera photocentre relative to controlling filed coordinate system, Δ T; Calculate Δ R=R _ar _g ^-1.

7. a camera multi-vision visual localization method, is characterized in that, comprises the following steps:

Step 1:

In N number of geographic position, point takes same object X, wherein N >=2, surveys the following amount of i-th observation point, wherein i ∈ (1,2 ..., N):

The position that (1) shows in video frame images (ui, vi);

(2) geographic position of shooting point the longitude of i-th shooting point respectively, latitude, highly;

(3) the attitude matrix R of i-th shooting point is recorded by claim 6 method _i;

The long f of inner parameter focal length of (4) i-th shooting points _i, principal point coordinate (c _xi, c _yi);

Step 2:

Calculate the N number of attitude matrix relative to the earth's core, i=1 ~ N, R _0i=R _ir _vi

Wherein

Step 3:

Utilize formula obtain the geocentric coordinate system coordinate T of N number of shooting point camera photocentre _0i={ Txi, Tyi, Tzi} ^ti=1 ~ N; Wherein { Δ Txi, Δ Txi, Δ Tzi} ^t=R _vi ^tΔ T _i; Δ T _iit is the camera optical centre bias vector that method according to claim 6 calculates; by longitude, latitude, height conversion becomes the function of the X-coordinate of geocentric coordinate system; by longitude, latitude, height conversion becomes the function of the Y-coordinate of geocentric coordinate system; by longitude, latitude, height conversion becomes the function of the Z coordinate of geocentric coordinate system;

Step 4

According to the attitude matrix R of shooting point N _0N, position [T _xNt _yNt _zN], the position coordinates (u shown in video frame images _n, v _n), the long f of camera focal length pixel _n, principal point coordinate (c _xN, c _yN) all meet following equation

$u_i=c_{\mathrm{xi}}+f_i\frac{r_{0i11}(X_x-T_{\mathrm{xi}})+r_{0i12}(X_y-T_{\mathrm{yi}})+r_{0i13}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

$v_i=c_{\mathrm{yi}}+f_i\frac{r_{0i21}(X_x-T_{\mathrm{xi}})+r_{0i22}(X_y-T_{\mathrm{yi}})+r_{0i23}(X_z-T_{\mathrm{zi}})}{r_{0i31}(X_x-T_{\mathrm{xi}})+r_{0i32}(X_y-T_{\mathrm{yi}})+r_{0i33}(X_z-T_{\mathrm{zi}})}$

Wherein r _0iabrepresent attitude matrix R _0ithe element value of a capable b row; I=1 ~ N is row 2N equation altogether, separates (Xx, Xy, Xz) three unknown numbers; Or adopt the photogrammetric front method that crosses to calculate object X and obtain geocentric coordinate system coordinate (Xx, Xy, Xz);

Finally obtain longitude and latitude corresponding to geocentric coordinate (Xx, Xy, Xz) by geodesy formula.

8. a monocular single-point terrestrial positioning method, is characterized in that, comprises step:

Step 1

Method according to claim 6 calculates the attitude matrix of camera $R=\mathrm{\&Delta;R}\&times;R_g=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right];$

Step 2

Camera capture video is chosen a bit on ground level, (u v), obtains the long f of inner parameter focal length of camera, principal point coordinate (c in advance to record its position shown in video frame images _x, c _y); This is as follows in the computing formula of the coordinate (X, Y) of local coordinate system:

Wherein h is camera height overhead;

Step 3

Local coordinate system coordinate (X, Y ,-h) is converted to longitude and latitude by Geodetic surveying method,

T _{the earth's core}=T _s+ R _v ^t([X, Y ,-h] ^t+ Δ T)

Wherein T _sintelligent machine geocentric coordinate,

Δ T is by the displacement vector of claim 6 method record intelligent machine camera photocentre relative to GPS anchor point;

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}, obtain T by geodesy formula _{the earth's core}corresponding longitude and latitude.

9. the position of monocular two point measurement perpendicular to the object on ground and a method for height, is characterized in that, comprise the following steps:

Step 1

1 image perpendicular to ground object is taken with intelligent machine camera, and following amount under measurements and calculations:

(1) method according to claim 6 calculates the attitude matrix R=Δ R × R of camera _g

(2) position (u ⊥, v ⊥) that object intersection point on the ground shows in video frame images is measured

(3) position (u shown in video frame images perpendicular to the top of ground object is measured _h, v _h)

(4) the long fx of inner parameter focal length of camera is obtained, principal point coordinate (c _x, c _y)

(5) camera overhead height h is obtained;

Step 2

Calculate following amount

$z_{\&perp;}=\frac{-h}{r_{13}\frac{u_{\&perp;}-c_x}{f}+r_{23}\frac{v_{\&perp;}-c_y}{f}+r_{33}}$

$p_{\&perp;}=r_{11}\frac{u_{\&perp;}-c_x}{f}+r_{21}\frac{v_{\&perp;}-c_y}{f}+r_{31}$

$q_{\&perp;}=r_{12}\frac{u_{\&perp;}-c_x}{f}+r_{22}\frac{v_{\&perp;}-c_y}{f}+r_{32}$

$p_h=r_{11}\frac{u_h-c_x}{f}+r_{21}\frac{v_h-c_y}{f}+r_{31}$

$q_h=r_{12}\frac{u_h-c_x}{f}+r_{22}\frac{v_h-c_y}{f}+r_{32}$

$r_h=r_{13}\frac{u_h-c_x}{f}+r_{23}\frac{v_h-c_y}{f}+r_{33}$

Step 3

Solution is matrix equation below, obtains [X Y H]

$\left[\begin{array}{ccc}1& & \\ & 1& \\ 1& & \frac{-p_h}{r_h}\\ & 1& \frac{-q_h}{r_h}\end{array}\right]\left[\begin{array}{c}X\\ Y\\ H\end{array}\right]=\left[\begin{array}{c}{z}_{\&perp;}{p}_{\&perp;}\\ z_{\&perp;}{q}_{\&perp;}\\ 0\\ 0\end{array}\right]$

Wherein (X, Y) is the coordinate of object under local coordinate system, and H is perpendicular to the height of ground object;

Step 4

The highest for object summit local coordinate system coordinate (X, Y, H) is converted to longitude and latitude by Geodetic surveying method; T _{the earth's core}=T _s+ R _v ^t([X, Y, H] ^t+ Δ T)

Wherein Ts is intelligent machine geocentric coordinate, and Δ T is the intelligent machine shooting that method according to claim 6 calculates

Head photocentre is relative to the displacement vector of GPS anchor point;

θ _sintelligent machine latitude, it is intelligent machine longitude;

Last according to geocentric coordinate T _{the earth's core}, obtain T by geodesy formula _{the earth's core}corresponding longitude and latitude.

10. an intelligent machine full-view image generation method, is characterized in that, comprise the following steps:

Step 1, intelligent machine open image at the different attitude shooting n of a certain point of fixity, adopt the method for claim 6 to obtain attitude matrix and are respectively R _i, i is the integer between 0 ~ n-1, n be more than or equal to 1 integer;

Step 2, open up the full-view image of a wide w height h size;

Each pixel on step 3, scanning full-view image, each pixel (x, y) for panorama sketch carries out (1) (2) process as follows

(1) horizontal ordinate x converts to ordinate y converts θ to;

wherein w, h are the wide of panoramic picture, high;

(2) according to pinhole imaging system formula

Wherein $K_i=\left[\begin{array}{ccc}{f}_i& & c_{\mathrm{ix}}\\ & f_i& c_{\mathrm{iy}}\\ & & 1\end{array}\right]$ F _ifor camera i focal length length in pixels, (c _ix, c _iy) be the principal point of camera i, utilize R _iwill convert the picture coordinate (u being mapped to each camera i to _i, v _i), if (u _i, v _i) in image range, then by camera i as coordinate (u _i, v _i) taking-up of place's color, weighting is rendered into panorama sketch (x, y) place; Weight and (u _i, v _i) relevant to this image border bee-line, the shorter weight of distance is less; If (the u of none camera _i, v _i) in its image range, rendering transparent look or background colour are to (x, y) place;

Scan on panoramic picture and filled a little corresponding color, namely completed playing up of panoramic picture;

If R _imatrix is inaccurate, then by R _ias initial value, LM adjustment Algorithm model in computer vision subject is adopted to be optimized, minimum to ensure the grey value difference of overlapping region respective pixel;

Or in step 2 first by panoramic picture color value all clear 0, open image in step 3 taken turns by each for the respective area on panoramic picture picture element scan n for n, weighted color value is accumulated in the pixel of panorama sketch by a point n wheel successively;

The three dimensional navigation of step 4 panoramic picture and retrieval location

(A) intelligent machine gesture stability browses three-dimensional panoramic image

Be observation point with the centre of sphere in virtual scene, sphere inner wall plays up a ball using panorama sketch as texture; Setting intelligent machine camera direction of observation { r ₃₁, r ₃₂r ₃₃, self upward direction {-r ₂₁,-r ₂₂,-r ₂₃; r _ijit is the i-th row jth column element in attitude R matrix;

(B) panorama sketch builds storehouse, retrieval and location

For effectively utilizing storage space, except the formula adopted in step 3 (1), go back available inks Kato conversion formula,

Convert panoramic picture to, keep building storehouse consistent with the conversion regime of retrieval;

Build storehouse

Extract the unique point set of panoramic picture, set up the characteristic point data storehouse associated with panoramic picture, in storehouse, each unique point points to the panoramic picture at its place;

Retrieval

Images uniting panoramic picture to be retrieved in c >=1, same place also extracts and plays up district's unique point, to each unique point, with the panoramic picture pointed by its most similar features point in inquiry storehouse, by unique point similarity degree, ballot is weighted to the panoramic picture pointed in storehouse, by the sequence of ballot summation size;

To play up district in panorama sketch to be retrieved and in the storehouse that obtains of voting, the set of pixels of panorama sketch corresponding region carries out relevant matches calculating, related coefficient is more large more similar; Export the result that the forward and related coefficient of ballot sequence is greater than preset value; If c=1, also can according to single shot attitude R, panoramic picture in storehouse is mapped to single shot image area and single shot image carries out relevant matches;

Location

In known GPS situation, the subset feature storehouse of the panoramic picture composition of retrieval GPS near zone;

Panoramic picture and its geographical position coordinates are bound together storage by server database, and result for retrieval is the image being numbered i*, can find out the geographic position corresponding to i* in a database.