CN1851618A - Single-eye vision semi-matter simulating system and method - Google Patents

Abstract

Said system of the present invention includes virtual imaging unit, picture projection unit and video camera realizing unit. In semi practicality emulation, said method includes: in virtual imaging unit, constructing and configuring virtual scene position stance through OpenGL etc software, through virtual video camera projecting virtual scene to computer screen, then picture projection unit projecting picture to projection plane, through video camera realizing unit to obtain projected picture on projection plane, finally through computer software to calculate virtual video camera inner parameter, virtual scene position stance; through comparing with set value and analysing to obtain emulation result accuracy error. Said invention has advantages of Said invention has advantages of simple equipment, easy realizing, and wide use range, especially suitable for realizing calibrating and vision guided navigation positional emulation.

Description

Single-eye vision semi-matter simulating system and method

Technical field

The present invention relates to a kind of single-eye vision semi-matter simulating system and method.

Background technology

Along with theory on computer vision and development of computer, computer vision has been widely used in aspects such as Target Recognition, visual guidance, commercial measurement, Industry Control.Semi-true object emulation technology is called hardware again at loop simulation, inserts material object under the situation of conditions permit as far as possible in analogue system, to replace the mathematical model of appropriate section, so more near actual conditions, thereby obtains more definite information.Run into some complex situations through regular meeting in the computer vision research process, such as often needing a large amount of scene images carry out algorithm research, but field condition complexity usually, being difficult in the research initial stage just carries out full-scale investigation and obtains a large amount of scene images.Like that both waste fund, influenced the research progress again.Therefore, it is very necessary carrying out emulation experiment at the initial stage of research work.Though Digital Simulation is completely economized on the use of funds, search time short, because complication system modeling difficulty is had to complication system is simplified, and a lot of states are made as perfect condition.Though also added noise, but be difficult to obtain very simulation result near actual conditions.Therefore, hardware-in-the-loop simulation has just embodied advantage in the initial stage of computer vision research.But also do not mention the problem that a kind of semi-matter simulating system can solve the complex situations emulation difficulty that is run in the computer vision research process in the available data, therefore designing a kind of single-eye vision semi-matter simulating system is very necessary and urgent in order to solve these difficulties.

Summary of the invention

The objective of the invention is to overcome the defective of prior art, a kind of single-eye vision semi-matter simulating system and method are provided, in order to be implemented in the simulation work of computer vision research under the complex environment, especially be fit to be applied to demarcate the The Realization of Simulation with the vision guided navigation location, its have equipment simple, be easy to realize and advantage that usable range is wide.

Monocular semi-matter simulating system of the present invention comprises that virtual image unit, image projection unit and shooting realize the unit, connect in turn between these three unit, and its relative position can not change.Described virtual image unit is that virtual scene is projected on the computer screen; Described image projection unit is that the virtual scene on the computer screen is projected on the projection plane; Described shooting realizes that the unit is to obtain image on the projection plane by monocular-camera.

Monocular hardware-in-the-loop simulation method of the present invention may further comprise the steps: at first, in the virtual image unit, by softwares such as OpenGL virtual scene is made up, set the position and attitude of virtual scene, virtual scene is projected on the computer screen by virtual video camera.Then, image projection unit projects image onto on the projection plane, realizes that through shooting the unit obtains projected image on projection plane.At last, calculate the inner parameter of virtual video camera by computer software, the position and attitude of virtual scene by comparing with setting value, is analyzed the trueness error that draws simulation result.

Advantage of the present invention mainly contains: (1) creatively combines computer vision and virtual reality technology, has well solved the problem of complex situations emulation difficulty; (2) equipment simple, be easy to realize that usable range is wide; (3), can obtain than complete Digital Simulation more near the simulation result of truth problem such as it is many to have solved full-scale investigation cost fund again, and setup time is long by this simulated environment.

Description of drawings

With embodiment single-eye vision semi-matter simulating system of the present invention and method are done explanation in further detail with reference to the accompanying drawings.

Fig. 1 is a system architecture schematic diagram of the present invention;

Fig. 2 is the structural representation of system of the present invention;

Fig. 3 is a method flow schematic diagram of the present invention;

Fig. 4 is that virtual video camera field angle of the present invention and scale factor concern synoptic diagram;

Fig. 5 is the plane target drone synoptic diagram of embodiments of the invention;

Fig. 6 is the projected image synoptic diagram of virtual scene of the embodiment of Fig. 5;

Fig. 7 is the image synoptic diagram of unique point of extraction of the embodiment of Fig. 6.

Embodiment

Single-eye vision semi-matter simulating system shown in Figure 1 comprises: virtual image unit, image projection unit and shooting realize the unit, connect in turn between these three unit, and its relative position can not change.Described virtual image unit is that virtual scene is projected on the computer screen; Described image projection unit is that the virtual scene on the computer screen is projected on the projection plane; Described shooting realizes that the unit is to obtain image on the projection plane by monocular-camera.

As shown in Figure 2, virtual image of the present invention unit comprises the computing machine 1 that virtual views software is housed, and described image projection unit comprises projector 2 and projection screen 3.Described shooting realization unit comprises video camera 4, camera pan-tilt 5 and the computing machine 6 of simulation software is housed.Computing machine 6 may command camera pan-tilts 5 make video camera 4 obtain best field positions, and projector 2, projection screen 3, video camera 4 and camera pan-tilt 5 all are fixed on indoor wall or the support, and relative variation can not appear in the position between them.The parameter of projector 2 immobilizes, it is connected with computing machine 1, on projection screen 3, launch the virtual three-dimensional visual pattern that generates by virtual views software in the computing machine 1, video camera 4 and computing machine 6 join, to be sent to computing machine 6 by the view data that video camera 4 is gathered earlier by image pick-up card, and the image of this time collection is handled by the simulation software in the computing machine 6, calculate the inner parameter of virtual video camera and the position and attitude of virtual scene, then, change the position and attitude of virtual scene, repeat again to gather and calculate.

As shown in Figure 3, monocular hardware-in-the-loop simulation method of the present invention may further comprise the steps: at first, in the virtual image unit, by softwares such as OpenGL virtual scene is made up, set the position and attitude of virtual scene, by virtual video camera virtual scene is projected on the computer screen, then, image projection unit projects image onto on the projection plane, realizes that through shooting the unit obtains projected image on projection plane, and is last, calculate the inner parameter of virtual video camera by computer software, the position and attitude of virtual scene by comparing with setting value, is analyzed the trueness error that draws simulation result.

Specify each step that this method realizes below in conjunction with accompanying drawing:

Step 1: set up by the imaging model of virtual scene to computer screen

The video camera linear model is as shown in the formula 1

$s_1\left[\begin{array}{c}a\\ b\\ 1\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{\α}}_x& 0& u_0& 0\\ 0& {\mathrm{\α}}_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]=M_{x1}{M}_{x2}\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]...\left(1\right)$

The homogeneous coordinates of putting on the wherein virtual scene are [x _w, y _w, z _w, 1] ^T, the homogeneous coordinates of putting on the image are [a, b, 1] ^Tα _xBe that virtual video camera is as the scale factor on the u axle on the plane, α _yBe that virtual video camera is as the scale factor on the v axle on the plane; [u ₀, v ₀] be that virtual video camera is as plane epigraph centre coordinate; s ₁It is a scale factor; R, T are rotation matrix and translation vector; M _X1By α _x, α _y, u ₀, v ₀Determine, be called as the inner parameter of virtual video camera; M _X2By R, T determines, is called as the external parameter of virtual video camera.

Step 1: determine the virtual video camera inner parameter

Setting the virtual video camera image resolution ratio is 1024 * 768.30 degree are got at the vertical field of view angle, and concrete geometric relationship is set α here with reference to Fig. 4 _x=α _y=384.0/tan (15.0 * π/180)=1433.1075357064; u ₀=512, v ₀=384.

Step 2: determine the virtual video camera external parameter

In virtual views software, need determine the locus of viewpoint, i.e. the D coordinates value T of viewpoint under the virtual world coordinate system for the outer ginseng of determining virtual video camera _w=[X _w, Y _w, Z _w]; Direction of visual lines driftage angle θ (rotating), luffing angle  (rotating) and lift-over angle φ (rotating) around Z-direction around Y direction around X-direction.

Here the initial position of setting the virtual video camera coordinate system overlaps with virtual world coordinate system position.Virtual video camera under original state according to first lift-over again the order of the last driftage of pitching be rotated, thereby determine the postrotational final position of virtual video camera, its rotation matrix is R, again the virtual video camera viewpoint is displaced to set position and has promptly finished the conversion that is tied to the virtual video camera coordinate system from the virtual world coordinate.

Rotation of coordinate matrix R can try to achieve with the Eulerian angle representation, according to the virtual video camera rotation of front defined order as can be known in the Eulerian angle representation rotation be in proper order: around Z axle → around Y-axis → around X-axis, the anglec of rotation is positive and negative to be determined by the right-handed coordinate system rule, angle around X-axis, Y-axis, Z axle is respectively θ, , behind the φ, can derive and try to achieve rotation matrix and be:

If the coordinate figure of virtual video camera viewpoint in the virtual world coordinate system is T _W, then translation vector is:

T＝-(R*T _W)

Finally obtaining virtual video camera external parameter matrix is:

$M_{x2}=\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]$

Step 2: set up by computer screen to the imaging model of video camera as the plane

Because in calibration process, be difficult to projector is demarcated separately, so, do as a wholely to consider with latter two projection process merging.

, not only to consider the foundation of concrete model, and will demarcate during to video camera setting up computer screen, specifically be divided into the parameter of system as the imaging model on plane:

Step 1: set up imaging model

In order to understand whole imaging model more accurately, earlier set up computer screen respectively to projection plane, projection plane is to the model of video camera as the plane, and last again two models merging forms by computer screen to the imaging model of video camera as the plane, specifically comprises:

1) sets up by the imaging model of computer screen to projection plane

Because computer screen is face-face projection to the projection relation of projection plane, so can obtain as drag:

$s_2\left[\begin{array}{c}{x}_t\\ y_t\\ 1\end{array}\right]=H_t\left[\begin{array}{c}a\\ b\\ 1\end{array}\right]...\left(2\right)$

S wherein ₂Be scale factor; H _tBe one 3 * 3 nonsingular matrix; The homogeneous coordinates of putting on the projection plane are [x _t, y _t, 1].

2) set up by the imaging model of projection plane to video camera

Because projection plane is a plane, projection plane can get to the imaging model of video camera:

$s_3\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{\mathrm{\α}}_{\mathrm{cx}}& 0& u_{c0}\\ 0& {\mathrm{\α}}_{\mathrm{cy}}& v_{c0}\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}{r}_{c1}& r_{c2}& T_c\end{array}\right]\left[\begin{array}{c}{x}_t\\ y_t\\ 1\end{array}\right]...\left(3\right)$

Wherein video camera is [u, v, 1] as the homogeneous coordinates of putting on the plane ^Ts ₃It is a scale factor; α _CxBe that video camera is as the scale factor on the u axle on the plane, α _CyBe that video camera is as the scale factor on the v axle on the plane; [u _C0, y _C0] be that video camera is as epigraph center, plane coordinate; r _C1, r _C2Be respectively R _cThe 1st, 2 row.

3) set up by computer screen to the imaging model of video camera as the plane

Formula (2) and formula (3) merging are formed by computer screen to the imaging model of video camera as the plane

Concrete formula is as follows:

$s_4\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{\mathrm{\α}}_{\mathrm{cx}}& 0& u_{c0}\\ 0& {\mathrm{\α}}_{\mathrm{cy}}& v_{c0}\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}{r}_{c1}& r_{c2}& t_c\end{array}\right]H_t\left[\begin{array}{c}a\\ b\\ 1\end{array}\right]=H_b\left[\begin{array}{c}a\\ b\\ 1\end{array}\right]...\left(4\right)$

H in the formula _bBe 3 * 3 matrixes; s ₄It is a scale factor.

Step 2: the demarcation of system

Utilize the model formation of formula (4), system is demarcated.

Detailed process is as follows:

(1) the varifocal method that proposes according to Lenz and Tsai is (referring to Lenz.R.K, Tsai.R.Y, Techniques for Calibrationof the Scale Factor and Image Center for High Accuracy 3-D Machine Vision Metrology, IEEETransactions on Pattern Analysis and Machine Intelligence.Volume 10.Issue 5.September 1988.Pages:713-720) try to achieve video camera as the picture centre on the plane.

That (2) utilizes proposition such as Zhang Guangjun realizes demarcation to the video camera coefficient of radial distortion based on the constant distortion of camera coefficient scaling method of double ratio (writing " machine vision ", Beijing: Science Press, 2005. referring to Zhang Guangjun).

(3) obtaining on the basis of video camera as picture centre on the plane and coefficient of radial distortion, after video camera carried out distortion correction as the image on the plane, can set up linear equation by (4) formula by a plurality of angle points on the target, utilize least square method can obtain matrix H _b, promptly finish the demarcation of parameter.

Step 3: set up overall calculation machine vision hardware-in-the-loop simulation environmental model

The mathematical model of front process is merged, can obtain the block mold of hardware-in-the-loop simulation environment:

$s_5\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=H_b\left[\begin{array}{cccc}{\mathrm{\α}}_x& 0& u_0& 0\\ 0& {\mathrm{\α}}_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]...\left(5\right)$

s ₅It is a scale factor; H wherein _bCan obtain by demarcation; R, T are artificial the setting.

According to above-mentioned model as can be seen, the homogeneous coordinates of putting on as the plane when known video camera [u, v, 1] ^TWith the homogeneous coordinates [x that puts on the virtual scene _w, y _w, z _w, 1] ^TThe time, the confidential reference items that just can calculate the virtual video camera in the formula (5) reach outer ginseng (R, T).

The step of monocular of the present invention half visual simulation in kind is described below by a specific embodiment: unmanned plane has been obtained illustrious military success on land battlefield, its outstanding performance has showed that to All Around The World unmanned plane is applied to the bright prospects of sea warfare, thereby is subjected to the attention of various countries naval.And one of key technical problem of carrier-borne unmanned plane with regard to after being to use warship reclaim problem, here I adopt the present invention to carry out unmanned plane and the warship emulation experiment.

Step 1:

At first by plane target drone of OpenGL software design of virtual image unit, as shown in Figure 5.Here get that four angle points of each black square are unique point in the target zone, each characteristic point coordinates is set in advance, the arrangement mode of wherein big or small black square be for the initial point of discerning the target coordinate system and X-axis convenient.

Step 2:

Realize that by shooting the unit obtains the image through the image projection unit projection on projection plane.In order to narrate for simplicity, only corresponding change a position and attitude here, be 45 degree with the angle of pitch, viewpoint is to be example in 150 centimeters, as shown in Figure 6.

Step 3:

By the image coordinate of unique point in the image processing function extraction image of Computer Simulation software, as shown in Figure 7.

Step 4:

M in known formula (5) ₁, M ₂, α _x, α _y, u ₀, v ₀Under the situation, according to formula (5), can solve unmanned plane position and attitude (R, T), concrete data are as follows:

R＝

0.702725 -0.001678 -0.714938

-0.000565 -0.999956 0.001809

-0.711514 -0.000596 -0.699186

T＝

-0.006923

-0.004578

150.141495

Setting value and actual measured value such as following table among the OpenGL

	θ (degree)	 (degree)	φ (degree)	T x(centimetre)	T y(centimetre)	T z(centimetre)
							Set	0	45	0	106.06601	0	106.066018
Calculate	-0.129	45.356	-0.137	106.82711	0.102277	105.504913
							Deviation	-0.129	0.356	-0.137	0.761095	0.102277	-0.561105

T=[T in the table _x, T _y, T _z] ^TBe the coordinate under the alive boundary of the video camera photocentre coordinate system.

Above-described only is preferred implementation of the present invention.Should be pointed out that for the person of ordinary skill of the art under the prerequisite that does not break away from the principle of the invention, can also make some modification and improvement, these also should be considered as belonging to protection scope of the present invention.

Claims (6)

1. single-eye vision semi-matter simulating system is characterized in that comprising: virtual image unit, image projection unit and shooting realize the unit, connect in turn between these three unit, and its relative position can not change.

2. single-eye vision semi-matter simulating system as claimed in claim 1, it is characterized in that described virtual image unit comprises the computing machine (1) that virtual views software is housed, described image projection unit comprises projector (2) and projection screen (3), described shooting realizes that the unit comprises video camera (4), camera pan-tilt (5) and the computing machine (6) of simulation software is housed, computing machine (6) may command camera pan-tilt (5), make video camera (4) obtain best field positions, described projector (2), projection screen (3), video camera (4) and camera pan-tilt (5) are fixed on indoor wall or the support, relative variation can not appear in the position between them, the parameter of projector (2) immobilizes, it is connected with computing machine (1), launch the virtual three-dimensional visual pattern that virtual views software generates from computing machine 1 on projection screen (3), described video camera (4) and computing machine (6) join.

3. a method of utilizing single-eye vision semi-matter simulating system as claimed in claim 1 to carry out hardware-in-the-loop simulation is characterized in that comprising the steps:

The first step in the virtual image unit, makes up virtual scene by softwares such as OpenGL, sets the position and attitude of virtual scene, by virtual video camera virtual scene is projected on the computer screen;

In second step, image projection unit projects image onto on the projection plane, realizes that through shooting the unit obtains projected image on projection plane;

The 3rd step calculated the inner parameter of virtual video camera by computer software, and the position and attitude of virtual scene by comparing with setting value, is analyzed the trueness error that draws simulation result.

4. single-eye vision semi-matter emulation mode as claimed in claim 3, it is characterized in that: the described first step comprises the step of the internal and external parameter of setting virtual video camera.

5. single-eye vision semi-matter emulation mode as claimed in claim 3 is characterized in that: described second step comprises: set up by the imaging model of computer screen to projection plane; Foundation is by the imaging model of projection plane to video camera; Foundation by computer screen to the imaging model of video camera as the plane.

6. single-eye vision semi-matter emulation mode as claimed in claim 3 is characterized in that: described the 3rd step comprises the block mold of setting up hardware-in-the-loop simulation:

$s_5\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=H_b\left[\begin{array}{cccc}{\mathrm{\α}}_x& 0& u_0& 0\\ 0& {\mathrm{\α}}_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]$

H wherein _bBe 3 * 3 matrixes, α _xBe that virtual video camera is as the scale factor on the u axle on the plane, α _yBe that virtual video camera is as the scale factor on the v axle on the plane, [u ₀, v ₀] be that virtual video camera is as plane epigraph centre coordinate, s ₅Be a scale factor, video camera is as the coordinate of putting on the plane [u, v, 1] ^T, the homogeneous coordinates [x that puts on the virtual scene _w, y _w, z _w, 1] ^T

Images