CN1851618A - Single-eye vision semi-matter simulating system and method - Google Patents

Abstract

The present invention relates to a monocular vision semi-physical simulation system and method. The system includes: a virtual imaging unit, an image projection unit and a camera realization unit. When performing semi-physical simulation, the following steps are included: in the virtual imaging unit, The virtual scene is constructed by software such as OpenGL, the position and posture of the virtual scene are set, and the virtual scene is projected onto the computer screen through the virtual camera. Then, the image projection unit projects the image onto the projection plane. Finally, the internal parameters of the virtual camera and the position and posture of the virtual scene are calculated by computer software, and the accuracy error of the simulation result is analyzed by comparing with the set value. The invention has the advantages of simple equipment, easy realization and wide application range, and is especially suitable for simulation realization of calibration and visual navigation and positioning.

Description

Monocular vision hardware-in-the-loop simulation system and method

technical field

The invention relates to a monocular vision semi-physical simulation system and method.

Background technique

With the development of computer vision theory and computer technology, computer vision has been widely used in target recognition, visual guidance, industrial measurement, industrial control, etc. Hardware-in-the-loop simulation technology is also called hardware-in-the-loop simulation. When conditions permit, physical objects are connected to the simulation system as much as possible to replace the corresponding part of the mathematical model, which is closer to the actual situation and thus more accurate information can be obtained. In the process of computer vision research, some complex situations are often encountered. For example, a large number of scene images are often required for algorithm research, but usually the site conditions are complicated, and it is difficult to conduct physical experiments in the early stage of research to obtain a large number of scene images. That would waste money and affect research progress. Therefore, it is very necessary to conduct simulation experiments in the initial stage of research work. Although complete digital simulation saves money and shortens the research time, due to the difficulty in modeling complex systems, complex systems have to be simplified and many states are set to ideal states. Although noise is also added, it is still difficult to obtain simulation results that are very close to the actual situation. Therefore, hardware-in-the-loop simulation has shown advantages in the early stages of computer vision research. However, there is no mention of a hardware-in-the-loop simulation system in the existing data that can well solve the problem of difficult simulation of complex situations encountered in the computer vision research process, so a monocular vision hardware-in-the-loop simulation system is designed to solve These difficulties are very necessary and urgent.

Contents of the invention

The purpose of the present invention is to overcome the defects of the prior art, to provide a monocular vision semi-physical simulation system and method, to realize the simulation work of computer vision research in a complex environment, especially suitable for calibration and visual navigation positioning simulation It has the advantages of simple equipment, easy implementation and wide application range.

The monocular hardware-in-the-loop simulation system of the present invention includes a virtual imaging unit, an image projection unit and a camera realization unit, these three units are connected in sequence, and their relative positions cannot be changed. The virtual imaging unit is to project the virtual scene on the computer screen; the image projection unit is to project the virtual scene on the computer screen onto the projection plane; image on the .

The monocular semi-physical simulation method of the present invention comprises the following steps: first, in the virtual imaging unit, the virtual scene is constructed by software such as OpenGL, the position and posture of the virtual scene are set, and the virtual scene is projected onto the computer screen by the virtual camera . Then, the image projection unit projects the image onto the projection plane, and obtains the projection image on the projection plane through the camera realization unit. Finally, the internal parameters of the virtual camera and the position and attitude of the virtual scene are calculated by computer software, and the accuracy error of the simulation result is analyzed by comparing with the set value.

The advantages of the present invention mainly include: (1) Creatively combining computer vision and virtual reality technology, it solves the problem of difficult simulation of complex situations; (2) The equipment is simple, easy to implement, and has a wide range of applications; (3) Through this simulation environment, not only the simulation results closer to the real situation can be obtained than the complete digital simulation, but also the problems of high cost and long preparation time for the physical experiment are solved.

Description of drawings

The monocular vision hardware-in-the-loop simulation system and method of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

Fig. 1 is a schematic diagram of the system structure of the present invention;

Fig. 2 is the structural representation of the system of the present invention;

Fig. 3 is a schematic diagram of the method flow chart of the present invention;

Fig. 4 is a schematic diagram of the relationship between the virtual camera field of view and the scale factor of the present invention;

Fig. 5 is the planar target schematic diagram of the embodiment of the present invention;

Fig. 6 is a schematic diagram of a projected image of a virtual scene in the embodiment of Fig. 5;

FIG. 7 is a schematic diagram of an image of extracted feature points in the embodiment of FIG. 6 .

Detailed ways

The monocular vision hardware-in-the-loop simulation system shown in Figure 1 includes: a virtual imaging unit, an image projection unit, and a camera realization unit. These three units are connected in sequence, and their relative positions cannot be changed. The virtual imaging unit is to project the virtual scene on the computer screen; the image projection unit is to project the virtual scene on the computer screen onto the projection plane; The image on the projection plane.

As shown in FIG. 2 , the virtual imaging unit of the present invention includes a computer 1 equipped with virtual scene software, and the image projection unit includes a projector 2 and a projection screen 3 . The camera realization unit includes a camera 4, a camera pan-tilt 5 and a computer 6 equipped with simulation software. The computer 6 can control the camera platform 5 so that the camera 4 can obtain the best field of view position, and the projector 2, the projection screen 3, the camera 4 and the camera platform 5 are all fixed on the indoor wall or the support, and the position between them is There can be no relative change. The parameters of the projector 2 are fixed, and it is connected with the computer 1. On the projection screen 3, the virtual three-dimensional visual image generated by the virtual view software in the computer 1 is projected. First transmit the image data collected by the camera 4 to the computer 6, and process the collected image through the simulation software in the computer 6, calculate the internal parameters of the virtual camera and the position and posture of the virtual scene, and then change the virtual scene position and posture, and then repeat the acquisition calculation.

As shown in Figure 3, the monocular hardware-in-the-loop simulation method of the present invention includes the following steps: first, in the virtual imaging unit, the virtual scene is constructed by software such as OpenGL, the position and posture of the virtual scene are set, and the virtual scene is set by the virtual camera. The scene is projected on the computer screen, and then the image projection unit projects the image onto the projection plane, and the projection image is obtained on the projection plane through the camera realization unit. Finally, the internal parameters of the virtual camera and the position of the virtual scene are calculated by computer software. Attitude, by comparing with the set value, analyze the accuracy error of the simulation result.

Each step that this method realizes is described in detail below in conjunction with accompanying drawing:

Step 1: Create an imaging model from the virtual scene to the computer screen

The linear model of the camera is as follows: 1

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

The homogeneous coordinates of points on the virtual scene are [x _w , y _w , z _w , 1] ^T , and the homogeneous coordinates of points on the image are [a, b, 1] ^T ; α _x is u on the image plane of the virtual camera The scale factor on the axis, α _y is the scale factor on the v-axis on the virtual camera image plane; [u ₀ , v ₀ ] is the image center coordinates on the virtual camera image plane; s ₁ is a scale factor; R, T are rotation Matrix and translation vector; M _x1 is determined by α _x , α _y , u ₀ , v ₀ and is called the internal parameter of the virtual camera; M _x2 is determined by R and T and is called the external parameter of the virtual camera.

Step 1: Determine the internal parameters of the virtual camera

Set the image resolution of the virtual camera to 1024×768. The vertical viewing angle is taken as 30 degrees. Refer to Figure 4 for the specific geometric relationship. Here, α _x =α _y =384.0/tan(15.0×π/180)=1433.1075357064; u ₀ =512, v ₀ =384.

Step 2: Determine the external parameters of the virtual camera

In the virtual scene software, in order to determine the external parameters of the virtual camera, it is necessary to determine the spatial position of the viewpoint, that is, the three-dimensional coordinate value of the viewpoint in the virtual world coordinate system T _w = [X _w , Y _w , Z _w ]; the line of sight yaw Angle θ (rotate around the X-axis), pitch angle  (rotate around the Y-axis), and roll angle φ (rotate around the Z-axis).

Here it is set that the initial position of the virtual camera coordinate system coincides with the position of the virtual world coordinate system. In the initial state, the virtual camera rotates in the order of first roll, then pitch, and finally yaw, so as to determine the final position of the virtual camera after rotation. The rotation matrix is R, and then the virtual camera viewpoint is displaced to the set position and it is completed. Transformation from the virtual world coordinate system to the virtual camera coordinate system.

The coordinate rotation matrix R can be obtained by the Euler angle representation. According to the virtual camera rotation sequence specified above, the rotation sequence in the Euler angle representation is: around the Z axis → around the Y axis → around the X axis, and the rotation angle is positive or negative Determined according to the right-handed coordinate system rule, after the angles around the X-axis, Y-axis, and Z-axis are θ, , φ respectively, the rotation matrix can be derived as:

Suppose the coordinate value of the virtual camera viewpoint in the virtual world coordinate system is T _W , then the translation vector is:

T＝-(R*T _W )

Finally, the external parameter matrix of the virtual camera is obtained as:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

Step 2: Establish an imaging model from the computer screen to the camera image plane

Since it is difficult to calibrate the projector individually during the calibration process, the latter two projection processes are combined and considered as a whole.

When establishing the imaging model from the computer screen to the camera image plane, not only the establishment of the specific model must be considered, but also the parameters of the system must be calibrated, specifically divided into:

Step 1: Building an Imaging Model

In order to understand the entire imaging model more accurately, first establish a model from the computer screen to the projection plane, and from the projection plane to the camera image plane, and finally combine the two models to form an imaging model from the computer screen to the camera image plane, specifically including:

1) Establish an imaging model from the computer screen to the projection plane

Since the projection relationship from the computer screen to the projection plane is a plane-plane projection, the following model can be obtained:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, s ₂ is a scaling factor; H _t is a 3×3 non-singular matrix; the homogeneous coordinates of points on the projection plane are [x _t , y _t , 1].

2) Establish an imaging model from the projection plane to the camera

Since the projection plane is a plane, the imaging model from the projection plane to the camera can be obtained as:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; cx</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; cy</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, the homogeneous coordinates of the point on the camera image plane are [u, v, 1] ^T ; s ₃ is a scale factor; α _cx is the scale factor on the u axis on the camera image plane, and α _cy is the v axis on the camera image plane scale factor on ; [u _c0 , y _c0 ] are the coordinates of the center of the image on the camera image plane; r _c1 , r _c2 are the first and second columns of R _c respectively.

3) Establish an imaging model from the computer screen to the camera image plane

Combine formula (2) and formula (3) to form an imaging model from computer screen to camera image plane

The specific formula is as follows:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; cx</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; cy</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\end{array}\right]{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; a</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Where H _b is a 3×3 matrix; s ₄ is a scaling factor.

Step 2: Calibration of the system

Use the model formula of formula (4) to calibrate the system.

The specific process is as follows:

(1) According to the zoom method proposed by Lenz and Tsai (see Lenz.R.K, Tsai.R.Y, Techniques for Calibration of the Scale Factor and Image Center for High Accuracy 3-D Machine Vision Metrology, IEEE Transactions on Pattern Analysis and Machine Intelligence.Volume 10.Issue 5.September 1988.Pages: 713-720) to obtain the image center on the camera image plane.

(2) Using the camera distortion coefficient calibration method proposed by Zhang Guangjun et al. based on cross-ratio invariance (see Zhang Guangjun edited, "Machine Vision", Beijing: Science Press, 2005.) to achieve the calibration of the camera radial distortion coefficient.

(3) On the basis of obtaining the image center and radial distortion coefficient on the camera image plane, after distortion correction is performed on the image on the camera image plane, a linear equation can be established by formula (4) through multiple corner points on the target, The matrix H _b can be obtained by using the least square method, that is, the calibration of the parameters is completed.

Step 3: Establish an overall computer vision hardware-in-the-loop simulation environment model

Combining the mathematical models of the previous process, the overall model of the hardware-in-the-loop simulation environment can be obtained:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

s ₅ is a proportional factor; among them, H _b can be obtained through calibration; R, T are artificially set.

According to the above model, it can be seen that when the homogeneous coordinates [u, v, 1] ^T of the point on the camera image plane and the homogeneous coordinates [x _w , y _w , z _w , 1] ^T of the point on the virtual scene are known , the internal and external parameters (R, T) of the virtual camera in formula (5) can be calculated.

The steps of the monocular semi-physical visual simulation of the present invention will be described below through a specific embodiment: UAVs have achieved outstanding results on the land battlefield, and their outstanding performance has demonstrated to the whole world the broad prospects for UAVs to be used in naval warfare , so it is valued by the navies of various countries. And one of the key technical problems of ship-borne unmanned aerial vehicles is exactly the problem of shipboard recovery after use, and here I adopt the present invention to carry out unmanned aerial vehicle landing simulation experiment.

step one:

First, a planar target is designed by the OpenGL software of the virtual imaging unit, as shown in Figure 5. Here, the four corners of each black square in the target area are taken as feature points, and the coordinates of each feature point are set in advance. The arrangement of the large and small black squares is for the convenience of identifying the origin and X axis of the target coordinate system.

Step two:

The image projected by the image projection unit is obtained by the camera realization unit on the projection plane. For the sake of simplification, the position and posture are only changed once accordingly, taking the pitch angle as 45 degrees and the viewpoint at 150 centimeters as an example, as shown in Figure 6.

Step three:

The image coordinates of the feature points in the image are extracted through the image processing function of the computer simulation software, as shown in Figure 7.

Step four:

In the case of known M ₁ , M ₂ , α _x , α _y , u ₀ , v ₀ in formula (5), according to formula (5), the position and attitude (R, T) of the UAV can be solved, specifically Data are as follows:

R=

0.702725 -0.001678 -0.714938

-0.000565 -0.999956 0.001809

-0.711514 -0.000596 -0.699186

T =

-0.006923

-0.004578

150.141495

The set value and actual measured value in OpenGL are as follows θ (degrees)  (degrees) φ (degree) T _x (cm) T _y (cm) _Tz (cm) set up 0 45 0 106.06601 0 106.066018 calculate -0.129 45.356 -0.137 106.82711 0.102277 105.504913 deviation -0.129 0.356 -0.137 0.761095 0.102277 -0.561105

In the table, T=[T _x , Ty _y , T _z ] ^T is the coordinate of the optical center of the camera in the world coordinate system.

What has been described above are only preferred embodiments of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the principles of the present invention, and these should also be regarded as belonging to the protection scope of the present invention.

Claims (6)

1. A monocular visual hardware-in-the-loop simulation system is characterized in that comprising: a virtual imaging unit, an image projection unit and a camera realization unit, which are connected in sequence between these three units, and their relative positions cannot be changed. 2. a kind of monocular vision hardware-in-the-loop simulation system as claimed in claim 1 is characterized in that described virtual imaging unit comprises the computer (1) that virtual view software is equipped with, and described image projection unit comprises projection Instrument (2) and projection screen (3), described camera realization unit comprises video camera (4), video camera pan-tilt (5) and the computer (6) that emulation software is housed, computer (6) can control video camera pan-tilt (5) ), so that the camera (4) obtains the best field of view position, and the projector (2), projection screen (3), camera (4) and camera pan/tilt (5) are fixed on the indoor wall or bracket, and they There can be no relative changes between the positions, the parameters of the projector (2) are fixed, it is connected with the computer (1), and the virtual three-dimensional vision generated by the virtual view software in the computer 1 is projected on the projection screen (3). image, the camera (4) is connected with a computer (6). 3. a method utilizing monocular vision semi-physical simulation system as claimed in claim 1 to carry out semi-physical simulation, is characterized in that comprising the steps: In the first step, in the virtual imaging unit, the virtual scene is constructed by software such as OpenGL, the position and posture of the virtual scene are set, and the virtual scene is projected onto the computer screen through the virtual camera; In the second step, the image projection unit projects the image onto the projection plane, and obtains the projection image on the projection plane through the camera realization unit; In the third step, the internal parameters of the virtual camera and the position and posture of the virtual scene are calculated by computer software, and the accuracy error of the simulation result is analyzed by comparing with the set value. 4. A monocular vision hardware-in-the-loop simulation method as claimed in claim 3, characterized in that: the first step includes the step of setting the internal and external parameters of the virtual camera. 5. a kind of monocular vision hardware-in-the-loop simulation method as claimed in claim 3 is characterized in that: described second step comprises: set up the imaging model by computer screen to projection plane; Set up the imaging by projection plane to video camera Model; establish an imaging model from the computer screen to the camera image plane. 6. a kind of monocular vision hardware-in-the-loop simulation method as claimed in claim 3, is characterized in that: described 3rd step comprises setting up the whole model of hardware-in-the-loop simulation: ${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$ where H _b is a 3×3 matrix, α _x is the scale factor on the u-axis on the virtual camera image plane, α _y is the scale factor on the v-axis on the virtual camera image plane, [u ₀ , v ₀ ] is the virtual camera image The coordinates of the image center on the plane, s ₅ is a scale factor, the coordinates of the point on the camera image plane [u, v, 1] ^T , the homogeneous coordinates of the point on the virtual scene [x _w , y _w , z _w , 1] ^T .

Images