KR20150007023A - Vehicle simulation system and method to control thereof - Google Patents

Abstract

According to the present invention, a vehicle simulation system includes: a boarding part on which a user can sit on; an infrared ray emission part which is worn on a head of the user; an infrared camera which is located on a front side of the boarding part and senses movement of the infrared ray emission part; a control part which generates an image in the direction of user′s vision corresponding to the movement of the infrared ray emission part recorded by the infrared ray camera; and a display part which outputs and displays the image generated by the control part. According to the present invention, multiple users can get on one vehicle simulation system at the same time and can practice flight simulation based on tasks for the users and training characteristics for one vehicle when both a pilot and a co-pilot practice flight training using simulation for a vehicle such as a helicopter and an airplane. During a flight training process using simulation, the image corresponding to a location and movement of the user is provided in real time. The vehicle simulation system can provide training images optimized for each user, and thus enables the users to more effectively practice flight training using simulation for a vehicle.

Description

Technical Field [0001] The present invention relates to a vehicle simulation system and a control method thereof,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a transportation device simulation system and a control method thereof, and more particularly, to a transportation device simulation system and a control method thereof for providing an optimized image in a user's position and direction using an infrared tracking technology.

In general, the transportation device simulation system is widely used for the pre-operation training of various transportation devices such as an aircraft. The simulation system displays a simulated image of a virtual environment on the display unit and while the user views the display unit, the simulator drives the corresponding transportation apparatus in accordance with the displayed image using the control unit very similar to the control unit of the actual transportation apparatus It is training.

In some cases, a transportation system simulation system is used by a single user, but a plurality of users may simultaneously use the system. For example, in the case of an aircraft simulator system, a main pilot, a co-pilot, and other passengers may be simultaneously riding to perform training to operate one transportation device.

However, in the case where a plurality of users are trained and boarded, the same image is displayed on each occupant, which is different from the external environment that the user sees during transportation of the actual transportation apparatus, and thus there is a problem in half the training effect .

For example, even if one of a plurality of users moves their head to view an image of a part hidden in a frame of the cockpit of an aircraft, it can not individually provide matching images even if they are not visible, There is a limit to the realistic sense of body training of the transportation system.

US Patent No. 8,194,193, entitled "Method and apparatus for a wide field of view display &

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a simulation apparatus and a simulation method of a transportation apparatus simulating a plurality of users, System and a control method thereof.

The transportation system simulation system according to the present invention is characterized in that the transportation system simulation system comprises a boarding passable portion for the user, an infrared ray emitting portion to be worn on the head portion of the user, an infrared ray camera for detecting the movement of the infrared ray emitting portion, A control unit for generating an image of the user's viewing direction corresponding to the movement of the infrared light emitting unit; and a display unit for outputting and displaying the image generated by the control unit.

The display unit may include a screen installed in front of the boarding unit and a projector for projecting an image on the screen.

Wherein the boarding section includes a first boarding section on which the first user boarded and a second boarding section boarded by the second user, the infrared ray emitting section includes a first infrared ray emitting section worn by the first user and a second infrared ray emitting section worn by the second user And a second infrared ray emitting unit.

Wherein the controller generates a first image corresponding to the movement of the first infrared ray emitting unit and a second image corresponding to the movement of the second infrared ray emitting unit and the projector includes a first projector for projecting the first image onto the screen, And a second projector for projecting the second image onto the screen.

The control unit generates a first image corresponding to the movement of the first infrared ray emitting unit and a second image corresponding to the movement of the second infrared ray emitting unit and the first image and the second image are polarized with different polarizations And can be projected by the projector alternately.

The controller generates a first image corresponding to the movement of the first infrared ray emitting portion and a second image corresponding to the movement of the second infrared ray emitting portion and the first image and the second image are polarization- The projector may include a first projector for projecting the first image onto the screen and a second projector for projecting the second image onto the screen.

A control method of a transportation device simulation system according to the present invention is a method of controlling a transportation device simulation system, comprising: detecting an infrared ray emitting unit worn by a user with an infrared camera; acquiring information on a viewing direction of the user according to a position of the infrared ray emitting unit sensed by the infrared camera; Generates an output image matched with the information on the viewing direction, and outputs the generated output image on the display unit.

Wherein the boarding section includes a first boarding section on which the first user boarded and a second boarding section boarded by the second user, the infrared ray emitting section includes a first infrared ray emitting section worn by the first user and a second infrared ray emitting section worn by the second user And a second image corresponding to the movement of the second infrared ray emitting unit is generated to generate a first image corresponding to the movement of the first infrared ray emitting unit and output from the display unit, .

Wherein the display unit comprises a first projector for projecting the first image onto the screen and a second projector for projecting the second image onto the screen, the projector having a screen installed in front of the boarding section and a projector for projecting an image on the screen, .

The first image corresponding to the movement of the first infrared ray emitting part and the second image corresponding to the movement of the second infrared ray emitting part can be polarized and displayed with different polarized light and displayed alternately on the display part.

A simulation system for a transportation device according to the present invention is a system in which a plurality of users are boarded together in a simulation system to perform a simulation exercise on a transportation device such as a helicopter and an aircraft, In this way, it is possible to provide training images that are optimized for each user by providing images corresponding to the positions and motions of the respective users in real time, thereby simulating more effective transportation devices This has the effect of enabling pilot training.

1 is a diagram illustrating a transportation device simulation system according to an embodiment of the present invention.
2 is a block diagram illustrating a transportation device simulation system according to an embodiment of the present invention.
3 is a flow chart for explaining the operation of the transportation device simulation system according to the embodiment of the present invention.
4 is a view for explaining a state in which a cockpit frame of a transportation apparatus shown on a screen of a transportation device simulation system appears.
5A, 5B and 5C are diagrams for explaining a state in which a plurality of users change their view.

Embodiments of a transportation device simulation system according to an embodiment of the present invention described below will be described with reference to the drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, Is provided to fully inform the user.

Hereinafter, a configuration of a transportation device simulation system according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a transportation device simulation system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a transportation device simulation system according to an embodiment of the present invention.

1 and 2, a transportation device simulation system according to an embodiment of the present invention includes an infrared light emitting device 110 mounted on a head portion of a user 110, A plurality of infrared cameras 150 positioned in front of the passenger compartments 100 and 101 to sense the movement of the infrared ray emitting units 131 and 141, A main controller 210 for generating an image of a direction of the user 110 or 111 in response to the movement of the infrared ray emitting units 131 and 141, And a display unit 160 for displaying an image.

The boarding sections 100 and 101 include a seat 120 on which the users 110 and 111 can be seated and a steering section 120 for steering the simulated transporting device in front of the seat. The first and second boarding passages 100 and 101 are located on the left and right sides of the first boarding part 100 and the second boarding part 101 on which the second user 111 boarded. In another embodiment, the first boarding portion 100 and the second boarding portion 111 may be positioned in the forward and backward directions.

The infrared ray emitting units 131 and 141 may be installed in the head mount apparatuses 130 and 140 to be worn by the users 110 and 111 seated in the boarding units 100 and 101. A plurality of infrared ray emitting units 131 and 141 may be installed at a plurality of positions of the head mounts 130 and 140 or a plurality of infrared ray emitting units 131 and 141 may be mounted on the head mounts 130 and 140, As shown in FIG. And may be installed at various other positions so as to be able to detect the movement of the head and the eyes of the user 110 (111) two-dimensionally or three-dimensionally.

 The infrared camera 150 includes infrared light emitting units 131 and 141 mounted on the head mount devices 130 and 140 placed in front of the boarding passages 100 and 101 and worn by the users 110 and 111, .

Accordingly, the infrared ray emitting units 131 and 141 may include a first infrared ray emitting unit 131 mounted on the first head mount apparatus 130 worn by the first user 110 and a second infrared ray emitting unit 131 mounted on the second user 111 The first infrared ray emitting unit 131 and the second infrared ray emitting unit 141 are distinguished from each other by a second infrared ray emitting unit 141 installed in the head mount apparatus 140. The infrared ray camera 150 identifies the first infrared ray emitting unit 131 and the second infrared ray emitting unit 141, .

The infrared ray emitting units 131 and 141 may be implemented by an infrared light emitting diode (LED) that emits infrared rays. The infrared ray emitting units 131 and 141 are used to obtain motion information (angle, direction, position, etc.) of the head mount apparatuses 130 and 140 worn by the users 110 and 111, A plurality of heads may be mounted on the head mount devices 130,

In another embodiment, the infrared ray emitting units 131 and 141 may be implemented with an infrared marker. When implemented with an infrared marker, the infrared marker is formed into a spherical shape, and the sphere may be coated with a reflective layer that reflects infrared rays. The structure and the embodiment of such an infrared marker can be modified by referring to a technique such as " Imagination Computer Services Ges.m.b.H. ", "iotracker"

The display unit 160 includes a screen 161 positioned behind the infrared camera 150 and at least one projector 162 or 163 for projecting an image on the screen 161. The projectors 162 and 163 may be implemented by one or two projectors.

When the projectors 162 and 163 are provided as one unit, the main controller 210 controls the first polarized image corresponding to the movement of the first infrared ray emitting unit 131 and the second polarized image corresponding to the movement of the second infrared ray emitting unit 141 Thereby generating a second polarized image. The first polarized light image and the second polarized light image can be polarized by different polarizations and projected from the one projector 162 or 163 to the screen 161 alternately. To this end, the projectors 162 and 163 may additionally include a rotatable polarizer for alternately projecting another polarized image.

When the two projectors 162 and 163 are used, the first projector 162 that projects the first polarized image corresponding to the movement of the first infrared ray emitting unit 131 to the screen 161, and the second infrared ray radiator And a second projector 163 that projects a second polarized image corresponding to the movement of the first polarized light image 141 and the second polarized light image to the screen 161. The first polarized light image and the second polarized light image are polarized with different polarization components, 161, respectively. Therefore, a polarizing filter fixed to each of the projectors 162 and 163 may be installed in the first projector 162 and the second projector 163 to project images of different polarization components.

Accordingly, the glass (such as the HMD or safety goggles) provided in the first head mount apparatus 130 worn by the first user 110 can be watched for the polarization-adjusted image with the first polarized light component (for example, "P" polarized light) And the glass provided on the second head mount apparatus 140 worn by the second user 111 is provided with a polarizing film for the polarized light of the second polarized light component (for example, "S" polarized light) A polarizing film may be provided to enable viewing.

The transportation system simulation system according to the embodiment of the present invention includes an infrared image control unit 170 for receiving and processing images of the infrared ray emitting units 131 and 141 taken by the infrared camera 150, A control unit 180 for receiving and processing operation data generated when the control unit 111 controls the control unit 120 and the operation of the head mount devices 130 and 140 worn by the user 110 A projector control unit 200 for controlling the operation of the projectors 162 and 163, and a main control unit 210. The head control unit 190 controls the operations of the head-mounted device control unit 190, the projectors 162 and 163,

The infrared image controller 170 implements a tracking algorithm for estimating the positions and directions of the plurality of infrared ray emitting units 131 and 141 taken by the plurality of infrared cameras 150. This tracking algorithm can be composed of feature detection & estimation, feature matching, and 3D motion estimation.

The feature point prediction and extraction algorithm is for extracting the center coordinates of the feature point after predicting the feature points of the 2D image plane from the result data of the 3D motion estimation algorithm and the feature point matching algorithm is consecutively performed considering the shape and number change of the feature points And is used to track feature points simultaneously viewed in perspective on the image plane of a plurality of infrared cameras 150 to track feature points. The 3D motion estimation algorithm is for estimating the position and the posture of the infrared light emitting units 131 and 141, that is, the head mount units 130 and 140 of the user.

The steering control unit 180 receives and processes the operation request signal of the transportation device generated when the user operates the control unit 120 and transmits the operation request signal to the main control unit 210. The head mounting control unit 190 controls the head- When an additional display is mounted on the glass of the head mounting apparatuses 130 and 140, an image necessary for the display is received from the main control unit 210 When a gyro sensor or the like is mounted on the head mounts 130 and 140, a signal indicating the movement of the users 110 and 111 is received from the head mounts 130 and 140, And transmits the data to the unit 210.

The projector control unit 200 controls the functions of transmitting the video signals transmitted from the main control unit 210 to the respective projectors 162 and 163 and other functions generally required in the projectors 162 and 163.

The main function of the main control unit 210 is to generate a video signal to be output to the projectors 162 and 163. That is, a basic background image and various condition images applied to the background image are merged, and an image with necessary effects for a simulation operation is generated.

Hereinafter, an embodiment of a method of controlling a transportation device simulation system according to an embodiment of the present invention will be described.

3 is a flow chart for explaining the operation of the transportation device simulation system according to the embodiment of the present invention.

3, the control method according to the embodiment of the present invention sets the reference position of the infrared ray emitting units 131 and 141 (S100), detects the movement of the infrared ray emitting units 131 and 141 (Step S120), generating an image corresponding to the tracking result (step S130), and outputting the generated simulation image (step S140) ).

The step S100 of setting the reference position of the infrared ray emitting units 131 and 141 is a step of setting an image to be output from the projectors 162 and 163 at the beginning of the operation and an operation start position for simulation training of the user . For example, in the case of a simulation training for an aircraft, when the user 110 (111) is seated in the control unit 120 while the aircraft is on the ground, the position can be set to a position where the operation of the simulated training simulation system is started have.

When the simulator training is started by using the simulator system, the infrared camera 150 transmits infrared light to the infrared light emitting unit 131 (FIG. 1) of the head mount apparatus 130 (140) worn by the first user 110 and the second user 111 (S110), and drives the tracking algorithm using the sensed image to calculate the three-dimensional position movement of the user (S120). The tracking algorithm can be implemented by feature point prediction and extraction algorithm, feature point matching algorithm, and 3D motion estimation algorithm.

The feature point prediction and extraction algorithm is an algorithm for predicting feature points of the head mount devices 130 and 140 using the estimated three-dimensional movement of the head mount devices 130 and 140 and then extracting the center points of the feature points. At this time, the center coordinate data of the feature points of the time t is used in the 3D motion estimation algorithm thereafter. Since this algorithm is implemented using an extended Kalman filter, it is possible to estimate the movement of the head-mounted device 130 (140) at time t and time t + 1. Accordingly, when the motion of the three-dimensional head mount apparatus 130 (140) at time t + 1 is converted into the two-dimensional motion of the image plane photographed by the infrared camera 150, the image coordinates of the feature point to be acquired at time t + can do. Perspective perspective method can be used to convert the three-dimensional coordinates of the minutiae to two-dimensional image coordinates.

The feature point matching algorithm is intended to track the feature points of each time by assigning unique numbers to the feature points of the head mount apparatuses 130 and 140. The feature point matching can be divided into finding the correspondence of the feature points formed on the image planes of the two infrared cameras 150 and matching the feature points on the previous time and the current time. The algorithm can be implemented by distinguishing the case of the former by the stereo matching and the latter case by the feature point map matching. The stereo matching algorithm is implemented so that the response points of feature points can be matched even when the number and types of feature points formed on the image plane of two infrared cameras 150 are different. The feature point map matching algorithm can be designed to track a feature point by assigning a unique number to a feature point even if the number and shape of the stereo-matched feature points change.

The 3D motion estimation algorithm is implemented using an extended Kalman filter and is performed using the resultant data of the feature point prediction and extraction algorithm and the feature point matching algorithm. The state variable vector consists of positional variables, velocity variables, and incremental angles in the head coordinate system for the camera frame. The rotation matrix for estimating the posture of the head mount devices 130 and 140 is configured with an incremental quaternion for calculating the Euler increment angle. Then, the current rotation matrix can be obtained by multiplying the rotation matrix of the previous time by estimating the incremental rotation matrix in each frame.

The three dimensional position of the head mount apparatus 130 (140) of the user 110 (111) and the view direction of the user 110 (111) can be estimated by using the tracking algorithm. Then, the infrared image controller 170 acquires data on the direction and position of the user's view by driving the tracking algorithm, and transmits the obtained data to the main controller 210. The main control unit 210 includes a central processing unit, a database, and a graphic card for processing the corresponding training image.

Accordingly, the central processing unit instructs the graphics card to generate an image corresponding to the user's view direction and position, and the image processing apparatus of the graphic card generates an image corresponding to the generated image and stores the image in the graphic memory And outputs an image through the rear projectors 162 and 163 (S120) (S130). FIG. 4 and FIGS. 5A, 5B and 5C are diagrams for explaining operation examples for such driving.

FIG. 4 is a view for explaining a state in which a frame of the cockpit displayed on the screen 161 of the transportation device simulation system is displayed, and FIGS. 5A, 5B and 5C are views for explaining a state in which a plurality of users change their visibility to be.

As shown in FIG. 4, when the simulation system is for simulation training on an aircraft, the image is output with the left or right frame portion of the aircraft obscured. At this time, when the actual aircraft is operated, the pilot can move the head to the left or right to secure the field of view of the obstructed area. However, in the conventional simulation system, the user 110 (111) does not provide the desired image in response to the movement of the user 110 (111). Further, when a plurality of simulated training pilots are aboard, the outputted images can not be classified according to the view of each pilot and can not be outputted.

However, in the embodiment of the present invention, when a plurality of users (110) 111 ride on a simulation system and perform simulated training, individual images optimized for the view and operation of each of the users (110) And simultaneously provide the optimized training images to the users 110 and 111. [0064] FIG.

For example, even if both users 110 and 111 are looking in the same direction as shown in FIG. 5A, since there is a difference in position between users, a different image should be actually provided. Accordingly, the first projector 162 and the second projector 163 can simultaneously output the first and second images having different polarization components to the images optimized for the view of the first user 110 and the second user 111, And projects it onto the screen 161.

At this time, since the first user 110 and the second user 111 wear head-mounted glasses capable of viewing images of different polarized components, the user views the optimized images independently of each other.

5B and 5C, when the first user 110 and the second user 111 change their viewing directions in opposite directions to each other, the infrared camera 150 transmits the first user 110 and the second user 111, The infrared ray emitting units 131 and 141 of the head mount devices 130 and 140 worn by the user 111 are detected to obtain tracking information on the user's movements and a new It is possible to provide an optimized simulated image according to the view of the user 110 (111) by projecting the image in real time on the screen 161 through the projector 162 (163).

Referring to FIG. 4 again, when the user moves his / her head forward to see the rear of the frame to secure a field of view on the left side frame of the aircraft, the infrared camera 150 detects So that a realistic simulated image can be provided by projecting an image of the back of the frame onto the screen 161. [

The embodiments of the present invention described above are for illustrative purposes only and are not intended to limit the present invention to the described embodiments. It is also to be understood that the detailed description of this specification does not limit the scope of the invention and that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and defined by the appended claims, To those skilled in the art.

100, 101 ... boarding section
110, 111 ... users
120 ... control section
130 ... first head mount device
131 ... First infrared ray emitting portion
140 ... second head mount unit
141 ... second infrared ray emitting portion
150 ... Infrared camera
160 ... display portion
170 ... Infrared image control unit
180:
190 ... Head mount apparatus control section
200 ... projector control unit

Claims (13)

A boarding section on which the user can board;
The infrared ray emitting part
An infrared camera positioned in front of the boarding portion and sensing movement of the infrared light emitting portion;
A control unit for generating an image corresponding to the movement of the infrared light emitting unit taken by the infrared camera;
And a display unit for outputting the image generated by the control unit.
The transportation system simulation system according to claim 1, wherein the display unit comprises a screen installed in front of the boarding section and a projector for projecting an image on the screen.
The transportation system simulation system of claim 1, wherein the boarding section includes a first boarding section on which the first user boards and a second boarding section boarded by the second users.
The transportation device simulation system according to claim 3, wherein the infrared ray emitting portion includes a first infrared ray emitting portion worn by the first user and a second infrared ray emitting portion worn by the second user.
The system according to claim 3, wherein the control unit generates a first image corresponding to the movement of the first infrared ray emitting unit and a second image corresponding to the movement of the second infrared ray emitting unit.
6. The system of claim 5, wherein the projector comprises a first projector for projecting the first image onto the screen and a second projector for projecting the second image onto the screen.
The apparatus of claim 3, wherein the controller generates a first image corresponding to a movement of the first infrared ray emitting unit and a second image corresponding to a movement of the second infrared ray emitting unit, And the other polarized light is polarized and projected by the projector in an alternating manner.
The apparatus of claim 3, wherein the controller generates a first image corresponding to the movement of the first infrared ray emitting unit and a second image corresponding to the movement of the second infrared ray emitting unit, Wherein the projector is polarized with different polarized light, the projector comprising: a first projector that projects the first image onto the screen; and a second projector that projects the second image onto the screen.
An infrared camera senses an infrared light emitting portion worn by the user,
Acquiring information on the viewing direction of the user according to the position of the infrared ray emitting portion sensed by the infrared ray camera,
And generating an output image matched with the information on the viewing direction and outputting the output image on a display unit.
10. The method according to claim 9, wherein the boarding section comprises a first boarding section on which the first user boards and a second boarding section boarding the second users.
The apparatus according to claim 10, wherein the infrared ray emitting unit includes a first infrared ray emitting unit worn by the first user and a second infrared ray emitting unit worn by the second user so that a first image corresponding to the motion of the first infrared ray emitting unit Generating a second image corresponding to the motion of the second infrared ray emitting unit, and outputting the second image to the display unit.
10. The apparatus according to claim 9, wherein the display unit includes a screen provided in front of the boarding section and a projector for projecting an image on the screen, the first projector projecting the first image onto the screen, And a second projector for projecting the light onto the second projector.
The display device according to claim 9, wherein the first image corresponding to the movement of the first infrared ray emitting part and the second image corresponding to the movement of the second infrared ray emitting part are polarized with different polarizations, A method of controlling a transportation device simulation system for displaying.

Images