KR102120553B1 - Multimode Driver Simulator for Multipurpose Vehicle using VR HMD and Monitor simultaneously - Google Patents

Abstract

The present invention relates to a driving simulator device for a special vehicle, and applies VR HMD (Head Mounted Display) as a basic video output device, but additionally arranges a monitor to selectively utilize the video output device according to the trainer's preference or to evaluate training instructors It can be configured to be applied. Head out driving by heading out of the hatch and heading out by closing the hatch and looking at the periscope inside, using the chair position and the motion recognition sensor device of the trainee The training video considering head in driving and perspective can be transmitted to the trainer in real time. When a head-in is driven through a separate small monitor placed inside or on the front periscope device, the actuator device is built in or installed under the chair assembly to show the appropriate driving image for each periscope position to the trainee and automatically transport the rear door and monitor. In addition, it may include a reaction force reproduction device that can transmit the driving vibration to the trainee.

Description

Multimode Driver Simulator for Multipurpose Vehicle using VR HMD and Monitor simultaneously}

The present invention relates to a driving simulator device for a special vehicle, and applies VR Head Mounted Display (HMD) as a basic image output device; By arranging additional monitors, it can be configured to be selectively used inside or outside the simulator system according to the training situation, so that the user can partially head out driving and head in according to the positions inside and outside the system. Driving) relates to a multi-mode driving simulator for a multi-purpose vehicle that can practice the operation of a variety of special-purpose multi-functional vehicles.

The existing driving simulator device provides a user or a trainee with the same driving experience as a real vehicle by manipulating various controls while viewing images in front of a vehicle displayed on a monitor, project, or screen in a structure space made of the same as a real vehicle. . A motion platform device is placed under the structure to simulate the driving vibration of the vehicle. At this time, the motion platform supports several hundred tons of superstructures and trainers while generating various dynamic posture control including vibration. It is common. In order to control the motion of such a heavy object, a separate control device for controlling the motion platform with high power in proportion was required. In addition, it was common to have a separate control facility by adding functions such as simulator device control and training situation monitoring. This has the advantage of strengthening facility safety through the application of multiple control devices, and simultaneously monitoring, evaluating and controlling multiple simulator training situations if necessary. However, on the contrary, having to go through a separate control facility every time a control operation such as driving and stopping of an individual simulator was obviously not a good method in terms of user convenience, and excessive initial cost was also a problem.

With the recent advancement in technology, introduction of virtual reality (VR), augmented reality (AR) and augmented reality (MR) and mixed reality (MR) has been actively performed, and simulators using this are also applied in various fields. have. With excellent image reproducibility, it is possible to increase the user's immersion compared to the existing method, and it is also possible to expect the effect of cost reduction by replacing the hardware structure with images. Among them, virtual reality technology is the most efficient to improve the immersion of the user, but when VR HMD is worn, it blocks the view in front of the user's eyes. You need to do it. In addition, devices and contents such as virtual reality are still in the introduction stage, and some users are unfamiliar with the application or even feel dizzy.

In addition, in the case of special or multi-functional vehicles, depending on the situation, the door or hatch of the operating system is opened and the head out driving and the door or hatch are closed and the system is operated or operated in a situation where a part of the operator's body comes out of the vehicle or system. In addition, there is a method of head in driving that is operated or driven through an in-vehicle device, an indirect device such as a periscope, or imaging equipment, but in the case of an existing training system or simulator, the head is pushed out of the vehicle. Looking at the head out driving training was only possible.

The present invention was created to solve and improve the problems of the existing training system or device, simulator as described above, and applies virtual reality technology as a basis to provide cost savings in system configuration and operation by reducing user immersion and hardware production. However, it is intended to provide a universal simulator device capable of satisfying some users who are not familiar with virtual reality technology. In addition, while providing a means to facilitate head-out driving and head-in driving, separate control devices and facilities are removed to reduce costs and ease of use, but the simulator device itself monitors all device controls and training instructors. Let's provide functions such as reviews and various functions for user convenience during simulator training.

The present invention for achieving the above object, the main idea is to first apply VR HMD as a basic video output means to improve the realism of training and immersiveness of users compared to existing methods with excellent image performance of virtual reality technology and to touch the body parts Outside of this area, the hardware production of the system or device to be trained is minimized to reduce cost, weight and volume. Feedback with external environments such as a target system or device, for example, a vehicle driving vibration may be replaced with a VR image capable of realizing virtual reality. In addition, a reaction device that combines a plurality of actuators may be applied to a chair or motion platform to which a user's body is contacted or weighted. Additionally, a monitor device is arranged for a user who prefers the conventional video output method, so that the VR HMD mode and the monitor mode can be selectively used. In addition to widening the range of user selection through the additional arrangement of the monitor, when training in VR HMD mode, the training image is displayed on the monitor in the same manner, thereby providing a monitoring image for the training instructor. In VR HMD, both driving training is possible by transmitting the image in front of the vehicle suitable for each of the head-in driving and head-out driving modes, and in the case of head-out driving during training in the monitor mode, it is out of the hatch as in the conventional simulator. When you head out, you train while looking at the images in the monitor. When you head in, you close the hatch and perform driving training while looking at the external monitor through a specially designed periscope. Two driving or operating modes can be implemented according to the user's selection. In addition, it may be set by sensing values of relative positions of the inside and outside of the structure assembly by a VR HMD or a motion recognition sensor device. By removing the high-power motion platform device and incorporating the monitoring function itself, a separate control device such as the existing one becomes unnecessary, and the simulator device itself can fully provide all functions.

As an embodiment for achieving the object of the present invention, a simulator device capable of simulated driving training of a special vehicle has a structure in which components can be arranged, and the components can be moved up and down or back and forth by a chair assembly and a user It may be configured to include a control device capable of transmitting a physical manipulation and a mechanical or electrical manipulation signal according to the instrument panel device indicating status information of the simulator device, and the like, and can exchange electrical signals with at least one of the constituent devices. A motion recognition sensor device for recognizing a user's relative position and motion in the structure, a control device for processing and controlling information and signals of the simulator device including the control device, the instrument panel device, and the motion recognition sensor device, and It may be configured to include a video output device including a VR HMD interlocked with the control device and a monitor disposed outside the structure, and selectively use the video output device using information of the motion recognition sensor device. By doing so, it is possible to implement a multi-mode simulator device.

The simulator device according to the present invention includes a structure assembly 1 in which the inside and the outside are visually divided and predetermined components are disposed; A control device 100 including driving control means such as image, sound, sensor signal processing, and actuator of the device, including control operation software (O/S); A moving device 200 that allows the device to move freely and be fixed and used at a desired position; The components include a steering device, an accelerator pedal, a brake pedal, a parking pedal, and a steering device 300 according to characteristics of the system, such as a transmission gear; In addition, an auxiliary device 700 such as a hatch device 701, a periscope device 702; Instrument panel device 800; And the like. In addition, a system, apparatus, or vehicle that may be selectively provided may further include a chair assembly 400 for transmitting feedback, vibration, etc. according to interaction with external environments, or supporting the user or trainer's riding; A motion recognition sensor device 500 that recognizes a user's motion and position change; An output device 600 such as a VR HMD 601 and a monitor 602 that displays training images; It may be provided with a configuration.

The structure assembly 1 is generally made of the same size of space and shape as actual equipment, thereby providing the trainee with a feeling of use such as the same spatial sense as the actual vehicle. During driving training, the linear actuator 11 may be vertically raised to position the monitor 602 in front of the trainee's field of view, and otherwise may be lowered for reasons such as convenience of storage and movement to be positioned within the structure. The rear door 21 is also constructed with a structure that can be opened and closed by applying the actuator 22, through which it can be opened when used, allowing the trainee to step on the upper part and board into the simulator device, and closed before and after use. Make it available for storage. At this time, the actuator 22 for the rear door can open/close the rear door 21 through expansion and contraction of the actuator by applying a linear actuator as shown in FIG. 2, and the rear door 21 and It is also possible to construct an open/closed structure by applying a rotary actuator to the hinge portion of the structure.

The control device 100 has built-in independent operation software, and provides video and sound for training in real time to the trainee while driving and controlling various sensors and actuators in the simulator device, and a network function that connects the simulator to the outside. It is responsible for the back. The control device may be largely divided into a main control device 101 and an electric control device 102 for each function as shown in FIG. 5. The main control device 101 includes operating software (O/S) and hardware including a GUI so that the simulator device can be started and operated, and the trainee applies the selected training course or scenario after driving the simulator device. It provides video and sound. At this time, in order to reproduce realistic images that apply the same physical laws as reality, such as accurate vehicle movement, gravity, and interference, the internal software is equipped with a physics engine with a built-in vehicle dynamics model. If necessary, it can be connected to other monitoring and evaluation systems and provides joint training functions by interworking with other simulator devices through a network. In addition, when inputting a signal from a motion sensor sensor 500 that detects a movement or position change of a controller sensor or a trainee according to a trainer's operation, a position and motion change signal is transmitted from the electric control device 102 to receive an image. It also reflects the change in the visual viewpoint according to the movement of the trainee's head to the trainee in real time. In addition, the movement of the vehicle according to the operation of the steering device is analyzed and implemented in the physical engine in consideration of the corresponding driving situation. At this time, the driving vibration of the vehicle is calculated in real time and transmitted to the electric control device 102 to be disposed together in the chair assembly. The reaction force according to the movement and vibration of the vehicle may be transmitted to a trainee in real time through a reaction force reproducing device. The electric control device 102 is disposed on a control device sensor, a monitor actuator 12, a rear door actuator 22, a reaction force reproducing device, and other chair assemblies 400 and ancillary devices 700 to change the posture of the device. It supplies the necessary power to various sensor devices recognizing and processes the electrical signals of the sensors, converts them into real physical values, delivers them to the main control device 101 so that it can be fed back to the user, and operates the user's GUI or physically. The control of the actuator is performed by receiving a signal from the main control device 101 generated by the switch operation. The actuator may include all actuator devices in the simulator device, such as the rear door, monitor, and reaction device for the chair assembly, and if necessary, detect the change in the length of the actuator in real time through the placement of the distance sensor 13 in the linear actuator, and then control the electricity again. It is possible to control a change in the length of the actuator as desired through the device 102, the main control device 101, and the electrical control device 102 in order.

The moving device 200 is disposed at the lower end of the structure 1 so that the simulator device can be freely moved and fixed to a desired position.

The control device 300 includes all mechanism devices capable of controlling the movement of the vehicle during the driving training by the trainee, and all individual control devices include a sensor device 311 capable of determining whether the control device is operated or not and the degree of operation. The mechanism device is a steering wheel, a shift gear, an accelerator pedal, a brake pedal, a parking pedal, and an operation lever for displaying a driving condition. The steering device is equipped with a sensor device that can determine whether to operate the steering wheel by changing the rotation of the steering wheel, changing the gear stage of the transmission gear, changing the position or rotation of the acceleration/braking/parking pedal, and other levers. 300). The sensor electrical signal generated when the control device 300 is operated is passed to the electric control device 102 to be converted into a physical value such as the steering angle of the steering wheel, which is then passed back to the main control device 101 to rotate the steering wheel. The vehicle in the image is also moved to the steering angle corresponding to the steering wheel angle. In addition to the sensor detecting the distance, rotation angle, position change, and the like, as shown in FIG. 4, the sensor device is a microprocessor in charge of controlling the sensor and a circuit capable of filtering the electrical signal of the sensor, and, if necessary, an analog electrical signal digitally. And an electrical circuit that can be converted and transmitted wirelessly.

The chair assembly 400 generally provides up/down, front/rear position movement and backrest adjustment functions so that the trainee can sit and train in a comfortable position to himself, and in this case, in the present invention, the position movement unit and the backrest adjustment rotation unit The sensor device is arranged to send a signal to the electrical control device 102 when the position or posture changes. Through this, it is possible to determine whether it is head-out driving training by raising the chair or head-down driving training by lowering the chair, and a training image in consideration of perspective can be transmitted on the VR HMD, and the changed position and posture of the chair is also provided to the trainee as an image. Give feedback. In addition, a reaction force reproducing device may be additionally installed under the chair in order to transmit a reaction force due to vibrations on the road surface when driving. The reaction force reproducing device is a device that consists of a plurality of actuators and reproduces the reaction/vibration/shock caused by unevenness of the vehicle, unevenness of the road surface, and the reaction force transmitted to the occupant during steering. When vibration/impact of the vehicle is caused by manipulation, the vibration profile value for the corresponding condition is calculated by the main control device and transmitted to the reaction force reproduction device for reproduction.

The user's motion recognition sensor device 500 collectively refers to all sensor devices capable of detecting a posture and position change of a trainer, such as an optical, mechanical, magnetic, and inertial measurement method. Sensors are attached to various parts such as the user's hands, feet, body, and head, or optical devices are placed in a position to check the shape of the trainer to detect changes in the position and posture of the trainer. With the relative position information of the inside and outside of the system or device, the mode can be switched based on the actual user's position in the operation or driving of the system. In addition, when the position and posture change, it is transmitted to the main control device and implemented as a video change of the trainee's avatar. The trainee operates various controls and realistic training while watching the avatar acting the same as the one superimposed on the train's video. can do.

The training image is displayed through the output device 600 such as the monitor 602 and the VR HMD 601. The output device can be selected and used as the monitor 602 or VR HMD 601 according to the taste of the trainee. Even if the VR HMD 601 is selected for training, the monitor 602 is the same as the trainer or observer. A video or an instructor or observer can selectively monitor the training video by manipulating the video view. The training sound may be simultaneously output from the monitor 602 and the VR HMD 601, or may be output by installing a separate speaker.

In the case of military vehicles, in general, a trainee opens the hatch 701 and drives the head out of the structure (head-out driving) or closes the hatch 701 and closes the daytime periscope 702 or the thermal periscope monitor 703 ) Or through the front/rear camera monitor 704, driving while looking at the front/rear (head-in driving). In the present invention, devices such as the hatch assembly 701, the periscope assembly 702, the thermal periscope monitor 703, and the front/rear camera monitor 704 through the auxiliary device 700 are configured in the same manner as the actual vehicle structure. . In addition, a sensor device capable of confirming the degree of opening and closing of the hatch is attached to the hatch rotating part 711, and when the state changes, the sensor signal is transmitted to the main control device 101 through the electrical control device 102 to move the actual hatch. This can be reflected as it is. The trainee can perform head-out and head-in driving in the same manner as in a real vehicle using the simulator of the present invention, and the method is as follows. For head-out driving, first open the hatch 701, then train the chair assembly 400 to a position where the trainee's field of view is fully visible while driving, and watch the monitor according to the trainer's preference or wear VR HMD to train. do. The control device 100 detects whether the position of the chair has changed through a signal from a sensor disposed in the position adjusting unit of the chair assembly 400, and grasps the position of the user's eye level through the motion recognition sensor device 500 of the trainer The corresponding driving image is transmitted to a trainee wearing VR HMD. In the case of head-in driving, after closing the hatch and descending the chair, you can drive while watching the monitor screen through a periscope or wear VR HMD to train. Here, when viewing an external monitor through the periscope device 702, for example, in a device composed of three periscopes of left, center, and right as shown in FIG. 1, the central periscope faces the monitor, but the screen is full of view. There is a problem that is not located in the center, and there is a problem that is not seen at all in the left/right periscope. Accordingly, a separate small monitor may be attached to the inside of the periscope device or to the front of the device, thereby displaying an image suitable for each periscope position to the trainee. The control device 100 determines whether the chair position has been changed through a signal from a sensor disposed in the position adjusting unit of the chair assembly 400, and the user's eye level position is detected through the motion recognition sensor device 500 of the trainer. Appropriate forward driving video is sent to the trainee wearing VR HMD. When wearing a VR HMD, when an avatar approaches with a periscope in virtual reality, you can see the landscape in front of the vehicle through the periscope to reproduce realistic training.

The instrument panel device 800 basically displays basic operation information such as vehicle speed, engine RPM, number of gears, and engine driving time on a panel such as an LCD or LED, and supplies electricity in the vehicle, starts the engine, and controls other vehicles. Switch and touch screen. On a panel such as an LCD or LED, real-time driving operation information of a vehicle calculated by the main control device can be expressed in graphs, text, etc., or a simple alarm or other information about the vehicle situation can be displayed in a simple phrase or picture.

According to the present invention described above, it is possible to selectively drive training in a head-in or head-out manner using a VR HMD or a monitor according to the user's selection or location information, and increase the effectiveness of training through additional devices such as a periscope device. At the same time, all the control of the simulator device is possible with its own control device inside the simulator without the need for additional external control facilities, and it is possible to freely move and fix, thereby maximizing user convenience and reducing development costs.

1 is an overall three-dimensional plan view of the present invention
2 is a rear view of the present invention
Figure 3 is a three-dimensional interior plan view of the present invention (excluding structures and devices near the structure)
4 is an internal side view of the present invention
5 is a control device configuration of the present invention
Figure 6 is a flow chart for explaining the operating method of the present invention

Hereinafter, a preferred operation example of the present invention will be described in detail with reference to the accompanying drawings. However, the working examples of the present invention are provided to more fully describe the present invention, and the actual implementation may be modified in various other forms. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on a user's or operator's intention or practice, and thus the definition should be made based on the contents of the present specification describing the present invention.

1 to 4 are diagrams of a simulator device for implementing the above-described idea. The simulator device is equipped with both a VR HMD 601 and a monitor 602, and is equipped with a video output device that can be selected according to a user's preference. At this time, the VR HMD 601 or the physical engine in the main control device 101 or Training images corresponding to each of the monitors 602 are provided. During training when the monitor 602 is selected, the scenery and driving image in front of the vehicle are continuously supplied to the monitor, but when operating the steering device 300 such as the steering wheel of the trainee, the acceleration/braking pedal, and the shift lever 300, it is disposed on each control device. The sensor senses the movement and transmits the electric signal to the electric control device 102. The signal is converted into physical values such as the steering angle of the steering wheel, the pedal operating angle, and the gear stage of the shift lever to the main control device 101. Will be delivered. The main control device 101 recalculates the movement change of the vehicle reflecting this and sends it to the monitor 602 to provide feedback to the trainee. When training VR HMD (601) is selected, the motion recognition sensor device (500) detects the position of the trainee and the direction in which he or she is looking at it, and transmits the corresponding image from the main control device (101). It gives the user a sense of reality like doing The vision problem that is obscured due to the wearing of the VR HMD 601 is a corresponding device from sensors disposed in the motion recognition sensor device 500 and the sensors 300, the chair assembly 400, the hatch assembly 701, and the like. It is solved by detecting the change of motion and reflecting it in VR HMD (601) image in real time. In addition to the image, a reaction force reproducing device composed of a plurality of actuators may be disposed under the chair assembly 400 to simulate the driving vibration of the vehicle, thereby making it possible to further increase the immersion of driving training. It calculates the vehicle's movement and driving vibration on the main control device 101 based on the surrounding environment such as the road surface at the time, the vehicle condition, and the operating condition of the trainee's control device 300 detected through the sensor, and based on this, the reaction force reproduction device It is reproduced through the individual actuator control. After all the training is completed, after the power is shut off, the hatch assembly 701 is closed and the chair assembly 400 is moved into place, and then the monitor 602 descends and moves into the structure and the rear door 21 is closed to store desired storage. Move the simulator device to a location and store it.

The existing simulator device required a separate dedicated control device for this to apply a large-capacity motion platform device that generates driving vibration while supporting the entire heavy structure device, and a separate control device for integrated control and monitoring of the training situation. A control room was needed. The driving simulator of the present invention reduces the system burden by minimizing the hardware application compared to the existing by applying VR virtual reality as the basic video output medium, and removes the motion platform or reduces the capacity of the chair assembly 400 with excellent immersion of virtual reality. It is possible to mount all control devices inside the simulator device by removing the control devices that were needed separately by applying only to. Individual simulators have built-in software that can be operated independently by the control device, and when training using VR HMD, the monitors for training instructors can be displayed and monitored through the monitor. A separate dedicated control device and facility are unnecessary and all are included in the simulator device, but interworking with an external system and network connection include functions within the control device, so it can be used if necessary.

According to the present invention having such a structure, a VR HMD and a monitor can be selectively used according to a trainer's preference for driving training for a special vehicle, and both head-in and head-out driving training are possible. Unlike the existing one, all control functions are embedded in the simulator device without the need for a separate control device and facility, and monitoring functions for training instructors are provided.

6 is a flowchart in which the main idea of the present invention is implemented. After moving and fixing the simulator device to a position to be trained by using the moving device 200, the actuator is operated to open the rear door 21 and move the monitor 602 upwards. When driving the simulator, the trainer can select VR HMD (601) or monitor (602) according to his preference, and open the hatch, raise the head out, and train the head out or seal the hatch and periscope inside. You can also choose a driving method such as head-in driving while watching and training. When training VR HMD 601 is selected, the chair height and steering wheel angle are adjusted according to the head-out or head-in driving method, and the hatch is opened or closed, and training is performed according to the image in the VR HMD worn. Through the monitor image, training instructors can monitor and evaluate the training contents. When training the monitor 602 is selected, the chair height and the steering wheel angle are also adjusted according to the head-out and head-in driving methods, and training is performed while the hatch is opened or closed and the monitor screen or periscope is viewed.

The control device 100 recognizes the position of the trainee's eyes in real time through signal analysis from the sensor and the motion recognition sensor device 500 arranged in the position adjusting unit of the chair assembly 400 to drive head in or head out and perspective. It transmits the training video considered by the back. In addition to the periscope, it is possible to observe the thermal periscope monitor 703 or the monitor 704 for the front/rear cameras and perform head-in driving training using the monitor.

It is understood that the embodiments shown for the purpose of describing the present invention are only one embodiment in which the present invention is embodied, and various types of combinations are possible to realize the gist of the present invention as described above.

Therefore, the present invention is not limited to the above-described embodiments, and various modifications can be made to anyone having ordinary knowledge in the field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims below. It will be said that there is a technical spirit of the present invention to the extent possible.

1: Simulator structure assembly 11: Monitor actuator device
21: rear door 22: actuator device for rear door
100: control device 101: main control device
102: electric control device 200: mobile device
300: control device 400: chair assembly
500: motion recognition sensor device 600: output device
601: VR HMD 602: Monitor
700: Accessories 701: Hatch assembly
702: daytime periscope device 703: thermal periscope monitor
704: Front/rear camera monitor 711: Hatch rotating part
800: instrument panel device

Claims (7)

In the simulator device capable of simulated driving training of a special vehicle,
A structure in which components can be arranged;
The configuration device
Chair assembly that can be moved up and down or back and forth;
In the chair assembly, a reaction force reproducing device composed of a plurality of linear actuators is disposed underneath,
A control device capable of transmitting a physical manipulation and a mechanical or electrical manipulation signal by the user;
An instrument panel device showing status information of the simulator device; With back
A motion recognition sensor device that exchanges electrical signals with at least one of the components and recognizes a user's relative position and motion in the structure;
A control device for processing and controlling information and signals of the simulator device including the control device, the instrument panel device, and the motion recognition sensor device;
A video output device including a VR HMD interlocked with the control device and a monitor disposed outside the structure; It is possible to selectively use the image output device using the information of the motion recognition sensor device, including;
The control device automatically transmits a training image in consideration of head-in or head-out driving and perspective by determining the position of the trainee's eye from the sensor disposed in the position adjusting portion of the chair assembly and the signal from the motion recognition sensor device. Giving is a multi-mode simulator device.
The method of claim 1, wherein the configuration device may further comprise a periscope device,
A multi-mode simulator device, characterized in that it comprises a separate monitor inside the periscope device or on the front of the device.




delete delete According to claim 1, The structure,
Multi-mode simulator device that can be used to match the eye level during training by applying an actuator that can automatically raise or lower the monitor, and to descend and store it inside the structure during training.



delete delete

Images