JPH08289227A - Virtual environment display device - Google Patents

Abstract

PURPOSE: To provide an inexpensive head mounted visual display device with which any feeling of restriction is not received. CONSTITUTION: A computer 2 for control generates the signal of a three- dimensional image from any arbitrary software package to a computer 1 for VR image signal generation based on signals from a gyro sensor inside a head mounted visual display device 4, gyro sensor inside space joy stick 5, pressure detection part, position/torque detection part inside a training machine 6 and optical card recording and reproducing device 3 and the operation of an operating part provided at the computer 2 for control. The direction of a glance for generating the three-dimensional image is decided by information showing an attitude from the gyro sensor inside the head mounted visual display device 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a virtual environment display device using a head-mounted visual display device, and more particularly to a signal indicating the line-of-sight direction of an image representing the virtual environment or the position and posture of a virtual hand. It relates to a means for realizing a signal.

[0002]

2. Description of the Related Art A so-called virtual reality method has been developed as a simulation system for displaying a three-dimensional image on a head-mounted visual display device to give a virtual environment. For example, as a training system for driving a car or an aircraft, it is known that a virtual environment is changed by an operation of a steering wheel or the like by an experienced person.

In virtual reality, a three-dimensional image is displayed on a head-mounted visual display device. The head-mounted visual display device is called an HMD, and an example of its basic structure is shown in FIG. A convex lens is arranged at a position corresponding to both eyes in the body shown in the figure. Left and right eye liquid crystal panels (LCDs) are arranged behind the left and right convex lenses, respectively, and a cover is attached behind them. When such an HMD is mounted on the head, an image on the LCD with left and right parallax can be seen enlarged in both eyes. You can see a stereoscopic image because of the left and right parallax.

The image on the LCD is created by a computer graphic. That is, an image for the right eye and an image for the left eye in which the background object set on the world coordinates is viewed from the viewpoints of the right eye and the left eye in arbitrary line-of-sight directions is calculated, and the LCD for the left eye and the LCD for the right eye are respectively calculated. Projected on. Thus, it is necessary to determine the line-of-sight direction in order to create an image of the virtual environment. Conventionally, the HMD is hung from the outside by a link so that the virtual world can be viewed according to the movement of the head, a sensor such as a potentiometer for detecting a rotation angle is attached to the link, and the line-of-sight direction is determined by the signal of the sensor.

Further, in order to show an image of a virtual hand in the image of the virtual environment, a signal indicating the position and orientation of the object held by the hand is required, and for that purpose, a link has been conventionally used.

[0006] In this way, if the HMD or the object gripped by the hand is supported by the link, a sense of discomfort is felt when the head or the hand is moved. In order to eliminate this discomfort, there is a Polhemus sensor as a non-contact spatial position sensor using magnetic induction, but it is very expensive.

[0007]

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and its purpose is to determine the position of the head and hands in virtual reality. As a sensor for obtaining posture information, it is an object to provide an inexpensive means that does not give a feeling of restraint.

[0008]

A virtual environment display device of the present invention is a head-mounted visual display device for providing a virtual environment, and an image signal for generating an image signal and outputting the image signal to the head-mounted visual display device. In a virtual environment display device including a generation device, a line-of-sight direction of a virtual environment image is determined according to a signal indicating a posture from the gyro sensor by attaching a gyro sensor to the head-mounted visual display device, and the image signal is It is configured to be generated.

Further, the virtual environment display device of the present invention comprises a head-mounted visual display device for providing a virtual environment and an image signal generation device for generating an image signal and outputting the image signal to the head-mounted visual display device. In the provided virtual environment display device, the viewpoint position and line-of-sight direction of the virtual environment image are determined according to a signal indicating the position and orientation from the gyro sensor by attaching a gyro sensor to the head-mounted visual display device,
The image signal is generated based on the viewpoint position and the line-of-sight direction.

Further, the virtual environment display device of the present invention includes a head-mounted visual display device for providing a virtual environment, and an image signal generation device for generating an image signal and outputting the image signal to the head-mounted visual display device. In a virtual environment display device including a space joystick for adding an image of a virtual hand to the virtual environment, a gyro sensor is attached to the space joystick, and the virtual joystick is responsive to a signal indicating a position and a posture from the gyro sensor. The configuration is such that the position and posture of the image of the hand change.

[0011]

Since the top of the gyro sensor rotates at a high speed and is supported inside the metal ring which can be tilted about mutually perpendicular axes, the attitude of the top remains unchanged even when the main body is tilted. Therefore, the rotation angle of the main body around the three axes with respect to the fixed coordinates is detected from the relative inclination between the top and the main body. On the other hand, the acceleration in the three axial directions fixed to the main body can be detected by the accelerometer.

The acceleration of the main body with respect to the fixed coordinates is calculated from the acceleration thus detected and the rotation angle of the main body around the three axes. The position of the main body on the fixed coordinates is calculated by integrating the acceleration with respect to the fixed coordinates twice.

By determining the line-of-sight direction of the virtual environment image from the signal indicating the rotation angle from the gyro sensor attached to the head-mounted visual display device, the virtual world can be overlooked according to the movement of the head. Further, by determining the line-of-sight position of the virtual environment image based on the signal indicating the position from the gyro sensor, it is possible to obtain the sensation of traveling in the virtual world.

Similarly, a gyro sensor is attached to the space joystick, the position of the virtual hand image is determined by the signal indicating the position, and the posture is determined by the signal indicating the rotation angle. You can move your hands freely. Since the gyro sensor does not need to be supported by a link, it does not feel that the head or hand is restrained.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a rehabilitation and diagnosis apparatus for a person with cerebral dysfunction using a virtual environment display apparatus according to an embodiment of the present invention. Reference numeral 1 shown in the figure is a computer for generating a VR image signal, which stores a plurality of software packages in an internal hard disk.
Reference numeral 2 denotes a control computer for controlling the VR image signal generating computer 1, the optical card recording / reproducing device 3, the space joystick 5, the gyro sensor in the space joystick 5, and the training machine 6.

The control computer 2 is connected to the VR image signal generation computer 1 by an RS-232C cable 8. Then, the control computer 2 has a gyro sensor in the head-mounted visual display device 4, a gyro sensor and a pressure detection unit in the space joystick 5, a position / torque detection unit in the training machine 6, and an optical card recording / reproducing device 3.
The VR image signal generation computer 1 is caused to generate a stereoscopic image signal from an arbitrary software package based on the signal from the above and the operation part of the control computer 2 and to generate a three-dimensional audio signal. The stereoscopic image is displayed on the display of the head-mounted visual display device 4, and the headphones are driven by the three-dimensional audio signal.

A background object is set on the world coordinates in the data of the software package, and the background images for the right eye and the left eye are calculated by giving an arbitrary viewpoint and the direction of the line of sight. Further, it also has image data of a virtual arm or the like to be superimposed on the background image, and the image of the image data is an arbitrary image at an arbitrary position on the background image by a signal indicating the position and orientation of the gyro sensor in the space joystick 5. It is projected in a posture.

The head-mounted visual display device 4 includes a display for displaying a stereoscopic image, headphones and a gyro sensor. The RGB signal of the image output from the computer 1 for generating the VR image signal is sent to the scan converter 14 by the pin cable 12, and the scan converter 14
Is converted into an image signal (composite signal) for display of the head-mounted visual display device 4. The image signal is input to the display of the head-mounted visual display device 4 via the connector 13.

The three-dimensional audio signal output from the VR image signal generating computer 1 is input to the headphones of the head-mounted visual display device 4 via the pin cable 11 and the connector 13. A signal indicating the posture of the gyro sensor in the head-mounted visual display device 4 (a signal indicating a rotation angle around three axes) is converted into an RS-232C signal by the decoder 15 and is transmitted via the RS-232C cable 9. It is input to the control computer 2.

The space joystick 5 is provided with a pressure detection unit and a gyro sensor, and a signal output from the pressure detection unit indicating the grip of the hand and a signal indicating the position / posture from the gyro sensor (position in the XYZ axis directions and Three
The signal indicating the rotation angle around the axis is the decoder / converter 1
6 converted to RS-232C signal, RS-23
It is input to the control computer 2 via the 2C cable 9.

The structure of the training machine 6 is shown in detail in FIG. That is, the AC servomotor 61 rotationally drives the output shaft 64 via the speed reducer 62, the drive belts 63, 63, ..., The torque limiter 70, or is driven by the output shaft 64. A crank 65 and a pedal 66 are attached to the output shaft 64. The rotation angle of the output shaft is detected by the rotation potentiometer 67, and the torque is detected by the torque detection unit 68 having the torque potentiometer 69. Further, the emergency stop button 71 is provided on the training machine 6.

As shown in FIG. 1, the training machine 6 is connected to the control computer 2 by a parallel cable 17, and the input / output relationship of signals between the training machine 6 and the control computer 2 is shown in FIG. Are detailed in. That is,
D / A converter (D / A) in control computer 2
The motor driver supplies current to the AC servomotor 61 based on the analog signal output from the AC servomotor 61. The signal from the torque detection unit 68 is converted into an input signal of the computer by the torque detection conversion unit and input to the A / D converter (A / D) in the control computer 2. Further, the signal from the rotary potentiometer 67 is converted into an input signal of the computer by the position detecting / converting unit, and the signal in the A in the controlling computer 2 is converted.
It is input to the / D converter (A / D). Further, the signal from the emergency stop button 71 is input to the interface (P I / O) in the control computer 2.

The training machine 6 is controlled by the control computer 2 to perform a training operation. The training operation has a passive mode and an active mode. In passive mode, it makes rotational movement at the set speed. In the active mode, the lower limit of the input torque is set, and only when the input torque exceeds the set torque, the rotational movement is performed while the rotation is changed by the torque difference.

The optical card recording / reproducing apparatus 3 is connected to the control computer 2 via the SCS1 cable 7,
Each patient reads the recorded contents of the optical card, outputs a signal indicating the data to the control computer 2, and records the data signal output from the control computer 2 in the optical card.

The recorded contents of the optical card include information on the doctor in charge (prescription, training schedule), information on patient confidence (name, sex, age, medical history, blood pressure, weight, height, facial photograph, etc.), device setting value information ( Training mode, torque etc.),
There are training history information and information from the device (information such as patient movements).

Next, the operation of the rehabilitation and diagnosis device will be described. First, when the optical card is inserted into the optical card recording / reproducing apparatus, the recorded information is read and sent to the control computer 2. Control computer 2
Sends the software package number to the VR image signal generating computer 1 according to the sent information. Then, the VR image signal generation computer 1 selects a software package and sends a selection completion signal to the control computer 2.

Next, the control computer 2 sends an initialization request signal to the VR image signal generation computer 1. In response to this, when the VR image signal generating computer 1 sends an acknowledge signal, the controlling computer 2 receives the position from the head-mounted visual display device 4 and the space joystick 5,
Information on the posture and the strength of grip is obtained, and the information is sent to the VR image signal generating computer 1 as initialization information.

When the VR image signal generation computer 1 finishes the initialization by the initialization information, the initialization completion signal is sent to the control computer 2. Then, the control computer 2 waits for the training start button to be pressed. When the start button is pressed, the control computer 2 sends a signal requesting the start of image signal generation of the virtual environment to V
It is sent to the R image signal generating computer 1. And VR
The image signal generating computer 1 starts to display the image of the virtual environment on the head-mounted visual display device 4.

In this way, the rehabilitation operation is started. During the rehabilitation, an image of the virtual environment is displayed and a control signal is sent to the AC servomotor of the training machine 6. Then, signals of the rotation angle and the torque are sent from the training machine 6 to the control computer 2. Further, the space joystick sends a signal indicating the position and posture or the strength of grip to the control computer 2.

The position of the viewpoint of the image of the virtual environment advances with the rotation of the output shaft of the training machine. The direction of the line of sight changes according to the signal indicating the rotation angle of the head-mounted visual display device 4. That is, the virtual environment can be looked around by turning the head. Further, the position and posture of the virtual hand change according to the movement of the space joystick. Further, when a virtual hand collides with a door or the like in the virtual environment, a collision sound is heard from the headphones.

FIG. 3 shows the situation where the device is used with the patient lying in bed. In this case, the pedal of the training machine 6 is rotated by the foot. FIG. 4 shows a situation where the device is used by a patient sitting in a wheelchair. In this case, the pedal of the training machine 6 is rotated by the foot. FIG. 5 shows a situation where the device is used by a patient sitting in a wheelchair. In this case, the pedal of the training machine 6 is rotated by hand.

When the virtual environment is a park, by turning the training machine 6, the situation in which one is progressing can be seen and the environment can be overlooked. As the rotation speed of the training machine 6 increases, the traveling speed also increases. Then, when the right pedal is strongly depressed, the patient moves to the right. If the left pedal is strongly depressed, the patient will move to the left.

If the virtual environment is indoors, the patient proceeds indoors. Then, by moving the space joystick 5, it is possible to move a virtual hand on the screen and turn a knob of a door on the screen. The situation of the patient's response in such training is stored in the VR image signal generation computer 1.

When the rehabilitation operation is completed,
The VR image signal generation computer 1 sends an end signal to the control computer 2. Then, the control computer 2
Requests the VR image signal generating computer 1 to transmit the information on the operating state of the patient. When the control computer 2 receives the information, it instructs the optical card recording / reproducing device 3 to record the information on the optical card. After the recording, the optical card recording / reproducing apparatus 3 ejects the optical card and waits for the next optical card to be set.

By thus reading the signal recorded on the optical card with the optical card recording / reproducing apparatus, the operation of the patient can be known and the diagnosis can be easily performed. The conditions for the next training can be set by the attending physician using another computer by referring to the records on the optical card.

The embodiment is constructed as described above,
It is also possible to determine the line-of-sight position of the virtual environment image by a signal indicating the position from a gyro sensor attached to the head-mounted visual display device. By doing so, a feeling of walking in the virtual world can be obtained.

[0037]

As described above, according to the virtual environment display device of the present invention, the visual line position in the virtual world, the visual line direction, the gyro sensor attached to the head-mounted visual display device or the space joystick are used. Since the information of the position of the virtual hand or the posture of the virtual hand is obtained, the feeling that the hand or head is restrained is not felt. Further, the gyro sensor is much cheaper than the non-contact space position sensor using magnetic induction, and the virtual environment display device can be manufactured at low cost.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a rehabilitation and diagnosis device for a person with cerebral dysfunction using the virtual environment display device of the present invention.

FIG. 2 is a diagram showing a structure of a training machine used in the apparatus.

FIG. 3 is a perspective view showing a usage state of the apparatus.

FIG. 4 is a perspective view showing another example of a usage state of the device.

FIG. 5 is a perspective view showing still another example of a usage state of the device.

FIG. 6 is an exploded perspective view showing the structure of a head-mounted visual display device.

[Explanation of symbols]

 1 VR image signal generation computer 2 Control computer 3 Optical card recording / reproducing device 4 Head mounted visual display device 5 Space joystick 6 Training machine 7 SCS1 cable 8, 9, 10 RS-232C cable 11, 12 pin cable 13 Connection Device 14 Scan converter 15 Decoder 16 Decoder / converter 17 Parallel cable

Claims (3)

[Claims] 1. A virtual environment display device comprising: a head-mounted visual display device for providing a virtual environment; and an image signal generation device for generating an image signal and outputting the image signal to the head-mounted visual display device. A gyro sensor is attached to the head-mounted visual display device, and the line-of-sight direction of the virtual environment image is determined according to a signal indicating the posture from the gyro sensor, and an image signal is generated. Environmental display device. 2. A virtual environment display device comprising: a head-mounted visual display device for providing a virtual environment; and an image signal generation device for generating an image signal and outputting the image signal to the head-mounted visual display device. A gyro sensor is attached to the head-mounted visual display device, and the viewpoint position and the line-of-sight direction of the virtual environment image are determined according to the signal indicating the position and the attitude from the gyro sensor, and the image signal is generated based on the viewpoint position and the line-of-sight direction. A virtual environment display device characterized by being configured to be generated. 3. A head-mounted visual display device for providing a virtual environment, an image signal generation device for generating an image signal and outputting it to the head-mounted visual display device, and a virtual hand image for the virtual environment. In a virtual environment display device including a space joystick for adding to the space joystick, a gyro sensor is attached to the space joystick, and a position and a posture of the virtual hand image change according to a signal indicating a position and a posture from the gyro sensor. A virtual environment display device characterized by being configured as follows.