JPH11338457A - Binocular parallax type three-dimensional display method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To smoothly display a three-dimensional only by processing the position data of an observing point without changing the conventional display system by displaying an image of the same observing point on both eyes only by a moving parallax when the observing point moving speed of a user is high, and making binocular parallax display when the observing point moving speed is low. SOLUTION: A magnetic sensor type head tracker having a magnetic sensor 2 on a head of a head mounted display(HMD) 1 is used for transmitting the observing point position of a user to a virtual workd description application 7. The magnetic sensor type head tracker is a device for transmitting the position and orientation of the magnetic sensor 2 in a magnetic field formed by a magnetic source 3 through a magnetic sensor controller 4 to a computer 6. Two sheets of images with biobular parallax are sent through a three- dimensional display controller 5 to the HMD1. When the observing point moving speed is high, an image of the same observing point is displayed on both eyes only by moving parallax and when the observing point moving speed is low, binocular parallax display is made.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for displaying a stereoscopic image and a recording medium, and more particularly to a computer graphic (CG).
The present invention relates to a stereoscopic display method for stereoscopically viewing a virtual space formed using a technique such as cs) using binocular parallax and motion parallax.

[0002]

2. Description of the Related Art In a conventional stereoscopic display method using binocular parallax, a parallax parameter corresponding to a binocular distance is a constant value corresponding to an adult's binocular distance. As described above, since the conventional method mimics the stereoscopic view of the real world, there are problems in the following points when stereoscopically viewing a virtual space generated by CG or the like.

A first problem is that the processing power of a computer is used unnecessarily even in a scene where visual characteristics are not effective. That is, when the object in the visual field moves rapidly due to the movement of the viewpoint, the depth detection sensitivity due to the binocular parallax decreases, and it is well known that the motion parallax becomes the dominant factor in depth detection instead of the binocular parallax. Have been. Unlike the binocular parallax, the motion parallax requires a single viewpoint image, and it is unnecessary to generate a two viewpoint image for binocular parallax display.

[0004] In general, drawing (rendering) a complicated CG model is a heavy load in computer processing. Therefore, the binocular parallax display that renders a motion parallax image twice has a lower frame rate than the motion parallax display. Therefore, displaying the binocular parallax under the condition that the detection sensitivity of the binocular parallax is low results only in reducing the frame rate and giving flicker.

[0005] The second problem is that, despite the fact that in the virtual world, a person with a high degree of freedom can be seen outside the real world, conventionally, nothing has been done beyond mimicking the state of a real human being. That is. In other words, it is well known that a distant object has a smaller parallax than a nearby object, so that it is difficult to grasp the depth. However, in the related art, there is a similar limitation in a virtual world.

On the other hand, if the field of view is widened in order to amplify the sense of depth at a distance, the parallax is too large when viewing a near object, so that stereoscopic viewing becomes difficult, so that it cannot be used.
Conversely, when trying to see a small object in detail due to lack of resolution of the display, it is necessary to bring the object closer to the eye than in reality. In that case, however, there is a problem that the parallax becomes too large and stereoscopic vision becomes difficult.

A third problem is that a head-mounted display (HMD) or a large-screen box-shaped display uses a head position detecting means such as a magnetic sensor to draw the world viewed from the user's viewpoint. However, because the accuracy of these position sensors is insufficient, even if the head is stopped, the field of view may fluctuate. In the virtual world, the brightness of the contour of an object may change significantly with a slight movement of the visual field due to a shortage of polygons of the object, simplification of lighting calculation, and the like. For this reason, there is a problem that flickering of the screen occurs due to the fluctuation of the visual field, and it becomes difficult to perform the stereoscopic viewing by the binocular parallax. On the other hand, if the detection sensitivity of the position sensor is reduced in order to reduce the flicker, there is a problem that the movement of the visual field with respect to the movement of the head is not smooth.

[0008]

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to process position data of a viewpoint without changing a conventional display system. It is an object of the present invention to provide a technology capable of performing a smooth three-dimensional display only by using the technique.

Another object of the present invention is to provide a technology capable of giving a sufficient sense of depth to an object at a long distance and suppressing an unpleasant sway of a visual field due to noise or the like of a position sensor. .

Another object of the present invention is to provide a technique capable of realizing high-quality stereoscopic display.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0012]

The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows.

(1) In the binocular parallax stereoscopic display method, the moving speed of the user's viewpoint is measured, and when the moving speed of the viewpoint is high, an image of the same viewpoint is displayed on both eyes only by the motion parallax. When the speed is low, binocular parallax is displayed.

(2) In the binocular parallax stereoscopic display method, the gaze distance from the user's viewpoint to the gaze point is measured, and the parallax amount is increased when the gaze distance is large, and the parallax is increased when the gaze distance is small. Decrease the amount.

(3) A binocular parallax stereoscopic display method using a sensor for detecting the position of the user's head, wherein a gaze time during which the user is gazing is measured. Decrease the detection sensitivity of the sensor.

(4) A stereoscopic display method of binocular parallax using a sensor for detecting a position of a head of a viewpoint user,
The gaze time during which the user is gazing is measured, and if the state in which the difference between the position and orientation of the gaze data of the user is smaller than the threshold has continued for a certain period of time or more, the “reference value” is not updated and the “drawing parameter” An image from the same viewpoint is drawn with the reference value unchanged, and in other cases, the “reference value” is updated, and an image from the updated viewpoint is drawn using the “drawing parameter” as the current value.

(5) A recording medium which records a stereoscopic display control program for performing binocular parallax stereoscopic display using a sensor for detecting the position and viewpoint of the user's head by a computer, wherein the user is watching. If the procedure of measuring the gaze time and the state in which the difference between the position and orientation of the user's gaze data is smaller than the threshold has continued for a certain period of time or more, the “reference value” is not updated, and the “drawing parameter” remains at the reference value. A means for drawing an image from the same viewpoint and a procedure for updating the “reference value” in other cases and drawing an image from the updated viewpoint using the “drawing parameter” as the current value are executed by the computer. And a computer-readable recording medium storing a program to be executed.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments (examples) of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a three-dimensional display device using a head mounted display (HMD) according to an embodiment (example) of the present invention, and 1 is a head mounted display (hereinafter, HMD). 2) a magnetic sensor, 3 a magnetic source, 4 a magnetic sensor controller, 5 a stereoscopic display controller, 6 a computer, and 7 a virtual world description application.

As shown in FIG. 1, the three-dimensional display device of this embodiment has a magnetic sensor 2 provided on the head of the HMD 1 for transmitting a user's viewpoint to a virtual world description application 7. A head tracker was used. The magnetic sensor type head tracker includes a magnetic sensor 2 in a magnetic field generated by the magnetic source 3.
Is a device for communicating the position and orientation of the computer to the computer 6 via the magnetic sensor controller 4. Two images with binocular parallax are sent to the HMD 1 through the stereoscopic display controller 5.

FIG. 2 is a flowchart showing a processing procedure (a processing procedure generally used conventionally) for displaying a virtual world on a CG according to the present embodiment. FIG. 2A shows the case of the motion parallax display, and this processing is performed when the image is displayed on a normal personal computer monitor. FIG. 2 (2) shows a case of binocular parallax display, and is a processing flow in the case of using the HMD 1 of the present embodiment, and in the case of stereoscopic viewing using glasses such as liquid crystal shutter glasses and polarizing glasses. When the display mode is binocular parallax, unlike the case of single viewpoint,
Some parameters must be added, such as the distance between the eyes.

In (1) and (2) of FIG. 2, the computer 6 first obtains viewpoint data of the user (S
101, S104), and based on the preset drawing parameters (S102, S105), calculate the field of view that should be seen from that viewpoint and draw (S103, S104).
S106). The drawing parameters must include everything necessary for drawing, such as a display mode for determining whether a single viewpoint (motion parallax display) or binocular viewpoint (binocular parallax display). The user's viewpoint data is one of the important parameters, which is updated in real time.

Conventionally, after initial setting of the parameters, one of the processing flows (1) and (2) of FIG. 2 has been fixedly used. However, the present invention is characterized in that the movement of the viewpoint of the user is detected and the processing flow is switched in real time. First, a method of switching the display mode will be described.

In the present invention, the binocular parallax display mode and the motion parallax display mode are switched according to the moving speed of the viewpoint.

FIG. 3 is a diagram for explaining the threshold value of the viewpoint moving speed and the display state. As shown in FIG. 3A, the user's viewpoint is defined by a position vector r from the origin and a direction vector s (unit vector) indicating the direction of the viewpoint from the coordinate system in the virtual space. You. In the present embodiment, in order to obtain the moving speed of the viewpoint, the inner product of the direction vector s2 of the current viewpoint and the direction vector s1 of the previously input viewpoint is calculated. Since the change of viewpoint vectors adjacent in time is small, the inner product of the vectors takes a smaller value as the change of the vector is larger, in a range where the difference between the two vectors is not so large. Therefore, when a certain threshold value is provided and (the inner product of s1 and s2)> threshold is satisfied, it can be determined that the moving speed of the viewpoint is low, and otherwise, it is determined that the moving speed of the viewpoint is high. Since the threshold value depends on the system performance such as the noise level of the magnetic sensor, it is necessary to set the threshold value according to the system to be used.

FIG. 3 (2) is a diagram showing how the display mode is switched when the inner product of the direction vector changes over time. The binocular parallax display and the motion parallax display alternately appear at the threshold.

FIG. 4 is a flowchart showing an example of a processing procedure for changing the display mode according to the present embodiment. In the display mode change processing of the present embodiment, as shown in FIG. 4, when a program is started, a loop for drawing starts to rotate. First, the user's viewpoint data is acquired first (S201), and then the difference from the previous position is calculated (S201).
202). Then, it is determined whether the difference is larger than the threshold (S2
03) If the difference is larger than the threshold, the drawing parameters for the binocular parallax display are set (S204), and the field of view with parallax is drawn in the two windows (S205). On the other hand, if the difference is smaller than the threshold value, the drawing parameters for the single viewpoint display mode (motion parallax display mode) are set (S20).
6) The visual field seen from the intermediate point connecting both eyes is drawn in one window, and the same image is copied to the other window (S207).

In this embodiment, the drawing time of the binocular parallax display is compared with the drawing of a virtual world of about 500 polygons with texture, and as a result, a time reduction effect of about 1: 0.8 is obtained. Under this comparison condition, the rendering load was not so large, so that there was not much difference between the two display modes. However, it is considered that the ratio approaches 1: 0.5 as the number of polygons increases. Of course, when the viewpoint is not moved, the display mode is not switched at all in the present invention, but in that case, there is no movement, so the binocular parallax display may be kept, and there is no effect of increasing the frame rate and increasing smoothness. There is no effect.

Although the display mode is instantaneously switched in the flowchart of FIG. 4, the distance between both eyes is 0 → 6.5c.
m, and conversely, a method of smoothly switching from 6.5 cm to 0 is also conceivable. By adjusting the distance between both eyes in this way, it is possible to make the user not feel switching.

Next, in the present invention, the parameter of the distance between the eyes is changed according to the magnitude of the gaze distance. FIG. 5 is a diagram for explaining the relationship between the gaze distance threshold and the distance between both eyes according to the present embodiment. FIG. 5A illustrates how to change the interocular distance when the gaze distance is large and small. Eyeballs when the eyeball interval is a standard value (about 6.5 cm) are referred to as a left eyeball 1 and a right eyeball 1, respectively. When viewing an object at a long distance in this state, the angle of convergence is small (angle of convergence 1) and the binocular parallax is also small, so that the depth of the object is small. However, when the same distant object is viewed with eyes having a larger eyeball distance (left eyeball 2, right eyeball 2), the convergence angle increases (convergence angle 2),
Since the binocular parallax also increases, the sense of depth is emphasized. Conversely, when viewing a short-distance object such as in front of the eyes, the convergence angle becomes too large, making stereoscopic viewing itself difficult. It can be seen that for such a short-distance object, stereoscopic viewing is facilitated by reducing the distance between the eyes.

FIG. 5 (2) is a diagram for explaining how to control the interocular distance when the user's gaze distance changes. Two thresholds are provided for the gaze distance, and threshold 2> threshold 1. If the gaze distance> threshold 2, the distance between both eyes is a long distance value (> standard value). If the threshold 2> the gaze distance> threshold 1, both eyes are set. If the interval is a standard value, and if the gaze distance <threshold 1, the binocular interval is a short distance value (<standard value).

FIG. 6 is a flowchart illustrating an example of a processing procedure for controlling the distance between eyes according to the present embodiment. As shown in FIG. 6, the binocular distance control process of the present embodiment
The user's viewpoint position data is fetched (S301), and the gaze distance is calculated (S302). There are several methods for obtaining the gaze distance from the viewpoint position data. In this embodiment, a third method is provided between the left and right eyes for drawing in addition to the left and right eyes.
The distance to the closest object in the third perpendicular direction was determined from the z-buffer value and the like, and this was taken as the gaze distance. As another method of obtaining the gaze distance, there is a method of measuring the line-of-sight vectors of both eyes with an eye movement measuring instrument or the like, and obtaining the position where two lines of sight intersect based on the principle of triangulation. next,
The gaze distance is compared with two thresholds (threshold 1 and threshold 2) (S
303), the drawing parameters are set by changing the distance between the eyes to three values of a long distance value, a standard value, and a short distance value according to the magnitude (S304, S305, S306), and the left and right parallax images are drawn (S307). , S308, S309). By performing this control in real time, if the center of the screen is a distant view, the distance between the eyes is automatically widened and the sense of depth is enhanced, and if there is an object very close to the center of the screen, both eyes are automatically set. The distance between eyes becomes narrower, making it easier to view stereoscopically. Also,
When viewing an object at a medium distance, the distance between the eyes is a standard distance between eyes, so that a stereoscopic effect similar to reality can be obtained. In this embodiment, only two thresholds are provided, but it is also possible to provide a large number of thresholds and change the binocular interval continuously.

Next, in the present invention, the detection sensitivity of the position sensor is changed according to the magnitude of the gaze time. FIG. 7 is a diagram for explaining a method of setting a gaze time threshold according to the present embodiment. As shown in (1) of FIG. 7, a threshold value is provided in a gaze area (three-dimensional area which is written in two dimensions in FIG. 7, but is actually deep) to create a threshold area. Then, the number of gaze positions (numbers 1, 2, 3,... N) that are continuously input from the gaze position (reference value: number 0) at which measurement is started is counted in the threshold region.

FIG. 7 (2) is a diagram for explaining a method of determining whether or not the pixel is in the threshold region. It is determined whether or not the inner product of the direction vector of the currently input viewpoint and the reference viewpoint is larger than a certain threshold value. Further, it is checked whether or not the absolute value of the difference between the position vectors of the two viewpoints is smaller than a certain threshold value. Actually, both the direction change and the parallel movement occur at the same time, so the determination is made based on the AND condition of the two conditions.

FIG. 8 is a flowchart showing an example of a processing procedure for controlling the sensor sensitivity according to this embodiment. In the process for controlling the sensor sensitivity according to the present embodiment, as shown in FIG. 8, after acquiring viewpoint data of the user (S401), a difference from reference viewpoint data is obtained (S402). It is determined whether this difference is equal to or smaller than the threshold (S40).
3) If the difference is equal to or smaller than the threshold, the counter is incremented by 1 (S
404), and compare with the threshold value of the count number (S405).
If the counted number is equal to or larger than the threshold value of the counter, the viewpoint position at that time is registered as a reference position (S40).
6) The drawing parameters are set (S407), and the left and right screens are drawn (S408). After that, if the viewpoint data is within the threshold, the input data taken in is ignored, and drawing is always continued at the reference viewpoint. As a result, when the gaze position remains within the threshold region for a time corresponding to the threshold value of the count number, the display field of view does not change even if there is a similar degree of viewpoint swing, and a calm stereoscopic image can be seen. But,
Even in this case, once the viewpoint is moved beyond the threshold region, the counter is set again and the display returns to the normal display mode (S413, S414, S415).

If the determination in step (S403) is equal to or greater than the threshold value, the counter is reset (S409), the reference value is updated and registered (S410), the drawing parameters are set (S411), and the left and right screens are displayed. Draw (S412), and return to step S401.

As can be seen from the above description, according to the present invention, when the viewpoint moving speed is high, an image of the same viewpoint is displayed on both eyes only by motion parallax, and when the viewpoint moving speed is low, binocular parallax display is performed. By doing so, during viewpoint movement in which it is difficult to perform stereoscopic viewing with binocular parallax, an expression based on motion parallax is made, so that the user is always given the illusion of stereoscopic display with binocular parallax.

In addition, in the case of the motion parallax display, the display frame rate is increased, so that a smoother display is possible than in the case of the conventional binocular parallax display alone. The viewpoint moving speed can be obtained from output data of a viewpoint moving means such as a magnetic sensor, a mouse, and a joystick.

Further, according to the present invention, a control is performed to measure the gaze distance from the user's viewpoint to the gaze point, and increase the parallax amount when the gaze distance is large and decrease the parallax amount when the gaze distance is small. Is performed, the parallax is originally small when viewing a long distance, and even if the binocular disparity is widened, the binocular parallax image can easily be stereoscopically viewed with many corresponding points, so that the sense of distance can be amplified without a sense of discomfort.

Also, if the eyes are sufficiently close to a small object, the angle of convergence becomes too large and the area corresponding to the two parallax images becomes difficult. However, according to the present invention, the distance between the eyes is automatically adjusted in this case. , The corresponding points can be easily viewed in a stereoscopic manner without a decrease in the corresponding points. The gaze distance can be generally determined by mounting an eye movement measuring device and measuring the intersection of the binocular gazes.

According to the present invention, considering that the object to be watched is often viewed in front of the face by moving the head so that the object to be watched is at the center of the field of view, the distance of the object at the center of the field of view is determined. Distance can be substituted, which is an effective method, especially when looking at the CG world.

According to the present invention, the gaze time measured by the user is measured, and when the gaze time is long, the detection sensitivity of the sensor is reduced. If it can be determined that the position sensor is detected, the detection sensitivity of the position sensor is low, so that a stable image can be observed without a change in the field of view due to the noise of the position sensor or slight shaking of the head.

Further, according to the present invention, when an operation of looking around is performed, even if the movement of the viewpoint stops, the stop time is not so long, and in that case, the detection sensitivity of the position sensor is high. Therefore, a smooth feeling of movement can be obtained. The gaze time is obtained by a method such as counting the number of image frames whose viewpoints are within the threshold value.

As described above, the invention made by the present inventor is:
Although the present invention has been described in detail with reference to the embodiment, the present invention is not limited to the embodiment, and it is needless to say that various changes can be made without departing from the scope of the invention.

[0045]

As described above, according to the present invention, a smooth three-dimensional display can be achieved only by processing the position data of the viewpoint without changing the conventional display system.

Further, a sufficient sense of depth can be given to an object located at a long distance.

Further, it is possible to suppress the unpleasant sway of the visual field due to the noise of the position sensor. As a result, high-quality stereoscopic display can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a stereoscopic display device using a head mounted display (HMD) according to an embodiment (example) of the present invention.

FIG. 2 is a flowchart illustrating an example of a processing procedure for displaying a virtual world on a CG according to the embodiment;

FIG. 3 is a diagram for explaining a threshold value and a display state of a viewpoint moving speed according to the embodiment;

FIG. 4 is a flowchart illustrating an example of a processing procedure for changing a display mode according to the embodiment.

FIG. 5 is a diagram for explaining a relationship between a gaze distance threshold and a binocular distance according to the embodiment;

FIG. 6 is a flowchart illustrating an example of a processing procedure for binocular distance control according to the embodiment;

FIG. 7 is a diagram for explaining a method of setting a gaze time threshold according to the embodiment;

FIG. 8 is a flowchart illustrating an example of a processing procedure for controlling sensor sensitivity according to the embodiment.

[Explanation of symbols]

1 ... HMD, 2 ... magnetic sensor, 3 ... magnetic source, 4 ...
Magnetic sensor controller, 5 ... 3D display controller, 6 ... Computer, 7 ... Virtual world description application.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Susumu Ichinose 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation

Claims (5)

[Claims] In a binocular parallax stereoscopic display method, a moving speed of a user's viewpoint is measured, and when the viewpoint moving speed is high, an image of the same viewpoint is displayed on both eyes only by a motion parallax. A stereoscopic display method wherein binocular parallax is displayed when is small. 2. In a binocular parallax-type stereoscopic display method, a gaze distance from a user's viewpoint to a gaze point is measured, and if the gaze distance is large, the amount of parallax is increased. Stereoscopic display method characterized by reducing the number of images. 3. A stereoscopic display method of a binocular parallax using a sensor for detecting a position of a user's head, wherein a gaze time during which the user is gazing is measured, and the sensor is used when the gaze time of the user is long. A stereoscopic display method characterized by lowering the detection sensitivity of an image. 4. A binocular parallax stereoscopic display method using a sensor that detects a position of a head of a viewpoint user, wherein a gaze time during which the user gazes is measured, and a position and an orientation of the user's gaze data are measured. If the state where the difference is smaller than the threshold continues for a certain period of time or more, the "reference value" is not updated,
Draw an image from the same viewpoint while keeping the “drawing parameter” as the reference value, and in other cases, update the “reference value”,
A three-dimensional display method characterized by drawing an image from an updated viewpoint using a “drawing parameter” as a current value. 5. A recording medium storing a stereoscopic display control program for performing binocular parallax stereoscopic display using a sensor that detects a position and a viewpoint of a user's head by a computer, wherein the user is gazing at the gazing point. When the time measurement procedure and the state in which the difference between the position and orientation of the user's gaze data is smaller than the threshold value have continued for a certain period of time or more, the “reference value” is not updated, and the “drawing parameter” remains the same as the reference value. Means for drawing an image from the viewpoint, and in other cases, the "reference value" is updated and the "drawing parameter"
And a procedure for drawing an image from an updated viewpoint with the current value as a current value.