JP3477441B2 - Image display device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention
It can be suitably implemented as a window display device.
The present invention relates to an image display device that can be used. 2. Description of the Related Art A head-mounted display device has two
Images with parallax can be displayed on one display surface
You. The user puts the head mounted display device on the head
To view the image on each display surface individually with the left and right eyes.
And a three-dimensional display created by stereoscopic display
You can experience the imaginary space. Another conventional technique is disclosed in Japanese Patent Laid-Open No.
Message information on display shown at 69363
There is a device that manages This device is placed on the display
When displaying the message, the eye tracker
To detect the position on the display where the user is
To display a message at that location. Gaze position
When the message moves along the message, the message
Is erased. [0004] The head mount disk
Playing devices require only stereoscopic display to the user.
This is the target device. Therefore, in this device,
Identify the position where the observer is gazing in the created virtual space
Can not be. In the device disclosed in Japanese Patent Application Laid-Open No. 10-69363,
Identify the location on the display where the user is watching
Can do things, but only in two dimensions
Is specific. Even if this device is used as it is,
The gaze position in the virtual space shown and created
It cannot be specified. An object of the present invention is to enable stereoscopic display.
Attention to the virtual space created by the user
To provide an image display device that can specify the position
It is to be. SUMMARY OF THE INVENTION The present invention is directed to a user's head.
On the left and right eyes of the user.
Display images with parallax on the two display surfaces
Display means to detect the gaze directions of the left and right eyes of the user.
Gaze detection means and the position and posture of the user's head
Detected by the position / posture detection unit that emits and the line of sight detection unit
Gaze direction and position / posture detection means
Display means based on the position and posture of the head
Created virtually by displaying images using
Processing for calculating the position of the user's gaze in the virtual space
Processing means, wherein the arithmetic processing means is
Image portion at a higher position than the rest of the image portion
And the distance in the perspective direction from the position where the user gazes
Display images with reduced sharpness as volume increases
Image display characterized by controlling display means to
Device. According to the present invention, the display means is a user's head.
In this state, the two display surfaces are placed on the left side of the user.
It is placed facing each right eye, and the user
To see the image on one display surface and
You can see the image on the other display surface. by this
The display means can provide a stereoscopic display to the user.
To create a three-dimensional virtual space for the user
it can. When viewing images that are stereoscopically displayed in this way,
The gaze directions of the left and right eyes of the user are detected by gaze detection means
And the position and posture of the user's head
Detected by the detecting means, and
Therefore, by the stereoscopic display by the display means,
The position where the user gazes in the virtual space created imaginatively
Can be calculated by the calculation processing means. this
By detecting each gaze direction of the left and right eyes
By specifying the position where the line of sight intersects, the user in the virtual space
The gaze position can be obtained by calculation. Again
The image part representing the position where the user gazes is the remaining image part
An image having higher sharpness is displayed by the display means.
Is shown. Moreover, the displayed image is watched by the user.
Sharpness increases as the amount of displacement in the distance direction from the position increases
Is lower. In this way, the user's
Gives a partial difference in the sharpness of the image depending on the
The gaze position is displayed more sharply than others, and the gaze position
The image is blurred as it shifts from the position in the perspective direction.
Therefore, the displayed image shows that the user has seen the real space.
As with the partially sharpened images obtained in
Images with different sharpness and display such images
can do. FIG. 1 shows an embodiment of the present invention. Form
FIG. 2 is a block diagram illustrating a head mounted display device 1.
is there. Head mounted display as an image display device
(HeadMounted Display: hereafter referred to as “HMD”
The device 1 is mounted on the head 3 of the user 2 and
2 arranged to face the left and right eyes 4L, 4R, respectively
Images having parallax are respectively displayed on the two display surfaces 6L and 6R.
The display 8 to be displayed and the left and right eyes 4L and 4R of the user 2
A line-of-sight detector 9 for detecting the line-of-sight direction, and the head 3 of the user 2
A position and orientation detector 11 for detecting the position and orientation of the
Each line-of-sight direction and position detected by the line detector 9
The position of the head 3 detected by the posture detector 11 and
Based on the posture, the virtual image
The position where the user gazes in the virtual space
An arithmetic processor that calculates the “gaze position”)
And a calculator 10. The HMD device 1 further includes a head of the user 2
3 is provided with a holder (not shown) attached to the
An indicator 8 is provided on the rudder, whereby the indicator 8
It is attached to the head 3 of the user 2 and is fixed to the head 3
Is held. The display 8, which is a display means, displays each display.
Two screen parts on which the surfaces 6L and 6R are respectively formed
Images on the materials 12L, 12R and the display surfaces 6L, 6R
It has two projectors 13L and 13R for projecting respectively. Each of the screen members 12L and 12R is
And a holder attached to the head 3
With the head facing the left and right eyes 4L and 4R, respectively.
3 is located forward. For example, one display surface
One screen member 12L having 6L is provided to the left eye 4L.
The other screen having the other display surface 6 </ b> R
The lean member 12R is disposed facing the right eye 4R. Each of the projectors 13L and 13R is, for example, a cathode.
It consists of a tube and a liquid crystal display element.
Each of the screen members 12 is projected by the projectors 13L and 13R.
The images are projected on the display surfaces 6L, 6R of the L, 12R,
Images are displayed on the display surfaces 6L and 6R. Each display surface 6L, 6
The images displayed on R are the left and right eyes 4L and 4R of the user 2.
Is an image having a parallax corresponding to the parallax of FIG. As described above, the screen composed of the half mirror
To project images onto the image members 12L and 12R to display images.
Therefore, when viewed from the user 2, the screen members 12L, 12L
Overlaid on top of the background behind R
The user 2 can use the display 8 to display
Not only the image displayed on the screen, but also the screen member 12L,
The background can be seen through 12R. Book like this
In the embodiment, the actual background and the image by the display 8 are used.
Therefore, a virtual space is created. The line-of-sight detector 9 serving as the line-of-sight detecting means
Eye cameras 1 that respectively image the eyes 4L and 4R
4L, 14R and two counters each having a reflecting surface formed.
Shooting members 15L, 15R and eye cameras 14L, 14R
The left and right eyes 4L, 4
An eye camera controller 16 for determining the line of sight of R
Have. Each of the eye cameras 14L,
14R and each reflecting member 15L, 15R,
Provided on the provided holder, thereby
14L, 14R and each reflecting member 15L, 15R,
It is attached to the head 3 of the user 2 and is fixed to the head 3
Is held. In this embodiment, the power is held by the holder.
Each screen member 12L mounted on the head 3 of the user 2,
12R, each projector 13L, 13R, each eye camera 14
H including L, 14R and each reflecting member 15L, 15R.
The MD main body 20 is configured. This HMD body 20 is the head
3 is attached to a fixed position. Each of the eye cameras 14L and 14R emits infrared rays.
Using the left and right eyes 4L, 4R, specifically each eye
Image a sphere. By using infrared, user 2
The left and right eyes 4L, 4R without affecting the vision of
Images can be taken. Infrared irradiation means not shown
And irradiate infrared rays to the left and right eyes 4L and 4R,
To facilitate imaging by the cameras 14L and 14R
May be. Each of the reflecting plates 15L and 15R is a half mirror.
With the holder attached to the head 3
Then, face the left and right eyes 4L and 4R, respectively, and
And then placed in front. For example, one reflection member 15
L is the left eye 4L and one screen member facing the left eye 4L.
12L, and the other reflecting member 15R is located between the right eye
The right eye 4R and the other screen member 12R facing the 4R
Placed between. As described above, the reflecting plates 15L and 15R are provided.
Each of the eye cameras 14L and 14R is
Image left and right eyes 4L and 4R through 5L and 15R
You. Specifically, one eye camera 14L has one
The left eye 4L is imaged through the firing plate 15L, and the other eye camera is taken.
14R captures the right eye 4R via the other reflector 15R.
Image. The left and right eyes 4 via the reflection plates 15L and 15R.
By taking images of L and 4R respectively,
LA uses 14L and 14R blocks the view of User 2
Obstructs the viewer 2 from seeing the image and background on the display 8
Can be prevented. The eye camera controller 16 stores each eye
Left and right eyes 4 captured by cameras 14L and 14R
A signal representing the L, 4R image is provided. Eye camera control
Troller 16 images left and right eyes 4L and 4R, respectively.
The left and right eyes are processed by image processing and analyzing the position of the pupil.
4L, 4R line-of-sight directions, specifically left and right with respect to head 3
The gaze direction of the eyes 4L and 4R is calculated and obtained. like this
Then, the line-of-sight detector 9 detects the left and right eyes 4L and 4R.
The gaze direction with respect to the head 3 can be detected. Position / posture detector 1 as position / posture detection means
1 has a sensor unit 24 and a transmitter unit 25
Magnetic head motion sensor (Head Motion Sens)
or: HMS).
Controller 26. The HMS sensor unit 24
HMS transmitter attached to the head 3 of the user 2
The unit 25 is located at a low position in the space where the user 2 is located, as will be described later.
If user 2 is a pilot, the cockpit is low.
Fixed in position. Made by transmitter unit 25
The intensity of the emitted magnetic field is detected by the sensor unit 24,
Is given to the HMS controller 26.
You. Based on this, the HMS controller 26
Sensor unit 2 for transmitter unit 25 in fixed position
The position and orientation of No. 4 are calculated and obtained. Like this
The position and orientation of the head 3 of the user 2 are detected by the position and orientation detector 11.
And posture can be detected. The position / orientation detector 11 has H
MS may be used. For example, optical and
Other types of HMS, such as a gyro type, may be used.
The position and posture of the head 3 of the user 2 can be detected.
Any method may be used. The arithmetic processing computer 10 as the arithmetic processing means
Represents the image data of the image to be displayed by the display 8.
Signal is given to each of the projectors 13L and 13R. This
Therefore, each of the projectors 13L and 13R is
Projecting an image based on the image data given by the
8, the image is displayed. The arithmetic processing computer 10 also has an HMS
Signal representing the position and posture of the head 3 from the controller 26
Is given. As a result, the arithmetic processing computer 10
Image based on the display 8 based on the position and orientation of the unit 3
Can be ordered. The arithmetic processing computer 10 has the position of the head 3
Position and attitude signals, plus an eye camera controller
Signals indicating the line-of-sight directions of the left and right eyes 4L and 4R are given from 16
available. As a result, the arithmetic processing computer 10
Based on the line-of-sight directions of the eyes 4L and 4R, the image
The user 2 in the virtual space created by the image gazes
Position can be calculated. The HMD 1 is displayed on the display 8.
Image information input means for inputting information about the image to be
Step 17 is included. This image information input means 17 is, for example,
Storage means for storing various kinds of information generated in advance;
Imaging means for imaging the interval and information for obtaining predetermined information
A configuration having at least one of acquisition means may be adopted.
No. The storage means is, for example, a hard disk drive.
This storage means is realized by image information
When used as the input means 17, the arithmetic processing computer 10
Are various types corresponding to the position and posture of the head 3 of the user 2.
Reads information, generates image data, and generates the image data
Is given to the display 8. The imaging means can be
A CCD camera that captures images using visual or infrared light
This imaging means is used as image information input means.
If there is, the arithmetic processing computer 10
A signal representing the image data is provided to the display 8. Information acquisition means
Is realized, for example, by sensors that detect various information.
When this imaging means is used as image information input means,
In this case, the arithmetic processing computer 10 displays a graph indicating the acquired information.
A signal representing image data of an image such as
You. The HMD device 1 has the head of the user 2
It is possible to display an image based on the position and orientation of the unit 3.
User 2 does not move or change direction
When the position and posture of the head 3 change,
In the virtual space according to the change of the position and posture of the unit 3
Display such that the position and posture of the head 3 change
be able to. In other words, the user 2 temporarily creates the virtual space.
You can move imaginatively. Produced in this way
A gaze detector detects the position where the user gazes in the virtual space
9 can be detected. FIG. 2 illustrates a stereoscopic display by the display unit 8.
FIG. One display surface 6 for the left eye 4L
The image 35L displayed on L and the other for right eye 4R
The image 35R displayed on the display surface 6R is as described above.
Have parallax with each other. Specifically, the example of FIG.
In the figure, a tree 36, a mountain 37 and a lake 38 are displayed.
Is the image 35L for the left eye 4L and the image 35R for the right eye 4R.
And the mutual positional relationship between the tree 36, the mountain 37 and the lake 38
Are different. With such a display, stereoscopic display can be realized.
can do. The image displayed on one display surface is displayed on the left and right eyes.
It is needless to say that the display seen in 4L and 4R is a flat table.
I can't show it. Separately for left and right eyes 4L, 4R
, If the same image, the left and right eyes 4R,
Since the line of sight of 4L is parallel, a sense of perspective cannot be obtained. One
For one side, for example, only the image 35R for the right eye 4R,
Information about the directions of trees 36, mountains 37 and lakes 38
Tree 36, Mountain 37, though it can be given to User 2
To provide user 2 with information on the distance to and from Lake 38
I can't do that. Therefore, the user 2 has no
I can't grip. In the HMD 1, an image 35 having parallax is displayed.
L and 35R are displayed on the left and right eyes 4L and 4R, respectively.
And the distance between trees 36, mountains 37 and lakes 38
User 2 can grasp the perspective
be able to. Taking the tree 36 as an example further,
In the image 35R for the eye 4R alone, the position indicated by the solid line
And which of the positions indicated by the virtual lines
An image 35 for the left eye 4L, which cannot be grasped
By grasping the direction by L, it exists at the intersection and a solid line
Can be grasped. Actually
The image is displayed on each of the display surfaces 6L and 6R.
It can be displayed as if there is a distance. this
Display in a stereoscopic manner as if
Can be produced. FIG. 3 shows the specification of the gaze position in the three-dimensional space.
FIG. 3 is a diagram for explaining the principle, and FIG.
3 shows the line of sight when looking at infinity, and FIG.
This shows the line of sight when the user 2 is looking at a specific object. Profit
User 2 looks at the entire field of view without gazing at one point
3A, the left and right eyes 4L, 4L, as shown in FIG.
R is looking at infinity, and the left and right eyes 4L,
The line-of-sight directions 30L and 30R of the 4R are parallel to each other,
And in the direction that the head 3 is facing (forward with respect to the head 3)
Parallel to the other. On the other hand, the user 2 has a specific object.
When gazing at the object 32, as shown in FIG.
Look at the object 32 with the left and right eyes 4L and 4R, respectively.
The line-of-sight directions 30L, 30R of the left and right eyes 4L, 4R are
Toward the object 32, in the direction in which the head 3 is facing
Different line-of-sight angles θL and θR, respectively.
You. These line-of-sight angles θL, θR and the left and right eyes 4L, 4R
Position where the user 2 is gazing according to the binocular distance X of
The left and right eyes 4L, 4R up to one point in the object 32
Gazing point distances LL and LR can be obtained respectively.
You. The gazing point distance LL between the left eye 4L and the gaze position of the user 2
LL = x / (sin θL−tan θR × cos θL) (1) by the following equation (1), and the gazing position of the right eye 4R and the user 2
The gazing point distance LR with the position can be obtained by the following equation (2) as LR = x / (tan θL × cos θR−sin θR) (2) Such a measurement principle is based on the fact that the actual space has
Of course, images with parallax are displayed on the left and right eyes 4L and 4R.
In the virtual space created by
In the HMD device 1, this measurement source
The gaze position of the user is specified by the logic. Elaborate
And the line-of-sight directions 30L,
30R are detected as the line-of-sight angles θL and θR. Ma
The distance D between the left and right eyes 4L and 4R of the user 2
It is possible to measure. These line-of-sight angles θL, θ
Equations (1) and (2) using R and the interocular distance x
The arithmetic processing is executed by the arithmetic processing computer 10. to this
Therefore, the background behind the screen members 12L and 12R,
In the virtual space created by the display on the display 8
Identify the position where the user 2 is watching
Can be. FIG. 4 shows an example of a display image on the display unit 8.
FIG. FIG. 4 shows an image for the right eye 4R shown in FIG.
4 shows a detailed image of the image 35R. HMD device of the present embodiment
In the device 1, the arithmetic processing computer 10 calculates the value as described above.
The image part at the position where the user 2 gazes is the remaining image part
To display images with higher sharpness than
The display 8 is controlled. Specifically, the image 3 for the right eye 4R
Trees 36, mountains 37 and lakes 38 similar to FIG.
In displaying, the virtual calculated as described above
Tree 36 is virtually present at the gaze position of user 2 in the space
The tree 36 is displayed with high sharpness
And other mountains 37 and lakes 38 are sharper than trees 36
Display in degrees. In other words, the tree 36 has its outline and
The pattern etc. are shown clearly and in detail, other mountains 37 and lakes
38 has an unknown outline and pattern compared to the tree 36
It is clearly and non-detailed. Thus, depending on the gaze position
The object at that gaze position has at least a clear outline
Is displayed with high sharpness, and the objects at the remaining positions are
Displayed with low sharpness. FIG. 4 shows an image 3 for the right eye 4R.
5R is shown, but the same applies to the image for the left eye 4L.
Will be displayed. The human eyes including the user 2 can be seen from a near view to a distant view.
Can focus on everything up to
Instead, the human including the user 2 selects the distance range and focuses.
I'm matching. This will be described with reference to a specific example.
Seeing the entire view as described with reference to
If user 2 is focused on infinity,
Landscapes, such as mountains, are in focus and sharp
Higher, the closer you get, the more you get out of focus
Lakes and trees in the foreground are out of focus and have low sharpness
Looking at the state. Also described with reference to FIG.
Thus, when looking at a particular object 32, for example, a lake,
User 2 is focused on the lake and has a distant view from the lake
As the distance from the lake becomes closer and closer to the lake
Out of focus and closer to the mountains and lakes than the lake
Trees and the like are out of focus and viewed with low sharpness. This
When the user 2 looks at the actual space,
Focus on the position and move away from the gaze position
Looking out of focus. In the HMD device 1 according to the present embodiment,
The sharpness of the object at the gaze position is
It can be displayed so that it is higher than
Similar to how user 2 focuses when looking at the actual space
In other words, the sharpness varies depending on the gaze position
When pseudo-focus processing is performed to see a more actual space
A stereoscopic display that gives a vision close to that of a stereoscopic image can be obtained.
In this way, the width of stereoscopic display is widened, and more advanced stereoscopic viewing
Can be displayed. Therefore, user 2
You can see the displayed image with little feeling
You. FIG. 5 is a diagram for explaining the pseudo focus processing.
FIG. 5A shows the whole image with high sharpness.
FIG. 5 (2) shows only the gaze position.
This is image data when displayed with high sharpness. FIG.
In (1) and FIG. 5 (2), they are separated by a line 40
The left part is the gaze image part that displays the gaze position,
Non-note where the right part shows the remaining position excluding the gaze position
This is a visual image portion. Single color, for ease of understanding
For example, an image represented by black density will be described as an example.
You. The image data is shown corresponding to some pixels in FIG.
As shown in FIG.
It is represented by the following numerical value, and as it approaches 100,
The degree increases. As shown in FIG. 5 (1), the whole is highly sharp.
When displayed in degrees, the gaze image part and the non-gaze image
Both parts have densities determined individually for each pixel
Is displayed as follows. On the other hand, only the gaze image
In the case of high sharpness, the gaze image part is
Although it is displayed so that the concentration is determined individually
In the non-gaze image part, the adjacent pixels have the same density.
For example, one of a plurality of adjacent pixels is
It is displayed so as to have the density of one pixel. The pseudo focus process is similar to the process shown in FIG.
The pixel group of a plurality of adjacent pixels is not limited.
It is sufficient if the same density is considered as one pixel, and the density is the same.
To equalize, average the numerical values of adjacent pixels
It may be a process for converting the data. In this case, the same effect can be obtained.
Can be In the example of FIG. 5, the number of pixels having the same density is
4, but may be 2, 3 and 5 or more. Further
In the virtual space from the gaze position
Therefore, the images at the position where the amount of displacement is small have the same density.
Reduce the prime number and use the same
By increasing the number of pixels
Achieving pseudo-focus processing close to the state of viewing space
Can be. In FIG. 5, a monochrome image is taken as an example.
For color images represented by red, blue and green densities,
The same applies for blue and green densities.
Thus, a similar effect can be obtained. FIG. 6 is a diagram showing a use state of the HMD device 1.
is there. The HMD device 1 according to the present embodiment includes a helicopter or the like.
Image information input means 1
7 has storage means. Map data is stored in the storage means.
Data, terrain data, building data, etc.
These are read by the arithmetic processing computer 10 and
The position of the user 2 who is a pilot in the arithmetic processing computer 10
Generates image data of images simulating the scenery seen from the viewpoint
Is done. In this way, the actual space 4 outside the aircraft 42
3. Display image 45 composed of images 35L and 35R is superimposed on 3
Together, a virtual space can be formed. Paraphrase
Then, a pseudo view can be formed. Such a book
The HMD device 1 according to the embodiment has its own
It is easy to grasp the position and assist in maneuvering
You. Actual visibility, especially at night, cloudy and rainy days
Is poor, the steering can be favorably supported. Ma
Screen members 12L and 12R are made up of half mirrors.
To see the actual space
Steering is possible. FIG. 7 shows an HMD according to another embodiment of the present invention.
It is a figure showing use condition of device 1A. H of the present embodiment
The MD device 1A is an HMD device according to the embodiment shown in FIGS.
Similar to 1 and corresponding components are denoted by the same reference numerals.
Only different points will be described. Of this embodiment
In the HMD device 1A, the arithmetic processing computer 10
It has a display content switching function instead of the focus processing function.
You. This arithmetic processing computer 10 is constructed as described above.
The gaze time at a predetermined position in the virtual space
When the time exceeds the fixed time, the display contents of the display 8 are changed.
The display 8 is controlled to change. More specifically, in the present embodiment,
Means that the image information input means 17 uses information instead of the storage means.
It has an acquisition means. The information acquisition means is based on
Detect flight-related information, including flight status such as speed and speed
It has a sensor, and the detected flight-related information is
It is given to the physical computer 10. The arithmetic processing computer 10 checks
Image 45 of the displayed flight-related information as a graph
Generate image data. Therefore, the HMD device 1A
A graph representing flight-related information is stored in an actual space 43 outside the airplane 42.
Image 45 is superimposed and displayed in front of the pilot.
A virtual space in which a graph exists can be created. The image 45 of the graph is the gaze position of the user 2.
Is switched between display and display stop. This
Display content to display a predetermined graph as shown
The display contents are switched according to the gaze position of the user 2.
It is. This switching is performed by the user 2 from the window of the aircraft 42
When the user is watching the space 43, a graph is displayed.
When the user 2 is watching the instrument panel 47 in the aircraft 42
To stop displaying the graph. Such display contents
The switching is performed when the gaze time of the outer space 43 and the instrument panel 47 is changed.
For example, when 0.5 seconds or more, switch the display
You. When displaying, the position and posture of the head 3
Irrespective of the display area, display is performed in a certain area on each of the display surfaces 6L and 6R.
You. The flight-related information displayed as a graph is the flight altitude
And important information such as flight speed. By switching the display contents in this way,
When looking at the space 43 outside the aircraft 42,
Simultaneous confirmation of important flight-related information while watching the outside space 43
You can detailed flight related information including other information
When looking at the instrument panel 47 for confirmation, the display 8
Display may be in the way, making it difficult to see the instrument panel 47.
And can be prevented. Also, check each line of sight of the left and right eyes 4L and 4R.
The user 2 gazes at the instrument panel 47 by issuing
Is there, or the sky outside the aircraft 42 behind the dashboard 47
It is possible to reliably detect whether you are looking at the interval 43
The switching as described above can be reliably performed according to the intention of the user 2.
Can be carried out. Further, as described above, the user gazes at a predetermined position.
This allows the display contents to be switched,
2 does not require a separate manual operation for display switching.
In addition, the labor for switching can be eliminated. But
Is very convenient and replaces the display switching operation.
Other operations can be performed at
Can be. The screen members 12L and 12R are
The effect of being composed of mirrors can be achieved as well.
Can be. FIG. 8 shows still another embodiment of the present invention.
It is a figure showing an example of display contents of HMD device 1B. Real truth
The HMD device 1B according to the embodiment is different from the HM device according to the embodiment in FIG.
It is similar to the D device 1A, and the corresponding components are the same.
And only different points will be described. This implementation
In the HMD device 1B according to the embodiment, the arithmetic processing computer 10
It has the same display content switching function as the HMD device 1A described above.
At the same time, image information is input in the same manner as in the above-described HMD device 1A.
Means 17 include an information acquisition unit. Form of this implementation
The information acquiring means of the HMD device 1B in the standby state
In addition to the flight conditions such as
The state of the ship, such as the temperature of the
Including the surrounding environment such as atmospheric pressure, temperature and humidity
Equipped with a sensor to detect flight-related information,
The computer 10 converts the detected flight-related information into a plurality of graphs.
The image data of the image 45 shown in a graph is generated.
Therefore, the HMD device 1 </ b> B
Image 4 of the graph representing the flight-related information detected during interval 43
5 is superimposed and the graph is displayed in front of the pilot.
You can create an existing virtual space. The graph image 45 is the main graph at the center.
Surround and surround image portion 48 and this main graph
And four sub-groups in this embodiment.
It has rough images 49a to 49d. Each image part 48,
The graphs 49a to 49d are different from each other.
Therefore, the information to be displayed is different from each other. Also the main graph image section
Minute 48 is an image of a detailed graph, and each subgraph image
The image parts 49a to 49d are simplified graph images.
And each subgraph image portion 49a-49d has
Each of which has operation button images 50a to 50d.
You. Each of these image portions 48, 49a to 49b is
Irrespective of the position and the posture of the display surface 6L, 6R.
It is displayed in a certain area. In the present embodiment, the user 2
Accordingly, the graph displayed in the main graph image portion 48
Is changed so as to change. As mentioned above
As shown in FIG.
The user 2 has images 50a to 50d of these operation buttons.
And therefore, by these images 50a to 50d,
Gaze at the operation buttons that are created intentionally for a predetermined time or more.
The sub button having the image of the operation button.
Replace the rough image part with the main graph image part 48
indicate. For example, the operation button image 50a
The gaze time of the operated operation button is, for example, 0.5 seconds.
When the above is reached, the image 50a of the operation button is provided.
The graph of the subgraph image portion 49a
The image is displayed in the image portion 48 and the graph of the main graph image portion 48 is displayed.
The rough is displayed in the subgraph image portion 49a. This
Change the placement and detail of multiple graphs
The display contents are switched according to the gaze position of the user 2.
You. By switching the display contents in this way,
User 2 looks at the instrument panel 47 in the aircraft 42
Many flights while watching the space 43 outside the aircraft 42
You can check related information. The left and right eyes 4L,
User 2 displays by detecting each line of sight of 4R
You are looking at the graph of the container 8, the space outside the aircraft 42
43 can be detected with certainty.
Such switching should be performed in accordance with the intention of the user 2 without fail.
Can be. Further, as described above, the user gazes at a predetermined position.
This allows the display contents to be switched,
2 does not require a separate manual operation for display switching.
In addition, the labor for switching can be eliminated. But
Is very convenient and replaces the display switching operation.
Other operations can be performed at
Can be. The screen members 12L and 12R are
The effect of being composed of mirrors can be achieved as well.
Can be. FIG. 9 shows still another embodiment of the present invention.
It is a figure showing an example of display contents of HMD device 1C. Real truth
The HMD device 1C according to the embodiment is different from the HM device according to the embodiment in FIG.
Similar to the D device 1B, and the corresponding components are the same.
And only different points will be described. This implementation
The HMD device 1C in the form of
In the same manner as in, detected in the actual space 43 outside the aircraft 42
Superimposed image 45 of graph showing flight related information
Shows the virtual space where the graph exists in front of the pilot.
Can be produced. Image 45 is the main graph image
Portion 48 and each subgraph image portion 49a-49d.
I do. Each of these image portions 48, 49a-49b
Irrespective of the position and orientation of the display surface 3,
It is displayed in a certain area. In the HMD device 1B of the above embodiment,
Each of the subgraph image portions 49a to 49d is
Although each had images 50a to 50d,
In the HMD device 1C according to the embodiment, the image 5 of the operation button is displayed.
0a to 50d, instead of the pilot
An operation button 51 manually operated by the user is separately provided.
The operation button 51 is provided, for example, on an aircraft such as a control stick.
It may be provided in the control input unit for control,
Although it may be provided at another position, it is provided at the control input section.
In such a configuration, the pilot does not control the aircraft 42.
Since the operation buttons 51 can be operated,
Excellent convenience. As described above, in the present embodiment, the user 2
Operation button 51 which is an operation means operated by
Contains. The arithmetic processing computer 9 is located at a predetermined position.
When the operation button 51 is operated while the
To change the display contents of the display 8
8 is controlled. Specifically, the operation button 51 is operated.
The main graph image according to the gaze position of the user 2 when
Display to change the graph displayed in image portion 48
The contents are switched. When the user 2 sets each graph image portion 49
a-49d, and thus each of these image portions 49a-4
Watched the graph virtually created by 9d
By operating the operation button 51 in the state, the user
The main graph is the sub-graph image part that 2 is watching
It is displayed in place of the image part 48. For example, a subgraph
Watch the graph produced by the image part 49a.
Then, when the operation button 51 is operated, the user is watching
The graph of the subgraph image portion 49a is
The image is displayed in the image portion 48 and the graph of the main graph image portion 48 is displayed.
Is displayed in the subgraph image portion 49a. like this
Like changing the placement and detail of multiple charts in
The display content is switched according to the gaze position of the user 2. By switching the display contents in this way,
User 2 looks at the instrument panel 47 in the aircraft 42
Many flights while watching the space 43 outside the aircraft 42
You can check related information. The left and right eyes 4L,
User 2 displays by detecting each line of sight of 4R
You are looking at the graph of the container 8, the space outside the aircraft 42
43 can be reliably detected,
Since the switching is based on the manual operation of the operation button 51,
The switching as described above is performed according to the intention of the user 2 without fail.
I can. Further, as described above, the user must not pay attention to a predetermined position.
By operating the operation button 51, the display content is displayed.
Can be switched, and when the user 2 switches the display,
There is no need for manual operation to select the display content,
The labor for switching can be reduced. But
Is very convenient and replaces the display switching operation.
It is possible to perform other operations more
I can help. Also, the screen members 12L, 12
The effect of R consisting of a half mirror is similarly achieved.
Can be achieved. An HMD device according to still another embodiment of the present invention
As a device that includes operating means.
You may. The HMD device according to the present embodiment is configured as shown in FIGS.
It is similar to the HMD device 1 of the embodiment, and has a corresponding structure.
The same reference numerals are given to the components, and the different points with reference to FIG.
Will be described only. The HMD device according to the present embodiment includes:
An operation for executing a predetermined operation as indicated by a virtual line in FIG.
Means 55 is further included. The arithmetic processing computer 10 includes:
Control function of operating means instead of pseudo focus processing function
have. The arithmetic processing computer 10 is configured as described above.
Of the specified position in the virtual space created
When the viewing time is equal to or longer than a predetermined time, the operation means 55
The switching of execution and stop of the operation is controlled. The operating means is, for example, an object in a virtual space.
Automatic tracking (lock-on) automatic tracking means
Person 2 is at a predetermined position in the virtual space, and therefore a predetermined object
By gazing at it for 0.5 seconds or more,
Automatic tracking of the object watched by the user 2 is started. And vice versa
During such automatic tracking, the object is
Auto-tracking is canceled by gazing for a predetermined time or more.
You. In this manner, the operating means is operated in accordance with the gaze position of the user 2.
Execution and stop are switched. Like this
By observing the fixed position, the so-called switch
Switching on / off to control predetermined operation
And eliminate the trouble of the operation for switching this operation.
Can be. Further, in another embodiment of the present invention, the HM
As the D device, the above-mentioned operation means and operation button 51 are used.
It may be implemented as a device that includes. Form of this implementation
The HMD device according to the embodiment is the HMD device according to the embodiment shown in FIGS.
Similar to the device 1 and the corresponding components are denoted by the same reference numerals.
And only the differences will be described with reference to FIG.
You. The HMD device of the present embodiment is shown by a virtual line in FIG.
As described above, the operation means 55 for executing a predetermined operation is different from the operation means 55 shown in FIG.
It includes the operation button 51 described in the embodiment. This fruit
Also in the embodiment, the arithmetic processing computer 10 executes
In place of the focus processing function, the
I have. The arithmetic processing computer 10 operates as described above.
At a specified position in the virtual space
When the operation button 51 is operated in a state where the
The control of the execution and stop of the operation by 55 is controlled. The operating means is, for example, an object in a virtual space.
Automatic tracking (lock-on) automatic tracking means
Person 2 determines a predetermined position in the virtual space, and thus a predetermined object.
When the operation button 51 is operated while gazing,
Automatic tracking of the object watched by the user 2 is started. Also
Conversely, in such a state where automatic tracking is performed,
When the operation button 51 is operated while watching the
Release the tail. Thus, according to the gaze position of the user 2
Thus, the execution and stop of the operation of the operation means are switched.
In this manner, the operation button 51 is
By operating the so-called switch on / off
Switching can be performed to control the specified operation.
Operation time for switching the operation of
Wear. Specifically, select an object to track automatically
Operations such as manual operation of the cursor buttons and
There is no need to move the cursor by moving the head 3
Yes, it is possible to select an object just by looking
Time can be reduced. FIG. 10 shows still another embodiment of the present invention.
It is a figure showing an example of the display contents of HMD device 1D. Book
The HMD device 1D according to the embodiment has the form shown in FIGS.
Is similar to the HMD device 1 in the
Are denoted by the same reference numerals, and only different points will be described.
The above-described HMD device 1 is mounted on the aircraft 42
Although configured, the HMD device 1D of the present embodiment is:
For example, it is used as a bodily sensation device. In the above-described HMD device 1, the screen member
Half mirrors were used for 12L and 12R
Also, the HMD device 1D of the present embodiment has light transmittance
No light-shielding mirror is used. Screen like this
Banana placed on table 60 on members 12L and 12R
The image 45 of the corner 61 and the apple 62 is displayed. this
The image 45 has parallax as in the above embodiments.
It is composed of an image 35L for the left eye and an image 35R for the right eye.
This image 45 is subjected to the pseudo focus processing as described above.
And the user 2 is watching the banana 61
It is displayed as being in focus. Such a display
Possible sensation devices allow for a sense of discomfort closer to the real space.
You can experience a small virtual space. Such a pseudo
The HMD device 1D as a bodily sensation device that forms a similar visual field is:
Amusement equipment (game machine), rehabilitation support equipment and relaxation
It can be used as a zation device. As described above, the HMD device is used as a bodily sensation device.
Even if the calculation processing computer 10
Instead of the focus processing function, the gaze time at a predetermined position is
Display content switching to switch the display content when it is longer than
Function may be provided, or the operation button 51 may be provided separately.
The operation button 51 in a state where the user gazes at a predetermined position.
Display contents switching to switch display contents by operating
A configuration having a function may be adopted. It also includes operating means,
The arithmetic processing computer 10 replaces the pseudo focus processing function.
When the gaze time at a predetermined position is longer than a predetermined time,
Has a function to switch between execution and stop of the operation of the operation means
Or the operation button 51 is separately provided,
Operate the operation button 51 while gazing at a predetermined position.
Switches between execution and stop of the operation of the operation means by
A configuration having a function may be adopted. Use as a bodily sensation device
In this case, the operating means is, for example, a gas having a predetermined scent.
It is a gas discharging means that blows out around the user 2,
Works by gazing at banana 61 as described above
When the means is executed, the gas having the scent of the banana 61 is
You may make it discharge. In this way, the visual
Give stimuli other than senses such as smell, hearing, touch, etc.
To achieve a more immersive experience.
Can be The above embodiments are merely illustrative of the present invention.
It is possible to change the configuration within the scope of the present invention.
it can. For example, the HMD device of each of the above-described embodiments is
It is also possible to use a combination of two or more devices.
In other words, as in the embodiment shown in FIGS.
Forming a field and implementing one of FIGS.
A configuration for displaying a graph in the form of (1) may be used. Also figure
As described with reference to 10, used as a bodily sensation device
Even in the case, pseudo focus function and display content switcher
It may have a configuration having both functions. 8 and
When displaying a graph as shown in Figure 9 and subgraph
The number of image parts is not limited to four,
Or five or more subgraph image parts
Will be placed next to one side of the main graph image
You may do it. When mounted on an aircraft as described above,
In detecting the position and orientation of the head 3 of the user 2,
To add more global positioning systems
You may use it. Moving objects other than aircraft, for example,
Mounted on a ship at night and when there is heavy fog
When the visibility of the ship is poor,
You may. As the image information input means 17,
A step may be used, and the imaging means is used in this manner.
If there is, for example, as in the embodiment of FIGS.
Onboard the aircraft, the pilot from User 2,
Use an imager to capture directions that are not visible due to obstacles such as the body
To form a simulated field of view to help maneuver
Can be suitably used. In addition, the imaging means is a robot
Or place it on a mobile object so that humans cannot stand
A pseudo vision similar to the state of entering a place is formed, and this human
Remotely monitor places where it cannot stand, or
Maintenance of such places and the equipment installed in those places
It can be suitably used for Also like this
Can be used to experience remote areas
For recreation, play, rehabilitation and relaxation
It can also be suitably used as a sensitive device. According to the present invention, stereoscopic display is performed
Identify the position where the user gazes in the displayed virtual space
Depending on the position where the user gazes
Thus, a predetermined process can be executed. Therefore
For example, depending on the position where the user gazes, the display means
By changing the display contents and display state,
In addition, a detailed and highly convenient display can be performed. Also
For example, depending on the position where the user gazes,
Specified by switch operation
If you try to perform the operation of
Can be reduced or eliminated. In this way the user
Performs predetermined processing according to the position
And can achieve the above excellent effects
You. Image with additional function based on gaze position
A display device can be realized. Also, obtained when the user looks at the real space
Like the image, the sharpness differs partially and the gaze position
Image that becomes sharper as it shifts from
Can be shown. Therefore, the user views the real space.
You can get a closer vision
You. Therefore, the user is in a state with very little discomfort
You can see the image. ## EQU00002 ##

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an HMD device 1 according to an embodiment of the present invention. FIG. 2 is a diagram for explaining stereoscopic display. FIG. 3 is a diagram illustrating a specific principle of a gaze position. FIG. 4 is a diagram showing an image 35R for the right eye in detail. FIG. 5 is a diagram for explaining a pseudo focus process. FIG. 6 is a diagram showing a use state of the HMD device 1. FIG. 7 is a diagram illustrating a use state of an HMD device 1A according to another embodiment of the present invention. FIG. 8 shows an HMD device 1 according to still another embodiment of the present invention.
FIG. 14 is a diagram showing an example of display contents of B. FIG. 9 is an HMD apparatus 1 according to still another embodiment of the present invention.
It is a figure showing an example of the display contents of C. FIG. 10 is a diagram showing an example of display contents of an HMD device 1D according to still another embodiment of the present invention. [Description of Signs] 1, 1A to 1D HMD device 2 User 3 Head 4L, 4R Eye 6L, 6R Display surface 8 Display 9 Eye gaze detector 10 Operation processing computer 11 Position and orientation detector 12L, 12R Screen member 13L, 13R Projector 14L, 14R Eye camera 15L, 15R Reflecting member 16 Eye camera controller 17 Image information input means 24, 25 HMS unit 26 HMS controller 51 Operation button 55 Operation means

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-11-195131 (JP, A) JP-A-11-295648 (JP, A) JP-A-5-191683 (JP, A) JP-A-2000-187553 (JP, A) JP-A-7-107518 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 13/00 G06T 15/00 G06F 3/033

Claims (1)

(57) [Claims 1] Images having parallax are respectively displayed on two display surfaces which are mounted on a user's head and are respectively arranged facing right and left eyes of the user. Display means; gaze detection means for detecting the gaze directions of the left and right eyes of the user; position / posture detection means for detecting the position and posture of the user's head; gaze directions detected by the gaze detection means; Arithmetic processing means for calculating, based on the position and orientation of the head detected by the position and orientation detection means, a position in the virtual space which is virtually created by displaying an image using the display means and in which the user gazes. The arithmetic processing means, wherein the image portion at the position where the user gazes has higher sharpness than the remaining image portion, and the shift amount in the perspective direction from the position where the user gazes becomes large Nitsu An image display device, wherein the display means is controlled so as to display an image having reduced sharpness.