JP5332127B2 - Head-mounted display device - Google Patents

Description

  The present invention relates to a head-mounted display device that provides images visually recognized by the wearer's left eye and right eye, for example, helicopters and aircraft pilots, training simulator users, consumer game machines and information The present invention relates to a head-mounted display device that provides an image to a user such as a terminal.

  As a display device for an information device in a form worn on the body, a form worn on the head is the mainstream (see, for example, Patent Document 1). In addition, the display device may be provided with a night vision function in an aircraft or in a special application having a purpose such as security, crime prevention or defense. Thus, the wearer can visually recognize the outside world even in an environment such as at night. The night vision function uses the faint amplification of the photomultiplier tube. For example, it amplifies light in the wavelength range from red visible light (red light) to near infrared (infrared light). To do.

FIG. 4 is a diagram showing an external configuration of a conventional head-mounted display device 100 worn on the head of the wearer, and FIG. 5 is a block diagram showing a schematic configuration of the head-mounted display device 100. is there. The head-mounted display device 100 is disposed on the left side of the helmet 1, the left side of the helmet 1, the left-eye night-vision camera 2 </ b> L that captures the image for the left eye, and the right side of the helmet 1. In addition, the right-eye night-vision camera 2R that captures the right-eye image, the display unit 10, the left-eye video signal from the left-eye night-vision camera 2L, and the right-eye video signal from the right-eye night-vision camera 2R. And a video signal processing unit 103 that outputs a left-eye video correction signal and a right-eye video correction signal to the display unit 10. In addition, the display unit 10 includes a display element (left-eye emitting means) 11L and a lens (left-eye optical system) 12L arranged on the upper left of the wearer P, and a display arranged on the upper right of the wearer P. An element (right eye emitting means) 11R and a lens (right eye optical system) 12R, and a visor (or combiner) (optical system) 13 disposed in front of the left eye EL and right eye ER of the wearer P Have. Then, the left-eye display light emitted from the display element (left-eye emitting means) 11L passes through the lens 12L and is reflected by the reflecting surface of the visor 13, so that the left eye EL of the wearer P is reflected. The right-eye display light that is guided and emitted from the display element (right-eye emitting means) 11R passes through the lens 12R and is reflected by the reflecting surface of the visor 13, whereby the right eye of the wearer P Guided by ER. Therefore, the wearer P can visually recognize the observation target in three dimensions, and can also visually recognize the real object in front by the light transmitted through the visor 13.
At this time, the video signal processing unit 103 receives the left-eye video signal from the left-eye night-vision camera 2L and the right-eye video signal from the right-eye night-vision camera 2R, whereby the left-eye video signal is input to the display unit 10. The video signal and the right-eye video signal are output as they are as the left-eye video correction signal and the right-eye video correction signal.
JP 2006-189634 A

  Here, the relationship between the left eye night vision camera 2L, the right eye night vision camera 2R, the left eye EL and the right eye ER of the wearer P, and the observation object S will be described. FIG. 6 is a diagram when a long-distance cube (observation object) S is observed with the left-eye night-vision camera 2L, the right-eye night-vision camera 2R, the left-eye EL, and the right-eye ER. FIG. 7 is a view when a nearby cube (observation object) S and sphere T are observed with the left-eye night-vision camera 2L, the right-eye night-vision camera 2R, the left-eye EL, and the right-eye ER. .

  Here, as shown in FIGS. 6 and 7, the position of the cube S includes a direction angle α from the left-eye night-vision camera 2L to the cube S and a direction angle from the right-eye night-vision camera 2R to the cube S. If β and the inter-camera distance dC between the left-eye night-vision camera 2L and the right-eye night-vision camera 2R are known, it can be uniquely calculated. A left-eye image captured by the left-eye night-vision camera 2L with a known viewing angle (for example, a viewing angle of −15 ° to 15 ° in the vertical direction and −20 ° to 20 ° in the horizontal direction) is created. Thus, the direction angle α is determined by the position of the cube image in the left-eye image. Similarly, the right-eye image captured by the right-eye night-vision camera 2R with a known viewing angle (for example, the viewing angle is −15 ° to 15 ° in the vertical direction and −20 ° to 20 ° in the horizontal direction). Is created, the direction angle β is determined by the position of the cube image in the right-eye image.

  At this time, as shown in FIG. 6, when observing the distant cube S, there is almost no difference between the direction angle θ and the direction angle α from the left eye EL of the wearer P to the cube S. There is almost no difference between the direction angle φ and the direction angle β from the right eye ER of the person P to the cube S. Therefore, in the head-mounted display device 100, the video signal processing unit 103 receives the left-eye video signal from the left-eye night camera 2L and the right-eye video signal from the right-eye night camera 2R. Even if the left-eye video signal and the right-eye video signal are directly output to the display unit 10 as the left-eye video correction signal and the right-eye video correction signal, the wearer P visually recognizes the cube at an accurate distance. It was done.

However, as shown in FIG. 7, when the cube S in the vicinity is observed, the difference between the direction angle θ and the direction angle α increases, and similarly, the difference between the direction angle φ and the direction angle β also increases. . Similarly, when the sphere T is observed, the difference between the direction angle θ and the direction angle α increases, and similarly, the difference between the direction angle φ and the direction angle β also increases. Therefore, in the head-mounted display device 100, the video signal processing unit 103 receives the left-eye video signal from the left-eye night-vision camera 2L and the right-eye video signal from the right-eye night-vision camera 2R. Thus, when the left-eye video signal and the right-eye video signal are directly output to the display unit 10 as the left-eye video correction signal and the right-eye video correction signal, the wearer P can visually recognize the cube at an accurate distance. I felt uncomfortable.
FIG. 8A is a diagram showing an image of a nearby cube S and sphere T taken by the left-eye night-vision camera 2L and the right-eye night-vision camera 2R. (B) is a figure which shows the image supposed to be observed with the left eye EL and the right eye ER.
Accordingly, an object of the present invention is to provide a head-mounted display device capable of visually recognizing an observation object at an accurate distance even when observing a nearby observation object.

The head-mounted display device of the present invention made to solve the above problems is mounted on the left side of the head of the wearer and is a left-eye night-vision camera for taking a left-eye image, and the mounting A right-eye night-vision camera that is mounted on the right side of the person's head and that captures a right-eye image, a display unit, and a left-eye video signal and right-eye night vision from the left-eye night camera. A video signal processing unit that receives a video signal for the right eye from the camera and outputs a video correction signal for the left eye and a video correction signal for the right eye to the display unit; and the display unit displays for the left eye A left-eye emitting unit that emits light; a right-eye emitting unit that emits right-eye display light; and an optical path of the left-eye display light is changed and guided to the left eye of the wearer, and the right eye A head-mounted display device having an optical system that changes the optical path of the display light for use and guides it to the right eye of the wearer Thus, the video signal processing unit stores the position of the left eye and the right eye of the wearer and the position of the left-eye night camera and the right-eye night camera as position data. And a left-eye video signal input from the left-eye night-vision camera and a right-eye video signal input from the right-eye night-vision camera based on the principle of triangulation using the position data. Te is moved in pixel units ranging recognized respectively If it is the observation object, the wearer's left eye and the image correction signal for the left eye are observed from the position of the right eye and the right eye image correction signal Correction signal generating means for correcting, left-eye output means for outputting a left-eye video correction signal to the left-eye emission means, and right-eye output for outputting a right-eye video correction signal to the right-eye emission means Output means.

According to the head-mounted display device of the present invention, the left-eye video signal input from the left-eye night-vision camera and the right-eye video signal input from the right-eye night-vision camera, for example, Wearer based on the principle of triangulation using the difference between the distance between the left and right eyes of the wearer and the distance between the left and right eye night vision cameras The left eye image correction signal and the right eye image correction signal that are observed from the left eye and right eye positions are corrected.
For example, as shown in FIG. 7, the position of the cube (observation target) S includes the direction angle α from the left-eye night-vision camera 2L to the cube S and the right-eye night-vision camera 2R to the cube S. If the direction angle β and the inter-camera distance dC between the left-eye night-vision camera 2L and the right-eye night-vision camera 2R are known, it can be uniquely calculated. Therefore, first, as shown in FIG. 3A, for example, the left eye has a known viewing angle (for example, the viewing angle is -15 ° to 15 ° in the vertical direction and -20 ° to 20 ° in the horizontal direction). By creating the left eye image GL photographed by the night vision camera 2L, the direction angle α is determined by the position of the cubic image in the left eye image GL. Next, the viewing angle of the position of the cubic image from which the direction angle α from the left-eye night vision camera 2L is calculated is known (for example, the viewing angle is −15 ° to 15 ° in the vertical direction and −20 in the horizontal direction). The direction angle β from the right-eye night-vision camera 2R is determined by searching the right-eye image GR taken by the right-eye night-vision camera 2R. Thereby, since the inter-camera distance dC is stored in the storage means, the position of the cube S is calculated.

Next, when the position of the cube S is calculated, since the distance dE between the left eye EL and the right eye ER of the wearer P is stored in the storage means, the position of the left eye EL of the wearer P is calculated. A direction angle θ that is observed and a direction angle φ that is observed from the position of the right eye ER of the wearer P are determined. Thereby, as shown in FIG. 3B, the position of the cube S is changed from the direction angle α when taken by the left-eye night-vision camera 2L to the direction angle θ observed from the position of the left eye EL. Correct so that
Similarly, the correction is performed so that the direction angle β when observed from the position of the right eye ER is changed from the direction angle β when captured by the right-eye night-vision camera 2R. Further, similarly, the position of the sphere T is corrected from the direction angle α ′ taken with the left-eye night vision camera 2L to the direction angle θ ′ observed from the position of the left eye EL. . Similarly, the correction is performed so that the direction angle β ′ when observed from the position of the right eye ER is changed from the direction angle β ′ when captured by the right-eye night-vision camera 2R.
As described above, the left-eye video signal input from the left-eye night vision camera 2L is corrected to the left-eye video correction signal that is observed from the position of the left eye EL of the wearer P, and the right eye is corrected. The right-eye video signal input from the night vision camera 2R is corrected to a right-eye video correction signal that is observed from the position of the right eye ER of the wearer P.

Therefore, according to the head-mounted display device of the present invention, the left eye image correction signal and the right eye image that are observed from the positions of the left eye and the right eye are applied to the left eye and the right eye of the wearer. Since the display light for the left eye and the display light for the right eye created by the correction signal are guided, the wearer can visually recognize the observation target at an accurate distance even when observing an observation target nearby.
In the head-mounted display device according to the present invention, the correction signal creating means may be arranged for the right eye for a missing portion caused by moving an image recognized as an observation object in the left eye video signal. An image using the video signal for the left eye is interpolated by inserting an image using the video signal, and an image using the video signal for the left eye is generated in the missing portion generated by moving the image recognized as the observation object in the video signal for the right eye. May be interpolated by inserting.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and it goes without saying that various aspects are included without departing from the spirit of the present invention.

  FIG. 1 is a diagram showing an external configuration of a head-mounted display device 20 according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a schematic configuration of the head-mounted display device 20. The head-mounted display device 20 is arranged on the left side of the helmet 1, the left side of the helmet 1, the left-eye night-vision camera 2 </ b> L that captures the left-eye image GL, and the right side of the helmet 1. The right-eye night-vision camera 2R that captures the right-eye image GR, the display unit 10, the left-eye video signal from the left-eye night-vision camera 2L, and the right-eye night-vision camera 2R A video signal processing unit 3 that receives a video signal and outputs a left-eye video correction signal and a right-eye video correction signal to the display unit 10 is provided.

The left-eye night-vision camera 2L and the right-eye night-vision camera 2R have a night-vision function that amplifies the amount of captured infrared light (for example, a wavelength of 610 nm or more). Thereby, the observation object S that emits or reflects external infrared light can be confirmed even in an environment such as at night.
The left-eye night-vision camera 2L and the right-eye night-vision camera 2R enable the stereoscopic vision of the observation object S, so that the left and right sides of the helmet 1 are separated by a certain distance (dC) between the cameras. Are arranged at the same height, and each photographing direction (optical axis) is perpendicular to the inter-camera distance (dC) direction.
Thereby, for example, a left-eye image GL photographed by the left-eye night-vision camera 2 </ b> L having a viewing angle of −15 ° to 15 ° in the vertical direction and −20 ° to 20 ° in the horizontal direction is created. A right-eye image GR photographed by the right-eye night-vision camera 2 </ b> R having angles of −15 ° to 15 ° in the vertical direction and −20 ° to 20 ° in the horizontal direction is created. Therefore, if the position in the left-eye image GL is known, the direction angle α from the left-eye night-vision camera 2L is determined. Similarly, if the position in the right-eye image GR is known, the direction angle β from the right-eye night-vision camera 2R is determined.

  The display unit 10 includes a display element (left-eye emitting means) 11L and a lens (left-eye optical system) 12L disposed on the upper left of the wearer P, and a display element (upper right of the wearer P) ( A right eye emitting means) 11R and a lens (right eye optical system) 12R, and a visor (optical system) 13 disposed in front of the left eye EL and right eye ER of the wearer P; Then, the left-eye display light emitted from the display element (left-eye emitting means) 11L passes through the lens 12L and is reflected by the reflecting surface of the visor 13, so that the left eye EL of the wearer P is reflected. The right-eye display light that is guided and emitted from the display element (right-eye emitting means) 11R passes through the lens 12R and is reflected by the reflecting surface of the visor 13, whereby the right eye of the wearer P Guided by ER. Therefore, the wearer P can visually recognize the observation object S in a three-dimensional manner, and can also visually recognize the front actual object by the light transmitted through the visor 13.

  The helmet 1 has a hemispherical shape and is worn on the head of the wearer P. The wearer P has the left eye EL and the right eye in the same direction as the shooting direction (optical axis) of the left-eye night-vision camera 2L and the right-eye night-vision camera 2R that pass through the center of the inter-camera distance (dC). The helmet 1 will be worn so that the center with the ER comes. At this time, it is further preferable that the center of the inter-camera distance (dC) and the center of the left eye EL and the right eye ER be the same.

The video signal processing unit 3 includes a CPU 3a, and further includes a memory (storage means) 32 for storing position data and the like, a left-eye display electronic circuit 3b, and a right-eye display electronic circuit 3c.
The control and calculation contents executed by the CPU 3a will be described separately for each function. As shown in FIG. 2, the correction signal generating means 30 for generating the left-eye video correction signal and the right-eye video correction signal, and the display element ( Left-eye output means 31L that outputs a left-eye video correction signal to 11L, and right-eye output means that outputs a right-eye video correction signal to the display element (right-eye emission means) 11R 31R.

The memory 32 indicates the distance dE between the left eye EL and the right eye ER of the wearer P and the inter-camera distance dC between the left eye night vision camera 2L and the right eye night vision camera 2R. For example, the wearer P stores the distance dE between the left eye EL and the right eye ER and the inter-camera distance dC in advance before use.
The memory 32 stores relative coordinates (XYZ coordinates). The relative coordinates are a three-dimensional coordinate system (XYZ coordinates) formed by the left-eye night-vision camera 2L and the right-eye night-vision camera 2R, and the origin and the direction of each coordinate axis can be arbitrarily determined. In the present embodiment, the direction from the left-eye night-vision camera 2L to the right-eye night-vision camera 2R is the X-axis direction, and the direction perpendicular to the X-axis direction and perpendicular to the photographing direction (optical axis) is the Y-axis direction. The direction perpendicular to the X-axis direction and the Y-axis direction is defined as the Z-axis direction, and the origin is defined as the midpoint between the left-eye night-vision camera 2L and the right-eye night-vision camera 2R. is there.

  The left-eye display electronic circuit 3b and the right-eye display electronic circuit 3c are respectively supplied to the display element (left-eye emission means) 11L and the display element (right-eye emission means) 11R from the data (for the left eye). Based on the video correction signal and the right-eye video correction signal), a drive signal, a control signal, and the like are output.

The correction signal creation means 30 stores the left-eye video signal input from the left-eye night-vision camera 2L and the right-eye video signal input from the right-eye night-vision camera 2R in the memory 32. Based on the principle of triangulation using position data, control is performed to correct the image correction signal for the left eye and the image correction signal for the right eye, which are assumed to be observed from the positions of the left eye EL and the right eye ER of the wearer P. Is.
For example, as shown in FIG. 3A, first, by creating a left-eye image GL photographed by the left-eye night-vision camera 2L, the direction angle α is changed to a cubic image in the left-eye image GL. Determined by position. At this time, it is preferable that the position of the cube image is determined in units of pixels in a range recognized as a cube, not one pixel in the left-eye image GL. Next, the position of the cubic image from which the direction angle α from the left-eye night-vision camera 2L is calculated is the right-eye image captured by the right-eye night-vision camera 2R using, for example, the area correlation method or the like. The direction angle β from the night-vision camera 2R for the right eye is determined by searching from the GR. Thereby, since the position data of the inter-camera distance dC is stored in the memory 32, the position of the cube S in relative coordinates (XYZ coordinates) is calculated.

Next, the direction angle θ assumed to be observed from the position of the left eye EL of the wearer P by the calculated position of the cube S and the distance dE between the left eye EL and the right eye ER of the wearer P. And a direction angle φ that is observed from the position of the right eye ER of the wearer P is determined. Thereby, as shown in FIG. 3B, the position of the cube S is changed from the direction angle α when taken by the left-eye night-vision camera 2L to the direction angle θ observed from the position of the left eye EL. Correct so that Similarly, the correction is performed so that the direction angle β when observed from the position of the right eye ER is changed from the direction angle β when captured by the right-eye night-vision camera 2R. Further, similarly, the position of the sphere T is corrected from the direction angle α ′ taken with the left-eye night vision camera 2L to the direction angle θ ′ observed from the position of the left eye EL. . Similarly, the correction is performed so that the direction angle β ′ when observed from the position of the right eye ER is changed from the direction angle β ′ when captured by the right-eye night-vision camera 2R.
As described above, the left-eye video signal input from the left-eye night vision camera 2L is corrected to the left-eye video correction signal that is observed from the position of the left eye EL of the wearer P, and the right eye is corrected. The right-eye video signal input from the night vision camera 2R is corrected to a right-eye video correction signal that is observed from the position of the right eye ER of the wearer P.
At this time, in the video correction signal for the left eye, the position indicating the direction angle α may be lost because the position of the cube S is corrected from the direction angle α to the direction angle θ. Assuming that there is an object at a distant location, if there is an object at a distant location, there is no difference in the direction angle indicated by the left-eye video signal and the right-eye video signal. Is interpolated by inserting an image of the object having the direction angle α. Similarly, in the video correction signal for the right eye, the position indicating the direction angle β may be lost because the position of the cube S is corrected from the direction angle β to the direction angle φ. Assuming that there is an object at that position in the distance, interpolation is performed by inserting an image of the object having the direction angle β in the left-eye video signal.

  The left-eye output unit 31L performs control to output a left-eye video correction signal to the left-eye display electronic circuit 3b, and the right-eye output unit 31R transmits to the right-eye display electronic circuit 3c. The control which outputs the image correction signal for the image is performed.

  As described above, according to the head-mounted display device 20, the left-eye video signal input from the left-eye night-vision camera 2L and the right-eye video input from the right-eye night-vision camera 2R. Based on the principle of triangulation using the difference between the distance dE between the left eye EL and the right eye ER of the wearer P and the inter-camera distance dC, the signal of the wearer P's left eye EL and right eye ER It can correct | amend to the video correction signal for left eyes and the video correction signal for right eyes which are supposed to be observed from the position. Accordingly, the left eye EL and the right eye ER of the wearer P have a left eye created from the left eye video correction signal and the right eye video correction signal that are observed from the positions of the left eye EL and the right eye ER. Since the display light for display and the display light for the right eye are guided, the wearer P can visually recognize the observation target at an accurate distance even when observing an observation target nearby.

(Other embodiments)
(1) In the head-mounted display device 20 described above, the video signal processing unit 3 is configured to output the left-eye video correction signal and the right-eye video correction signal to the display unit 10. When it is determined that there is no object, the left-eye video signal and the right-eye video signal may be directly output to the display unit as the left-eye video correction signal and the right-eye video correction signal.

  INDUSTRIAL APPLICABILITY The present invention can be used for, for example, a head-mounted display device that provides an image to a user of a helicopter, an aircraft pilot, a training simulator, a consumer game machine, an information terminal, or the like. .

It is a figure which shows the external appearance structure of the head mounted display apparatus which is one Embodiment of this invention. 1 is a block diagram illustrating a schematic configuration of a head-mounted display device that is an embodiment of the present invention. It is a figure for demonstrating the relationship between the night vision camera for left eyes, the night vision camera for right eyes, the left eye and the right eye, and an observation target. It is a figure which shows the external appearance structure of the conventional head mounted display apparatus with which a wearer's head is mounted | worn. It is a block diagram which shows schematic structure of the conventional head mounted display apparatus. It is a figure when the observation target object is observed with the night vision camera for left eyes, the night vision camera for right eyes, the left eye, and the right eye. It is a figure when the observation target object is observed with the night vision camera for left eyes, the night vision camera for right eyes, the left eye, and the right eye. It is a figure for demonstrating the relationship between the night vision camera for left eyes, the night vision camera for right eyes, the left eye and the right eye, and an observation target.

Explanation of symbols

1 Helmet (or head holding mechanism such as headgear)
2L Night-vision camera for left eye 2R Night-vision camera for right eye 3,103 Video signal processing unit 3a CPU
3b Display electronic circuit for left eye 3c Display electronic circuit for right eye 10 Display unit 11L Display element (left eye emitting means)
11R display element (emitter for right eye)
12L lens (optical system for left eye)
12R lens (optical system for right eye)
13 Visor (or combiner) (optical system)
20,100 Head-mounted display device 30 Correction signal generation means 31L Output means for left eye 31R Output means for right eye 32 Memory (storage means)
EL Left eye ER Right eye P Wearer

Claims (2)

A left-eye night-vision camera that is mounted on the left side of the wearer's head and that captures a left-eye image;
A right-eye night-vision camera that is mounted on the right side of the wearer's head and that captures a right-eye image;
A display unit;
The left-eye video signal is input from the left-eye night camera and the right-eye video signal is input from the right-eye night camera, and the left-eye video correction signal and the right-eye video correction signal are input to the display unit. An output video signal processing unit,
The display unit includes left-eye emission means for emitting left-eye display light;
Right-eye emitting means for emitting right-eye display light;
A head-mounted display having an optical system that changes the optical path of the display light for the left eye and guides it to the left eye of the wearer and guides the optical path of the display light for the right eye to the right eye of the wearer A device,
The video signal processing unit, the storage means for storing the position of the left eye and the right eye of the wearer, and the position of the night vision camera for the left eye and the night vision camera for the right eye as position data;
The left-eye video signal input from the left-eye night-vision camera and the right-eye video signal input from the right-eye night-vision camera are each calculated based on the principle of triangulation using the position data. The image is moved in units of pixels within the range recognized as the observation object, and corrected to the left-eye video correction signal and the right-eye video correction signal that are observed from the left and right eye positions of the wearer. Correction signal generating means;
Left-eye output means for outputting a left-eye video correction signal to the left-eye emission means;
A head-mounted display device comprising: right-eye output means for outputting a right-eye video correction signal to the right-eye emitting means. The correction signal generating means interpolates by inserting an image using the right-eye video signal into a missing portion generated by moving an image recognized as an observation object in the left-eye video signal. ,
The interpolation is performed by inserting an image using the left-eye video signal into a missing portion generated by moving an image recognized as an observation object in the right-eye video signal. The head-mounted display device according to 1.

Images