JPH05241540A - Visual information display device - Google Patents

Abstract

PURPOSE:To provide a visual information display device synthesizing a real image changing the direction and the position following up the change of the direction and the position of a head and a CG image and displaying it in front of a mounting person. CONSTITUTION:This device is constituted by providing a half mirror 50 arranged on the visual line of a viewer in the job site a base mount 10 mounted on viewer's head H to hold the half mirror 50 on the position, a sensor 80 detecting the direction and the position of the head H, a computer graphic device 90 outputting the computer image of the part corresponding to the direction and the position of the head H based on the detected result of the senser 80, a televisin camera 102 picking up a prescribed image pickup area corresponding to the direction and the position of the head part H, an image synthesizing device 110 synthesizing the photographic image of the television camera 102 on the output computer image of the computer graphic device 90 and a liquid crystal display device 30 held by the base mount 10 for displaying synthesized image from the image synthesizing device 110 and projecting it on the half mirror 50.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual information display device capable of displaying an image created by a computer graphic device on an actual image picked up by a television camera.

[0002]

2. Description of the Related Art For example, when displaying a completed landscape of a building such as a building as a video image, a completion image of the building is created by computer graphic (hereinafter abbreviated as CG) and a monitor screen is displayed. It is generally displayed above.

[0003]

[Problems to be Solved by the Invention] By the way, the person in charge of work at the planned construction site can move at any place while moving on site.
It would be convenient if it was possible to grasp the progress of the construction work by superimposing the image of the planned completion by CG on the actual image of the current site, and when the work manager moved his head to change the direction. Needless to say, it is more convenient to change the orientation and position of the actual image or the image to be completed by CG in accordance with the change in the directing direction. The present invention has been made by paying attention to the above points,
When the wearer changes the orientation or position of the head to change the directional direction, the orientation or position of the real image or the CG image is changed by following the change in the directional direction, and the real image and the CG image are changed. An object of the present invention is to provide a visual information display device that can be synthesized and displayed in front of the wearer.

[0004]

In order to achieve the above object, the present invention provides a screen arranged in the line of sight of a wearer and a head mounted on the wearer to hold the screen at the position. Holding means, a sensor that detects the orientation and position of the wearer's head, and a computer that outputs a computer image of a portion corresponding to the orientation and position of the wearer's head based on the detection results of the sensor. A graphic device, a television camera that captures a predetermined imaging region corresponding to the orientation and position of the wearer's head, and image synthesis that synthesizes an image captured by the television camera with a computer image output by the computer graphic device. And a video display device which is held by the holding means and which displays an image combined by the image combining device and projects it on the screen. It was obtain configuration.

Further, in the present invention, the computer graphic device is composed of two computer graphic devices for outputting computer images of portions corresponding to the direction and position of the head of the wearer with different directivities. The television camera is composed of two television cameras that capture images of the predetermined image capturing area portion with different directivities, and the image synthesizing device displays one of the computer image and the image captured by the television camera. The first image synthesizing device for synthesizing, and the second image synthesizing device for synthesizing the other computer image and the image captured by the television camera. The video display device is configured by the first and second image synthesizing devices. It is characterized in that it is composed of two video display devices for individually displaying the combined images.

Further, according to the present invention, the television camera is
A switching unit that is composed of two television cameras for capturing the predetermined image pickup area portions with different directivities, and alternately switches the images from the two television cameras at predetermined intervals and outputs the images to the image synthesizing apparatus. Further, the computer graphic device is configured so that two computer images each having a different directivity in a portion corresponding to a direction and a position of a head of the wearer are alternately displayed on the image synthesizing device in synchronization with the switching means. It is characterized by outputting.

In addition, according to the present invention, the computer graphic device is configured to superimpose and output two computer images having different directivities in portions corresponding to the direction and position of the head of the wearer. The television camera is composed of two television cameras that image the predetermined image pickup area portion with different directivities, and superimposes the images from the two television cameras and outputs them to the image synthesizing device. It further comprises an image superimposing means and a lenticular lens for separating the image displayed on the video display device into the two computer images having different directivities and projecting the computer image on the screen. Further, the present invention provides a pan head on which the television camera is mounted, a driving unit provided on the pan head, which displaces the directivity direction and the position of the television camera in a three-dimensional direction, and the detection result of the sensor. And a control means for controlling the drive of the drive means based on the above.

[0008]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a side view showing a schematic configuration of a display portion of a visual information display device according to an embodiment of the present invention, and FIG. 2 is A in FIG.
It is an arrow view. In the present embodiment, a case will be described in which an image of a building to be completed, such as a building under construction or a building to be built, is overlaid on an actual scene at the construction site. In FIG. 1, reference numeral 10 denotes a base mount as a holding means, which is attached to a helmet M, a headband (not shown) or the like as shown in FIG. 3 and has a head H of a work manager (wearer) at a construction site.
To be attached to. As shown in FIG. 1, the upper surface 11 of the base mount 10 is positioned on the line of sight of both eyes E of the work manager when the helmet M, headband, etc. are mounted on the head H of the work manager. A liquid crystal display device 3 as a video display device so that it is directed to the front of the person in charge of work.
0 and 30 are arranged in an inverted state one by one via a slide frame 20 that is slidable in the line-of-sight direction L of the work manager. Further, the slide frame 20
The front base mount 10 is provided with two transmission windows 15, 15 as shown in FIGS. 1 and 2, and a Fresnel lens 60 is fitted in each transmission window 15.

Further, as shown in FIGS. 1 and 2, a reflection mirror 40 is provided on the upper surface 11 of the base mount 10 so as to face the two transmission windows 15 and to be inclined at 45 ° with respect to the base mount 10. It is provided in. On the other hand, FIG.
The L-shaped piece 14 attached to the lower surface 13 of the base mount 10 as shown in FIG.
0 faces the two transmissive windows 15 and has an inclination of 45 ° with respect to the base mount 10, and has an inclination of 90 ° with respect to the reflection mirror 40.
It is attached so that it is located on the line of sight of. Further, as shown in FIG. 1, a holding piece 51 is attached to the surface of the half mirror 50 opposite to the eye E side of the person in charge of work.
By this holding piece 51, the polarizing glass 70 for adjusting the brightness of the image reflected by the reflection mirror 40 and projected on the half mirror 50 is detachably held so as to be located on the line of sight of the person in charge of work. There is.

As shown in FIG. 3, the helmet M is composed of, for example, a receiver for receiving radio waves from GPS satellites, a geomagnetic sensor, a gyro, and the like. A sensor 80 for detecting the orientation and the position is provided, and this sensor 80 is used for the coordinate setting unit 9 of two CG creation computers 90 described later.
1 is connected by wire or wirelessly.

Next, the left and right liquid crystal display devices 30, 30
An example of the configuration for displaying an image will be described with reference to FIG. In FIG. 3, reference numeral 31 is a receiving unit for receiving a video signal of the NTSC system and displaying it on the liquid crystal display device 30, and for example, an ultra-high frequency (UH) wave is transmitted through the antenna 32.
A tuner (not shown) for receiving a video signal in the F) band according to the NTSC system is provided. In this embodiment, two sets of tuners for receiving the video signals of different frequencies are provided in the receiving section 31 corresponding to the left and right liquid crystal display devices 30, 30.

Further, in FIG. 3, numeral 33 indicates the receiving unit 3
There are two transmission units provided corresponding to 1, and each transmission unit 3
An image synthesizing device 110 is connected to 3 via an RGB = NTSC converter 34.
A computer image created on the assumption that the construction work of the building under construction and its surroundings is completed is output to 0,
A CG creation computer 90 as a computer graphic device is connected.

Each transmitter 33 transmits to the tuner in the receiver 31 at a frequency different from each other, and each CG
From the creation computer 90, the signals of the computer images of the same portions whose directivities are slightly different from each other are output to the image synthesizing devices 110 so as to correspond to the left and right eyes E of the work manager, and the respective image synthesizing devices 110 are output. In the above, the combination processing of the computer image and the captured image of the television camera 102 described later is performed by so-called chroma processing.

On the other hand, reference numeral 100 in FIG. 3 denotes a platform which is provided at a position away from the person in charge of work.
Two TV cameras 102 and 102 are mounted on the display unit 0. The television cameras 102 and 102 are for photographing substantially the same area with different directivities, and the images photographed by the television cameras 102 and 102 are NTSC.
= Two image composition devices 1 via the RGB converter 35
Input to 10. Further, on the platform 100, for example, a gear group and a motor (not shown) for rotationally driving the gear group, for displacing the directivity direction and the position of the television cameras 102, 102 in a three-dimensional direction, and the like. , Pan head 10
A drive unit 101 including XYZ axes for holding 0 for traveling and rotation is provided, and the same as the sensor 80 for detecting the pointing directions and positions of the two TV cameras 102, 102. Sensor 103 according to configuration
Is provided and the sensor 103 is connected to the coordinate setting unit 94 of the two CG creation computers 90 by wire or wirelessly.

As shown in FIG. 3, the orientations of the head H of the person in charge of work and the two television cameras 102, 102 are determined on the two computer 90 for computer creation based on the detection results of the sensors 80, 103, respectively. And coordinate setting units 91 and 94 for setting coordinate values indicating the position. The two coordinate values set by the coordinate setting units 91, 94 are compared by the coordinate comparing unit 95, and based on the comparison result, the coordinate comparing unit 95 determines that the pointing directions and the positions of the television cameras 102, 102 are the same. A control signal for controlling the drive of the drive unit 101 is generated so as to be displaced following the displacement of the direction and position of the head H of the work manager, and is output to the drive unit 101. That is, in this embodiment, the coordinate comparison unit 95 corresponds to the control means.

Further, as shown in FIG. 3, each CG creating computer 90 holds a CG data holding unit for holding the data of the computer image created on the assumption that the construction work of the building under construction and its periphery is completed. 96 and coordinate setting unit 9
Based on the coordinate values set in 4, the data of the display area to be output to the liquid crystal display device 30 is extracted from the computer image data held in the CG data holding unit 96,
In addition, the orientation and size of the computer image of the extracted area are converted into those corresponding to the directivity and position of the television cameras 102, 102, and by extension, the orientation and position of the head H of the work manager, and the conversion. The image conversion unit 92 for converting the directivity of the converted computer image to the left or the right to convert it into a computer image corresponding to the left or right eye E of the work manager, and the data of the converted computer image. , And a background erasing unit 93 for erasing the data of the background (background) portion. Then, the computer image from which the data of the background portion is erased is input from each CG creating computer 90 to each of the two image synthesizing apparatuses 110.

Therefore, for example, when the head H of the person in charge of work or the line of sight of the eye E moves up or down or left or right, the computer image scrolled up or down or left or right in response to this, the image conversion unit 92. Output from the head H of the work manager
When is moved in the front-back direction, a computer image enlarged or reduced in response to this is output from the image conversion unit 92, and the background portion is erased from the output computer image by the background erasing unit 93 to create a CG. It is output to the outside of the computer 90.

In the visual information display apparatus of this embodiment having the above-mentioned configuration, the left-eye and right-eye computer images from the CG creating computer 90 and the television camera 102 and the captured image signal are two images. The image signals of the combined image that are respectively combined by the combining device 110 and output from each image combining device 110 are converted into NTSC signals by the RGB = NTSC converter 34, and are transmitted to the air by the ultra-high frequency wave from the transmitter 33, and the antenna 32 is transmitted. It is received by the receiving unit 31 via the. Then, the left and right liquid crystal display devices 30 respectively display composite images of different directivities for the left eye and the right eye according to the received video signal in an inverted state, and the liquid crystal display device 30 displays the composite images in an inverted state. Each image is reflected by the reflection mirror 40 and projected on the half mirror 50 in an upright state.

In this case, when the direction and position of the head H of the work manager moves, the detection results of the sensors 80 and 103 and the coordinate values set by the coordinate setting units 91 and 94 change accordingly, and the computer image Is scrolled or scaled up / down / left / right in the coordinate conversion unit 92, and the pointing direction of the television camera 102 is displaced following the movement of the head H, so that it is displayed on the liquid crystal display device 30. The synthetic image projected on the mirror 50 changes in accordance with the change in the direction and position of the head H of the work manager, that is, the line of sight and the visual field of the eye E.

The size of the composite image projected on the half mirror 50 is enlarged by the Fresnel lens 60,
The combined image for the left eye and the right eye projected on the half mirror 50 is visually recognized by the work manager as if it is in front of the work manager, for example, 1 to 5 m ahead. The half mirror 5
The brightness of the composite image projected on 0 can be set to a desired brightness by arbitrarily selecting the polarization degree of the polarizing glass 70 and holding it by the holding piece 51. It is preferable to use polarizing glass having a degree of polarization that allows total internal reflection. In addition, the focal length of the composite image displayed on the left and right liquid crystal display devices 30 and 30 and projected on the half mirror 50 is the slide frame 20.
Is appropriately adjusted by moving in the line-of-sight direction L with respect to the base mount 10.

In the visual information display device having such a structure,
When the direction and position of the head H of the person in charge of working who looks at the half mirror 50 with the left and right eyes E changes, and the direction of the line of sight and the field of vision change, the computer image and the image captured by the TV camera 102 are tracked according to the change. Even if the line of sight of the work manager's eye E changes, it is possible to see the actual image and computer image according to the line of sight in a superposed manner, and at a glance recognize the appearance of the building at the time of completion You will be able to accurately grasp the progress status of. Moreover, since the left and right liquid crystal display devices 30, 30 respectively display a composite image of a computer image and a captured image that are slightly offset from each other in correspondence with the left and right eyes E of the work manager, the left and right eyes E are displayed.
The person in charge of viewing the half mirror 50 can see the composite image showing the completed state of the building under construction and its surroundings as a stereoscopic three-dimensional image, and can further increase the sense of presence.

In the present embodiment, two CG making computers 90 for outputting the signals of the computer images of the same portion whose directivities are slightly deviated from each other are provided, and the computer image signals from the respective CG making computers 90 are provided. The signals of the captured images of the two TV cameras 102 and 102 are combined by the two image combining devices 110 and 110, and each combined image is divided into two.
Although the liquid crystal display devices 30 and 30 are used for displaying, the liquid crystal display device, the computer for CG creation, the television camera, the image synthesizing device, the tuner of the receiving unit, the transmitting unit, the R
A so-called monaural image may be displayed with one set of GB = NTSC converter and one set of NTSC = RGB converter, or may be configured as follows.

For example, as shown in FIG. 4, for example, as shown in FIG. 4, a switching device as a switching means uses signals of images taken by the two television cameras 102, 102, which image the same imaging region with different directivities. At 120, the images are switched at predetermined intervals and alternately output. On the other hand, the image conversion unit 92 of the single computer 90 for CG creation causes the computer image to be extracted from the CG data holding unit 96, and the extracted computer image is The switching device 120 converts the signals of the two converted computer images into two computer images whose directivities are slightly shifted from each other.
The output signals of the switching device 120 and the output signal of the computer 90 for CG creation are combined by a single image combining device 110 to be output by a single liquid crystal display device 30. It may be displayed.

With such a structure, the two composite images corresponding to the left and right eyes E displayed on the two liquid crystal display devices 30 and 30 in the previous embodiment are combined into a single liquid crystal display device. Since the images are alternately displayed at predetermined intervals at 30 and projected on the half mirror 50, the work manager can visually recognize this projected image as a stereoscopic image, and the liquid crystal display device, the computer for CG creation, and the image. A combination device, a tuner of a receiving unit, a transmitting unit, an RGB = NTSC converter, and an NTSC = RGB converter can be set as one set each.

As another example, as shown in FIG. 5, for example, as shown in FIG. 5, the signals of the picked-up images of the two television cameras 102 and 102, which pick up the same picked-up areas with different directivities, are superimposed on each other. The image is superimposed and output by the image superimposing device 130 as described above. On the other hand, the image conversion unit 92 of the single CG creation computer 90 causes the computer image to be extracted from the CG data holding unit 96, and the extracted computer image is The signals are converted into two computer images whose directivities are slightly deviated from each other, and the signals of the converted two computer images are superimposed and output, and the output signal of the image superimposing device 130 and the output signal of the computer 90 for CG creation are combined. Are synthesized by a single image synthesizing device 110, and an image obtained by superimposing two synthetic images having different directivities is formed on the single liquid crystal display device 3. , The displayed composite superimposed image is transmitted through the lenticular lens 140 held by the base mount 10 as shown in FIG. 5 to be separated into two composite images having directivities different from each other. The two combined images thus formed may be projected on the half mirror 50.

With such a structure, the image projected on the half mirror 50 can be visually recognized as a stereoscopic image by the person in charge of the work, and the images of the liquid crystal display device, the computer for CG creation, the image synthesizing device, and the receiving unit are displayed. A tuner, a transmitter, an RGB = NTSC converter, and an NTSC = RGB converter can be set as one set each.

The liquid crystal display device 30 is used in these embodiments.
The image is projected on the half mirror 50 after it is arranged in an inverted state and reflected by the reflection mirror 40. However, when a liquid crystal display device that displays an image in which the image is horizontally reversed is used, The mirror may be omitted and the liquid crystal display device may directly project onto the half mirror. When two liquid crystal display devices 30 are provided, two reflection mirrors 40 and two half mirrors 50 are provided correspondingly. May be. In addition, the transparent window 15 provided in the base mount 10
Although the Fresnel lens 60 is fitted to the slide frame 2, the Fresnel lens 60 may be omitted and the slide frame 2
0 does not have to be movable in the line-of-sight direction L with respect to the base mount 10, and the computer image output from the computer 90 for CG creation is converted into an NTSC signal and wirelessly transmitted. Computer 9
It may be directly connected to the liquid crystal display device 30 from 0 by wire.

Further, although a small CRT or other display device may be used as the image display device, if the liquid crystal display device 30 is used as in the above embodiment, the weight is smaller than that of the small CRT or other display device. It is possible to reduce the weight, and it is possible to reduce the degree of fatigue of the person in charge of work when used for a long time. In addition, although a half mirror is used as the screen in the above embodiments, the screen in the present invention may be any screen as long as it can project an image displayed by a video display device such as a liquid crystal display device.

Further, in the above embodiment, when the image conversion unit 92 converts the orientation and size of the computer image into those corresponding to the orientation and position of the head H of the work manager, the television cameras 102, 102. The conversion is performed based on the detection result of the sensor 103 that detects the orientation direction and the position of the head. However, the conversion is performed based on the detection result of the sensor 80 that detects the direction and the position of the head H of the work manager. Alternatively, the television camera may be attached to the side portion of the helmet M, and the platform 100, the driving device 101, and the configuration for controlling the driving device 101 may be omitted. It goes without saying that the visual information display device of the present invention can be applied to various uses other than the uses shown in this embodiment.

[0030]

As described above, according to the present invention, a screen arranged on the line of sight of the wearer, and a holding means which is mounted on the head of the wearer and holds the screen at the position, A sensor that detects the orientation and position of the wearer's head, and a computer graphic device that outputs a computer image of a portion corresponding to the orientation and position of the wearer's head, based on the detection result of the sensor, A television camera that captures a predetermined imaging region corresponding to the orientation and position of the wearer's head; an image synthesizer that synthesizes an image captured by the television camera with a computer image output by the computer graphic device; And a video display device which is held by a holding means and which displays the image combined by the image combining device and projects it on the screen. Therefore, as the orientation and position of the wearer's head changes, the orientation and position of the composite image of the computer image by the computer graphic device and the image captured by the television camera are changed and projected on the screen. be able to. Therefore, when the computer image is viewed by superimposing it on the actual image of the television camera, it is possible to always see the composite image of the place and size corresponding to the field of view of the wearer.

[Brief description of drawings]

FIG. 1 is a side view showing a schematic configuration of a display portion of a visual information display device according to an embodiment of the present invention.

FIG. 2 is a view on arrow A in FIG.

FIG. 3 is an explanatory diagram showing a schematic block diagram of a schematic configuration of a visual information display device according to an embodiment of the present invention.

FIG. 4 is an explanatory diagram showing a schematic block diagram of a schematic configuration of a visual information display device according to another embodiment of the present invention.

FIG. 5 is an explanatory diagram showing, in a partial block diagram, a schematic configuration of a visual information display device according to still another embodiment of the present invention.

[Explanation of symbols]

10 base mount (holding means) 30 liquid crystal display device (video display device) 50 half mirror (screen) 80 sensor 90 computer for CG creation (computer graphic device) 95 coordinate comparison unit (control means) 100 pan head 101 drive device (drive) Means) 102 TV camera 110 Image synthesizing device 120 Switching device (switching device) 130 Image superimposing device (image superimposing device) E Eye H Head

Claims (5)

[Claims] 1. A screen arranged in the line of sight of the wearer, a holding means mounted on the head of the wearer and holding the screen in the position, and a direction and a position of the head of the wearer. A sensor for detecting, a computer graphic device that outputs a computer image of a portion corresponding to the orientation and position of the head of the wearer based on the detection result of the sensor, and the orientation and position of the head of the wearer. A television camera that captures a corresponding predetermined image capturing area, an image synthesizer that synthesizes an image captured by the television camera with a computer image output by the computer graphic device, and the image synthesizer that is held by the holding unit. A visual information display device, comprising: a video display device that displays an image that has been combined and projected onto the screen. 2. The computer graphics device comprises:
The television camera includes two computer graphic devices that output computer images of portions corresponding to the orientation and position of the wearer's head with different directivities, and the television camera directs the predetermined imaging area portions. The image synthesizing device is composed of two television cameras that capture images with different sexes, and the image synthesizing device includes a first image synthesizing device that synthesizes one of the computer images and an image captured by the television camera, and the other computer image and the television. A second image synthesizing device for synthesizing images taken by the camera, and the video display device comprises two video display devices for individually displaying the images synthesized by the first and second image synthesizing devices. The visual information display device according to claim 1, wherein the visual information display device is configured. 3. The television camera is composed of two television cameras which image the predetermined image pickup area portion with different directivities, and the images from the two television cameras are alternately switched at predetermined intervals. The computer graphic device further includes switching means for outputting to the image synthesizing device, and the computer graphic device causes the switching means to output two computer images having different directivities of portions corresponding to the direction and position of the head of the wearer. The visual information display device according to claim 1, wherein the images are alternately output in synchronization with each other. 4. The computer graphic device comprises:
The television camera is configured to superimpose and output two computer images each having a different directivity in a portion corresponding to the orientation and position of the wearer's head, and the television camera directs each of the predetermined imaging area portions. Image with different sex 2
An image superimposing means that is composed of two television cameras and superimposes the images from the two television cameras and outputs the images to the image synthesizing device; and an image displayed on the video display device that has two different directivities. The visual information display device according to claim 1, further comprising a lenticular lens for separating the computer image and projecting the computer image on the screen. 5. A pan head on which the television camera is mounted,
A driving means provided on the platform for displacing the directivity and position of the television camera in a three-dimensional direction; and a control means for controlling driving of the driving means based on a detection result of the sensor. Claim 1, characterized in that
The visual information display device according to 2, 3, or 4.