JP2009251428A - Information display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information display system capable of quickly and reliably transmitting information without burdening an instructor and an operator with presenting and receiving information. <P>SOLUTION: The information display system includes a head-mounted image display device (HMD) having a display unit displaying an image based on image data and capable of seeing through the outside, and an information providing device for providing the HMD with image data. The information providing device has a tag information adding part for adding tag information indicating the urgency of the image data, to the image data. The HMD has a tag information analysis part for analyzing the tag information; a shutter for adjusting the quantity of external light transmitted through the display unit; and a shutter driving part for controlling driving of the shutter. The shutter driving part controls driving of the shutter to lower the quantity of external light transmitted through the display unit when the tag information analysis part discriminates that the image data is urgent data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information display system, and more particularly to an information display system provided with a head-mounted video display device.

  Conventionally, an image (hereinafter also referred to as an image) that is detachably attached to the observer's head or face and obtained from an image display element such as a small CRT or a liquid crystal display element is displayed on the observer's eyeball by an eyepiece optical system. A head-mounted image display device, that is, an HMD (Head Mount Display) is known in which a virtual image can be observed as if the image is enlarged and projected in the air.

  The HMD is used as a display device for viewing content videos such as DVDs and videos, and for remote operation of industrial equipment or medical equipment, and has a wide variety of uses.

  For remote operation, an operator wears an HMD equipped with an electronic camera, works while observing the image of the work object photographed by the electronic camera, and the instructor uses the same image transmitted from the HMD. In addition, a remote work support system is known in which appropriate information is provided by voice or video to a worker and a work instruction is given while observing on a monitor such as a host computer provided in a remote place (for example, patent document) 1).

In addition, a so-called see-through type HMD that displays various information necessary for the work provided by the instructor superimposed on a natural image (hereinafter also referred to as a see-through image) by external light incident through the display unit of the HMD is used. Various remote work support systems have been studied.
JP 2002-132487 A

  However, in the remote work support system using the see-through type HMD as described above, when the worker wearing the HMD is devoted to the work while observing the see-through image of the work target, Even if information desired to be viewed immediately is transmitted from a host computer or the like provided in the network, there is a problem that the transmitted information is not looked at unless the operator notices it. This leads to a fatal problem as the urgency of information increases.

  The present invention has been made in view of the above problems, and is an information display system capable of transmitting information promptly and reliably without imposing a burden on the instructor or worker to present and receive the information. The purpose is to provide.

  The above object is achieved by the invention described in any one of the following items 1 to 3.

1. A head-mounted image display device having a display unit that displays an image based on image data and can see through the outside world;
An information providing device for providing the image data to the head-mounted video display device;
An information display system comprising:
The information providing apparatus includes:
A tag information adding unit that adds tag information indicating the degree of urgency of the image data to the image data;
The head-mounted video display device is
A tag information analysis unit for analyzing the tag information;
A shutter for adjusting the amount of light from the outside that passes through the display unit;
A shutter drive unit for controlling the drive of the shutter;
The shutter drive unit
When the tag information analysis unit determines that the image data is highly urgent data, the driving of the shutter is controlled so that the amount of light from the outside that passes through the display unit is reduced. Information display system.

2. The display unit is
A light transmissive display element for displaying an image based on the image data;
A backlight for illuminating the light transmissive display element;
A backlight drive unit for controlling the drive of the backlight,
The backlight driver is
2. The driving of the backlight is controlled to increase the brightness of the backlight when the tag information analysis unit determines that the image data is highly urgent data. Information display system.

3. The head-mounted video display device is
An image data converter for converting the image data;
The image data converter is
When the tag information analysis unit determines that the image data is highly urgent data, the image data is converted so that the saturation of an image displayed based on the image data is increased. The information display system according to 1 or 2 above.

  According to the present invention, the HMD is configured to reduce the amount of light from the outside that passes through the display unit when the image data provided from the information providing device is highly urgent data. Thereby, for example, even when the worker wearing the HMD is devoted to the work while gazing at the see-through image of the work object, information (image) provided from the host computer (information providing device) or the like by the instructor If the data) is highly urgent information, the amount of light from the outside that passes through the display unit can be reduced to make it difficult to see the see-through image. The line of sight can be directed to the display image. As a result, it is possible to quickly and reliably transmit information that is highly urgent and that is desired to be viewed immediately.

  Embodiments of an information display system according to the present invention will be described below with reference to the drawings. In addition, although this invention is demonstrated based on embodiment of illustration, this invention is not limited to this embodiment.

  First, the configuration of the information display system 1 will be described with reference to FIG. FIG. 1 is a schematic diagram showing an outline of the overall configuration of an information display system 1 according to an embodiment of the present invention.

  As shown in FIG. 1, the information display system 1 includes an HMD 2, an information providing device 4, and the like. The information providing apparatus 4 includes a host computer 5, a video / audio recording server 6, a content server 7, and the like.

  The host computer (hereinafter also referred to as host PC) 5 is connected to the video / audio recording server 6 and the content server 7 via the Ethernet EN, and the host PC 5 and the HMD 2 are connected to the Internet IN, mobile, PHS, wireless LAN, etc. Are connected via a public network PN.

  In the information display system 1 having such a configuration, the worker B wears a see-through type HMD 2 equipped with a camera unit 28 to be described later, and works while observing a see-through image of the work target. While observing an image of a work object photographed by the camera unit 28 on a monitor 53 (to be described later) of the information providing apparatus 4 provided at a remote place, the information providing apparatus 4 provides an image or sound to the worker B via the HMD 2 Providing appropriate information and giving work instructions.

  Next, the configuration of the HMD 2 corresponding to the head-mounted image display device according to the present invention will be described with reference to FIG. FIG. 2 is a perspective view showing an appearance of the HMD 2 according to the embodiment of the present invention.

  As shown in FIG. 2, the HMD 2 includes temples 21R and 21L, a bridge 22, nose pads 23R and 23L, transparent substrates 24R and 24L, a display unit 25 (LCD display unit 26, an eyepiece optical system 27), a camera unit 28, and an earphone. 29R, 29L, a microphone 30, a control unit 31, and the like.

  The HMD 2 displays an image captured by the camera unit 28 or an image provided from the information providing apparatus 4 on the LCD display unit 26 provided in the display unit 25, and the image obtained from the LCD display unit 26 is displayed as an eyepiece. By directly projecting onto the eyeball of the wearer (hereinafter also referred to as worker B) by the optical system 27, it is possible to observe a virtual image as if the image is magnified and projected in the air.

  The temples 21R and 21L are elongate members made of a flexible elastic material or the like, and are hung on the wearer's ears or the temporal region. This is for adjusting the mounting position. The temples 21R and 21L are configured to be rotatable in the directions of arrows P and Q in the rotating portions 21Ra and 21La. When the HMD 2 is not used, the temples 21R and 21L are rotated in the directions of the arrows P and Q. It can be made compact by moving it along the transparent substrates 24R and 24L.

  The bridge 22 includes nose pads 23R and 23L that hold the HMD 2 on the face of the wearer, and is a short rod-like member that is stretched over a predetermined position on the transparent substrates 24R and 24L facing each other. The transparent substrate 24L is held in a relative positional relationship with a fixed gap.

  The transparent substrates 24R and 24L are substantially flat transparent bodies that form a U-shaped space 24Rs at a position corresponding to one eyeball. An eyepiece optical system 27 is fitted in a U-shaped space 24Rs formed by being surrounded by the transparent substrate 24R.

  The display unit 25 includes an LCD display unit 26, an eyepiece optical system 27, and the like. A video or the like related to the given work instruction is displayed.

  The camera unit 28 includes a zoom lens 281 (not shown), a CCD (Charge Coupled Device) 282, an image processing unit 283, and the like. The camera unit 28 photographs an external scene around the wearer, and is connected by the zoom lens 281. The imaged subject optical image is photoelectrically converted by the CCD 282 to generate an image signal (video signal), and the generated image signal is subjected to predetermined image processing by the image processing unit 283 or the like.

  The earphones 29R and 29L are inserted into the ears of the wearer and provide sound to the wearer.

  The microphone 30 converts the voice of the wearer into an audio signal.

  The control unit 31 includes a microcomputer and includes a power switch 351, an operation switch 352, and the like. A display operation performed by the HMD 2 in response to signals from these various operation switches, a control signal from the information providing device 4, and the like. And image signal processing operations are comprehensively controlled.

  Here, the configuration of the display unit 25 will be described with reference to FIG. FIG. 3 is a side sectional view from the left side of the display unit 25 in the HMD 2 according to the embodiment of the present invention, and mainly shows the internal configuration of the display unit 25.

  As shown in FIG. 3, the display unit 25 includes a housing 261, a backlight 262, a collimator lens 263, an LCD display unit 26 made up of an LCD (Liquid Crystal Display) 264, a prism 271, an HOE (Holographic Optical Element) 272, An eyepiece optical system 27 including a shutter 273 is configured.

  In the state where the backlight 262, the collimator lens 263, and the LCD 264 are built in the housing 261 of the LCD display unit 26, the housing 261 is inclined upward and forward (at the upper end of the prism 271 of the eyepiece optical system 27). In FIG. 3, it is attached in such a manner as to protrude obliquely upward to the right.

  The backlight 262 is a point light source that is composed of a white light emitting diode (LED) including a predetermined wavelength color and illuminates the LCD 264.

  The collimator lens 263 converts the LED light of the backlight 262 into substantially parallel light and projects it onto the LCD 264.

  The LCD 264 is provided from the information providing device 4 via a video signal (hereinafter also referred to as video data or image data) generated by the camera unit 28 or a communication unit 36 described later provided in the control unit 31. An image is generated based on image data related to a work instruction, and is, for example, a light transmission type liquid crystal display panel.

  The prism 271 is a substantially plate-shaped transparent member made of glass, transparent resin, or the like, and causes light from the LCD 264 to be reflected a plurality of times inside. The upper end portion of the prism 271 is formed in a wedge shape so that the front side (opposite side of the eyepiece surface) protrudes to be thicker upward so that most of the light from the LCD 264 can be collected inside. The thick part 271b thus formed is formed.

  An inclined surface 271a is formed at the lower end of the prism 271. The prism 271 is bonded (for example, bonded) to the inclined surface 24Ra formed on the transparent substrate 24R via the HOE 272. The front and back surfaces of the prism 271 are flush with the front and back surfaces of the transparent substrate 24R. Thereby, the prism 271 is integrated with the transparent substrate 24R in a single plate shape.

  The HOE 272 is a volume phase type holographic optical element having a positive power, which is constituted by a so-called free-form surface that is optically asymmetric with respect to the axis, and is supported at a lower end portion of the prism 271 with a predetermined inclination angle. The HOE 272 provides a hologram image to the eyeball E using the light interference phenomenon when irradiated with the light guided by the prism 271.

  In the display unit 25 having the above configuration, the light emitted from the LED of the backlight 262 illuminates the LCD 264 through the collimator lens 263, and the image light generated by the LCD 264 by this illumination is within the prism 271. After a plurality of total reflections, it is diffracted by the HOE 272 and guided to the eyeball E of the wearer as a virtual image.

  Further, the prism 271 guides light incident from the front to the eyeball E of the wearer. As a result, the wearer can see through the outside world (front subject), and is provided from the information providing device 4 via the image captured and generated by the camera unit 28 or the communication unit 36 described later provided in the control unit 31. The video or the like related to the given work instruction is visually recognized by being superimposed on the outside world (front subject).

  The shutter 273 is provided on the front surface of the prism 271 and the transparent substrate 24R (the surface facing the subject), and adjusts the amount of light from the outside that passes through the prism 271 and the transparent substrate 24R. Can be used. The details of driving the shutter 273 will be described later.

  Next, returning to FIG. 1, the configuration of the information providing apparatus 4 will be described. The information providing apparatus 4 includes a host PC 5, an audio / video recording server 6, a content server 7, and the like.

  As shown in FIG. 1, the host PC 5 is, for example, a personal computer including a computer main body 51, a monitor 53, a mouse 55, a keyboard 57, and the like.

  The host PC 5 displays on the monitor 53 an image of the work object photographed by the camera unit 28 of the HMD 2 attached to the worker B. The instructor A operates the host PC 5 with the mouse 55, the keyboard 57, and the like while observing the image of the work object displayed on the monitor 53, and information necessary for work recorded in the content server 7 in the HMD 2 ( Image data) and provide work instructions to worker B.

  The audio / video recording server 6 stores work records such as video captured by the camera unit 28 of the HMD 2 and audio exchanged between the worker B and the instructor A.

  The content server 7 stores information such as video and audio necessary for the worker B to perform work.

  Next, the block configuration of the information display system 1 will be described with reference to FIG. FIG. 4 is a block diagram showing a main configuration of the information display system 1 according to the embodiment of the present invention. In FIG. 4, the same members as those shown in FIGS. 1 to 3 are given the same numbers.

  First, the main part of the information providing apparatus 4 is composed of a host PC 5, a content server 7, and the like.

  The host PC 5 includes a computer main body 51, a monitor 53, a mouse 55, a keyboard 57, and the like.

  The computer main body 51 has a tag information adding unit 511.

  The tag information adding unit 511 determines the urgency of the image data related to the work instruction from the content server 7 provided to the HMD 2, that is, whether the image based on the image data is an image that the worker B wants to see immediately. Is added to the image data.

  Next, the main part of the HMD 2 includes a display unit 25, a camera unit 28, a control unit 31, and the like.

  The display unit 25 includes an LCD display unit 26 and an eyepiece optical system 27. Since the operations performed in each part have been described above, the description thereof will be omitted.

  The camera unit 28 includes a zoom lens 281, a CCD (Charge Coupled Device) 282, an image processing unit 283, and the like.

  The CCD 282 is a color area imaging sensor in which R (red) light, G (green) light, and B (blue) light transmission filters are arranged in a checkered pattern in pixel units (pixel units), and is connected by a zoom lens 281. The imaged subject light image is photoelectrically converted into image signals (signals consisting of a signal sequence of pixel signals received in units of pixels) of each color component of R (red) light, G (green) light, and B (blue) light. To convert.

  The image processing unit 283 includes, for example, a black level correction unit, a pixel interpolation unit, a white balance control unit, a gamma correction unit, a matrix calculation unit, a shading correction unit, and an image compression unit (not shown), and is read from the CCD 282. The image signal is subjected to known image signal processing. Then, the image signal that has undergone predetermined processing at these parts is stored in an image memory 33 of the control unit 31 described later.

  The control unit 31 includes a control unit 32, an image memory 33, a VRAM (Video Random Access Memory) 34, an operation unit 35, a communication unit 36, and the like.

  The control unit 32 reads a ROM (Read Only Memory) that stores each control program (not shown), a RAM (Random Access Memory) that temporarily stores data such as arithmetic processing and control processing, and a control program from the ROM. CPU (central processing unit) etc. to be executed. The control unit 32 receives signals from the power switch 351 and various operation switches 352, the control signal from the information providing device 4, and the like, and comprehensively controls display operations and image signal processing operations performed in the HMD 2. It is. Further, the control unit 32 changes the display image based on the see-through image and the image data displayed on the display unit 25 based on the tag information added to the image data provided from the information providing device 4. Details of the display control operation based on the tag information performed by the control unit 32 will be described later.

  The image memory 33 is a temporary memory used as a work area for performing various processing on the image signal by the control unit 32.

  The VRAM 34 has a recording capacity of an image signal corresponding to the number of pixels of the LCD 264 in the LCD display unit 26, and is a buffer memory of an image signal that constitutes an image reproduced and displayed on the LCD 264.

  The communication unit 36 connects the HMD 2 to the public line network PN, communicates with the information providing apparatus 4, and inputs and outputs video, audio, various control signals, and the like.

  In the information display system 1 having such a configuration, the present invention provides a case where the information providing device 4 provides highly urgent information (image data) that the instructor A at a remote location wants the worker B to see immediately. The operator B is easily noticed while wearing the HMD 2 and gazing at the see-through image of the work object so that the line of sight is directed to the display image based on the image data displayed on the display unit 25. For this purpose, the aspect of the see-through image or the display image is changed. Details will be described below.

  As shown in FIG. 4, the control unit 32 of the HMD 2 includes a shutter drive unit 323, a tag information analysis 324, and the like.

  The tag information analysis unit 324 detects tag information added to the image data provided from the information providing device 4, and determines the urgency of the image data based on the detected tag information.

  The shutter driving unit 323 controls the driving of the shutter 273, and transmits through the display unit 25 when the tag information analysis unit 324 determines that the image data provided from the information providing device 4 is highly urgent data. The driving of the shutter 273 is controlled so that the amount of light from the outside world is reduced.

  The state of the screen of the display unit 25 at this time is shown in FIG. FIG. 5A is a schematic diagram showing the state of the screen D when the urgency of the image data provided from the information providing device 4 is low, and FIG. 5B is the state of the screen D when the urgency is high. It is a schematic diagram which shows.

  When the urgency of the image data provided from the information providing device 4 is high, as shown in FIG. 5B, the see-through image of the work object X can be obtained by suppressing the amount of light from the outside that passes through the display unit 25. S2 becomes darker and visibility is lower than the see-through image S1 in the case of FIG. Thereby, it is possible to make the worker B easily aware of the change and direct the operator B's line of sight toward the display image H1. As a result, it is possible to quickly and reliably transmit information that is highly urgent and that is desired to be viewed immediately.

  Furthermore, as shown in FIG. 4, the control unit 32 is provided with a backlight driving unit 321 that controls the driving of the backlight 262, and the backlight is based on the urgency of the image data provided from the information providing device 4. The brightness of H.262 may also be controlled.

  When the tag information analysis unit 324 determines that the image data provided from the information providing device 4 is highly urgent data, the backlight driving unit 321 increases the brightness of the backlight 262. The driving of 262 is controlled. Thereby, in addition to the brightness of the see-through image S1 being suppressed by the shutter drive unit 323, the contrast of the display image H1 is increased, so that the display image H1 becomes a more prominent image, and the line of sight of the operator B can be more easily Can be directed to the display image H1.

  Further, as shown in FIG. 4, the control unit 32 is provided with an image data conversion unit 322 that converts image data provided from the information providing device 4, and based on the urgency of the image data provided from the information providing device 4. The image data may be converted so that the saturation of the image displayed based on the image data is increased.

  When the tag information analysis unit 324 determines that the image data provided from the information providing device 4 is highly urgent data, the image data conversion unit 322 determines the color of the image displayed based on the image data. The image data is converted so that the degree increases.

  The state of the screen of the display unit 25 at this time is shown in FIG. 6A is a schematic diagram showing a state of the screen D when the urgency level of the image data provided from the information providing device 4 is low, and FIG. 6B is a state of the screen D when the urgency level is high. It is a schematic diagram which shows.

  When the urgency of the image data provided from the information providing device 4 is high, the brightness of the see-through image S2 is suppressed by the shutter driving unit 323 as shown in FIG. Since the saturation is increased, the display image H2 becomes more prominent, and the line of sight of the operator B can be more easily directed toward the display image H1.

  The present invention has been described above with reference to the embodiments. However, the present invention should not be construed as being limited to the above-described embodiments, and can be changed or improved as appropriate. For example, when the image data provided from the information providing device 4 to the HMD 2 is stream image data that changes with time, a time when the operator wants to pay attention may be added to the tag information. For example, when an image desired to be watched by an operator is reproduced 10 minutes after the point of provision, information indicating the time (after 10 minutes) is added to the tag information and provided to the HMD 2. The HMD 2 performs a normal display operation for 10 minutes from the time when the image data is received, and when 10 minutes have elapsed, the shutter drive unit 323, the backlight drive unit, the 321 image data conversion unit 322, etc. The display mode of the display image is changed as described above. As a result, it is not necessary to update the tag information again at a time when the operator wants to pay attention, and information can be provided efficiently.

It is a mimetic diagram showing the whole information display system composition concerning an embodiment of the present invention. It is a perspective view which shows the external appearance of HMD which concerns on embodiment of this invention. It is a sectional side view of the display unit which concerns on embodiment of this invention. 1 is a block diagram showing a schematic configuration of an information display system according to an embodiment of the present invention. It is a schematic diagram which shows an example of the information display in the display unit which concerns on embodiment of this invention. It is a schematic diagram which shows another example of the information display in the display unit which concerns on embodiment of this invention.

Explanation of symbols

1 Information display system 2 Head-mounted video display device (HMD)
21R, 21L Temple 22 Bridge 23R, 23L Nose pad 24R, 24L Transparent substrate 25 Display unit 26 LCD display unit 261 Housing 262 Backlight 263 Collimator lens 264 LCD
27 Eyepiece optical system 271 Prism 272 HOE
273 Liquid crystal shutter 28 Camera unit 281 Zoom lens 282 CCD
283 Image processing unit 29R, 29L Earphone 30 Microphone 31 Control unit 32 Control unit 321 Backlight drive unit 322 Image data conversion unit 323 Shutter drive unit 324 Tag information analysis unit 33 Image memory 34 VRAM
35 Operation Unit 351 Power Switch 352 Operation Switch 36 Communication Unit 4 Information Providing Device 5 Host Computer (Host PC)
51 Computer Main Body 511 Tag Information Adding Unit 53 Monitor 55 Mouse 57 Keyboard 6 Video / Audio Recording Server 7 Content Server A Instructor B Worker EN Ethernet E Eye IN Internet PN Public Line Network X Work Object

Claims (3)

A head-mounted image display device having a display unit that displays an image based on image data and can see through the outside world;
An information providing device for providing the image data to the head-mounted video display device;
An information display system comprising:
The information providing apparatus includes:
A tag information adding unit that adds tag information indicating the degree of urgency of the image data to the image data;
The head-mounted video display device is
A tag information analysis unit for analyzing the tag information;
A shutter for adjusting the amount of light from the outside that passes through the display unit;
A shutter drive unit for controlling the drive of the shutter;
The shutter drive unit
When the tag information analysis unit determines that the image data is highly urgent data, the driving of the shutter is controlled so that the amount of light from the outside that passes through the display unit is reduced. Information display system. The display unit is
A light transmissive display element for displaying an image based on the image data;
A backlight for illuminating the light transmissive display element;
A backlight drive unit for controlling the drive of the backlight,
The backlight driver is
2. The drive of the backlight is controlled to increase the brightness of the backlight when the tag information analysis unit determines that the image data is highly urgent data. The information display system described. The head-mounted video display device is
An image data converter for converting the image data;
The image data converter is
When the tag information analysis unit determines that the image data is highly urgent data, the image data is converted so that the saturation of an image displayed based on the image data is increased. The information display system according to claim 1 or 2.

Images