JP6561606B2 - Display device and control method of display device - Google Patents

Description

  The present invention relates to a display device and a display device control method.

  2. Description of the Related Art Conventionally, a display device is known that can ensure the visibility of an image or video to be displayed even when the display device is affected by external light (also referred to as external light) (for example, patents). Reference 1). A head mounted display (HMD) disclosed in Patent Document 1 displays an image and irradiates light to the outside in order to ensure a suitable field of view even in a dark place. This display device makes it easy to distinguish the image to be displayed from the outside world by making the color directly recognized by the user from the outside light different from the color recognized from the image light irradiated by the display device.

JP 2006-139124 A

As described in Patent Document 1, when light and light are output from the device to display an image or an image, the appearance of the light output from the device changes due to the influence of other light. Patent Document 1 discloses an example in which the wavelength of image light is set to a wavelength different from the irradiation light that the apparatus irradiates to the outside. In this example, the wavelength of the irradiation light is known. On the other hand, when the wavelength that can affect the visibility of the image displayed by the display device cannot be specified in advance, no countermeasures can be taken.
The present invention has been made in view of the above circumstances, and when the visibility of an image output and displayed by the apparatus is influenced by light other than the image light, the image can be viewed appropriately. The purpose is to be able to adjust the sex.

In order to achieve the above object, a display device of the present invention provides a user with a display unit that outputs image light including a plurality of color lights to display a display object, a light detection unit that detects light, A control unit that controls a color tone of the image light output from the display unit based on a detection result of the light detection unit.
According to the present invention, the color tone of image light can be changed in accordance with the light detected by the light detection unit. For this reason, for example, when the light detection unit detects light incident on the user's field of view other than the image light, control is performed so that the color tone of the light and the image light are harmonized, or the visibility of the image light is increased. You can control the color tone. Thereby, the visibility of the image which a display apparatus displays can be adjusted appropriately.

Moreover, the present invention is characterized in that, in the display device, the light detection unit detects light from a direction different from the image light output from the display unit.
According to the present invention, it is possible to control the color tone of the image light and the light incident on the user's visual field in addition to the image light, or to control the color tone so as to improve the visibility of the image light. The visibility of the image displayed by can be adjusted appropriately.

In the display device according to the present invention, the display unit is mounted on the user's head and outputs the image light in a state where external light can enter the user's field of view. The light detection unit detects the outside light incident on the field of view of the user.
ADVANTAGE OF THE INVENTION According to this invention, the visibility of an image can be adjusted by changing the color tone of the image which the display apparatus with which a user's head mount | wears displays according to the external light which injects into a user's visual field.

In order to achieve the above object, the display device of the present invention is provided adjacent to the display unit for displaying an image by outputting image light including a plurality of color lights to the user. A light detection unit that detects light from the front of the user's face, and a control unit that controls a color tone of the image light output from the display unit based on a detection result of the light detection unit. It is characterized by.
According to the present invention, the color tone of the image light can be changed in accordance with the light detected by the light detection unit provided adjacent to the display unit. Therefore, it is possible to control the light detected by the light detection unit in the vicinity of the display unit and the color tone of the image light, or to control the color tone so as to improve the visibility of the image light. Thereby, the visibility of the image which a display apparatus displays can be adjusted appropriately.

In the display device according to the aspect of the invention, the control unit may cause the display unit to output the color light constituting the image based on image data including a plurality of color data, and color data included in the image data. The color tone of the image light is changed by changing the tone value.
According to the present invention, the visibility of an image can be adjusted by data processing by changing the gradation value for each color included in the image data.

Moreover, the present invention is characterized in that, in the above display device, the light detection unit includes a sensor for detecting the intensity of received light for each of a plurality of different wavelengths.
According to the present invention, it is possible to obtain information relating to the color of light incident on the user's visual field.

In the display device according to the present invention, the control unit controls the color tone of the image light output from the display unit based on the intensity of light for each wavelength detected by the light detection unit. It is characterized by.
ADVANTAGE OF THE INVENTION According to this invention, the visibility of the image which a display apparatus displays can be adjusted appropriately by controlling the color tone of image light according to the color of the light which injects into a user's visual field.

Moreover, the present invention is characterized in that, in the display device, the control unit controls a color tone of the image light so as to improve visibility for the user to visually recognize the image light.
ADVANTAGE OF THE INVENTION According to this invention, the visibility of the image which a display apparatus displays can be improved corresponding to the color of the light which injects into a user's visual field.

In the display device according to the aspect of the invention, the control unit may adjust the luminance of each color light constituting the image light so that the color tone of the image light approaches the color tone of the light detected by the light detection unit. It is characterized by controlling.
ADVANTAGE OF THE INVENTION According to this invention, the color tone of image light can be adapted to the color of the light which injects into a user's visual field, and the image which a display apparatus displays can be harmonized with colors, such as external light.

In the display device according to the aspect of the invention, the control unit may improve visibility of the user viewing the image light based on light intensity for each wavelength detected by the light detection unit. The color tone of the image light is adjusted by the light detection unit based on the first processing for controlling the luminance of each of the color lights constituting the image light and the intensity of light for each wavelength detected by the light detection unit. A second process for controlling the brightness of each of the color lights constituting the image light so as to approximate the color tone of the light to be detected. One of the first process and the second process is selected and executed.
According to the present invention, it is possible to select and execute a process for improving the visibility of an image with respect to external light or the like and a process for harmonizing the image with external light or the like according to the attribute of the image displayed on the display device.

In the display device according to the aspect of the invention, when the control unit executes the first process, the image light is detected when an intensity of light detected by the light detection unit is lower than a preset intensity. The ratio of the luminance of each of the color lights included in is changed to a preset ratio.
According to the present invention, the visibility of an image can be appropriately adjusted even when the intensity of light such as outside light is low.

Further, in the display device according to the aspect of the invention, the control unit may be configured such that when the intensity of light detected by the light detection unit is lower than a preset intensity, the luminance of each of the color lights included in the image light The ratio is changed to a preset ratio.
According to the present invention, the visibility of an image can be appropriately adjusted even when the intensity of light such as outside light is low.

In the display device according to the present invention, the light detection unit can switch between a first detection state and a second detection state suitable for detecting light having a lower intensity than the first detection state. The control unit is characterized in that, when the light detection unit is switched to the second detection state, a luminance ratio of each of the color lights included in the image light is changed to a preset ratio.
According to the present invention, it is possible to detect light in a wide range with high accuracy by switching the detection state of the light detection unit according to the intensity of light to be detected.

In the display device according to the aspect of the invention, the control unit may detect image data of an image displayed by the display unit when the intensity of light detected by the light detection unit is lower than a preset intensity. Contrast correction processing and / or edge correction processing is executed.
According to the present invention, when the intensity of light such as outside light is low, the visibility of an image can be effectively enhanced by performing the contrast correction process and / or the edge correction process.

In the display device according to the present invention, the display unit includes a display area located in the visual field of the user, and the control unit is external light incident on the visual field of the user by the light detection unit. The display area is divided into a plurality of parts based on the detection result of detecting the color, and the color tone of the image light is controlled for each part.
According to the present invention, the visibility of an image can be adjusted for each portion of the display area.

Moreover, the present invention is characterized in that, in the display device, the light detection unit can detect the intensity of each of light incident on the user's visual field from a plurality of directions.
ADVANTAGE OF THE INVENTION According to this invention, the visibility of an image can be adjusted corresponding to each of the light which injects into a user's visual field from several directions.

In order to achieve the above object, the present invention provides a display unit that outputs image light including a plurality of color lights to a user to display an image, and a direction different from the image light output by the display unit. And a light detection unit that detects light incident on the user's field of view, and based on a detection result of the light detection unit, the image light output by the display unit Controlling the color tone.
According to the present invention, the color tone of image light can be changed in accordance with light incident on the user's field of view in addition to image light. For this reason, it is possible to control the color tone of the image light and the light incident on the user's visual field in addition to the image light, or to control the color tone to enhance the visibility of the image light. Thereby, the visibility of the image which a display apparatus displays can be adjusted appropriately.

Explanatory drawing which shows the external appearance structure of the head mounted display apparatus of 1st Embodiment. The figure which shows the structure of the optical system of an image display part. The functional block diagram of each part which comprises a head mounting | wearing type display apparatus. It is a figure which shows the display state of a head mounted display apparatus, (A) is explanatory drawing which shows the position of a display area, (B) shows a 1st display example, (C) shows a 2nd display example. It is a flowchart which shows operation | movement of a head mounted display apparatus, (A) shows operation | movement of a control apparatus, (B) shows operation | movement of an image display part. The flowchart which shows an example of the image process which a head mounted display apparatus performs. The flowchart which shows another example of image processing. The flowchart which shows another example of image processing. It is explanatory drawing of operation | movement of the head mounted display apparatus of 2nd Embodiment, (A) shows the area | region provided in a captured image corresponding to a display area, (B) shows a display area. 10 is a flowchart illustrating an example of image processing in the second embodiment. 10 is a flowchart illustrating an example of image processing in the third embodiment. The functional block diagram of the head mounted display apparatus of 4th Embodiment.

[First Embodiment]
FIG. 1 is an explanatory diagram showing an external configuration of an HMD (Head Mounted Display) 100 according to a first embodiment to which the present invention is applied.
The HMD 100 (display device) includes an image display unit 20 (display unit) that allows a user to visually recognize a virtual image while being mounted on the user's head, and a control device 10 that controls the image display unit 20. Yes. The control device 10 also functions as a controller for the user to operate the HMD 100.

  The image display unit 20 is a wearing body that is worn on the user's head, and has a glasses shape in the present embodiment. The image display unit 20 includes a right holding unit 21, a right display driving unit 22, a left holding unit 23, a left display driving unit 24, a right optical image display unit 26, a left optical image display unit 28, and a camera 61. (Imaging part) and a microphone 63 are provided. The right optical image display unit 26 and the left optical image display unit 28 are arranged so as to be positioned in front of the right and left eyes of the user when the user wears the image display unit 20, respectively. One end of the right optical image display unit 26 and one end of the left optical image display unit 28 are connected to each other at a position corresponding to the eyebrow of the user when the user wears the image display unit 20.

  The right holding unit 21 extends from the end ER which is the other end of the right optical image display unit 26 to a position corresponding to the user's temporal region when the user wears the image display unit 20. It is a member. Similarly, the left holding unit 23 extends from the end EL which is the other end of the left optical image display unit 28 to a position corresponding to the user's temporal region when the user wears the image display unit 20. It is a member provided. The right holding unit 21 and the left holding unit 23 hold the image display unit 20 on the user's head like a temple of glasses.

  The right display drive unit 22 and the left display drive unit 24 are disposed on the side facing the user's head when the user wears the image display unit 20. The right display drive unit 22 and the left display drive unit 24 are collectively referred to simply as “display drive unit”, and the right optical image display unit 26 and the left optical image display unit 28 are simply referred to as “optical image display unit”. Also called.

The display drive units 22 and 24 include liquid crystal displays 241 and 242 (hereinafter referred to as “LCDs 241 and 242”), projection optical systems 251 and 252, which will be described later with reference to FIG.
The right optical image display unit 26 and the left optical image display unit 28 include light guide plates 261 and 262 (FIG. 2) and a light control plate 20A. The light guide plates 261 and 262 are formed of a light transmissive resin or the like, and guide the image light output from the display driving units 22 and 24 to the user's eyes. The light control plate 20A is a thin plate-like optical element, and is disposed so as to cover the front side of the image display unit 20 which is the side opposite to the user's eye side. As the light control plate 20A, various materials such as a material that has almost no light transmission, a material that is nearly transparent, a material that attenuates the amount of light and transmits light, and a material that attenuates or reflects light of a specific wavelength may be used. it can. By appropriately selecting the optical characteristics (light transmittance, etc.) of the light control plate 20A, the external light amount incident on the right optical image display unit 26 and the left optical image display unit 28 from the outside is adjusted to visually recognize the virtual image. You can adjust the ease. In the present embodiment, the HMD 100 uses at least a light control plate 20 </ b> A having such a light transmittance that a user wearing the HMD 100 can visually recognize an outside scene. The light control plate 20A protects the right light guide plate 261 and the left light guide plate 262, and suppresses damage to the right light guide plate 261 and the left light guide plate 262, adhesion of dirt, and the like.

  The light control plate 20A may be detachable from the right optical image display unit 26 and the left optical image display unit 28. In this case, a plurality of types of light control plates 20A may be exchanged and mounted. For example, a configuration may be adopted in which one type can be selected from a plurality of types of light control plates having different visible light transmittances and attached to the image display unit 20. Moreover, you may select from the multiple types of light control board from which the color (transmission spectrum in visible region) of the light to permeate | transmits differs. The image display unit 20 may be configured to be usable even when the light control plate 20A is not attached.

The camera 61 is disposed at the boundary between the right optical image display unit 26 and the left optical image display unit 28. In a state where the user wears the image display unit 20, the position of the camera 61 is substantially in the middle of both eyes of the user in the horizontal direction and above the eyes of the user in the vertical direction. The camera 61 is a digital camera that includes an imaging element such as a CCD or CMOS, an imaging lens, and the like, and may be a monocular camera or a stereo camera.
The camera 61 images at least a part of the outside scene in the front side direction of the HMD 100, in other words, the user's view direction when the HMD 100 is worn. Although the angle of view of the camera 61 can be set as appropriate, it is preferable that the imaging range of the camera 61 overlaps with the outside world visually recognized by the user through the right optical image display unit 26 and the left optical image display unit 28. Furthermore, it is more preferable that the imaging range of the camera 61 is set so that the entire field of view of the user through the light control plate 20A can be imaged.

In addition, an external light sensor 68 (light detection unit) that detects light is disposed in the frame 2. The external light sensor 68 is an ambient light sensor that receives light and outputs a detection value corresponding to the received light quantity or the intensity of the received light.
The external light sensor 68 is disposed in the vicinity of the camera 61, receives light emitted from the direction including the angle of view of the camera 61 toward the external light sensor 68, and detects the amount of light. In addition, since the external light sensor 68 is provided adjacent to the image display unit 20, the external light sensor 68 receives light from substantially the same direction as the light irradiated or incident on the image display unit 20 from the outside. Accordingly, the external light sensor 68 receives light from the same direction as the light that is transmitted through the image display unit 20 and is incident on the user's eyes, and is thus regarded as receiving external light that is incident on the user's eyes. Can do.

Here, outside light refers to light incident on the user's eyes in addition to the light emitted by the right display drive unit 22 and the left display drive unit 24. Specifically, it refers to light emitted from a light source different from the right display drive unit 22 and the left display drive unit 24 shown in FIG. 2 among light incident on the user's eyes, and direct light and reflection directly incident from the light source. Either light or light may be included.
As shown in FIG. 1, the external light sensor 68 receives light emitted from the outside of the image display unit 20, and is particularly provided in a direction in which light emitted from the front of the user is easily received. The external light that actually enters the user's eyes includes light that has passed through the right optical image display unit 26, the left optical image display unit 28, and the dimming plate 20A, but the external light sensor 68 has a right optical image display unit. 26 or the left optical image display unit 28 may not be configured to receive the light itself. That is, the present invention can be implemented if the detection value of the external light sensor 68 can be used as an index of the amount or intensity of external light incident on the user's eyes. For example, as shown in FIG. 1, there is a configuration in which the external light sensor 68 can receive a part of light irradiated on the outer surface of the image display unit 20 from substantially the same direction as the external light incident on the user's eyes. . In the present embodiment, a configuration in which one external light sensor 68 is provided in the frame 2 is taken as an example, but a plurality of external light sensors 68 may be provided. Further, the position of the outside light sensor 68 may be provided at the end ER or the end EL in addition to the center in the width direction of the frame 2 as shown in FIG.
With this configuration, the light detected by the external light sensor 68 can be regarded as external light in the outside scene direction that is viewed by the user through the right light guide plate 261 and the left light guide plate 262. In other words, the external light sensor 68 detects the external light OL incident on the user's eyes as the background light of the image light L shown in FIG.

FIG. 2 is a principal plan view showing the configuration of the optical system provided in the image display unit 20. FIG. 2 shows the user's left eye LE and right eye RE for explanation.
The left display driving unit 24 includes a left backlight 222 having a light source such as an LED and a diffusion plate. The left display driving unit 24 includes a transmissive left LCD 242 disposed on the optical path of light diffused by the diffusion plate of the left backlight 222, a lens group that guides the image light L transmitted through the left LCD 242, and the like. The left projection optical system 252 is provided. The left LCD 242 is a transmissive liquid crystal panel in which a plurality of pixels are arranged in a matrix.

The left projection optical system 252 includes a collimator lens that converts the image light L emitted from the left LCD 242 into a parallel light beam. The image light L converted into a parallel light beam by the collimating lens is incident on the left light guide plate 262 (optical element). The left light guide plate 262 is a prism formed with a plurality of reflecting surfaces that reflect the image light L, and the image light L is guided to the left eye LE side through a plurality of reflections inside the left light guide plate 262. The left light guide plate 262 is formed with a half mirror 262A (reflection surface) positioned in front of the left eye LE.
The image light L reflected by the half mirror 262A is emitted from the left optical image display unit 28 toward the left eye LE, and this image light L forms an image on the retina of the left eye LE so that the user can visually recognize the image.

  The right display driving unit 22 is configured to be symmetrical with the left display driving unit 24. The right display driving unit 22 includes a right backlight 221 having a light source such as an LED and a diffusion plate. The right display driving unit 22 includes a transmissive right LCD 241 disposed on the optical path of light diffused by the diffusion plate of the right backlight 221, a lens group that guides the image light L transmitted through the right LCD 241, and the like. A right projection optical system 251 is provided. The right LCD 241 is a transmissive liquid crystal panel in which a plurality of pixels are arranged in a matrix.

The right projection optical system 251 has a collimator lens that converts the image light L emitted from the right LCD 241 into a light beam in a parallel state. The image light L converted into a parallel light beam by the collimating lens is incident on the right light guide plate 261 (optical element). The right light guide plate 261 is a prism formed with a plurality of reflecting surfaces that reflect the image light L, and the image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 261. The right light guide plate 261 is formed with a half mirror 261A (reflection surface) located in front of the right eye RE.
The image light L reflected by the half mirror 261A is emitted from the right optical image display unit 26 toward the right eye RE, and this image light L forms an image on the retina of the right eye RE to make the user visually recognize the image.

  The image light L reflected by the half mirror 261A and the external light OL transmitted through the light control plate 20A are incident on the right eye RE of the user. The image light L reflected by the half mirror 262A and the external light OL transmitted through the light control plate 20A are incident on the left eye LE. In this way, the HMD 100 superimposes the image light L of the internally processed image and the external light OL on the user's eyes and allows the user to see the external scene through the light control plate 20A. The image by the image light L is visually recognized. As described above, the HMD 100 functions as a see-through display device.

  The left projection optical system 252 and the left light guide plate 262 are collectively referred to as “left light guide unit”, and the right projection optical system 251 and the right light guide plate 261 are collectively referred to as “right light guide unit”. The configuration of the right light guide and the left light guide is not limited to the above example, and any method can be used as long as a virtual image is formed in front of the user's eyes using image light. It may be used, or a semi-transmissive reflective film may be used.

  The image display unit 20 (FIG. 1) is connected to the control device 10 via the connection unit 40. The connection unit 40 is a harness including a body cord 48, a right cord 42, a left cord 44, and a connecting member 46 that are connected to the control device 10. The right cord 42 and the left cord 44 are branched into two main body cords 48, and the right cord 42 is inserted into the housing of the right holding portion 21 from the distal end portion AP in the extending direction of the right holding portion 21, and the right display driving portion. 22 is connected. Similarly, the left cord 44 is inserted into the housing of the left holding unit 23 from the distal end AP in the extending direction of the left holding unit 23 and connected to the left display driving unit 24. The right cord 42, the left cord 44, and the main body cord 48 only need to be capable of transmitting digital data, and can be constituted by, for example, a metal cable or an optical fiber. Further, the right cord 42 and the left cord 44 may be combined into a single cord.

  The connecting member 46 is provided at a branch point between the main body cord 48, the right cord 42 and the left cord 44, and has a jack for connecting the earphone plug 30. A right earphone 32 and a left earphone 34 extend from the earphone plug 30. A microphone 63 is provided in the vicinity of the earphone plug 30. The earphone plug 30 to the microphone 63 are combined into one cord, and the cord branches from the microphone 63 and is connected to each of the right earphone 32 and the left earphone 34.

  For example, as shown in FIG. 1, the microphone 63 is arranged such that the sound collection unit of the microphone 63 faces the user's line of sight, collects sound, and outputs a sound signal. For example, the microphone 63 may be a monaural microphone or a stereo microphone, may be a directional microphone, or may be an omnidirectional microphone.

  The image display unit 20 and the control device 10 transmit various signals via the connection unit 40. The end of the main body cord 48 opposite to the connecting member 46 and the control device 10 are provided with connectors (not shown) that fit together. By fitting the connector of the main body cord 48 and the connector of the control device 10 or removing this fitting, the control device 10 and the image display unit 20 can be contacted and separated.

  The control device 10 has a box-shaped main body that is separate from the main body of the image display unit 20, and controls the HMD 100. The control device 10 includes switches including an enter key 11, a lighting unit 12, a display switching key 13, a luminance switching key 15, a direction key 16, a menu key 17, and a power switch 18. In addition, the control device 10 includes a track pad 14 that is operated by a user with fingers.

  The determination key 11 detects a pressing operation and outputs a signal for determining the content operated by the control device 10. The lighting unit 12 includes a light source such as an LED (Light Emitting Diode), and notifies the operation state (for example, power ON / OFF) of the HMD 100 according to the lighting state of the light source. The display switching key 13 outputs, for example, a signal instructing switching of the image display mode in response to the pressing operation.

  The track pad 14 has an operation surface for detecting a contact operation, and outputs an operation signal according to an operation on the operation surface. The detection method on the operation surface is not limited, and an electrostatic method, a pressure detection method, an optical method, or the like can be adopted. The luminance switching key 15 outputs a signal instructing increase / decrease of the luminance of the image display unit 20 in response to the pressing operation. The direction key 16 outputs an operation signal in response to a pressing operation on a key corresponding to the up / down / left / right direction. The power switch 18 is a switch for switching on / off the power of the HMD 100.

FIG. 3 is a functional block diagram of each part constituting the HMD 100.
The control device 10 includes a control unit 110 (second control unit) that controls the control device 10 and the image display unit 20. The control unit 110 is composed of, for example, a microprocessor, and is connected to a memory 121 that temporarily stores data processed by the control unit 110 and a storage unit 122 that stores data processed by the control unit 110 in a nonvolatile manner. Is done. Both the memory 121 and the storage unit 122 are configured by semiconductor elements, and are connected to the control unit 110 via a data bus.

The control unit 110 includes a power supply control unit 123, a UI (user interface) control unit 124, a wireless I / F (interface) control unit 125, an audio control unit 126, a sensor IC 127, and an external I / F (interface) unit 128. Is connected.
The HMD 100 includes a primary battery or a secondary battery as a power source, and the power control unit 123 includes an IC connected to these batteries. The power supply control unit 123 detects the remaining capacity of the battery according to the control of the control unit 110, and outputs detection value data or data indicating that the remaining capacity is equal to or less than the set value to the control unit 110.

  The UI control unit 124 includes the determination key 11, the display switching key 13, the track pad 14, the luminance switching key 15, the direction key 16, and the menu key 17 shown in FIG. 1, the lighting unit 12, and the track pad. 14 is an IC to be connected. Each of the operation units functions as an input unit, and the lighting unit 12 and the track pad 14 function as an output unit, and constitute a user interface of the HMD 100. The UI control unit 124 detects an operation in the operation unit, and outputs operation data corresponding to the operation to the control unit 110. Further, the UI control unit 124 performs lighting / extinguishing of the lighting unit 12 and display on the track pad 14 according to the control of the control unit 110.

The wireless I / F control unit 125 is a control IC connected to a wireless communication interface (not shown), and executes communication using the wireless communication interface according to the control of the control unit 110. The wireless communication interface included in the control device 10 performs wireless data communication conforming to a standard such as a wireless LAN (WiFi (registered trademark)), Miracast (registered trademark), Bluetooth (registered trademark), or the like.
The audio control unit 126 is an IC that is connected to the right earphone 32, the left earphone 34, and the microphone 63 and includes an A / D (analog / digital) converter, an amplifier, and the like. The sound control unit 126 outputs sound from the right earphone 32 and the left earphone 34 based on the sound data input from the control unit 110. In addition, the voice control unit 126 generates voice data based on the voice collected by the microphone 63 and outputs the voice data to the control unit 110.

  The sensor IC 127 includes, for example, a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor, and includes, for example, one IC including the sensor. The sensor IC 127 performs detection according to the control of the control unit 110 and outputs data indicating the detection value of each sensor to the control unit 110. The number and types of sensors provided in the sensor IC 127 are not limited, and an external light sensor, a temperature sensor, a pressure sensor, and the like may be provided.

  The external I / F unit 128 is an interface that connects the HMD 100 to an external device. For example, an interface corresponding to a wired connection such as a USB interface, a micro USB interface, a memory card interface, or the like can be used, and a wireless communication interface may be used. Various external devices that supply content to the HMD 100 can be connected to the external I / F unit 128. These external devices can also be referred to as image supply devices that supply images to the HMD 100. For example, a personal computer (PC), a mobile phone terminal, a portable game machine, or the like is used. In addition, the external I / F unit 128 may be provided with terminals connected to the right earphone 32, the left earphone 34, and the microphone 63. In this case, the analog audio signal processed by the audio control unit 126 passes through the external I / F unit 128. Input and output.

An I / F (interface) unit 115 is connected to the control unit 110. The I / F unit 115 is an interface including a connector connected to one end of the connection unit 40, and the other end of the connection unit 40 is connected to the I / F unit 155 of the image display unit 20.
The control unit 110 performs data communication with the sub-control unit 150 included in the image display unit 20 via the connection unit 40.

  The control unit 110 executes a program stored in the built-in ROM and controls each unit of the HMD 100. The control unit 110 acquires the detection value of the sensor based on the data input from the sensor IC 127 and stores it in the memory 121. At this time, the control unit 110 adds and stores time stamp information indicating the time when the detection value is acquired in association with the detection value of the sensor.

  In addition, the control unit 110 receives data indicating detection values of sensors (external light sensor 68, 9-axis sensor 162, and GPS 163) included in the image display unit 20 via the connection unit 40. The control unit 110 stores the received data in the memory 121. The data received by the control unit 110 includes time stamp information added by the sub-control unit 150. The control unit 110 adds the time stamp information added to the detection value of the sensor IC 127 as described above in a manner that can be distinguished from the time stamp information added by the sub control unit 150 and stores the time stamp information in the memory 121. The memory 121 stores the sensor detection value in a data format to which time stamp information is added as one of the data attributes. Here, the control unit 110 may store the sensor detection value data in the storage unit 122.

  The control unit 110 receives content data from an external device connected by the external I / F unit 128 or the wireless I / F control unit 125 and stores the data in the storage unit 122. The content data is data such as text and images displayed on the image display unit 20, and may include audio data output from the right earphone 32 and the left earphone 34. The control unit 110 controls the HMD 100 to reproduce content. Specifically, the control unit 110 transmits content display data to the sub-control unit 150 to execute display, and outputs audio data of the content to the audio control unit 126 to output audio. When the content data received from the external device includes data indicating a condition related to reproduction, the control unit 110 reproduces the content according to the condition. For example, when a detection value of a sensor such as a position or inclination detected in the image display unit 20 satisfies a condition, a text or an image corresponding to the detection value is displayed.

The image display unit 20 includes a sub-control unit 150 that performs communication with the control unit 110 and controls each unit of the image display unit 20. The sub control unit 150 is configured by, for example, a microprocessor, is connected to the connection unit 40 by the I / F unit 155, and executes data communication with the control unit 110 via the connection unit 40.
An external light sensor 68, a 9-axis sensor 162, and GPS 163 sensors are connected to the sub control unit 150. As described above, the ambient light sensor 68 is an IC of an ambient light sensor (ALS), or an IC in which a plurality of sensors including the ambient light sensor and peripheral circuits of the sensors are unitized. The 9-axis sensor 162 is an IC including a 3-axis acceleration sensor, a 3-axis gyro sensor, and a 3-axis geomagnetic sensor.

  The external light sensor 68 is a color sensor that receives light and detects the light amount or intensity in a plurality of different wavelength ranges (frequency bands). In the present embodiment, the external light sensor 68 is an RGB color sensor. The frequency sensitivity characteristic of the external light sensor 68 has a peak frequency corresponding to each of the three colors R (red), G (green), and B (blue). For example, the received light intensity is detected in each of a wavelength range of 570 to 700 nm corresponding to R, a wavelength range of 450 to 630 nm corresponding to G, and a wavelength range of 400 to 540 nm corresponding to B. The outside light sensor 68 preferably has, for example, characteristics suitable for human visual efficiency. The external light sensor 68 may be a sensor module in which a sensor (detection unit) main body and a circuit unit (detection circuit unit) that outputs a detection value of the sensor as digital data are modularized.

  The external light sensor 68 is driven by the control of the sub-control unit 150, and outputs, for example, a detection value indicating the amount of received light or the received light intensity in each frequency band of R, G, B to the sub-control unit 150 as digital data.

  The 9-axis sensor 162 is driven by the control of the sub-control unit 150 and outputs data indicating the detection value of each built-in sensor to the sub-control unit 150.

  The GPS 163 receives a position detection signal transmitted from a GPS satellite or a pseudo GPS transmitter (not shown) installed indoors, calculates the current position of the image display unit 20, and outputs the calculated data to the sub-control unit. 150. The GPS 163 may be configured to have only a function as a receiver that receives a signal for position detection. In this case, the sub-control unit 150 may perform a process of calculating the current position based on data output from the GPS 163.

The EEPROM 165 (setting data storage unit) stores data related to processing executed by the sub-control unit 150 in a nonvolatile manner.
In addition, the camera 61 is connected to the sub-control unit 150, and the sub-control unit 150 controls the camera 61 to execute imaging, and transmits captured image data of the camera 61 to the control unit 110.

  Connected to the sub-control unit 150 are an LCD drive unit 167 that performs drawing by driving the right LCD 241 and an LCD drive unit 168 that performs drawing by driving the left LCD 242. The sub-control unit 150 receives content data from the control unit 110, generates display data for displaying text and images included in the received data, outputs the display data to the LCD driving units 167 and 168, and executes display.

  The sub control unit 150 is connected to a backlight driving unit 169 that drives the right backlight 221 and a backlight driving unit 170 that drives the left backlight 222. The sub control unit 150 outputs control data including timing data for PWM control to the backlight driving units 169 and 170. The backlight driving units 169 and 170 supply driving voltages and pulses to the right backlight 221 and the left backlight 222 based on the control data input from the sub control unit 150, and the right backlight 221 and the left backlight 222. Lights up.

  Further, the sub control unit 150 designates the pulse width or duty of the pulse output from the backlight drive unit 169 to the right backlight 221 based on the data output to the backlight drive unit 169. Duty refers to the ratio between the on period and the off period of a pulse. Similarly, the sub control unit 150 specifies the pulse width or duty of the pulse output from the backlight drive unit 170 to the left backlight 222 based on the data output to the backlight drive unit 170. The right backlight 221 and the left backlight 222 are individual light sources such as LEDs, and the brightness of light emission, that is, the luminance can be adjusted by PWM control. Therefore, the amount of the image light L (FIG. 2) incident on the user's eyes can be adjusted by the control of the sub control unit 150. Further, the sub control unit 150 outputs different data to each of the backlight driving unit 169 and the backlight driving unit 170, and can individually adjust the luminance of the right backlight 221 and the luminance of the left backlight 222.

The backlight drive unit 169 can adjust the luminance of the right backlight 221 in steps, and the backlight drive unit 170 can also adjust the luminance of the left backlight 222 in steps. In the present embodiment, a configuration in which the brightness can be adjusted in 256 steps is given as an example. The sub control unit 150 outputs data specifying the luminance of the right backlight 221 and the left backlight 222 to the backlight driving units 169 and 170. This data is a luminance value from 0 to 255 indicating the luminance level. The backlight driving units 169 and 170 generate a pulse corresponding to the luminance value specified by the data input from the sub control unit 150 and output the pulse to the right backlight 221 and the left backlight 222, respectively.
In the sub-control unit 150, a luminance value set in consideration of luminance suitable for display and gamma values of the right LCD 241 and the left LCD 242 is set as an initial value.

The connection unit 40 that connects the control unit 110 and the sub-control unit 150 has a plurality of data buses including a control data bus 41A, an image data bus 41B, and display data buses 41C and 41D. These data buses can transmit data independently of each other, but the signal lines constituting each data bus may be physically partitioned, and each data bus may have a common signal line. It may be configured virtually or logically.
The control data bus 41A transmits control data transmitted from the control unit 110 to the sub control unit 150, sensor detection value data transmitted from the sub control unit 150 to the control unit 110, and the like. The image data bus 41 </ b> B transmits captured image data of the camera 61 from the sub control unit 150 to the control unit 110. The display data bus 41C transmits data to be displayed by the right display drive unit 22, and the display data bus 41D transmits data to be displayed by the left display drive unit 24.

  The sampling periods of a plurality of sensors including the external light sensor 68, the 9-axis sensor 162, and the GPS 163 included in the image display unit 20 may be greatly different. For example, the sampling period (sampling frequency) of the acceleration sensor of the 9-axis sensor 162 may be 200 times / second or more. On the other hand, the sampling period of the external light sensor 68 may be slower, and it is considered that 1 to 10 times / second (1000 to 100 ms period) is sufficiently useful. In these sensors, the sub-control unit 150 sets a sampling cycle, and the sub-control unit 150 acquires a detection value according to the set sampling cycle. The sub-control unit 150 transmits detection value data sampled from each sensor to the control unit 110 in a time division manner on the control data bus 41A.

  For this reason, the control data bus 41A is not occupied for a long time in order to control a sensor having a slow sampling cycle (in other words, a low sampling frequency or a long sampling interval). As a result, the overhead of the control data bus 41A can be reduced, and the detection values of a large number of sensors can be efficiently transmitted on the control data bus 41A. Further, the sub-control unit 150 has a RAM (not shown), and temporarily stores the detection value of the sensor in the RAM. The sub control unit 150 adjusts the transmission timing of the data stored in the RAM and sends the data to the control data bus 41A. Therefore, the operation of the sub-control unit 150 is also not easily restricted by the sampling period of each sensor, and the situation where the processing of the sub-control unit 150 is occupied for sensor control can be prevented.

  The control unit 110 performs AR display by the image display unit 20 by transmitting and receiving data via each bus of the connection unit 40. The AR display has a so-called AR (Augmented Reality) effect in accordance with an object in real space that is transmitted through the image display unit 20 (the right optical image display unit 26 and the left optical image display unit 28) and is visually recognized by the user. This is an operation for displaying an image (hereinafter referred to as an AR image). When the AR display is performed, the user can see the AR image at a position overlapping the object or a position corresponding to the object while viewing the object existing in the real space. For this reason, since the virtual display object including a character and an image is added to the object in the real space and can be visually recognized for the user, there is an effect as if the reality is expanded.

4A and 4B are diagrams showing the display state of the HMD 100, FIG. 4A is an explanatory diagram showing the position of the display area, FIG. 4B shows a first display example, and FIG. 4C shows a second display example. Indicates. 4A to 4C, VR indicates a user's visual field, and VS indicates an outside scene that the user can see in the visual field VR.
V1 indicates an area where the HMD 100 can display an image and allow the user to visually recognize it, that is, a display area of the image display unit 20. As shown in FIG. 4A, the display area V1 is located in the approximate center of the user's visual field VR and is narrower than the visual field VR. The display area V1 may be the same size as the visual field VR, and the size and position are not limited to the example of FIG.

  The display region V1 is formed by reflecting image light to the user's eyes by the half mirror 261A (FIG. 2) of the right light guide plate 261 and the half mirror 262A of the left light guide plate 262. The display area V1 corresponds to the right LCD 241 and the left LCD 242. For example, when an image is displayed on the entire displayable area (displayable area) of the right LCD 241 and the left LCD 242, the user can visually recognize an image having the entire size of the display area V1.

The HMD 100 displays a display object such as an image or text constituting the AR image in the display area V1. The display position and display size of the display object are determined in accordance with the position of the target object in the real space (outside scene) visually recognized by the user. For example, the control unit 110 analyzes the captured image of the camera 61 as will be described later, and displays the display position of the display object based on data that determines the relative position between the angle of view (imaging range) of the camera 61 and the display area V1. Performs processing to determine the size. As described above, the angle of view (imaging range) of the camera 61 overlaps the visual field VR of the user, and more preferably, the angle of view of the camera 61 includes the entire visual field VR. Therefore, an object in real space that the user visually recognizes in the visual field VR can be reflected in the captured image of the camera 61, and the object can be detected from the captured image of the camera 61.
The object may be a movable object, an object that cannot be moved, that is, a part or all of a fixed building, or a part of the surface of the object may be the object.

  In the example of FIG. 4 (A), the visual field VR of the user includes an object OB1 that is a door existing in real space. In this case, the user sees the object OB1 through the image display unit 20. The HMD 100 detects the position of the object OB <b> 1 that is the target for AR display from the captured image of the camera 61. The position detected here is the relative position of the object OB1 with respect to the camera 61. In addition, the HMD 100 stores data indicating the relative positional relationship between the position in the captured image of the camera 61 and the display area V1 in the storage unit 122 in advance. This data can be included in the setting data 122b, for example. In this case, the HMD 100 calculates the position of the object OB1 in the display area V1 from the position of the object OB1 in the captured image using the setting data 122b.

  FIG. 4B shows an example in which the image AR1 is displayed with higher visibility than the object OB1, that is, in a conspicuous manner. When performing the display of the example of FIG. 4B, the HMD 100 determines the display position of the image AR1 so as to overlap the position where the object OB1 is visually recognized, and sets the display size of the image AR1 to the maximum size in the display region V1. Set to. The image AR1 in FIG. 4B is preferably displayed so as to be more conspicuous than the outside scene VS. For example, the display brightness in the display area V1 is set to high brightness and displayed in a highly visible color.

  FIG. 4C shows an example of AR display that complements the outside scene VS. The images AR2 and AR3 in FIG. 4C are images that have an effect of allowing the user to visually recognize the color and texture of the object OB1 of the outside scene VS as different from the color of the outside scene VS. The image AR2 is displayed so as to overlap the object OB1 in accordance with the shape of the object OB1. Further, the image AR3 is displayed at a position in contact with the image AR2 in a size that matches the image AR2. In the example of FIG. 4C, it is preferable that the images AR2 and AR3 are displayed so as to become familiar with the outside scene VS. That is, it is preferable that the images AR2 and AR3 do not impair the visibility of the object OB1 and the outside scene VS around the object OB1. For this reason, for example, the display brightness of the images AR2 and AR3 in the display area V1 is set to a brightness that does not impair the visibility of the outside scene VS.

Data for displaying the images AR1 to AR3, which are display objects shown in FIGS. 4B and 4C, are stored in the storage unit 122 as content data 122a. The content data 122a may include raster image data of a plurality of display objects, or may include vector image data for generating display objects. Alternatively, it may include parameters, functions, text data, etc. for generating a display object. The display object is not limited to an image, and may include characters, a still image, or a moving image.
Further, data for displaying a plurality of display objects may be included in the content data 122a. The content data 122a may include audio data, and audio may be output from the audio control unit 126 based on the audio data of the content data 122a when a display object is displayed.

  The visibility of the display object displayed by the HMD 100 in the AR display and the visibility of the outside scene VS that is visible to the user through the image display unit 20 are transmitted through the image display unit 20 from the external environment. It is affected by light incident on the eye, that is, external light. When the amount of external light is large (the intensity is high), the visibility of the outside scene VS is increased, but the visibility of the display object is decreased as compared with the case where the amount of light is small (the intensity is low). In addition, the visibility of the display object may be affected by the color of external light. The color of external light can also be said to be the balance of wavelength components contained in external light. When external light with high light intensity in some wavelengths in the visible region is incident on the user's eyes, the visibility of the wavelength region with high light intensity in the display object is higher than when the external light is white light. Decreases. For this reason, the appearance of the color of the display object changes depending on the color of the external light, and a phenomenon such as a reduction in the visibility of a part of the display object may occur.

  The HMD 100 includes an adjustment function that changes the display mode of the display object displayed in the display area V1 according to the color of the external light detected by the external light sensor 68, and maintains the visibility of the display object. Since the external light sensor 68 is composed of a color sensor, for example, the intensity of external light can be detected as the intensity for each wavelength region. Further, as a detection result, the frequency spectrum of the external light may be obtained from the detection value of the external light sensor 68. Further, the chromaticity coordinates of the color of the external light may be calculated from the detection value of the external light sensor 68, and the detection result may be obtained. The detection results relating to the color of these external lights are collectively referred to as color tone. The HMD 100 obtains the color tone of the external light including any of the above from the detection value of the external light sensor 68. Then, the HMD 100 adjusts the display color and display luminance of the display object based on the detection result of the color tone of the external light.

The control unit 110 (FIG. 3) that controls the HMD 100 includes an imaging control unit 111, an image processing unit 112, and a display control unit 113 as functional units that execute the AR display and the adjustment function.
The imaging control unit 111 causes the camera 61 to perform imaging by transmitting a command to the sub-control unit 150 via the I / F units 115 and 155. The sub control unit 150 receives an imaging command transmitted from the imaging control unit 111, causes the camera 61 to execute imaging, and transmits captured image data of the camera 61 to the imaging control unit 111.

  The display control unit 113 causes the image display unit 20 to display a display object based on the content data 122 a stored in the storage unit 122. The display control unit 113 transmits the content data 122 a or display object data generated based on the content data 122 a via the I / F units 115 and 155. Further, the display control unit 113 transmits data indicating the display position and size of the display object, a command for instructing display of the display object, and the like. The sub-control unit 150 receives commands and data transmitted from the control unit 110 and drives the LCD driving units 167 and 168 and the backlight driving units 169 and 170 based on the received data to execute display.

  When performing display based on the content data 122a, the display control unit 113 performs processing of extracting image data of the display object from the content data 122a or processing of generating image data of the display object based on the content data 122a.

  The display control unit 113 performs processing for obtaining the display position of the display object in the display area V1, that is, the display position in the LCD driving units 167 and 168. When the normal display is performed without performing the AR display, the display control unit 113 obtains the display position of the display object based on the data included in the content data 122a or the setting data 122b set in advance, and designates the display position. Send a command.

  Further, when performing AR display, for example, when the attribute data indicating that AR display is included in the content data 122a, the display control unit 113 matches the position where the user visually recognizes the object OB1 of AR display, Processing for determining the display position of the display object is performed. In this process, the display control unit 113 detects an image of the object OB1 from the captured image. In the HMD 100, for the image of the object OB to be detected, data relating to the feature amount such as shape and color is stored in the storage unit 122 as setting data 122b, for example. The display control unit 113 detects the image of the object OB1 from the captured image using the setting data 122b stored in the storage unit 122, and specifies the position of the object OB1 in the captured image. Further, the display control unit 113 specifies the display position of the display object on the LCD driving units 167 and 168, and displays the display object.

Further, the display control unit 113 performs a process of adjusting the color tone and / or luminance of the display object displayed on the image display unit 20 based on the detection value of the external light sensor 68. The display control unit 113 acquires the received light intensity for each wavelength range of the external light from the detection value of the external light sensor 68, and based on the acquired received light intensity, the image data of the display object to be transmitted to the image display unit 20 is displayed as an image. Processing is performed by the processing unit 112.
The image processing unit 112 performs a process of changing the color tone or a process of adjusting the luminance of the image data extracted by the display control unit 113 from the content data 122a or the image data generated by the display control unit 113 based on the content data 122a. Execute. For example, the image processing unit 112 corrects the RGB gradation value of each pixel constituting the image data to be processed according to the correction parameter specified by the display control unit 113.

  When performing the AR display, the display control unit 113 displays a display mode in which the display object is displayed so as to overlap the object OB1 and a display mode in which the display object is displayed so as to be viewed behind the target OB1. You may switch. Further, the image processing unit 112 may process the image data so that the display object is displayed in a display form that is visually recognized as having a shadow.

FIG. 5 is a flowchart showing the operation of the HMD 100, (A) shows the operation of the control device 10, and (B) shows the operation of the image display unit 20.
When the start of the display operation is instructed by an operation on the control device 10, the control unit 110 generates an activation command and transmits it to the sub-control unit 150 (step S11). This command is transmitted via the control data bus 41A, and the sub-control unit 150 receives the command (step S31).

  The sub control unit 150 starts the operation, initializes the sub control unit 150 according to the command, and initializes each part of the image display unit 20 including the camera 61, the external light sensor 68, the 9-axis sensor 162, and the GPS 163. It performs (step S32).

  The control unit 110 acquires the image data of the display object constituting the AR image based on the content data 122a, and transmits it to the sub-control unit 150 (step S12). The sub control unit 150 receives the image data and starts displaying an image based on the received image data (step S33). Here, the sub control unit 150 operates the backlight driving units 169 and 170 to turn on the right backlight 221 and the left backlight 222, and operates the LCD driving units 167 and 168 based on the received image data. Display an image.

Then, the control part 110 produces | generates the detection command which instruct | indicates execution of detection by the external light sensor 68, and transmits to the sub-control part 150 (step S13).
The sub control unit 150 receives the detection command (step S34), operates the external light sensor 68 to acquire the detection value, and transmits the detection result, that is, data indicating the detection value, to the control unit 110 (step S35). .

  The control unit 110 receives the detection result of the external light sensor 68 (step S14), and executes image processing (step S15). This image processing is processing for adjusting the visibility of the display object based on the detection result of the external light sensor 68, and details will be described later.

The control unit 110 generates an imaging command that instructs execution of imaging by the camera 61 and transmits the imaging command to the sub-control unit 150 (step S16). The sub control unit 150 receives and executes the imaging command (step S36), causes the camera 61 to perform imaging, and transmits the captured image data to the control unit 110 (step S37).
The control unit 110 receives the captured image data (step S18), and determines the display position and display size of the display object constituting the AR image based on the captured image data (step S18). In step S18, the control unit 110 detects the target object OB1 that is the target of AR display from the captured image as described above, and displays the display position of the display object and the display object based on the position and size of the target object OB1 in the captured image. Determine the size. The control unit 110 transmits display position data specifying the display position and display size of the display object to the sub-control unit 150 (step S19).

  The sub control unit 150 receives the display position data (step S38), and updates the display position and size of the display object on the right LCD 241 and the left LCD 242 (step S39).

Control unit 110 determines whether or not to end the display (step S20). When the display is not finished (step S20; NO), the control unit 110 returns to step S13. Further, when the end of the display is instructed by the operation of the control device 10, the control unit 110 determines to end the display (step S20; YES), and stops the transmission of the image data (step S21). The control unit 110 generates an end command for instructing the end of display and transmits it to the sub-control unit 150 (step S22).
The sub-control unit 150 determines whether an end command has been received (step S40). When the end command is received (step S40; YES), the sub control unit 150 stops the LCD driving units 167 and 168 and the backlight driving units 169 and 170 and ends the display (step S41). If the end command is not received (step S40; NO), the sub-control unit 150 returns to step S34.

FIG. 6 is a flowchart showing in detail the image processing executed by the control unit 110 in step S15 of FIG.
The control unit 110 acquires the color tone of the external light from the detection result of the external light sensor 68 received in step S14 (FIG. 5) (step S101). The color tone of the external light is, for example, a detection value such as the above-described received light intensity for each wavelength, the intensity of each wavelength region of R, G, and B, the received light intensity at a specific wavelength set in advance, or the frequency obtained from the detected value. The spectrum or the chromaticity coordinates of external light. The “color tone” described in the present embodiment may be referred to as “color tone information” including at least one of the above-described various types of information regarding the color tone of external light.

The control unit 110 generates or acquires a correction parameter corresponding to the color tone of external light (step S102). In step S102, the control unit 110 may generate a correction parameter by performing arithmetic processing based on the color tone of the external light acquired in step S101. In addition, a table (not shown) that associates the color tone of external light and the correction parameter may be stored in the storage unit 122 in advance, and the correction parameter may be acquired using this table. The table in this case may be included in the setting data 122b.
Then, the control unit 110 performs a process of correcting the image data of the display object using the correction parameter generated or acquired in Step S102 (Step S103).

As specific examples of the processing shown in FIG. 6, there are three types of processing.
The first process is a process for maintaining the visibility of the display object so as not to deteriorate. In other words, when the visibility of the display object may be lowered due to the influence of external light, this is a process for preventing the visibility from being lowered.

  When there is a possibility that the color tone of the display object is changed due to the influence of the color tone of the external light, the control unit 110 changes the gradation value of the image data of the display object so as to cancel the effect of the external light. to correct. The right LCD 241 and the left LCD 242 are configured by a liquid crystal display panel that displays, for example, three colors R, G, and B, and have a color tone characteristic that matches the white light emitted by the right backlight 221 and the left backlight 222. ing. Therefore, when the external light is so-called white light, the display object is visually recognized in the original color. When the color tone of the external light is different from that of white light, the image light constituting the display object and the external light overlap and enter the user's eyes, so the display object differs from the original color tone It looks in color. In this case, the control unit 110 corrects each of the R, G, and B color data constituting the image data so that the display object is visually recognized as in the case where the external light is white light. For example, the control unit 110 performs gamma correction for each color of the image data.

The second process is a process for increasing the visibility of the display object.
The control unit 110 corrects the image data of the display object so that the contrast with the color tone of the external light becomes high. In the second process, the control unit 110 corrects the image data of the display object so that the contrast with the color tone of the external light becomes high. As a result of the second processing, the display object may be visually recognized with a color tone different from that when the external light is white light.

  The first and second processes are useful for a display object that is desirably visually recognized by the user so as to be conspicuous with respect to the outside scene VS, for example, an image AR1 illustrated in FIG.

The third process is a process of changing the color tone of the display object in accordance with the color tone of the external light. When performing the third processing, the control unit 110 corrects the gradation value of the image data so that the color tone of the display object is close to the color tone of the external light.
According to the third process, it is possible to obtain a visual effect such that the display object melts into the outside scene VS. This third process is useful for display objects that bring about a visual effect when viewed simultaneously with the real space (object), such as images AR2 and AR3 shown in FIG. 4C, for example. .

FIG. 7 is a flowchart showing in detail another example of the image processing executed by the control unit 110 in step S15 of FIG.
In the image processing of FIG. 7, the control unit 110 corrects the image data in accordance with the brightness of external light, that is, the received light intensity received by the external light sensor 68.

  The control unit 110 calculates the intensity of external light from the detection result of the external light sensor 68 received in step S14 (FIG. 5) (step S111). The intensity of the external light may be, for example, the light reception intensity of the external light sensor 68 at a specific wavelength, or may be a value calculated from the light reception intensity at a plurality of preset wavelengths or wavelength ranges.

  The controller 110 determines whether or not the intensity of external light is darker than a preset setting value (step S112). If it is darker than the set value (step S112; YES), the control unit 110 generates or acquires a correction parameter for dark place from the setting data 122b (step S113), and corrects the image data of the display object using this correction parameter. (Step S114). If the intensity of the external light is the same as or brighter than the set value (step S112; NO), the control unit 110 ends the process without correcting the image data.

  Among rod cells and cone cells constituting human photoreceptor cells, rod cells are known to work predominantly when the intensity of external light is weak (external light is dark). The photosensitive characteristics (light absorption characteristics) of rod cells have a peak at 500 to 550 nm, and the peak wavelength of the photosensitive characteristics is on the shorter wavelength side than cone cells. Therefore, when the outside light is dark, the light on the blue side is more intense than white. In the image processing of FIG. 7, the image data of the display object is corrected so that the blue color becomes strong so that the image light of the display object can be felt strongly by the rod cells. Thereby, when the intensity | strength of the light which injects into a user's eyes is weak, a display object can be made to be visually recognized favorably. The image processing of FIG. 7 has an effect that the visibility of the display object can be improved so that the contrast between the display object and the outside scene VS does not become extremely large.

The control unit 110 can also execute a combination of the image processing shown in FIG. 6 and the image processing shown in FIG. An example of this is shown in FIG.
FIG. 8 is a flowchart showing in detail another example of the image processing executed by the control unit 110 in step S15 of FIG. In FIG. 8, the same step numbers are assigned to the processing steps that perform the same processing as in FIGS.

  In the image processing of FIG. 8, the control unit 110 determines the attribute of the image data, that is, the attribute of the display object (step S121). When the image processing shown in FIG. 8 is performed, data indicating the attributes of the display object is added to the image data of the display object included in the content data 122a. The attribute of the display object indicates, for example, whether or not the display object is conspicuous with respect to the outside scene VS, that is, whether or not the visibility of the display object is increased. The image AR1 shown in FIG. 4B and the images AR2 and AR3 shown in FIG. 4C have different attributes.

  The control unit 110 determines whether or not to perform a process for improving the visibility of the display object based on the attribute of the display object acquired in step S121 (step S122). When performing the process of improving the visibility (step S122; YES), the control unit 110 executes the processes of steps S111 to S112 illustrated in FIG. 7, and whether the intensity of external light is darker than a preset setting value. It is determined whether or not (step S112).

When it is darker than the set value (step S112; YES), the control unit 110 acquires a dark place correction parameter from the setting data 122b (step S113).
Further, the control unit 110 acquires the color tone of the external light from the detection result of the external light sensor 68 (step S123), and generates a correction parameter to be applied to the image data (step S124). In step S124, the control unit 110 generates a correction parameter in which the correction parameter for the dark place acquired in step S113 is combined with the correction parameter for improving the visibility of the display object corresponding to the color tone of the external light. Note that the correction parameter generated by the control unit 110 in step S124 can ensure the visibility in the dark place as in the image processing of FIG. 7, and the first or second processing described with reference to FIG. Any parameter can be used as long as the same effect can be obtained, and the calculation method is arbitrary.

  The control unit 110 corrects the image data of the display object using the generated correction parameter (step S125). Further, the control unit 110 executes at least one of a contrast correction process for correcting the contrast and an edge correction process on the corrected image data (step S126). The contrast correction process can be, for example, a process that enhances the contrast and enhances the visibility of dark images. Further, the edge correction process can be, for example, a process for enhancing the visibility of a dark image by performing edge enhancement. Alternatively, the contrast may be reduced by the contrast correction process, and the edge blur process may be performed by the edge correction process to obtain an effect of making the display object familiar with external light.

  If the intensity of the external light is equal to or brighter than the set value (step S112; NO), the control unit 110 acquires the color tone of the external light from the detection result of the external light sensor 68 (step S131). Then, a correction parameter is generated or acquired (step S132). The correction parameter generated in step S132 is a correction parameter that maintains or enhances the visibility of the display object with respect to external light, and has the same effect as the first or second processing described with reference to FIG. This is the parameter obtained. After generating or acquiring the correction parameter, the control unit 110 proceeds to step S125 and corrects the image data.

  On the other hand, when the process for increasing the visibility of the display object is not performed based on the attribute of the image data (step S122; NO), the control unit 110 acquires the color tone of the external light from the detection value of the external light sensor 68 (step S122). S133). The control unit 110 generates or acquires a correction parameter that matches the color tone of the image data of the display object with the color tone of the external light (step S134), and corrects the image data using the correction parameter (step S135). The processing in steps S133 to S135 is the same processing as the third processing described with reference to FIG.

  When performing the image processing of FIG. 8, the control unit 110 corrects the visibility of the display object by correcting the image data of the display object according to the brightness of the external light and the attribute of the display object. It can be.

  Further, when the outside light sensor 68 has a configuration in which the range (detection range) of the intensity of light to be detected can be switched in a plurality of stages, the range of the outside light sensor 68 may be switched in the operations of FIGS. . Specifically, the control unit 110 designates the detection range of the outside light sensor 68 by a detection command (FIG. 5A) transmitted to the sub control unit 150. When the received light intensity indicated by the detection result of the external light sensor 68 is a value smaller than the upper limit of the currently set range, close to the upper limit, an upper limit value, or a value exceeding the upper limit value, the external light sensor A detection command for switching the 68 range to the high intensity side is transmitted. Thereby, even if it is a case where the intensity | strength of external light is high, external light can be detected appropriately. In addition, the controller 110 detects that the received light intensity indicated by the detection result of the external light sensor 68 is a value close to the lower limit of the currently set range, a lower limit value, or a value lower than the lower limit value. Send a detection command to switch the range to the low intensity side. Thereby, even if it is a case where the intensity | strength of external light is weak, external light can be detected appropriately. Then, the control unit 110 may acquire or generate a correction parameter corresponding to the range of the external light sensor 68 and correct the image data. In this case, the controller 110 replaces the process of determining the intensity from the detection result of the external light sensor 68 in step S112 (FIGS. 7 and 8) based on the range set by the external light sensor 68. Whether the light intensity is darker than the set value can be determined.

As described above, the HMD 100 according to the first embodiment to which the present invention is applied includes the image display unit 20 that outputs image light including a plurality of color lights to the user and displays a display object. The HMD 100 includes an external light sensor 68 that detects light. The control unit 110 controls the color tone of the image light output from the image display unit 20 based on the detection result of the external light sensor 68. Thereby, the color tone of the image light can be changed in accordance with the light detected by the external light sensor 68. For this reason, it is possible to control the color tone of the image light and the light incident on the user's visual field in addition to the image light, or to control the color tone to enhance the visibility of the image light. Thereby, the visibility of the image which HMD100 displays can be adjusted appropriately.
Since the external light sensor 68 detects light from a different direction from the image light output from the image display unit 20, control is performed so that the color tone of the image light and the light incident on the user's field of view other than the image light are harmonized. Or the color tone can be controlled to improve the visibility of the image light.

The image display unit 20 is mounted on the user's head and has a configuration for outputting image light in a state where the external light can be incident on the user's field of view. The external light sensor 68 detects external light. . For this reason, the visibility of an image can be adjusted by changing the color tone of the image which HMD100 with which a user's head is mounted | worn according to the external light which injects into a user's visual field.
In addition, since the external light sensor 68 is provided adjacent to the image display unit 20, the light detected by the external light sensor 68 in the vicinity of the image display unit 20 and the color tone of the image light are controlled to be in harmony. The color tone can be controlled to enhance the visibility of light.

  Further, the control unit 110 causes the image display unit 20 to output the color light that constitutes the image based on the image data including a plurality of color data, and changes the gradation value of the color data included in the image data, whereby the image is displayed. Change the color of light. For this reason, the visibility of an image can be adjusted by data processing by changing the gradation value for each color included in the image data.

Further, since the external light sensor 68 includes a sensor that detects the intensity of received light for each of a plurality of different wavelengths, information regarding the color of light incident on the user's field of view can be obtained.
Further, the control unit 110 may control the color tone of the image light output from the image display unit 20 based on the light intensity for each wavelength detected by the external light sensor 68. In this case, the visibility of the image displayed by the HMD 100 can be appropriately adjusted by controlling the color tone of the image light corresponding to the color of the light incident on the user's visual field.

  Moreover, since the control part 110 controls the color tone of image light so that the visibility which a user visually recognizes image light is improved, HMD100 displays according to the color of the light which injects into a user's visual field. The visibility of an image can be improved.

  In addition, the control unit 110 controls the luminance of each color light constituting the image light so that the color tone of the image light approaches the color tone of the light detected by the external light sensor 68. Thus, the color of the image light can be adapted to the color of light incident on the user's field of view, and the image displayed by the HMD 100 can be harmonized with the color of external light or the like.

  Further, as described with reference to FIG. 6, the control unit 110 enhances the visibility with which the user visually recognizes the image light based on the light intensity for each wavelength detected by the external light sensor 68. The first and second processes for controlling the luminance of each color light constituting the image light can be performed. In addition, the control unit 110 configures the image light so that the color tone of the image light approaches the color tone of the light detected by the external light sensor 68 based on the light intensity for each wavelength detected by the external light sensor 68. A third process for controlling the luminance of each color light can be performed. Then, as illustrated in FIG. 8, the control unit 110 can select and execute the first and second processes and the third process according to the attribute of the image displayed on the image display unit 20. For this reason, it is possible to select and execute a process for improving the visibility of an image with respect to external light or the like and a process for harmonizing the image with external light or the like according to the attribute of the image displayed by the HMD 100.

Here, in step S12 of FIG. 5, the visibility of the display object such as a high visibility mode for performing a process for improving the visibility of the display object, a high harmony mode for performing a process for displaying the display object in harmony with the outside scene, and the like. The color correction curve, the γ characteristic, and the edge correction processing may be different depending on the type of processing related to the property. Further, the color correction curve, the γ characteristic, and the edge correction process may be different depending on the scene using the HMD 100.
For example, a plurality of operation modes may be provided for the visibility of the display object and the outside scene. Examples of operation modes include: (1) “Low Illuminance Environment Mode” that enhances the visibility of both image light and the outside scene in a dark environment with low illuminance (such as when the outside is visible only in black in the dark). Is mentioned. Further, for example, (2) a “blue sky document viewing mode” is provided that enhances the visibility and readability of display objects including manuals such as work instructions in the state where the illuminance of outside light is higher than a set value, such as hot summer sun. This operation mode can also be called a “high visibility mode” in which priority is given to visibility. Further, for example, (3) “document priority mode” that improves the visibility of characters and the like in a display object including a document such as a work standard and facilitates work while viewing the document. This operation mode can include an operation for improving the readability by displaying a work standard character in a fluorescent color or the like. Further, for example, (4) “additional image fusion mode” or “color fitting mode” in which the visibility of a display object displayed by AR display or the like is highly harmonized with the outside scene can be cited. The HMD 100 may have a plurality of lookup tables (LUT) related to display of display objects and arithmetic expressions (correction curves: combinations of modes based on blurring, etch enhancement, etc.) for each operation mode. Further, the operation mode may include a plurality of operation modes for changing the image display brightness corresponding to the ambient light color.

  Further, when executing the first or second processing, the control unit 110, when the intensity of the light detected by the external light sensor 68 is lower than a preset intensity, each color light included in the image light The luminance ratio is changed to a preset ratio. Specifically, the image data is corrected using correction parameters for dark places. For this reason, even when the intensity of light such as outside light is low, the visibility of the image can be adjusted appropriately.

  In addition, when the intensity of light detected by the external light sensor 68 is lower than a preset intensity, the control unit 110 changes the luminance ratio of each color light included in the image light to a preset ratio. . Specifically, the image data is corrected using correction parameters for dark places. For this reason, even when the intensity of light such as outside light is low, the visibility of the image can be adjusted appropriately.

  Further, the outside light sensor 68 may be configured to be able to switch the range of received light intensity to be detected. In this case, one of the ranges corresponds to the first detection state, and the other range corresponds to the second detection state suitable for detecting light having a lower intensity than the first detection state. The control unit 110 can generate or acquire a correction parameter for correcting the image data in response to the range switching or setting state of the external light sensor 68. Therefore, the external light sensor 68 can detect external light with high accuracy.

  In addition, as described with reference to FIG. 8, the control unit 110 displays an image of an image displayed by the image display unit 20 when the intensity of light detected by the external light sensor 68 is lower than a preset intensity. Contrast correction processing and / or edge correction processing is performed on the data. For this reason, when the intensity | strength of light, such as external light, is low, the visibility of an image can be improved more effectively.

[Second Embodiment]
FIG. 9 is an explanatory diagram of the operation of the HMD 100 in the second embodiment to which the present invention is applied. (A) shows an area provided in the captured image corresponding to the display area, and (B) shows the display area. . Since the HMD 100 in the second embodiment is common to the first embodiment except for the configuration of the external light sensor 68, the configuration of the HMD 100 is denoted by the same reference numerals and the description thereof is omitted.

  The outside light sensor 68 in the second embodiment has a configuration in which a detection range for detecting outside light is set, and the color tone of outside light in the detection range can be detected in association with the position in the detection range. This configuration can be realized, for example, by adopting a digital camera capable of color imaging as the external light sensor 68.

  In the example shown in FIG. 9A, the external light detection range V2 is set corresponding to the visual field VR of the user. The outside light detection range V2 is a range in which the outside light sensor 68 detects outside light, and is divided into four regions V21, V22, V23, and V24. The external light sensor 68 detects the received light intensity of the external light for each of the four regions V21 to V24 and outputs a detection value.

The outside light detection range V2 is associated with the display area V1. In the second embodiment, the control unit 110 divides the display area V1 into four small areas V11, V12, V13, and V14 as shown in FIG. 9B, and displays each of the small areas V11 to V14. You can correct the color of the object.
The external light sensor 68 may be disposed at a position distant from the right optical image display unit 26 and the left optical image display unit 28 and may not directly detect external light that actually enters the display region V1. In this case, the outside light detection range V2 of the outside light sensor 68 is a position away from the position where the outside light sensor 68 actually receives outside light. The control unit 110 can indirectly detect the color tone of the external light in the external light detection range V2 by converting the actual detection value of the external light sensor 68 into the color tone of the external light in the external light detection range V2. It is also possible to store parameters and arithmetic expressions for associating the display area V1 with the outside light detection range V2 in the storage unit 122 and use the parameters and arithmetic expressions by the control unit 110.

  FIG. 10 is a flowchart showing the operation of the HMD 100 in the second embodiment. The operation in FIG. 10 is executed in step S12 in FIG. 5A, and corresponds to another example of the operations in FIGS. 6, 7, and 8 described above.

  The control unit 110 acquires each color tone of the regions V21 to V24 in the external light detection range V2 based on the detection value of the external light sensor 68 (step S201). The color tone acquired for each of the regions V21 to V24 is the same as the color tone acquired in step S101 of the first embodiment. The controller 110 generates or acquires correction parameters for each of the small areas V11 to V14 of the display area V1 based on the color tone acquired in step S201 (step S202).

  The control unit 110 acquires image data of the display object and divides this image data corresponding to the small areas V11 to V14 (step S203). Then, correction is performed on each of the divided image data using the correction parameter in step S202 (step S204).

  As described above, the control unit 110 divides the display area V1 into the plurality of small areas V11 to V14 based on the detection result of detecting the external light incident on the user's visual field by the external light sensor 68, and the small area V11. The color tone of the image light is controlled for each V14. According to the second embodiment, when the color tone of the external light incident on the visual field VR varies depending on the location of the visual field VR, the color tone of the image light constituting the display object can be finely corrected corresponding to the external light. For this reason, when there is a difference in color tone of external light in the visual field VR, the visibility of the display object can be effectively adjusted.

[Third Embodiment]
Next, a third embodiment to which the present invention is applied will be described. Since the HMD 100 in the third embodiment is common to the first embodiment, the configuration of the HMD 100 is denoted by the same reference numerals and description thereof is omitted.

  In the third embodiment, the control unit 110 corrects the image data of the display object in accordance with the color tone of the light control plate 20A. As described above, since the light control plate 20A is positioned outside the right optical image display unit 26 and the left optical image display unit 28, external light incident on the visual field VR is transmitted through the light control plate 20A. For this reason, the color tone of external light is affected by the light transmission characteristics (transmission spectrum in the visible light region) of the light control plate 20A. In addition, the light control plate 20A can be configured to be removable as described above, and the light control plate 20A is used in a non-mounted state, and a plurality of light control plates 20A having different light transmission characteristics. It is possible to replace and install the In the third embodiment, an example will be described in which the color tone of the display object is adjusted in accordance with the type of the light control plate 20A attached to the image display unit 20.

  FIG. 11 is a flowchart showing the operation of the HMD 100 in the third embodiment. The operation of FIG. 11 is executed in step S12 of FIG. 5A, and corresponds to another example of the operations of FIG. 6, FIG. 7, and FIG.

  The control unit 110 acquires the color tone of the external light from the detection result of the external light sensor 68 received in step S14 (FIG. 5) (step S211). The color tone of the external light is the same as the color tone acquired in step S101 in FIG. The control unit 110 generates or acquires a correction parameter corresponding to the color tone of outside light (step S212).

The control unit 110 acquires or determines the type of the light control plate 20A attached to the image display unit 20 (step S213). In step S213, when the light control plate 20A is not attached, it is determined that it is not attached.
For example, when data indicating the mounting state of the light control plate 20A, that is, the type of the light control plate 20A that is mounted is stored in the storage unit 122, the control unit 110 is based on the data stored in the storage unit 122. The type of the light control plate 20A is acquired. This data is input by operating the control device 10 by the user, for example. Further, a detection mechanism for detecting the type of the light control plate 20A in the image display unit 20 may be provided. For example, the dimming plate 20A is provided with a structure (for example, a protrusion or a notch) corresponding to the type of the dimming plate 20A, and the frame 2 has a detecting unit that mechanically contacts the structure of the dimming plate 20A ( A configuration provided with a not shown) can be employed. In this case, if the detection unit detects contact / non-contact of the structure of the light control plate 20A, the control unit 110 can determine the type of the light control plate 20A based on the detection result.

The control unit 110 acquires a correction parameter corresponding to the type of the light control plate 20A (step S214). In the third embodiment, correction parameters corresponding to the type of light control plate 20A are stored in the storage unit 122 in advance. The control unit 110 generates a correction parameter to be applied to the image data based on the correction parameter acquired in step S214 and the correction parameter generated or acquired in step S212 (step S215). The correction parameter generated by the control unit 110 in step S215 is a correction parameter that takes into account the change in the color tone of the external light transmitted through the light control plate 20A with respect to the color tone of the external light detected by the external light sensor 68. is there. Thereby, the control part 110 can perform correction | amendment corresponding to the color tone of the external light which permeate | transmitted the light control board 20A.
The control unit 110 performs a process of correcting the image data of the display object using the correction parameter generated in step S215 (step S216).

According to the third embodiment, in the configuration in which the external light that does not pass through the light control plate 20A is detected by the external light sensor 68, when the color tone of the external light changes by transmitting through the light control plate 20A, The color tone of the display object can be corrected in consideration of changes. For this reason, the visibility of a display object can be adjusted corresponding to the influence of the external light which actually injects into a user's eyes.
As a method of obtaining the same effect as that of the third embodiment, the external light sensor 68 can receive external light that passes through the light control plate 20A. In this case, the light receiving surface or light receiving portion of the external light sensor 68 may be arranged at a position covered by the light control plate 20A.

[Fourth Embodiment]
FIG. 12 is a block diagram showing a configuration of the fourth embodiment to which the present invention is applied.
In the fourth embodiment, in the HMD 100, the sub-control unit 150 provided in the image display unit 20 includes a processing unit 114 that corrects image data based on the detection result of the external light sensor 68. That is, the CPU (not shown) constituting the sub control unit 150 executes the function of the processing unit 114 by executing a program stored in the ROM (not shown) of the sub control unit 150 or the storage unit 122.

  In this case, the control unit 110 does not execute the operations shown in steps S13 to S15 in FIG. 5A, and the sub-control unit 150 receives external light from the processing unit 114 without receiving a command from the control unit 110. Detection by the sensor 68 is executed. Further, the sub control unit 150 causes the processing unit 114 to execute the image processing shown in step S15 of FIG. As this image processing, any of the processes described with reference to FIGS. 6, 7, 8, 10, and 11 can be executed.

According to this configuration, the same effects as those of the first to third embodiments described above can be obtained. In addition, the sub control unit 150 can be configured to execute the functions of the imaging control unit 111 and the display control unit 113 of the control unit 110 by a program.
Further, all or some of the functions of the control unit 110 can be configured to be executed by the sub-control unit 150. In addition to the control unit 110, the memory 121, the storage unit 122, the power supply control unit 123, the UI control unit 124, the wireless I / F control unit 125, the audio control unit 126, the sensor IC 127, and the external I / F unit 128 are included. Alternatively, the image display unit 20 may be mounted. Further, only the UI control unit 124, the sensor IC 127, and the external I / F unit 128 may be provided in the control device 10, and other functional units may be provided in the image display unit 20. Further, the control device 10 and the image display unit 20 may be connected by a wireless communication line.

In addition, this invention is not restricted to the structure of said each embodiment, In the range which does not deviate from the summary, it can be implemented in a various aspect.
For example, in the above embodiments, the spectacle-type frame 2 is illustrated as an example of the main body, but the shape of the main body is not limited to the spectacle-type. What is necessary is just to provide the display part which mounts or fixes to a user's head and displays an image corresponding to a user's left eye and right eye. For example, in addition to the above-described glasses type, a snow goggle-like shape that covers the upper part of the user's face may be used, or a shape that is arranged in front of each of the left and right eyes of the user like binoculars. There may be.
Further, for example, instead of the image display unit 20, another type of image display unit such as an image display unit worn like a hat may be employed. The display device of the present invention may be configured as a head mounted display mounted on a vehicle such as an automobile or an airplane. Further, for example, it may be configured as a head-mounted display built in a body protective device such as a helmet.

As the control device 10, a notebook computer, a tablet computer, or a desktop computer may be used. Alternatively, a game machine, a portable phone, a portable electronic device including a smartphone or a portable media player, other dedicated devices, or the like may be used as the control device 10.
For example, as a configuration for generating image light in the image display unit 20, a configuration including an organic EL (Organic Electro-Luminescence) display and an organic EL control unit may be used. Further, as a configuration for generating image light, LCOS (Liquid crystal on silicon, LCoS is a registered trademark), a digital micromirror device, or the like can be used.

The optical system that guides the image light to the user's eyes may include an optical member that transmits external light that is incident on the apparatus from the outside and that is incident on the user's eyes together with the image light. Moreover, you may use the optical member which is located ahead of a user's eyes and overlaps a part or all of a user's visual field. Further, a scanning optical system that scans a laser beam or the like to obtain image light may be employed. Further, the optical member is not limited to guiding the image light inside the optical member, and may have only a function of guiding the image light by refracting and / or reflecting it toward the user's eyes.
For example, the present invention can also be applied to a laser retinal projection type head mounted display. That is, the light emitting unit includes a laser light source and an optical system that guides the laser light source to the user's eyes, and the laser light is incident on the user's eyes to scan the retina and form an image on the retina. A configuration that allows the user to visually recognize an image may be employed.
Further, the present invention can be applied to a display device that employs a scanning optical system using a MEMS mirror and uses MEMS display technology. That is, a light emitting unit includes a signal light forming unit, a scanning optical system having a MEMS mirror that scans light emitted from the signal light forming unit, and an optical member on which a virtual image is formed by light scanned by the scanning optical system. You may prepare. In this configuration, the light emitted from the signal light forming unit is reflected by the MEMS mirror, enters the optical member, is guided through the optical member, and reaches the virtual image forming surface. When the MEMS mirror scans the light, a virtual image is formed on the virtual image forming surface, and the user recognizes the virtual image with the eyes, thereby recognizing the image. The optical component in this case may be one that guides light through a plurality of reflections, such as the right light guide plate 261 and the left light guide plate 262 of the above embodiment, and may use a half mirror surface. .

  Further, at least a part of each functional block shown in FIG. 3 may be realized by hardware, or may be realized by cooperation of hardware and software, as shown in FIG. It is not limited to a configuration in which independent hardware resources are arranged. Further, each functional unit shown in FIG. 3 is not limited to an example constituted by a microprocessor and an IC, and a configuration in which a plurality of functional units are mounted on a larger scale integrated circuit may be adopted. -a-chip) or the like. Further, the configuration formed in the control device 10 may be formed in the image display unit 20 in an overlapping manner.

  DESCRIPTION OF SYMBOLS 10 ... Control apparatus, 20 ... Image display part (display part), 20A ... Light control board, 22 ... Right display drive part, 24 ... Left display drive part, 40 ... Connection part, 61 ... Camera, 63 ... Microphone, 68 ... External light sensor (light detection unit), 100 ... head-mounted display device (display device), 110 ... control unit, 111 ... imaging control unit, 112 ... image processing unit, 113 ... display control unit, 114 ... processing unit, 121 ... Memory, 122 ... Flash memory, 122a ... Content data, 122b ... Setting data, 125 ... Wireless I / F control unit, 127 ... Sensor IC, 128 ... External I / F unit, 150 ... Sub control unit, 155 ... I / F section, 162 ... 9-axis sensor, 163 ... GPS, 165 ... EEPROM, 167, 168 ... LCD drive section, 169, 170 ... backlight drive section, 221 ... right backlight, 222 Left backlight, 241 ... right liquid crystal display, 242 ... left a liquid crystal display, 251 ... right projection optical system, 252 ... left projection optical system.

Claims (12)

A display unit that displays image by outputting image light including a plurality of color lights to a user;
A light detection unit for detecting light;
A control unit that controls the color tone of the image light output from the display unit based on the detection result of the light detection unit;
The light detection unit includes a sensor that detects the intensity of received light for each of a plurality of different wavelengths.
Each of the color lights constituting the image light is configured so that the control unit enhances the visibility for the user to visually recognize the image light based on the intensity of light for each wavelength detected by the light detection unit. A first process for controlling the brightness of
Based on the light intensity for each wavelength detected by the light detection unit, the color light of each of the color lights constituting the image light is made closer to the color tone of the light detected by the light detection unit. A second process for controlling the brightness, and configured to be executable.
Selecting and executing either the first process or the second process according to an attribute of an image displayed on the display unit;
A display device. The light detection unit detects light from a direction different from the image light output from the display unit;
The display device according to claim 1. The display unit is mounted on the user's head, and has a configuration for outputting the image light in a state in which external light can be incident on the user's visual field.
The light detector detects the external light incident on the user's field of view;
The display device according to claim 2. A display unit that displays image by outputting image light including a plurality of color lights to a user;
A light detection unit that is provided adjacent to the display unit and detects light from the front of the user's face;
A control unit that controls the color tone of the image light output from the display unit based on the detection result of the light detection unit;
The light detection unit includes a sensor that detects the intensity of received light for each of a plurality of different wavelengths.
Each of the color lights constituting the image light is configured so that the control unit enhances the visibility for the user to visually recognize the image light based on the intensity of light for each wavelength detected by the light detection unit. A first process for controlling the brightness of
Based on the light intensity for each wavelength detected by the light detection unit, the color light of each of the color lights constituting the image light is made closer to the color tone of the light detected by the light detection unit. A second process for controlling the brightness, and configured to be executable.
Selecting and executing either the first process or the second process according to an attribute of an image displayed on the display unit;
A display device. The control unit causes the display unit to output the color light constituting the image based on image data including a plurality of color data,
Changing the tone of the image light by changing the tone value of the color data included in the image data;
The display device according to claim 1, wherein: When the control unit executes the first process, the ratio of the luminances of the color lights included in the image light when the intensity of light detected by the light detection unit is lower than a preset intensity. To a preset ratio,
Display device according to claim 1, wherein the 5. When the intensity of light detected by the light detection unit is lower than a preset intensity, the control unit changes a luminance ratio of each color light included in the image light to a preset ratio. about,
The display device according to claim 1, wherein: The light detection unit is capable of switching between a first detection state and a second detection state suitable for detecting light having a lower intensity than the first detection state,
The control unit, when switching the light detection unit to the second detection state, to change the luminance ratio of each color light included in the image light to a preset ratio,
The display device according to claim 7 . The control unit performs contrast correction processing and / or edge correction processing on image data of an image displayed by the display unit when the intensity of light detected by the light detection unit is lower than a preset intensity. To do,
Display device according to claim 6, wherein 8. The display unit includes a display area located within the visual field of the user,
The control unit divides the display area into a plurality of parts and controls the color tone of the image light for each part based on a detection result of detecting the external light incident on the user's visual field by the light detection part. about,
The display device according to claim 1, wherein: The light detection unit is capable of detecting the intensity of each of light incident on the user's visual field from a plurality of directions;
The display device according to claim 10 . The display unit that outputs image light including a plurality of color lights to a user to display an image, and a sensor that detects the intensity of light received at a plurality of different wavelengths, and the display unit outputs the image A method of controlling a display device comprising: a light detection unit that detects light incident on the user's visual field from a direction different from light;
Based on the detection result of the light detection unit, the color tone of the image light output from the display unit is controlled, and based on the intensity of light for each wavelength detected by the light detection unit, the user The image is based on the first processing for controlling the luminance of each of the color lights constituting the image light and the light intensity for each wavelength detected by the light detection unit so as to improve the visibility for visually recognizing the image. One of the second processing for controlling the luminance of each of the color lights constituting the image light is displayed on the display unit so that the color tone of the light approaches the color tone of the light detected by the light detection unit. Select and execute according to image attributes,
A control method of a display device characterized by the above.

Images