JP2013114123A - Transmission type display device, display method and display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a transmission type display device capable of corresponding to actual conditions of eyes of a user, and further to provide a display method and a display program.SOLUTION: A transmission type display device comprises: displays 110A and 110B that transmit light from outside through image taking out areas and output image light to the image taking out areas; sensors 130A and 130B for detecting conditions of eyes of a user; an operation input section 270 for accepting an operation input from the user; and a controller 230 that, in a case where the conditions of eyes of the user is determined to become predetermined conditions on the basis of the conditions of eyes of the user detected by the sensors 130A and 130B, displays predetermined images by the displays 110A and 110B and, in a case where acceptance of a predetermined operation input in response thereto from the user by the operation input section 270 is detected, switches display modes of the displays 110A and 110B.

Description

  The present invention relates to a transmissive (see-through) display device, a display method, and a display program.

  Patent Document 1 discloses a head-mounted display device (HMD: head mounted display) in which an image display element including a liquid crystal panel and a backlight (BL) for irradiating the liquid crystal panel from behind is housed in the device body. A method of driving the backlight is disclosed (see Patent Document 1).

Japanese Patent No. 3635825

  However, the backlight driving method of the head-mounted display device disclosed in Patent Document 1 requires further development in that it corresponds to the fatigue of the user's actual eyes (eyes). .

  The present invention has been made in view of the above points, and an object of the present invention is to provide a transmissive display device, a display method, and a display program that can cope with the actual eye condition of the user.

  The present invention has been made in order to solve the above-described problem, and transmits a light from the outside to the image extraction area and outputs the image light to the image extraction area, and a user's eye A state detection sensor; an operation input unit that receives an operation input from the user; and the state of the user's eye based on the state of the user's eye detected by the sensor When it is determined, a predetermined image is displayed by the display unit, and when it is detected that a predetermined operation input from the user is received by the operation input unit accordingly, the display unit And a control unit that switches a display mode.

  With this configuration, the control unit displays a predetermined image when it is determined that the state of the user's eye is in a predetermined state, and an operation input from the user determined in accordance therewith is displayed. When it is detected that the display has been accepted, the display mode of the display unit is switched. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, according to the present invention, the control unit determines the number of blinks of the user's eye as the state of the user's eye based on the state of the user's eye detected by the sensor, In one or both of a case where it is determined that the number of eye blinks is equal to or greater than a first predetermined value and a case where it is determined that the number of eye blinks is equal to or less than a second predetermined value, It is determined that the state of the user's eyes is in the predetermined state.

  With this configuration, the control unit determines that the number of blinks of the user's eyes is equal to or greater than a first predetermined value and determines that the number of blinks of the user's eyes is equal to or less than a second predetermined value. In one or both of the above, it is determined that the state of the user's eyes is in a predetermined state, a predetermined image is displayed, and a predetermined operation from the user is performed accordingly When it is detected that an input is accepted, the display mode of the display unit is switched. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, the present invention is the transmissive display device, wherein the control unit switches to a display mode in which an eye resting image is displayed by the display unit as the display mode switching.

  With this configuration, the control unit displays a predetermined image when it is determined that the state of the user's eye is in a predetermined state, and an operation input from the user determined in accordance therewith is displayed. When it is detected that the image has been received, the display mode of the display unit is switched to a display mode in which the display unit displays an eye resting image. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, the present invention is the transmissive display device, wherein the control unit switches to a display mode in which a backlight of the display unit is turned off as switching of the display mode.

  With this configuration, the control unit displays a predetermined image when it is determined that the state of the user's eye is in a predetermined state, and an operation input from the user determined in accordance therewith is displayed. When it is detected that the display has been accepted, the display mode of the display unit is switched to a display mode in which the backlight of the display unit is turned off. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, according to the present invention, the control unit determines the size of a pupil of the user's eye as the state of the user's eye based on the state of the user's eye detected by the sensor, and the user The eye state of the user is determined in advance in one or both of the case where it is determined that the size of the pupil of the eye of the user has decreased and the size of the pupil of the eye of the user has been increased. The transmissive display device is characterized in that it is determined that the state has been reached.

  With this configuration, in one or both of the case where the control unit determines that the size of the pupil of the user's eye has decreased and the case where the size of the pupil of the user's eye has increased, the control unit When it is determined that the user's eye state is in a predetermined state, a predetermined image is displayed, and it is detected that a predetermined operation input from the user has been accepted in response to this Switch the display mode to. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, according to the present invention, when the control unit determines that the size of the pupil of the user's eye is reduced, the control unit displays a predetermined image by the display unit, and the operation input unit accordingly When it is detected that a predetermined operation input from the user has been received, the display mode is switched to a display mode for reducing the brightness of the backlight of the display unit. Type display device.

  With this configuration, the control unit displays a predetermined image by the display unit when it is determined that the size of the pupil of the user's eye is reduced, and the user predetermined by the operation input unit is displayed accordingly. When it is detected that an operation input from is received, the display mode is switched to a lowering of the backlight brightness of the display unit. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  Further, according to the present invention, when the control unit determines that the size of the pupil of the user's eye has increased, the control unit displays a predetermined image by the display unit, and the operation input unit accordingly When it is detected that a predetermined operation input from the user has been accepted, the display mode is switched to a display mode for increasing the transmittance of the shade of the display unit. It is a display device.

  With this configuration, when the control unit determines that the size of the pupil of the user's eye has increased, the control unit displays a predetermined image by the display unit, and accordingly, the user predetermined by the operation input unit is displayed. When it is detected that an operation input from is received, the display mode is switched to a display mode for increasing the transmittance of the shade of the display unit. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  In the present invention, the display unit is provided in a display device mounted on a user's head, and the display unit includes a display element, a projection lens that projects image light from the display element, and the projection. A light guide unit that is fixed with respect to the lens and guides the image light from the projection lens to the image extraction area. The image light image from the display element and the light from the outside through the light guide unit It is a transmissive display device characterized by being capable of simultaneously observing the image.

  With this configuration, the control unit displays a predetermined image when it is determined that the state of the user's eye is in a predetermined state, and an operation input from the user determined in accordance therewith is displayed. When it is detected that the display has been accepted, the display mode of the display unit is switched. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  According to the present invention, the display unit transmits light from the outside to the image extraction area and outputs the image light to the image extraction area, and the control unit detects the eye of the user's eye detected by the sensor. When it is determined that the state of the user's eyes has become a predetermined state based on the state, a predetermined image is displayed by the display unit, and the user predetermined by the operation input unit is displayed accordingly And a step of switching the display mode of the display unit when it is detected that an operation input from is received.

  By this method, the control unit displays a predetermined image when it is determined that the state of the user's eyes has become a predetermined state, and an operation input from the user predetermined according to this is displayed. When it is detected that the display has been accepted, the display mode of the display unit is switched. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  In addition, the present invention provides a procedure in which the display unit transmits light from the outside to the image extraction area and outputs the image light to the image extraction area, and a procedure in which the sensor detects the state of the user's eyes. The operation input unit receives an operation input from the user, and the control unit is in a state where the state of the user's eye is predetermined based on the state of the user's eye detected by the sensor. The display unit displays a predetermined image when the display unit determines that the operation input from the user is received by the operation input unit according to the image. And a display program in a transmissive display device for causing a computer to execute a procedure for switching the display mode.

  With this program, the control unit displays a predetermined image when it is determined that the state of the user's eyes has become a predetermined state, and an operation input from the user predetermined according to this is displayed. When it is detected that the display has been accepted, the display mode of the display unit is switched. Thereby, the transmissive display apparatus can cope with the actual eye condition of the user.

  As described above, according to the present invention, the transmissive display device displays a predetermined image when it is determined that the state of the user's eyes is in a predetermined state, and is determined in advance accordingly. When it is detected that an operation input from the user is received, the display mode of the display unit is switched. Thereby, it can respond to a user's actual eye condition.

1 is an external view of a transmissive display device according to an embodiment of the present invention. It is a top view which shows the structural example of the display part for left eyes of the display apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structural example of the transmissive display apparatus which concerns on one Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process performed in the transmissive display apparatus which concerns on 1st Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process performed in the transmissive display apparatus which concerns on 2nd Embodiment of this invention. It is an external view of the transmissive display apparatus which concerns on other one Embodiment of this invention.

[First Embodiment]
A first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is an external view of a transmissive display device (in this embodiment, a transmissive head mounted display) according to an embodiment of the present invention.
The transmissive display device according to the present embodiment includes a display device 100 having a shape like glasses, and a control device (controller) 200 having a size that can be held by a user's hand. Prepare.

Further, the display device 100 and the control device 200 are connected so as to be communicable by wire or wireless.
In the present embodiment, the left eye side and the right eye side of the display device 100 and the control device 200 are communicably connected via a wired cable. Each of the left eye side and the right eye side of the display device 100 and the control device 200 communicate image signals (image signals) and control signals (control signals) via this cable.

The display device 100 includes a left-eye display unit (left-eye display unit) 110A, a right-eye display unit (right-eye display unit) 110B, an optical sensor 120, and a left-eye sensor (left eye). Sensor) 130A and a right eye sensor (right eye sensor) 130B.
The control device 200 includes an operation unit 210 and an operation button unit 250.

The left-eye display unit 110A includes a left-eye image forming unit (left-eye image forming unit) 10A, a left-eye light guide unit (the left-eye light guide unit 20A illustrated in FIG. 2), and a left-eye display unit 110A. An eye reflecting portion (left eye reflecting portion) 21A and a left eye shade (left eye shade) 420A are provided.
The right-eye display unit 110B includes a right-eye image forming unit (right-eye image forming unit) 10B and a right-eye light guiding unit (the same right as the left-eye light guiding unit 20A illustrated in FIG. 2). An eye light guide 20B), a right eye reflector (right eye reflector) 21B, and a right eye shade (right eye shade) 420B.

FIG. 2 is a plan view illustrating a configuration example of the left-eye display unit 110A of the display device 100 according to an embodiment of the present invention. 2 shows a left eye 310A of a user (person) wearing the display device 100 and a left-eye sensor 130A for the left eye 310A of the user.
The left-eye display unit 110A and the right-eye display unit 110B have a bilaterally symmetric configuration, and only the configuration example of the left-eye display unit 110A will be described here.

The left-eye image forming unit 10A includes a left-eye image generation unit (left-eye image generation unit) 11A and a left-eye projection optical system (left-eye projection optical system) 12A.
The left-eye image generation unit 11A includes a left-eye backlight light source (left-eye backlight light source) 410A and a left-eye light modulation element (left-eye light modulation element) 411A.

In the present embodiment, the left-eye backlight light source 410A is composed of a set of light sources for each emission color such as red, green, and blue. As each light source, for example, a light emitting diode (LED) can be used.
As the left-eye light modulation element 411A, for example, a liquid crystal display device which is a display element can be used.

With reference to FIG. 1 and FIG. 2, the outline of the transmissive display device according to the present embodiment will be described.
An image signal for the left eye is input from the control device 200 to the image generation unit 11A for the left eye.
In the left-eye image generation unit 11A, each light source of the left-eye backlight light source 410A emits red light, green light, and blue light. The left-eye image generation unit 11A diffuses red light, green light, and blue light emitted from each light source of the left-eye backlight light source 410A and projects the diffused light onto the left-eye light modulation element 411A. The left-eye light modulation element 411A spatially modulates the projected red light, green light, and blue light in accordance with the image signal input from the control device 200 to the left-eye image generation unit 11A, thereby generating the image. The image light corresponding to the signal is emitted.
The left-eye projection optical system 12A includes, for example, a group of projection lenses that project input image light, and projects image light emitted from the left-eye light modulation element 411A of the left-eye image generation unit 11A. Thus, the luminous flux is in a parallel state.

  The left-eye light guide unit 20A converts image light that has been converted into a parallel light beam by the left-eye projection optical system 12A into a predetermined surface (semi-transmissive reflection surface) of a triangular prism included in the left-eye reflection unit 21A. ). Here, a reflective coating such as a mirror layer is applied on the side facing the user's left eye 310A when worn, out of the front and back of the transflective surface formed on the left eye reflecting portion 21A. The image light projected on the transflective surface formed on the left-eye reflecting portion 21A is totally reflected toward the user's left eye 310A by this reflectively coated surface. Accordingly, image light corresponding to the projected image light is output to an area (image extraction area) at a predetermined position of the left-eye reflecting unit 21A.

  In the present embodiment, the image light transmitted by the light guide plate of the left-eye light guide unit 20A is reflected by the transflective surface formed on the left-eye reflection unit 21A, and the reflected light is the user's left eye 310A. Then, a virtual image is formed on the retina of the left eye 310A. In the present embodiment, the left-eye light guide 20A is fixed to the left-eye projection optical system 12A and guides the image light from the left-eye projection optical system 12A to a predetermined position.

Further, at least part of the light incident on the left-eye reflecting portion 21A from the outside passes through the above-described semi-transmissive reflecting surface formed on the left-eye reflecting portion 21A and is guided to the user's left eye 310A. Thereby, the user sees the image formed by the left-eye image forming unit 10 </ b> A and the optical image from the outside world superimposed.
The left-eye shade 420A reduces light incident on the left-eye reflecting unit 21A (the user's left eye 310A) from the outside world according to the transmittance.

When the user wears the display device 100 on the head, the image is output from the image extraction area of the display device 100 (image extraction area of the left eye reflection unit 21A, image extraction area of the right eye reflection unit 21B). When an image corresponding to light appears to be displayed, the image can be recognized. Further, the user allows at least part of light from the outside world to pass through the image extraction area of the display device 100 (the image extraction area of the left-eye reflection unit 21A and the image extraction area of the right-eye reflection unit 21B). The outside world can be seen with the display device 100 mounted on the head. That is, light from the outside can pass through the image extraction area (the image extraction area of the left-eye reflection unit 21A and the image extraction area of the right-eye reflection unit 21B).
In this way, the user simultaneously views the image to be displayed by the transmissive display device and the image of the outside world through the light guide unit (the left eye light guide unit 20A and the right eye light guide unit 20B) (observation). And a virtual image can be visually recognized. Further, the user can input an operation to the operation unit 210 and the operation button unit 250 of the control device 200.

Here, the left-eye image forming unit 10A is provided on the left temple of the display device 100 so as to be positioned on the left side of the user's head when worn. The left-eye image forming unit 10A forms an image for the user's left eye, and outputs the light of the formed image toward a predetermined position (position of the image extraction area) of the left-eye reflecting unit 21A. .
The image extraction area of the left-eye reflecting portion 21A is provided in the left rim portion of the display device 100 so as to be positioned in front of the user's left eye 310A when worn.

Further, the right-eye image forming unit 10B is provided in the right temple of the display device 100 so as to be positioned on the right side of the user's head when worn. The right-eye image forming unit 10B forms an image for the user's right eye, and outputs the light of the formed image toward a predetermined position (image extraction area position) of the right-eye reflecting unit 21B. .
The image extraction area of the right-eye reflection unit 21B is provided on the right rim of the display device 100 so as to be positioned in front of the user's right eye when worn.

Hereinafter, the transmissive display device according to the present embodiment will be described in more detail.
FIG. 3 is a block diagram illustrating a configuration example of the transmissive display device according to the present embodiment.
In addition to the operation unit 210 and the operation button unit 250, the control device 200 includes a control unit 230, a storage unit 240, and a power supply unit 260.
In the control device 200, one or both of the operation unit 210 and the operation button unit 250 constitute an operation input unit 270 that receives an operation input from the user.

Each component in the transmissive display device according to the present embodiment will be described.
First, the display device 100 will be described.
Provided in the left-eye image forming unit 10A, the left-eye light guiding unit 20A, the left-eye reflecting unit 21A, the left-eye shade 420A, and the left-eye image forming unit 10A provided in the left-eye display unit 110A. The left-eye image generation unit 11A, the left-eye projection optical system 12A, the left-eye backlight light source 410A and the left-eye light modulation element 411A included in the left-eye image generation unit 11A are shown in FIGS. It has the configuration described with reference to perform the operation.

  The right eye image forming unit 10B, the right eye light guide unit 20B, the right eye reflecting unit 21B, the right eye shade 420B, and the right eye image forming unit 10B provided in the right eye display unit 110B are also provided. The right-eye image generation unit 11B, the right-eye projection optical system 12B, the right-eye backlight light source 410B and the right-eye light modulation element 411B included in the right-eye image generation unit 11B are for the left eye. It has a configuration similar to that described with reference to FIGS. 1 and 2 and operates.

The optical sensor 120 detects the level of external light (environmental light) and transmits the detection result to the control unit 230 of the control device 200.
Here, as the level of ambient light detected by the optical sensor 120, for example, the brightness level of ambient light can be used.

  In the present embodiment, the optical sensor 120 has a rim portion or a bridge portion (in this embodiment, a bridge in the present embodiment) so that the optical sensor 120 is positioned in front of the user's head when the user wears the display device 100. At the center of the section). In the present embodiment, the optical sensor 120 includes a shade (left eye shade 420A, right eye shade 420B) and an image extraction area (image extraction area of the left eye reflection unit 21A, image of the right eye reflection unit 21B) from the outside. It is arranged so as to detect ambient light in the same (or almost the same) direction as the direction to the extraction area. Thereby, the optical sensor 120 has a shade (left-eye shade 420A, right-eye shade 420B) and an image extraction area (image extraction area of the left-eye reflection unit 21A, image extraction area of the right-eye reflection unit 21B) from the outside. ) To detect the level of ambient light incident on (or a level corresponding thereto).

The left-eye sensor 130 </ b> A detects the state of the user's left eye 310 </ b> A when the user wears the display device 100. The left eye sensor 130 </ b> A transmits the detection result to the control unit 230.
In the present embodiment, a CCD (Charge Coupled Device) camera is used as the left-eye sensor 130A, and the camera is placed at the upper left position with respect to the image extraction area of the left-eye reflection unit 21A. It is arranged in a direction to image 310A (position where it is supposed to come). In the present embodiment, the left-eye sensor 130A captures the user's left eye 310A (position where it is expected to come), and the captured image (for example, a moving image) is detected as a result of detecting the state of the user's left eye 310A. To the control unit 230.

The right eye sensor 130B detects the state of the right eye of the user when the user wears the display device 100. The right eye sensor 130 </ b> B transmits the detection result to the control unit 230.
In the present embodiment, a CCD (Charge Coupled Device) camera is used as the right eye sensor 130B, and this camera is placed at the upper right position with respect to the image extraction area of the right eye reflection unit 21B. (Position that is supposed to come) is arranged in the direction of imaging. In the present embodiment, the right-eye sensor 130B captures the user's right eye (position assumed to come) and controls the captured image (for example, a moving image) as a detection result of the state of the user's right eye. To the unit 230.

Next, the control device 200 will be described.
The operation unit 210 detects an operation input to the operation surface provided in the casing of the control device 200 and outputs the detection result to the control unit 230.
Here, the operation unit 210 detects, for example, the position on the operation surface touched by the finger of the user who performs the operation input, using absolute coordinates on the operation surface. The operation unit 210 is, for example, a track pad.

  The operation button unit 250 includes a plurality of operation buttons. When each operation button is pressed by the user, a signal indicating a process associated with the pressed operation button in advance is output to the control unit 230.

The storage unit 240 stores various types of information. Specifically, the storage unit 240 stores, for example, information on a program executed by the control unit 230 and information used in various processes.
The power supply unit 260 supplies power to each component that requires the power supply provided in the control device 200 and the display device 100.

The control unit 230 executes various processes in the control device 200. The control unit 230 is configured using, for example, a CPU (Central Processing Unit).
Specifically, for example, when the operation input detection result is input from the operation unit 210, the control unit 230 has a function of executing processing according to the detection result, and an operation button pressed in the operation button unit 250. A function of executing a process corresponding to the signal when a signal indicating a previously associated process is input; a function of writing information in the storage unit 240; a function of reading out information stored in the storage unit 240; Has a function of controlling the supply state of power to each component.

  In addition, the control unit 230 has, for example, a function of acquiring a detection result of the level of ambient light input from the optical sensor 120, a detection result of the state of the user's left eye 310A input from the left-eye sensor 130A (this embodiment) In the embodiment, a function of acquiring an image of the user's left eye 310A), a detection result of the state of the user's right eye input from the right eye sensor 130B (in this embodiment, an image of the user's right eye imaged) ).

  Further, the control unit 230 determines the user's left eye based on the detection result of the state of the user's left eye 310A input from the left eye sensor 130A (in this embodiment, an image obtained by imaging the user's left eye 310A). Based on the function of determining the state of the eye 310A and the detection result of the state of the right eye of the user input from the right-eye sensor 130B (in this embodiment, an image obtained by capturing the right eye of the user) It has a function of determining the state of the eye.

In the present embodiment, the control unit 230 detects (determines) the number of blinks of the user's eyes as the state of the user's eyes based on an image obtained by imaging the user's eyes (right eye, left eye). It has a function. Such detection of the number of blinks may be performed by arbitrary image processing, and can be realized using, for example, known image processing.
Normally, in the human eye, the number of blinks is more than usual when the eyes are tired, and the number of blinks is less than usual when the eyes are in a dry eye state.

  In addition, the control unit 230 has, for example, a function of controlling the luminance (backlight luminance) of light emitted from each light source of the left-eye backlight light source 410A, and light emitted from each light source of the right-eye backlight light source 410B. For controlling the brightness of the eye (backlight brightness), the function for controlling the transmittance of the left-eye shade 420A, the function for controlling the transmittance of the right-eye shade 420B, and the gamma (γ of the image displayed for the left eye ) A function for controlling a value and a function for controlling a gamma (γ) value of an image displayed for the right eye.

Further, for example, the control unit 230 outputs a left-eye image signal to the left-eye image generation unit 11A for the image to be displayed to the user by the display device 100, and the right-eye image generation unit 11B It has a function of outputting an image signal for the eye.
Here, the image signal for the left eye and the image signal for the right eye that are simultaneously displayed to the user may be, for example, the same image signal or different image signals depending on the parallax of the image. It may be.

In addition, as the image displayed by the display device 100, images having various contents may be used.
Specifically, as an image displayed by the display device 100, for example, an image of a moving image content such as a movie or an image (warning image) indicating a warning for a predetermined user can be used. The warning image can be displayed on an on-screen display (OSD) with respect to the content image, for example.

The control unit 230 has a function of switching a plurality of display modes (display modes), for example.
Here, various display modes may be used.
Specifically, as a plurality of display modes, for example, the luminance of the backlight light source (left eye backlight light source 410A, right eye backlight light source 410B), shade (left eye shade 420A, right eye shade) 420B), the displayed image (left eye image, right eye image), the size of the displayed image (left eye image, right eye image), and the displayed image (left eye image). Image, right eye image) background color, display image (left eye image, right eye image) color, display image (left eye image, right eye image) Modes in which one or more aspects of the gamma value or the like are different can be used.

  In the present embodiment, the control unit 230 detects the state of the user's eyes, and based on the detection result, determines that the detected state of the user's eyes has become a predetermined state that is considered to be fatigued, for example. In such a case, a predetermined warning image is displayed to the user, and when a predetermined operation input is received from the user, the display mode is switched as predetermined.

An example of an operation performed in the transmissive display device according to the present embodiment will be described.
FIG. 4 is a flowchart illustrating an example of a procedure of processing performed in the transmissive display device according to the first embodiment of the present invention.
First, when the transmissive display device according to the present embodiment is in the off state, a predetermined operation button or the like provided in the operation button unit 250 of the control device 200 of the transmissive display device is operated by the user. Then, the transmissive display device is turned on, and power is supplied to each component from the power supply unit 260 of the control device 200. Thereby, use of the transmissive display device is started (step S1).

Here, it is assumed that the display device 100 of the transmissive display device according to the present embodiment is mounted by the user.
The control unit 230 includes information input from the left eye sensor 130A (in this embodiment, an image of the user's left eye 310A) and information input from the right eye sensor 130B (in this embodiment, the user The number of blinks of the user's eyes in a predetermined period (for example, 1 minute) is detected based on the image of the right eye) and the number of blinks detected (for example, 20 times / minute) Is stored in the storage unit 240 as an initial value (step S2).

Here, as the timing of the start point for detecting the initial value of the number of blinks of the user, for example, any predetermined timing can be used.
Specifically, as the timing of the starting point for detecting the initial value of the number of blinks of the user, for example, the control unit 230 detects that the user wears the display device 100 (for example, between the left eye and the right eye). The timing at which the control unit 230 detects that an image of the user's eyes has been acquired for one or both of the timings, or the timing at which a predetermined time has elapsed from this point, or the control The timing at which the control unit 230 detects that a predetermined operation button provided in the operation button unit 250 of the apparatus 200 is operated by the user can be used.

  For example, the control unit 230 may re-detect the initial value of the number of blinks of the user every predetermined period or every predetermined number of detections. In this case, for example, every time the initial value of the number of blinks of the user is detected again, the process may return to step S3.

Further, the state of the user's eyes (in this embodiment, the number of blinks) may be, for example, detecting the state of only one of the user's left eye and right eye, or the user's left eye and right Both eye states may be detected. In the present embodiment, the state of both the left eye and the right eye of the user is detected as the state of the user's eye.
When detecting the state of both the left eye and the right eye of the user as the state of the user's eye, for example, for the result detected for the left eye and the result detected for the right eye, the one with the larger value is selected. A configuration to be employed, a configuration to employ a smaller value, a configuration to employ an average value of the values, or the like can be used.

Moreover, in this embodiment, the control part 230 sets the 1st threshold value-the 8th threshold value according to the initial value of the number of blinks, and memorize | stores it in the memory | storage part 240, and memorize | stores it as needed after that. The first to eighth threshold values stored in the unit 240 are read and used.
In the present embodiment, the first threshold> the second threshold> the third threshold> the fourth threshold> the initial value> the fifth threshold> the sixth threshold> the seventh threshold> the eighth threshold. In addition, as previously determined, the control unit 230 sets each threshold value. As a method for obtaining each threshold value from the initial value, for example, a method in which a result obtained by performing a predetermined calculation on the initial value is used as a threshold value, or a correspondence between the initial value and each threshold value is determined in advance. Based on the above, a method for obtaining each threshold corresponding to the initial value can be used.

As an example, when the initial value = 20 times / minute, the first threshold = 100 times / minute, the second threshold = 80 times / minute, the third threshold = 60 times / minute, the fourth threshold = 30 times / minute, 5th threshold = 10 times / minute, 6th threshold = 7 times / minute, 7th threshold = 5 times / minute, 8th threshold = 3 times / minute .
Various values may be used as the respective threshold values (first threshold value to eighth threshold value).

  Thereafter, the control unit 230 receives information input from the left-eye sensor 130A (in this embodiment, an image obtained by imaging the user's left eye 310A) and the right-eye sensor 130B at a predetermined timing. The number of blinks of the user in a predetermined period (for example, 1 minute) is detected based on the information (in this embodiment, an image obtained by capturing the right eye of the user). A predetermined determination is made (step S3).

  Here, as the predetermined timing at which the control unit 230 detects the number of blinks of the user, an arbitrary timing may be used. For example, a timing with a certain period (for example, 10 minutes) is used. Can do.

  Then, as a determination in the process of step S3, the control unit 230 determines whether the detected number of blinks of the user is equal to or more than a fourth threshold value (for example, 30 times / minute), or a fifth threshold value (an example). Or less), or a value that is less than the fourth threshold value and exceeds the fifth threshold value (whether it is regarded as an initial value).

If the result of determination in step S3 is that the detected number of blinks of the user is greater than or equal to the fourth threshold, the control unit 230 considers that the user's eyes are in a fatigued state. To call attention (step S4).
In addition, as a result of the determination in the process of step S3, when the control unit 230 determines that the detected number of blinks of the user is less than or equal to the fifth threshold, the user's eyes are at risk of dry eye (Step S13) and call attention (step S4).

  Here, as an alert in the process of step S4, for example, the control unit 230 has an image extraction area of the display device 100 (an image extraction area of the left eye reflection unit 21A, an image extraction area of the right eye reflection unit 21B). In addition, the light of the content image for alerting the user is output and the image is displayed. As this image, for example, an image including a text message such as “Let's rest your eyes” can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.

  As a result of the determination in the process of step S3, when it is determined that the number of blinks of the detected user is a value that is less than the fourth threshold and exceeds the fifth threshold, the control unit 230 performs normal operation. (Step S12), the process proceeds to Step S3 (Step S3).

  After the process of step S4, the control unit 230 receives information (in the present embodiment, information input from the left eye sensor 130A at a predetermined timing (for example, the next timing in the timing of the fixed period described above)). An image obtained by imaging the user's left eye 310A) and information input from the right-eye sensor 130B (in this embodiment, an image obtained by imaging the user's right eye) (for example, 1 The number of blinks of the user in minutes etc. is detected, and a predetermined determination is made with respect to the number of blinks detected (step S5).

  Then, the control unit 230 determines whether or not the number of blinks of the detected user is equal to or more than a third threshold value (as an example, 60 times / minute) or a sixth threshold value (an example) as a determination in the process of step S5. Or less) or less than the third threshold value and exceeding the sixth threshold value (whether it is about the initial value).

As a result of the determination in step S5, when it is determined that the number of blinks of the detected user is greater than or equal to the third threshold, the control unit 230 regards that the user's eyes are more fatigued. Then, it is determined to urge the user to rest (step S6), and attention is drawn to urge the user to rest (step S7).
Further, as a result of the determination in the process of step S5, when the control unit 230 determines that the detected number of blinks of the user is equal to or less than the sixth threshold, the user's eyes are considered to be in a dry eye state. (Step S14), it is determined to urge the user to rest (Step S6), and attention is drawn to urge the user to rest (Step S7).

  As a result of the determination in the process of step S5, when the control unit 230 determines that the detected number of blinks of the user is less than the third threshold and exceeds the sixth threshold, the normal operation is performed. (Step S12), the process proceeds to Step S3 (Step S3).

  Here, as an alert in the process of step S7, for example, the control unit 230 has an image extraction area of the display device 100 (an image extraction area of the left eye reflection unit 21A, an image extraction area of the right eye reflection unit 21B). In addition, the light of the content image for alerting the user is output and the image is displayed. As this image, for example, an image (warning image) including a text message such as “Do you want to rest?” Can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.

  Further, for example, in the alert in the process of step S7, the control unit 230 further controls the image extraction area of the display device 100 (image extraction area of the left eye reflection unit 21A, image extraction area of the right eye reflection unit 21B). In addition, it is also possible to display the image by outputting light of the image including the explanation information effective for reducing eye fatigue or dry eye. In this case, the user can be prompted to take a rest actively.

Moreover, the control part 230 waits for the operation input from a user, after performing alerting in the process of step S7.
The user operates the operation unit 210 or the operation button unit 250 of the control device 200 in response to the display of an image (warning image) including a text message such as “Would you like to rest?” And “YES (Yes: Rest). ”) Or“ NO (No: do not rest) ”.

After performing the alert in the process of step S7, when the control unit 230 detects that an operation input indicating “YES (yes: rest)” is performed by the user, The display mode is switched (step S8).
As a method of switching the display mode, for example, a predetermined eye-restoration image is displayed from a display mode in which a predetermined eye-restoration image is not displayed (for example, a normal image is displayed). From the display mode for displaying the image, or from the display mode for displaying a normal image (video), the display of the image is temporarily stopped and the backlight (backlight light source 410A for the left eye, right eye backlight) is displayed. A method of switching to a display mode in which the light source 410B) is turned off can be used.

  As another configuration example, a configuration in which a user can select and instruct a desired display mode can be employed. In this configuration, for example, the user operates the operation unit 210 or the operation button unit 250 of the control device 200 to select and instruct a desired display mode, and the control unit 230 enters the display mode instructed according to the instruction. Switch. In this case, the control unit 230 may cause the user to display information indicating one or more display modes that can be switched (for example, a list of display modes that can be switched). A desired display mode is selected and instructed by looking at the contents of the display.

  Here, in the present embodiment, as the display mode after switching, a display mode that is considered to be able to relieve the user's eye fatigue and dry eye state compared to the display mode before switching.

On the other hand, after performing the alert in the process of step S7, the control unit 230 determines in advance if it is detected that an operation input indicating "NO (No: No rest)" is performed by the user. As described above, the frequency of detecting the state of the user's eyes (in this embodiment, the number of blinks) is increased (step S15).
Here, as a method of increasing the frequency of detecting the state of the user's eyes (in this embodiment, the number of blinks), for example, a method of shortening a constant cycle for detecting the state of the user's eyes (for example, changing the cycle) A method such as changing from 10 minutes to 5 minutes) can be used.

  Then, as a determination in the process of step S15, the control unit 230 determines that the detected number of blinks of the user is greater than or equal to the second threshold (for example, 80 times / minute) or the seventh threshold (for example, 5 times / minute). Min) or less than or equal to or greater than or equal to the first threshold value (as an example, 100 times / minute) or less than the eighth threshold value (as an example, 3 times / minute). Determine whether.

  If the result of determination in step S15 is that the controller 230 determines that the number of blinks of the detected user is one of the second threshold and the seventh threshold, the detected user If the number of blinks is equal to or greater than the second threshold, the user's eyes are considered to be more fatigued. If the detected number of blinks is equal to or less than the seventh threshold, the user's eyes are further dried. Assuming that the eye is in the state, it is determined to urge the user to rest, and the process proceeds to step S6 (step S6).

  In addition, as a result of the determination in the process of step S15, the control unit 230 detects when the number of blinks of the detected user is either the first threshold value or the eighth threshold value or less. If the number of blinks of the user is greater than or equal to the first threshold, it is considered that the user's eyes are very tired, and if the number of blinks of the detected user is less than or equal to the eighth threshold, the user's eyes Is determined to be in a very dry eye state, it is determined to forcibly switch the display mode regardless of the user's intention, and the process proceeds to step S8 (step S8).

After switching the display mode in the process of step S8, the control unit 230 determines whether or not the user has a subjective symptom of eye improvement according to an operation input from the user (step S9).
Specifically, in the process of step S9, the control unit 230 provides the user with an image extraction area of the display device 100 (an image extraction area of the left-eye reflection unit 21A and an image extraction area of the right-eye reflection unit 21B). On the other hand, the light of the image of the content which asks whether there exists a subjective symptom of the eye improvement condition is output, and the said image is displayed. As this image, for example, an image including a text message such as “Did you get tired of eyes?” Can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.

The control unit 230 waits for an operation input from the user after performing the image display in the process of step S9.
The user operates the operation unit 210 or the operation button unit 250 of the control device 200 in response to the display of the image including the text message such as “Did you get tired of eyes”? The operation input indicating "I was able to take it off" or "NO (No: I can't get tired)" is performed.

  After performing the image display in the process of step S9, the control unit 230 performs normal operation when it is detected that an operation input indicating “YES (yes: tired)” is performed by the user. It is decided to perform, and the process proceeds to step S12 (step S12).

On the other hand, after performing the image display in the process of step S9, the control unit 230, when detecting that an operation input indicating “NO (No: can not get tired)” is performed by the user, In the image extraction area of the display device 100 (the image extraction area of the left-eye reflection unit 21A and the image extraction area of the right-eye reflection unit 21B) of the display device 100. The light of the image of the content for warning that it transfers to a state is output, and the said image is displayed (step S10). As this image, for example, an image including a text message such as “enter system standby” can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.
Then, the control unit 230 shifts to a system standby state (step S11).

  Here, in the example of the flowchart shown in FIG. 4, alerting (for example, warning display) or switching of the display mode is performed both when the number of blinks of the user's eyes is larger than a predetermined value and when the number of blinks is smaller than the predetermined value. However, as another example, alerting (for example, warning display) or switching of the display mode only when the number of blinks of the user's eyes is greater than a predetermined value or less than a predetermined value It can also be set as the structure which performs.

  As described above, in the transmissive display device according to the present embodiment, the control unit 230 detects the state of the user's eyes (in this embodiment, the number of blinks), and detects the detected user based on the detection result. When it is determined that the eye condition has become a predetermined state that is considered to be, for example, fatigued, a predetermined warning image is displayed to the user, and the user determines a predetermined warning image accordingly. When an operation input is accepted, the display mode is switched as determined in advance.

  Thereby, in the transmission type display device according to the present embodiment, the user's actual eye condition (in this embodiment, the condition of fatigue or the condition of dry eye estimated based on the number of blinks of the user's eyes). The display mode can be switched according to the actual eye condition of the user.

  According to the transmissive display device according to the present embodiment, the degree of fatigued eyes or dry eyes can be estimated by detecting the number of blinks as monitoring of the user's eye condition compared to the initial use. To do. Usually, the number of blinks per unit time differs between the number of blinks in the initial stage and the number of blinks when the display apparatus 100 becomes tired or dry eyes are used. When the number of blinks detected during the time exceeds a predetermined number (for example, frequency) or falls below a predetermined number (for example, frequency), the user is alerted (for example, a warning image is displayed). indicate). When the user desires eye rest, the display mode is switched to realize the user's eye rest.

  In the present embodiment, the control unit 230 performs switching to the eye-restoration image or stops the image and turns off the backlight (turns off) as the display mode for realizing the rest of the user's eyes. . Thereby, the health of a user's eye at the time of continuous use of the display apparatus 100 can be protected, and the discomfort of a user's eye can be reduced.

  As a specific example, when displaying an image of a natural image as a recreation image of the eye, especially by displaying an image such as a distant mountain range or natural green, the focal position of the user's eye can be changed and It is possible to improve eye fatigue by making gentle colors appear.

  As another example, as the display mode switching, the control unit 230 changes the luminance of the backlight, changes the transmittance of the shade, changes the image to be displayed, and changes the size of the image to be displayed. One or more of the following: changing the background color of the displayed image, changing the color of the displayed image, changing the gamma value of the displayed image, and the like.

  As a specific example, an image to be displayed is made smaller, an eye escape area (for example, a place where the distance can be seen) is provided at the corner of the image extraction area of the reflecting portions 21A and 21B, and the color of the image to be displayed is changed from color. Changing to black and white, changing the color of the displayed image from 8-bit display to 2-bit display, changing the background color of the displayed image closer to green, changing the gamma value of the displayed image to green Or a state in which the image light is output and displayed for both eyes of the image extraction area of the left-eye reflection unit 21A and the image extraction area of the right-eye reflection unit 21B ( The burden on the user's eyes is reduced by switching from a binocular display state to a display mode that displays an image for only one eye (single-eye display state). Rukoto can.

In the past, closed-type HMDs (head-mounted displays) have been the mainstream, and users watch images (videos) over a long period of time in a dark state, which may cause effects such as eye fatigue and dry eyes. There was sex.
On the other hand, in the transmissive display device according to the present embodiment, in the see-through type HMD, it is possible to reduce the effects of eye fatigue, dry eyes, etc. due to long-term use, and further, the user can view eye-care images. As a result, it is possible to reduce or improve eye fatigue and dry eye of the user. Thereby, the user can use the transmissive display device more comfortably.

[Second Embodiment]
A second embodiment of the present invention will be described in detail with reference to the drawings.
The transmissive display device according to the present embodiment schematically has the same configuration as that of the transmissive display device according to the first embodiment shown in FIGS. 1, 2, and 3. For this reason, in this embodiment, it demonstrates using the code | symbol same as what was shown in FIGS. 1-3.
In the following, the configuration and operation different from the case of the first embodiment will be mainly described in detail.

In the present embodiment, the control unit 230 detects (determines) the size of the user's pupil as the state of the user's eye based on an image obtained by imaging the user's eye (left eye, right eye). Have Such detection of the size of the pupil may be performed by arbitrary image processing, and can be realized using, for example, known image processing.
Normally, in the human eye, the pupil size is large when viewing a dark object, and the pupil size is small when viewing a bright object.

An example of an operation performed in the transmissive display device according to the present embodiment will be described.
FIG. 5 is a flowchart showing an example of a procedure of processing performed in the transmissive display device according to the second embodiment of the present invention.
First, when the transmissive display device according to the present embodiment is in the off state, a predetermined operation button or the like provided in the operation button unit 250 of the control device 200 of the transmissive display device is operated by the user. Then, the transmissive display device is turned on, and power is supplied to each component from the power supply unit 260 of the control device 200. Thereby, use of the transmissive display device is started (step S21).

Here, it is assumed that the display device 100 of the transmissive display device according to the present embodiment is mounted by the user.
The control unit 230 includes information input from the left eye sensor 130A (in this embodiment, an image of the user's left eye 310A) and information input from the right eye sensor 130B (in this embodiment, the user Based on the image of the right eye of the user), the size of the pupil of the user's eye is detected, and the detected size of the pupil is stored in the storage unit 240 as an initial value (step S21). Thereafter, the control unit 230 reads and uses the initial value stored in the storage unit 240 when necessary.

Here, as a timing for detecting the initial value of the pupil size of the user's eye, for example, an arbitrary predetermined timing can be used.
Specifically, as the timing of detecting the initial value of the pupil size of the user's eye, for example, the control unit 230 detects that the user wears the display device 100 (for example, the left eye and the right eye). Timing when the controller 230 detects that an image of the user's eyes has been acquired for one or both of them, or timing when a predetermined time has elapsed from this point, or The timing at which the control unit 230 detects that a predetermined operation button provided in the operation button unit 250 of the control device 200 is operated by the user can be used.

Further, the state of the user's eye (in this embodiment, the size of the pupil) may be, for example, detecting the state of only one of the user's left eye and right eye, or Both states of the right eye may be detected. In the present embodiment, the state of both the left eye and the right eye of the user is detected as the state of the user's eye.
When detecting the state of both the left eye and the right eye of the user as the state of the user's eye, for example, for the result detected for the left eye and the result detected for the right eye, the one with the larger value is selected. A configuration to be employed, a configuration to employ a smaller value, a configuration to employ an average value of the values, or the like can be used.

  Thereafter, the control unit 230 acquires information input from the optical sensor 120 (in this embodiment, the level of ambient light detected by the optical sensor 120), and based on the acquired information, the light source of the backlight (left The eye backlight light source 410A and the right eye backlight light source 410B) are set to optimum luminance (step S22).

  Here, in the present embodiment, information input from the optical sensor 120 to the control unit 230 (in the present embodiment, the level of ambient light detected by the optical sensor 120) and a light source of the backlight (backlight for the left eye) The correspondence with the optimal luminance of the light source 410A and the right-eye backlight light source 410B) is stored in the storage unit 240 in advance. Based on the correspondence stored in the storage unit 240, the control unit 230 sets the backlight light sources (the left-eye backlight light source 410A and the right-eye backlight light source 410B) to the optimum luminance. In this case, the content of this correspondence may change depending on the transmittance of the shade (left eye shade 420A, right eye shade 420B), for example.

  Thereafter, the control unit 230 receives information input from the left-eye sensor 130A (in this embodiment, an image obtained by imaging the user's left eye 310A) and the right-eye sensor 130B at a predetermined timing. Based on the information (in this embodiment, an image obtained by capturing the right eye of the user), the size of the pupil of the user's eye is detected, and a predetermined determination is made with respect to the detected size of the pupil (step S23). ).

  Here, as the predetermined timing at which the control unit 230 detects the size of the pupil of the user, an arbitrary timing may be used. For example, a timing with a certain period (for example, 10 minutes) is used. be able to.

  Specifically, the control unit 230 determines whether the size of the detected user's pupil is not changed from the initial value or is smaller than the initial value as the determination in the process of step S23. Alternatively, it is determined whether it is larger than the initial value.

Here, when the controller 230 compares the size of the detected user's pupil with the initial value, for example, the detected user's pupil size is equal to or less than (initial value−9th threshold value). Sometimes it is determined to be smaller than the initial value, and when the detected user's pupil size is greater than or equal to (initial value + tenth threshold), it is determined to be larger than the initial value, and at other times (ie A configuration in which the size of the detected user's pupil exceeds (initial value−9th threshold value) and (less than (initial value + 10th threshold value)) determines that there is no change compared to the initial value. It can also be used. The ninth threshold value and the tenth threshold value are positive values.
In this configuration, various values may be used as the respective threshold values (the ninth threshold value to the tenth threshold value).

  In the present embodiment, in step S23, the control unit 230 updates the initial value of the user's pupil size with the size of the user's pupil detected this time. That is, in the present embodiment, in the process of step S23, except for the initial stage, the size of the user's pupil detected last time is compared with the size of the user's pupil detected this time, and the change is detected.

  As another configuration example, the control unit 230 returns to the process of step S21 to detect the initial value of the pupil size, for example, every predetermined period or every predetermined number of detections. It can also be set as the structure to do.

  As a result of the determination in the process of step S23, when it is determined that the size of the detected user's pupil is unchanged, the control unit 230 performs a normal operation (step S24), and proceeds to the process of step S23. (Step S23).

  As a result of the determination in step S23, when the control unit 230 determines that the size of the detected user's pupil is small, the display device 100 allows the user to view an image (in this embodiment, Considering that the ambient light has become bright, the user is alerted (step S25).

  Here, as an alert in the process of step S25, for example, the control unit 230 has an image extraction area of the display device 100 (an image extraction area of the left eye reflection unit 21A, an image extraction area of the right eye reflection unit 21B). In addition, the light of the content image for alerting the user is output and the image is displayed. As this image, for example, an image (warning image) including a text message such as “Is it too bright?” Can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.

Moreover, the control part 230 waits for the operation input from a user, after performing alerting in the process of step S25.
The user operates the operation unit 210 or the operation button unit 250 of the control device 200 in response to the display of an image (warning image) including a text message such as “Is it too bright?” And “YES (Yes: Bright Operation input indicating "NO" (No: not too bright).

  After performing the alert in the process of step S25, when the control unit 230 detects that an operation input indicating “YES (yes: too bright)” is performed by the user, The display mode is switched (step S26).

As a method of switching the display mode, in this embodiment, as previously determined, the display mode is switched to a display mode for reducing the luminance of the backlight light source (left eye backlight light source 410A, right eye backlight light source 410B). Use the method.
Various methods may be used as a method for reducing the backlight luminance. For example, a correspondence between the size of the pupil or the amount of change thereof and the luminance of the backlight or the amount of change thereof is set in advance. Thus, a method can be used in which the control unit 230 controls the luminance of the backlight based on the correspondence stored in the storage unit 240. In this case, the content of this correspondence may change depending on the transmittance of the shade (left eye shade 420A, right eye shade 420B), for example.

  Here, in the present embodiment, as the display mode after switching, a display mode that is considered to be able to relieve the user's eye fatigue and dry eye state compared to the display mode before switching. .

  On the other hand, after calling attention in the process of step S25, the control unit 230 detects that an operation input indicating "NO (No: not too bright)" is performed by the user, the process of step S23 The process proceeds to processing (step S23).

  Further, as a result of the determination in the process of step S23, when the control unit 230 determines that the size of the detected user's pupil is large, an environment in which the user views an image on the display device 100 (this embodiment) Now, it is considered that the ambient light has become dark, and the user is alerted (step S27).

  Here, as an alert in the process of step S27, for example, the control unit 230 has an image extraction area of the display device 100 (an image extraction area of the left eye reflection unit 21A, an image extraction area of the right eye reflection unit 21B). In addition, the light of the content image for alerting the user is output and the image is displayed. As this image, for example, an image (warning image) including a text message such as “Is it too dark?” Can be used. Moreover, this image may be displayed as a normal image, for example, or can be displayed by OSD.

Moreover, the control part 230 waits for the operation input from a user, after performing alerting in the process of step S27.
The user operates the operation unit 210 or the operation button unit 250 of the control device 200 in response to the display of an image (warning image) including a text message such as “Is it too dark?” Operation input indicating “too much” or “NO (No: not too dark)”.

  After performing the alert in the process of step S27, when the control unit 230 detects that an operation input indicating “YES (yes: too dark)” has been performed by the user, as described in advance The display mode is switched (step S28).

As a method for switching the display mode, in this embodiment, a method for switching to a display mode for increasing the transmittance of the shade (the left-eye shade 420A and the right-eye shade 420B) is used.
Various methods may be used as a method for increasing the transmittance of the shade. For example, a correspondence between the size of the pupil or a change amount thereof and the transmittance of the shade or the change amount thereof is set in advance. A method can be used in which the control unit 230 controls the transmittance of the shade based on the correspondence stored in the storage unit 240 and stored in the storage unit 240. In this case, the contents of this correspondence may change depending on the luminance of the backlight light source (left eye backlight light source 410A, right eye backlight light source 410B), for example.

  Here, in the present embodiment, as the display mode after switching, a display mode that is considered to be able to relieve the user's eye fatigue and dry eye state compared to the display mode before switching. .

  In addition, after switching the display mode, the control unit 230 sets the backlight light sources (the left-eye backlight light source 410A and the right-eye backlight light source 410B) to the optimum luminance based on the level of the ambient light. The process proceeds to S22 (step S22).

  On the other hand, after calling attention in the process of step S27, the control unit 230 detects that an operation input indicating "NO (No: not too dark)" is performed by the user, the process of step S23 The process proceeds to processing (step S23).

  Here, in the example of the flowchart shown in FIG. 5, alerting (for example, warning display) and switching of the display mode are performed both when the pupil size of the user's eye becomes smaller and larger. However, as another example, it is configured to alert (for example, display a warning) or switch the display mode only when the pupil size of the user's eye becomes smaller or larger. You can also.

  As described above, in the transmissive display device according to the present embodiment, the control unit 230 detects the state of the user's eyes (in this embodiment, the size of the pupil), and detects the detection based on the detection result. For example, if it is determined that the user's eye condition has become a predetermined state that is considered fatigued, for example, a predetermined warning image is displayed to the user, and the user's eye is determined accordingly. When an operation input is received, the display mode is switched as determined in advance.

  Thus, in the transmissive display device according to the present embodiment, the actual condition of the user's eyes (in this embodiment, the condition of the user's eyes estimated based on the size of the pupil (the situation that seems to be fatigued) ), And the display mode can be switched according to the actual eye condition of the user.

  According to the transmissive display device according to the present embodiment, the degree of ambient brightness is estimated by detecting the size of the pupil as monitoring of the state of the user's eyes. Then, the backlight brightness and shade transmittance are optimized.

As a specific example, the pupil usually spreads in a dark place or when the weather changes to dark, but if the backlight is too bright, the pupil will be small. Prevent fatigue.
Further, when the pupil becomes larger, the eye transmittance of the user is prevented by increasing the transmittance of the shade. Note that the luminance of the backlight can be increased instead of increasing the transmittance of the shade, but the power consumption can be reduced by increasing the transmittance of the shade.

  In general, if there is a difference between ambient brightness (for example, considering the transmittance of the shade) and backlight brightness, eyes are likely to get tired. For this reason, in this embodiment, it is also possible to reduce the burden on the user's eyes by detecting the ambient light (external light) with the optical sensor 120 to set the optimal backlight luminance for the ambient light. is there. Further, for example, it is possible to reduce power consumption by selecting an optimal backlight luminance.

[Summary of Embodiments Shown above]
Here, the optical sensor 120 may be provided at an arbitrary position.
As another example of the left eye sensor 130A and the right eye sensor 130B, a sensor (infrared sensor) that emits infrared light to the user's eye and detects the state of the user's eye based on the reflected light. An optical sensor) can also be used. In this case, the left-eye sensor 130A and the right-eye sensor 130B can be provided, for example, at the same position as shown in FIG.

As another example of the left-eye sensor 130A and the right-eye sensor 130B, a pressure sensor is provided inside the temple (temple) of the display device 100 and the display device 100 is mounted. It is also possible to use the pressure sensor that detects the movement of the muscle that moves the eye and determines the state of the user's eye based on the detection result.
FIG. 6 is an external view of a transmissive display device according to another embodiment of the present invention when such a pressure sensor is used.
The transmissive display device illustrated in FIG. 6 includes a left-eye sensor 510A and a right-eye sensor 510B configured by pressure sensors instead of the left-eye sensor 130A and the right-eye sensor 130B illustrated in FIG. . The other components shown in FIG. 6 are the same as those shown in FIG. 1, and are denoted by the same reference numerals as those shown in FIG.

  As another example of the state of the user's eyes, the degree of eye opening can be detected. Usually, it can be considered that the eye has become fatigued as the degree of eye opening (for example, the average value of the degree of eye opening over a predetermined period) becomes smaller. In this case, for example, a camera, an infrared light sensor, a pressure sensor, or the like may be used as the left eye sensor or the right eye sensor.

Further, the left-eye shade 420A and the right-eye shade 420B may be separate bodies or may be integrated, for example.
Further, the shade (shade for left eye 420A, shade for right eye 420B) may be provided integrally with the display unit (display unit for left eye 110A, display unit for right eye 110B), for example, or , May be provided in a detachable configuration.
In the above embodiment, the display device 100 including the shade (the left eye shade 420A and the right eye shade 420B) is shown. However, as another configuration example, when the control related to the shade is not performed, the display device 100 is provided. 100 may not be provided with a shade.

  The power supply unit 260 may be provided in the display device 100 instead of the control device 200, or both the control device 200 and the display device 100 may be provided with a power supply unit.

Further, for the left eye component and the right eye component that are paired, for example, the same control may be performed for the left eye and the right eye, or for the left eye and the right eye. Different controls may be used for different applications.
Moreover, even if control (same control or different control) is performed on both the left-eye component and the right-eye component unit, for example, for the left-eye component unit and the right-eye component unit. Alternatively, only one of the left eye and the right eye may be controlled.

In the above embodiment, the user operates the operation input unit 270 (one or both of the operation unit 210 and the operation button unit 250 in the present embodiment) of the control device 200, so that the operation input is performed. Although the example detected by the control unit 230 has been shown, as another configuration example, a part or all of the operation input unit that receives an operation input from the user may be provided in the display device 100, for example.
As another configuration example, one or both of the left-eye sensor 130A and the right-eye sensor 130B, or another sensor (for example, a camera) is used as an operation input unit to move the eye of the user. Based on this, an operation input from the user may be received. As a specific example, a configuration in which an operation input of “YES” or “NO” is accepted when the state where the user's eyes are closed continues for a predetermined time or more, or one of the user's left eye and right eye When the eye on the other side is opened and the eye on the other side is closed for a time longer than a predetermined value, a configuration in which an operation input of “YES” or “NO” is accepted is used. it can.

In the above embodiment, a transmissive display device (a transmissive display device for both eyes) having both left-eye and right-eye components is shown as an example. However, as another configuration example, The present invention may be applied to a transmissive display device for only one of the left eye and the right eye (one-eye transmissive display device).
In the above embodiment, an example in which the light from the outside world is applied to a see-through display device (transmission type display device) that transmits light from the outside to the image extraction area has been described. However, light from the outside world is transmitted to the image extraction area. It is also possible to apply to a closed type display device (non-transmission type display device) that is not allowed to be used.

In addition, the present invention may be applied to, for example, a light guide system that uses a hologram reflection surface.
In addition, the present invention may be applied to, for example, an apparatus in which image light is simply reflected by a reflecting surface without using a light guide plate.
For example, various known techniques may be applied to the present invention. As a specific example, the display device 100 may include a pupil division type see-through optical system as an optical system, or may include a transflective type see-through optical system. As a specific example, the display device 100 includes, as an optical system, an optical system included in an image display device described in Japanese Patent Application Laid-Open No. 2008-20770, and a head-mounted display described in Japanese Patent No. 4636164. An optical system, an optical system provided in a beam scanning display device described in International Publication No. 2009/041055, or an optical system provided in a display device described in Japanese Patent No. 4055283 may be provided.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  It should be noted that a program for realizing the function of an arbitrary component in the transmissive display device described above is recorded on a computer-readable recording medium, and the program is read into a computer system and executed. Good. The “computer system” mentioned here includes an OS (Operating System) and hardware such as peripheral devices. “Computer-readable recording medium” means a portable disk such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD (Compact Disk) -ROM, or a hard disk built in a computer system. Refers to the device. Further, the “computer-readable recording medium” means a volatile memory (RAM: Random Access) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Memory that holds a program for a certain period of time, such as Memory).

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 10A ... Image formation part for left eyes, 10B ... Image formation part for right eyes, 11A ... Image generation part for left eyes, 11B ... Image generation part for right eyes, 12A ... Projection optical system for left eyes, 12B ... For right eyes Projection optical system, 20A: light guide for left eye, 20B ... light guide for right eye, 21A ... reflection for left eye, 21B ... reflection for right eye, 100 ... display device, 110A ... display for left eye 110B ... right eye display unit, 120 ... light sensor, 130A ... left eye sensor, 130B ... right eye sensor, 200 ... control device, 210 ... operation unit, 230 ... control unit, 240 ... storage unit, 250 ... Operation button section, 260 ... Power supply section, 270 ... Operation input section, 310A ... User's left eye, 410A ... Backlight light source for left eye, 410B ... Backlight light source for right eye, 411A ... Light modulation element for left eye, 411B ... Right-eye light modulation element, 420A ... Eye shade, 420B ... right eye shade, 510A ... left-eye sensor, 510B ... right eye sensor,

Claims (10)

A display that transmits light from the outside to the image extraction area and outputs image light to the image extraction area;
A sensor that detects the user's eye condition;
An operation input unit that receives an operation input from the user;
When it is determined that the state of the user's eye has become a predetermined state based on the state of the user's eye detected by the sensor, a predetermined image is displayed by the display unit, and accordingly A control unit that switches a display mode of the display unit when it is detected that a predetermined operation input from the user is received by the operation input unit;
A transmissive display device comprising: The controller is
Based on the state of the user's eyes detected by the sensor, determine the number of blinks of the user's eyes as the state of the user's eyes,
One or both of a case where it is determined that the number of blinks of the user's eyes is equal to or greater than a first predetermined value and a case where it is determined that the number of blinks of the users' eyes is equal to or less than a second predetermined value And determining that the state of the eyes of the user has reached the predetermined state.
The transmissive display device according to claim 1. The control unit is switched to a display mode for displaying an eye resting image by the display unit as the display mode switching.
The transmissive display device according to claim 1, wherein the transmissive display device is provided. The control unit switches to a display mode for turning off the backlight of the display unit as the switching of the display mode.
The transmissive display device according to claim 1, wherein the transmissive display device is provided. The controller is
Based on the state of the user's eye detected by the sensor, the size of the pupil of the user's eye is determined as the state of the user's eye,
In one or both of the case where it is determined that the size of the pupil of the user's eye is reduced and the case where it is determined that the size of the pupil of the user's eye is increased, the state of the eye of the user is the Determine that it has reached a predetermined state,
The transmissive display device according to claim 1. The control unit displays a predetermined image by the display unit when it is determined that the size of the pupil of the user's eye is reduced, and the user predetermined by the operation input unit in response thereto When it is detected that an operation input from has been received, the display mode is switched to a display mode that reduces the backlight brightness of the display unit,
The transmissive display device according to claim 5. The control unit displays a predetermined image by the display unit when it is determined that the size of the pupil of the user's eye has increased, and the user predetermined by the operation input unit in response thereto When it is detected that an operation input from has been received, the display mode is switched to a display mode that increases the shade transmittance of the display unit.
The transmissive display device according to claim 5, wherein the transmissive display device is provided. The display unit is provided in a display device mounted on a user's head,
The display unit includes a display element, a projection lens that projects image light from the display element, and a light guide unit that is fixed to the projection lens and guides the image light from the projection lens to the image extraction area. With
An image of image light from the display element and an image of light from the outside can be simultaneously observed through the light guide unit.
The transmissive display device according to any one of claims 1 to 7, wherein the transmissive display device is provided. A display unit transmitting light from the outside to the image extraction area and outputting the image light to the image extraction area;
When the control unit determines that the user's eye state is in a predetermined state based on the user's eye state detected by the sensor, the control unit displays a predetermined image by the display unit. A step of switching the display mode of the display unit when it is detected that an operation input from the user determined in advance by the operation input unit is received,
A display method in a transmissive display device, comprising: A procedure in which the display unit transmits light from the outside to the image extraction area and outputs the image light to the image extraction area;
A procedure for the sensor to detect the condition of the user's eye;
A procedure in which the operation input unit receives an operation input from the user;
When the control unit determines that the state of the user's eye has become a predetermined state based on the state of the user's eye detected by the sensor, the display unit displays a predetermined image. In response to this, a procedure for switching the display mode of the display unit when it is detected that an operation input from the user predetermined by the operation input unit is received,
Display program in a transmissive display device for causing a computer to execute the program.

Images