WO2016072271A1 - Display device, method for controlling display device, and control program therefor - Google Patents

Abstract

The present invention provides a display device, a method for controlling the display device, and a control program therefor with which it is possible to easily and quickly display a desired screen image. The display device is equipped with: a display unit having a display member provided with a screen capable of displaying an image; a detection device having a detection area, the detection device detecting the direction of movement in which an indication unit moved by a user moves in the detection area and generating an output; and a control unit for controlling the screen display of the display unit on the basis of the output of the detection device. When it is detected on the basis of the output from the detection device that the indication unit moved by the user has moved in the detection area, the control device takes the direction of that movement as the first direction of movement, performs a screen display control that corresponds to the first direction of movement, and furthermore starts using a timer to measure a duration from when the first direction of movement is detected. Even when an output from the detection device indicating that the indication unit has passed through in a direction of movement differing from the first direction of movement is generated at least once while the timer is in operation from when the measurement started to when a prescribed amount of time elapses, the control device skips screen display control that corresponds to the different direction of movement.

Description

Display device, display device control method, and control program therefor

The present invention relates to a display device, a display device control method, and a control program therefor.

In recent years, mobile terminals such as smartphones that have been rapidly developed are often used for business and home work assistance. A general portable terminal has a touch panel screen that serves both as an image display and a user interface. By touching this screen, a user performs necessary input to display a desired image or input information. Etc. can be performed. However, there are cases where you want to operate the mobile device without touching the screen, such as when the user's hand is wet or dirty, and how to perform the input at that time is an issue .

Patent Document 1 discloses a mobile computing device including an infrared LED and an infrared proximity sensor. According to the technique of Patent Document 1, infrared light emitted from an infrared LED is reflected by a hand that is close to a computing device and reflected, and the reflected light is detected by an infrared proximity sensor, whereby the movement of the hand is detected. It can be detected, and a desired input can be performed in a non-contact manner by the movement (gesture) of the hand to perform operations such as swipe. Therefore, there is no risk of contaminating the computing device even if the user's hand is dirty.

Special table 2013-534209 JP 2013-206412 A

By the way, there are cases where the user wants to perform screen scrolling and screen transition (page feed) continuously. In such a case, if the touch panel screen can be touched, a desired operation can be easily and quickly performed by repeatedly rubbing the screen in one direction with the user's finger. However, in the gesture detection using the infrared proximity sensor disclosed in Patent Document 1, when the user reciprocates the hand within the detection area of the infrared proximity sensor, the computer that has received a signal from the infrared proximity sensor that has detected this is detected. On the device side, it is recognized as reciprocating scroll or reciprocating page feed, and there is a problem that continuous scrolling and page turning in one direction cannot be performed well. After moving the user's hand from one end of the detection area of the infrared proximity sensor to the other end, once removed from the detection area, further moved to one end outside the detection area, and again If it is moved from one end of the detection area to the other end, continuous scrolling and page turning in one direction are possible, but the user is forced to perform complex operations and is not easy to use. There's a problem.

In Patent Document 2, as an example using a non-contact user interface, when a user's hand enters the imaging range of the camera, the user's hand is extracted from the captured image and the gesture of the hand is recognized. Thus, a head mounted display (hereinafter referred to as “HMD”) that displays an image on the next page is disclosed.
Since a general HMD has a display placed in front of the user's eyes, it is inherently difficult to perform input by touching the screen, so it can be said that it is preferable to use gesture input. However, when it is desired to continuously perform screen scrolling, page turning, etc., as in the technique of Patent Document 1, in order to avoid recognizing the user's hand as a reciprocating motion, After moving from one end of the imaging range to the other end, once removed from the imaging range, further moved to one end outside the imaging range, and again from one end of the imaging range to the other A complicated operation of moving to the end of the is required. In addition, since the imaging range of the camera is often wider than the detection area of the infrared proximity sensor, there is a problem that the burden on the user who performs the gesture operation becomes large.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a display device, a display device control method, and a control program thereof that can easily and quickly display a desired screen.

The display device of the present invention includes a display unit having a display member having a screen capable of displaying an image,
A detection device having a detection region and generating an output by detecting a moving direction in which the pointing unit moved by the user moves in the detection region;
A control device for controlling the screen display of the display unit based on the output of the detection device,
When the control unit detects that the pointing unit has moved in the detection area based on the output of the detection device, the control unit sets the moving direction of the pointing unit as the first moving direction and responds to the first moving direction. Screen display control,
Furthermore, the timer starts counting from the time of detection of the first movement direction, and the instruction unit moves the first movement at least once during the operation of the timer from the time measurement until the specified time elapses. Even if the output of the detection device indicating that the movement is in a different movement direction from the direction is generated, the screen display control corresponding to the different movement direction is skipped.

The display device control method of the present invention includes a display unit having a display member having a screen capable of displaying an image, a detection area, and an indication unit that is moved by a user detects a moving direction in which the detection area moves. A display device comprising: a detection device that generates an output; and a control method for the display device that controls screen display of the display unit based on the output of the detection device,
When the instruction unit detects that the detection unit has moved in the detection area based on the output of the detection device, the movement direction of the instruction unit is set as the first movement direction, and screen display control according to the first movement direction is performed. Done
Further, the indication unit indicates that the instruction unit has moved in a movement direction different from the first movement direction at least once from the time when the first movement direction is detected until the specified time elapses from the start of timing. Even when the output of the detection device is generated, the screen display control corresponding to the different movement directions is skipped.

A control program for a display device according to the present invention detects a moving direction in which a display unit having a display member having a screen capable of displaying an image, a detection area, and an instruction unit moved by a user moves in the detection area. A control program for a display device comprising: a detection device that generates an output; and a control device that controls screen display of the display unit based on the output of the detection device,
In the control device,
When the instruction unit detects that the detection unit has moved in the detection area based on the output of the detection device, the movement direction of the instruction unit is set as the first movement direction, and screen display control according to the first movement direction is performed. Let
Further, the indication unit indicates that the instruction unit has moved in a movement direction different from the first movement direction at least once from the time when the first movement direction is detected until the specified time elapses from the start of timing. Even if the output of the detection device is generated, the screen display control corresponding to the different movement directions is skipped.

According to the present invention, it is possible to provide a display device, a control method for the display device, and a control program for the display device that allow a user to easily and quickly perform an intended screen display operation such as screen scrolling or continuous screen transition operation. .

1 is a perspective view of a head mounted display (HMD) according to a first embodiment. It is the figure which looked at HMD from the front. It is the figure which looked at HMD from the upper part. It is a schematic sectional drawing which shows the structure of a display unit. It is an enlarged view of a proximity sensor. It is a block diagram of the main circuit of HMD. It is a front view when a user wears HMD on the head. It is the side view (a) and partial top view (b) when a user wears HMD on the head. It is a figure which shows the image which a user visually recognizes through a see-through type image display part. It is a figure which shows the example of the signal waveform which a some light reception area | region outputs. It is a figure which shows the other example of the signal waveform which a some light reception area | region outputs. It is a flowchart which shows the screen display control process based on gesture operation which a processor performs. It is a figure which shows the state which a user visually recognizes his hand HD through a see-through type image display part at the time of lock mode setting. It is a time chart which shows the example of the screen display control process based on gesture operation. It is a figure which shows the motion of hand HD at the time of performing the screen display control to the same direction at the time of lock mode OFF setting. It is a figure which shows the example of the image switched by gesture operation. It is the figure which looked at the smart phone 200 concerning 2nd Embodiment from the front.

(First embodiment)
The present embodiment will be described below with reference to the drawings. FIG. 1 is a perspective view of an HMD 100 including a display device according to the present embodiment. FIG. 2 is a front view of the HMD 100 according to the present embodiment. FIG. 3 is a view of the HMD 100 according to the present embodiment as viewed from above. Hereinafter, the right side and the left side of the HMD 100 refer to the right side and the left side for the user wearing the HMD 100.

As shown in FIGS. 1 to 3, the HMD 100 of the present embodiment includes a frame 101 and a display unit 104 that performs screen display. The frame 101 constitutes a mounting member that holds the display unit 104 on the head so as to be positioned in front of the eyes. A frame 101 that is U-shaped when viewed from above has a front part 101a to which two spectacle lenses 102 are attached, and side parts 101b and 101c extending rearward from both ends of the front part 101a. The two spectacle lenses 102 attached to the frame 101 may or may not have refractive power.

The cylindrical main body 103 is fixed to the front part 101 a of the frame 101 on the upper part of the spectacle lens 102 on the right side (which may be on the left side according to the user's dominant eye). A display unit 104 is provided in the main body 103 which is a part of the mounting member. In the main body 103, a display control unit 104DR (see FIG. 6 described later) that controls display of the display unit 104 based on an instruction from the processor 121 described later is disposed. If necessary, a display unit may be arranged in front of both eyes.

FIG. 4 is a schematic cross-sectional view showing the configuration of the display unit 104. The display unit 104 includes an image forming unit 104A and an image display unit 104B. The image forming unit 104A is incorporated in the main body unit 103, and includes a light source 104a, a unidirectional diffuser 104b, a condenser lens 104c, and a display element 104d. On the other hand, the image display unit 104B, which is a so-called see-through type display member, is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1). The eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h.

The light source 104a has a function of illuminating the display element 104d. For example, 462 ± 12 nm (B light), 525 ± 17 nm (G light), 635 ± 11 nm (R It is composed of RGB-integrated LEDs that emit light in three wavelength bands. Thus, the light source 104a emits light having a predetermined wavelength width, whereby the image light obtained by illuminating the display element 104d can have a predetermined wavelength width, and the hologram optical element 104h transmits the image light. When diffracted, the image can be observed by the user over the entire observation angle of view at the position of the pupil B. Further, the peak wavelength for each color of the light source 104a is set in the vicinity of the peak wavelength of the diffraction efficiency of the hologram optical element 104h, so that the light use efficiency is improved.

Further, since the light source 104a is composed of LEDs that emit RGB light, the cost of the light source 104a can be reduced, and a color image is displayed on the display element 104d when the display element 104d is illuminated. The color image can be visually recognized by the user. In addition, since each of the RGB LED elements has a narrow emission wavelength width, the use of a plurality of such LED elements enables high color reproducibility and bright image display.

The display element 104d displays an image by modulating the light emitted from the light source 104a in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels that serve as light transmitting regions in a matrix. Has been. Note that the display element 104d may be of a reflective type.

The eyepiece prism 104f totally reflects the image light from the display element 104d incident through the base end face PL1 by the opposed parallel inner side face PL2 and outer side face PL3, and passes through the hologram optical element 104h to the user's pupil. On the other hand, external light is transmitted and guided to the user's pupil, and is composed of, for example, an acrylic resin together with the deflecting prism 104g. The eyepiece prism 104f and the deflection prism 104g are joined by an adhesive with the hologram optical element 104h sandwiched between inclined surfaces PL4 and PL5 inclined with respect to the inner surface PL2 and the outer surface PL3.

The deflection prism 104g is joined to the eyepiece prism 104f, and becomes a substantially parallel flat plate integrated with the eyepiece prism 104f. By joining the deflecting prism 104g to the eyepiece prism 104f, it is possible to prevent distortion in the external image observed by the user through the display unit 104.

That is, for example, when the deflecting prism 104g is not joined to the eyepiece prism 104f, external light is refracted when passing through the inclined surface PL4 of the eyepiece prism 104f, so that an external image observed through the eyepiece prism 104f is distorted. . However, by joining the deflecting prism 104g having the inclined surface PL5 complementary to the eyepiece prism 104f to form an integral substantially parallel plate, external light is transmitted through the inclined surfaces PL4 and PL5 (hologram optical element 104h). The refraction at that time can be canceled by the deflecting prism 104g. As a result, it is possible to prevent distortion in the external image observed through the see-through. In addition, if the spectacle lens 102 (refer FIG. 1) is mounted | worn between the display unit 104 and a user's pupil, it will be possible for the user who normally uses spectacles to observe an image without a problem.

The hologram optical element 104h diffracts and reflects the image light (light having a wavelength corresponding to the three primary colors) emitted from the display element 104d, guides it to the pupil B, enlarges the image displayed on the display element 104d, and enlarges the user's pupil. It is a volume phase type reflection hologram guided as a virtual image. The hologram optical element 104h has, for example, three wavelength ranges of 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected). Here, the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak, and the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half maximum of the diffraction efficiency peak. is there.

The reflection-type hologram optical element 104h has high wavelength selectivity, and only diffracts and reflects light having a wavelength in the above-mentioned wavelength range (near the exposure wavelength). The hologram optical element 104h is transmitted, and a high external light transmittance can be realized.

Next, the operation of the display unit 104 will be described. The light emitted from the light source 104a is diffused by the unidirectional diffusion plate 104b, condensed by the condenser lens 104c, and enters the display element 104d. The light incident on the display element 104d is modulated for each pixel based on the image data input from the display control unit 104DR, and is emitted as image light. Thereby, a color image is displayed on the display element 104d.

Image light from the display element 104d enters the eyepiece prism 104f from its base end face PL1, is totally reflected a plurality of times by the inner side face PL2 and the outer side face PL3, and enters the hologram optical element 104h. The light incident on the hologram optical element 104h is reflected there, passes through the inner side surface PL2, and reaches the pupil B. At the position of the pupil B, the user can observe an enlarged virtual image of the image displayed on the display element 104d, and can visually recognize it as a screen formed on the image display unit 104B. In the present specification, “screen” may refer to an image to be displayed.

On the other hand, since the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h transmit almost all of the external light, the user can observe an external field image (real image) through them. Therefore, the virtual image of the image displayed on the display element 104d is observed so as to overlap with a part of the external image. In this manner, the user of the HMD 100 can simultaneously observe the image provided from the display element 104d and the external image via the hologram optical element 104h. Note that when the display unit 104 is in the non-display state, the image display unit 104B is transparent, and only the external image can be observed. In this example, a display unit is configured by combining a light source, a liquid crystal display element, and an optical system. However, instead of a combination of a light source and a liquid crystal display element, a self-luminous display element (for example, an organic EL display) is used. Element) may be used. Further, instead of a combination of a light source, a liquid crystal display element, and an optical system, a transmissive organic EL display panel having transparency in a non-light emitting state may be used. In any case, when the screen is arranged so as to fall within the visual field of the user's eye facing the image display unit 104B, and preferably at least partially overlaps the effective visual field, the user can easily visually recognize the image. Can do.

Further, in FIGS. 1 and 2, on the front side of the main body 103, a proximity sensor 105 constituting a detection device arranged near the center of the frame 101 and a lens 106 a of a camera 106 arranged near the side are in front. It is provided to face. As will be described later, in this embodiment, the movement of the user's hand is detected based on the output from the proximity sensor 105, and screen display control such as screen switching is performed according to the hand passing direction.

In this specification, the “proximity sensor” refers to a detection region in the proximity range in front of the detection surface of the proximity sensor in order to detect that an object, for example, a part of the human body is close to the user's eyes. This means that a signal is output by detecting whether or not it exists. The proximity range may be set as appropriate according to the operator's characteristics and preferences. For example, the proximity range from the detection surface of the proximity sensor may be within a range of 200 mm. If the distance from the proximity sensor is 200 mm or less, the user can put the palm in and out of the user's field of view with the arm bent, and can be easily operated by gestures using the hand. In addition, there is less risk of erroneously detecting a human body or furniture other than the user. Here, the control device determines that the object exists in the proximity range based on a signal output from the proximity sensor when the object enters the detection area in the proximity range in front of the proximity sensor. When an object enters the detection area, an effective signal may be output from the proximity sensor to the control device.

There are two types of proximity sensors: passive type and active type. A passive proximity sensor has a detection unit that detects invisible light and electromagnetic waves emitted from an object when the object approaches. As a passive proximity sensor, there are a pyroelectric sensor that detects invisible light such as infrared rays emitted from an approaching human body, and a capacitance sensor that detects a change in electrostatic capacitance between the approaching human body and the like. The active proximity sensor includes an invisible light and sound wave projection unit, and a detection unit that receives the invisible light and sound wave reflected and returned from the object. Active proximity sensors include infrared sensors that project infrared rays and receive infrared rays reflected by objects, laser sensors that project laser beams and receive laser beams reflected by objects, and project ultrasonic waves. Then, there is an ultrasonic sensor that receives ultrasonic waves reflected by an object. Note that the passive proximity sensor is excellent in low power consumption. An active proximity sensor is easy to improve the certainty of detection. For example, even when the user is wearing an infrared opaque glove, it can be detected. A plurality of types of proximity sensors may be used in combination.

Proximity sensors are generally smaller and cheaper than cameras, and consume less power. The proximity sensor cannot perform complicated detection such as detection of the shape of the object, but can determine the approach or separation of the object, so that the HMD is operated by passing the hand or holding the hand. In addition, complicated image processing required for gesture recognition by analysis of the image captured by the camera is also unnecessary.

In this specification, the “indicator that is moved by the user” refers to an object that is driven according to the user's intention, such as a part of the user's body (for example, a hand or a finger), It may be.

FIG. 5 is an enlarged view of the proximity sensor 105 used in the present embodiment as viewed from the front. In the present embodiment, an example in which a pyroelectric sensor is used as the proximity sensor 105 will be described. In FIG. 5, the proximity sensor 105 includes a light receiving unit 105 a that receives invisible light such as infrared light emitted from a human body as detection light. The light receiving unit 105a forms light receiving areas RA to RD arranged in 2 rows and 2 columns, and when detecting light is received, corresponding signals are individually output from the light receiving areas RA to RD. It has become. If it is a pyroelectric sensor that detects infrared light emitted from the human body, it is difficult to falsely detect an object other than the exposed human body. For example, when working in a narrow space, the false detection is effectively prevented. There is an advantage that you can.

1 and 2, the right sub-body portion 107 is attached to the right side portion 101b of the frame 101, and the left sub-body portion 108 is attached to the left side portion 101c of the frame 101. The right sub-main body portion 107 and the left sub-main body portion 108 have an elongated plate shape, and have elongated protrusions 107a and 108a on the inner side, respectively. By engaging the elongated protrusion 107 a with the elongated hole 101 d of the side portion 101 b of the frame 101, the right sub-body portion 107 is attached to the frame 101 in a positioned state, and the elongated protrusion 108 a is attached to the side of the frame 101. By engaging with the long hole 101e of the portion 101c, the left sub-main body portion 108 is attached to the frame 101 in a positioned state.

In the right sub-main body 107, a geomagnetic sensor 109 (see FIG. 6 to be described later) for detecting geomagnetism, and a gyro and acceleration sensor 110 (see FIG. 6 to be described later) that generates an output corresponding to the posture. And a speaker (or earphone) 111A and a microphone 111B (see FIG. 6 to be described later) are provided in the left sub-main body section 108. The main main body 103 and the right sub main body 107 are connected so as to be able to transmit signals through a wiring HS, and the main main body 103 and the left sub main body 108 are connected so as to be able to transmit signals through a wiring (not shown). Yes. As schematically illustrated in FIG. 3, the right sub-main body 107 is connected to the control unit CTU via a cord CD extending from the rear end. A 6-axis sensor in which a gyro and an acceleration sensor are integrated may be used. Further, the HMD can be operated by sound based on an output signal generated from the microphone 111B according to the input sound. Further, the main main body 103 and the left sub main body 108 may be configured to be wirelessly connected.

FIG. 6 is a block diagram of main circuits of the HMD 100. The control unit CTU includes a processor 121 as a control device, an operation unit 122, a GPS reception unit 123 that receives radio waves from GPS satellites, a communication unit 124 that exchanges data with the outside, and a ROM 125 that stores programs and the like. A RAM 126 for storing image data and the like; a power supply circuit 128 and a battery 127 for supplying power to each unit; and a storage device 129 such as an SSD or a flash memory. The processor 121 can use an application processor used in a smartphone or the like, but the type of the processor 121 is not limited. For example, if an application processor has built-in hardware necessary for image processing such as GPU and Codec as a standard, it can be said that the processor is suitable for a small HMD. Here, the processor 121, the proximity sensor 105, the display control unit 104DR, and the display unit 104 constitute a display device.

Further, when the light receiving unit 105a detects the detection light emitted from the human body, the signal is input to the processor 121. The processor 121 controls image display on the display unit 104 via the display control unit 104DR.

The processor 121 receives power from the battery 127 via the power supply circuit 128, operates according to a program stored in at least one of the ROM 124 and the storage device 129, and operates according to an operation input such as power-on from the operation unit 122 according to an operation input such as power-on. Can be input and stored in the RAM 126 and communicated with the outside via the communication unit 123 as necessary. Furthermore, as will be described later, it is possible to detect a gesture operation in a non-contact manner and execute image control corresponding to the detected gesture operation. Further, the processor 121 can execute the lock mode in accordance with a control program stored in at least one of the ROM 124 and the storage device 129. Details of the lock mode will be described later.

FIG. 7 is a front view when the user US wears the HMD 100 of the present embodiment on the head. FIG. 8 is a side view (a) and a top view (b) when the user US wears the HMD 100 on the head, and shows it together with the user's hand. FIG. 9 is a diagram showing an image visually recognized by the user US through the see-through type image display unit 104B. Here, the gesture operation is an operation in which at least the hand HD of the user US approaches or separates from the proximity sensor 105 and can be detected by the processor 121 of the HMD 100 via the proximity sensor 105.

As shown in FIG. 9, the screen 104i of the image display unit 104B is arranged so as to overlap the effective visual field EV of the user's eye facing the image display unit 104B (here, positioned in the effective visual field EV). The Further, the detection area SA of the proximity sensor 105 is in the visual field of the user's eye facing the image display unit 104B. Preferably, the detection area SA is located within the stable focus field of the user's eye or inside the visual field (within about 90 ° horizontal and within about 70 ° vertical), and more preferably located inside the stable focus field. The proximity sensor 105 may be installed with its arrangement and orientation adjusted so as to overlap with the effective visual field EV or the inner visual field (horizontal within about 30 °, vertical within about 20 °).

FIG. 9 shows an example in which the detection area SA overlaps the screen 104i. In this way, by setting the detection region SA of the proximity sensor 105 within the visual field of the eye of the user US while the user US is wearing the frame 101 that is a mounting member on the head, the screen is displayed through the screen 104i. While observing the hand, the approach and retraction of the hand to the detection area SA of the proximity sensor 105 can be reliably recognized without accompanying eye movement. In particular, by making the detection area SA of the proximity sensor 105 within the stable field of view or the inside visual field, even if the user observes the screen, the gesture operation can be reliably performed while recognizing the detection area SA. . Further, by making the detection area SA of the proximity sensor 105 within the effective visual field EV or the visual field inside the effective visual field EV, the gesture operation can be performed more reliably. If the detection area SA overlaps the screen 104i, the gesture operation can be performed more reliably. When the proximity sensor has a plurality of light receiving regions as in the present embodiment, the entire plurality of light receiving regions are regarded as one light receiving unit, and the maximum detection range of the light receiving unit is regarded as a detection region. As shown in FIG. 9, when the detection area SA of the proximity sensor 105 is set to overlap the screen 104i, an image indicating the detection area SA is displayed on the screen 104i (for example, the range of the area SA is displayed by a solid line). ), The user can surely recognize the detection area SA, so that the operation by the gesture can be performed more reliably.

Describes gesture operation detection. If there is nothing in front of the user US, the light receiving unit 105a does not receive the detection light, so the processor 121 determines that no gesture operation is performed. On the other hand, as shown in FIG. 8, when the user's US hand HD is approached in front of the user US, detection light emitted from the hand HD can be detected by the light receiving unit 105a as shown by a dotted line. . Accordingly, the processor 121 determines that a gesture operation has been performed.

As described above, the light receiving unit 105a has the light receiving regions RA to RD arranged in 2 rows and 2 columns (see FIG. 5). Therefore, when the user US moves the hand HD closer to the front of the HMD 100 from either the left, right, up, or down directions, the output timings of signals detected in the light receiving areas RA to RD are different.

10 and 11 are examples of signal waveforms of the light receiving areas RA to RD, where the vertical axis represents the signal intensity of the light receiving areas RA to RD and the horizontal axis represents the time. For example, when the user US moves the hand HD from the right to the left in front of the HMD 100 with reference to FIGS. 8 and 9, the detection light emitted from the hand HD is incident on the light receiving unit 105a. In this case, the light receiving areas RA and RC first receive the detection light. Therefore, as shown in FIG. 10, first, the signals in the light receiving areas RA and RC rise, the signals in the light receiving areas RB and RD rise later, and the signals in the light receiving areas RA and RC fall, and then the light receiving areas RB, The RD signal falls. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the right to the left (here, passing through the detection region). Accordingly, the processor 121 controls the display control unit 104DR to change the display so as to perform page turning from the image G1 to the image G2, for example, in accordance with the movement of the hand HD as shown in FIG. (FIG. 9 shows a state in which the image G1 is being switched while sliding from the image G1 to the image G2 or in the middle of being switched to the image G2 while the image G1 is being wiped).

In the example of FIG. 11, the signals of the light receiving areas RA and RB rise first, the signals of the light receiving areas RC and RD rise after a delay, and the signals of the light receiving areas RA and RB further fall, and then the signals of the light receiving areas RC and RD. Is falling. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the top to the bottom (here, passing through the detection region).

According to the present embodiment, the presence and movement of the hand HD can be reliably detected by using the proximity sensor and positioning the detection region within the visual field of the user's eye facing the image display unit. Accordingly, the user US can view the hand operation and the operation in conjunction with each other, and can realize an intuitive operation. Furthermore, since the proximity sensor is small and consumes less power than the camera, the continuous operation time of the HMD 100 can be extended.

When it is desired to further reduce the power consumption of the HMD 100 or when priority is given to the operation by the user interface other than the gesture operation, the execution of the gesture detection process by the processor 121 or the energization to the proximity sensor is stopped to detect the gesture operation. It may be temporarily interrupted. The operation for detecting the gesture operation can be started and stopped by any means (for example, the operation unit 122).

The processor 121 may start and stop the operation for detecting the gesture operation by detecting the gesture operation. For example, the proximity sensor 105 is operated intermittently, the user US approaches the hand HD in front of the proximity sensor 105, and the hand is held as it is for a predetermined time (for example, about 1 second). 105a continuously outputs the intermittent signal during that time, so that the processor 121 that detects this can start control for detecting the gesture operation, or the proximity sensor 105 can be returned to the normal detection operation.

In the case where a sleep mode in which the entire HMD 100 is in a power saving state is provided, the above gesture operation can also be used to cancel the sleep mode. In addition, since the light receiving unit 105a of the proximity sensor 105 can detect the relative light reception amount of the detection light, a change in the light reception amount can be achieved by moving the hand HD close to or away from the proximity sensor 105. As a result, the processor 121 can determine different operations of the hand HD. It is conceivable to detect this and assign it to the operation of starting or stopping the gesture operation.

However, since the proximity sensor 105 can detect an object other than the hand HD of the user US (for example, another person's hand) from its detection characteristics, the hand HD is detected when starting or stopping the operation for detecting the gesture operation. It is also preferable to use means other than that from the viewpoint of reducing malfunctions. As an example, by providing the processor 121 with a voice recognition function, for example, a voice such as “start” and “stop” of the user US is acquired by the microphone 111B, and an output signal from the microphone generated accordingly May be activated or stopped by the processor 121 analyzing and recognizing the voice. When listening to music, the volume can be adjusted by voice recognition.

Next, screen display control processing based on gesture operations including the display device control method executed by the processor 121 will be described. In gesture recognition using a proximity sensor or camera, if the user reciprocates the hand within the detection area of the proximity sensor for continuous scrolling or page turning in one direction, it is recognized as reciprocating scrolling or reciprocating page turning. There is a risk that. After moving the user's hand from one end of the proximity sensor's detection area to the other end, once removed from the detection area, then moved to one end outside the detection area, then detected again Although it is conceivable to move from one end of the region to the other end, in this way, a complicated operation is forced on the user, and the usability deteriorates. Therefore, in the present embodiment, when it is detected that the user's hand has passed the detection area SA of the proximity sensor 105 based on the output of the proximity sensor 105, the movement direction of the user's hand is stored as the first movement direction. In addition, the screen display control according to the first movement direction is performed, and the user's hand is first at least once from the time when the first movement direction is detected until the specified time elapses. Even if the output of the proximity sensor 105 indicating that the vehicle has passed in a movement direction different from the movement direction is generated, the screen display control corresponding to the different movement direction is skipped. Accordingly, the user can easily and quickly perform the intended screen display operation such as screen scrolling and continuous screen transition without forcing the user to perform complicated operations.

Hereinafter, the screen display control process based on the gesture operation will be described in detail with reference to the drawings. First, after the gesture operation detection operation is started by the above-described operation, the processor 121 waits until the light receiving unit 105a detects detection light (in this case, detection light) in step S101 in the flowchart of FIG. When light is detected, the approach direction of the hand HD is determined from the timing of the signals in the areas RA to RD as follows in step S102.
(1) Signal rise in areas RA and RC first, followed by signal rise in areas RB and RD: The hand HD has entered in the direction from right to left.
(2) Signal rise in areas RB and RD first, followed by signal rise in areas RA and RC: The hand HD has entered in the direction from left to right.
(3) Signal rise in areas RA and RB first, followed by signal rise in areas RC and RD: The hand HD has entered in the direction from top to bottom.
(4) Signal rise in areas RC and RD first, then signal rise in areas RA and RB: The hand HD has entered in the direction from bottom to top.

In step S103, the processor 121 stores the determined approach direction of the hand HD. Next, in step S104, the processor 121 waits until the light receiving unit 105a detects no detection light. If the detection is not detected, in step S105, the processor 121 determines the direction of detachment of the hand HD from the signals of the regions RA to RD. Judgment from the timing is as follows.
(5) Signal fall in areas RA and RC first, followed by signal fall in areas RB and RD: Hand HD left in the direction from right to left.
(6) Signal fall in areas RB and RD first, followed by signal fall in areas RA and RC: Hand HD left in the direction from left to right.
(7) Signal fall in areas RA and RB first, followed by signal fall in areas RC and RD: Hand HD left in the direction from top to bottom.
(8) The signal fall in the areas RC and RD first, and then the signal fall in the areas RA and RB: The hand HD has left in the direction from the bottom to the top.

In step S106, the processor 121 determines whether or not the entry direction and the withdrawal direction of the hand HD match. If the entering direction and the leaving direction of the hand HD do not match, there is a possibility of erroneous detection of the gesture operation. In this example, the gesture operation is not detected and the process proceeds to step S123. Note that a case where the entering direction and the leaving direction of the hand HD are different may be detected and used for another control. Alternatively, the time from the entry to the withdrawal of the hand HD may be measured, and if the movement is within the determined time, it may be determined that the gesture operation is correct. For example, when a signal indicating withdrawal is not output even after one second or more from the signal indicating entry, the gesture operation may be ignored or determined as another pattern gesture operation.

On the other hand, if it is determined in step S106 that the approach direction and the withdrawal direction of the hand HD are the same, the processor 121 determines in step S107 whether the built-in lock mode timer is operating. The timer for the lock mode operates for a specified time exceeding 0 seconds (for example, 0.5 seconds) from the start to the stop, and the lock mode is set while the timer is operating. Since there are individual differences in the moving speed and behavior of the hand HD, it is possible to optimize it for each user by adjusting the prescribed time for setting the lock mode. If the specified time is too long, the time that the user who wants to scroll in the reverse direction has to wait becomes longer. Therefore, it is desirable that the maximum time is 3 seconds, preferably 1 second or less. Note that this specified time can be adjusted according to the user's preference, thereby providing a user-friendly HMD.

When the hand HD is detected for the first time, since the timer is not yet operated (the lock mode is not set), the processor 121 resets and starts the timer in step S108 to start the time measurement. Set the lock mode.

The processor 121 further determines a gesture operation in step S109. Specifically, when the processor 121 determines that the gesture operations (1) and (5) are continuously performed, the screen moves from right to left (page feed or scroll) in step S110. In this manner, the display unit 104 is controlled via the display control unit 104DR. On the other hand, when the processor 121 determines that the gesture operations (2) and (6) are continuously performed, the processor 121 displays the display via the display control unit 104DR so that the screen moves from left to right in step S111. The unit 104 is controlled. Further, when the processor 121 determines that the gesture operations (3) and (7) are continuously performed, in step S112, the processor 121 displays the display through the display control unit 104DR so that the screen moves from top to bottom. The unit 104 is controlled. If the processor 121 determines that the gesture operations (4) and (8) are continuously performed, the processor 121 displays the display via the display control unit 104DR so that the screen moves from bottom to top in step S113. The unit 104 is controlled. Thus, according to the present embodiment, the direction in which the user's hand approaches or separates from the detection area is determined according to the timing at which the plurality of light receiving areas detect the user's hand, and the direction according to the determined direction. Since the screen display is controlled, variations in control according to the signal from the proximity sensor can be increased.

The user US can view the movement of the hand HD entering and leaving in front of the user through the image display unit 104B, and can synchronize with it and perform page turning and scrolling of the image in the same direction as the movement of the hand. The desired operation can be performed intuitively, so it is easy to use.

Further, in the next step S114, the processor 121 stores the detected movement direction of the hand HD (direction passing through the detection area SA) in the RAM 126 or the register area in the processor 121 as the first movement direction, and then the flow. To step S123.

On the other hand, when the processor 121 determines that the timer is operating in step S107, since the lock mode has already been set, the process proceeds to step S115, and the movement direction of the hand HD detected immediately before step S107 is It is determined whether or not the first moving direction is different.

If the processor 121 determines that the detected moving direction of the hand HD is different from the initial moving direction, the processor 121 further determines in step S116 whether the detected moving direction of the hand HD intersects the initial moving direction. . If the processor 121 determines that the detected moving direction of the hand HD does not intersect the initial moving direction (that is, moves in the opposite direction), the processor 121 shifts the flow to step S123. That is, while the lock mode is set, even if the hand HD is moved in the direction opposite to the initial movement direction, the processor 121 skips the screen display control according to the movement direction different from the initial movement direction, and moves in the reverse direction. The page is not paged or scrolled. Accordingly, it is possible to avoid a screen display operation different from the user's intention, such as going back and forth between the previous and next pages and scrolling back and forth. In addition, while the lock mode is set (during timer operation), an icon IND (see FIG. 13) indicating the lock mode setting (during timer operation) can be displayed in the image display unit 104B. The convenience of the user US can be enhanced. If the direction of the arrow included in the icon IND indicates the initial movement direction, it is desirable because the user can recognize that the gesture operation in the other direction is ignored.

When it is determined in step S116 that the detected moving direction of the hand HD intersects the initial moving direction, the processor 121 determines that the user desires to release the lock mode, and the timer is set in step S117. Stop and release lock mode. That is, the user can release the lock mode by moving the hand HD in a direction intersecting the initial movement direction at an arbitrary timing while the lock mode is set, so that the mode can be quickly released. After that, the flow moves to step S123, and when the gesture operation detection control process is continued, the movement direction of the hand HD detected next becomes the first movement direction (No in step S107).

On the other hand, when the processor 121 determines in step S115 that the detected moving direction of the hand HD is the same as the initial moving direction, first, in step S118, the processor 121 checks the count value of the timer. By confirming the count value of the timer, the moving speed of the hand HD can be grasped, and further, the reciprocating speed when the hand HD is reciprocated can be grasped. Then, based on the speed of the reciprocation of the hand HD, the processor 121 performs a screen movement amount per operation (for example, for page control such as page turning and scrolling as screen display control according to the passing direction of the hand HD). Then, it is determined whether or not adjustment of the page feed amount or scroll amount is necessary (step S119). For example, two thresholds relating to the moving speed of the hand HD are set in advance, and it is determined whether the moving amount of the screen needs to be adjusted based on a comparison between the moving speed of the hand HD and these thresholds. The movement speed is compared with the previous movement speed of the hand HD to determine whether or not the adjustment of the movement amount of the screen is necessary. If necessary, the movement amount is adjusted in step S120. For example, when the speed of reciprocation of the hand HD is equal to or higher than a first threshold, the page feed amount and scroll amount are made larger than the standard value, and when the speed is less than the second threshold, the page feed amount and scroll amount are made standard. Make it smaller than the value. Alternatively, when the change with respect to the previous movement speed is positive, the page feed amount or scroll amount is increased according to the change amount, and when the change is negative, the page feed amount or scroll amount is decreased according to the change amount. . In step S121, the processor 121 causes the image to be paged or scrolled in the first movement direction. As is clear from the above, while the lock mode is set, as shown in FIG. 13, the user shakes the hand HD to the left and right while passing through the detection area SA, for example, so that only the direction in which the hand HD first passes is shown. Since the image can be repeatedly paged and scrolled, display control according to the intention of the user US can be performed at high speed. That is, the user US only needs to reciprocate the hand HD in the same direction and in the opposite direction in order to continuously page and scroll the image in one direction. There is no need to perform operations, reducing the burden on the user. Further, by increasing the speed of shaking hands and fingers, the number of pages to be advanced by one operation and the width of scrolling can be increased, so that the operability is further improved.

The processor 121 further resets the timer in step S122 and restarts the time measurement to reset the lock mode until the next specified time is over. As a result, as long as the user desires, it is possible to repeatedly feed and scroll the image. However, in order to prevent malfunction, an upper limit of the number of repetitions may be provided, and the screen display control may be stopped when the upper limit is exceeded.

The processor 121 returns the flow to step S101 and continues control unless a signal instructing the end of the gesture operation detection operation is input in step S123. On the other hand, when determining in step S123 that the user US has input a signal for instructing the end of the gesture operation detection operation, the processor 121 ends the screen display control process by the gesture operation.

Since gesture operations are likely to be carried out continuously, it usually waits for detection of the next gesture operation. However, if the user US forgets the stop operation, it is consumed if the non-operation time is long. You may make it stop detection of gesture operation for electric power reduction. In addition, the lock mode can be turned on / off by operating the operation unit 122. When the lock mode is set to ON, the lock mode can be set according to the flowchart of FIG. 12, and when the lock mode is set to OFF, the timer is forcibly set to the non-operating state so that the lock mode is not set.

The gesture operation detection control process performed by the processor 121 will be specifically described with reference to a schematic time chart shown in FIG. As shown in FIG. 14A, when the user shakes his / her hand from side to side (see FIG. 13), the sensor 105 outputs a signal that alternately repeats on and off as shown in FIG. 14B. Here, when the hand is not detected, the signal value of the sensor 105 decreases, and when the hand is detected, the signal value increases.

Fig. 14 (c) shows the screen display control when the lock mode is turned on. At time T1 when the sensor 105 detects the movement of the hand from the right to the left from the initial home screen, the timer is operated to set the lock mode, and the home screen is moved to the left to move to the next screen. (First screen) is displayed.

Next, at the time T2 when the sensor 105 detects the movement of the hand from left to right, since the lock mode is set, the first screen remains displayed. Further, at time T3 when the movement of the hand from the right to the left is detected by the sensor 105, before the specified time has elapsed, the timer is reset to extend the lock mode and the first screen is moved to the left. Move to display the next screen (second screen). Next, at time T4 when the sensor 105 detects the movement of the hand from left to right, since the lock mode is set, the second screen remains displayed. Thereafter, screen feed is continuously performed in one direction until a timeout occurs.

Fig. 14 (d) shows the screen display control when the lock mode is set to OFF. From the initial home screen, at time T1 when the sensor 105 detects the movement of the hand from right to left, the home screen is moved leftward to display the next screen (first screen). Next, at time T2 when the sensor 105 detects the movement of the hand from left to right, the first screen is moved rightward to display the original home screen. Further, at time T3 when the sensor 105 detects the movement of the hand from right to left from the home screen, the home screen is moved to the left to display the next screen (first screen). Next, at time T4 when the sensor 105 detects the movement of the hand from left to right, the first screen is moved rightward to display the original home screen. That is, when the lock mode is set to OFF, two screens are alternately displayed in accordance with the movement of the hand.

When it is desired to repeat page turning and scrolling in one direction with the lock mode off, the hand HD moves in one direction when passing the detection area SA of the sensor 105 as shown in FIG. When returning, the gesture operation is performed so as to pass outside the detection area SA, that is, to turn the hand in front of the user.

FIG. 16 is a diagram illustrating an example of an image that is switched by a gesture operation. As for the transition between the screens, the movement of the hand up and down, left and right can be recognized as a swipe operation, and can be switched by sliding in accordance with this. Here, when moving the hand HD up and down, it is preferable to move the palm and the elbow of the user while keeping the palm and the elbow horizontally, or to move the palm up and down with the elbow as a fulcrum in order to avoid erroneous detection. The following operation is controlled by the processor 121 in accordance with a signal from the proximity sensor 105. When the HMD 100 is powered on, the home screen G11 is displayed on the display unit 104. On the home screen G11 shown in FIG. 16, the current date, temperature, humidity, and the like are displayed at the top of the screen. Here, when a gesture operation is performed by moving the hand HD from the bottom to the top, a non-display mode in which no screen display is performed is entered, and the display unit 104 is switched to the non-display screen G01. In such a state, the user US can observe only the external image through the display unit 104. From this screen display (that is, during the execution of the non-display mode), when the gesture operation is performed by moving the hand HD from top to bottom, the non-display mode is canceled and the display unit 104 is switched to the home screen G11.

Further, when the gesture operation is performed by moving the hand HD from the right to the left from the display of the home screen G11, the music display mode is displayed and the music title display screen G10 is displayed on the display unit 104. On the song name display screen G10, the title, author, and the like flowing from the speaker or earphone are displayed at the top of the screen, and the volume control VOL is displayed on the left side of the screen. Each time the user US moves the hand HD from the bottom to the top while the song title display screen G10 is displayed, the volume of the song to be played increases one by one, and conversely the hand HD from the top to the bottom. Each time you move the volume, the volume decreases by one tick and the display changes with it. In other words, if a hand approaches or moves away from the detection area while the volume control image is displayed, the direction is determined and the function (volume) associated with the volume image is adjusted, and the display is displayed along with it. Updated. From this screen display, when the hand HD is moved from left to right and a gesture operation is performed, the display unit 104 is switched to the home screen G11.

In addition, when the gesture operation is performed by moving the hand HD from the left to the right from the display of the home screen G11, the display mode 104 is displayed and the imaging field angle display screen G12 is displayed on the display unit 104. It can be seen that the user US can capture the subject within the rectangular frame displayed on the imaging angle-of-view display screen G12 with the camera 106 as a still image or a moving image. The imaging may be started / finished in accordance with the switching of the screen, or the imaging may be started / finished after the standby time has elapsed from that point. Imaging may be started / stopped by operation of the operation unit or by voice. From this screen display, when the hand HD is moved from right to left and a gesture operation is performed, the display unit 104 is switched to the home screen G11.

Also, when a gesture operation is performed by moving the hand HD from the top to the bottom from the display on the home screen G11, the screen is switched to a content browsing display screen and an image G21n of a specific page of the content is displayed. Each time the hand HD is moved from right to left on the display screen G21n and a gesture operation is performed, the display decrements the page and switches to the image G21n-1 of the previous page, and moves from left to right. Every time the HD is moved and a gesture operation is performed, the image is switched to the image G21n + 1 of the next page. When a gesture operation is performed by moving the hand HD from the top to the bottom from the display screen G21, the setting mode is displayed and the setting display screen G31 is displayed on the display unit 104. When the hand HD is moved from the right to the left on the setting display screen G31 and a gesture operation is performed, the display unit 104 switches to another setting display screen G30, or the hand HD is moved from the left to the right. When a gesture operation is performed, the display unit 104 is switched to another setting display screen G32, and each different setting can be performed using, for example, the operation unit 122. When a gesture operation is performed by moving the hand HD from the bottom to the top on the setting display screen G31, the display unit 104 is switched to the home screen G11 via the browsing screen G21. It is effective to use the above-described lock mode when, for example, the screen is continuously changed from the setting screen G30 to the setting screen G32 or when page feed is performed on the display screen G21. In addition, you may perform the movement to the vertical direction or horizontal direction for selecting a candidate item from one display screen (menu screen) containing a some selection item using the detection of gesture operation. The amount of page feed and the amount of movement of items on the menu screen may be changed according to the reciprocation speed of the hand HD as described above.

The user US can operate the screen by moving the hand HD in front of the display unit 104 while viewing the display screen of the display unit 104. Therefore, it is not necessary to move the viewpoint to view the operation unit like a device having an operation panel. It can appear to move and operate. Therefore, an easy-to-use user interface can be provided. Moreover, unlike the gesture recognition by the camera, the recognized area can be accurately grasped.

Note that the display content of each screen is merely an example, and arbitrary items may be added or replaced from various screens including images associated with other functions. Moreover, what screen should be switched according to the moving direction of the hand, how to set the animation at the time of switching may be set as appropriate according to the preference of the user.

In this embodiment, a pyroelectric sensor, which is a kind of passive sensor, is used as the proximity sensor. Instead, a light emitting unit that emits invisible light such as infrared light and a light emitting unit emit light. An active type sensor having a light receiving unit that detects invisible light reflected by an object may be used. If an active sensor is used, there is an advantage that gesture operation can be detected even when the user is wearing gloves.

(Second Embodiment)
FIG. 17 is a front view of the smartphone 200 according to the second embodiment, which is shown together with the user's hand HD. The smartphone 200 has the same configuration as that of the above-described embodiment (see FIG. 6). Here, the configuration having the same function is denoted by the same reference numeral and description thereof is omitted, but the smartphone 200 has a proximity sensor as a detection device. Instead, the camera 106 is used. However, you may provide a proximity sensor similarly to HMD100 mentioned above.

Here, it is assumed that the imaging range of the camera 106 that is the imaging apparatus constitutes the detection area SA. When executing a screen display process based on a gesture operation, the built-in processor stores in advance each frame of a video image, which is image information taken by the camera 106, and stores the shape data and color feature data of the user's hand HD. And whether or not the hand HD has entered the frame of the camera 106 is determined. After recognizing that the hand HD has entered, the processor stores the features and feature points of the hand HD as tracking targets. If there is a hand HD having the same feature in the next frame, the processor measures the moving distance, direction, and speed.

For example, when the user moves the hand HD in one direction from the right side to the left side of the screen, the processor exceeds the minimum movement distance (for example, about half the distance of the screen) for which the movement of the hand HD is specified, and If it is determined that it has moved within the specified speed range, it is determined that the hand HD has passed the detection area, and unless the timer for the lock mode is activated, the lock mode is set and this direction is set as the first moving direction. As well as screen scrolling and screen transition in the first moving direction. At this time, the hand HD does not necessarily have to pass from the right to the left outside the frame of the camera 106.

Therefore, after confirming the first movement direction in the frame of the video image, the processor moves through the video image of the camera 106 that the hand HD has moved in the direction opposite to the first movement direction, as in the above-described embodiment. Even if it is detected, screen scrolling, screen transition, etc. are not performed in the reverse direction. Furthermore, when the processor detects through the video image of the camera 106 that the hand HD has moved in the direction opposite to the initial movement direction, it is possible to cause the screen to scroll or transition in the first movement direction. .

However, if the moving speed of the hand is too fast, it will be impossible to recognize the hand reliably in the frame image, and if it is slow, it can be regarded as another action, so it is desirable to define the speed range. Although the hand recognition rate depends on the frame rate of the camera, a rate of about 15 fps or more is recommended.

The present invention is not limited to the embodiments described in the present specification, and includes other embodiments and modifications based on the embodiments and technical ideas described in the present specification. It is obvious to The description and embodiments herein are for illustrative purposes only, and the scope of the present invention is indicated by the following claims.

100 HMD
101 Frame 101a Front part 101b Side part 101c Side part 101d Long hole 101e Long hole 102 Eyeglass lens 103 Main body part 104 Display 104A Image forming part 104B Image display part 104DR Display control part 104a Light source 104b Unidirectional diffuser 104c Condensing lens 104d Display element 104f Eyepiece prism 104g Deflection prism 104h Hologram optical element 104i Screen 105 Proximity sensor 105a Light receiving part 106 Camera 106a Lens 107 Right sub-main part 107a Protrusion 108 Left sub-main part 108a Protrusion 109 Acceleration sensor 110 Gyro 111A Speaker (or earphone)
111B Microphone 121 Processor 122 Operation unit 123 GPS reception unit 124 Communication unit 125 ROM
126 RAM
127 Battery 200 Smartphone CD code CTU Control unit HD Hand HS Wiring IND Icon PL1 Base end surface PL2 Inner surface PL3 Outer surface PL4 Inclined surface PL4, PL5 Inclined surface RA-RD Light receiving area SA Detection area EV Effective field US User

Claims (14)

1. A display unit having a display member having a screen capable of displaying an image;
   A detection device having a detection region and generating an output by detecting a moving direction in which the pointing unit moved by the user moves in the detection region;
   A control device for controlling the screen display of the display unit based on the output of the detection device,
   When the control unit detects that the pointing unit has moved in the detection area based on the output of the detection device, the control unit sets the moving direction of the pointing unit as the first moving direction and responds to the first moving direction. Screen display control,
   Furthermore, the timer starts counting from the time of detection of the first movement direction, and the instruction unit moves the first movement at least once during the operation of the timer from the time measurement until the specified time elapses. A display device that skips screen display control in accordance with the different movement direction even if an output of the detection device that indicates movement in a movement direction different from the direction is generated.
2. When the control device generates an output of the detection device indicating that the instruction unit has moved in the first movement direction during the operation of the timer, the first movement direction in which the instruction unit has moved is generated. The display device according to claim 1, wherein screen display control is performed in accordance with the display.
3. The control device resets the timing result of the timer when the output of the detection means indicating that the instruction unit has moved in the same direction as the initial movement direction is generated during the operation of the timer. In addition, when the timer is operated again and the output of the detection device indicating that the instruction unit has moved in a movement direction different from the initial movement direction is generated during the operation of the timer, the different movement directions The display device according to claim 1, wherein the screen display control according to is skipped.
4. The display device according to any one of claims 1 to 3, wherein the control device causes the display unit to display that the timer is operating.
5. The movement of the instruction unit is associated with the movement of the screen of the display unit,
   The detection of the instruction unit according to a time from when the first movement direction is detected until an output of the detection device indicating that the instruction unit has moved in the same movement direction as the first movement direction is generated. The display device according to any one of claims 1 to 4, wherein the amount of movement of the screen per operation corresponding to movement in the area is adjusted.
6. The control device determines in advance a time from when the first moving direction is detected until an output of the detecting device is generated indicating that the instruction unit has moved in the same moving direction as the first moving direction. The display device according to claim 5, wherein the amount of movement of the screen per operation is adjusted based on the comparison result.
7. The detection device can detect movement of the instruction unit in a plurality of directions intersecting each other in the detection area, and the control unit can detect whether the instruction unit is in the first movement direction during the operation of the timer. When the output of the detection device indicating movement in the intersecting direction is generated, the time measurement by the timer is stopped, and the output of the detection means indicating that the instruction unit has moved after the time measurement by the timer is stopped is The display device according to any one of claims 1 to 6, wherein when generated, the screen control is performed in accordance with movement of the instruction unit.
8. The display device according to any one of claims 1 to 7, wherein the detection device is an imaging device that images the instruction unit and outputs image information to the control device.
9. The display device according to any one of claims 1 to 7, wherein the detection device includes a light receiving unit that detects detection light emitted from the instruction unit or reflected by the instruction unit and outputs a signal to the control unit. .
10. 10. The display device according to claim 1, wherein the specified time is adjustable.
11. A mounting member that is mounted on the head of the user and holds the display unit so that the display member is positioned in front of at least one eye of the user, and the detection device is held by the mounting member The display device according to any one of claims 1 to 10.
12. The display device according to claim 11, wherein the display member of the display unit is a see-through type.
13. A display unit having a display member having a screen capable of displaying an image; and a detection device having a detection area and generating an output by detecting a moving direction in which the pointing unit moved by the user moves in the detection area. In the display device, the display device control method for controlling the screen display of the display unit based on the output of the detection device,
    When the instruction unit detects that the detection unit has moved in the detection area based on the output of the detection device, the movement direction of the instruction unit is set as the first movement direction, and screen display control according to the first movement direction is performed. Done
    Further, the indication unit indicates that the instruction unit has moved in a movement direction different from the first movement direction at least once from the time when the first movement direction is detected until the specified time elapses from the start of timing. A control method for a display device, wherein screen display control corresponding to the different movement directions is skipped even if an output of the detection device is generated.
14. A display unit having a display member having a screen capable of displaying an image; a detection device having a detection region; and a detection unit that generates an output by detecting a moving direction in which the pointing unit moved by the user moves in the detection region; and the detection A control device for controlling the screen display of the display unit based on the output of the device, and a display device control program comprising:
    In the control device,
    When the instruction unit detects that the detection unit has moved in the detection area based on the output of the detection device, the movement direction of the instruction unit is set as the first movement direction, and screen display control according to the first movement direction is performed. Let
    Further, the indication unit indicates that the instruction unit has moved in a movement direction different from the first movement direction at least once from the time when the first movement direction is detected until the specified time elapses from the start of timing. A display device control program for skipping screen display control in accordance with the different movement directions even when an output of a detection device is generated.

Images