JP2013083731A - Image display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to start image display without a switch operation and to prevent the image display from starting against the user's will, in an image display system including a head-mounted display.SOLUTION: An image display system includes: a head sensor 11 and a body sensor 12 which are put on the head and the body of a user respectively and detect turning movements of the head part and the body part respectively; a camera 13 which detects a viewing direction of the user; an image display unit 14 which displays an image within a visual field of the user; and a turning angle determination part 21 which obtains turning angles θand θof the head of the user on the basis of detection results of the head sensor 11 and the body sensor 12. The image display unit 14 starts image display on the basis of the viewing direction and the turning angles θand θof the head of the user.

Description

  The present invention relates to an image display system. More specifically, the present invention relates to an image display system in which an image display unit that displays an image in a user's field of view is mounted on the user's head, for example, a head mounted display (HMD). Regarding improvements.

  A so-called head-mounted display (hereinafter referred to as “HMD”) is an image display terminal that is attached to a user's head and visually recognizes an image from a close range. For example, a half mirror is arranged in the vicinity of the user's eyes, and an image displayed on the liquid crystal panel is reflected by the half mirror, so that the external light transmitted through the half mirror is superposed on the user's field of view. A transmissive HMD that displays an image is known.

  By using the HMD, the user can view the display image while moving. In addition, you can hold the terminal by hand like a general portable display terminal, and it is not necessary to turn your face to the terminal when browsing, so you can browse images without interrupting the work if you are working be able to. For this reason, if the related information of work is displayed using HMD, the work efficiency can be improved. For example, when performing a picking operation in a warehouse, an operator can wear the HMD and display information necessary for the picking operation on the HMD, thereby improving work efficiency.

  Patent Document 1 describes an HMD worn by a helicopter pilot. This HMD detects the orientation of the pilot's head and displays an image corresponding to the orientation of the head. For this reason, a display image can be switched by hands-free which does not require switch operation etc., and a pilot can concentrate on maneuvering. Further, not only the pilot's helmet but also the boarding machine is provided with a gyro sensor, and the difference between the gyro sensors is obtained to accurately detect the direction of the pilot's head.

JP 2001-103395 A

  In general, during image display by the HMD, the user's attention is attracted to the display image, and work mistakes are likely to occur. In addition, since the HMD reduces the visibility of the surrounding environment by displaying an image, it is not preferable to display an image while the user is moving from the viewpoint of ensuring safety. Therefore, it is conceivable that the user who is working controls whether or not to display an image. Also in this case, if it is possible to perform hands-free control without performing a switch operation or the like, work efficiency can be improved.

  However, when the conventional HMD is improved and configured to control the presence / absence of image display according to the orientation of the head, image display is started against the will of the user, causing work errors or safety. There is a risk of damage. For example, when the user turns around to check the surrounding environment, if the HMD starts image display according to the movement of the head, the visibility of the surrounding environment is reduced and the display image is noticed by the user. Since it is attracted, it is considered that the convenience of the HMD is remarkably lowered.

  The present invention has been made in view of the above circumstances, and an object thereof is to suppress the start of image display against the user's will in an image display system including an HMD. In particular, it is possible to start image display based on the user's intention without requiring a switch operation, and to suppress the start of image display against the user's intention.

  Another object of the present invention is to improve the working efficiency of a user who performs work while wearing an HMD. Furthermore, it aims at improving the safety | security of the user who mounts | wears and moves.

  An image display system according to a first aspect of the present invention is mounted on a user's head and displays an image in an information display area within the user's field of view, and is mounted on the head and rotates the head. A head sensor for detecting movement; a line-of-sight detection unit for detecting the user's line-of-sight direction; and a swing angle determination unit for determining the swing angle of the user based on the detection result of the head sensor; The image display unit is configured to start image display in the information display area based on the line-of-sight direction and the swing angle.

  According to such a configuration, in order to start displaying an image in the information display area, the user does not need to interrupt the work and perform a switch operation or the like, and work efficiency can be improved. Further, by determining whether or not to start image display based on the line-of-sight direction and the swing angle, it is possible to prevent image display from being started against the user's will and reducing work efficiency. Further, by suppressing unnecessary image display, it is possible to improve the safety of the user who is working.

  The rotation operation of the user's head detected by the head sensor may be a vertical rotation operation or a horizontal rotation operation. The information display area may be the entire display area where the image display unit displays an image, or may be a part of the display area.

  In the image display system according to the second aspect of the present invention, in addition to the above configuration, the image display unit is in a display start area in which the swing angle is larger than the image display angle and the line-of-sight direction does not include the front direction. The image display in the information display area is started.

  With such a configuration, it is possible to prevent the image display from starting when the user is facing the front direction. Further, even when the user swing angle exceeds the image display angle, image display is not started unless the line-of-sight direction is within the display start area. For this reason, it is possible to prevent image display from being started against the user's will.

  An image display system according to a third aspect of the present invention includes, in addition to the above configuration, a swing direction determination unit that determines a swing direction of the user based on a detection result of the head sensor, and the image display unit includes: When the swing angle is larger than the image display angle and the line-of-sight direction is opposite to the swing direction, the image display in the information display area is started.

  With such a configuration, it is possible to prevent the image display from starting when the user is facing the front direction. Even when the user's swing angle exceeds the image display angle, the image display is not started unless the line-of-sight direction is opposite to the swing direction. In general, the movement in which the line-of-sight direction and the swing direction are opposite to each other is an unnatural movement that must be performed consciously. Therefore, by starting image display based on such a condition, the user can It is possible to prevent the image display from being started contrary to the intention.

  An image display system according to a fourth aspect of the present invention includes, in addition to the above configuration, a torso sensor that is mounted on the torso of the user and detects the rotation of the torso, and the swing angle determination unit includes the head sensor And the said swing angle regarding the left-right direction is calculated | required based on the detection result of the said fuselage sensor.

  With such a configuration, it is possible to accurately detect the swinging motion in the horizontal direction by removing the rotational motion in the horizontal direction of the body from the rotational motion in the horizontal direction of the head detected by the head sensor. . For this reason, compared with the case where only the detection result of the head sensor is used, the swing angle of the user in the left-right direction can be obtained with high accuracy, and image display is prevented from starting against the user's will. Can do.

  In addition to the above configuration, the image display system according to the fifth aspect of the present invention is configured such that the swing angle determination unit obtains the swing angle in the vertical direction based on the detection result of the head sensor.

  With such a configuration, the user can start screen display in the information display area by rotating the head up and down so that the swing angle is larger than the image display angle.

  An image display system according to a sixth aspect of the present invention includes, in addition to the above configuration, a swing direction determination unit that determines a swing direction of the user based on a detection result of the head sensor, and the image display unit includes: An image corresponding to the swing direction is displayed.

  With such a configuration, the user can select an image displayed in the information display area by selecting a swing direction.

  In the image display system according to the seventh aspect of the present invention, in addition to the above configuration, the line-of-sight detection unit captures the user's eyes and a pupil position determination unit that determines the position of the pupil in the captured image. And the line-of-sight direction is determined from the position of the pupil. With such a configuration, the user's line-of-sight direction can be detected.

  In an image display system according to an eighth aspect of the present invention, in addition to the above configuration, the image display unit includes an image output unit that outputs the image, and a half mirror disposed in the vicinity of the user's eye, The light emitted from the image output unit reflected by the half mirror is superimposed on the external light transmitted through the half mirror, and is incident on the eye.

  With such a configuration, it is possible to visually recognize the surrounding environment to some extent even during image display by displaying the image display superimposed on the surrounding environment that can be normally visually recognized. In addition to this, by suppressing unnecessary image display, the user can pay attention to the surrounding environment, and the surrounding environment can be more visually recognized. For this reason, while improving work efficiency, the safety of the user during work can be improved.

  An image display system according to a ninth aspect of the present invention, in addition to the above configuration, identifies a voice command from a microphone to which the user's voice is input and the voice input during the image display in the information display area. A voice command identification unit for generating a signal, and configured not to generate the input signal from the voice input while the image is not displayed in the information display area.

  With such a configuration, it is possible to accept a voice command if an image is displayed in the information display area, and not accept a voice command if no image is displayed. Normally, a user inputs a voice command by looking at the image display. If the voice command is not accepted when the image is not displayed, noise during operation is erroneously recognized as a voice command. Can be prevented.

  An image display system according to a tenth aspect of the present invention is an image display system comprising a portable display terminal and a display control device capable of wireless communication with each other, wherein the portable display terminal is mounted on a user's head and the user's field of view. An image display unit that displays an image in an information display area, a head sensor that detects the rotational movement of the head, and a line-of-sight detection unit that detects the user's line-of-sight direction, and the display control device includes: Based on the detection result of the head sensor, a swing angle discriminating unit for determining the swing angle of the user, and for controlling image display in the information display area based on the line-of-sight direction and the swing angle A display control unit that generates a display control signal, and the image display unit is configured to start image display in the information display area based on the display control signal.

  According to the image display system of the present invention, image display in the information display area in the user's field of view is started based on the line-of-sight direction and the swing angle. For this reason, when it is desired to display an image in the information display area, it is not necessary for the user to interrupt the work and perform a switch operation or the like, and work efficiency can be improved. Further, compared to a case where it is determined whether or not to start image display based only on the swing angle, it is possible to prevent image display from being started against the user's will and a reduction in work efficiency. Further, by suppressing unnecessary image display, it is possible to improve the safety of the user who is working.

  According to the image display system of the present invention, the image display unit in the information display area starts when the swing angle is larger than the image display angle and the line-of-sight direction is in the display start area that does not include the front direction. Is done. For this reason, it is possible to prevent image display from being started when the user's head is facing the front. If the line-of-sight direction is not within the display start area, the image display is not started, and the image display can be prevented from starting against the user's will.

  According to the image display system of the present invention, when the swing angle is larger than the image display angle and the line-of-sight direction is opposite to the swing direction, image display in the information display area is started. For this reason, it is possible to prevent image display from being started when the user's head is facing the front. In addition, by starting image display based on an unnatural operation, it is possible to prevent image display from being started against the user's will.

  Further, according to the image display system of the present invention, a voice command is accepted if an image is displayed in the information display area, and a voice command is not accepted if an image is not displayed. For this reason, it is possible to prevent erroneous recognition of noise during work as a voice command.

1 is a system diagram showing a configuration example of an image display system 100 according to Embodiment 1 of the present invention. It is the external view which showed a mode that the head unit 1A of FIG. 1 was seen from back. FIG. 2 is a block diagram illustrating an example of a functional configuration of the image display system 100 in FIG. 1. It is explanatory drawing about the discrimination | determination process in the swing angle discrimination | determination part 21 of FIG. It is explanatory drawing which showed the range of the swing angle where an image display is performed as a virtual area | region seen from the user. It is the figure which showed an example of the display start area | region 5 used for the discrimination | determination process of a gaze direction. It is the flowchart which showed an example of the display control process. It is the flowchart which showed an example of the audio | voice input process. It is the block which showed an example of the function structure of the image display system 101 by Embodiment 2 of this invention. It is explanatory drawing about the gaze direction discrimination | determination process by Embodiment 2 of this invention. 10 is a flowchart illustrating an example of display control processing in the image display system 101 of FIG. 9. It is the figure which showed an example of the image display by Embodiment 3 of this invention.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating a configuration example of an image display system 100 according to the present embodiment, and FIG. 2 is an external view illustrating a state in which the head unit 1A of FIG. 1 is viewed from the rear. The image display system 100 includes an HMD 1 worn by a user and a management server 2 connected to the HMD 1 via wireless communication.

  The HMD 1 is a portable display terminal that is worn on the user's body and displays an image received from the management server 2 within a user's field of view from a close range. In addition, the user's line-of-sight direction and head movement are detected, and a detection signal is transmitted to the management server 2. The management server 2 receives these detection signals and performs display control of the HMD 1. In other words, the management server 2 determines whether or not to perform image display based on the detection signal from the HMD 1. When performing image display, the management server 2 transmits a display control signal including image data to the HMD 1. An image display based on the image data is performed.

  This image display is started when the user's line-of-sight direction and the swing angle satisfy certain conditions. For this reason, image display can be started when necessary without interrupting the operation and performing a switch operation or the like. Moreover, since the image display is started when both the condition of the line-of-sight direction and the condition of the swing angle are satisfied, it is possible to suppress the start of the image display against the user's will.

  In the present embodiment, when the user's line-of-sight direction is within the display start area and the user's head swing angle exceeds the image display angle, image display by the HMD 1 is started. For this reason, it can suppress more effectively that an image display is started contrary to a user's will compared with the case where an image display is started only based on a user's swing operation.

  The HMD 1 includes a head unit 1A that is mounted on the user's head and a body unit 1B that is mounted on the user's body. Here, the case where the head unit 1A and the torso unit 1B are connected to each other via the communication cable 32 will be described, but they may be connected to each other via a wireless communication line such as Bluetooth.

  The head unit 1 </ b> A includes a glasses-type frame 30, a main module 31, a half mirror 142, and a microphone 15. The spectacles type frame 30 has a shape similar to that of general spectacles, and is a mounting means for mounting the head unit 1A on the user's head. A camera 13, a microphone 15, a main module 31, and a half mirror 142 are attached to the eyeglass frame 30.

  The camera 13 is a line-of-sight detection unit that captures the user's eyes in order to detect the user's line-of-sight direction, and is disposed on the user side of the lens frame of the eyeglass-type frame 30. For example, a CCD camera or a CMOS image sensor can be used as the camera 13. By identifying the position of the pupil in the captured image output by the camera 13, the user's line-of-sight direction can be determined.

  The microphone 15 is a voice input unit for inputting user's voice. When a voice is input to the microphone 15, a voice command included in the voice is identified. Therefore, the user can perform input without performing a switch operation or the like.

  The main module 31 is disposed in the vicinity of the user's left eye. The main module 31 incorporates an image output unit 141 and a head sensor 11. The image output unit 141 is an image output unit that outputs an image based on the image data received from the management server 2. For example, a liquid crystal panel is used. The head sensor 11 is a sensor for detecting the vertical movement and the horizontal movement of the user's head. For example, a well-known gyro sensor is used.

  The half mirror 142 is attached to the main module 31 and arranged in the visual field of the user's left eye. Part of the external light incident from the user's surrounding environment passes through the half mirror 142 and enters the left eye of the user. The output light from the image output unit 141 is reflected by the half mirror 142 and enters the left eye of the user. That is, both external light and output light are incident on the user's eyes, and the image of the image output unit 141 is superimposed on the external light and displayed in the user's field of view. However, in general, the visibility of the surrounding environment is significantly reduced during image display by the HMD 1.

  The body unit 1B is connected to the head unit 1A via the communication cable 32, and is a controller that controls the head unit 1A. For example, the body unit 1B is attached to a user's waist. The body unit 1B is provided with an antenna for communicating with the management server 2 and has a body sensor 12 incorporated therein.

  The body sensor 12 is a sensor for detecting the rotational movement of the user's body in the left-right direction. For example, a well-known gyro sensor is used. By using the rotation operation in the left-right direction detected by the head sensor 11 and the rotation operation in the left-right direction detected by the body sensor 12, the swing angle of the user in the left-right direction can be accurately determined.

  FIG. 3 is a block diagram illustrating an example of a functional configuration of the image display system 100 in FIG. 1, and illustrates detailed configuration examples of the HMD 1 and the management server 2. The HMD 1 includes a communication unit 10, a head sensor 11, a body sensor 12, a camera 13, an image display unit 14, and a microphone 15. The management server 2 includes a communication unit 20, a swing angle determination unit 21, a swing direction determination unit 22, a gaze direction determination unit 23, a voice command identification unit 25, and a display control unit 26.

  The communication units 10 and 20 are communication means for performing wireless communication, and the HMD 1 and the management server 2 are connected via a wireless communication line. For example, the management server 2 is connected to a wireless LAN access point, and the HMD 1 and the management server 2 are connected to each other via the wireless LAN by performing wireless communication with the access point. Is done. Here, it is assumed that the detection signals S1 to S4 are transmitted from the HMD 1 to the management server 2 and the display control signal C1 is transmitted from the management server 2 to the HMD 1 via the communication units 10 and 20.

The head sensor 11 detects the rotation of the user's head and outputs a detection signal S1. Here, using two gyro sensors, an angular velocity ω _V of the vertical rotation operation and an angular velocity ω _H1 of the horizontal rotation operation are obtained, and a detection signal S1 including these angular velocities ω _V and ω _H1 is obtained. Is generated.

The torso sensor 12 detects the rotation operation of the user's torso and outputs a detection signal S2. Here, the angular velocity ω _H2 of the rotational movement in the left-right direction is obtained using one gyro sensor, and the detection signal S2 including this angular velocity ω _H2 is generated.

  The camera 13 captures the user's eyes and outputs a detection signal S3 including the captured image. The image display unit 14 includes an image output unit 141 and a half mirror 142. The image output unit 141 generates an image based on a display control signal C1 from the display control unit 26, and the half mirror 142 displays the image as a user. Display within the field of view. The microphone 15 converts the user's voice into an electrical signal and generates a detection signal S4 including the voice signal.

  The swing angle discriminating unit 21 obtains a user's swing angle based on the detection signals S1 and S2, and compares these swing angles with a predetermined image display angle. In this image display system 100, the image display condition is that a swing operation with a swing angle larger than the image display angle is performed. For this reason, the swing angle determination unit 21 determines whether or not this start condition is satisfied, and outputs the determination result to the display control unit 26.

The swing angle is an angle formed by the front direction of the user's face with respect to the reference direction, and the swing angle determination unit 21 obtains swing angles θ _V and θ _H in the vertical direction and the horizontal direction, respectively.

Swing angle theta _V in the vertical direction, the front direction of the face in a vertical plane is defined as the angle formed with respect to the reference direction. Here, the horizontal direction as a reference direction, the front direction of the face is the angle formed with respect to the horizontal direction, it assumed to be determined as a vertical direction of the swing angle theta _V. Such a vertical swing angle θ _V can be obtained by integrating the vertical angular velocity ω _V detected by the head sensor 11. If the head sensor 11 is an angle sensor, the head swing angle θ _{V in the} vertical direction is directly obtained by the head sensor 11.

The horizontal swing angle θ _H is defined as an angle formed by the front direction of the face with respect to the reference direction in the horizontal plane. Here, a body in the front direction reference direction of the user, the front direction of the face is the angle formed relative to the fuselage in the front direction, it assumed to be determined as a left-right direction of the swing angle theta _H. Such a lateral swing angle θ _H is obtained by integrating the difference between the lateral angular velocity ω _V1 detected by the head sensor 11 and the lateral angular velocity ω _V2 detected by the body sensor 12. .

The head swing direction determination unit 22 determines the user swing direction based on the detection signals S1 and S2. In the image display system 100, an image corresponding to the swing direction is displayed. For this reason, the swing direction determination unit 22 determines the swing direction and outputs the determination result to the display control unit 26. In this case, the swing direction determination unit 22 determines which of the four directions (up, down, left, and right) the head is swung. Such determination can be performed by comparing the vertical swing angle theta _V to swing angle determination unit 21 has determined, the absolute value of the lateral direction of the swing angle theta _H.

  The line-of-sight direction determination unit 23 determines whether the user's line-of-sight direction satisfies the image display disclosure condition based on the detection signal S3 from the camera 13. In this image display system 100, the image display start condition is that the line-of-sight direction is within a predetermined display start area. Therefore, the line-of-sight direction determination unit 23 obtains the user's line-of-sight direction, determines whether the line-of-sight direction is within the display start area, and outputs the determination result to the display control unit 26. Here, it is assumed that the image of the user's pupil is identified from the captured image of the camera 13 and the line-of-sight direction of the user is obtained based on the position of the pupil in the captured image.

  The voice command identification unit 25 generates an input signal based on the detection signal S4 from the microphone 15. In the image display system 100, when a voice is input during image display, a voice command is extracted from the voice and an input process corresponding to the voice command is performed. Therefore, when the detection signal S4 is input, the voice command identification unit 25 identifies a voice command included in the voice signal from the microphone 15 and generates an input signal corresponding to the voice command. For example, feature data of a voice command is given in advance, and by matching the voice signal with the feature data, the voice command is extracted from the voice signal and an input signal corresponding to the voice command is generated.

  However, such voice command identification processing is performed for voice input during image display, and is not performed for voice input during image non-display. In general, since voice input is performed during image display, by not accepting voice input while images are not displayed, noise other than the user's voice can be prevented from being erroneously recognized as a voice command. . Here, it is assumed that the voice command identification unit 25 performs voice command identification processing based on the display state signal C2. The display state signal C <b> 2 is a signal indicating whether or not an image is being displayed, and is generated by the display control unit 26.

  The display control unit 26 outputs a display control signal C1 and controls image display by the image display unit 14. The display control unit 26 selects, as a display image, an image previously associated with the user's swing direction based on the determination result of the swing direction determination unit 22. For this reason, an image corresponding to the swinging direction of the user is displayed by the image display unit 14. In addition, image display is performed based on an input signal from the voice command identification unit 25. For example, when the user inputs a voice command that designates a display image, the display control unit 26 selects the designated image as the display image of the HMD 1.

  Furthermore, if no image is displayed but the image is not being displayed, the display control unit 26 starts image display based on the determination results of the swing angle determining unit 21 and the line-of-sight direction determining unit 23. For this reason, in the non-display state of the image, the image display by the image display unit 14 is started when the swing angle and the line-of-sight direction of the user satisfy the predetermined conditions.

  On the other hand, if the image being displayed is being displayed, the display control unit 26 ends the image display based on the determination result of the swing angle determination unit 21. That is, if the user's swing angle is smaller than the image display angle, the image display is immediately terminated, or the image display is terminated after a predetermined time has elapsed.

FIG. 4 is an explanatory diagram of the discrimination processing in the swing angle discriminating unit 21 of FIG. 3, and (a) in the figure shows a situation when the user wearing the HMD 1 is viewed from above. The middle (b) shows the situation when the user is viewed from the right. The hatched areas in the figure are areas where the user swing angles θ _H , θ _V exceed the image display angles θ _H1 , θ _V1 , θ _V2 , and are areas where image display by the HMD 1 is performed.

Not swing angle judgment unit 21, the absolute value of the lateral direction of the swing angle theta _H compared with the image display angle theta _H1, a large swing than the image display angle theta _H1 to the left or right direction has been performed Is determined. For example, when the image display angle θ _H1 is 40 °, image display is not performed within an angle range of 80 ° across the reference direction of the front of the body, and the head is swung by 40 ° or more on either side. In this case, an image is displayed. In general, the limit angle of swinging in the left-right direction is about 60 °. Therefore, if the image display angle θ _H1 is 40 °, there is a margin of about 20 ° before the limit angle. If the head is swung in an angle range of 40 to 60 ° in any of the above, an image can be displayed.

Further, the swing angle determination unit 21 compares the vertical swing angle θ _V with the image display angles θ _V1 and θ _V2, and determines the image display angle θ _V1 in the upward direction or the image display angle θ _{V2 in the} downward direction. It is determined whether or not a large swing has occurred. For example, when the image display angle θ _V1 in the upward direction is 30 ° and the image display angle θ _{V2 in the} downward direction is 40 °, the image display is not performed within the angle range of 70 ° across the horizontal direction, and the upward direction If the head swings 30 ° or more or 40 ° or more downward, image display is performed. Generally, the upper and lower head swing limit angle is narrower in the upward direction than in the downward direction. For this reason, the image display angle θ _V1 in the upper direction is different from the image display angle θ _{V2 in the} lower direction.

In the present embodiment, an example in which the left and right image display angles θ _H1 are matched will be described. However, in consideration of the user's operation during work, the image display angles θ _H1 are set to each other. It can be different. Here, an example in which the image display angles θ _V1 and θ _V2 in the upper direction and the lower direction are different from each other will be described. However, they can be matched as necessary.

  FIG. 5 is an explanatory diagram showing the range of the swing angle in which image display is performed as a virtual region viewed from the user, and is shown three-dimensionally by combining (a) and (b) of FIG. It corresponds to the figure.

  A reference direction 40 in the figure is a front direction of the user when the head is not swung, that is, a direction in which the left-right direction is the front direction of the trunk and the vertical direction is the horizontal direction. The non-display zone 41 is a virtual area where the image display by the HMD 1 is terminated when the user turns the face, and the display zone 42 continues the image display by the HMD 1 when the user turns the face, or If the condition of the line-of-sight direction is satisfied, this is a virtual region for starting image display by the HMD 1 and is shown with hatching.

The non-display zone 41 is a rectangular area including the reference direction 40. The height of the rectangular area corresponds to the sum of the image display angles in the vertical direction (θ _V1 + θ _V2 ), and the width corresponds to the sum 2θ _H1 of the image display angles in the horizontal direction.

  On the other hand, the display zone 42 is a rectangular frame region formed around the non-display zone 41. The display zone 42 includes four display zones 42a to 42d having different swing directions. That is, this is a display zone when the swing direction is the left direction, 42a is the right direction, 42b is the right direction, 42c is the upward direction, and 42d is the downward direction, and an image corresponding to the swing direction of the user is displayed.

  FIG. 6 is a diagram illustrating an example of the display start area 5 used for the line-of-sight direction determination process. In this figure, the display start area 5 is shown as the lower left area in the left lens. The line-of-sight direction determination unit 23 determines whether or not the user's line-of-sight direction is within the display start area 5. If the line-of-sight direction is not within the display start area 5, the image display start condition is not satisfied, and if the line-of-sight direction is within the display start area 5, the image display start condition is satisfied. Here, an example in which the display start area 5 is the lower left area in the left lens is shown, but the display start area 5 may be a predetermined area that does not include at least the front direction. It can be arbitrarily determined.

  Steps S101 to S108 in FIG. 7 are flowcharts showing an example of display control processing in the image display system 100 in FIG. This display control process is a process that is started when the rotation of the user's head is detected, and is executed based on the detection signals S1 and S2 from the head sensor 11 or the torso sensor 12.

First, the swing angle discriminating unit 21 obtains a horizontal swing angle θ _H and a vertical swing angle θ _V based on the detection signals S1 and S2 (step S101). Next, the swing angle determination unit 21 determines whether the user's face is facing the non-display zone 41 or the display zone 42 based on the determined swing angles θ _H and θ _V (step S40). S102). That, together with comparing the absolute value of the lateral direction of the swing angle theta _H and horizontal direction of the image display angle theta _H1, comparing the vertical swing angle theta _V image display angle theta _V1 in the horizontal _direction, and theta _V2 . As a result, if any of the swing angles θ _H and θ _V is larger than the image display angle, it is determined that the user's face is facing the display zone 42. On the other hand, if the swing angles θ _H and θ _V are both equal to or smaller than the image display angle, it is determined that the user's face is facing the non-display zone 41.

  As a result, if it is determined that the user's face is facing the non-display zone 41, the image is not displayed and the display control process is terminated (step S103). That is, if the user swing angle is smaller than the image display angle, the image display is terminated.

  On the other hand, when it is determined in step S102 that the user's face is facing the display zone 42, the operation differs based on the display state of the HMD 1 (step S104). That is, if an image already displayed is being displayed, the process proceeds to step S107. On the other hand, if the image is not being displayed and the image is not being displayed, the process proceeds to step S105, and the line-of-sight direction determination process is performed as an image display start condition.

  When the HMD 1 is not displaying an image, the line-of-sight direction determination unit 23 obtains the user's line-of-sight direction based on the detection signal S3 from the camera 13 (step S105). Next, the line-of-sight direction determination unit 23 determines whether or not the obtained line-of-sight direction is within the display start area 5 (step S106).

  As a result, if it is determined that the user's line-of-sight direction is outside the display start area 5, the display control process ends. That is, the image display is not started, and the HMD 1 maintains the non-display state. On the other hand, if it is determined that the user's line-of-sight direction is within the display start area 5, the display control unit 26 generates a display control signal for starting image display based on the determination result, and the HMD 1 Transition from the non-display state to the image display state.

  If an image is being displayed in step S104, or if the line-of-sight direction is within the display start area 5 in step S106, the swing direction determination unit 22 determines the user's swing direction (step S107). . For example, it is determined which of the four directions (up, down, left, and right) the direction in which the user swings his / her head is. Based on the determination result, the display control unit 26 selects image data corresponding to the swing direction, generates a display control signal C1 including the image data, and based on the display control signal C1, the image display unit 14 performs image display corresponding to the swing direction (step S108).

  Steps S201 to S204 in FIG. 8 are flowcharts showing an example of voice input processing in the image display system 100 in FIG. The voice input process is a process that is started when the user performs voice input, and is executed based on the detection signal S4 from the microphone 15.

  When sound is input to the microphone 15, the operation is different based on the display state of the HMD 1 (step S201). That is, if the image is not displayed, the voice input process is terminated without performing anything. That is, voice input while images are not displayed is not accepted.

  On the other hand, when the image is being displayed, the voice command identifying unit 25 identifies the voice command included in the voice signal. That is, it is determined whether or not a predetermined voice command is included in the voice signal (step S202). As a result, if the voice command cannot be identified, the process ends (step S203).

  If the voice command can be extracted from the voice signal in step S203, the voice command identifying unit 25 generates an input signal corresponding to the voice command, and ends the voice input process (step S204). For example, if the identified voice command is an input for display control, the input signal is output to the display control unit 26. If the identified voice command is an input other than display control, it is output to another control unit (not shown).

  In the image display system 100 according to the present embodiment, the head sensor 11 mounted on the user's head detects the rotation operation of the user's head, and the swing angle determination unit 21 based on the detection result, The user's swing angle is obtained. Further, the line-of-sight direction determination unit 23 detects the direction of the user's line of sight using a captured image of the camera 13 mounted on the user's head. Based on these detection results, image display by the HMD 1 is started.

  For this reason, in order to start the image display, the user does not need to interrupt the operation and perform a switch operation or the like, and the work efficiency can be improved. Further, by determining whether or not to start image display based on the line-of-sight direction and the swing angle, it is possible to prevent image display from being started against the user's will and reducing work efficiency. Further, by suppressing unnecessary image display, it is possible to improve the safety of the user who is working.

  In image display system 100 according to the present embodiment, swing angle determination unit 21 compares the user's swing angle with the image display angle, and determines whether or not the swing angle is larger than the image display angle. ing. The line-of-sight direction determination unit 23 determines whether the line-of-sight direction is within the display start area 5 that does not include the front direction of the user's face. Then, image display is started based on these detection results.

  For this reason, it is possible to prevent the image display from being started when the user is facing the front direction. Even if the user's swing angle is larger than the image display angle, if the line-of-sight direction is not within the display start area, the image display is not started, and the image display is contrary to the user's will. It can be suppressed from starting.

  Further, in the image display system 100 according to the present embodiment, the torso sensor 12 mounted on the user's torso detects the rotation operation of the torso of the user, and the swing angle determination unit 21 has the head sensor 11 and the torso sensor 12. Based on the detection result, the horizontal swing angle is obtained.

  For this reason, compared with the case where only the detection result of the head sensor 11 is used, it is possible to accurately detect the swing angle in the left-right direction, and to suppress the start of image display against the intention of the user. Can do.

  In the image display system 100 according to the present embodiment, the microphone 15 converts the user's voice into a voice signal, and the voice command identifying unit 25 identifies the voice command included in the voice signal input during the image display. The input signal is generated based on the identification result. Moreover, an input signal is not generated even if voice input is performed while an image is not displayed, and voice input while an image is not displayed is not accepted.

  For this reason, in order to perform an input, the user does not need to interrupt the work and perform a switch operation or the like, and work efficiency can be improved. In addition, since voice input during image non-display is not accepted, it is possible to suppress erroneous recognition of noise or the like as a voice command.

Embodiment 2. FIG.
In the first embodiment, the image display system 100 that starts image display when the swing angle is larger than the image display angle and the line-of-sight direction is within the display start area 5 has been described. In contrast, in the present embodiment, an image display system 101 that starts image display when the swing angle is larger than the image display angle and the line-of-sight direction is opposite to the swing direction will be described. . Since the basic configuration of the image display system 101 according to the second embodiment is the same as that of the image display system 100 according to the first embodiment, only the differences between them will be described here.

  FIG. 9 is a block diagram illustrating an example of a functional configuration of the image display system 101 according to the second embodiment of the present invention. Compared to the configuration of the image display system according to the first embodiment (FIG. 3), the difference is that the swing direction determination result by the swing direction determination unit 22 is input to the line-of-sight direction determination unit 23.

  The line-of-sight direction determination unit 23 determines whether or not the user's line-of-sight direction is the direction opposite to the swing direction. Here, since the swing direction determination unit 22 determines whether the swing direction is the four directions of up, down, left, and right, the gaze direction determination unit 23 also includes the gaze direction of the four directions of up, down, left, and right. It is determined in which direction. Then, by comparing the determined line-of-sight direction with the swing direction, it is determined whether or not the directions are opposite to each other.

  FIG. 10 is an explanatory diagram of the line-of-sight direction determination process according to the second embodiment of the present invention. In the drawing, four line-of-sight direction determination areas 5a to 5d are shown as partial areas in the left lens. The line-of-sight direction determination unit 23 determines that the line-of-sight direction is the left direction if the user's line-of-sight direction is within the line-of-sight direction determination region 5a. Similarly, if it is in the line-of-sight direction determination regions 5b, 5c, 5d, it is determined that the direction is the right direction, the upward direction, and the downward direction, respectively. If it is possible to determine whether the line-of-sight direction is up, down, left, or right in this way, it is possible to determine whether the directions are opposite to each other by comparing the determination result with the swing direction. In addition, when it does not belong to any of the direction discrimination | determination parts 5a-5d, it is discriminate | determined that it is a front direction and it is judged that it is not a direction opposite to a swing direction.

  Steps S301 to S308 in FIG. 11 are flowcharts showing an example of display control processing in the image display system 101 in FIG. This flowchart is the same as the flowchart of FIG. 7 except for steps S304, S306, and S307.

  When it is determined in step S302 that the user's face is facing the display zone 42, the swing direction determination unit 22 determines the user's swing direction (step S304). For example, it is determined which of the four directions (up, down, left, and right) is the direction in which the user swings his / her head.

  Next, when the HMD 1 is displaying an image, the display image of the HMD 1 is switched to an image corresponding to the swing direction, and the display control process ends (steps S305 and S308).

  On the other hand, when the HMD 1 is not displaying an image, the line-of-sight direction determination unit 23 determines which of the four directions (up, down, left, and right) the user's line-of-sight direction is (step S306). Then, it is determined whether the determined line-of-sight direction is the direction opposite to the swinging direction (step S307).

  As a result, if it is determined that the line-of-sight direction is not the direction opposite to the swinging direction, the display control process ends. That is, the image display is not started, and the HMD 1 maintains the non-display state. On the other hand, if it is determined that the user's line-of-sight direction is opposite to the swing direction, the display control unit 26 generates a display control signal C1 for starting image display based on the determination result, and the HMD 1 , Transition from the image non-display state to the image display state.

  In the image display system according to the present embodiment, image display is started when the swing angle is larger than the image display angle and the line-of-sight direction is opposite to the swing direction. In general, the movement in which the line-of-sight direction and the swing direction are opposite to each other is an unnatural movement that must be performed consciously. Therefore, by starting image display based on such a condition, the user can It is possible to prevent the image display from being started contrary to the intention.

Embodiment 3 FIG.
In the first and second embodiments, the image display systems 100 and 101 in which the user controls the presence / absence of image display by the HMD 1 based on the swing angle and the line-of-sight direction have been described. On the other hand, in the third embodiment, a case will be described in which the same control is performed on a part of the display area by the HMD 1 instead of the entire display area.

  FIG. 12 is a diagram illustrating an example of an image display according to the third embodiment of the present invention, in which an example of a display image of HMD 1 visually recognized by a user wearing the HMD 1 and performing a picking operation is illustrated. Further, (a) in the figure shows a state during image non-display, and (b) shows a state during image display.

  The HMD 1 has a rectangular image display area 60 that is visible to the user. The image display area 60 is divided into an upper constant display area 61 and a lower information display area 62.

  The constant display area 61 is a display area for displaying basic information such as the radio wave state of the wireless communication, the remaining battery level, the user name, and the work name, and is always displayed while the HMD 1 is turned on. The presence or absence cannot be controlled by the user. However, since the constant display area 61 is a relatively narrow area provided at the peripheral edge of the image display area 60, even if it is always displayed, the user's work efficiency is not lowered, and safety is also improved. Has little effect.

  On the other hand, the information display area 62 is a display area in which detailed information necessary for work is displayed, and the presence / absence of the display can be controlled by the user's head swing operation and line-of-sight direction. Since the information display area 62 occupies most of the image display area 60 and includes the center of the image display area 60, the user may display an image when necessary and may not display an image when unnecessary. desirable.

  In the image display system according to the present embodiment, the presence or absence of image display in the information display area 62 is controlled based on the user's line-of-sight direction and swing angle. That is, if the information display area is a partial area of the image display area 60 of the HMD 1, the user can perform display or non-display control for the partial area. Needless to say, the entire image display area of the HMD 1 may be the information display area 62.

  In the first to third embodiments, the case where the image display systems 100 and 101 include the management server 2 has been described. However, the present invention is not limited to such a configuration. For example, by incorporating the function of the management server 2 into the body unit 1B, an image display system that does not require the management server 2 can be realized.

  Further, by incorporating the function of the torso sensor 12 into the head unit 1A except for the torso sensor 12, the HMD 1 that does not require the torso unit 1B can be realized. In this case, if the function of the management server 2 is also incorporated into the head unit 1A, an image display system including only the head unit 1A can be realized.

  In the above embodiment, the case where the vertical and horizontal swing angles are detected, the two-dimensional orientation of the user's face is determined, and the display or non-display of the image is controlled. The invention is not limited to such a case. For example, only the horizontal swing angle can be detected, and display or non-display of an image can be controlled based on the horizontal direction of the user's face. Similarly, only the vertical swing angle can be detected, and display or non-display of an image can be controlled based on the vertical direction of the user's face.

  Moreover, although the case where the head sensor 11 and the torso sensor 12 respectively detect the angular velocities of the head and the torso has been described in the above embodiment, the present invention is not limited to such a case. For example, the rotation angles of the head and the torso may be detected.

  In the above embodiment, the image display system including the half mirror 142 has been described. However, the present invention is not limited to such a case. For example, the present invention can be applied to an image display system in which an image is visually recognized by scanning beam light on the retina.

42a to 42d Display zone 1 HMD (head mounted display)
1A head unit 1B body unit 10, 20 communication unit 11 head sensor 12 body sensor 13 camera 14 image display unit 141 image output unit 142 half mirror 15 microphone 2 management server 21 swing angle determination unit 22 swing direction determination unit 23 Line-of-sight direction determination unit 25 Voice command identification unit 26 Display control unit 30 Eyeglass-type frame 31 Main module 32 Communication cable 40 Reference direction 41 Non-display zone 42 Display zone 5 Display start area 5a to 5d Line-of-sight direction determination area 60 Image display area 61 Display area 62 Information display area 100, 101 Image display system C1 Display control signal C2 Display status signals S1 to S4 Detection signals θ _H , θ _V swing angles θ _H1 , θ _V1 , θ _V2 Image display angles ω _V , ω _H1 , ω _H2 angular velocity

Claims (10)

An image display unit that is mounted on a user's head and displays an image in an information display area within the user's field of view;
A head sensor that is mounted on the head and detects rotational movement of the head;
A line-of-sight detection unit for detecting the user's line-of-sight direction;
Based on the detection result of the head sensor, a swing angle determining unit for determining the swing angle of the user,
The image display system, wherein the image display unit starts image display in the information display area based on the line-of-sight direction and the swing angle.   The image display unit starts image display in the information display area when the swing angle is larger than the image display angle and the line-of-sight direction is in a display start area not including the front direction. The image display system according to claim 1. Based on the detection result of the head sensor, a swing direction discriminating unit for determining the swing direction of the user is provided,
The image display unit starts image display in the information display area when the swing angle is larger than the image display angle and the line-of-sight direction is opposite to the swing direction. The image display system according to claim 1. A torso sensor that is mounted on the torso of the user and detects the rotation of the torso,
4. The image display system according to claim 1, wherein the swing angle determination unit obtains the swing angle in the left-right direction based on detection results of the head sensor and the body sensor. .   The image display system according to claim 1, wherein the swing angle determining unit obtains the swing angle in the vertical direction based on a detection result of the head sensor. Based on the detection result of the head sensor, a swing direction discriminating unit for determining the swing direction of the user is provided,
The image display system according to claim 1, wherein the image display unit displays an image corresponding to the swing direction.   The line-of-sight detection unit includes an imaging unit that captures the eyes of the user and a pupil position determination unit that determines a position of a pupil in the captured image, and obtains a line-of-sight direction based on the position of the pupil. The image display system according to claim 1. The image display unit includes an image output unit that outputs the image, and a half mirror disposed in the vicinity of the user's eye,
2. The image display system according to claim 1, wherein light emitted from the image output unit reflected by the half mirror is superimposed on external light transmitted through the half mirror and is incident on the eye. A microphone to which the user's voice is input;
A voice command identifying unit that identifies a voice command from the voice input during image display in the information display area and generates an input signal;
The image display system according to claim 1, wherein the input signal is not generated from the sound input while the image is not displayed in the information display area. In an image display system comprising a portable display terminal and a display control device capable of wireless communication with each other,
The above mobile display terminal
An image display unit that is mounted on a user's head and displays an image in an information display area within the user's field of view;
A head sensor for detecting the rotational movement of the head;
A gaze detection unit that detects the gaze direction of the user,
The display control device, based on the detection result of the head sensor, a swing angle determination unit for determining the swing angle of the user,
A display control unit that generates a display control signal for controlling image display in the information display area based on the line-of-sight direction and the swing angle;
The image display unit starts image display in the information display area based on the display control signal.

Images