JP4927631B2 - Display device, control method therefor, program, recording medium, and integrated circuit - Google Patents

Description

  The present invention relates to a display device that displays information, and more particularly to a hands-free operation technique for a head-mounted display device such as an HMD (head mounted display).

  Conventionally, a hands-free operation method that eliminates the need for a user operation by hand has been proposed for a head-mounted display device such as an HMD (head-mounted display). As a typical hands-free operation method, there is a method using an eye camera (eyeball camera).

  For example, there is a method for estimating a user's point of gaze by photographing a user's eyeball with an eye camera and analyzing the moving image (for example, see Patent Document 1). According to this method, it is possible to estimate a display element viewed by the user from among a plurality of display elements displayed on a screen such as an HMD. Accordingly, the user can operate the HMD by moving the line of sight without performing a button operation or the like.

There is also a method for estimating the direction in which the user is looking using something other than an eye camera.
For example, there is a method in which an angular velocity sensor is mounted on the user's head and the direction in which the user is looking is estimated based on an output from the angular velocity sensor (see, for example, Patent Document 2). According to this method, it is possible to estimate the area that the user is viewing to some extent without using an eye camera.
JP-A-8-286141 Japanese Patent Laid-Open No. 7-24141

  As described above, there are a method using an eye camera and a method using an angular velocity sensor as a method for determining a display element viewed by the user from a plurality of display elements displayed on the HMD. When these two methods are compared, the method using the eye camera has a problem that the hardware cost of the eye camera and the light source and the calculation cost of the image processing for estimating the gazing point are high. On the other hand, the method using the angular velocity sensor has an advantage that the hardware cost and the calculation cost are relatively low, but there is a problem that the accuracy of estimating the direction in which the user is looking is relatively low. The reason for the low accuracy is that when the user moves only the eyeball without moving the head, the movement of the eyeball cannot be detected by the angular velocity sensor. As described above, according to the method using the angular velocity sensor, there is a problem that when the display element that the user is viewing is determined from the plurality of display elements displayed on the HMD, the determination cannot be performed with high accuracy.

  The present invention solves the above-described problems, and an object of the present invention is to provide a display device that enables highly accurate hands-free operation while adopting a low-cost configuration.

In order to solve the conventional problem, a display device according to the present invention is a display device that displays information to a user, and obtains a display element that is information displayed to the user. Determining means for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element; and display for displaying the display element at the determined display position Means, a detecting means for detecting the direction of change when the user's head orientation changes, a selecting means for visually selecting the display element displayed in the detected changing direction, and the selected Executing means for executing a predetermined process related to the display element; and changing means for changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction. The display means displays the display element at the changed display position, and the selection means is provided that a portion of the display element not less than a predetermined ratio does not enter an area that can be seen by the user. The display element is not selected, and the display element is selected in response to the portion of the predetermined ratio or more entering the area, and the execution unit selects the display element by the selection unit. The process assigned to the display element is executed.
In order to solve the conventional problem, a display device according to the present invention is a display device that displays information to a user, and acquires a display element that is information displayed to the user. An acquisition unit; a determination unit that determines a display position at which the user cannot see a part of the display element as the display position of the acquired display element; and the display element is displayed at the determined display position. Display means for detecting, when the head orientation of the user changes, detection means for detecting the change direction, selection means for selecting the display element displayed in the detected change direction, and the selected Execution means for executing a predetermined process related to the display element.

  With this configuration, since the user cannot see a part of the display element, the user's head naturally moves when the user looks at the display element. As a result, a display element displayed in the change direction of the head direction is selected, and a predetermined process related to the selected display element is executed. This enables a highly accurate hands-free operation while adopting a low-cost configuration that only detects the direction of change in head orientation.

  Here, the display device further includes changing means for changing a display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction, and the display means is changed. The display element may be displayed at the displayed position.

  With this configuration, for example, when the user moves the head to the right to see the display element displayed on the right side, the display element moves to the left side, and when the head is turned to the front, Since attention can be paid to the contents, selection mistakes can be reduced.

  Further, the selection means may select the display element when the display position of the display element is in a state where the user can see all of the display elements.

  With this configuration, the display element is not selected unless the user's head moves until the user can see all of the display elements. Therefore, the user can select only the display elements that the user wants to see. It becomes possible.

  Further, the selection means may select the display element when the user can see a portion where the display position of the display element is equal to or greater than a predetermined ratio of the display element.

  With this configuration, the display element is not selected unless the user's head moves until the user can see a portion (for example, most) of the display element that exceeds a predetermined ratio. It is possible to select only the display elements.

  The selection means may select the display element when the predetermined display state continues for a predetermined time or more.

  According to this configuration, since the display element is not selected unless the predetermined display state continues for a predetermined time or longer, the display element viewed by the user for a moment is not selected, and selection mistakes can be reduced.

  The determining means may determine the display position of the display element such that the display element straddles a boundary line of the user's visual field.

  With this configuration, when the display area of the display means is wider than the user's field of view, the display element is displayed across the boundary of the user's field of view, so that the user cannot see part of the display element. It becomes possible to.

  Further, the determining means may determine the display position of the display element such that the display element straddles a boundary line of the display area of the display means.

  With this configuration, when the display area of the display means is narrower than the user's field of view, the display element is displayed across the boundary of the display area of the display means, so that the user can see a part of the display element. It becomes possible to make it impossible.

  The determining means may display a part of the left side of the display element at the right end of the display area of the display means so that a part of the right side of the display element is displayed at the left end of the display area of the display means. The display means such that the lower part of the display element is displayed at the upper end of the display area of the display means, or the upper part of the display element is displayed at the lower end of the display area of the display means. The display position of the element may be determined.

  With this configuration, it is possible to easily prevent a user from seeing some of the display elements on a general rectangular display screen.

  The display device further includes a changing unit that changes a display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction. Even when the display positions of the display elements are changed, the display positions of a plurality of the display elements may be determined so that the number of selected display elements is one.

  With this configuration, the number of display elements to be displayed is one, so that it is possible to reliably select one display element intended by the user from a plurality of display elements.

  The detecting means may detect the change direction based on an output from an angular velocity sensor mounted on the user's head.

  With this configuration, it is only necessary to detect the change direction of the head direction based on the output from the angular velocity sensor, so there is no need to use an expensive eye camera, and the gaze point can be determined from the moving image captured by the eye camera. Since the image processing cost to be recognized becomes unnecessary, the overall cost can be greatly reduced.

  The execution means may execute processing assigned to the button when the display element is a button.

  With this configuration, when the user gazes at the button, the process assigned to the button is executed, so that it is possible to give a hands-free instruction to press the button.

  The execution means may start the reproduction process of the moving image when the display element is a moving image.

  With this configuration, when a user gazes at a moving image, the moving image reproduction process is started, so that it is possible to give a hands-free instruction to start the moving image reproduction.

  The display means displays a selection frame, which is a frame for selecting the display element, at a predetermined display position, and the selection means displays the display element when the display element is within the selection frame. May be selected.

  With this configuration, the display element can be selected by an easy-to-understand method called a selection frame, so that there is an effect that the usability is improved.

The display means may be mounted on the user's head.
With this configuration, the present invention can be applied to the HMD.

  Note that the present invention can be realized not only as such a display device but also as an integrated circuit including characteristic means included in such a display device, or as a characteristic feature included in such a display device. It can also be realized as a control method of a display device using means as steps, or as a program for causing a computer to execute these steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  As is clear from the above description, according to the display device according to the present invention, the user cannot see a part of the display element. It will move. As a result, a display element displayed in the change direction of the head direction is selected, and a predetermined process related to the selected display element is executed. This enables a highly accurate hands-free operation while adopting a low-cost configuration that only detects the direction of change in head orientation.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is an external view of a display device according to Embodiment 1 of the present invention. In the first embodiment, a case where the HMD 40 is employed as a display device is illustrated.

  The HMD 40 is a display device that is worn on the user's head. As shown in this figure, a small projection device 42 is placed on ordinary glasses 41. Image data, power, and the like are sent from the main body 44 to the projection device 42 through the cable 43. The image data sent to the projection device 42 is projected on a display prism mounted along the lens of the glasses 41 with a viewing angle of about 27 degrees.

  When the image data is not projected, it is very normal to see the surrounding landscape through the glasses. On the other hand, when image data is projected, the image being projected appears in the landscape.

  FIG. 2 is a configuration diagram of the display device according to Embodiment 1 of the present invention. This display device is a display device (for example, HMD 40) worn on the user's head, and functionally includes a display element determination unit 101, a display element acquisition unit 102, a boundary display determination unit 103, and detection. Unit 104, display position change unit 105, selection unit 110, execution unit 108, and display unit 109. The selection unit 110 includes a boundary display determination unit 106 and a selection element determination unit 107.

Hereinafter, each component and its relationship will be described with reference to the configuration diagram of FIG.
The display element determination unit 101 determines a display element. The display element is a component of information displayed to the user, and includes GUI (Graphical User Interface) parts such as icons, buttons, and windows, and contents such as moving images, still images, characters, and symbols. Information that is not displayed to the user, such as voice information, may be combined with the display element. The display element may be a television program, mail, homepage, clock, timetable, stock price, sign, character, advertisement / advertisement, and the like. The number of display elements determined to be displayed may be one or plural.

  The display element acquisition unit 102 is an example of an acquisition unit according to the present invention, and acquires the display element determined by the display element determination unit 101. The acquisition method may be a method of acquiring via a wired or wireless communication channel or broadcast channel, or a method of acquiring via a medium such as an optical disk or a semiconductor memory. Further, it may be acquired prior to the determination by the display element determination unit 101. When acquiring a plurality of display elements, they may be acquired simultaneously or sequentially.

  The boundary display determination unit 103 is an example of a determination unit according to the present invention, and a display position where the user cannot see a part of the display element as the display position of the display element acquired by the display element acquisition unit 102 To decide. Specifically, since the display element determination unit 101 is interposed between the display element acquisition unit 102 and the boundary display determination unit 103, the user can see a part of the display elements determined by the display element determination unit 101. The display position that cannot be determined is determined. Here, displaying the display element at a display position where the user cannot see a part of the display element is referred to as “boundary display”. In other words, the boundary display is a display in which a part of the display element is visible to the user and the remaining part is not visible. For example, by displaying only the left half of the display element at the right end of the HMD screen, or by displaying only the upper part of the display element at the lower end of the screen, the user can see some but not all Can do. On the other hand, displaying a display element at a display position where the user can see all of the display elements is hereinafter referred to as “display all”.

  Note that, when there are a plurality of display elements, the boundary display determination unit 103 determines the display position of each display element so that at least one display element is boundary-displayed. That is, not all display elements need to be displayed as boundaries. For example, when there are two display elements, a position where only one of them is displayed as a boundary may be determined. Alternatively, one may be displayed at the left end of the HMD screen and the other may be displayed in the vicinity of the right end of the screen. In addition, when there are a plurality of display elements, only one of them may be displayed, and the display position of each display element may be determined so that the others are displayed as borders. Then, when the user wants to finally select one display element, the process in the selection element determination unit 107 described later can be simplified.

  Note that the display range of the HMD (display area of the display unit 109) may not be rectangular. For example, the upper end of the HMD screen is not a horizontal straight line, but may be a curved line or a broken line with the center portion lowered. Further, by performing a display different from the display element in front of the display element, a boundary display may be realized by hiding a part of the display element.

  In addition, when the display range of HMD is wider than a user's visual field, you may make it a display element straddle a user's visual field boundary line. In this way, boundary display can be realized as in the case where the display element straddles the boundary line of the display area of the display unit 109. The visual field boundary line here may be a binocular visual field boundary line or a monocular visual field boundary line. For example, as the boundary line of the binocular visual field, the boundary line of the visual field range from the front of the face to the left and right 60 degrees, the upper 60 degrees, and the lower 75 degrees can be adopted. Further, as the boundary line of the monocular field of view, the boundary line of the visual field range from the front of the face to the left and right 100 degrees, the upper 60 degrees, and the lower 75 degrees can be adopted. In consideration of individual differences in the visual field range, different angles may be employed within a range of 20 degrees from the specific angle exemplified here.

  The “field of view” mentioned above is mainly used in the meaning of the peripheral visual field, but the boundary line may be obtained for the central visual field instead of the peripheral visual field. That is, the human visual field can be broadly divided into a central visual field and a peripheral visual field. In the central field of view, the shape of the object can be discriminated in detail and the color can be recognized. On the other hand, in the peripheral vision, the shape of the object is vague and the color is not well understood. If the display element straddles the boundary line of the central visual field of the user, a part of the display element is blurred to the user. As a result, the consciousness of seeing all the display elements in detail works, and the tendency of the user's head to move naturally increases.

  The detection unit 104 is an example of a detection unit according to the present invention, and detects the amount of head movement of the user. The user's head movement amount can be detected based on an output from an angular velocity sensor (an angular velocity sensor mounted on the display device) mounted on the user's head. Instead of the angular velocity sensor, an acceleration sensor, a geomagnetic sensor, a radio wave sensor, an ultrasonic sensor, or the like may be employed. Moreover, even if it is not attached to the head, it may be anything that can detect the movement of the head, and a camera, a sensor, an electronic tag reader, or the like may be employed. It is also possible to employ a semiconductor sensor using a semiconductor element. The user's head movement amount may be an angle change, a movement amount, or an acceleration amount thereof. A flag indicating whether or not the user's head has moved left or right may be considered as the head movement amount. That is, the head movement amount may be a value that can detect the change direction of the head direction.

  “Head orientation” refers to the direction in which the head is facing with respect to the trunk, and “head direction change direction” refers to the direction in which the head orientation has changed. For example, when the head facing the front is turned to the left, a change direction of “left direction” is detected, and when the head facing left is turned to the original front, the change direction of “right direction” is detected. Will be detected.

  The display position changing unit 105 is an example of a changing unit according to the present invention. The display position changing unit 105 detects the head movement amount detected by the detecting unit 104 so that the display element moves in a direction substantially opposite to the change direction of the head direction. The display position of the display element is changed accordingly. As a result, when the user's head moves to the right, for example, the display in the HMD moves to the left. Here, the expression “substantially reverse direction” is because the reverse direction (direction different by 180 degrees) is not necessarily required. That is, when the display element moves in a direction different from the change direction of the head direction by 90 degrees or more and 270 degrees or less, it is assumed that the display element moves in a direction substantially opposite to the change direction of the head direction.

  Note that the method of moving the display element to the left is not limited to the method of changing the display position to the left without changing the display size or shape of the display element. For example, a method of changing the left end position of the display element to the left or changing the right end position to the right by enlarging / reducing or deforming the display size of the display element may be employed. The same applies to movement to the right and movement up and down.

  The selection unit 110 is an example of a selection unit according to the present invention, and selects display elements displayed in the change direction of the head direction detected by the detection unit 104. For example, when the display position of the display element is changed to a display position where the user can see all of the display elements, the display element is selected. In order to realize such a selection process, the selection unit 110 performs a boundary display determination process and a selection element determination process. Hereinafter, the selection unit 110 that performs the boundary display determination process is referred to as a “boundary display determination unit 106”, and the selection unit 110 that performs the selection element determination process is referred to as a “selection element determination unit 107”.

  The boundary display determination unit 106 determines the boundary display state of the display element whose display position has been changed by the display position changing unit 105. The boundary display state is a state indicating whether each display element is displayed as a boundary. The value may be a binary value of “boundary display” and “display all”, or a value indicating a displayed ratio. Further, the boundary display state for one display element is not limited to one. For example, one display element may be divided into a plurality of areas and the boundary display state may be determined for each area. Further, a past boundary display state and a future boundary display state (predicted value) may be determined for one display element.

  The selection element determination unit 107 determines the selection state of the display element based on the boundary display state determined by the boundary display determination unit 106. For example, the display elements that are all displayed are determined to be “selected”, and the display elements that are boundary-displayed are determined to be “non-selected”. A display element having a large number of display portions may be determined as “selected”, and other determination criteria may be employed. Further, instead of selecting one display element, a plurality of display elements may be selected. The selection state is a state indicating whether each display element is selected by the user. The value may be a binary value of “selected” and “non-selected”, a value indicating the degree of selection, or a value indicating the possibility of selection. As described above, the display element determined to be “selected” by the selection element determination unit 107 can be said to be a display element selected by the selection unit 110.

  The execution unit 108 is an example of an execution unit according to the present invention, and a predetermined element related to the display element selected by the selection unit 110 (that is, the display element determined to be “selected” by the selection element determination unit 107). Execute the process. The predetermined processing here is various and not particularly limited, but typically means processing for changing the display operation of the display element. The change in display operation includes a change in display size, shape, position, color, brightness, depth, contents, a change in reproduction state, and a change in display presence / absence. Further, not changing these is considered as one form of changing the display operation. In addition to changing the display operation of the display element itself, the display operation of information other than the display element may be changed, for example, by displaying a frame around the display element.

  The display unit 109 is an example of the display unit according to the present invention, and displays the display element at the display position determined by the display element determination unit 101 or the display position changed by the display position change unit 105. Further, the processing result executed by the execution unit 108 is displayed. Specifically, a display unit such as an HMD, FMD (face mount display), HUD (head-up display), or glasses-type display corresponds to the display unit 109 here. The display unit 109 may be mounted on the head, such as the frontal region, temporal region, occipital region, nose, ears, forehead, face, hair, mouth, or the upper body such as the neck, shoulders, chest, or back. May be. The display method may be a display / projection method using a transmissive liquid crystal, a reflective liquid crystal, an organic EL (electroluminescence) element, or the like, or a display / projection method using an optical semiconductor or a laser.

  2 may be incorporated in one HMD or may not be incorporated. For example, all the components in FIG. 2 may be included in one HMD, or the display element acquisition unit 102 may be in another device. Each unit may be distributed on a plurality of devices. For example, the detection unit 104 and the display unit 109 may be in different devices. Further, there may be a plurality of each part in FIG. For example, there may be two display units 109. Further, a plurality of users may share each part in FIG.

Next, the operation will be described.
FIG. 3 is a diagram showing the operation of the display device according to Embodiment 1 of the present invention. In this description, it is assumed that four moving images (each of which is described below as display element 1, display element 2, display element 3, display element 4, and the like) are displayed on the HMD as four display elements at a time. explain. Specifically, of the four displayed moving images, only one moving image that the user is viewing is set to the playback state, and the remaining three moving images that are not viewed by the user are set to the paused state. explain.

(S100) The operation is started and the operation proceeds to S101.
(S101) The display element determination unit 101 determines a display element to be displayed on the display unit 109, and the display element acquisition unit 102 acquires the display element determined by the display element determination unit 101, and proceeds to the operation of S102. In this description, it is assumed that the display element determination unit 101 determines that four display elements of display element 1, display element 2, display element 3, and display element 4 are displayed. The decision criteria include suitability to users' preferences and purposes, suitability to user situations such as the external world situation, physical situation, psychological situation, social situation and interpersonal situation, HMD display characteristics, Device characteristics such as communication characteristics and information storage characteristics of the display element acquisition unit 102 or the communication path may be used.

  (S102) The boundary display determination unit 103 determines the display position where each display element is boundary-displayed, and proceeds to the operation of S103. The display position of each display element is determined by comparing the display range of the HMD with the display size of each display element.

  FIG. 4A is a diagram showing a display example in Embodiment 1 of the present invention. FIG. 4B is a diagram showing the display position management table 10 according to Embodiment 1 of the present invention. The display position management table 10 manages “display size”, “display position”, “boundary display state”, and “selected state” for each display element, and includes a boundary display determination unit 103, a display position change unit 105, The table is referred to / updated by the boundary display determination unit 106, the selection element determination unit 107, and the execution unit 108. In this description, it is assumed that the display range of the HMD is rectangular, the horizontal width is 100 pixels, and the vertical width is 70 pixels. On the other hand, each display element is also rectangular, and the display size is 40 pixels wide and 30 pixels long. Based on the HMD display range and the display size of each display element, the boundary display determination unit 103 obtains a display position so that each display element is displayed as a boundary, and the obtained display position is obtained as shown in FIG. 4B. This is added to the “display position” column of the display position management table 10. In this description, the display position of the display element 1 is such that the upper left vertex of the display element 1 is located at −10 pixels to the right (that is, 10 pixels to the left) and 0 pixels downward from the upper left vertex of the HMD display range. To decide. Similarly, the display element 2 has a position of 70 pixels to the right and 0 pixels down, the display element 3 has a position of −10 pixels to the right and 50 pixels down, and the display element 4 has a position of 70 pixels right and 50 pixels down. Decide to display to the position. As a result, as shown in FIG. 4A, each display element is displayed as a boundary.

  As described above, in order to obtain the position where the display element is displayed as a boundary, when the HMD display range or the display element is rectangular, the horizontal direction of the display element is set so that the boundary of the display range is between the left end and the right end of the display element. The position in the direction may be obtained, or the position in the vertical direction of the display element may be obtained so that the boundary of the display range fits between the upper end and the lower end of the display element. When the boundary display determining unit 103 determines the display position, the display unit 109 may display the display element before moving to the operation of S103.

  (S103) The display position changing unit 105 changes the display position of each display element according to the head movement amount detected by the detection unit 104, and proceeds to the operation of S104. In this description, the case where the head movement amount is detected using an angular velocity sensor mounted on the head will be described. From the angular velocity sensor, it is possible to acquire the angle change in the vertical direction and the horizontal direction of the head. When the display position change unit 105 acquires the angle change amount via the detection unit 104, the display position change unit 105 converts the angle change amount so as to correspond to the pixel interval of the display unit 109. For example, when the horizontal width of the HMD display range is 100 pixels and the horizontal viewing angle is 50 degrees, the conversion constant from the angle change amount to the pixel is expressed as “display range horizontal pixel count / horizontal viewing angle × (−1)”. As a result, -2 is obtained. The reason for multiplying by (-1) is to reverse the direction. For example, when 10 degrees is acquired as the angle change amount in the right direction of the head, the display position changing unit 105 moves the display element to the right by −20 pixels multiplied by −2 that is the conversion constant. That is, when the user swings his head 10 degrees to the right, the display element moves 20 pixels to the left.

  5A and 6A are diagrams showing display examples in the first embodiment of the present invention. 5B and 6B are diagrams showing the display position management table 10 according to Embodiment 1 of the present invention. 5 and 6 are the same as FIG. 4 except that the assumed scenes are different.

  That is, FIG. 4 assumes a case where the head angle change amount is 0 degree. In FIG. 5, it is assumed that the right angle change amount is −5 degrees and the display element is moved to the right by 10 pixels. In FIG. 6, it is assumed that the right angle change amount is 10 degrees and the downward angle change amount is 5 degrees as compared with FIG. 5, and the display element is moved to the right by −20 pixels and −10 pixels below. Yes. In this way, the display position changing unit 105 acquires the display position management table 10 from the boundary display determining unit 103 and adds the obtained movement amount to the display position described in the “display position” column of the display position management table 10. Change the display position with.

  In this description, the conversion constant is set to a fixed value of −2. However, depending on the shape of the display unit 109 and the pixel interval, the conversion constant may be obtained individually according to the display area without being a constant. Further, regarding the movement amount of the display element, another method may be used instead of the method of using the number of pixels obtained by multiplying the angle change amount by the conversion constant as the movement amount. For example, a method that does not move the display element if the angle changes within a certain range, or a method that increases the amount of movement of the display element if the angle changes suddenly may be used.

  If the display element is moved according to the head movement amount, the display element may be completely out of the display range. Thus, in order to avoid a problem that the display element is not visible to the user, the display element may not be moved more than a certain amount.

  In this description, the movement amounts of the four display elements are all the same, but different movement amounts may be used for each display element.

  Note that when the display position changing unit 105 changes the display position, the display unit 109 may display the display element before proceeding to the operation of S104.

  (S104) The boundary display determination unit 106 determines whether each display element is boundary-displayed, and proceeds to the operation of S105. The boundary display determination unit 106 acquires the display position management table 10 from the display position change unit 105 and adds the value of “boundary display” or “display all” to the “boundary display state” column of the display position management table 10. To do. In this description, as shown in FIGS. 4 to 6, “display all” is displayed when the entire display element is displayed, and “boundary display” is displayed when the boundary is displayed.

  Even if the display element is partially displayed at the edge of the screen, if the edge of the screen moves closer to the center of the user's field of view due to the user's head facing toward that edge, the display element It may be determined that it is not displayed.

  (S105) The selection element determination unit 107 determines the selection state of each display element, and proceeds to the operation of S106. The selection element determination unit 107 acquires the display position management table 10 from the boundary display determination unit 106 and adds a value of “selected” or “non-selected” to the “selected state” column of the display position management table 10. In this description, as shown in FIG. 4 to FIG. 6, when the boundary display state of the display element is “boundary display”, it is “not selected”, and when it is “display all”, it is “selected”.

  (S106) The execution unit 108 changes the display of the display unit 109 according to the selection state of each display element, and proceeds to the operation of S107. In this description, reproduction is started for display elements whose selection state is “selected”, and reproduction is stopped for display elements whose selection state is “non-selected”. Thereby, it is possible to reproduce only the display element (moving image) that the user is viewing.

  (S107) The execution unit 108 determines whether or not to end the operation. If it is determined that the operation is to be ended, the operation proceeds to S108. If it is determined that the operation is not to be ended, the operation proceeds to S101.

(S108) The operation ends.
Hereinafter, a specific example will be described.

  7A to 7C are diagrams showing state transition of the display unit 109 according to Embodiment 1 of the present invention. The start button 21 is an example of a display element, and is a button for instructing start of various processes. The end button 22 is an example of a display element, and is a button for instructing the end of use of the apparatus. A dotted rectangle shown in the drawing represents the display area 11 of the display unit 109.

  In the initial state, as shown in FIG. 7A, a part of the start button 21 is displayed outside the display area 11, and a part of the end button 22 is also displayed outside the display area 11. In this state, the user cannot see a part of the start button 21 and cannot see a part of the end button 22. Here, when the user wants to instruct the start of various processes, the user's head naturally turns to the left. As a result, the start button 21 and the end button 22 are moved in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). As a result, as shown in FIG. 7B, when the start button 21 is entirely within the display area 11, the start button 21 is selected. When the start button 21 is selected in this manner, as shown in FIG. 7C, a movie button 23, a music button 24, an Internet button 25, a game button 26 (hereinafter, these buttons are collectively referred to as a “submenu button”). May be displayed). A part of the submenu button is displayed outside the display area 11, and in this state, the user cannot see a part of the submenu button.

  8A to 8C are diagrams showing another state transition of the display unit 109 according to Embodiment 1 of the present invention. FIG. 7 illustrates an example in which the start button 21 is selected when all of the start buttons 21 are within the display area 11, but here, a portion of the start button 21 that is a predetermined ratio or more (for example, a portion of 90% or more) ) Exemplifies a mode in which the start button 21 is selected when it fits in the display area 11.

  That is, in the initial state, as shown in FIG. 8A, a part of the start button 21 is displayed outside the display area 11, and a part of the end button 22 is also displayed outside the display area 11. In this state, the user cannot see a part of the start button 21 and cannot see a part of the end button 22. Here, when the user wants to instruct the start of various processes, the user's head naturally turns to the left. As a result, the start button 21 and the end button 22 are moved in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). As a result, as shown in FIG. 8B, when 90% or more of the start button 21 fits in the display area 11, the start button 21 is selected. When the start button 21 is selected in this way, a submenu button is displayed as shown in FIG. 8C. A part of the submenu button is displayed outside the display area 11, and in this state, the user cannot see a part of the submenu button.

  9A to 9C are diagrams showing another state transition of the display unit 109 according to Embodiment 1 of the present invention. A dotted rectangle shown in the figure represents the display area 11 of the display unit 109, and an ellipse shown by a solid line in the figure represents the user's visual field 12. Although FIG. 7 illustrates an example in which the display position of the display element is determined so as to cross the boundary line of the display area 11, the display position of the display element is determined so as to cross the boundary line of the visual field 12 of the user. The form which is made is illustrated. Whether the display element crosses the boundary line of the display region 11 or the boundary line of the user's visual field 12 is the same in that the user cannot see a part of the display element.

  That is, in the initial state, as shown in FIG. 9A, a part of the start button 21 is displayed outside the user's visual field 12 and a part of the end button 22 is also displayed outside the user's visual field 12. In this state, the user cannot see a part of the start button 21 and cannot see a part of the end button 22. Here, when the user wants to instruct the start of various processes, the user's head naturally turns to the left. As a result, the start button 21 and the end button 22 are moved in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). As a result, as shown in FIG. 9B, when the start button 21 is entirely within the user's field of view 12, the start button 21 is selected. When the start button 21 is selected in this way, a submenu button is displayed as shown in FIG. 9C. Some of the submenu buttons are displayed outside the user's field of view 12, and in this state, the user cannot see some of the submenu buttons.

  10A to 10C are diagrams showing another state transition of the display unit 109 according to Embodiment 1 of the present invention. FIG. 9 illustrates a mode in which the start button 21 is selected when all of the start buttons 21 are within the user's field of view 12, but here, a portion of the start button 21 that is equal to or greater than a predetermined ratio (for example, 90% or more). An example is shown in which the start button 21 is selected when (part) falls within the user's field of view 12.

  That is, in the initial state, as shown in FIG. 10A, a part of the start button 21 is displayed outside the user's visual field 12 and a part of the end button 22 is also displayed outside the user's visual field 12. In this state, the user cannot see a part of the start button 21 and cannot see a part of the end button 22. Here, when the user wants to instruct the start of various processes, the user's head naturally turns to the left. As a result, the start button 21 and the end button 22 are moved in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). As a result, as shown in FIG. 10B, when 90% or more of the start button 21 falls within the user's visual field 12, the start button 21 is selected. When the start button 21 is selected in this way, a submenu button is displayed as shown in FIG. 10C. Some of the submenu buttons are displayed outside the user's field of view 12, and in this state, the user cannot see some of the submenu buttons.

  11A to 11F are diagrams showing another state transition of the display unit 109 according to Embodiment 1 of the present invention. Here, the state transition after the submenu button is displayed will be described. The submenu button is an example of a display element according to the present invention, and specifically includes a movie button 23, a music button 24, an internet button 25, and a game button 26.

  As already described, immediately after the start button 21 is selected, as shown in FIG. 11A, the user cannot see a part of the submenu buttons. Here, when the user wants to watch a movie, the user's head naturally turns to the upper left. As a result, the submenu button moves in the direction opposite to the head direction (upper left direction) and in the substantially opposite direction (lower right direction). As a result, as shown in FIG. 11B, when the movie button 23 is entirely within the display area 11, the movie button 23 is selected. When the movie button 23 is selected in this way, the stopped moving images 31 to 34 are displayed as shown in FIG. 11C. A part of the moving images 31 to 34 is displayed outside the display area 11, and in this state, the user cannot see a part of the moving images 31 to 34. Here, when the user wants to view the moving image 34, the user's head naturally turns to the lower right. Accordingly, the moving images 31 to 34 move in a direction substantially opposite to the head direction (lower right direction) (upper left direction). As a result, as shown in FIG. 11D, when the entire moving image 34 fits in the display area 11, the moving image 34 is selected. When the moving image 34 is selected in this way, as shown in FIG. 11E, the moving image 34 is displayed, for example, in the entire display area 11, and the reproduction process of the moving image 34 is started as shown in FIG. 11F. The

  12A to 12F are diagrams showing another state transition of the display unit 109 according to Embodiment 1 of the present invention. In the example of FIG. 7, it is not possible to return to the main menu after shifting to the submenu. The submenu here is a menu constituted by submenu buttons, and the main menu is a menu constituted by a start button 21 and an end button 22. However, the button may be selected by mistake, and in such a case, it is inconvenient that the previous menu cannot be returned. Here, a description will be given of a mode in which the user can return to the main menu as necessary even when the submenu button is entered.

  That is, in the initial state, as shown in FIG. 12A, a part of the start button 21 is displayed outside the display area 11, and a part of the end button 22 is also displayed outside the display area 11. In this state, the user cannot see a part of the start button 21 and cannot see a part of the end button 22. Here, when the user wants to instruct the start of various processes, the user's head naturally turns to the left. As a result, the start button 21 and the end button 22 are moved in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). As a result, as shown in FIG. 12B, after all of the start buttons 21 are accommodated in the display area 11, as shown in FIG. 12C, when the start button 21 moves to approximately the center of the display area 11, the start button 21 is displayed. Is selected. When the start button 21 is selected in this way, as shown in FIG. 12D, the submenu buttons are displayed radially with the start button 21 as the center. A part of the submenu button is displayed outside the display area 11, and in this state, the user cannot see a part of the submenu button.

  Here, when the user wants to watch a movie, the user's head naturally turns to the upper left. As a result, the submenu button moves in the direction opposite to the head direction (upper left direction) and in the substantially opposite direction (lower right direction). As a result, as shown in FIG. 12D, when all the movie buttons 23 fit in the display area 11, the movie button 23 is selected. On the other hand, in the state shown in FIG. 12D, when the user wants to instruct the end of various processes, the user's head naturally turns to the right. As a result, the submenu button, the start button 21 and the end button 22 move in a direction substantially opposite (leftward) to the direction of change in head direction (rightward). As a result, as shown in FIG. 12F, when the end button 22 is entirely within the display area 11, the end button 22 is selected. Of course, the timing at which the end button 22 is selected may not be when all of the end buttons 22 are within the display area 11. For example, 90% or more of the end button 22 may fit in the display area 11, or the end button 22 may move to approximately the center of the display area 11.

  As described above, according to the display device in the first embodiment of the present invention, the user cannot see a part of the display element. It will move. As a result, a display element displayed in the change direction of the head direction is selected, and a predetermined process related to the selected display element is executed. This enables a highly accurate hands-free operation while adopting a low-cost configuration that only detects the direction of change in head orientation. For example, when four moving images (display element 1, display element 2, display element 3, and display element 4) are displayed as four display elements at a time on the HMD, the user views the four moving images that are displayed. It is possible to set only one moving image in the playback state and the remaining three moving images not viewed by the user in the paused state.

  Note that either of the operations from S101 to S105 shown in FIG. 3 may be performed first. For example, the element to be actually displayed may be determined after acquiring the display element, or the display element may be determined after determining the boundary display position. Further, when each step is repeatedly executed, if there is a step that adopts the same processing result as the previous time, no processing is required at that step. For example, when S101 is re-executed after executing S108, the display elements do not need to be changed. Further, the operations shown in FIG. 3 may be performed one after another immediately, may be operated after a certain time, or may be operated in parallel.

  Note that the processing from S101 to S108 shown in FIG. 3 may be processing results with probability. For example, the boundary display state of a certain display element may be determined as “boundary display with a probability of 20%”, or the selection state may be determined as “selected with a probability of 15%”. .

  In the above description, when a display element fits in the display area 11, the display element is selected. However, the present invention is not limited to this. That is, as will be described below, when a display element falls within the selection frame, the display element may be selected.

  13A to 13C are diagrams illustrating display examples when the selection frame F is provided. This selection frame F is a frame for selecting a display element. That is, the display unit 109 displays the selection frame F at a predetermined position (for example, the center) of the display area 11 as shown in FIG. 13A. The display position of the selection frame F is fixed.

  Here, when the user turns his head to the left, as shown in FIG. 13B, the start button 21 and the end button 22 move in the direction opposite to the head direction (left direction) and in the substantially opposite direction (right direction). Will do. As a result, as shown in FIG. 13C, when all of the start buttons 21 are within the selection frame F, the start button 21 is selected.

  As described above, even with the configuration using the selection frame F, the same effect as described above can be obtained. In addition, since the display element can be selected by an easy-to-understand method called the selection frame F, there is an effect that the usability is improved.

  Here, the start button 21 is selected when all of the start buttons 21 are within the selection frame F, but the timing for selecting the display element is not limited to this. That is, the start button 21 may be selected when a portion of the start button 21 that is equal to or greater than a predetermined ratio is within the selection frame F.

  14A to 14C are diagrams illustrating another display example when the selection frame F is provided. Although FIG. 13 illustrates the case where one display element is selected from two display elements, FIG. 14 illustrates the case where one display element is selected from a large number of display elements. In this case, it is necessary to avoid a problem that two or more display elements are simultaneously accommodated in the selection frame F. Therefore, it is preferable that the size of the selection frame F is a size that can include one display element and cannot include two or more display elements. When such a selection frame F is used, one display element can be easily selected from a large number of display elements.

  Although the rectangular selection frame F is illustrated here, the shape of the selection frame F is not particularly limited, and for example, a circular selection frame F may be adopted. The color of the selection frame F is not particularly limited, and for example, a transparent selection frame F may be adopted.

  A color or a pattern may be added to one or both of the inner area and the outer area of the selection frame F. In this case, the boundary line of the color or pattern can be considered as the selection frame F. It is also possible to prevent the user from observing the inner area or the outer area by reducing the transparency of the color or pattern.

(Embodiment 2)
In the first embodiment, the HMD is exemplified as the display device, but in the second embodiment, a large-screen television is exemplified as the display device. Hereinafter, the second embodiment will be described with a focus on differences from the first embodiment.

  FIG. 15 is a diagram illustrating a usage state of the display device according to the second embodiment of the present invention. The large-screen television 50 is an example of a display device according to the present invention. The user U is watching a movie displayed on the television 50. Operations on the television 50 can be performed hands-free.

  FIG. 16 is a configuration diagram of a display device according to Embodiment 2 of the present invention. This display device is a stationary display device (for example, the television 50) and functionally includes a display element determination unit 101, a display element acquisition unit 102, a boundary display determination unit 103, a detection unit 104, A selection unit 110, an execution unit 108, and a display unit 109 are provided.

  The display unit 109 according to the second embodiment is not mounted on the user U's head, but is installed on an indoor wall or the like. Therefore, the display device according to the second embodiment does not include the display position changing unit 105 described in the first embodiment. The detailed reason why the display position changing unit 105 becomes unnecessary will be described later. Other points are the same as in the first embodiment.

  Note that the display method of the display unit 109 may be liquid crystal or plasma and is not particularly limited. Although not shown, it is assumed that an angular velocity sensor (detection unit 104) is attached to the head of the user U.

  FIG. 17 is a diagram illustrating the operation of the display device according to Embodiment 2 of the present invention. As described above, the display device according to the second embodiment does not include the display position changing unit 105 described in the first embodiment. Therefore, in the second embodiment, the operation of the display position changing unit 105 (the operation corresponding to S103 in FIG. 3) described in the first embodiment is not necessary. Other points are the same as in the first embodiment.

  18A and 18B are diagrams showing state transition of the display unit 109 according to Embodiment 2 of the present invention. The button 51 is an example of a display element, and is a button for decreasing the volume of the television 50. The button 52 is an example of a display element, and is a button for increasing the volume of the television 50.

  In the drawing, a range indicated by a dotted line represents the central visual field of the user U. As shown in FIG. 18A, when the user U is facing the front, the button 51 and the button 52 are displayed so as to cross the boundary line of the central visual field of the user U. In this case, it is necessary to grasp the viewing position of the user U on the television 50 side, but the method is not particularly limited. For example, if the camera is installed indoors, the viewing position of the user U can be obtained by analyzing the video captured by the camera.

  In the state shown in FIG. 18A, a part of the button 51 and the button 52 is blurred to the user U. Here, when the user U wants to increase the volume of the television 50, the consciousness of seeing all of the buttons 52 in detail works, and the head naturally turns to the right. As a result, as shown in FIG. 18B, when all of the buttons 52 fall within the user's U field of view, the button 52 is selected.

  As described above, even according to the second embodiment, the user cannot see a part of the display element. Therefore, when the user looks at the display element, the user's head naturally moves. This enables a highly accurate hands-free operation while adopting a low-cost configuration that only detects the direction of change in head orientation. In other words, the present invention can be applied to a stationary display device.

  In the second embodiment, only the case where the display element crosses the boundary line of the user's field of view is illustrated, and the case where the display element crosses the boundary line of the display region is not illustrated. This is because when a stationary display device is employed, the user's head cannot be expected to move naturally even if the display element crosses the boundary of the display area. Hereinafter, this point will be described in detail.

  19A and 19B are diagrams conceptually showing the display area 11 that can be seen by the user U wearing the HMD. Here, as shown in FIG. 19A, it is assumed that the user U is observing the display area 11 slightly forward and to the left.

  In this case, even if the head of the user U moves, the relative position of the display area 11 does not change as shown in FIG. 19B. Therefore, in the first embodiment, the display position changing unit 105 moves the display elements in the display area 11 as if the scenery seen from the window 11 changes. In this way, when it is desired to select a display element that crosses the boundary line of the display area 11, the head of the user U naturally faces the display element.

  On the other hand, when the stationary display device is adopted, as shown in FIG. 18, when the head of the user U moves, the relative position of the television 50 changes accordingly. In other words, the user U in this case is only observing a natural image. Therefore, it is not necessary to change the display position of the display element on the display device side, and the display position changing unit 105 is not necessary. According to the configuration not including the display position changing unit 105, the display position of the display element is not changed even if the head is moved. Therefore, even if there is a display element that straddles the boundary line of the display area 11, the tendency of the user U's head to move naturally to select the display element is reduced.

  In the second embodiment, the angular velocity sensor mounted on the head of the user U is exemplified as the detection unit 104. However, the detection unit 104 can detect the movement of the user U's head. I just need it. For example, if the camera is installed indoors, it is possible to detect the movement of the user U's head by analyzing the video captured by the camera. In this case, there is an advantage that convenience is improved because it is not necessary to attach anything to the head of the user U.

  Each of the above-described embodiments is realized by interpreting and executing predetermined program data stored in a storage device (ROM, RAM, hard disk, etc.) that can execute the above-described processing procedure. In this case, the program data may be introduced into the storage device via the recording medium, or may be directly executed from the recording medium. The recording medium refers to a recording medium such as a semiconductor memory such as a ROM, a RAM, or a flash memory, a magnetic disk memory such as a flexible disk or a hard disk, an optical disk such as a CD-ROM, DVD, or BD, or a memory card such as an SD card. . The recording medium is a concept including a communication medium such as a telephone line or a conveyance path.

  The present invention can be applied to uses such as an HMD, FMD, HUD, glasses-type display, and large-screen television that require a highly accurate hands-free operation while adopting a low-cost configuration.

FIG. 1 is an external view of a display device according to Embodiment 1 of the present invention. FIG. 2 is a configuration diagram of the display device according to Embodiment 1 of the present invention. FIG. 3 is a diagram showing the operation of the display device according to Embodiment 1 of the present invention. FIG. 4A is a diagram showing a display example in Embodiment 1 of the present invention. FIG. 4B is a diagram showing a display position management table in Embodiment 1 of the present invention. FIG. 5A is a diagram showing a display example in Embodiment 1 of the present invention. FIG. 5B is a diagram showing a display position management table according to Embodiment 1 of the present invention. FIG. 6A is a diagram showing a display example in the first embodiment of the present invention. FIG. 6B is a diagram showing a display position management table according to Embodiment 1 of the present invention. FIG. 7A is a diagram showing state transition of the display unit according to Embodiment 1 of the present invention. FIG. 7B is a diagram showing state transition of the display unit according to Embodiment 1 of the present invention. FIG. 7C is a diagram showing state transition of the display unit according to Embodiment 1 of the present invention. FIG. 8A is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 8B is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 8C is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 9A is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 9B is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 9C is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 10A is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 10B is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 10C is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11A is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11B is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11C is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11D is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11E is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 11F is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12A is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12B is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12C is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12D is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12E is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 12F is a diagram showing another state transition of the display unit according to Embodiment 1 of the present invention. FIG. 13A is a diagram illustrating a display example when a selection frame is provided. FIG. 13B is a diagram illustrating a display example when a selection frame is provided. FIG. 13C is a diagram illustrating a display example when a selection frame is provided. FIG. 14A is a diagram showing another display example when a selection frame is provided. FIG. 14B is a diagram illustrating another display example when a selection frame is provided. FIG. 14C is a diagram illustrating another display example when a selection frame is provided. FIG. 15 is a diagram illustrating a usage state of the display device according to the second embodiment of the present invention. FIG. 16 is a configuration diagram of a display device according to Embodiment 2 of the present invention. FIG. 17 is a diagram illustrating the operation of the display device according to Embodiment 2 of the present invention. FIG. 18A is a diagram showing state transition of the display unit according to Embodiment 2 of the present invention. FIG. 18B is a diagram showing state transition of the display unit according to Embodiment 2 of the present invention. FIG. 19A is a diagram conceptually showing a display area visible to a user wearing the HMD. FIG. 19B is a diagram conceptually illustrating a display area that is visible to a user wearing the HMD.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Display element determination part 102 Display element acquisition part 103 Boundary display determination part 104 Detection part 105 Display position change part 106 Boundary display determination part 107 Selection element determination part 108 Execution part 109 Display part 110 Selection part

Claims (16)

A display device for displaying information to a user,
Obtaining means for obtaining a display element which is information displayed to the user;
Determining means for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element;
Display means for displaying the display element at the determined display position;
When the user's head orientation changes, detection means for detecting the change direction;
A selection means for visually selecting the display element displayed in the detected change direction;
Execution means for executing a predetermined process related to the selected display element;
Changing means for changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction;
The display means displays the display element at the changed display position;
The selection means does not select the display element unless a portion of the display element equal to or greater than a predetermined ratio enters an area that can be viewed by the user, and a portion greater than the predetermined ratio has entered the area. Select the display element accordingly,
The display device is characterized in that, when the display element is selected by the selection means, the execution means executes a process assigned to the display element . Said selecting means, when said all of the display elements in a state where the user can see, the display device according to claim 1, wherein the selecting the display element. The display device according to claim 1, wherein the selection unit selects the display element when the predetermined display state continues for a predetermined time or longer. The display device according to claim 1, wherein the determining unit determines a display position of the display element such that the display element crosses a boundary line of the visual field of the user. The display device according to claim 1, wherein the determining unit determines a display position of the display element such that the display element crosses a boundary line of a display area of the display unit. The determining means is configured such that a left part of the display element is displayed at a right end of the display area of the display means so that a right part of the display element is displayed at a left end of the display area of the display means. The lower part of the display element is displayed at the upper end of the display area of the display means, or the upper part of the display element is displayed at the lower end of the display area of the display means. The display device according to claim 1, wherein a display position is determined. Before SL determining means, even when the display positions of the plurality of the display element is changed, so that the number of the display elements to be selected becomes one, characterized by determining the display positions of the plurality of display elements The display device according to claim 1. The display device according to claim 1, wherein the detection unit detects the change direction based on an output from an angular velocity sensor mounted on the head of the user. The display element is a button or icon;
The display device according to claim 1, wherein the execution unit executes a process assigned to the button or the icon . The display device according to claim 1, wherein, when the display element is a moving image, the execution unit starts a reproduction process of the moving image. The display means displays a selection frame that is a frame for selecting the display element at a predetermined display position,
The display device according to claim 1, wherein the selection unit selects the display element when the display element falls within the selection frame. The display device according to claim 1, wherein the display unit is attached to a head of the user. A control method for a display device for displaying information to a user,
An acquisition step of acquiring a display element which is information displayed to the user;
A display position determining step for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element;
A display step of displaying the display element at the determined display position;
When the user's head orientation changes, a detection step of detecting the change direction;
A selection step of visually selecting the display element displayed in the detected change direction;
An execution step of executing a predetermined process related to the selected display element;
Changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction,
In the display step, the display element is displayed at the display position changed by the change step,
In the selection step, the display element is not selected unless a portion of the display element that is equal to or greater than a predetermined ratio enters the area that the user can see, and the part that exceeds the predetermined ratio is included in the area. Select the display element accordingly,
In the execution step, when the display element is selected in the selection step, a process assigned to the display element is executed . A program for controlling a display device that displays information to a user,
An acquisition step of acquiring a display element which is information displayed to the user;
A display position determining step for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element;
A display step of displaying the display element at the determined display position;
When the user's head orientation changes, a detection step of detecting the change direction;
A selection step of visually selecting the display element displayed in the detected change direction;
An execution step of executing a predetermined process related to the selected display element ;
Causing the computer to execute a changing step of changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction;
In the display step, the display element is displayed at the display position changed by the change step,
In the selection step, the display element is not selected unless a portion of the display element that is equal to or greater than a predetermined ratio enters the area that the user can see, and the part that exceeds the predetermined ratio is included in the area. Select the display element accordingly,
In the execution step, when the display element is selected in the selection step, a process assigned to the display element is executed.
A program characterized by that . A computer-readable recording medium recording a program for controlling a display device that displays information to a user,
The program is
An acquisition step of acquiring a display element which is information displayed to the user;
A display position determining step for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element;
A display step of displaying the display element at the determined display position;
When the user's head orientation changes, a detection step of detecting the change direction;
A selection step of visually selecting the display element displayed in the detected change direction;
An execution step of executing a predetermined process related to the selected display element;
Causing the computer to execute a changing step of changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction;
In the display step, the display element is displayed at the display position changed by the change step,
In the selection step, the display element is not selected unless a portion of the display element that is equal to or greater than a predetermined ratio enters the area that the user can see, and the part that exceeds the predetermined ratio is included in the area. Select the display element accordingly,
In the execution step, when the display element is selected in the selection step, processing assigned to the display element is executed . An integrated circuit for controlling a display device for displaying information to a user,
Obtaining means for obtaining a display element which is information displayed to the user;
Determining means for determining a display position at which the user cannot see a part of the display element as a display position of the acquired display element;
Display means for displaying the display element at the determined display position;
When the user's head orientation changes, detection means for detecting the change direction;
A selection means for visually selecting the display element displayed in the detected change direction;
Execution means for executing a predetermined process related to the selected display element;
Changing means for changing the display position of the display element so that the display element moves in a direction substantially opposite to the detected change direction;
The display means displays the display element at the changed display position;
The selection means does not select the display element unless a portion of the display element equal to or greater than a predetermined ratio enters an area that can be viewed by the user, and a portion greater than the predetermined ratio has entered the area. Select the display element accordingly,
The integrated circuit according to claim 1, wherein when the display element is selected by the selection means, the execution means executes a process assigned to the display element .

Images