JP2018085125A - Terminal device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an operation method which enables selection in a short period of time while preventing erroneous operation.SOLUTION: A terminal device comprises: a detection section which detects an operation point of a user in a virtual space; an objection arrangement section which arranges an object, which can be selected through user operation, in the virtual space; a display control section which causes a display section to display a first determination region to determine whether or not the selection object arranged in the virtual space is selected and a second determination region which is different from the first determination region; a first determination section which determines that the selection object is selected under a condition of a lapse of a predetermined period since when a positional relation where the first determination region displayed on the display section is not overlapped with the operation point is shifted to the positional relation where the first determination region is overlapped with the operation point; and a second determination section which determines that the selection object is selected under the condition that the positional relation of the second determination region with the operation point satisfies a predetermined condition regardless of the lapse of the predetermined period.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a terminal device and a program.

  There is a head mounted display (HMD) that can be mounted on a user's head and can display an image in a virtual space on a display arranged in front of the user's eyes. For example, in a game using HMD, there is a game that plays a game using a device such as a controller that is held and operated in addition to a display worn on the head. However, since the user can not observe the hand because the field of view of the real space is covered by the HMD, an erroneous operation may occur when the finger is once removed from the controller. In view of this, there is an HMD that detects a motion such as a tilt of the head and performs a display / non-display operation of options (for example, Patent Document 1).

  In recent years, some smartphones have been used as HMDs with higher performance. For example, it can be used as an HMD by attaching a smartphone to the head using a simple attachment. In this case, an input device such as a touch panel provided in the smartphone cannot be used by being worn on the head. Therefore, for example, there is a method of realizing an operation for an option displayed on the HMD by detecting the movement of the head using a sensor such as a gyro built in the smartphone.

Japanese Patent No. 5767386

  As described above, in the operation in the HMD, it is determined that a predetermined time (for example, 3 seconds) has elapsed with the head direction corresponding to the user's line-of-sight direction within a specific range (option determination region). There is an operation method in which the selection is confirmed when the In such an operation method, erroneous operation is prevented by providing a determination period, but a waiting time of a certain period is required until selection is confirmed. For this reason, the user may be forced to wait for a long time in scenes where selection is often made.

  An object of some aspects of the present invention is to provide a terminal device and a program that realize an operation method that can be selected in a short time while suppressing erroneous operations.

  Another object of another aspect of the present invention is to provide a terminal device and a program that can achieve the effects described in the embodiments to be described later.

  In order to solve the above-described problem, one aspect of the present invention provides a detection unit that detects an operation point of a user in a virtual space, and an object arrangement that arranges a selection object that can be selected by the user's operation in the virtual space. A first determination area for determining whether or not the selected object arranged in the virtual space has been selected, the operation point detected by the detection section, and the first determination area A display control unit that displays a different second determination region on the display unit, and a predetermined positional relationship after the first determination region displayed on the display unit and the operation point do not overlap with each other. The positional relationship between the first determination unit that determines that the selected object has been selected, and the second determination region and the operation point satisfies a predetermined condition on the condition that the time elapses. Condition the door, and a second determination unit determines that the selected object need not be the predetermined time has elapsed is selected, a terminal device comprising a.

  One embodiment of the present invention is a program for causing a computer to function as the above terminal device.

1 is an external view showing an example of an HMD system according to a first embodiment. The figure which shows an example of the hardware constitutions of the terminal device which concerns on 1st Embodiment. The figure which shows the definition of the direction of the virtual space which concerns on 1st Embodiment. Explanatory drawing of the visual field direction and gaze point which concern on 1st Embodiment. The figure which shows an example of the selection menu screen which concerns on 1st Embodiment. The figure which shows an example of the selection menu screen with which the gaze point overlapped the determination area | region of the selection object. The figure which shows an example of the selection menu screen after a display content changes because the gaze point overlapped the determination area | region of the selection object. The figure which shows an example of the selection menu screen with which the gaze point overlapped the selection confirmation area | region of the selection object. The block diagram which shows an example of a function structure of the terminal device which concerns on 1st Embodiment. 6 is a flowchart illustrating an example of a selected object arrangement process according to the first embodiment. 6 is a flowchart showing an example of determination processing for determining whether or not a selected object is selected according to the first embodiment. The figure which shows an example of the selection menu screen which concerns on 2nd Embodiment. The figure which shows an example of the selection menu screen which concerns on 3rd Embodiment. The figure which shows an example of the selection menu screen of the state which the gazing point moved to the position of the starting point area | region. The figure which shows an example of the selection menu screen of the state where selection was confirmed because the gaze point moved to the end point area. 14 is a flowchart illustrating an example of determination processing for determining whether or not a selected object is selected according to the third embodiment. The figure which shows an example of the selection menu screen which concerns on 4th Embodiment. The figure which shows an example of the selection menu screen which concerns on 5th Embodiment. The figure which shows an example of the selection menu screen of the state in which the elapsed time gauge follows the movement of a gazing point. The figure which shows an example of the selection menu screen which concerns on 6th Embodiment. The figure which shows the other example of the selection menu screen which concerns on 6th Embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[First embodiment]
A first embodiment of the present invention will be described.
FIG. 1 is an external view showing an example of an HMD (Head Mounted Display) system according to the present embodiment. The HMD is worn on the user's head and can display a view image representing the view from a virtual viewpoint in the virtual space. For example, the HMD can display a stereoscopic image using binocular parallax between the right eye and the left eye. Also, the HMD is equipped with a sensor that detects the movement and tilt of the HMD such as a gyro, detects the movement of the head of the user who is wearing it, changes in the tilt, etc., and displays the virtual on the display according to the change. A view image in the view direction of the space is displayed. For example, the visual field image displayed on the HMD changes to a right visual field image in the virtual space when the user's head is directed rightward, and changes to an upward visual image in the virtual space when the user heads upward. It is possible to give the user an immersive feeling as if they were on the spot.

  The illustrated HMD system 1 is configured such that the terminal device 10 can be used as an HMD by attaching the terminal device 10 including the display unit 12 to the attachment 2. The attachment 2 can be attached to the terminal device 10 so as to cover the field of view of the front of the user while attached to the user's head, and the right-eye lens for visually recognizing the display unit 12 of the attached terminal device 10 3R and the left-eye lens 3L, and a strap 5 for mounting the HMD system 1 on the user's head. The user can visually recognize the visual field image displayed on the terminal device 10 via the right-eye lens 3R and the left-eye lens 3L by wearing the HMD system 1 on the head.

[Hardware Configuration of Terminal Device 10]
The terminal device 10 is a portable computer device that can be used as at least a part of the HMD system 1, and is a mobile phone such as a smartphone or a feature phone, a personal digital assistant (PDA), a tablet PC, or a home game. Machine, etc. can be applied. In the present embodiment, the terminal device 10 will be described as a smartphone.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the terminal device 10. The terminal device 10 includes, for example, a display unit 12, a sensor 13, a timer 14, a storage unit 15, a communication unit 16, and a CPU (Central Processing Unit) 17.

  The display unit 12 is a display that displays information such as images and texts, and includes, for example, a liquid crystal display panel, an organic EL (ElectroLuminescence) display panel, and the like. For example, the display unit 12 displays a stereoscopic image (a right-eye image and a left-eye image) using binocular parallax as a view field image from a virtual viewpoint (the user's eye wearing the HMD) in the virtual space. To do. In addition, the display unit 12 displays various objects arranged in the virtual space together with the view field image. The various objects are displayed, for example, for a user interface (UI) such as a selection object (option) that can be selected by a user's operation, an object for enabling a gaze point corresponding to the gaze direction from the virtual viewpoint, and the like. It is an object for. Here, the selected object has a determination area for determining whether or not a selection is made by a user operation. For example, a part or all of the selected object is a determination area. The gaze point corresponding to the line-of-sight direction from the virtual viewpoint is a point indicating the position of the line of sight ahead of the line-of-sight direction.

  The sensor 13 is a sensor that detects information related to the direction of the terminal device 10. For example, the sensor 13 is a gyro sensor that detects an angle, an angular velocity, an angular acceleration, and the like of an object. The sensor 13 may be a sensor that detects a change in direction, or a sensor that detects the direction itself. For example, the sensor 13 is not limited to a gyro sensor, and may be an acceleration sensor, a tilt sensor, a geomagnetic sensor, or the like.

  The timer 14 has a time measuring function for measuring time.

  The storage unit 15 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and a ROM (Read-Only Memory). Virtual space data (image data), data of objects arranged in the virtual space, game programs using the virtual space, and the like are stored.

  The communication unit 16 communicates with other devices via the network NW.

  The CPU 17 functions as a control center that controls each unit included in the terminal device 10. For example, the CPU 17 functions as a control unit that controls each unit of the terminal device 10 by executing various programs stored in the storage unit 15.

  The above-described components are connected to each other via a bus so that they can communicate with each other. Further, the terminal device 10 may be configured to include a hardware configuration such as a speaker (not shown), an audio output terminal, a camera, a GPS (Global Positioning System) receiving module, an operation button that accepts a user operation input, or a touch panel. .

  For example, the terminal device 10 executes a game program using the HMD. This game is played with the user wearing the HMD system 1 while changing the visual field range of the field-of-view image (game screen) displayed on the display unit 12 by changing the direction of the head or the like. The terminal device 10 detects information related to the direction of the terminal device 10 based on the detection result of the sensor 13, and causes the display unit 12 to display a visual field image (game screen) in the visual field direction based on the detection result. The direction of the terminal device 10 corresponds to the direction of the head of the user wearing the HMD on the head. In addition, when receiving an input of a selection operation by the user, the terminal device 10 arranges a selection object (option) having a determination region for determining whether or not there is a selection from the user in the virtual space, and displays it on the display unit 12 together with the view image. Display.

[Definition of virtual space]
FIG. 3 is a diagram showing the definition of the direction of the virtual space according to the present embodiment. In the present embodiment, the vertical direction in which the user stands upright is the Z axis, the direction orthogonal to the Z axis and connecting the user and the display unit 12 is the X axis, and is orthogonal to the Z axis and the X axis. The axis is the Y axis.

  Here, a change in the rotation direction around the Z axis is also referred to as a change in the yaw direction (left and right direction), and a change in the rotation direction around the Y axis is also referred to as a change in the pitch direction (vertical direction). A change in the rotation direction about the X axis is also referred to as a change in the roll direction. For example, the sensor 13 described above detects an angular velocity or an angular acceleration in the rotation direction (yaw direction, pitch direction, and roll direction) of each axis. Note that a change in the yaw direction may be referred to as a change in the left-right direction, and a change in the pitch direction may be referred to as a change in the vertical direction.

  FIG. 4 is an explanatory diagram of the viewing direction and the point of gaze according to the present embodiment. In this figure, when the virtual viewpoint K (user's virtual viewpoint) in the virtual space is the intersection (origin) of the X-axis, Y-axis, and Z-axis, and the user's line-of-sight direction is the X-axis direction, the view from the virtual viewpoint The range of the visual field image in the direction (that is, the visual field) is the yaw angle α (the internal angle between the broken line a and the broken line b and the internal angle between the broken line c and the broken line d) and the pitch angle around the visual line direction (X-axis direction). This is a range determined by β (inner angle between the broken line a and the broken line d and inner angle between the broken line b and the broken line c). Here, the yaw angle α and the pitch angle β are angles set in advance as the angle of view of the visual field image of the virtual space displayed on the HMD system 1. When the user's head changes in the pitch direction or the yaw direction, the line-of-sight direction changes from the X-axis direction to the pitch direction or the yaw direction according to the change, and the visual field direction also changes in the pitch direction or the yaw direction. When the user's head changes in the roll direction, the visual line direction rotates in the roll direction while the line-of-sight direction remains in the X-axis direction.

  In the virtual space, various objects are arranged as necessary. For example, various objects are arranged on a plane orthogonal to the viewing direction in the virtual space. For example, when a plane on which an object is placed in the virtual space is a plane L1 as shown in the drawing, the gazing point P1, which is an object for enabling a gazing point corresponding to the line-of-sight direction to be visually recognized, is viewed on the plane L1. Is arranged at the center of the range (view) surrounded by a quadrangle with vertexes a1, b1, c1, d1 with the broken lines a, b, c, d indicating the range (ie, the intersection of the plane L1 and the X axis). The Note that the position at which the gazing point P1 is arranged is not limited to the center of the range (field of view) surrounded by a rectangle with the intersections a1, b1, c1, and d1 as vertices on the plane L1, but other positions (for example, from the center It may be arranged at a slightly lower position or the like. In addition, a selection object (option) having a determination area for determining whether or not there is a selection by a user operation is arranged on the surface L1 in the virtual space. The determination area may be a surface (two-dimensional shape) or a solid (three-dimensional shape). For example, a selection object (option) arranged in a range (view) surrounded by a rectangle with the intersections a1, b1, c1, d1 of the plane L1 and the broken lines a, b, c, d1 as vertices, and a gazing point P1 Is displayed on the display unit 12 together with the view field image. In the present embodiment, the gazing point P1 corresponding to the user's line-of-sight direction is an operation point indicating the position of the operation for selecting the selected object (option). Various objects arranged in the virtual space may be arranged on the same plane (same layer) or may be arranged separately on a plurality of planes (plural layers).

  When the user's head changes in the pitch direction or the yaw direction, the viewing direction and the line-of-sight direction change in the pitch direction or the yaw direction according to the change. Move so that it appears in the center. Further, when the user's head changes in the roll direction, the gazing point P1 rotates according to the change in the roll direction at the position where it is placed. On the other hand, the selected object does not move from the position where the selected object is arranged even if the visual field direction changes in the pitch direction, the yaw direction, or the roll direction. That is, when the view direction changes, the selected object arranged in the changed view is displayed.

  When displaying stereoscopic images using binocular parallax, there are field-of-view directions and line-of-sight directions corresponding to virtual viewpoints for the right eye and left-eye, respectively, and field-of-view images and various objects in the respective field-of-view directions and line-of-sight directions. The right-eye image and the left-eye image are displayed. In the present embodiment, the right-eye image and the left-eye image will be described without distinction in order to facilitate the description.

[Display example and operation example of selected object]
Next, a display example and an operation example of the selected object will be described. Hereinafter, a screen displayed on the display unit 12 so that a plurality of selected objects can be selected is also referred to as a “selection menu screen”. The selection menu screen is a screen on which a view object in the view direction in the virtual space is used as a background image and a selection object that is also arranged in the virtual space and a gazing point P1 corresponding to the view direction are displayed.

  FIG. 5 is a diagram illustrating an example of a selection menu screen according to the present embodiment. In the illustrated selection menu screen G10, a gazing point P1, which is an object arranged at a gazing point corresponding to the line-of-sight direction in the virtual space, is displayed at the center of the screen. In addition, two selection objects arranged side by side in the Y-axis direction in the virtual space are displayed on the left and right with the gazing point P1 in between. The selected object has a determination area for determining the presence / absence of selection. Here, it is assumed that the display area of the selected object is a determination area for determining the presence / absence of selection. On the left side of the gazing point P1, a selection object of the option A having the determination area R1 is displayed. In addition, a selection object of option B having the determination region R2 is displayed on the right side of the gazing point P1. Note that the display area of the selected object and the determination area for determining the presence / absence of selection may be different areas. Since the gazing point P1 is in a positional relationship that does not overlap with the determination area of any selected object (option) in the line-of-sight direction, no option is selected.

  If the orientation of the user's head changes in the state where the selection menu screen G10 shown in FIG. 5 is displayed, the relative position between the selected object and the gazing point P1 changes. The gazing point P1 changes in view direction according to the change in head direction, but moves to follow the view direction so as to keep the position of the line of sight, so it is fixed and displayed at the center of the screen. . On the other hand, the selected object moves on the screen in accordance with the change in the viewing direction due to the change in the orientation of the head.

  FIG. 6 is a diagram illustrating an example of a selection menu screen in which the gazing point overlaps the determination area of the selected object. The selection menu screen G11 shown in the drawing shows a state in which the gazing point P1 and the determination area R of the selection object of the option A overlap with each other in the line-of-sight direction. Hereinafter, the selection object of option A is simply referred to as “option A”. When the gazing point P1 overlaps the determination area of the option A, the display content of the selection menu screen changes as shown in FIG.

  FIG. 7 is a diagram illustrating an example of the selection menu screen after the display content is changed due to the point of interest overlapping the determination area of the selected object. In the illustrated selection menu screen G12, an elapsed time gauge T1 and a selection confirmation region R11 are displayed.

  The elapsed time gauge T1 is an object for making it possible to visually recognize the elapsed time since the gazing point P1 overlaps the determination area R1 of the option A, and is arranged and displayed in the vicinity of the determination area R1 of the option A. For example, the elapsed time gauge T1 is a circular gauge, and when the gazing point P1 overlaps the determination region R1, the length of the bar increases in the circumferential direction as time elapses, and a predetermined time or more elapses. This bar extends for one lap. Here, the predetermined time is a determination time set in advance as a threshold for determining whether or not the selected object is selected, and an elapsed time after the gazing point P1 overlaps the determination region R1 for a predetermined time. Is equal to or longer than a predetermined time (for example, 3 seconds), it is determined that the option A in the determination region R1 has been selected. For example, when the bar of the elapsed time gauge T1 is one round, it is determined that the option A in which the gazing point P1 overlaps the determination region R1 is selected. On the other hand, if the elapsed time since the point of interest P1 overlaps the determination region R1 is less than the predetermined time, it is determined that the option A of the determination region R1 has not yet been selected. For example, if the gazing point P1 deviates from the determination region R1 before the bar of the elapsed time gauge T1 has made one round, it is determined that the option A is not selected, and the measurement time (elapsed time) is reset. .

  The selection confirmation region R11 is a region different from the determination region R1, and is displayed as a region for confirming selection to the option A when the gazing point P1 overlaps the determination region R1 of the option A. For example, the selection confirmation area R11 is displayed in an area smaller than the determination area R1 in the determination area R1 of the option A.

  FIG. 8 is a diagram illustrating an example of a selection menu screen in which the gazing point overlaps the selection confirmation area of the selected object. If the gazing point P1 that overlaps the determination area R1 of the option A further overlaps the selection confirmation area R11 by changing the orientation of the head, the bar of the elapsed time gauge T1 is before one round. (That is, even if the predetermined time has not elapsed), it is determined that the option A has been selected. That is, when the user wants to confirm the selection quickly, the user should move the gazing point P1 to the selection confirmation region R11.

  For example, the display color of the selection object of option A is blue in a state where the gazing point P1 does not overlap the selection confirmation region R11 and in a state where the gazing point P1 does not overlap the selection confirmation region R11 even if it overlaps, the selection point of the gazing point P1 is confirmed. When it overlaps with the region R11 (or when a predetermined time has elapsed), it is changed to red indicating that the selection is confirmed.

  In the examples shown in FIGS. 7 and 8, the selection confirmation region R11 is displayed near the center of the determination region R1, but the position of the selection confirmation region R11 is changed depending on where the gazing point P1 enters the determination region R1. You may change it. For example, when the gazing point P1 enters the determination region R1 from the right, the selection confirmation region R11 may be displayed on the left side in the determination region R1. Thereby, it becomes easy to prevent an erroneous operation. That is, the position of the selection confirmation region R11 may be determined depending on where the gazing point P1 enters the determination region R1. Note that the position of the selection confirmation region R11 may be determined based on the current position of the gazing point P1.

  In the example shown in FIGS. 7 and 8, an example is shown in which the entire selection confirmation region R11 is included in the determination region R1, but a part of the selection confirmation region R11 may be included. . Further, the selection confirmation region R11 may be displayed at a position that does not overlap in the vicinity of the determination region R1.

  The size of the selected object (determination area) may be changed according to the positional relationship between the gazing point P1 and the determination area of the selection object. For example, when the gazing point P1 does not overlap with the determination area R1 of the option A but overlaps with the determination area R1, the size of the selection object (determination area R1) of the option A may be enlarged. Further, when the selection object (determination area R1) of option A is enlarged, the position where the selection object of another option (here, option B) is arranged may be changed. In this case, the user can easily perform an operation when changing the orientation of the head so that the gazing point P1 that overlaps the determination area R1 of the option A further overlaps the selection confirmation area R11.

  Further, when the gazing point P1 overlaps the determination area R1 of the option A, other selected objects (for example, the option B) arranged in the virtual space are erased or are not noticeable by increasing the transmittance. May be displayed. As a result, the option that the user is trying to select is displayed in a relatively emphasized manner. Thereby, it is possible to easily determine the option that the user is trying to select and the state of the selected option.

[Functional Configuration of Terminal Device 10]
Next, the functional configuration of the terminal device 10 will be described with reference to FIG. FIG. 9 is a block diagram illustrating an example of a functional configuration of the terminal device 10 according to the present embodiment. The terminal device 10 includes a control unit 110 as a functional configuration realized by the CPU 17 executing a program stored in the storage unit 15. The control unit 110 includes a detection unit 111, a determination unit 112, an object placement unit 113, and a display control unit 114.

  The detection unit 111 detects information related to the direction of the terminal device 10 based on the detection result of the sensor 13. Further, the detection unit 111 detects a gazing point corresponding to the line-of-sight direction from the virtual viewpoint in the virtual space, based on the information regarding the direction of the terminal device 10. For example, the detection unit 111 detects the direction of view of the user wearing the HMD system 1 to which the terminal device 10 is attached or the change in the view direction based on the direction detected by the sensor 13 or information on the direction change. While detecting by the direction in which 10 is facing, the center of the field of view is set as the position of the point of interest. Note that the detection unit 111 may detect information related to the change speed of the visual field direction.

  The determination unit 112 determines whether or not the selected object is selected based on the positional relationship between the user's gazing point P1 and the determination area of the selected object (function as a first determination unit). For example, the determination unit 112 counts the elapsed time after the gazing point P1 and the determination area of the selected object overlap from the positional relationship that does not overlap in the line-of-sight direction, and selects the selection based on the measured elapsed time. Determine presence or absence. For example, the determination unit 112 determines that the selected object has been determined (selected) when a predetermined time or more has elapsed since the positional relationship where the gazing point P1 and the determination area of the selected object overlap. On the other hand, when the elapsed time after the gazing point P1 and the determination area of the selected object overlap is less than a predetermined time, the determination unit 112 has not yet selected the selection object (no selection). Is determined.

  Further, the determination unit 112 determines that the selected object has been selected even if the predetermined time has not elapsed, on the condition that the positional relationship between the user's gazing point P1 and the selection confirmation area of the selected object satisfies the predetermined condition. Determine (function as second determination unit). For example, the determination unit 112 determines that the selected object has been selected even if the predetermined time has not elapsed when the positional relationship is overlapped from the positional relationship where the gazing point P1 and the selection confirmation region do not overlap.

  The object placement unit 113 places various objects in the virtual space. For example, the object placement unit 113 places the gazing point P1 corresponding to the line-of-sight direction in the virtual space based on the detection result of the detection unit 111. The object placement unit 113 places a selection object, a selection confirmation region, an elapsed time gauge, and the like that can be selected by a user operation in the virtual space. The object placement unit 113 may place the various objects on the same plane (same layer) in the virtual space, or may be divided into a plurality of planes (plural layers). The object placement unit 113 acquires time information such as elapsed time by using the timer 14.

  For example, the object arrangement unit 113 arranges a selected object having a determination area for determining whether or not it has been selected by the user's operation in the virtual space, and overlaps the determination area and the gazing point P1 from a positional relationship that does not overlap. When the positional relationship is reached, the selection confirmation area of the selected object is arranged in the virtual space.

  The display control unit 114 causes the display unit 12 to display a visual field image in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111 as a background image. The view image data in each direction in the virtual space is stored in the storage unit 15 as virtual space data. The display control unit 114 acquires a view image corresponding to the view direction from the virtual space data and causes the display unit 12 to display the view image as a view image.

  The display control unit 114 also has various objects (gaze point, selection object, selection confirmation area, elapsed time gauge, etc.) arranged in the field of view from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. Is superimposed on the view field image and displayed on the display unit 12. For example, the display control unit 114 displays the selected object (that is, the determination region) arranged in the virtual space on the display unit 12 and changes the positional relationship from the positional relationship in which the determination region and the gazing point P1 do not overlap. When it becomes, the selection confirmation area | region arrange | positioned in virtual space is displayed on the display part 12. FIG.

[Selected object placement processing operation]
Next, the operation of the selected object placement process according to the present embodiment will be described with reference to FIG. FIG. 10 is a flowchart illustrating an example of the selected object placement process according to the present embodiment. It is assumed that a visual field image in the visual field direction and a gazing point P1 are displayed on the display unit 12. First, upon receiving a selection menu display trigger, control unit 110 places a plurality of selected objects in the virtual space (step S100). Accordingly, a selection menu screen (for example, see FIG. 5) including the view field image and the plurality of selection objects is displayed on the display unit 12. Here, the trigger of the selection menu display means that the user in the game, such as that the game has started, that a predetermined time has elapsed since the game was started, or that a specific action has been performed in the game. It was set in a situation where confirmation of intention was required.

  The control unit 110 determines whether or not the gazing point P1 and the determination area of the selected object overlap in the line-of-sight direction (step S102). When it is determined that the gazing point P1 and the determination area of the selected object overlap in the line-of-sight direction (for example, see FIG. 6) (YES), the control unit 110 selects the selected object in which the gazing point P1 overlaps the determination area. Are arranged and displayed on the selection menu screen (step S104). For example, when it is determined that the gazing point P1 and the determination area R1 of the option A overlap in the line-of-sight direction, the control unit 110 displays the selection confirmation area R11 in the determination area R1 (see FIG. 7).

  On the other hand, when it is determined that the gazing point P1 and the determination area of the selected object do not overlap in the line-of-sight direction (for example, see FIG. 5) (NO), the control unit 110 hides the selection confirmation area. For example, the control unit 110 hides the selection confirmation area if it is not displayed, and hides it if it is displayed (step S106).

  Next, the control unit 110 determines whether or not a selection menu display-off trigger has been received (step S108). Here, the trigger for turning off the display of the selection menu is to select an option for turning off the display of the selection menu when a selected object is selected, or when a predetermined time has elapsed since the selection object was displayed. And so on. When it is determined that the trigger for turning off the display of the selection menu is received (YES), the control unit 110 deletes the selection object arranged in the virtual space (step S110) and ends the selection object arrangement processing.

  On the other hand, when it is determined that the trigger for turning off the display of the selection menu is not received (NO), the control unit 110 detects the angular velocity based on the detection result of the sensor 13 (step S112). And the control part 110 determines whether there exists any change of a visual field direction based on the detected angular velocity (step S114). When it is determined that there is no change in the viewing direction (NO), the control unit 110 returns to the process of step S108 and displays the selection menu because there is no change in the relative position between the gazing point P1 and the determination area of the selected object. It is determined whether an off trigger is accepted. On the other hand, if it is determined that there is a change in the viewing direction (YES), a change occurs in the relative position between the gazing point P1 and the determination area of the selected object, and thus the control unit 110 returns to the process of step S102, It is determined whether P1 and the determination area of the selected object overlap in the line-of-sight direction. The control unit 110 keeps the selection confirmation area displayed when the gazing point P1 and the determination area of the selected object are continuously overlapped, and when the overlapping state is reached from the non-overlapping state. The selection confirmation area is displayed, and the selection confirmation area is not displayed when the overlapping state is changed to the non-overlapping state.

[Operation of selection judgment processing]
Next, with reference to FIG. 11, the operation of the determination process for determining whether or not the selected object arranged in the object arrangement process according to this embodiment is selected will be described. FIG. 11 is a flowchart illustrating an example of determination processing for determining whether or not a selected object is selected according to the present embodiment. First, upon receiving a selection menu display trigger, the control unit 110 displays a selection menu screen (for example, see FIG. 5) on the display unit 12 by arranging a plurality of selection objects in the virtual space (step 5). S200). Moreover, the control part 110 resets measurement time (step S202).

  Next, the control unit 110 determines whether or not the gazing point P1 and the determination area of the selected object overlap in the line-of-sight direction (step S204). When it is determined that the gazing point P1 and the determination area of the selected object do not overlap in the line-of-sight direction (for example, see FIG. 5) (NO), the control unit 110 returns the process to step S202. On the other hand, when it is determined that the gazing point P1 and the determination area of the selected object overlap in the line-of-sight direction (for example, see FIG. 6) (YES), the control unit 110 starts counting elapsed time and In the space, an elapsed time gauge T1 is arranged in the vicinity of the selected object determined to overlap (step S206). Here, it is assumed that the gazing point P1 overlaps the determination area R1 of the option A as illustrated in FIG. Thereby, the elapsed time gauge T1 is displayed in the vicinity of the option A.

  Next, the control unit 110 determines whether or not a predetermined time or more has elapsed since the gazing point P1 and the determination area R1 of the option A overlap (step S208). When it is determined that a predetermined time or more has elapsed since the gazing point P1 and the determination area R1 of the option A overlap (YES), the control unit 110 determines that the option A has been selected and ends the determination process ( Step S220).

  On the other hand, when it is determined that a predetermined time or more has not elapsed since the gazing point P1 and the determination area R1 of the option A overlap (NO), the control unit 110 overlaps the gazing point P1 and the selection confirmation area R11. It is determined whether or not (step S210). When it is determined that the gazing point P1 and the selection confirmation region R11 overlap (YES, see FIG. 8), the control unit 110 does not exceed a predetermined time after the gazing point P1 and the determination region R1 of the option A overlap. Even if it has not elapsed, it is determined that the option A has been selected, and the determination process is terminated (step S220). On the other hand, when it is determined that the gazing point P1 and the selection confirmation region R11 do not overlap (NO), the control unit 110 returns to the process of step S204 and continues to measure the elapsed time after the overlap. When it is determined in step S204 that the gazing point P1 and the determination area R1 of the option A do not overlap (NO), the control unit 110 resets the measurement time measured so far (step S202). .

  When the point of sight P1 and the determination area of the selected object change from the overlapping state to the non-overlapping state, the control unit 110 temporarily does not reset the measurement time measured until then, It may be stopped. When the point of interest P1 and the determination area of the selected object overlap again, the stop may be released and the time measurement may be resumed. In addition, when the point of interest P1 and the selected object determination area change from a state in which they overlap each other to a state in which they do not overlap, the control unit 110 may delay the advance of time compared to the elapsed time measurement in the state in which they overlap. However, the time may be set back. That is, the control unit 110 may perform different timing depending on whether the gazing point P1 and the determination area of the selected object are in a positional relationship that does not overlap in the line-of-sight direction or in a positional relationship that overlaps.

  In this case, even if the gazing point P1 once deviates from the determination area of the selected object, if the gazing point P1 overlaps by moving the line of sight again within the determination area of the same selected object, the elapsed time is restarted from the next time. Even if the line of sight is mistakenly deviated from the determination area for a moment, it is not necessary to re-measure the elapsed time from zero.

  When a predetermined time (for example, 10 seconds) elapses after the gazing point P1 deviates from the determination area of the selected object, the control unit 110 assumes that the user does not intend to continue the selection, and the measurement time It may be reset. For example, the control unit 110 determines the elapsed time measured after the positional relationship of the gazing point P1 and the determination area of the selected object as a positional relationship that does not overlap the gazing point P1 and the determination area of the selected object. When the time elapses, the measurement time (elapsed time) may be reset.

  Further, when the gazing point P1 moves to the determination area of another selected object (for example, option B), the measured time (elapsed time) is reset, and the elapsed time is started from 0. For example, the determination unit 112 has the positional relationship in which the watch point P1 overlaps the determination region of another selected object with respect to the elapsed time counted after the positional relationship of the watch point P1 and the determination region of the selected object overlap. If this happens, reset the measurement time (elapsed time).

[Summary of First Embodiment]
As described above, the terminal device 10 according to the present embodiment is at least one of the HMD systems that can display a view image representing a view from a virtual viewpoint in a virtual space as a stereoscopic image using binocular parallax. It is a terminal device that can be used as a part. The terminal device 10 includes a detection unit 111, an object placement unit 113, a display control unit 114, and a determination unit 112. The detection unit 111 detects a user's point of sight P1 (an example of an operation point) in the virtual space. For example, the detection unit 111 detects information related to the direction of the terminal device 10 and detects a gazing point P1 corresponding to the line-of-sight direction from the virtual viewpoint in the virtual space as the operation point of the user. The object placement unit 113 places selected objects that can be selected by a user operation in the virtual space. The display control unit 114 includes a determination region R1 (an example of a first determination region) for determining whether or not a selected object arranged in the virtual space has been selected, a gazing point P1 detected by the detection unit 111, A selection confirmation region R11 (an example of a second determination region) different from the determination region R1 is displayed on the display unit 12. For example, the display control unit 114, based on information related to the direction of the terminal device 10, and a selection object (for example, a determination area R1, a selection confirmation area R11, etc.) arranged in the visual field direction from the virtual viewpoint in the virtual space. The gazing point P1 is displayed on the display unit 12 according to the direction of the terminal device 10. Then, the determination unit 112 selects the selected object on the condition that a predetermined time has elapsed since the positional relationship where the determination region R1 displayed on the display unit 12 and the gazing point P1 do not overlap with each other. (Example of function as first determination unit). Further, the determination unit 112 determines that the selected object has been selected even if the predetermined time has not passed, on the condition that the positional relationship between the selection confirmation region R11 and the gazing point P1 satisfies the predetermined condition (first order). 2 is an example of a function as a determination unit).

  As a result, the terminal device 10 has a predetermined time after the gazing point P1 overlaps the determination region R1 in addition to the determination region R1 where the selection is fixed on the condition that a predetermined time has elapsed since the gazing point P1 overlapped. Since the selection confirmation region R11 in which the selection is confirmed without the elapse of time is provided, an operation method that can be selected in a short time while suppressing erroneous operations can be realized. For example, since there are two types, a determination area R1 and a selection confirmation area R11, the user may perform an operation of overlaying the gazing point P1 on the determination area R1 and wait for confirmation of selection slowly. Both the operation methods can be selected by confirming the selection in a short time by performing the operation of overlapping the gazing point P1 on the region R11.

  For example, the determination unit 112 determines that the predetermined time has elapsed when the selection confirmation region R11 (an example of the second determination region) and the gazing point P1 (an example of the operation point) are in a positional relationship that overlaps with each other. Even if not, it is determined that the selected object has been selected.

  Thereby, the terminal device 10 confirms the selection by the user's operation so that the gazing point P1 overlaps the selection confirmation region R11 even if a predetermined time has not elapsed after the gazing point P1 overlaps the determination region R1. Therefore, the selected object can be selected in a short time.

  Further, the selection confirmation region R11 (an example of the second determination region) is a region smaller than the determination region R1 in the determination region R1 (an example of the first determination region).

  Thereby, since the selection is confirmed by the user overlapping the smaller selection confirmation region R11 in the determination region R1 after the user has superimposed the gazing point P1 on the determination region R1, the terminal device 10 can be shortened while suppressing erroneous operations. It becomes possible to select the selected object by time.

  The display control unit 114 displays the determination region R1 (an example of the first determination region) and the gazing point P1 (an example of the operation point) on the display unit 12, and then the determination region R1 and the gazing point P1 do not overlap. When the positional relationship is overlapped with the positional relationship, the selection confirmation region R11 (an example of the second determination region) is further displayed on the display unit 12.

  Thereby, since the terminal device 10 confirms the selection by further overlapping the gazing point P1 on the selection confirmation region R11 that is displayed after the user overlaps the gazing point P1 with the determination region R1, the terminal device 10 can suppress the erroneous operation for a short time. The selected object can be selected with. Moreover, the terminal device 10 can make it easier to determine the selection object that the user wants to select by not displaying unnecessary information at the stage of selecting the selection object.

  In addition, the display control unit 114 causes the display unit 12 to display a visual field image in the visual field direction from the virtual viewpoint in the virtual space based on the information regarding the direction of the terminal device 10. Thereby, the terminal device 10 can provide a visual field image in the visual field direction in the virtual space so that the user can visually recognize it according to the movement of the head of the user wearing the HDM system 1. Further, since the terminal device 10 can simultaneously display the view field image and the selected object in the virtual space, the user can select the game device with reference to the game information in the virtual space.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 9, characteristic processing in the present embodiment will be described. In the first embodiment, when the gazing point P1 overlaps the selection confirmation area of the selected object, an example is described in which it is determined that the selected object has been selected even if the predetermined time has not elapsed. In the present embodiment, an example will be described in which when the gazing point P1 overlaps the selection confirmation area of the selected object, the time until the selection determination is shortened by increasing the time count.

  FIG. 12 is a diagram illustrating an example of a selection menu screen according to the present embodiment. The illustrated selection menu screen G20 is in a state where the gazing point P1 overlaps the determination area R1 of the option A, and an elapsed time gauge T1 and a selection confirmation area R12 are displayed. In the selection confirmation region R12, an icon indicating that the time measurement speed is increased is displayed (fast forward display).

  The determination unit 112 measures the elapsed time after the gazing point P1 and the determination area of the selected object overlap with each other in a line-of-sight direction, and the gazing point P1 does not overlap the selection confirmation area. When the positional relationship overlaps from the positional relationship, the elapsed time is measured faster. As a result, the terminal device 10 can shorten the time until the selection of the selected object is confirmed by performing an operation (an operation of changing the head direction) in which the user puts the gazing point P1 in the selection confirmation area. it can.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 9, characteristic processing in the present embodiment will be described. In the first embodiment, when the gazing point P1 overlaps the selection confirmation area of the selected object, an example is described in which it is determined that the selected object has been selected even if the predetermined time has not elapsed. In the present embodiment, after the gazing point P1 overlaps the selection confirmation area of the selected object, when the gazing point P1 further moves in a predetermined manner, it is determined that the selected object has been selected even if the predetermined time has not elapsed. An example will be described.

  FIG. 13 is a diagram illustrating an example of a selection menu screen according to the present embodiment. The illustrated selection menu screen G30 is in a state where the gazing point P1 overlaps the determination area R1 of the option A, and an elapsed time gauge T1 and a selection confirmation area R13 are displayed. The selection confirmation region R13 according to the present embodiment is a rectangular region having a long side in the Y-axis direction. The start point region R131 is arranged on one end side in the Y-axis direction in this region, and the end point is located on the other end side. Region R132 is arranged. The object placement unit 113 places a selection confirmation region R13 including a start point region R131 and an end point region R132 as shown in the figure when the positional relationship is overlapped from the positional relationship where the gazing point P1 and the determination region do not overlap. Thereby, the display control unit 114 causes the display unit 12 to display a selection menu screen in which the selection confirmation region R13 as shown in FIG.

  After the user moves the gazing point P1 to a position overlapping the selection confirmation area R13, first, the user moves to the position of the start point area R131 in the selection confirmation area R13. FIG. 14 is a diagram showing the selection menu screen G31 in a state where the gazing point P1 has moved to the position of the start point region R131. When the gazing point P1 enters the start point region R131, the display mode (color or size) of the start point region R131 may be changed to indicate that the gazing point P1 has entered the start point region R131. Next, by moving within the selection confirmation region R13 from the start point region R131 to the end point region R132 in the direction of the arrow shown in FIG. 15, it is confirmed that the option A has been selected even if the predetermined time has not elapsed. . FIG. 15 is a diagram showing a selection menu screen G32 in a state where selection is confirmed as the gazing point P1 moves from the start point region R131 to the end point region R132.

  Note that when the gazing point P1 moves out of the selection confirmation region R13 while moving from the starting point region R131 to the ending point region R132, the fact that the gazing point P1 overlaps the starting point region R131 is reset. In addition, when the time measurement by the elapsed time gauge T1 is completed before the operation in the selection confirmation region R13 is completed (when a predetermined time or more has elapsed), the selection is made without waiting for the operation in the selection confirmation region R13. Is fixed. The selection is confirmed at the earlier timing. In the examples of FIGS. 13 to 15, the start point region R131 is arranged at the right end and the end point region R132 is arranged at the left end. Further, of both ends of the selection confirmation region R13, the side close to the position where the gazing point P1 enters the selection confirmation region R13 may be set as the start point region R131, and the opposite side may be set as the end point region R132. Further, in response to the point of sight point P1 entering the selection confirmation region R13, two specific regions with no start point or end point are first arranged in the selection confirmation region R13, and the gazing point P1 of the two specific regions is arranged. The specific region in which the first entered is the start point region R131, and one may be the end point region R132. Moreover, the arrow shown in FIG. 15 may be displayed as a guide.

  In other words, the determination unit 112 has a positional relationship that overlaps from a positional relationship in which the gazing point P1 and the selection confirmation region do not overlap, and a predetermined time has elapsed when the gazing point P1 moves with respect to the selection confirmation region. Even if not, it is determined that the selected object has been selected. The predetermined movement is, for example, a movement corresponding to a preset route in the selection confirmation area. In the above-described example, the gazing point P1 moves from the start point area R131 to the end point area R132.

  FIG. 16 is a flowchart illustrating an example of a determination process for determining whether or not a selected object is selected according to the present embodiment. With reference to this figure, the operation of the determination process for determining whether or not the selected object is selected according to the present embodiment will be described. In FIG. 16, processes corresponding to the processes shown in FIG. 11 are denoted by the same reference numerals, and description thereof is omitted. The process shown in FIG. 16 is different from the process after it is determined (YES) in step S210 in FIG. 11 that the gazing point P1 and the selection confirmation area overlap. Here, as shown in FIG. 13, for example, the selection confirmation region R13 is displayed when the gazing point P1 overlaps the determination region R1 of the option A, and the gazing point P1 and the selection confirmation region R11 overlap.

  When it is determined in step S210 that the gazing point P1 and the selection confirmation area R11 overlap (YES), the control unit 110 determines whether or not the gazing point P1 and the start point area R131 overlap (step S212). . When it is determined that the gazing point P1 and the start point region R131 do not overlap (NO), the control unit 110 returns to the process of step S208, and if the predetermined time has not elapsed, the gazing point P1 and the start point region R131 are returned. Wait for and to overlap. On the other hand, when it is determined that the gazing point P1 and the start point region R131 overlap (YES, see FIG. 14), the control unit 110 then determines whether or not the gazing point P1 and the end point region R132 overlap ( Step S214).

  When it is determined that the gazing point P1 and the end point region R132 do not overlap (NO), the control unit 110 determines whether or not the gazing point P1 and the selection confirmation region R11 overlap (step S216). If it is determined (YES), the process returns to step S208, and if the predetermined time or more has not elapsed, the process waits for the gazing point P1 and the end point region R132 to overlap. If it is determined in step S216 that the gazing point P1 and the selection confirmation region R11 do not overlap before the gazing point P1 and the end point region R132 overlap (NO), the control unit 110 performs the process in step S204. It returns and resets that there was operation to start point field R131.

  On the other hand, when it is determined in step S214 that the gazing point P1 overlaps the end point region R132 (YES), the control unit 110 has passed a predetermined time or more after the gazing point P1 overlaps the determination region of the selected object. If not, it is determined that the selected object has been selected, and the determination process is terminated (step S220).

  In the example illustrated in FIG. 16, when the gazing point P1 is out of the selection confirmation region R11 after the gazing point P1 and the starting point region R131 overlap, the gazing point P1 overlaps the starting point region R131 again. It is necessary to operate so that the gazing point P1 and the end point region R132 overlap. However, after the gazing point P1 and the start point region R131 overlap, even if the gazing point P1 deviates from the selection confirmation region R11, the fact that the gazing point P1 and the start point region R131 overlap is not reset, A configuration may be adopted in which selection of the selected object is confirmed even if the gazing point P1 and the end point region R132 are operated to overlap.

  In the present embodiment, the start point region R131 and the end point region R132 are displayed in the selection confirmation region R13 in the determination region R1, but the selection confirmation region R13 is not displayed, and the start point region R131 is displayed. The end point region R132 may be displayed in the determination region R1 as the selection confirmation region.

  As described above, the determination unit 112 of the terminal device 10 according to the present embodiment has a positional relationship in which the selection confirmation region R13 (an example of the second determination region) and the gazing point P1 (an example of the operation point) do not overlap. When the gazing point P1 moves in a predetermined manner relative to the selection confirmation area R13, the selected object corresponding to the selection confirmation area R13 is selected even if the predetermined time has not elapsed. judge.

  As a result, the terminal device 10 is not merely an operation for confirming the selection, but also an operation for superimposing the gazing point P1 on the selection confirmation region R13. Therefore, the terminal device 10 unintentionally superimposes the gazing point P1 on the selection confirmation region R13. The erroneous operation that occurs can be prevented, and the time for finalizing the selection of the selected object can be shortened.

  For example, the predetermined movement is a movement corresponding to a preset route in the selection confirmation region R13. For example, the preset route is a movement from the start point region R131 to the end point region R132.

  Thereby, since the terminal device 10 needs an intentional operation within the selection confirmation region R13 in order to confirm the selection of the selected object, an erroneous operation can be prevented.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 9, characteristic processing in the present embodiment will be described. As described with reference to FIGS. 13 to 15, in the third embodiment, the predetermined movement that confirms the selection of the selected object is the movement of the gazing point P1 with respect to the path from the start point region R131 to the end point region R132. However, the present invention is not limited to this. For example, the movement of the gazing point P1 with respect to a route passing through a plurality of areas may be used. For example, a specific area may be further provided on the route from the start point area R131 to the end point area R132, and the movement of the gazing point P1 with respect to the path including the specific area may be used.

  FIG. 17 is a diagram illustrating an example of a selection menu screen according to the present embodiment. The illustrated selection menu screen G40 is in a state where the gazing point P1 overlaps the determination area R1 of the option A, and an elapsed time gauge T1 and a selection confirmation area R14 are displayed. The selection confirmation region R14 according to the present embodiment is a U-shaped region. A start point region R141 is disposed at one end of the selection confirmation region R14, and an end point region R144 is disposed at the other end. In addition, a passage point region R142 and a passage point region R143 are sequentially arranged as specific regions in the middle of the U-shaped route from the start point region R141 to the end point region R144 (in each of two corners on the route). Yes. After the user moves the gazing point P1 to a position overlapping the selection confirmation area R14, the user first moves to the position of the start point area R141 in the selection confirmation area R14, and then passes through the passing point area R142 in the selection confirmation area R13. The point A is moved so as to pass through the point region R143 in order, and finally moved to the end point region R144 (by moving in the direction of the arrow shown in FIG. 17), so that the option A can be selected even if the predetermined time has not elapsed. The selection is confirmed. Note that the arrow shown in FIG. 17 may be displayed as a guide.

  For example, the determination unit 112 determines whether or not the gazing point P1 (an example of the operation point) has passed through a specific region existing on the route in the selection confirmation region (an example of the second determination region) (for example, in a predetermined order). It is determined whether or not the gazing point P1 has made a predetermined movement with respect to the selection confirmation area within the selection confirmation area.

  Thereby, since the terminal device 10 needs a complicated operation in the selection confirmation region R13 in order to confirm the selection of the selected object, it is possible to further prevent an erroneous operation. For example, in an important selection-related part such as purchase settlement by billing, a complicated operation as in this embodiment is effective for the purpose of enhancing prevention of erroneous operations.

  In the present embodiment, an example in which the predetermined movement that confirms the selection of the selected object is a movement in which the gazing point P1 passes through the start point region R141, the pass point region R142, the pass point region R143, and the end point region R144 in this order. However, the order of passing through the plurality of regions may not be determined. That is, the determination unit 112 may determine that a predetermined movement has been made with respect to the selection confirmation region, regardless of the order in which the gazing point P1 passes through the plurality of regions.

  Further, the start point region R141, the pass point region R142, the pass point region R143, and the end point region R144 may not be displayed at a time from the beginning. For example, only the start point region R141 is initially displayed. When the gazing point P1 overlaps the start point region R141, the next passing point region R142 is displayed. When the gazing point P1 overlaps the passing point region R142, the next passing point region R143 is further displayed. Each area may be sequentially displayed as if it is displayed. Further, the number and position of the passing point regions and the shape of the route can be arbitrarily determined.

  Also, in this embodiment, has the gazing point P1 moved to draw a U-shape depending on whether or not the gazing point P1 has passed in the order of the starting point region R141, the passing point region R142, the passing point region R143, and the ending point region R144? Although it is determined whether or not, each area may not be set. For example, what kind of trajectory the gazing point P1 draws may be determined based on a difference between a preset trajectory and the trajectory that the gazing point P1 moves.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 9, characteristic processing in the present embodiment will be described. In the present embodiment, the elapsed time gauge is displayed as the selection confirmation area, and the selection of the selected object is confirmed when the gazing point P1 moves with respect to the elapsed time gauge.

  FIG. 18 is a diagram showing an example of a selection menu screen according to the present embodiment. The illustrated selection menu screen G50 is in a state in which the gazing point P1 overlaps the determination area R1 of the option A, and a selection confirmation area R15 is displayed. The selection confirmation region R15 according to the present embodiment also serves as an elapsed time gauge. That is, the elapsed time gauge is displayed as the selection confirmation region R15. The selection confirmation region R15 is a linear elapsed time gauge extending in the Y-axis direction, and the length of the bar is one end (here, the left end) in the linear direction as time passes after the gazing point P1 overlaps the determination region R1. Increases toward the other end (here, the right end), and this bar extends to the right end when a predetermined time has elapsed. When the gazing point P1 approaches the tip portion B1 on the side extending toward the right end of the bar before a predetermined time or more has elapsed, the tip portion B1 and the gazing point P1 are brought into close contact with each other, and the movement of the gazing point P1 is caused. The tip portion B1 is interlocked. That is, the length of the bar follows the movement of the gazing point P1 regardless of the elapsed time.

  FIG. 19 is a diagram illustrating an example of a selection menu screen in a state where the elapsed time gauge follows the movement of the gazing point P1. The selection menu screen G51 shown in the figure shows a state in which the gazing point P1 approaches the tip part B1 and the tip part B1 of the bar follows the movement of the gazing point P1. In this state, when the gazing point P1 is moved to the right (in the direction of the arrow), the length of the bar extends to the right regardless of the elapsed time. When the length of the bar extends to the right end, it is determined that option A has been selected. In this state, when the gazing point P1 is moved in the left direction (the direction opposite to the arrow), the length of the bar may be shortened to the left regardless of the elapsed time, or the length of the bar You may make it not change.

  For example, when the determination area R1 and the gazing point P1 do not overlap with each other, the object placement unit 113 makes it possible to visually recognize the progress of the elapsed time after the overlapping position relation. An elapsed time gauge is arranged as the selection confirmation region R15. That is, the display control unit 114 is configured to make it possible to visually recognize the progress of the elapsed time since the overlapping positional relationship when the determination region R1 and the gazing point P1 are not overlapped. The elapsed time gauge is displayed on the display unit 12 as the selection confirmation region R15. Then, the determination unit 112 determines that the selected object has been selected even if the predetermined time has not elapsed when the gazing point P1 has made a predetermined movement with respect to the elapsed time gauge. The predetermined movement is a movement that extends to the length of the bar after a predetermined time has elapsed. For example, as described with reference to FIGS. 18 and 19, the gazing point P1 is the tip of the bar of the elapsed time gauge. This is a movement of moving to the right end of the elapsed time gauge (selection confirmation region R15) after approaching the portion B1.

  In this way, the terminal device 10 is forced and intuitive without waiting for the passage of time by pulling and extending the bar of the elapsed time gauge that is measuring the elapsed time at the gazing point P1 by the user's operation. The selection can be confirmed. In addition, when the user pulls and extends the bar of the elapsed time gauge halfway at the gazing point P1, the time until the selection is confirmed is shortened.

[Sixth Embodiment]
Next, a sixth embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 9, characteristic processing in the present embodiment will be described. In the first to fourth embodiments, the example in which the selection confirmation area is displayed in the selection object (determination area) has been described. However, the selection confirmation area may be displayed outside the selection object (determination area).

  FIG. 20 is a diagram illustrating an example of a selection menu screen according to the present embodiment. The illustrated selection menu screen G60 is in a state in which the gazing point P1 overlaps the determination area R1 of the option A, and in addition to the elapsed time gauge T1, a selection confirmation area R161 and a cancellation area R162 are displayed. The selection confirmation area R161 and the cancellation area R162 are displayed outside the determination area R1 of the option A. The selection confirmation region R161 is a region for confirming the selection to the selection object (here, option A) where the gazing point P1 overlaps. The cancel area R162 is an area for canceling the selection to the selected object (here, option A) where the gazing point P1 overlaps.

  In this state, when the user moves the gazing point P1 to a position overlapping the selection confirmation region R161, it is confirmed that the option A has been selected even if a predetermined time has not elapsed. When the user moves the gazing point P1 to a position overlapping the cancel area R162, the fact that the gazing point P1 overlaps the determination area R1 of the option A is canceled, and the selection confirmation area R161 and the cancellation area R162 are hidden. Become. When the gazing point P1 is in a position that does not overlap any of the determination region R1, the selection confirmation region R161, and the cancellation region R162, the counting of the elapsed time after the gazing point P1 overlaps the determination region R1 is interrupted (elapsed time) The bar of the gauge T1 is stopped without extending or contracting).

  For example, the object placement unit 113 is located outside the determination region R1 and in the vicinity of the determination region R1 when the determination region R1 and the gazing point P1 do not overlap with each other. Region R162 is arranged. That is, the display control unit 114 is outside the determination region R1 and in the vicinity of the determination region R1 and cancels the selection confirmation region R161 and the cancel when the determination region R1 and the gazing point P1 do not overlap. The region R162 is displayed on the display unit 12.

  The place where the selection confirmation area R161 is arranged may be a position set in advance with reference to the position of the determination area R1, or a position set in advance based on the position of the gazing point P1 overlapping the determination area R1. It is good. For example, the object placement unit 113 is located outside the determination region R1 and in a predetermined direction and distance from the gazing point P1 when the determination region R1 and the gazing point P1 do not overlap with each other. A selection confirmation region R161 may be arranged. For example, if the selection confirmation region is always displayed at a fixed distance on the upper left with respect to the gazing point P1, the same operation (after the user places the gazing point P1 in the determination region of any selected object) Since the selection can be confirmed by turning to the upper left), convenience is enhanced. Similarly, the cancellation region R162 may be set in advance with reference to the position of the determination region R1, or may be set in advance with reference to the position of the gazing point P1 overlapping the determination region R1. .

  Note that the cancel region R162 does not have to be displayed at the timing when the selection confirmation region R161 is displayed. FIG. 21 is a diagram illustrating an example of a selection menu screen on which only the selection confirmation region R161 is displayed in the present embodiment. In this case, the cancel area R162 may be displayed when the gazing point P1 is out of the determination area R1 of the option A, or for a while after the gazing point P1 is out of the determination area R1 of the option A (for example, 2 -After 3 seconds) Cancel region R162 may be displayed. Further, after a short time (for example, after 2 to 3 seconds) after the gazing point P1 deviates from the determination area R1 of the option A, the selection confirmation area R161 may be changed to the cancellation area R162, or the cancellation area R162 is displayed. The selection confirmation region R161 may be canceled without being canceled.

[Modification]
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiment, and includes a design and the like within a scope not departing from the gist of the present invention. For example, the configurations described in the above first to sixth embodiments can be arbitrarily combined.

  In the above-described embodiment, the example in which the selection confirmation area is displayed when the gazing point P1 overlaps the determination area of the selected object has been described. However, the present invention is not limited to this, and the selection confirmation area is not the selection object (determination area). ) And the selected object may be displayed from the beginning.

  In the above embodiment, the positional relationship between the gazing point P1 and the selection confirmation region overlaps after the positional relationship where the gazing point P1 and the determination region do not overlap each other, or the gazing point P1 is further changed. Although an example has been described in which it is determined that a selected object has been selected even if a predetermined time has not passed on condition that a predetermined movement has been made, the present invention is not limited to this. For example, the determination unit 112 has a positional relationship that overlaps from a positional relationship in which the selection confirmation region (an example of the second determination region) and the gazing point P1 do not overlap, and the detection unit 111 has an X-axis direction (a user's line-of-sight direction). When the information on the change in direction of the terminal device 10 corresponding to the change in the roll direction (rotation direction) with the rotation axis as the rotation axis is detected, even if it is determined that the selected object has been selected even if the predetermined time has not elapsed Good. Here, the direction of the terminal device 10 corresponds to the direction of the head of the user wearing the HMD 1 on the head. As a result, the terminal device 10 suppresses erroneous operations because the selection is confirmed by the user tilting his / her head in the roll direction more than a certain amount even if a predetermined time has not elapsed since the gazing point P1 overlaps the determination region R1. The selected object can be selected in a short time. Note that the terminal device 10 moves the head within a predetermined time (for example, within one second) after the gazing point P1 overlaps the determination region R1 (how much it has been rotated more than a certain amount from side to side or up and down), etc. Thus, it may be determined that the selected object has been selected even if the predetermined time has not elapsed.

  In the above embodiment, the example in which the arrangement direction of the selection objects is the Y-axis direction has been described. However, the arrangement direction of the selection objects is not limited to the Y-axis direction. It may be a direction having a Y-axis component and a Z-axis component. When the selected objects are arranged in three dimensions, the arrangement direction of the selected objects may be a direction having an X-axis component, a Y-axis component, and a Z-axis component.

  In the present embodiment, an example in which the line-of-sight direction is detected based on the direction of the terminal device 10 has been described. However, the present invention is not limited to this. For example, the line-of-sight direction is detected by detecting the direction of the user's eyeball. (Eye tracking) may be used. In that case, since the line-of-sight direction is based on detection different from the movement of the head, the gazing point P1 moves in the view field image, not in a fixed position in the view field image.

  Moreover, in the said embodiment, although the structure of the HMD system 1 which can be utilized as HMD by mounting | wearing the terminal device 10 with the attachment 2 was demonstrated to the example, it is not restricted to this, For example, the terminal device 10 is an attachment. Even if 2 is not required, it may be a form that can be mounted on the head itself, that is, an HMD. An HMD that does not require an attachment is used as a display device by being console-connected to, for example, a personal computer or a game machine. In this case, for example, the display unit 12 and the sensor 13 shown in FIG. 2 are provided on the HMD side, and the timer 14, the storage unit 15, and the CPU 17 are provided on the console side (such as a personal computer or a game machine). The communication unit 16 is provided on both the HMD side and the console side, and communicates with each other. A CPU for communication control and the like is also provided on the HMD side.

  Moreover, in the said embodiment, although the terminal device 10 demonstrated the structural example which can be mounted | worn to a user's head as HMD (HMD system) by which at least one part is comprised, HMD can mount | wear to a head completely. The configuration is not limited to a simple shape. For example, the terminal device 10 that can be used as an HMD maintains the positional relationship between the user's eyes and the display unit 12 in a predetermined relationship by holding the user's eyes in front of the eyes regardless of whether or not there is an attachment. It may be a device (for example, a shape like binoculars) that makes it possible to visually recognize the view field image and various objects.

  In the above embodiment, an example of a selection object as an option displayed on the selection menu screen has been described. However, the present invention is not limited to this. For example, the selected object may be an object arranged as a character, an item, or the like having a determination area in a virtual space in a game executed by the terminal device 10. As an example, the process related to the selected object described in the first embodiment may be applied to a process for a plurality of enemy objects (selected objects) flying in the virtual space of the game. For example, an attack is irradiated based on the user's gazing point P1, and the enemy object (selected object) is shot down when the irradiation (positional relationship where the determination area and the gazing point P1 overlap) elapses for a predetermined time. The present invention may be applied to a process of selecting a selected object.

  In the above embodiment, the selection menu screen displayed in the HMD system 1 has been described as an example, but the present invention is not limited to the HMD system 1. That is, the terminal device 10 on which the selection menu screen is displayed may not be worn on the head, and may be used, for example, as it is held by a user or placed on a desk. Further, the terminal device 10 may display a selection menu screen on an installed television or monitor. In this case, the operation method for the selection menu screen is an operation method using a touch panel that accepts a touch operation on the screen, a pointing device such as a mouse, a joystick, or a touch pad, or a pointing device that recognizes an operation caused by a human movement such as a gesture. There may be. That is, the gazing point P1 may be an operation point operated by each operation method.

  Further, the control unit 110 is recorded by recording a program for realizing the functions of the control unit 110 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. You may perform the process of. Here, “loading and executing a program recorded on a recording medium into a computer system” includes installing the program in the computer system. The “computer system” here includes an OS and hardware such as peripheral devices. Further, the “computer system” may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and dedicated line. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM. The recording medium also includes a recording medium provided inside or outside that is accessible from the distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program that can be executed by the terminal device. That is, the format stored in the distribution server is not limited as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device. Note that the program may be divided into a plurality of parts, downloaded at different timings, and combined in the terminal device, or the distribution server that distributes each of the divided programs may be different. Furthermore, a “computer-readable recording medium” holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or client when the program is transmitted via a network. Including things. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  Further, part or all of the functions of the control unit 110 described above may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function described above may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

[Appendix]
From the above description, the present invention is grasped as follows, for example. In order to facilitate understanding of the present invention, reference numerals in the accompanying drawings are appended in parentheses for convenience, but the present invention is not limited to the illustrated embodiment.

  (Supplementary Note 1) A terminal device (10) according to one aspect of the present invention includes a detection unit (111, S112) that detects an operation point of a user in a virtual space, and a selection object that can be selected by the operation of the user. An object placement unit (113, S100) placed in the space, a first determination region (R1) for judging whether or not the selected object placed in the virtual space has been selected, and detection by the detection unit The display control unit that causes the display unit (12) to display the operated operation point (P1) and a second determination region (R11, R12, R13, R14, R15, R161) different from the first determination region. (114, S100, S104) and a predetermined time after the first determination area displayed on the display unit and the operation point do not overlap with each other. The first determination unit (112, S208) that determines that the selected object has been selected on the condition that it has passed, and that the positional relationship between the second determination region and the operation point satisfies a predetermined condition. As a condition, a second determination unit (112, S210) that determines that the selected object has been selected even if the predetermined time has not elapsed.

  According to the configuration of Supplementary Note 1, in addition to the first determination area in which the selection is confirmed on the condition that a predetermined time has elapsed since the operation points overlap, the terminal device has the operation point in the first determination area. Since the second determination area in which selection is confirmed even if a predetermined time does not elapse after overlapping, an operation method that allows selection in a short time while suppressing erroneous operation can be realized.

  (Additional remark 2) Moreover, 1 aspect of this invention is a terminal device of Additional remark 1, Comprising: As for the said 2nd determination part, the said 2nd determination area | region (R11) and the said operation point (P1) overlap. When the positional relationship is overlapped from the positional relationship that does not exist, it is determined that the selected object has been selected even if the predetermined time has not elapsed.

  According to the configuration of Supplementary Note 2, the terminal device is operated by the user so that the operation point overlaps the second determination area even if a predetermined time has not elapsed since the operation point overlaps the first determination area. Since the selection is finalized, the selected object can be selected in a short time.

  (Additional remark 3) Moreover, 1 aspect of this invention is a terminal device of Additional remark 1, Comprising: The said 2nd determination part is said 2nd determination area | region (R13, R14, R15) and said operation point (P1). When the operation point has a predetermined movement with respect to the second determination area, the selected object is selected even if the predetermined time has not elapsed. It is determined that

  According to the configuration of Supplementary Note 3, the terminal device does not intend to overlap the operation point with the second determination region because the operation for confirming the selection is not simply an operation to overlap the operation point with the second determination region. Thus, it is possible to prevent an erroneous operation caused by the error, and to shorten the time for finalizing the selection of the selected object.

  (Additional remark 4) Moreover, 1 aspect of this invention is a terminal device of Additional remark 3, Comprising: The said predetermined motion respond | corresponds to the preset path | route in the said 2nd determination area | region (R13, R14). It is a movement to do.

  According to the configuration of Supplementary Note 4, since the terminal device needs an intentional operation within the second determination region in order to confirm the selection of the selected object, it is possible to prevent an erroneous operation.

  (Additional remark 5) Moreover, 1 aspect of this invention is a terminal device of Additional remark 4, Comprising: The said 2nd determination part is a specific area | region which exists on the said path | route in the said 2nd determination area | region (R14). Based on whether or not the operation point (P1) has passed through (R142, R143), whether or not the operation point has made a predetermined movement with respect to the second determination region within the second determination region Determine whether.

  According to the configuration of Supplementary Note 5, since the terminal device requires a complicated operation in the second determination region in order to confirm the selection of the selected object, it is possible to further prevent an erroneous operation. For example, in a portion related to an important selection such as purchase settlement by billing, a complicated operation is effective for the purpose of enhancing prevention of erroneous operation.

  (Additional remark 6) Moreover, 1 aspect of this invention is a terminal device of Additional remark 1, Comprising: As for the said display control part, the said 1st determination area | region (R1) and the said operation point (P1) do not overlap. When the positional relationship is overlapped from the positional relationship, an elapsed time gauge for making it possible to visually recognize the progress of the elapsed time since the overlapping positional relationship is displayed on the display unit as the second determination region (R15). The second determination unit determines that the selected object has been selected even if the predetermined time has not elapsed when the operation point has made a predetermined movement with respect to the elapsed time gauge.

  According to the configuration of Supplementary Note 6, the terminal device can be forced without waiting for the passage of time by pulling and extending the bar of the elapsed time gauge that measures the elapsed time at the point of interest P1 by the user's operation, and Intuitive selection can be confirmed.

  (Additional remark 7) Moreover, 1 aspect of this invention is a terminal device as described in any one of Additional remark 1 to Additional remark 6, Comprising: Said 2nd determination area | region is said 1st in said 1st determination area | region. This area is smaller than the determination area.

  According to the configuration of Supplementary Note 7, the terminal device confirms the selection by overlapping the operation point on the first determination area and then overlapping the second determination area in the first determination area. Therefore, it is possible to select the selected object in a short time while suppressing erroneous operations.

  (Supplementary Note 8) One aspect of the present invention is the terminal device according to any one of Supplementary Notes 1 to 7, wherein the display control unit includes the first determination region and the operation point. After being displayed on the display unit, the second determination region is further displayed on the display unit when the first determination region and the operation point are in a positional relationship that overlaps from a positional relationship that does not overlap.

  According to the configuration of Supplementary Note 8, since the terminal device further determines the selection by overlapping the operation point on the second determination area displayed after the user overlaps the operation point on the first determination area, It is possible to select the selected object in a short time while suppressing it. Further, the terminal device can make it easier to determine the selection object that the user wants to select by not displaying unnecessary information at the stage of selecting the selection object.

  (Supplementary Note 9) One aspect of the present invention is the terminal device according to any one of Supplementary Note 1 to Supplementary Note 8, in which a stereoscopic image using binocular parallax is viewed from a virtual viewpoint in a virtual space. A terminal device that can be used as at least a part of a head-mounted display system (HMD) capable of displaying a field-of-view image representing a field of view, wherein the detection unit detects information related to a direction of the terminal device and A gazing point corresponding to a line-of-sight direction from the virtual viewpoint in space is detected as the operation point, and the display control unit is configured to view the field of view from the virtual viewpoint in the virtual space based on information on the direction of the terminal device. The selected object and the gazing point arranged in the direction are displayed on the display unit according to the direction of the terminal device.

  According to the configuration of Supplementary Note 9, the terminal device that can be used as at least a part of the HMD includes, in addition to the first determination region in which selection is determined on the condition that a predetermined time has passed since the gazing points overlapped, Since the second determination area is provided in which the selection is confirmed even if a predetermined time has not elapsed after the gazing point overlaps the first determination area, an operation method that allows selection in a short time while suppressing erroneous operations can be realized.

  (Additional remark 10) Moreover, 1 aspect of this invention is a terminal device of Additional remark 9, Comprising: The direction of the said terminal device respond | corresponds to the direction of the head of the user who mounted | worn the said head mounted display system on the head The second determination unit has a positional relationship that overlaps from a positional relationship in which the second determination region and the gazing point do not overlap, and the detection unit rotates about the user's line-of-sight direction as a rotation axis When information related to the change in direction of the terminal device corresponding to the change in direction is detected, it is determined that the selected object has been selected even if the predetermined time has not elapsed.

  According to the configuration of Supplementary Note 10, the terminal device determines the selection by tilting the head more than a certain amount in the roll direction even if a predetermined time has not elapsed since the gazing point overlaps the first determination region. It is possible to select the selected object in a short time while suppressing erroneous operations.

  (Additional remark 11) Moreover, 1 aspect of this invention is a terminal device of Additional remark 9 or Additional remark 10, Comprising: The said display control part is the said in the said virtual space based on the information regarding the direction of the said terminal device. A visual field image in the visual field direction from the virtual viewpoint is displayed on the display unit.

  According to the configuration of Supplementary Note 11, the terminal device can provide a visual field image in the visual field direction in the virtual space so that the user can visually recognize the movement according to the movement of the head of the user wearing the HDM system.

  (Additional remark 12) Moreover, the program which concerns on 1 aspect of this invention is a program for functioning a computer as a terminal device as described in any one of Additional remark 1 to Additional remark 11.

  According to the configuration of Supplementary Note 12, the program overlaps the first determination area in addition to the first determination area where the selection is determined on the condition that a predetermined time has passed since the operation points overlapped. Since the second determination area in which the selection is confirmed even if a predetermined time has not elapsed since then, an operation method that can be selected in a short time while suppressing erroneous operations can be realized.

  1 HMD system, 2 attachment, 3R lens for right eye, 3L lens for left eye, 5 strap, 10 terminal device, 12 display unit, 13 sensor, 14 timer, 15 storage unit, 16 communication unit, 17 CPU, 110 control unit, 111 Detection unit, 112 determination unit, 113 object placement unit, 114 display control unit

Claims (12)

A detection unit for detecting a user's operation point in the virtual space;
An object placement unit for placing a selection object selectable by the user's operation in the virtual space;
A first determination region for determining whether or not the selected object arranged in the virtual space has been selected, the operation point detected by the detection unit, and a second different from the first determination region A display control unit for displaying the determination area on the display unit,
It is determined that the selected object has been selected on the condition that a predetermined time has elapsed since the first determination area displayed on the display unit and the operation point do not overlap with each other. A first determination unit to perform,
A second determination unit that determines that the selected object has been selected even if the predetermined time has not passed, provided that a positional relationship between the second determination region and the operation point satisfies a predetermined condition; ,
A terminal device comprising: The second determination unit includes
If the second determination area and the operation point do not overlap with each other, the determination is that the selected object has been selected even if the predetermined time has not elapsed.
The terminal device according to claim 1. The second determination unit includes
When the second determination area and the operation point do not overlap with each other, the predetermined time elapses when the position relationship overlaps and the operation point moves with respect to the second determination area. Even if not, it is determined that the selected object is selected.
The terminal device according to claim 1. The predetermined movement is a movement corresponding to a preset route in the second determination area.
The terminal device according to claim 3. The second determination unit includes
Based on whether or not the operation point has passed through a specific area existing on the route in the second determination area, the operation point is compared with the second determination area in the second determination area. To determine whether or not it has made a certain movement,
The terminal device according to claim 4. The display control unit
When the first determination area and the operation point do not overlap with each other, the elapsed time gauge for enabling visual recognition of the progress of the elapsed time since the overlapped positional relationship is obtained. Display on the display unit as a second determination area,
The second determination unit includes
When the operation point has made a predetermined movement with respect to the elapsed time gauge, it is determined that the selected object has been selected even if the predetermined time has not elapsed.
The terminal device according to claim 1. The second determination area is an area smaller than the first determination area in the first determination area.
The terminal device as described in any one of Claims 1-6. The display control unit
After the first determination area and the operation point are displayed on the display unit, the first determination area and the operation point are further overlapped with each other when the first determination area and the operation point do not overlap with each other. 2 determination areas are displayed on the display unit,
The terminal device as described in any one of Claims 1-7. As the stereoscopic image using binocular parallax, the terminal device that can be used as at least a part of a head-mounted display system capable of displaying a visual field image representing a visual field from a virtual viewpoint in a virtual space,
The detector is
Detecting information related to the direction of the terminal device, and detecting a gazing point corresponding to a line-of-sight direction from the virtual viewpoint in the virtual space as the operation point;
The display control unit
Based on the information regarding the direction of the terminal device, the selection object and the gazing point arranged in the visual field direction from the virtual viewpoint in the virtual space are displayed on the display unit according to the direction of the terminal device. ,
The terminal device as described in any one of Claims 1-8. The direction of the terminal device corresponds to the direction of the head of the user wearing the head mounted display system on the head,
The second determination unit includes
The direction of the terminal device is a positional relationship that overlaps from a positional relationship in which the second determination area and the gazing point do not overlap, and the detection unit responds to a change in the rotation direction with the user's line-of-sight direction as the rotation axis If the information related to the change in the position is detected, it is determined that the selected object has been selected even if the predetermined time has not elapsed.
The terminal device according to claim 9. The display control unit
Based on the information related to the direction of the terminal device, the visual field image of the visual field direction from the virtual viewpoint in the virtual space is displayed on the display unit.
The terminal device according to claim 9 or 10.   The program for functioning a computer as a terminal device as described in any one of Claims 1-11.

Images