JP2018010702A - Terminal device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operation property when performing a selection operation to choices displayed on a HMD.SOLUTION: A terminal device which can be used as a head mount display system detects information related to a direction of a terminal device, arranges a visual line position object corresponding to a visual line direction from a virtual viewpoint in a virtual space, arranges plural selection objects having a determination area, in the virtual space in a prescribed direction, then, according to the detection result, displays at least some of the plural selection objects arranged in the virtual space and the visual line position object on a display part, then, based on a timing in which, a relative position of the visual line position and a determination area of the one selection object included in the plural selection objects, is in a specific state, determines presence/absence of selection of one selection object, and if it is determined that, a view direction is changed, moves the selection object while limiting movement of the plural selection object in an arrangement direction, based on change of the view direction.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a terminal device and a program.

  There is a head mounted display (HMD) that can be mounted on a user's head and can display an image in a virtual space on a display arranged in front of the user's eyes. For example, in a game using HMD, there is a game that plays a game using a device such as a controller that is held and operated in addition to a display worn on the head. However, since the user can not observe the hand because the field of view of the real space is covered by the HMD, an erroneous operation may occur when the finger is once removed from the controller. In view of this, there is an HMD that detects a motion such as a tilt of the head and performs a display / non-display operation of options (for example, Patent Document 1).

  In recent years, some smartphones have been used as HMDs with higher performance. For example, it can be used as an HMD by attaching a smartphone to the head using a simple attachment. In this case, an input device such as a touch panel provided in the smartphone cannot be used by being worn on the head. For this reason, for example, it is necessary to realize an operation for an option displayed on the HMD by detecting the movement of the head using a sensor such as a gyro built in the smartphone.

Japanese Patent No. 5767386

  When performing an operation using the movement of the head for an option displayed on the HMD, the direction of the head is set within a specific range (within the range of the option determination area), and in that state, a predetermined time ( For example, there is an operation method for confirming the selection after elapse of 3 seconds. However, in such an operation method, when the selection must be made from a large number of options, the range of the determination area of each option displayed on the HMD becomes small, so that the unstable head When the operation is performed by movement, an erroneous input such as unintentional deviation from the range of the determination area occurs, and the operability is poor.

  It is an object of some aspects of the present invention to provide a terminal device and a program that improve operability when performing a selection operation on options displayed on the HMD.

  Another object of another aspect of the present invention is to provide a terminal device and a program that can achieve the effects described in the embodiments to be described later.

  In order to solve the above-described problem, one aspect of the present invention provides at least a head-mounted display system capable of displaying a visual field image representing a visual field from a virtual viewpoint in a virtual space as a stereoscopic image using binocular parallax. A terminal device that can be used as a part, and a plurality of selection objects having a detection unit that detects information related to the direction of the terminal device and a determination region that determines whether or not there is a selection from the user, in the virtual space An object placement unit arranged side by side in a predetermined direction, and a display control unit that causes the display unit to display the selected object placed in the view direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit And the object placement unit has a visual field direction from the virtual viewpoint in the virtual space based on a detection result of the detection unit. If it is determined that the phased moves the selected object while limiting the movement to the predetermined direction based on the change of the viewing direction, a terminal device.

  One embodiment of the present invention is a program for causing a computer to function as the above terminal device.

1 is an external view showing an example of an HMD system according to a first embodiment. The figure which shows an example of the hardware constitutions of the terminal device which concerns on 1st Embodiment. The figure which shows the definition of the direction of the virtual space which concerns on 1st Embodiment. Explanatory drawing of the visual field direction and gaze position which concern on 1st Embodiment. The figure which shows an example of a selection menu screen. The figure which shows an example of the selection menu screen in which the eyes | visual_axis position overlapped with the selection object. The figure which shows an example of the selection menu screen from which the determination area | region changed. The figure which shows the example of a selection menu screen when selection of a selection object is decided. The figure which shows an example of the selection menu screen in case there are many selection objects. The figure which shows the example of a selection menu screen where the eyes | visual_axis position overlapped with the selection object. The block diagram which shows an example of a function structure of the terminal device which concerns on 1st Embodiment. 6 is a flowchart illustrating an example of object placement processing according to the first embodiment. 6 is a flowchart illustrating an example of a selection object determination process according to the first embodiment. The figure which shows an example of the selection menu screen with which the eyes | visual_axis position and the determination area | region of the selection object overlap. The figure which shows an example of the selection menu screen used as the state by which a visual line position and the determination area | region of the selection object did not overlap. 6 is a flowchart illustrating an example of a selection object determination process according to the first embodiment. The figure which shows an example of the selection menu screen which concerns on 2nd Embodiment. The figure which shows an example of the selection menu screen when a user's visual field direction changes to a pitch direction. The figure which shows an example of the selection menu screen when a user's visual field direction changes to a yaw direction. The figure which shows an example of the selection menu screen when a user's visual field direction changes to a roll direction. 10 is a flowchart illustrating an example of object placement processing according to the second embodiment. The figure which shows an example of the selection menu screen which concerns on 3rd Embodiment. 10 is a flowchart illustrating an example of object placement processing according to the third embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[First embodiment]
A first embodiment of the present invention will be described.
FIG. 1 is an external view showing an example of an HMD (Head Mounted Display) system according to the present embodiment. The HMD is worn on the user's head and can display a view image representing the view from a virtual viewpoint in the virtual space. For example, the HMD can display a stereoscopic image using binocular parallax between the right eye and the left eye. Also, the HMD is equipped with a sensor that detects the movement and tilt of the HMD such as a gyro, detects the movement of the head of the user who is wearing it, changes in the tilt, etc., and displays the virtual on the display according to the change. A view image in the view direction of the space is displayed. For example, the visual field image displayed on the HMD changes to a right visual field image in the virtual space if the user's head is directed to the right, and changes to an upward visual image in the virtual space if the user heads upward. It is possible to give the user an immersive feeling as if they were there.

  The illustrated HMD system 1 is configured such that the terminal device 10 can be used as an HMD by attaching the terminal device 10 including the display unit 12 to the attachment 2. The attachment 2 can be attached to the terminal device 10 so as to cover the field of view of the front of the user while attached to the user's head, and the right-eye lens for visually recognizing the display unit 12 of the attached terminal device 10 3R and the left-eye lens 3L, and a strap 5 for mounting the HMD system 1 on the user's head. The user can visually recognize the visual field image displayed on the terminal device 10 via the right-eye lens 3R and the left-eye lens 3L by wearing the HMD system 1 on the head.

[Hardware Configuration of Terminal Device 10]
The terminal device 10 is a portable computer device that can be used as at least a part of the HMD system 1, and is a mobile phone such as a smartphone or a feature phone, a personal digital assistant (PDA), a tablet PC, or a home game. Machine, etc. can be applied. In the present embodiment, the terminal device 10 will be described as a smartphone.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the terminal device 10. The terminal device 10 includes, for example, a display unit 12, a sensor 13, a timer 14, a storage unit 15, a communication unit 16, and a CPU (Central Processing Unit) 17.

  The display unit 12 is a display that displays information such as images and texts, and includes, for example, a liquid crystal display panel, an organic EL (ElectroLuminescence) display panel, and the like. For example, the display unit 12 displays a stereoscopic image (a right-eye image and a left-eye image) using binocular parallax as a view image from a virtual viewpoint in the virtual space. In addition, the display unit 12 displays various objects arranged in the virtual space together with the view field image. The various objects include, for example, a user interface such as a selection object (option) having a determination area for determining the presence / absence of selection by the user, and an object for enabling the visual line position corresponding to the visual line direction from the virtual viewpoint to be visible. This is an object to be displayed for UI).

  The sensor 13 is a sensor that detects information related to the direction of the terminal device 10. For example, the sensor 13 is a gyro sensor that detects an angle, an angular velocity, an angular acceleration, and the like of an object. The sensor 13 may be a sensor that detects a change in direction, or a sensor that detects the direction itself. For example, the sensor 13 is not limited to a gyro sensor, and may be an acceleration sensor, a tilt sensor, a geomagnetic sensor, or the like.

  The timer 14 has a time measuring function for measuring time.

  The storage unit 15 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and a ROM (Read-Only Memory). Virtual space data (image data), data of objects arranged in the virtual space, game programs using the virtual space, and the like are stored.

  The communication unit 16 communicates with other devices via the network NW.

  The CPU 17 functions as a control center that controls each unit included in the terminal device 10. For example, the CPU 17 functions as a control unit that controls each unit of the terminal device 10 by executing various programs stored in the storage unit 15.

  The above-described components are connected to each other via a bus so that they can communicate with each other. Further, the terminal device 10 may be configured to include a hardware configuration such as a speaker (not shown), an audio output terminal, a camera, a GPS (Global Positioning System) receiving module, an operation button that accepts a user operation input, or a touch panel. .

  For example, the terminal device 10 executes a game program using the HMD. This game is played with the user wearing the HMD system 1 while changing the visual field range of the field-of-view image (game screen) displayed on the display unit 12 by changing the direction of the head or the like. The terminal device 10 detects information related to the direction of the terminal device 10 based on the detection result of the sensor 13, and causes the display unit 12 to display a visual field image (game screen) in the visual field direction based on the detection result. In addition, when receiving an input of a selection operation by the user, the terminal device 10 arranges a selection object (option) having a determination region for determining whether or not there is a selection from the user in the virtual space, and displays it on the display unit 12 together with the view image. Display.

[Definition in virtual space]
FIG. 3 is a diagram showing the definition of the direction of the virtual space according to the present embodiment. In the present embodiment, the vertical direction in which the user stands upright is the Z axis, the direction orthogonal to the Z axis and connecting the user and the display unit 12 is the X axis, and is orthogonal to the Z axis and the X axis. The axis is the Y axis.

  Here, a change in the rotation direction around the Z axis is also referred to as a change in the yaw direction (left and right direction), and a change in the rotation direction around the Y axis is also referred to as a change in the pitch direction (vertical direction). A change in the rotation direction about the X axis is also referred to as a change in the roll direction. For example, the sensor 13 described above detects an angular velocity or an angular acceleration in the rotation direction (yaw direction, pitch direction, and roll direction) of each axis. Note that a change in the yaw direction may be referred to as a change in the left-right direction, and a change in the pitch direction may be referred to as a change in the vertical direction.

  FIG. 4 is an explanatory diagram of the viewing direction and the line-of-sight position according to the present embodiment. In this figure, when the virtual viewpoint K (user's virtual viewpoint) in the virtual space is the intersection (origin) of the X-axis, Y-axis, and Z-axis, and the user's line-of-sight direction is the X-axis direction, the view from the virtual viewpoint The range of the visual field image in the direction (that is, the visual field) is the yaw angle α (the internal angle between the broken line a and the broken line b and the internal angle between the broken line c and the broken line d) and the pitch angle around the visual line direction (X-axis direction). This is a range determined by β (inner angle between the broken line a and the broken line d and inner angle between the broken line b and the broken line c). Here, the yaw angle α and the pitch angle β are angles set in advance as the angle of view of the visual field image of the virtual space displayed on the HMD system 1. When the user's head changes in the pitch direction or the yaw direction, the line-of-sight direction changes from the X-axis direction to the pitch direction or the yaw direction according to the change, and the visual field direction also changes in the pitch direction or the yaw direction. When the user's head changes in the roll direction, the visual line direction rotates in the roll direction while the line-of-sight direction remains in the X-axis direction.

  In the virtual space, various objects are arranged as necessary. For example, various objects are arranged on the same plane (same layer) orthogonal to the line-of-sight direction in the virtual space. For example, when the surface on which the object is arranged in the virtual space is a surface L1 as illustrated, the line-of-sight position P1 that is an object for making the line-of-sight position visible is a broken line indicating the range of the visual field direction on the surface L1. It is arranged at the center (ie, the intersection of the plane L1 and the X axis) of the range (field of view) surrounded by a quadrangle having the intersections a1, b1, c1, d1 as the vertices with a, b, c, d. Note that the position at which the line-of-sight position P1 is arranged is not limited to the center of the range (field of view) surrounded by a rectangle having the intersections a1, b1, c1, and d1 as vertices on the plane L1, but other positions (for example, from the center It may be arranged at a slightly lower position or the like. In addition, a selection object (option) having a determination area for determining the presence / absence of selection from the user is arranged on the surface L1 in the virtual space. The determination area may be a surface (two-dimensional shape) or a solid (three-dimensional shape). For example, a selection object (option) arranged in a range (view) surrounded by a rectangle having vertexes of intersections a1, b1, c1, and d1 of the surface L1 and broken lines a, b, c, and d, and a line-of-sight position P1 Is displayed on the display unit 12 together with the view field image.

  When the user's head changes in the pitch direction or the yaw direction, the visual field direction and the visual line direction change in the pitch direction or the yaw direction according to the change, so the visual line position P1 is a position corresponding to the visual line direction (of the visual field image). Move so that it appears in the center. Further, when the user's head changes in the roll direction, the line-of-sight position P1 rotates according to the change in the roll direction with the position being arranged. On the other hand, the selected object does not move from the position where the selected object is arranged even if the visual field direction changes in the pitch direction, the yaw direction, or the roll direction. That is, when the view direction changes, the selected object arranged in the changed view is displayed.

  When displaying stereoscopic images using binocular parallax, there are field-of-view directions and line-of-sight directions corresponding to virtual viewpoints for the right eye and left-eye, respectively, and field-of-view images and various objects in the respective field-of-view directions and line-of-sight directions. The right-eye image and the left-eye image are displayed. In the present embodiment, the right-eye image and the left-eye image will be described without distinction in order to facilitate the description.

[Display example and operation example of selected object]
Next, a display example and an operation example of the selected object will be described. Hereinafter, a screen displayed on the display unit 12 so that a plurality of selected objects can be selected is also referred to as a “selection menu screen”. The selection menu screen is a screen on which a selection object arranged in the virtual space and a line-of-sight position P1 corresponding to the line-of-sight direction are displayed using a field-of-view image in the virtual space as a background image.

  FIG. 5 is a diagram illustrating an example of the selection menu screen. In the illustrated selection menu screen G10, a line-of-sight position P1, which is an object arranged at a line-of-sight position corresponding to the line-of-sight direction in the virtual space, is displayed at the center of the screen. In addition, two columns in which each of the four selected objects arranged in the virtual space are arranged in the Z-axis direction are displayed side by side with a predetermined interval. In the left column, four selection objects of option A, option C, option E, and option G are displayed side by side with a predetermined interval in the Z-axis direction. In the right column, four selection objects of option B, option D, option F, and option H are displayed side by side with a predetermined interval in the Z-axis direction. Here, it is assumed that the display area of the selected object is a determination area for determining the presence / absence of selection. Note that the display area of the selected object and the determination area for determining the presence / absence of selection may be different areas. Since the line-of-sight position P1 is in a positional relationship that does not overlap with the determination area of any selected object in the line-of-sight direction, no selected object (option) is selected.

  If the orientation of the user's head changes while the selection menu screen G10 shown in FIG. 5 is displayed, the relative position between the selected object and the line-of-sight position P1 changes. The line-of-sight position P1 follows and moves so as to maintain the line-of-sight position in accordance with the change in the orientation of the head, so that it is fixed and displayed at the center of the screen. On the other hand, since the view direction changes according to the change in the orientation of the head, the selected object moves on the screen according to the change in the view direction.

  FIG. 6 is a diagram illustrating an example of a selection menu screen in which the line-of-sight position P1 overlaps the selected object. In the illustrated selection menu screen G11, the line-of-sight position P1 and the option C have a positional relationship in which they overlap in the line-of-sight direction. When the line-of-sight position P1 overlaps the determination area of the selected object, the determination area of the selected object (here, option C) changes.

  FIG. 7 is a diagram illustrating an example of a selection menu screen in which the determination area is changed. In the illustrated selection menu screen G12, the determination area of the selected object (here, option C) where the line-of-sight position P1 overlaps the determination area is enlarged. This makes it easy to maintain the line-of-sight position P1 within the determination area of the selected object to be selected, and to easily prevent an erroneous operation that erroneously moves the line-of-sight position P1 outside the determination area.

  The determination area of the selected object may be enlarged by a method of enlarging the determination area arranged in the virtual space, or selected by moving the position of the determination area as it is to the user (virtual viewpoint K). You may make it display relatively enlarged rather than an object. In addition, here, the display area is enlarged as well as the determination area for option C, but only the determination area for option C is enlarged, and the display area may not be enlarged.

  In addition, the display color of the selected object (here, option C) in which the line-of-sight position P1 overlaps the determination region may be changed. For example, the display color of the selected object whose line-of-sight position P1 does not overlap the determination area is blue, and the display color of the selection object (here, option C) where the line-of-sight position P1 overlaps the determination area is changed from blue to red. May be. This makes it easier to visually recognize that the item is selected. As described above, when the determination area of the selected object is enlarged, the display mode of the selected object may be changed.

  In addition, an elapsed time that is an object for making it possible to visually recognize an elapsed time since the line-of-sight position P1 overlaps the determination area in the vicinity of the selected object (here, option C) where the line-of-sight position P1 overlaps the determination area. A gauge T1 is arranged and displayed. For example, the elapsed time gauge T1 is a circular gauge, and the bars increase in a circular shape with the passage of time after the line-of-sight position P1 overlaps the determination region, and increase in a circular shape when a predetermined time or more elapses. The bar extends for one lap. Here, the predetermined time is a determination time set in advance as a threshold for determining whether or not the selected object is selected, and the elapsed time after the line-of-sight position P1 overlaps the determination region is equal to or longer than the predetermined time. In such a case, it is determined that the selected object in the determination area is selected. On the other hand, if the elapsed time since the line-of-sight position P1 overlaps the determination area is less than the predetermined time, it is determined that the selected object in the determination area has not yet been selected. For example, when the line-of-sight position P1 deviates from the determination area before the circularly increasing bar reaches one round, it is determined that the selected object in the determination area is not selected, and the measurement time (elapsed time) Is reset.

On the other hand, when the bar of the elapsed time gauge T1 increases in a circle, the selection of the selected object (here, the option C) where the line-of-sight position P1 overlaps the determination region is confirmed.
FIG. 8 is a diagram illustrating an example of a selection menu screen when selection of a selected object is confirmed. In the selection menu screen G13 shown in the figure, a selection object (here, option C) in which the circularly increasing bar of the elapsed time gauge T1 extends for one round and the line-of-sight position P1 overlaps the determination region is selected. Confirmed. At this time, other selected objects arranged in the virtual space are erased or displayed inconspicuously by increasing the transmittance. As a result, the selected selected object is displayed in a relatively emphasized manner.

  Note that the above-described enlargement of the determination area of the selected object is an enlargement in a range that does not overlap with the determination areas of other selection objects in the line-of-sight direction. Further, the enlargement of the determination area of the selected object may be an enlargement within a predetermined interval in the direction orthogonal to the line-of-sight direction from the determination area of another selected object. Therefore, when the number of selected objects increases, it becomes difficult to enlarge the determination area of the selected object. Therefore, when expanding the determination area of the selected object, the terminal device 10 may change the position of the other selected object.

  FIG. 9 is a diagram illustrating an example of a selection menu screen when the number of selected objects is large. In the selection menu screen G14 shown in the figure, two columns in which each of five selected objects arranged in the virtual space are arranged in the Z-axis direction are displayed side by side with a predetermined interval. In the left column, five selection objects of option A, option C, option E, option G, and option I are displayed side by side with a predetermined interval in the Z-axis direction. In the right column, five selection objects of option B, option D, option F, option H, and option J are displayed side by side with a predetermined interval in the Z-axis direction. In addition, the line-of-sight position P1 arranged in the virtual space is displayed at the center of the screen. The line-of-sight position P1 has a positional relationship that does not overlap with the determination area of any selected object in the line-of-sight direction.

  FIG. 10 is a diagram illustrating an example of the selection menu screen in which the line-of-sight position P1 overlaps the selected object on the selection menu screen illustrated in FIG. In the illustrated selection menu screen G15, the line-of-sight position P1 and the option E overlap with each other in the line-of-sight direction. When the line-of-sight position P1 overlaps the determination area of the selected object, the determination area of the selected object (here, option E) is enlarged, but the arrangement interval between the selection objects adjacent in the Z-axis direction is narrow. Therefore, when the determination area of the selected object (here, option E) is enlarged, the selection object above the option E (here, options A and C) and the lower side are secured so as to secure the area to be enlarged. Each of the selected objects (here, options G and I) moves in a direction away from option E.

  Other information may be displayed in the enlarged display area of the selected object. For example, when the selection object is a selection object of a purchase item, the item name is normally displayed on the selection object, but when the selection object is selected, a purchase price or the like may be displayed in addition to the item name.

[Functional Configuration of Terminal Device 10]
Next, the functional configuration of the terminal device 10 will be described with reference to FIG. FIG. 11 is a block diagram illustrating an example of a functional configuration of the terminal device 10 according to the present embodiment. The terminal device 10 includes a control unit 110 as a functional configuration realized by the CPU 17 executing a program stored in the storage unit 15. The control unit 110 includes a detection unit 111, a determination unit 112, an object placement unit 113, and a display control unit 114.

  The detection unit 111 detects information related to the direction of the terminal device 10 based on the detection result of the sensor 13. For example, the detection unit 111 detects the direction of view of the user wearing the HMD system 1 to which the terminal device 10 is attached or the change in the view direction based on the direction detected by the sensor 13 or information on the direction change. Detect by the direction in which 10 is facing. Note that the detection unit 111 may detect information related to the change speed of the visual field direction.

  The determination unit 112 determines whether or not the selected object has been selected based on the positional relationship between the user's line-of-sight position P1 and the determination area of the selected object. For example, the determination unit 112 measures the elapsed time after the position relationship where the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction overlaps, and selects based on the measured elapsed time. Determine presence or absence. For example, the determination unit 112 determines that the selected object has been determined (selected) when a predetermined time or more has elapsed since the line-of-sight position P1 and the determination region of the selected object have overlapped. On the other hand, when the elapsed time after the line-of-sight position P1 and the determination area of the selected object overlap is less than a predetermined time, the determination unit 112 determines that the selected object has not yet been selected (not selected). judge.

  The object placement unit 113 places a selected object having a determination area for determining whether or not there is a selection from the user in the virtual space. Further, the object placement unit 113 determines the line-of-sight direction from the virtual viewpoint based on the detection result of the detection unit 111, and an object (for example, the line-of-sight position P1) for enabling visual recognition of the line-of-sight position corresponding to the line-of-sight direction. Place in the virtual space. In addition, the object arrangement unit 113 arranges an object (for example, an elapsed time gauge T1) for enabling the elapsed time to be visually recognized in the virtual space. For example, the object placement unit 113 may place the above-described various objects on the same surface (same layer) in the virtual space, or may be placed on a plurality of surfaces (a plurality of layers). The object placement unit 113 acquires time information such as elapsed time by using the timer 14.

  The display control unit 114 causes the display unit 12 to display a visual field image in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. Further, the display control unit 114 displays various objects (selected objects, line-of-sight positions, elapsed time gauges, etc.) arranged in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. 12 is displayed. Note that the data of the view image in each direction in the virtual space is stored in the storage unit 15 as virtual space data. The display control unit 114 acquires a view image corresponding to the view direction from the virtual space data and causes the display unit 12 to display the view image as a view image.

[Details of object placement processing]
Next, the object arrangement processing according to the present embodiment will be described in detail.
The object placement unit 113 changes the determination area of the selected object based on the positional relationship between the line-of-sight position P1 corresponding to the line-of-sight direction from the virtual viewpoint in the virtual space and the determination area of the selected object. For example, the object placement unit 113 widens the determination area of the selected object when the line-of-sight position P1 and the determination area of the selection object overlap with each other from the position relation that does not overlap in the line-of-sight direction. Here, when the determination area of the selected object is widened, the object placement unit 113 widens the determination area so as not to overlap the determination area of another selected object in the line-of-sight direction.

  Note that, when the selection area of the selected object is widened, the object placement unit 113 widens the selection object in a range where a predetermined interval is provided in a direction orthogonal to the determination area of the other selection objects. Further, the object placement unit 113 may change the position of the other selected object when the selection object determination area is widened. Moreover, the object arrangement | positioning part 113 may change the display mode of a selection object, when enlarging the determination area | region of a selection object. In addition, the object placement unit 113 may select the determination area of the selected object or the determination area after a predetermined time has elapsed since the position relationship where the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction has changed. The display area corresponding to may be widened.

  Next, with reference to FIG. 12, the operation of the object placement processing according to the present embodiment will be described. FIG. 12 is a flowchart illustrating an example of the object placement process according to the present embodiment. It is assumed that a visual field image in the visual field direction and a visual line position P1 are displayed on the display unit 12. First, upon receiving a selection menu display trigger, control unit 110 places a plurality of selected objects in the virtual space (step S100). Accordingly, a selection menu screen (for example, see FIG. 5) including the view field image and the plurality of selection objects is displayed on the display unit 12. Here, the trigger of the selection menu display is that the game is started, the game is started, a predetermined time has elapsed, a specific action is performed in the game, etc. It is set in situations where confirmation is required.

  The control unit 110 determines whether or not the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction (step S102). When it is determined that the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction (for example, see FIG. 6) (YES), the control unit 110 sets the determination area of the selected object to be enlarged (for example, , See FIG. 7), that is, the determination region is widened (step S104). At this time, the control unit 110 arranges the elapsed time gauge T1 in the vicinity of the selected object (for example, the option C) where the line-of-sight position P1 overlaps the determination region.

  On the other hand, when it is determined that the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction (for example, see FIG. 5) (NO), the control unit 110 sets the determination area of the selected object to an initial value. That is, the determination area is not changed (step S106).

  Next, the control unit 110 determines whether or not a selection menu display-off trigger has been received (step S108). Here, the trigger for turning off the display of the selection menu means that when any selected object is selected, an option for turning off the display of the selection menu is selected when a predetermined time has elapsed since the selected object was displayed. If, etc. If it is determined that the trigger for turning off the display of the selection menu has been received (YES), the control unit 110 deletes the selected object arranged in the virtual space (step S110) and ends the object arrangement processing.

  On the other hand, when it is determined that the trigger for turning off the display of the selection menu is not received (NO), the control unit 110 detects the angular velocity based on the detection result of the sensor 13 (step S112). And the control part 110 determines whether there exists any change of a visual field direction based on the detected angular velocity (step S114). If it is determined that there is no change in the viewing direction (NO), since the relative position between the line-of-sight position P1 and the determination area of the selected object does not change, the control unit 110 returns to the process of step S108 and displays the selection menu. It is determined whether an off trigger is accepted. On the other hand, when it is determined that there is a change in the viewing direction (YES), a change occurs in the relative position between the line-of-sight position P1 and the determination area of the selected object. Therefore, the control unit 110 returns to the process of step S102, and the line-of-sight position It is determined whether P1 and the determination area of the selected object overlap in the line-of-sight direction. If it continues to overlap, leave the selected object's judgment area set to enlargement, and if it overlaps from the non-overlapping state, change the selection object's judgment area from the initial value to enlargement. When the overlapping state does not overlap, the determination area of the selected object is changed from enlarged to the initial value.

[Operation of selection judgment processing]
Next, with reference to FIG. 13, the operation of the determination process for determining whether or not the selected object arranged in the object arrangement process according to this embodiment is selected will be described. FIG. 13 is a flowchart illustrating an example of a determination process for determining whether or not a selected object is selected according to the present embodiment. First, upon receiving a selection menu display trigger, the control unit 110 displays a selection menu screen (for example, see FIG. 5) on the display unit 12 by arranging a plurality of selection objects in the virtual space (step 5). S200). Moreover, the control part 110 resets measurement time (step S202).

  Next, the control unit 110 determines whether or not the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction (step S204). When it is determined that the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction (for example, see FIG. 5) (NO), the control unit 110 returns the process to step S202. On the other hand, when it is determined that the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction (for example, see FIG. 6) (YES), the control unit 110 starts measuring elapsed time and measures time The elapsed time gauge T1 on which a bar corresponding to the elapsed time is displayed is placed in the vicinity of the selected object determined to overlap in the virtual space (step S206). Thereby, the elapsed time gauge T1 is displayed in the vicinity of the selected object (see, for example, FIG. 7).

  Next, the control unit 110 determines whether or not a predetermined time has elapsed since the line-of-sight position P1 overlaps with the determination area of the selected object (step S208). When it is determined that the predetermined time or more has not elapsed (NO), the control unit 110 returns to the process of step S204 and continues to measure the elapsed time after overlapping. On the other hand, when it is determined that a predetermined time or more has elapsed since the line-of-sight position P1 and the determination area of the selected object overlap (YES), the control unit 110 determines that the selected object has been selected (step S210), and the determination The process ends.

[Another example of selection processing]
In the determination process for determining whether or not the selected object is selected, the example in which the presence or absence of the selection is determined based on the elapsed time after the line-of-sight position P1 overlaps the determination area of the selected object has been described. Another example of the timing specification will be described.
FIG. 14 is a diagram illustrating an example of a selection menu screen in which the line-of-sight position P1 and the selection object determination area overlap. In the illustrated selection menu screen G16, the line-of-sight position P1 and the selected object (here, option B) overlap, and an elapsed time gauge T1 indicating the elapsed time since the overlap is displayed. This elapsed time is in a state where the predetermined time for determining that the time has been selected has not been reached. Here, when the user turns to the left, the selected object moves to the right relative to the line-of-sight position P1.

  FIG. 15 is a diagram illustrating an example of a selection menu screen in which the line-of-sight position P1 and the determination area of the selected object do not overlap with each other because the selected object has moved to the right relative to the line-of-sight position P1. . The selection menu screen G17 shown in the drawing shows that the line-of-sight position P1 and the determination area of the selected object (here, option B) that overlap in FIG. When the line-of-sight position P1 does not overlap the selected object, the measurement time is reset and the elapsed time becomes 0 in the determination process illustrated in FIG. 13, but the elapsed time may be stopped without being reset. For example, in the example shown in FIG. 15, the elapsed time is stopped, and the elapsed time gauge T1 bar indicates the elapsed time up to that point (when the line-of-sight position P1 overlaps the determination area of the selected object). It shows that it has stopped. In addition, when the line-of-sight position P1 and the selected object determination area change from the overlapping state to the non-overlapping state, the advance of the time may be delayed or the time may be set backward compared to the elapsed time measurement in the overlapping state. You may let them. In this manner, the determination unit 112 may perform different timing depending on whether the line-of-sight position P1 and the determination area of the selected object are in a positional relationship where they do not overlap in the line-of-sight direction.

  In this case, even if the line-of-sight position P1 deviates from the determination area of the selected object, if the line-of-sight position P1 overlaps by moving the line of sight again within the determination area of the same selected object, the elapsed time is restarted from the next time. Even if the line of sight is mistakenly deviated from the determination area for a moment, it is not necessary to re-measure the elapsed time from zero.

  When a predetermined time (for example, 10 seconds) elapses when the line-of-sight position P1 is out of the selected object determination area, the measurement time may be reset assuming that the user does not intend to continue the selection. . For example, the determination unit 112 determines the elapsed time measured after the positional relationship where the line-of-sight position P1 overlaps the determination area of the selected object as the positional relationship where the line-of-sight position P1 does not overlap the determination area of the selected object. When the time elapses, the measurement time (elapsed time) may be reset.

  When the line-of-sight position P1 moves to the determination area of another selected object (option A, C, etc.), the measured time (elapsed time) is reset, and the elapsed time is started from 0. . For example, the determination unit 112 has a positional relationship in which the visual line position P1 overlaps the determination region of another selected object with respect to the elapsed time measured after the positional relationship of the visual line position P1 overlaps with the determination region of the selected object. If this is the case, the measurement time (elapsed time) is reset.

  Next, with reference to FIG. 16, the operation of a determination process for determining whether or not a selected object is selected based on the above-described elapsed time measurement specification will be described. FIG. 16 is a flowchart illustrating an example of a determination process for determining whether or not a selected object is selected according to the present embodiment. Steps S300, S302, S304. S306. Each process of S308 and S310 is performed in steps S200, S202, S204. S206. Corresponding to each process of S208 and S210, only the subsequent process when it is determined NO in the process of step S308 is different from the determination process shown in FIG. Here, processing different from the determination processing illustrated in FIG. 13 will be described, and description of similar processing will be omitted.

  If it is determined in step S308 that the predetermined time or more has not elapsed since the line-of-sight position P1 and the selection object determination area overlap (NO), the control unit 110 determines that the line-of-sight position P1 and the selection object determination area are It is determined whether or not they overlap in the line-of-sight direction (step S312). If it is determined that the line-of-sight position P1 and the determination area of the selected object do not overlap (NO), the control unit 130 temporarily stops counting and displaying the elapsed time (the bar of the elapsed time gauge T1) (step S314). . At this time, instead of temporarily measuring and displaying the elapsed time, it may be timed and displayed so that the passage of time progresses slowly, or timed and displayed so that the passage of time goes backward. Also good.

  Next, the control unit 110 determines whether or not a predetermined time or more has elapsed since the line-of-sight position P1 and the determination area of the selected object are not overlapped (step S316). When it is determined that the predetermined time or more has elapsed (YES), the control unit 110 considers that the user has no intention to continue the selection, returns the process to step S302, and resets the measurement time.

  On the other hand, when it is determined that the predetermined time or more has not passed since the line-of-sight position P1 and the determination area of the selected object do not overlap (NO), the control unit 110 returns the process to step S312 and returns to the line-of-sight position It is determined whether P1 and the determination area of the selected object overlap in the line-of-sight direction. When it is determined that the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction (YES), the control unit 110 determines that the selection object that overlaps the line-of-sight position P1 is the selection object that previously overlapped. It is determined whether or not there is (step S318). If it is determined that the selected object overlapped last time, the control unit 110 returns the process to step S308 and restarts counting and displaying the elapsed time. In step S314, when the timekeeping and display are performed so that the passage of time progresses slowly, or when the timekeeping and display are performed such that the passage of time is retreated, the control unit 110 returns to the original timekeeping and display. Return to. On the other hand, when it is determined that the selected object on which the line-of-sight position P1 overlaps is another selected object (NO), the control unit 110 returns the process to step S302 and resets the measurement time.

[Summary of First Embodiment]
As described above, the terminal device 10 according to the present embodiment is at least the HMD system 1 capable of displaying a visual field image representing a visual field from a virtual viewpoint in a virtual space as a stereoscopic image using binocular parallax. It is a terminal device that can be used as a part. For example, the terminal device 10 includes a detection unit 111, an object placement unit 113, and a display control unit 114. The detection unit 111 detects information related to the direction of the terminal device 10. The object placement unit 113 places a selected object having a determination area for determining whether or not there is a selection from the user in the virtual space. The display control unit 114 causes the display unit 12 to display the selected object arranged in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. Then, the object placement unit 113 changes the determination area of the selected object based on the positional relationship between the line-of-sight position P1 corresponding to the line-of-sight direction from the virtual viewpoint in the virtual space and the determination area of the selection object.

  Here, the change of the determination region may be a change in the size of the determination region, for example, a change in area (width) or a change in volume, or a change in the position where the determination region is arranged. Good. Further, the change of the determination area may be a change of the shape of the determination area. In addition, the above-described positional relationship may be a two-dimensional planar positional relationship projected in the line-of-sight direction, or a three-dimensional positional relationship when the object is a solid (three-dimensional shape). May be.

  Thereby, the terminal device 10 can change the size (apparent size) of the determination region according to the positional relationship between the line-of-sight position P1 and the determination region of the selected object. The operability at the time of performing can be improved. Note that the object placement unit 113 may make it easier to visually recognize the selected area by changing the color or brightness of the determination area.

  For example, on the selection menu screen, there is a case where it is necessary to reduce the determination area of one option when it is not desired to disturb the display of the view field image as a background or when there are many selection items. In such a case, if the determination area is small, the operability is difficult such that the line-of-sight position is unintentionally outside the determination area due to an erroneous operation within the period for determining whether or not to select. Therefore, for example, the object placement unit 113 widens the determination area of the selected object when the line-of-sight position P1 and the determination area of the selected object overlap with each other from the position relation that does not overlap in the line-of-sight direction. Thereby, since the terminal device 10 expands the determination area when the line-of-sight position P1 enters the determination area of the selected object, it is possible to suppress the occurrence of an erroneous operation that causes the line-of-sight position P1 to be out of the determination area without the user's intention. .

  For example, in the case where the determination area of the selected object is widened, the object placement unit 113 widens the determination area so as not to overlap the determination area of another selected object in the line-of-sight direction. As a result, when the user intentionally changes to selection of another selected object, the terminal device 10 causes an erroneous operation that causes the line-of-sight position P1 to deviate from the currently overlapping determination region so that the selection can be made. Generation can be suppressed. Further, since the terminal device 10 can maintain the independence of the determination areas of the respective selected objects, it is possible to prevent an erroneous operation when it is desired to select any one selected object.

  Further, in the case where the determination area of the selected object is widened, the object arrangement unit 113 may widen the determination area of the selected object so that a predetermined interval is provided in a direction orthogonal to the line-of-sight direction. Accordingly, when the user intentionally changes the selection to another selected object, the terminal device 10 can immediately remove the line-of-sight position P1 from the currently overlapping determination area.

  Further, the object placement unit 113 may change the position of the other selected object when the selection object determination area is widened. Thereby, even when there are a large number of selected objects, the terminal device 10 can secure an area for expanding the determination area by moving other selected objects.

  The object placement unit 113 changes the display mode of the selected object when widening the determination area of the selected object. For example, the object placement unit 113 may display other information on the enlarged selected object. For example, if the selected object is a purchase item option, the item name is displayed on the selection object having a normal size before the line-of-sight position P1 overlaps, and the item name is displayed when the line-of-sight position P1 overlaps and expands. In addition, a purchase price or the like may be displayed. Thereby, since the terminal device 10 can display the information related to the decision making of the selection on the selected object, the user can prevent erroneous selection. Further, the object placement unit 113 may change the color of the selected object being enlarged to a color different from that before enlargement. In this case, the terminal device 10 can improve operability by improving the visibility of the selected object to be selected.

  In addition, the object placement unit 113 may select the determination area of the selected object or the determination area after a predetermined time has elapsed since the position relationship where the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction has changed. The display area corresponding to may be widened. That is, the terminal device 10 does not immediately expand the determination area of the selected object even when the line-of-sight position P1 enters the determination area of the selected object, but does not immediately expand the determination area of the selected object. A time lag of about 3 seconds) may be provided. Thereby, since the terminal device 10 can suppress the determination area from being enlarged when the line-of-sight position P1 passes over the determination area in a short time even though there is no intention to select, the visual recognition when the selection operation is performed. Can be improved. For example, if the selected object is enlarged each time the line-of-sight position P1 passes, it becomes annoying and poor visibility. Note that the time lag is provided only in the display area of the selected object, and the time lag may not be provided in the determination area. Also, a time lag may be provided in the same manner for both the display area and the determination area of the selected object. Also, a time lag may be set depending on the speed of entry of the line-of-sight position P1 into the determination area. For example, a time lag may be provided when the intrusion speed is greater than or equal to a predetermined value, and no time lag may be provided when the intrusion speed is less than a predetermined value.

  In addition, the object placement unit 113 determines the line-of-sight direction from the virtual viewpoint based on the detection result of the detection unit 111, and places an object for enabling visual recognition of the line-of-sight position P1. Thereby, the terminal device 10 can determine, for example, the line-of-sight position P1 corresponding to the line-of-sight direction in the view image (for example, the center of the view image) according to the orientation of the head of the user wearing the HMD system 1. It is possible to improve operability by making it easier for the user to recognize the positional relationship between the line-of-sight position P1 and the selected object.

  In the present embodiment, the example in which the line-of-sight direction is detected based on the direction of the terminal device 10 has been described. However, the present invention is not limited to this. For example, the line-of-sight direction is detected by detecting the direction of the user's eyeball. (Eye tracking) may be used. In this case, since the line-of-sight direction is based on detection different from the head movement, the line-of-sight position P1 moves in the view field image, not in a fixed position in the view field image.

  Further, the terminal device 10 includes a determination unit 112 that determines whether or not a selected object is selected based on the positional relationship between the line-of-sight position P1 and the determination region of the selected object. Thereby, the terminal device 10 can perform an operation UI for selecting an option using the user's line of sight in the HMD system 1.

  For example, the determination unit 112 measures the elapsed time after the position relationship where the line-of-sight position P1 and the determination area of the selected object do not overlap in the line-of-sight direction overlaps, and selects based on the measured elapsed time. Determine presence or absence. As a result, the terminal device 10 determines that the line of sight position P1 has been selected on the condition that a predetermined time has elapsed since the line of sight position P1 overlaps the determination area of the selected object. It is possible to prevent the position P1 from being selected only by passing.

  Moreover, the object arrangement | positioning part 113 arrange | positions the object for enabling visual recognition of elapsed time in virtual space. Thereby, the terminal device 10 can present the timing when the selection is confirmed so that the user can easily recognize.

  In addition, the determination unit 112 measures different times depending on whether the line-of-sight position P1 and the determination area of the selected object are in a positional relationship that does not overlap in the line-of-sight direction. For example, the determination unit 112 performs time counting when the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction, and stops time counting when they do not overlap (does not reset to 0). To maintain the elapsed time). Note that when the positions do not overlap, the time may be measured so that the time progresses slowly, or the time may be set backward, without stopping the time measurement. As a result, even if the line-of-sight position P1 deviates from the determination area due to an erroneous operation, the terminal device 10 does not start counting again from 0 and moves the elapsed time from the next time if the line-of-sight position P1 is moved again to the same determination area. Since the timing is resumed, operability can be improved.

  Further, the determination unit 112 determines the elapsed time measured after the line-of-sight position P1 and the determination area of the selected object overlap in the line-of-sight direction. If the overlapping position relationship is reached, reset. As a result, the terminal device 10 assumes that the user does not intend to continue selection when the line-of-sight position P1 moves to the determination area of another selected object, and returns the elapsed time measured to 0 to the same. Even when the line-of-sight position P1 unintentionally moves again to the determination region, the measurement of the elapsed time is not resumed from the continuation, and an erroneous operation can be prevented.

  In addition, the determination unit 112 does not overlap the line-of-sight position P1 and the determination area of the selected object in the line-of-sight direction with respect to the elapsed time measured since the line-of-sight position P1 and the determination area of the selection object overlap in the line-of-sight direction. It may be reset when a predetermined time elapses due to the positional relationship. Thereby, the terminal device 10 considers that the user does not intend to continue the selection when a predetermined time (for example, 10 seconds) elapses after the line-of-sight position P1 deviates from the determination region, and the elapsed time measured. Therefore, even when the line-of-sight position P1 unintentionally moves again to the same determination area, the measurement of the elapsed time is not resumed and it is possible to prevent an erroneous operation from occurring.

  Further, the determination unit 112 may change the speed of time measurement depending on which position in the determination area of the selected object the line-of-sight position P1 overlaps in the line-of-sight direction. For example, the determination unit 112 may increase the timing speed as it is closer to the center of the determination region. In this case, the closer to the center of the determination area, the faster the bar of the elapsed time gauge T1 extends by one turn, and the shorter the period during which it is determined that the bar has been selected as a result. Thereby, the terminal device 10 can also provide the user with a method of shortening the period until it is determined to be selected while providing a determination time for preventing malfunction. Note that, instead of changing the speed of timing, the determination time for determining (determining) selection may be changed. For example, the determination unit 112 may shorten the determination time as it is closer to the center of the determination region.

  In addition, an area where the speed of time measurement (speed at which the bar of the elapsed time gauge T1 extends) is increased after the line-of-sight position P1 enters the determination area may be provided. The area may be an area that does not require elapsed time (that is, an area for which selection is immediately determined). In this case, the area that does not require the elapsed time is set to a position that does not include the line-of-sight position P1 in the determination area so that the line-of-sight position P1 does not enter the user unintentionally.

  In addition, the display control unit 114 causes the display unit 12 to display a visual field image in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. Thereby, the terminal device 10 can provide a visual field image in the visual field direction in the virtual space so that the user can visually recognize it according to the movement of the head of the user wearing the HDM system 1. Further, since the terminal device 10 can simultaneously display the view field image and the selected object in the virtual space, the user can select the game device while referring to the game information in the virtual space.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 11, characteristic processing in the present embodiment will be described. In the present embodiment, a process for restricting a change in the relative position of the selected object with respect to the line-of-sight position in a predetermined direction when changing the view direction on the selection menu screen and performing a selection operation will be described.

  FIG. 17 is a diagram illustrating an example of a selection menu screen according to the present embodiment. In the illustrated selection menu screen G20, the line-of-sight position P1 is displayed at the center of the screen, and four selection objects (option A, option B, option C, and the like arranged in the Z-axis direction in the virtual space are arranged. And option D) are displayed. For example, if the user changes the viewing direction (gaze direction) to the pitch direction (vertical direction) that is the arrangement direction (Z-axis direction) of the selected objects in order to select option A, the viewing direction (gaze direction) changes. Accordingly, the visual field image in the virtual space changes in the pitch direction (vertical direction). Further, the four selected objects arranged in the virtual space change in the pitch direction (up and down direction) relative to the line-of-sight position P1 when the view field image changes in the pitch direction (up and down direction).

  FIG. 18 is a diagram illustrating an example of a selection menu screen when the user's visual field direction (gaze direction) changes in the pitch direction (upward). In the illustrated selection menu screen G21, when the user changes the viewing direction (gaze direction) upward, the selected object changes downward relative to the gaze position P1, and the gaze position P1 is the option A. It shows a state where it overlaps the judgment area.

On the other hand, with respect to a change in the user's field of view direction (line-of-sight direction) other than the direction in which the selected objects are arranged (here, other than the pitch direction), the selected object is moved in accordance with the change, thereby You may restrict | limit the change of a relative position.
FIG. 19 is a diagram showing an example of the selection menu screen when the user's viewing direction (gaze direction) changes to the yaw direction (right direction) on the selection menu screen shown in FIG. In the selection menu screen G22 shown in the drawing, the visual field image changes to the right as the user changes the visual field direction (gaze direction) to the right, but the change in the relative position of the selected object with respect to the visual line position P1 is small. A broken line M1 indicates the position of the selected object before moving (the position at the center of the determination area). On the other hand, a broken line M2 indicates the position of the selected object after movement (the position at the center of the determination area). That is, the object placement unit 113 moves the selected object from the position centered on the broken line M1 to the position centered on the broken line M2 when the user's view direction (line of sight direction) changes to the yaw direction (right direction). .

  The object placement unit 113 also rotates the selected object in response to the change because the change in the roll direction of the user's view direction (gaze direction) is also a change in a direction other than the arrangement direction of the selected objects. Let FIG. 20 is a diagram illustrating an example of a selection menu screen when the user's viewing direction (gaze direction) changes to the roll direction. The selection menu screen G23 shown in the drawing shows the arrangement of the selected object when the user's visual field direction (gaze direction) changes by the angle θ in the roll direction (here, counterclockwise about the X axis). A broken line M1 indicates the position of the selected object (the center position of the determination area) before the rotational movement. On the other hand, a broken line M2 indicates the position of the selected object after rotation (position at the center of the determination area). That is, the object placement unit 113 changes the rotation direction between the line-of-sight position P1 and the selected object when the user's line-of-sight direction (line-of-sight direction) changes by an angle θ in the roll direction (here, counterclockwise about the X axis). The selected object is moved in the roll direction (here, counterclockwise about the X axis) by rotating the angle θ so that the relative position does not change.

  It should be noted that the angle at which the user's visual field direction (line-of-sight direction) has changed may or may not coincide with the angle by which the selected object is rotated. For example, the angle by which the selected object is rotated may be an angle obtained by multiplying the angle at which the user's visual field direction (gaze direction) is changed by a predetermined ratio, or an angle obtained by adding or subtracting a predetermined angle. In addition, an upper limit may be provided for the angle at which the selected object is rotated.

  As described above, when the object placement unit 113 determines that the view direction (gaze direction) from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit 111, the object placement unit 113 changes the view direction (gaze direction). Based on the change, the selected object is moved while restricting the movement of the selected object in the arrangement direction. For example, the object placement unit 113 moves the selected object according to the change in the visual field direction (gaze direction) so that the relative position between the gaze position P1 and the selected object does not change in directions other than the arrangement direction of the selected objects. .

  Note that the object placement unit 113 may move the selected object according to a change in the viewing direction (gaze direction) so that the gaze position P1 falls within a predetermined range in directions other than the arrangement direction of the selected objects. Here, the predetermined range is, for example, a width (for example, a width indicated by W in FIG. 19) of the determination area of the selected object in a direction orthogonal to the arrangement direction of the selected objects. Further, the predetermined range may be, for example, a range of angles in the rotation direction with the line-of-sight direction as an axis.

  Next, the operation of the object placement process according to the present embodiment will be described with reference to FIG. FIG. 21 is a flowchart illustrating an example of the object placement process according to the present embodiment. First, upon receiving a selection menu display trigger, the control unit 110 arranges a plurality of selected objects in a predetermined direction (for example, the Z-axis direction) in the virtual space (step S400). Accordingly, a selection menu screen (for example, see FIG. 17) including the view field image and the plurality of selection objects is displayed on the display unit 12.

  The controller 110 detects the angular velocity based on the detection result of the sensor 13 (step S402). And the control part 110 determines whether there exists a change of a visual field direction (gaze direction) based on the detected angular velocity (step S404).

  When it is determined that there is a change in the viewing direction (gaze direction) (YES), the control unit 110 restricts the movement in a predetermined direction (the arrangement direction of the selected objects, for example, the Z-axis direction), and the viewing direction. The selected object is moved based on the change in the (sight line direction) (see FIGS. 18 and 19), and the process proceeds to step S408 (step S406). On the other hand, when it determines with there being no change of a visual field direction (gaze direction) (NO), the control part 110 progresses to the process of step S408.

  Next, the control unit 110 determines whether or not a selection menu display-off trigger has been received (step S408). When it is determined that the trigger for turning off the display of the selection menu is not received (NO), the control unit 110 returns the process to step S402. On the other hand, when it is determined that the trigger for turning off the display of the selection menu has been received (YES), the control unit 110 deletes the selected object arranged in the virtual space (step S410) and ends the object arrangement processing. .

[Summary of Second Embodiment]
As described above, the terminal device 10 according to the present embodiment is capable of displaying at least one of the HMD systems 1 capable of displaying a view image representing a view from a virtual viewpoint in a virtual space as a stereoscopic image using binocular parallax. It is a terminal device that can be used as a part. For example, the terminal device 10 includes a detection unit 111, an object placement unit 113, and a display control unit 114. The detection unit 111 detects information related to the direction of the terminal device 10. The object arrangement unit 113 arranges a plurality of selection objects having a determination area for determining whether or not there is a selection from the user in a predetermined direction in the virtual space. The display control unit 114 causes the display unit 12 to display the selected object arranged in the visual field direction from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111. Then, when it is determined that the view direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit 111, the object placement unit 113 moves in a predetermined direction based on the change in the view direction. The selected object is moved while restricting.

  As a result, the terminal device 10 can restrict the relative position between the line-of-sight position P1 and the determination area of the selected object so as not to change in a direction unnecessary for selection. Operability can be improved.

  Further, the object placement unit 113 determines the line-of-sight direction from the virtual viewpoint based on the detection result of the detection unit 111. Moreover, the object arrangement | positioning part 113 arrange | positions the object (for example, gaze position P1) for enabling visual recognition of the gaze position corresponding to the gaze direction from the virtual viewpoint in virtual space. Thereby, the terminal device 10 can determine, for example, the line-of-sight position P1 corresponding to the line-of-sight direction in the view image (for example, the center of the view image) according to the orientation of the head of the user wearing the HMD system 1. It is possible to improve operability by making it easier for the user to recognize the positional relationship between the line-of-sight position P1 and the selected object.

  For example, the object placement unit 113 places an object (for example, the line-of-sight position P1) for making the line-of-sight position visible and the selected object on the same plane in the virtual space. The same plane is, for example, a plane orthogonal to the viewing direction in the virtual space. The same surface is not limited to exactly the same surface, and may be substantially the same surface. Further, the same surface is not limited to a surface orthogonal to the line-of-sight direction, and may be a surface in any direction in the virtual space, and may be a flat surface or a curved surface. Thereby, the terminal device 10 can realize an option selection operation UI based on the relative position (positional relationship) between the line-of-sight position P1 and the selected object. Specifically, the terminal device 10 arranges the line-of-sight position P1 and the selected object on the same plane in the virtual space, so that the line-of-sight position P1 and the selected object have the same focal length in stereoscopic vision using binocular parallax. Therefore, the positional relationship between the two can be easily recognized.

  When it is determined that the view direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit 111, the object placement unit 113 selects the line-of-sight position P1 for directions other than the arrangement direction of the selected objects. The selected object is moved according to the change in the viewing direction so that the line-of-sight position falls within the width in the direction orthogonal to the arrangement direction of the object determination regions. Thereby, the terminal device 10 prevents the line-of-sight position P1 from deviating from the range of the width of the determination area of the selected object in directions other than the direction necessary for selecting the selected object (that is, the direction unnecessary for the operation). In order to limit, it is possible to facilitate the selection operation for the options. In addition, with such a restriction, even if the selected object is made smaller (decreasing the width), there is little influence on the operation. Therefore, displaying the selected object in a small size prevents the display of the field-of-view image as the background of the selected object. Can also be reduced.

  Further, the object placement unit 113 may change the line-of-sight position P1 beyond the width of the selected object in order to leave a cancel area for canceling the selection (the line-of-sight position P1 is removed from the determination area). .

  When it is determined that the view direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit 111, the object placement unit 113 determines the line-of-sight position P1 and the selected object for directions other than the arrangement direction of the selected objects. The selected object is moved in accordance with the change in the viewing direction so that the relative position with respect to does not change. As a result, the terminal device 10 makes the relative relationship between the line-of-sight position P1 and the determination area of the selected object other than the direction in which the line-of-sight position P1 necessary for selecting the selected object is moved (that is, the direction unnecessary for the operation). Since the position is restricted so as not to change, it is possible to facilitate the selection operation for the option. In addition, with such a restriction, even if the selected object is made smaller (decreasing the width), there is little influence on the operation. Therefore, displaying the selected object in a small size prevents the display of the field-of-view image as the background of the selected object. Can also be reduced.

  In addition, the terminal device 10 includes a determination unit 112 that determines whether or not a selected object is selected based on a relative position (positional relationship) between the line-of-sight position P1 and the determination region of the selected object. Thereby, the terminal device 10 can perform an operation UI for selecting an option using the user's line of sight in the HMD system 1.

  If the object placement unit 113 determines that the field of view from the virtual viewpoint in the virtual space has changed to the rotation direction about the line-of-sight direction based on the detection result of the detection unit 111, the object placement unit 113 determines the line-of-sight position P1 and the selected object. The selected object is rotated in the rotation direction so that the relative position in the rotation direction does not change. Thereby, even when the user tilts the head in the roll direction, the terminal device 10 maintains the relative relationship between the vertical and horizontal directions with respect to the user's head and the arrangement direction of the selected objects. Can make it easier to do.

  Note that the object placement unit 113 displays the selected object in the rotation direction even when it is determined that the field of view from the virtual viewpoint in the virtual space has changed to the rotation direction about the line-of-sight direction based on the detection result of the detection unit 111. It is not necessary to rotate it. In this case, on the selection menu screen, the selected object is displayed obliquely together with the visual field image as the background as the selected object is changed in the rotation direction.

  Further, the display control unit 114 causes the display unit 12 to display a field-of-view image from the virtual viewpoint in the virtual space according to the detection result of the detection unit 111, whereby the terminal device 10 can display the HDM system 1. According to the movement of the head of the user wearing the, the view image in the view direction in the virtual space can be provided so that the user can visually recognize it. Further, since the terminal device 10 can simultaneously display the view field image and the selected object in the virtual space, the user can select the game device while referring to the game information in the virtual space.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
Since the basic configurations of the HMD system 1 and the terminal device 10 according to the present embodiment are the same as the configurations illustrated in FIGS. 1, 2, and 11, characteristic processing in the present embodiment will be described. In the present embodiment, when a plurality of columns in which a plurality of selection objects are arranged are arranged, the selection object to be determined for selection is switched for each column based on the movement of the user's head. An example will be described.

  FIG. 22 is a diagram illustrating an example of a selection menu screen on which a selection object to be determined can be switched for each column. In the illustrated selection menu screen G25, two columns in which five selected objects are arranged in the Z-axis direction (the left column aligned on the broken line M1 and the right column aligned on the broken line M2) are arranged in the left-right direction (Y-axis direction). They are displayed side by side at a predetermined interval. In the left column, five selection objects of option A, option C, option E, option G, and option I are displayed side by side with a predetermined interval in the Z-axis direction. In the right column, five selection objects of option B, option D, option F, option H, and option J are displayed side by side with a predetermined interval in the Z-axis direction. The selection menu screen G25 has a specification in which the selection target is switched for each column. For example, when the left (A) column is a selection target, any one of the five selection objects of option A, option C, option E, option G, and option I can be selected. On the other hand, when the column on the right side (B) is a selection target, any one of the five selection objects of option B, option D, option F, option H, and option J can be selected.

  For example, when the movement of the user's head is the rotation direction (roll direction) about the X axis, the column to be selected is switched. Specifically, when the user's head movement is a right rotation, the column to be selected is switched from the left column to the right column, and when the user's head movement is a left rotation, the column is a selection target. The column switches from the right column to the left column. Note that if the movement of the user's head is rotating, regardless of the rotation direction of the movement of the user's head, the column to be selected is switched.

  Further, the column to be selected may be switched based not only on the rotation direction but also on the rotation direction and the rotation speed. For example, when the rotation speed is greater than or equal to a predetermined value, the view direction rotates and the column to be selected is switched. When the rotation speed is less than the predetermined value, the view direction is only rotated and the column to be selected is switched. It may not be possible.

  FIG. 23 is a flowchart illustrating an example of the object placement process according to the present embodiment. First, upon receiving a selection menu display trigger, the control unit 110 arranges a plurality of selected objects in a predetermined direction (for example, the Z-axis direction) in a plurality of columns in the virtual space (step S500). As a result, a selection menu screen (for example, see FIG. 22) including the view field image and a plurality of selection objects in two columns on the left and right is displayed on the display unit 12.

  The controller 110 detects the angular velocity based on the detection result of the sensor 13 (step S502). Then, based on the detected angular velocity, the control unit 110 determines whether or not there is a change in the visual field direction (gaze direction) (change in the rotation direction about the X axis) (step S504).

  When it is determined that there is a change in the rotation direction about the X axis (YES), the control unit 110 determines whether or not the change speed in the rotation direction is equal to or greater than a predetermined value (step S506). If it is determined that the change speed in the rotation direction is equal to or greater than the predetermined value (YES), the control unit 110 switches the column of the selection objects to be determined (step S508). On the other hand, when it is determined that the speed of change in the rotation direction is not equal to or greater than the predetermined value (NO), the control unit 110 does not switch the column of the selected objects to be determined.

  Next, control unit 110 determines whether or not a selection menu display-off trigger has been received (step S510). When it is determined that the trigger for turning off the display of the selection menu is not received (NO), the control unit 110 returns the process to step S502. On the other hand, when it is determined that the trigger for turning off the display of the selection menu has been received (YES), the control unit 110 deletes the selected object arranged in the virtual space (step S512) and ends the object arrangement processing. .

[Summary of Third Embodiment]
As described above, in the terminal device 10 according to the present embodiment, the object placement unit 113 places a plurality of rows in which a plurality of selected objects are arranged in the virtual space. The determination unit 112 determines whether or not the selected object is selected for each column, and switches the determination column based on the detection result of the detection unit 111. Thereby, when the terminal device 10 selects an option for each of a plurality of columns, the user only changes the direction of the head while restricting the movement of the selected object in a predetermined direction based on the change in the viewing direction. Since the selection target column can be switched, the operability when performing a selection operation on the option can be improved.

  For example, if the determination unit 112 determines that the field of view from the virtual viewpoint has changed in the rotation direction about the line-of-sight direction from the virtual viewpoint, the determination unit 112 switches the determination column according to the changed rotation direction. Thereby, when the terminal device 10 selects an option for each of a plurality of columns, the user can switch the column to be selected to a column in the corresponding direction only by changing the head direction to the roll direction. It is possible to improve the operability when performing a selection operation on the.

  The detection unit 111 detects information related to the direction change speed as information related to the direction of the terminal device 10. The determination unit 112 switches the determination column when the detection unit 111 detects information related to a change rate equal to or greater than a predetermined value. Thereby, when the terminal device 10 selects an option for each of a plurality of columns, the user can switch the column to be selected by changing the head direction at a speed equal to or higher than a predetermined value. Therefore, the operability when performing the selection operation can be improved.

  Note that the determination unit 112 uses both the change in the rotation direction around the line of sight from the virtual viewpoint as the axis of view and the case where the detection unit 111 detects information about the change speed greater than or equal to the predetermined value. The column to be determined based on one may be switched, or the column to be determined based on either one may be switched. For example, the determination unit 112 may switch the determination column to the right side or to the right side only because the rotation direction around the visual line direction from the virtual viewpoint changes to the right direction. When a change speed that is greater than or equal to a predetermined value is detected, the determination column may be switched to the right side. In addition, the determination unit 112 determines the column to be determined on the right side or the left side (for example, if there are two columns, from one to the other 3) when a change speed of a predetermined value or more is detected regardless of the change in the rotation direction. If it is more than the column, it may be switched to the side column set in advance.

[Modification]
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiment, and includes a design and the like within a scope not departing from the gist of the present invention. For example, the configurations described in the first to third embodiments can be arbitrarily combined. For example, in the second and third embodiments, after the line-of-sight position P1 overlaps the determination region, the elapsed time is measured and the elapsed time gauge T1 is displayed, as in the first embodiment. The presence or absence of selection is determined (determined) based on the time.

  In the above embodiment, the example in which the arrangement direction of the selection objects is the Z-axis direction has been described. However, the arrangement direction of the selection objects is not limited to the Z-axis direction, and may be, for example, the Y-axis direction. The direction may have a Y-axis component and a Z-axis component. When the selected objects are arranged in three dimensions, the arrangement direction of the selected objects may be a direction having an X-axis component, a Y-axis component, and a Z-axis component. Further, in the example illustrated in FIG. 22, an example in which a plurality of columns in which a plurality of selection objects are arranged in the Z-axis direction is arranged in the left and right (Y-axis direction), but a plurality of columns in which a plurality of selection objects are arranged in the Y-axis direction. Are arranged in the vertical direction (Z-axis direction), and the column may be switched in the vertical direction (Z-axis direction).

  In the above-described embodiment, the configuration of the HMD system 1 that can be used as an HMD by attaching the terminal device 10 to the attachment 2 has been described as an example. However, the configuration is not limited thereto. Even if 2 is not required, it may be a form that can be mounted on the head itself, that is, an HMD. Moreover, in the said embodiment, although the terminal device 10 demonstrated the structural example which can be mounted | worn to a user's head as HMD (HMD system) by which at least one part is comprised, HMD can mount | wear to a head completely. The configuration is not limited to a simple shape. For example, the terminal device 10 that can be used as an HMD maintains the positional relationship between the user's eyes and the display unit 12 in a predetermined relationship by holding the user's eyes in front of the eyes regardless of whether or not there is an attachment. It may be a device (for example, a shape like binoculars) that makes it possible to visually recognize the view field image and various objects.

  In the above embodiment, an example of a selection object as an option displayed on the selection menu screen has been described. However, the present invention is not limited to this. For example, the selected object may be an object arranged as a character, an item, or the like having a determination area in a virtual space in a game executed by the terminal device 10. As an example, the process related to the selected object described in the first embodiment may be applied to a process for a plurality of enemy objects (selected objects) flying in the virtual space of the game. For example, when an attack is irradiated based on the user's line-of-sight position and the irradiation to the enemy object (selected object) (positional relationship where the determination area and the line-of-sight position overlap) passes for a predetermined time, the enemy object is shot down (selected object) You may apply in the process of having selection.

  Further, the control unit 110 is recorded by recording a program for realizing the functions of the control unit 110 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. You may perform the process of. Here, “loading and executing a program recorded on a recording medium into a computer system” includes installing the program in the computer system. The “computer system” here includes an OS and hardware such as peripheral devices. Further, the “computer system” may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and dedicated line. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM. The recording medium also includes a recording medium provided inside or outside that is accessible from the distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program that can be executed by the terminal device. That is, the format stored in the distribution server is not limited as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device. Note that the program may be divided into a plurality of parts, downloaded at different timings, and combined in the terminal device, or the distribution server that distributes each of the divided programs may be different. Furthermore, the “computer-readable recording medium” holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or a client when the program is transmitted via a network. Including things. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  Further, the one line-of-sight position part or all of the functions of the control unit 110 described above may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function described above may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

[Appendix]
From the above description, the present invention is grasped as follows, for example. In order to facilitate understanding of the present invention, reference numerals in the accompanying drawings are appended in parentheses for convenience, but the present invention is not limited to the illustrated embodiment.

  (Additional remark 1) The terminal device (10) which concerns on 1 aspect of this invention is a head mounted display system which can display the visual field image showing the visual field from the virtual viewpoint in virtual space as a stereoscopic vision image using binocular parallax. A detection unit (111, S112, S402, S502) that detects information related to the direction of the terminal device, and a determination area that determines whether or not there is a selection from the user An object arrangement unit (113, S100, S200, S300, S400, S500) that arranges a plurality of selected objects in a predetermined direction in the virtual space, and the virtual space according to the detection result of the detection unit A display control unit (114, for displaying the selected object arranged in the view direction from the virtual viewpoint on the display unit) 100, S200, S300, S400, S500), and the object placement unit determines that the viewing direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit Then, the selected object is moved while restricting movement in the predetermined direction based on the change in the viewing direction (S406).

  According to the configuration of Supplementary Note 1, the terminal device can restrict the relative position between the line-of-sight position and the determination area of the selected object so that it does not change in a direction unnecessary for selection. The operability at the time of performing can be improved.

  (Additional remark 2) Moreover, 1 aspect of this invention is a terminal device of Additional remark 1, Comprising: The said object arrangement | positioning part determines the gaze direction from the said virtual viewpoint based on the detection result of the said detection part, An object for enabling visual recognition of the visual line position corresponding to the visual line direction from the virtual viewpoint is arranged in the virtual space.

  According to the configuration of Supplementary Note 2, for example, the terminal device determines the line-of-sight position corresponding to the line-of-sight direction in the visual field image (for example, the center of the visual field image) according to the orientation of the head of the user wearing the HMD system. It is possible to improve the operability by making it easier for the user to recognize the positional relationship between the line-of-sight position and the selected object.

  (Additional remark 3) Moreover, 1 aspect of this invention is a terminal device of Additional remark 2, Comprising: The said object arrangement | positioning part makes the said virtual space the object for enabling the said visual line position to be visually recognized, and the said selection object. It arrange | positions in the same surface (perpendicular to the said visual line direction).

  According to the configuration of Supplementary Note 3, the terminal device can realize an option selection operation UI based on the relative position (positional relationship) between the line-of-sight position and the selected object. Specifically, the terminal device 10 arranges the line-of-sight position P1 and the selected object on the same plane in the virtual space, so that the line-of-sight position P1 and the selected object have the same focal length in stereoscopic vision using binocular parallax. Therefore, the positional relationship between the two can be easily recognized.

  (Additional remark 4) Moreover, 1 aspect of this invention is a terminal device of Additional remark 2 or Additional remark 3, Comprising: The said object arrangement | positioning part is the said virtual viewpoint in the said virtual space based on the detection result of the said detection part. When it is determined that the viewing direction from the direction has changed, the line-of-sight position falls within the width of the direction perpendicular to the predetermined direction in the determination area of the selected object for directions other than the predetermined direction. In addition, the selected object is moved in accordance with the change in the viewing direction.

  According to the configuration of Supplementary Note 4, the terminal device deviates from the range of the width of the determination area of the selected object in directions other than the direction necessary for selecting the selected object (that is, the direction unnecessary for the operation). In order to limit the selection, it is possible to facilitate the selection operation for the options.

  (Additional remark 5) Moreover, 1 aspect of this invention is a terminal device as described in any one of Additional remark 2 to Additional remark 4, Comprising: The said object arrangement | positioning part is based on the detection result of the said detection part. When it is determined that the viewing direction from the virtual viewpoint has changed, the direction of view other than the predetermined direction is changed according to the change in the viewing direction so that the relative position between the viewing position and the selected object does not change. To move the selected object.

  According to the configuration of Supplementary Note 5, the terminal device determines the line-of-sight position and the determination area of the selected object, except for the direction in which the line-of-sight position necessary for selecting the selected object is moved (that is, the direction unnecessary for the operation). Since the relative position is limited so as not to change, it is possible to facilitate the selection operation for the option.

  (Additional remark 6) Moreover, 1 aspect of this invention is a terminal device as described in any one of Additional remark 2 to Additional remark 5, Comprising: Based on the relative position of the said gaze position and the determination area | region of the said selection object, A determination unit (112, S204 to S210, S304 to S310) that determines whether or not the selected object is selected is provided.

  According to the configuration of Supplementary Note 6, the terminal device can perform an operation UI for selecting an option using the user's line of sight in the HMD system.

  (Additional remark 7) Moreover, 1 aspect of this invention is a terminal device of Additional remark 6, Comprising: The said object arrangement | positioning part arrange | positions the several row | line | column in which the said several selection object each arranges in the said virtual space. The determination unit determines whether the selected object is selected for each column, and switches the determination column based on the detection result of the detection unit (S508).

  According to the configuration of Supplementary Note 7, when the terminal device selects an option for each of a plurality of columns, the terminal device restricts the movement of the selected object in a predetermined direction based on the change in the visual field direction, Since the selection target column can be switched simply by changing the operability, it is possible to improve the operability when performing the selection operation on the option.

  (Additional remark 8) Moreover, 1 aspect of this invention is a terminal device of Additional remark 7, Comprising: The said determination part is a rotation direction centering around the visual line direction from the said virtual viewpoint on the said visual field from the said virtual viewpoint. When it is determined that the change has occurred, the determination column is switched according to the changed rotation direction (S504, S508).

  According to the configuration of Supplementary Note 8, when the terminal device selects an option for each of a plurality of columns, the user can switch the column to be selected to a column in a corresponding direction simply by changing the head direction to the roll direction. Therefore, it is possible to improve the operability when performing a selection operation on the options.

  (Additional remark 9) Moreover, 1 aspect of this invention is a terminal device of Additional remark 7 or Additional remark 8, Comprising: The said detection part detects the information regarding the change speed of this direction as information regarding the direction of the said terminal device. The determination unit switches the determination column when the detection unit detects information about the change rate that is equal to or greater than a predetermined value (S506, S508).

  According to the configuration of Supplementary Note 9, when the terminal device selects an option for each of a plurality of columns, the user can switch the column to be selected by changing the head direction at a speed equal to or higher than a predetermined value. Thus, it is possible to improve the operability when performing a selection operation on the options.

  (Additional remark 10) Moreover, 1 aspect of this invention is a terminal device as described in any one of Additional remark 2 to Additional remark 9, Comprising: The said object arrangement | positioning part is in the said virtual space based on the detection result of the said detection part. The selected object so that the relative position in the rotation direction between the line-of-sight position and the selection object does not change when it is determined that the field of view from the virtual viewpoint changes in the rotation direction around the line-of-sight direction. Is rotated in the rotation direction.

  According to the configuration of Supplementary Note 10, even when the user tilts his / her head in the roll direction, the relative relationship between the up / down / left / right direction with respect to the user's head and the arrangement direction of the selected object is maintained. It is possible to facilitate the selection operation for.

  (Additional remark 11) Moreover, 1 aspect of this invention is a terminal device as described in any one of Additional remark 1 to Additional remark 10, Comprising: The said display control part is in the said virtual space according to the detection result of the said detection part. A visual field image in the visual field direction from the virtual viewpoint is displayed on the display unit. In addition, since the terminal device can simultaneously display the field-of-view image in the virtual space and the selected object, the user can select the game while referring to the game information in the virtual space.

  According to the configuration of Supplementary Note 11, the terminal device can provide a visual field image in the visual field direction in the virtual space so that the user can visually recognize the movement according to the movement of the head of the user wearing the HDM system.

  (Additional remark 12) Moreover, the program which concerns on 1 aspect of this invention is a program for functioning a computer as a terminal device as described in any one of Claims 1-11.

  According to the configuration of Supplementary Note 12, since the program can limit the change in the relative position between the line-of-sight position and the determination area of the selected object depending on the direction, the operability when performing a selection operation on an option can be improved. Can do.

  1 HMD system, 2 attachment, 3R lens for right eye, 3L lens for left eye, 5 strap, 10 terminal device, 11 input unit 11, 12 display unit, 13 sensor, 14 timer, 15 storage unit, 16 communication unit, 17 CPU, 110 control unit, 111 detection unit, 112 determination unit, 113 object placement unit, 114 display control unit

Claims (5)

As a stereoscopic image using binocular parallax, a terminal device that can be used as at least a part of a head-mounted display system capable of displaying a visual field image representing a visual field from a virtual viewpoint in a virtual space,
A detection unit for detecting information on the direction of the terminal device;
A plurality of selection objects having a determination area for determining whether or not there is a selection from a user, while arranging a line-of-sight position object for enabling visual recognition of a line-of-sight position corresponding to the line-of-sight direction from the virtual viewpoint. An object arrangement unit arranged in a predetermined direction in the virtual space;
A display control unit that displays at least a part of the plurality of selection objects arranged in the virtual space and the line-of-sight position object on a display unit according to a detection result of the detection unit;
A determination unit that determines whether or not the one selected object is selected based on a time when the relative position between the line-of-sight position and the determination region of the one selected object included in the plurality of selected objects is in a specific state. ,
With
The object placement unit is
When it is determined that the view direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit, the plurality of selected objects are moved in the arrangement direction based on the change in the view direction. Moving the selected object while restricting,
Terminal device. The object placement unit is
An object indicating the timing situation is arranged in the virtual space,
The display control unit
According to the detection result of the detection unit, the display unit displays an object indicating the timing situation arranged in the virtual space.
The terminal device according to claim 1. The object placement unit is
Arranging the line-of-sight position object and one selection object included in the plurality of selection objects on the same plane in the virtual space;
The terminal device according to claim 2. The object placement unit is
When it is determined that the view direction from the virtual viewpoint in the virtual space has changed based on the detection result of the detection unit, the line-of-sight position in the determination area of the selected object is the direction other than the arrangement direction. Moving the selected object according to the change in the viewing direction so as to be within the width of the direction orthogonal to the arrangement direction;
The terminal device according to claim 2 or claim 3.   The program for functioning a computer as a terminal device as described in any one of Claims 1-4.

Images