JP6587364B2 - Program and image display system - Google Patents

Description

  The present invention relates to a technique for displaying an image obtained by imaging a virtual space.

  Conventionally, a technique has been proposed in which an image captured by a virtual camera (hereinafter referred to as “virtual camera”) installed in a virtual space is displayed on a display device. For example, Patent Document 1 discloses a configuration in which a menu image is displayed in a virtual space according to the tilt of a head mounted display.

Japanese Patent Laid-Open No. 2006-115122

  Conventionally, contents such as games that communicate with objects such as characters in a virtual space have been proposed. In a scene in which this type of content is displayed on a head-mounted display, there is a problem that user operations are restricted. If an operation device separate from the head-mounted display is prepared, restrictions on the operation by the user can be eased, but there is a problem that the device configuration becomes complicated. In view of the above circumstances, an object of the present invention is to easily realize an operation on an object in a virtual space.

  In order to solve the above problems, a program according to a preferred aspect of the present invention is an image obtained by capturing a virtual space with a virtual camera whose direction is controlled according to the direction of the head mounted display, A display control unit for displaying the used stereoscopic image on the display device of the head mounted display, and rotation in the roll direction of the head mounted display, and the direction in the virtual space changes according to the direction of the head mounted display The computer is caused to function as an operation control unit that controls the operation of the object according to the positional relationship between the reference line to be performed and the object in the virtual space.

  An image display system according to a preferred aspect of the present invention is an image display system including a head mounted display and an information processing device, and the direction of the information processing device is controlled according to the direction of the head mounted display. A display controller that displays a stereoscopic image using binocular parallax on a display device of the head-mounted display, which is an image of a virtual space captured by a virtual camera, and rotation of the head-mounted display in the roll direction; A motion control unit that controls the motion of the object according to a positional relationship between a reference line whose direction in the virtual space changes according to the direction of the head mounted display and the object in the virtual space.

1 is a perspective view illustrating the appearance of an image display system according to a first embodiment of the invention. It is a block diagram which illustrates the composition of an image display system. 1 is a block diagram illustrating a functional configuration of an image display system. It is explanatory drawing of virtual space. It is explanatory drawing of the image which a display apparatus displays. It is a flowchart which illustrates the content of the process which a control apparatus performs. It is explanatory drawing of rotation of HMD in a roll direction. It is explanatory drawing of the image which a display apparatus displays. It is explanatory drawing of the movement of the virtual camera in modification A4. It is explanatory drawing of the change of the imaging range in modification A5. It is a block diagram which illustrates the functional structure of the image display system in 2nd Embodiment. It is explanatory drawing of the image displayed when a reference line does not cross | intersect a character in 2nd Embodiment. It is explanatory drawing of the image displayed when a reference line cross | intersects a character in 2nd Embodiment. It is explanatory drawing of the relationship between the character and instruction | indication image in 2nd Embodiment. It is a flowchart which illustrates the content of the process which the control apparatus of 2nd Embodiment performs. It is explanatory drawing of the time-dependent change of the instruction | indication image in modification B2. It is explanatory drawing of the instruction | indication image in modification B3. It is explanatory drawing of the instruction | indication image in modification B4. It is a perspective view which illustrates the appearance of the image display system in modification C5.

[A: First Embodiment]
FIG. 1 is a perspective view illustrating the appearance of an image display system 1A according to the first embodiment of the invention. The image display system 1A is a video device for displaying an image, and includes a terminal device 10 and a mounting tool 20. The terminal device 10 is a portable information terminal such as a smartphone or a tablet terminal. The terminal device 10 is detachably installed on the mounting tool 20. The mounting tool 20 is a tool for mounting the terminal device 10 on the user U's head. For example, as illustrated in FIG. 1, a goggle type attachment having a belt wound around the head of the user U is suitable as the wearing tool 20.

  FIG. 2 is a block diagram illustrating the configuration of the terminal device 10. As illustrated in FIG. 2, the terminal device 10 is a computer system including a control device 11, a storage device 12, a display device 13, and a detection device 14. The control device 11 is a processor such as a CPU (Central Processing Unit) and comprehensively controls each element of the image display system 1A. The storage device 12 stores a program executed by the control device 11 and various data used by the control device 11. For example, the storage device 12 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of types of recording media.

  The display device 13 displays an image under the control of the control device 11. For example, a flat display such as a liquid crystal display panel or an organic EL (Electroluminescence) display panel is used as the display device 13. As understood from FIG. 1, in the state where the wearing tool 20 is mounted on the head of the user U, the display device is placed in front of both eyes of the user U with the display surface facing the head of the user U. 13 is arranged. The display device 13 according to the first embodiment displays a stereoscopic image that allows the user U to perceive a stereoscopic effect. The stereoscopic image includes a right-eye image and a left-eye image to which binocular parallax is added. By causing the right eye image to be visually recognized by the right eye of the user U and the left eye image being visually recognized by the left eye of the user U, the user U perceives a stereoscopic effect.

  As illustrated in FIG. 2, the display device 13, the detection device 14, and the mounting tool 20 constitute a head mounted display (hereinafter referred to as “HMD”) 30, and the control device 11 and the storage device 12 display an image on the HMD 30. The information processing apparatus 40 to be displayed is configured. That is, the image display system 1 </ b> A includes the HMD 30 and the information processing device 40. As described above, in the first embodiment, the information processing device 40, the display device 13, and the detection device 14 are realized by the single terminal device 10. The information processing apparatus 40 may be grasped as an element of the HMD 30.

  The detection device 14 in FIG. 2 is a sensor that outputs a detection signal corresponding to the attitude of the HMD 30. Specifically, the detection device 14 is selected from a plurality of types of sensors such as a gyro sensor that detects angular velocity, an acceleration sensor that detects acceleration, an inclination sensor that detects an inclination angle, and a geomagnetic sensor that detects a direction by geomagnetism. One or more types of sensors. By analyzing the detection signal output from the detection device 14, it is possible to identify the posture of the HMD 30 or a change in the posture. In other words, the detection device 14 is a sensor that outputs a detection signal corresponding to the attitude of the display device 13 or the terminal device 10. Illustration of an amplifier that amplifies the detection signal and an A / D converter that converts the detection signal from analog to digital is omitted for convenience.

  As illustrated in FIG. 1, the attitude of the HMD 30 is defined by three axes (X axis, Y axis, and Z axis) orthogonal to each other at the origin O. The X axis is an axis perpendicular to the display surface of the display device 13 and corresponds to the front-rear direction of the user U. The Y axis and the Z axis are axes parallel to the display surface of the display device 13. The Y axis corresponds to the horizontal direction of the user U, and the Z axis corresponds to the vertical direction of the user U. Focusing on the display surface of the display device 13, the X axis corresponds to the depth direction of the display surface, the Y axis corresponds to the horizontal direction of the display surface, and the Z direction corresponds to the vertical direction of the display surface. The direction of the circumference around the X axis is the roll direction, the direction of the circumference around the Y axis is the pitch direction, and the direction of the circumference around the Z axis is the yaw direction. The direction of the X axis is determined by the angle in the pitch direction and the angle in the yaw direction. The rotation in the roll direction corresponds to an operation of tilting the head left and right with the user U facing forward. That is, for example, when the user U raises his / her neck, the HMD 30 rotates in the roll direction. In the following description, as illustrated in FIG. 1, one side in the roll direction is denoted as “first side” and the other side is denoted as “second side”. One of clockwise and counterclockwise rotation around the X axis is the first side, and the other is the second side.

  The control device 11 in FIG. 2 functions as an attitude analysis unit 41, a display control unit 42A, and an operation control unit 43A as illustrated in FIG. 3 by executing a program stored in the storage device 12. A part or all of the functions of the control device 11 may be realized by a dedicated electronic circuit.

  The posture analysis unit 41 identifies the posture of the HMD 30 by analyzing the detection signal output from the detection device 14. Specifically, the posture analysis unit 41 sequentially generates posture data regarding the posture of the HMD 30 at a predetermined cycle. The attitude data is data representing the attitude of the HMD 30 or a change in the attitude.

  The display control unit 42A causes the display device 13 to display an image captured by the virtual camera E in the virtual space V. FIG. 4 is an explanatory diagram of the virtual space V. As illustrated in FIG. 4, the virtual camera E is installed in the virtual space V and images a specific range (hereinafter referred to as “imaging range”) R in the virtual space V. The imaging range R is a range over a predetermined angle in the vertical direction and the horizontal direction with respect to the optical axis of the virtual camera E. The optical axis of the virtual camera E corresponds to the virtual line of sight of the user U in the virtual space V. The display control unit 42A of the first embodiment causes the display device 13 to display a stereoscopic image (right-eye image and left-eye image) using binocular parallax. Therefore, the virtual camera E includes a first virtual camera that captures an image for the left eye and a second virtual camera that captures an image for the right eye. Image data representing a stereoscopic image captured by the virtual camera E is sequentially supplied from the display control unit 42A to the display device 13, whereby the stereoscopic image is displayed on the display device 13.

  The display control unit 42A controls the direction of the virtual camera E in the virtual space V according to the direction of the HMD 30 specified by the posture analysis unit 41. Specifically, the direction of the optical axis of the virtual camera E (and the imaging range R) changes in conjunction with the rotation of the X axis around the origin O. For example, when the HMD 30 rotates in the yaw direction, the optical axis of the virtual camera E rotates left and right around the origin O, and when the HMD 30 rotates in the pitch direction, the optical axis of the virtual camera E changes to the origin O Rotate up and down around the center. In the first embodiment, even when the HMD 30 rotates in the roll direction, the attitude of the virtual camera E (an angle with respect to the optical axis) does not change. However, the inclination of the virtual camera E around the optical axis may be changed according to the rotation of the HMD 30 in the roll direction.

  As illustrated in FIG. 4, the character C is arranged in the virtual space V where the virtual camera E is installed. The character C of the first embodiment is an object that represents a virtual creature (for example, a human being, an animal, or a monster) that is active in the virtual space V, and includes a head and a torso. A game that interacts with the character C while being in contact (skinship) with the character C at any time in the virtual space V is provided to the user U by the image display system 1A.

  A reference line Q is set in the virtual space V. The reference line Q is a straight line whose direction in the virtual space V changes according to the direction of the HMD 30 (the direction of the X axis). In the first embodiment, as illustrated in FIG. 4, the optical axis of the virtual camera E is illustrated as the reference line Q. For example, if the user U turns his head to the right, the reference line Q rotates to the right, and if the user U turns his head to the left, the reference line Q rotates to the left. As understood from the above description, the reference line Q is a virtual line of sight of the user U in the virtual space V.

  FIG. 5 is a schematic diagram of an image displayed on the display device 13. As illustrated in FIG. 5, the display control unit 42 </ b> A causes the display device 13 to display a stereoscopic image captured by the virtual camera E in the virtual space V. When the character C is located within the imaging range R, the character C is displayed on the display device 13. Further, the display control unit 42A causes the display device 13 to display an image (hereinafter referred to as “instruction image”) G representing the direction of the reference line Q. The instruction image G is arranged at a predetermined position on the reference line Q in the virtual space V. Since the reference line Q in the first embodiment is the optical axis of the virtual camera E, the instruction image G is displayed at a predetermined position in the display surface of the display device 13.

  The motion control unit 43A in FIG. 3 controls the motion of the character C in the virtual space V. The motion control unit 43A controls the motion of the character C according to the progress of the game, and causes the character C to perform a motion corresponding to the contact of the user U in the virtual space V (hereinafter referred to as “reaction motion”). Specifically, the motion control unit 43A determines whether or not the user U has touched the character C in the virtual space V, and causes the character C to perform a reaction motion to the touch by the user U. For example, various actions such as a change in posture, a change in facial expression, or pronunciation of a specific line are typical examples of reaction actions.

  The motion control unit 43A determines whether or not the user U has touched the character C in the virtual space V in accordance with the change in the posture of the HMD 30 specified by the posture analysis unit 41. The motion control unit 43A of the first embodiment determines that the user U has touched the character C in the virtual space V when the HMD 30 rotates in the roll direction. That is, the rotation of the HMD 30 in the roll direction corresponds to an instruction (motion instruction) for the user U to contact the character C in the virtual space V. Specifically, when the HMD 30 rotates in the roll direction, the motion control unit 43A contacts the user U at a point P (hereinafter referred to as “contact point”) where the reference line Q intersects the surface of the character C. Is determined. The contact point P is located on the surface of the character C. When the reference line Q intersects the surface of the character C at a plurality of points, the point closest to the virtual camera E (that is, the user U) among the plurality of points is selected as the contact point P.

  As described above, in the first embodiment, the motion of the character C in the virtual space V is controlled according to the rotation of the HMD 30 in the roll direction. Therefore, the instruction from the user U is reflected in the action of the character C without requiring an operating device for the user U to input an instruction for controlling the action of the character C separately from the HMD 30. be able to. Particularly in the first embodiment, since it is determined that the user U has touched the character C when the reference line Q and the character C intersect, the user U captures the character C in the field of view. The character C can be operated. That is, there is an advantage that the user U can operate the character C more intuitively.

  FIG. 6 is a flowchart illustrating specific contents of a process (hereinafter referred to as “first control process”) executed by the operation control unit 43A. The operation control unit 43A repeatedly executes the first control process of FIG. 6 at a predetermined cycle. The storage device 12 stores contact determination data representing whether or not the user U is in contact with the character C in the virtual space V (hereinafter referred to as “contact state”). The contact determination data is, for example, a flag indicating either a contact state or a non-contact state.

  When the first control process is started, the operation control unit 43A determines whether or not the contact determination data indicates a contact state (Sa1). When the contact determination data indicates a non-contact state (Sa1: NO), the operation control unit 43A determines whether the HMD 30 has rotated in the roll direction (Sa2). Specifically, the motion control unit 43A determines whether or not the HMD 30 has rotated to the first side in the roll direction (for example, clockwise) by referring to the posture data sequentially supplied from the posture analysis unit 41. To do.

  FIG. 7 is an explanatory diagram of processing for determining whether or not the HMD 30 has rotated in the roll direction. As illustrated in FIG. 7, the operation control unit 43 </ b> A determines that the HMD 30 has rotated in the roll direction when the angle θ that the HMD 30 has rotated to the first side in the roll direction exceeds the threshold θt. The angle θ is an angle obtained by rotating the Z axis or the Y axis about the X axis, and is defined with the vertical direction as a reference (θ = 0), for example. On the other hand, when the rotation angle θ of the HMD 30 is less than the threshold θt, the operation control unit 43A does not determine that the HMD 30 has rotated in the roll direction. As described above, in the first embodiment, it is not determined that the HMD 30 has rotated in the roll direction when the rotation is less than the threshold θt. Therefore, it is possible to reduce the possibility that the HMD 30 is determined with an excessive frequency when it rotates. There is also an advantage that the possibility that the user U is determined to have rotated although the user U does not intend to rotate the HMD 30 (that is, the possibility of erroneous operation) can be reduced.

  Not only when the rotation in the roll direction alone occurs in the HMD 30, but also when the rotation in the pitch direction or the yaw direction occurs along with the rotation in the roll direction, the rotation angle in the roll direction exceeds the threshold θt. Is determined that the HMD 30 has rotated in the roll direction.

  In the first control process of FIG. 6, if it is determined that the HMD 30 has rotated to the first side in the roll direction (Sa2: YES), the motion control unit 43A determines whether the reference line Q intersects the character C in the virtual space V It is determined whether or not (Sa3). That is, it is determined whether or not the character C exists on the reference line Q. As illustrated in FIG. 8, when the reference line Q intersects the character C in the virtual space V (Sa3: YES), the motion control unit 43A is in a contact state in which the user U contacts the contact point P of the character C. It is determined that there is (Sa4). Specifically, the contact determination data is changed to a numerical value indicating the contact state. As understood from the above description, the motion control unit 43A determines that the user U has touched the contact point P of the character C in the virtual space V when the HMD 30 rotates in the roll direction. That is, the user U can instruct the character C to contact the contact point P by rotating the HMD 30 in the roll direction.

  When the state transitions to the contact state, the motion control unit 43A causes the character C to execute a reaction motion (Sa5). Specifically, the action control unit 43A causes the character C to execute a reaction action according to the position of the contact point P where the reference line Q intersects in the character C. That is, the reaction action performed by the character C changes according to the position of the contact point P. For example, a table that defines the type of reaction action for each of a plurality of regions that segment the surface of the character C is stored in the storage device 12. The motion control unit 43A causes the character C to perform a reaction motion defined in the table for a region including the contact point P among the plurality of regions on the surface of the character C. For example, when the contact point P is located at the head of the character C, the action control unit 43A performs a reaction action (for example, a favorable action such as rejoicing, laughing, or approaching) that accepts the contact by the user U. Let C execute. On the other hand, when the contact point P is located on the torso of the character C, the action control unit 43A performs a reaction action (for example, a negative action such as anger, sadness, or separation) that rejects contact by the user U. Cause character C to execute. According to the above configuration, the reaction action of the character C can be variously changed according to the position of the contact point P.

  On the other hand, when the HMD 30 does not rotate to the first side in the roll direction (Sa2: NO), or when the reference line Q does not intersect the character C (Sa3: NO), the motion control unit 43A is a numerical value indicating a non-contact state. Maintain contact judgment data. That is, even when the HMD 30 rotates to the first side in the roll direction, it is determined that the user U is not in contact with the character C when the reference line Q does not intersect the character C as illustrated in FIG.

  When the user U touches the character C in the virtual space V, it is determined that the contact determination data indicates a contact state in step Sa1 of the first control process thereafter. When the contact determination data indicates a contact state (Sa1: YES), the motion control unit 43A refers to the posture data sequentially supplied from the posture analysis unit 41, so that the HMD 30 is on the second side in the roll direction (e.g. It is determined whether or not it has been rotated clockwise. Specifically, the operation control unit 43A determines that the HMD 30 has rotated to the second side in the roll direction when the rotation angle θ with respect to the second side in the roll direction exceeds a predetermined threshold value θt. When the HMD 30 rotates to the second side in the roll direction (Sa6: YES), the motion control unit 43A determines that the contact with the character C has been released (Sa7). Specifically, the operation control unit 43A changes the contact determination data to a numerical value indicating a non-contact state. On the other hand, when the HMD 30 does not rotate to the second side in the roll direction (Sa6: NO), the operation control unit 43A maintains the contact determination data at a numerical value indicating the contact state.

  Note that when the contact with the character C is released, it is preferable that the motion control unit 43A causes the character C to perform an action that reacts to the release of the contact. A configuration that is not executed is also assumed.

  As understood from the above description, the motion control unit 43A determines that the user U has touched the character C when the HMD 30 rotates to the first side in the roll direction (Sa2: YES) (Sa4). On the other hand, in a state where the user U is in contact with the character C (Sa1: YES), it is determined that the contact with the character C is released when the HMD 30 rotates to the second side in the roll direction (Sa6: YES). According to the above configuration, the user U can instruct the generation and release of contact with the character C in the virtual space V according to the direction of rotation of the HMD 30 in the roll direction. Since rotations opposite to each other in the roll direction correspond to the opposite actions of generation and release of contact with the character C, there is also an advantage that the user U can easily grasp the generation and release of contact with the character C intuitively. is there.

  In the first embodiment, for example, the configurations exemplified below may be adopted.

[Modification A1]
The motion control unit 43A in the modification A1 causes the character C to execute a reaction motion corresponding to the time length during which the contact with the character C is maintained in the virtual space V. Specifically, the operation control unit 43A changes the reaction operation over time within a period during which the contact state continues (hereinafter referred to as “contact period”). The contact period is a period from when the HMD 30 rotates to the first side in the roll direction until it rotates to the second side. Note that a period from when the contact determination data is set to a numerical value indicating the contact state to when the contact determination data is changed to a numerical value indicating the non-contact state may be set as the contact period. The action control unit 43A changes the reaction action of the character C over time from the start point to the end point of the contact period. Therefore, the reaction action of the character C changes according to the time length of the contact period. According to the above configuration, the reaction action of the character C can be variously changed according to the length of time that the user U contacts the character C.

[Modification A2]
The storage device 12 in the modified example A2 stores parameters relating to the character C. Specifically, the preference of the character C with respect to the user U is stored in the storage device 12 as a parameter. Favorability is a parameter indicating the degree of favorable emotion from the character C to the user U. The motion control unit 43A changes the likability according to the progress of the game, and also changes the likability from the character C to the user U according to the contact with the character C. Specifically, the greater the degree of contact (for example, the number of times or time) of the user U with the character C, the greater the preference from the character C to the user U is set to a larger numerical value. In the above example, the character C's favorability with respect to the user U has been described. However, the favorability stored in the storage device 12 is the favorability from the character C with respect to the character (player character) used by the user U. But you can. The user U can have a plurality of player characters in the virtual space V.

  In addition, the action control unit 43A changes the reaction action according to the favorability stored for the character C. That is, when the user U touches the character C (Sa2: YES, Sa3: YES), the type of reaction action performed by the character C (Sa4) changes according to the favorableness of the character C. For example, in a state where the number of times or time of contact with the character C is insufficient and the liking is low, when the user U touches the head, the character C executes a reaction action that rejects the contact. On the other hand, in a state where the number of times or time of contact with the character C is sufficiently secured and the likability is high, when the user U touches the head, the character C executes a reaction action that accepts the contact. According to the above configuration, the reaction action of the character C can be variously changed according to the parameter related to the character C.

  It should be noted that the parameter that affects the reaction action of the character C is not limited to the favorability exemplified above. For example, various parameters such as the degree of growth (level) of the character C or the familiarity between the user U and the character C change according to the contact of the character C by the user U, and according to the parameter. Thus, the reaction action of the character C is controlled.

[Modification A3]
The motion control unit 43A in the modified example A3 causes the character C to execute a reaction motion corresponding to the angle θ that the HMD 30 rotates in the roll direction. For example, the rotation angle θ of the HMD 30 corresponds to a virtual pressure at which the user U contacts the character C in the virtual space V. For example, when the rotation angle θ is small, it means that the user U is lightly touching the character C, and when the rotation angle θ is large, the user U strongly presses the character C. Means the state. For example, when the rotation angle θ is small, the motion control unit 43A causes the character C to perform a reaction operation that accepts contact by the user U, and rejects contact by the user U when the rotation angle θ is large. The character C is caused to execute a reaction action. According to the above configuration, the reaction action of the character C can be variously changed according to the rotation angle θ of the HMD 30 (that is, a virtual pressure at which the user U contacts the character C).

[Modification A4]
The display control unit 42A in the modification A4 moves the virtual camera E (that is, a virtual viewpoint in the virtual space V) in the virtual space V according to the rotation of the HMD 30 in the roll direction. Specifically, the display control unit 42A changes the positional relationship between the virtual camera E and the character C in the virtual space V in conjunction with the rotation of the HMD 30. For example, the distance between the virtual camera E and the character C is controlled according to the rotation angle θ of the HMD 30. That is, the virtual camera E moves in the direction of the object in conjunction with the rotation of the HMD 30 in the roll direction.

  For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A moves the virtual camera E by an amount of movement corresponding to the rotation angle θ (θ> 0) of the HMD 30 as indicated by an arrow a1 in FIG. The character C is brought close to the virtual space V. On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A moves the virtual camera E by the amount of movement corresponding to the rotation angle θ (θ <0) of the HMD 30 as indicated by an arrow a2 in FIG. The character C is separated from the virtual space V. According to the above configuration, the user U can move the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[Modification A5]
The display control unit 42A in the modified example A5 changes the imaging range R (that is, the angle of view of the virtual camera E) by the virtual camera E according to the rotation of the HMD 30 in the roll direction. That is, the range displayed on the display device 13 in the virtual space V changes according to the rotation of the HMD 30 in the roll direction. For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A reduces the imaging range R at a ratio corresponding to the rotation angle θ of the HMD 30 as indicated by an arrow b1 in FIG. Therefore, for example, the character C in the virtual space V is zoomed in (enlarged). On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A expands the imaging range R at a ratio according to the rotation angle θ of the HMD 30 as indicated by an arrow b2 in FIG. Therefore, for example, the character C in the virtual space V is zoomed out (reduced). According to the above configuration, the user U can change the imaging range R of the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[Modification A6]
In the first embodiment, it is determined that the contact with the character C has been released when the HMD 30 rotates to the second side in the roll direction (Sa6: YES). However, the condition for determining that the contact with the character C has been released is as follows. It is not limited to the above illustration. For example, the motion control unit 43A may determine that the contact with the character C has been released when the HMD 30 returns from the state rotated in the roll direction (Sa2: YES). Specifically, the motion control unit 43A sets the rotation angle θ of the HMD 30 to an initial angle (for example, θ = 0) at the time when rotation for contact with the character C is started in step Sa6 in FIG. If it has changed, it is determined that the contact with the character C has been released (Sa7). On the other hand, in step Sa2, the motion control unit 43A determines that the user U has touched the character C when the rotation angle θ changes regardless of the direction of rotation of the HMD 30 (first side / second side). To do. As understood from the above description, the occurrence and release of contact with the character C can be determined by monitoring the rotation angle θ of the HMD 30 without determining the rotation direction (first side / second side) of the HMD 30. Can be determined. That is, in determining contact with the character C, it is not always necessary for the motion control unit 43A to determine the direction of rotation of the HMD 30.

[Modification A7]
In the first embodiment, when the reference line Q intersects the character C in the virtual space V (Sa3: YES), the character C is caused to perform a reaction action to the contact by the user U. The conditions regarding the positional relationship with C are not limited to the above examples. For example, the reaction action may be executed by the character C on condition that the distance between the reference line Q and the character C is below a predetermined threshold. When the distance between the reference line Q and the character C is less than a predetermined threshold, the case where the reference line Q is separated from the character C within a range below the threshold is included in addition to the case where the reference line Q intersects the character C. . Further, the reaction action may be executed by the character C on condition that the reference line Q intersects a specific part of the character C. When the reference line Q intersects other than a specific part of the character C, the motion control unit 43A does not cause the character C to execute a reaction motion. As understood from the above description, the motion control unit 43A according to the first embodiment determines the character C according to the positional relationship between the reference line Q and the character C (for example, the relationship between the reference line Q and the character C). It is expressed comprehensively as an element that controls the operation.

[B: Second Embodiment]
A second embodiment of the present invention will be described. In the following examples, elements having the same functions as those of the first embodiment are diverted using the same reference numerals used in the description of the first embodiment, and detailed descriptions thereof are appropriately omitted.

  The image display system in the second embodiment has the same configuration as that of the first embodiment illustrated in FIGS. 1 and 2. That is, the HMD 30 of the second embodiment includes the display device 13 and the detection device 14 of the terminal device 10 and the mounting tool 20, and the information processing device 40 includes the control device 11 and the storage device 12 of the terminal device 10. To do.

  FIG. 11 is a block diagram illustrating a functional configuration of the terminal device 10 according to the second embodiment. As illustrated in FIG. 11, the control device 11 of the second embodiment functions as a posture analysis unit 41, a display control unit 42 </ b> B, and an operation control unit 43 </ b> B by executing a program stored in the storage device 12. A part or all of the control device 11 may be realized by a dedicated electronic circuit. As in the first embodiment, the posture analysis unit 41 sequentially generates posture data related to the posture of the HMD 30 by analyzing the detection signal output from the detection device 14.

  Similar to the display control unit 42A of the first embodiment, the display control unit 42B displays an image in the imaging range R by the virtual camera E in the virtual space V and an instruction image G representing the direction of the reference line Q. 13 is displayed. As described above for the first embodiment, the reference line Q is a virtual straight line whose direction in the virtual space V changes according to the direction of the HMD 30.

  The display control unit 42B of the second embodiment changes the display mode of the instruction image G according to the positional relationship between the reference line Q and the character C (example of object) in the virtual space V. Specifically, the display control unit 42B makes the display mode of the instruction image G different when the reference line Q intersects the character C and when the reference line Q does not intersect the character C. The display mode means a property of an image that the user U can visually discriminate. For example, in addition to hue (color tone), saturation and lightness (gradation), which are the three attributes of color, size and image content (for example, a pattern or shape) are also included in the concept of display mode.

  When the reference line Q does not intersect the character C, the display control unit 42B causes the display device 13 to display the image G1 as the instruction image G as illustrated in FIG. The image G1 is a circular or dot-like object arranged in the virtual space V, and is arranged on the reference line Q in the virtual space V. On the other hand, when the reference line Q intersects the character C, the display control unit 42B causes the display device 13 to display an image G2 different from the image G1 as the instruction image G as illustrated in FIG. The image G2 is a planar object that schematically represents the user U's hand. As illustrated in FIG. 14, the display control unit 42 </ b> B arranges the image G <b> 2 at a point that intersects the reference line Q (that is, the contact point P) on the surface of the character C in the virtual space V. That is, the image G2 contacts the contact point P on the surface of the character C in the virtual space V. The contact point P is also referred to as a point where the image G2 contacts on the surface of the character C.

  In the above description, switching between the image G1 and the image G2 is exemplified, but the change in the display mode of the instruction image G includes switching between display and non-display. That is, the display control unit 42B may display the image G2 as the instruction image G when the reference line Q intersects the character C, and may hide the instruction image G when the reference line Q does not intersect the character C. .

  As understood from the above description, in the second embodiment, the display mode of the instruction image G representing the direction of the reference line Q in the virtual space V changes according to the positional relationship between the reference line Q and the character C. . Therefore, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V. Further, since the instruction image G (image G2) is arranged on the surface of the character C in the virtual space V, the user U can easily perceive the state in contact with the surface of the character C in the virtual space V. There is. Since the instruction image G is arranged on the surface of the character C, there is an advantage that the user U can easily grasp the contact point P visually.

  The motion control unit 43B in FIG. 11 controls the motion of the character C in the virtual space V, similarly to the motion control unit 43A in the first embodiment. Specifically, the motion control unit 43B causes the character C to perform a motion that reacts to the contact of the user U in the virtual space V (that is, a reaction motion). A typical example of the reaction action is a change in posture, a change in facial expression, or pronunciation of a specific dialogue.

  Specifically, the motion control unit 43B causes the character C to perform a motion according to the position of the contact point P where the reference line Q intersects the character C. For example, when the contact point P is located at the head of the character C, the motion control unit 43B causes the character C to perform a reaction action that accepts the contact by the user U, and the contact point P is the body part of the character C. If the character C is located, the character C is caused to perform an action of refusing contact by the user U. According to the above configuration, the action of the character C can be variously changed according to the position of the contact point P.

  FIG. 15 is a flowchart illustrating the specific contents of processing (hereinafter referred to as “second control processing”) executed by the display control unit 42B and the operation control unit 43B of the second embodiment. The second control process of FIG. 15 is repeatedly executed at a predetermined cycle.

  When the second control process is started, the display control unit 42B determines whether or not the reference line Q intersects the character C in the virtual space V (Sb1). When the reference line Q intersects the character C (Sb1: YES), the display control unit 42B uses the image G2 arranged at the contact point P where the reference line Q and the character C intersect as an instruction image G on the display device 13. Display (Sb2). On the other hand, when the reference line Q does not intersect the character C (Sb1: NO), the display control unit 42B causes the display device 13 to display the image G1 arranged on the reference line Q as the instruction image G (Sb3).

  In a state where the image G2 overlapping the character C is displayed, the motion control unit 43B determines whether or not a motion instruction (hereinafter referred to as “motion command”) for the character C has been received from the user U (Sb4). The action instruction is an instruction for generating an action on the character C in the virtual space V. The action instruction in the second embodiment means an instruction that the user U contacts the character C in the virtual space V.

  The operation control unit 43B of the second embodiment accepts a specific change related to the attitude of the HMD 30 as an operation instruction from the user U. Specifically, the operation control unit 43B determines that the operation instruction has been received when the change in the posture of the HMD 30 satisfies a predetermined condition (hereinafter referred to as “instruction determination condition”). For example, similarly to the first embodiment, the operation control unit 43B determines that an operation instruction is given from the user U when the HMD 30 rotates in the roll direction. That is, the rotation of the HMD 30 in the roll direction is the instruction determination condition. On the other hand, when the change in the attitude of the HMD 30 does not satisfy the instruction determination condition, the operation control unit 43B determines that the operation instruction is not received. As described above with reference to FIG. 7, it is preferable to determine that the HMD 30 has rotated when the rotation angle θ of the HMD 30 exceeds the threshold θt.

  When a motion instruction is received from the user U (Sb4: YES), the motion control unit 43B determines that the user U has touched the character C in the virtual space V, and performs a motion that reacts to the touch by the user U. The character C is executed (Sb5). Specifically, the action control unit 43B causes the character C to perform an action according to the position of the contact point P where the reference line Q intersects among the characters C as described above. As will be understood from the above description, in the second embodiment, the user U gives a motion instruction while moving the reference line Q so as to intersect the desired position of the character C. It is possible to touch a desired position of the character C in the space V.

  In addition, in the structure which recognizes the change of the attitude | position of HMD30 as an operation instruction, since direct operation, such as a touch operation with respect to a display surface, is unnecessary, there exists a problem that the user U cannot feel the sense of contact. In the second embodiment, since the display mode of the instruction image G changes according to the positional relationship between the reference line Q and the character C (specifically, whether the reference line Q intersects the character C), the instruction image Compared with the configuration in which G is displayed in a fixed manner, there is an advantage that the user U can easily feel the sense that the user U has touched the character C in the virtual space V.

  In the second embodiment, for example, the following configuration may be employed.

[Modification B1]
In the second embodiment, the character C is caused to perform a reaction action according to the position of the contact point P. However, similarly to the first embodiment, according to the amount of change in the attitude of the HMD 30 (for example, the rotation angle θ in the roll direction). The reaction action may be executed by the character C. For example, it is assumed that the change amount of the posture of the HMD 30 is a virtual pressure at which the user U contacts the character C in the virtual space V. The motion control unit 43B of the modified example B1 causes the character C to execute a reaction motion that accepts contact by the user U when the amount of change in the posture of the HMD 30 is small, and uses it when the amount of change in posture is large. The character C is caused to perform a reaction action that rejects contact by the person U. According to the above aspect, the character C can be made to perform various reaction actions according to the posture of the HMD 30.

[Modification B2]
The operation control unit 43B of the modification B2 receives an operation instruction from the user U within a specific period on the time axis (hereinafter referred to as “instruction reception period”). That is, the motion instruction given by the user U within the instruction acceptance period is effectively reflected in the motion of the character C, but the motion instruction given by the user U after the instruction acceptance period has been ignored. The instruction reception period is, for example, a period of a predetermined time length (for example, several seconds) starting from the time point when the reference line Q starts to cross the character C. The start point or end point of the instruction reception period is not limited to the above examples. For example, a time point when a specific event starts in the game may be set as the start point of the instruction reception period. As described above, by restricting the reception of the operation instruction to the instruction reception period, it is possible to give a moderate tension to the user U and consequently to prevent the game from being bored.

  The display control unit 42B changes the display mode of the instruction image G (image G2) over time within the instruction reception period. For example, as illustrated in FIG. 16, it is assumed that the image G2 displayed as the instruction image G when the reference line Q intersects the character C includes an image G21 and an image G22. The image G21 is an image representing the hand of the user U like the image G2 of the first embodiment, and the image G22 is a circular image disposed behind the image G21. The display control unit 42B reduces the size of the image G22 over time from the start point to the end point of the instruction reception period. The size of the image G21 does not change. According to the above configuration, there is an advantage that the user U can visually grasp the instruction reception period (for example, remaining time) in which the operation instruction is received from the display mode of the instruction image G.

[Modification B3]
The display control unit 42B of the modification B3 changes the display mode of the instruction image G according to the position of the contact point P where the reference line Q intersects among the characters C in the virtual space V. Specifically, as illustrated in FIG. 17, the instruction image is displayed when the contact point P is in the first position on the surface of the character C and in the second position on the surface of the character C in the virtual space V. The display mode of G is different. The first position is, for example, a position where the character C accepts contact by the user U (for example, the body of the character C), and the second position is, for example, a position where the character C rejects contact by the user U (for example, the character C). C head).

  Although FIG. 17 illustrates the case where the contact point P is at the first position or the second position, the display control unit 42B displays the instruction image G in parallel with the change in the position of the contact point P in the character C. The aspect may be changed over time. That is, when the user U moves the contact point P on the surface of the character C according to the posture of the HMD 30, the display control unit 42B changes the display mode of the instruction image G over time in conjunction with the movement of the contact point P. To change. For example, the display control unit 42B changes the display mode of the instruction image G from the display mode at the first position to the display mode at the second position within the period in which the contact point P moves from the first position to the second position in FIG. Change continuously or step by step.

  In the modification B3, the display mode of the instruction image G changes according to the position of the contact point P. Therefore, by visually recognizing the display mode of the instruction image G, the user U can guess the user U about the action of the character C at the time of contact. That is, by confirming the display mode of the instruction image G, the contact point P is gradually moved while estimating the motion of the character C at the time of contact, and the contact point P is maintained at a position where a desired motion is expected. It is possible to provide the user U with an interest in giving an operation instruction.

[Modification B4]
The display control unit 42B of the modification B4 controls the character C so that the line of sight of the character C tracks the reference line Q. For example, the character C is controlled such that the line of sight faces the contact point P where the reference line Q intersects the character C. Specifically, as illustrated in FIG. 18, the eyeball (for example, the pupil) and the head of the character C rotate so as to track the reference line Q. Only one of the eyeball and the head of the character C may track the reference line Q. According to the modified example B4, there is an advantage that the user U can easily grasp the sense that the user U is affecting the character C.

[Modification B5]
In the second embodiment, the display mode of the instruction image G is changed depending on whether or not the reference line Q intersects the character C. However, the conditions for changing the display mode of the instruction image G are as described above. It is not limited. For example, the display control unit 42B may change the display mode of the instruction image G continuously or stepwise according to the distance between the reference line Q and the character C. The intersection of the reference line Q and the character C is not an essential condition for changing the display mode of the instruction image G. Further, the display mode of the instruction image G may be changed depending on whether or not the reference line Q intersects a specific part of the character C. In a state where the reference line Q intersects other than a specific part of the character C, the instruction image G is displayed in the same display manner as when the reference line Q does not intersect with the character C. As understood from the above description, the motion control unit 43B of the second embodiment is comprehensively expressed as an element that changes the display mode of the instruction image G in accordance with the positional relationship between the reference line Q and the character C. .

[Modification B6]
The configuration of the second embodiment in which the display mode of the instruction image G is changed according to the positional relationship between the reference line Q and the character C is used while the user U is holding the terminal device 10 (for example, a smartphone) by hand. The same applies to the case. Similarly to the second embodiment, the imaging range R displayed on the display device 13 in the virtual space V changes according to the attitude of the terminal device 10.

  The terminal device 10 of the modified example B6 includes a touch panel that detects the contact of the user U with the display surface of the display device 13. In 2nd Embodiment, although the position of the contact point P in the virtual space V was changed according to the attitude | position of HMD30, in modification B6, the point where the user U contacted the display surface of the display apparatus 13 is shown. An instruction image G is displayed as the contact point P. For example, a reference line Q that passes through a contact point P with which the user U has touched is set in the virtual space V. In other words, a virtual straight line in the direction indicated by the user U by contact with the display surface of the display device 13 (that is, touch operation) is set in the virtual space V as the reference line Q. The display control unit 42B displays the image G2 on the display device 13 as the instruction image G when the reference line Q intersects the character C, and displays the image G1 as the instruction image G when the reference line Q does not intersect the character C. Is displayed on the display device 13. Even with the above configuration, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V (for example, the state where the user U is in contact with the character C in the virtual space V). There are advantages.

  As understood from the illustration of the modification B6, in the second embodiment, the configuration in which the display device 13 is mounted on the head of the user U can be omitted. The display control unit 42B in a configuration that is not assumed to be worn on the head of the user U displays an image obtained by capturing the virtual space V with the virtual camera E whose posture is controlled according to the posture of the display device 13. 13 is expressed as an element to be displayed. In the display device 13 that is used while being held by the user U, a virtual straight line in a direction instructed by an operation (for example, a touch operation) on the display device 13 is a “reference line”.

[C: Other variations]
Specific modifications to the first embodiment or the second embodiment will be exemplified below. Two or more aspects arbitrarily selected from the following examples may be appropriately combined as long as they do not contradict each other.

[Modification C1]
The reference line Q is not limited to the optical axis of the virtual camera E exemplified in each of the above embodiments. For example, a straight line perpendicular to the display surface of the display device 13 is used as the reference line Q. A straight line that forms a predetermined angle with respect to a straight line perpendicular to the optical axis of the virtual camera E or the display surface may be used as the reference line Q. Note that in the HMD 30 equipped with an eye tracking (line-of-sight measurement) function for estimating the direction of the line of sight of the user U, the line of sight estimated by the function may be used as the reference line Q. As understood from the above examples, the reference line Q is comprehensively expressed as a virtual straight line set in the virtual space V.

[Modification C2]
In each of the above-described embodiments, the character C is exemplified as the object arranged in the virtual space V, but the object that the user U contacts in the virtual space V is not limited to the character C. For example, inanimate elements such as buildings and natural objects in the virtual space V are also included in the concept of objects. For the biological object (character C), “motion” is, for example, the behavior, action, state, or attitude of the object. For an inanimate object, “motion” is, for example, a change in the form of the element. For example, an operation of opening a door of a building or an operation of deforming or moving a natural object such as a rock is exemplified as an operation of an object.

[Modification C3]
In each of the above-described embodiments, when the HMD 30 rotates in the roll direction, it is determined that the user U has touched the character C in the virtual space V and the character C performs a reaction action. That is, the operation instruction) is not limited to the above examples. For example, the character C may be caused to perform a reaction action with the rotation of the HMD 30 in the pitch direction or the yaw direction as contact by the user U. Further, for example, the reaction to the character C is performed on the condition that the state in which the reference line Q intersects the specific range of the character C (that is, when the user U stares in the range for a long time) continues for a predetermined time. An operation may be executed. The character C may be caused to perform a reaction action on condition that the user U operates an operating device (not shown) connected to the HMD 30. As understood from the above examples, a configuration in which a change in the posture of the HMD 30 (particularly rotation in the roll direction) is triggered by the reaction action of the character C may be omitted.

[Modification C4]
In each of the above-described embodiments, the rotation of the HMD 30 in the roll direction is accepted as an operation instruction (instruction for contact with the character C), but the change in the posture of the HMD 30 determined as the operation instruction is not limited to the rotation in the roll direction. For example, the rotation of the HMD 30 in the pitch direction or the yaw direction may be accepted as an operation instruction.

[Modification C5]
The modified examples (modified examples A1 to A7) exemplified for the first embodiment are similarly applied to the second embodiment. Moreover, the modification (Modification B1 to Modification B6) illustrated about 2nd Embodiment is applied similarly to 1st Embodiment.

[Modification C6]
In each of the above-described embodiments, the configuration in which the display device 13 and the detection device 14 of the HMD 30 and the information processing device 40 that controls the HMD 30 are realized by the single terminal device 10 is illustrated. However, as in the image display system 1B of FIG. The HMD 30 and the information processing device 40 may be realized as separate devices. The HMD 30 and the information processing apparatus 40 can communicate with each other by wire or wireless. The communication method between the HMD 30 and the information processing apparatus 40 is arbitrary, but short-range wireless communication such as bluetooth (registered trademark) is suitable.

  Similar to the first embodiment, the HMD 30 includes the display device 13, the detection device 14, and the mounting tool 20, and is mounted on the user U's head. The information processing device 40 is a control device that displays various images on the HMD 30 by communicating with the HMD 30, and includes a control device 11 and a storage device 12. For example, various information terminals such as a mobile phone, a smartphone, a tablet terminal, a personal computer, or a home game device are used as the information processing device 40. It does not matter whether the information processing apparatus 40 is portable or stationary. The control device 11 executes the program stored in the storage device 12 to perform the posture analysis unit 41, the display control unit 42 (42A, 42B), and the operation control unit in the same manner as in the first embodiment or the second embodiment. 43 (43A, 43B). Accordingly, the same effects as those of the first embodiment or the second embodiment are realized in the configuration of FIG.

[Modification C7]
A preferable aspect of the present invention is realized by the cooperation of a computer (specifically, the control device 11) and a program as illustrated in the above-described embodiments. The programs according to the above-described embodiments are provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium Including a recording medium of the form. Note that the non-transitory recording medium includes any recording medium except for a transient propagation signal (transitory, propagating signal), and does not exclude a volatile recording medium. It is also possible to provide a program to a computer in the form of distribution via a communication network.

[D: Appendix]
From the above description, for example, a preferred aspect of the present invention is grasped as follows. In addition, in order to facilitate understanding of each aspect, the following description is not intended to limit the present invention to the illustrated aspect.

[Aspect 1]
A program according to a preferred aspect (aspect 1) of the present invention is an image obtained by capturing a virtual space (V) with a virtual camera (E) whose direction is controlled according to the direction of the head mounted display (30), A display control unit (42A) for displaying a stereoscopic image using binocular parallax on the display device (13) of the head mounted display (30), and rotation of the head mounted display (30) in the roll direction; According to the positional relationship between the reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head mounted display (30) and the object (C) in the virtual space (V). The computer (11) is caused to function as the motion control unit (43A) that controls the motion of the object (C). In the above aspect, since the operation of the object (C) in the virtual space (V) is controlled according to the rotation of the head mounted display (30) in the roll direction, an instruction for controlling the operation of the object (C) The instruction from the user (U) is reflected on the operation of the object (C) without preparing an operating device for the user (U) to input separately from the head mounted display (30). Can do.

  The “head mounted display” is an image display device that can be mounted on the head of the user (U). The head mounted display (30) in a preferred embodiment includes a display device (13) for displaying an image, and a mounting tool (20) for mounting the display device (13) on the head of the user (U). To do. In addition to the configuration in which the dedicated display device (13) is fixedly installed on the mounting device (20) for mounting the display device (13) on the head (so-called head-mounted display), it is mounted on the head. The structure which can be attached or detached with respect to a mounting tool (20) also includes the general purpose display apparatus (13) which can be utilized also in the state except having been made. That is, the display device (13) of the head-mounted display (30) includes not only a device that is used while being mounted on the head, but also a state that is not mounted on the head and a state that is mounted on the head. The apparatus which can be used by both is also included. In the configuration in which the display device (13) is detachable from the wearing tool (20), for example, a portable terminal device (10) such as a smartphone or a tablet terminal is used as the display device (13).

  “Roll direction” means a circumferential direction centered on an axis in the front-rear direction of the user (U) (for example, the direction perpendicular to the display screen on which an image is displayed on the head mounted display (30)). For example, when a user (U) wearing the head mounted display (30) on the head tilts the head left and right while facing forward, the head mounted display (30) rotates in the roll direction. The head mounted display (30) does not move in the roll direction not only when the rotation in the roll direction alone occurs in the head mounted display (30) but also in the pitch direction or the yaw direction with the rotation in the roll direction. It can be determined that it has rotated. It does not matter whether the posture of the virtual camera (E) is changed according to the rotation in the roll direction. That is, both the configuration in which the virtual camera (E) is rotated around the optical axis in conjunction with the rotation in the roll direction and the configuration in which the virtual camera (E) is not rotated even when the rotation in the roll direction occurs are described in the first aspect. It is included in the range.

  An “object” is a virtual object installed in the virtual space (V). A typical example of the object (C) is a character (for example, a human being, an animal or a monster), but an inanimate element such as a building or a natural object in the virtual space (V) may be included in the concept of the object. The “object motion” for a character is, for example, the behavior, action, appearance, or attitude of the character. For an inanimate element, the “object motion” is, for example, a change in the form of the element (for example, a door or window of a building is opened, a natural object such as a rock is deformed or moved).

  The “reference line” is a virtual straight line set in the virtual space (V). For example, the optical axis of the virtual camera (E), the center line of the display screen by the display device (13), or a straight line forming a predetermined angle with respect to these straight lines is a preferred example of the reference line (Q). In addition, in the structure which can utilize the eye tracking function which estimates a user's (U) eyes | visual_axis, you may utilize a user's (U) eyes | visual_axis as a reference line (Q) other than the straight line illustrated above.

[Aspect 2]
In a preferred example of aspect 1 (aspect 2), the positional relationship is a relationship in which the reference line (Q) and the object (C) intersect. According to the above aspect, the instruction according to the intersection of the reference line (Q) and the object (C) can be reflected in the operation of the object (C).

[Aspect 3]
In a preferred example (Aspect 3) of Aspect 1 or Aspect 2, when the head mounted display (30) rotates in the roll direction, the motion control unit (43A) causes the object ( C), it is determined that the point (P) where the reference line (Q) intersects, and the object (C) is caused to perform an action corresponding to the contact. According to the above aspect, the user (U) rotates the head mounted display (30) in the roll direction, thereby making contact with the point (P) that intersects the reference line (Q) in the object (C). Can be directed.

[Aspect 4]
In a preferred example (aspect 4) of aspect 3, the motion control unit (43A) causes the object (C) to perform an action according to the length of time during which contact with the object (C) is maintained. According to the above aspect, the operation of the object (C) can be variously changed according to the length of time that the head mounted display (30) is maintained in the state of rotating in the roll direction.

[Aspect 5]
In a preferred example (Aspect 5) of Aspect 3 or Aspect 4, the motion control unit (43A) changes a parameter related to the object (C) in response to contact with the object (C). According to the above aspect, the parameter regarding the object (C) in the virtual space (V) can be changed by a simple operation of rotating the head mounted display (30) in the roll direction.

[Aspect 6]
In a preferred example (Aspect 6) of any one of Aspects 3 to 5, the motion control unit (43A) performs an operation according to a position of the object (C) at which the reference line (Q) intersects. The object (C) is executed. According to the above aspect, the operation of the object (C) can be variously changed according to the position of the point (P) where the reference line (Q) in the virtual space (V) intersects the object (C). .

[Aspect 7]
In a preferred example (aspect 7) of any one of Aspects 3 to 6, the operation control unit (43A) is configured such that when the head mounted display (30) rotates to one side in the roll direction, the object (C) When the head mounted display (30) is rotated to the other side in the roll direction in the state of being in contact with the object (C), or from the rotation of the head mounted display to the one side When returning to the original state, it is determined that the contact with the object (C) has been released. According to the above aspect, the user (U) can instruct generation | occurrence | production and cancellation | release of the contact with respect to the object (C) in virtual space (V) according to the direction of rotation in a roll direction.

[Aspect 8]
In a preferred example (aspect 8) of any one of Aspects 1 to 7, the operation control unit (43A) performs an operation according to a rotation angle (θ) in the roll direction of the head mounted display (30) on the object. (C). According to the above aspect, it is possible to cause the object (C) to execute various operations according to the rotation angle (θ) of the head mounted display (30).

[Aspect 9]
In a preferred example (aspect 9) of any one of the aspects 1 to 8, the display control unit (42A) moves the virtual camera (E) to the virtual camera (E) according to the rotation of the head mounted display (30) in the roll direction. Move in the virtual space (V). According to the above aspect, the virtual camera (E) (virtual viewpoint) approaches the object (C) in the virtual space (V) by a simple operation of rotating the head mounted display (30) in the roll direction. Can be separated. The direction of movement of the virtual camera (E) is arbitrary, but for example, a configuration in which the virtual camera (E) is moved in the direction of the object (C) is suitable. For example, when the head mounted display (30) is rotated to one side in the roll direction, the virtual camera (E) is moved closer to the object (C), and the head mounted display (30) is rotated to the other side in the roll direction. A configuration in which the virtual camera (E) is separated from the object (C) is preferable.

[Aspect 10]
In a preferred example (aspect 10) of any one of Aspects 1 to 9, the display control unit (42A) captures images by the virtual camera (E) according to the rotation of the head mounted display (30) in the roll direction. The range (R) is changed. According to the above aspect, the range (that is, the angle of view) captured by the virtual camera (E) in the virtual space (V) can be changed by the simple operation of rotating the head mounted display (30) in the roll direction. it can. Note that the change in the imaging range (R) of the virtual camera (E) means enlargement (zoom in) or reduction (zoom out) of an element (for example, the object (C)) to be imaged in the virtual space (V).

[Aspect 11]
In a preferred example (aspect 11) of any one of the aspects 1 to 10, the operation control unit (43A) has a rotation angle (θ) in the roll direction of the head mounted display (30) exceeding a threshold value (θt). In this case, it is determined that the head mounted display (30) has rotated in the roll direction. In the above aspect, when the rotation angle (θ) exceeds the threshold value (θt), it is determined that the head mounted display (30) has rotated in the roll direction. That is, it is not determined that the head-mounted display (30) has rotated in the roll direction when the rotation is less than the threshold value (θt). Therefore, it is possible to reduce the possibility that the head mounted display (30) is determined with an excessive frequency that the head mounted display (30) is rotated.

[Aspect 12]
An image display system (1A, 1B) according to a preferred aspect (aspect 12) of the present invention is an image display system (1A, 1B) comprising a head-mounted display (30) and an information processing device (40). The information processing device (40) is an image obtained by capturing the virtual space (V) with a virtual camera (E) whose direction is controlled according to the direction of the head mounted display (30), and uses binocular parallax. The display control unit (42A) that displays the stereoscopic image thus displayed on the display device (13) of the head mounted display (30), the rotation of the head mounted display (30) in the roll direction, and the head mounted display (30 ) And the positional relationship between the reference line (Q) whose direction in the virtual space (V) changes according to the direction of the object (C) in the virtual space (V), Depending includes an operation control unit for controlling the operation of the object (C) (43A) with. In the above aspect, since the operation of the object (C) in the virtual space (V) is controlled according to the rotation of the head mounted display (30) in the roll direction, an instruction for controlling the operation of the object (C) The instruction from the user (U) is reflected on the operation of the object (C) without preparing an operating device for the user (U) to input separately from the head mounted display (30). Can do.

DESCRIPTION OF SYMBOLS 1A, 1B ... Image display system, 10 ... Terminal device, 11 ... Control device, 12 ... Storage device, 13 ... Display device, 14 ... Detection device, 20 ... Wearing tool, 30 ... HMD, 40 ... Information processing device, 41 ... Posture analysis unit, 42A, 42B ... display control unit, 43A, 43B ... motion control unit, V ... virtual space, E ... virtual camera, C ... character, R ... imaging range, Q ... reference line, G ... instruction image, P ... the point of contact.

Claims (12)

Display control unit for displaying a stereoscopic image using binocular parallax on a display device of the head mounted display, which is an image obtained by capturing a virtual space with a virtual camera whose direction is controlled according to the direction of the head mounted display ,and,
The movement of the head-mounted display according to the rotation of the head-mounted display and the positional relationship between a reference line whose direction in the virtual space changes according to the direction of the head-mounted display and the object in the virtual space. A program that causes a computer to function as an operation control unit that controls the computer. The program according to claim 1, wherein the positional relationship is a relationship in which the reference line and the object intersect. When the head mounted display rotates in the roll direction, the motion control unit determines that the object touches a point in the virtual space where the reference line intersects, and performs a motion corresponding to the touch. The program according to claim 1 or 2, wherein the program is executed by an object. The program according to claim 3, wherein the motion control unit causes the object to perform a motion corresponding to a length of time during which contact with the object is maintained. The program according to claim 3 or 4, wherein the motion control unit changes a parameter related to the object in accordance with contact with the object. The program according to any one of claims 3 to 5, wherein the motion control unit causes the object to perform a motion according to a position of the object at which the reference line intersects. The operation control unit determines that the head-mounted display is in contact with the object when the head-mounted display is rotated in one direction in the roll direction. The program according to any one of claims 3 to 6, wherein it is determined that contact with the object is released when the head-mounted display is rotated or when the head-mounted display is restored from the rotation to the one side. The program according to any one of claims 1 to 7, wherein the operation control unit causes the object to perform an operation according to a rotation angle in a roll direction of the head mounted display. The program according to any one of claims 1 to 8, wherein the display control unit moves the virtual camera in the virtual space in accordance with rotation of the head mounted display in a roll direction. The program according to any one of claims 1 to 9, wherein the display control unit changes an imaging range of the virtual camera in accordance with rotation of the head mounted display in a roll direction. The program according to any one of claims 1 to 10, wherein the operation control unit determines that the head mounted display has rotated in the roll direction when a rotation angle of the head mounted display in the roll direction exceeds a threshold value. An image display system comprising a head mounted display and an information processing device,
The information processing apparatus includes:
Display control unit for displaying a stereoscopic image using binocular parallax on a display device of the head mounted display, which is an image obtained by capturing a virtual space with a virtual camera whose direction is controlled according to the direction of the head mounted display When,
The movement of the head-mounted display according to the rotation of the head-mounted display and the positional relationship between a reference line whose direction in the virtual space changes according to the direction of the head-mounted display and the object in the virtual space. An image display system comprising: an operation control unit that controls the operation.

Images