JP6628331B2 - Program and image display system - Google Patents

Description

  The present invention relates to a technique for displaying an image of a virtual space.

  2. Description of the Related Art A technique for displaying an image captured by a virtual camera (hereinafter, referred to as a “virtual camera”) installed in a virtual space on a display device has been conventionally proposed. For example, Patent Document 1 discloses a configuration in which a menu image is displayed in a virtual space according to the inclination of a head mounted display.

JP-A-2006-115122

  2. Description of the Related Art Content such as games that communicate with objects such as characters in a virtual space has been conventionally proposed. In a scene in which this type of content is displayed on a head-mounted display, a virtual straight line (hereinafter, referred to as a “reference line”) such as an optical axis of a virtual camera or a line of sight of a user and an object such as a character in a virtual space. There is a problem that it is difficult for a user to grasp the positional relationship. In view of the above circumstances, an object of the present invention is to make it easy for a user to grasp the positional relationship between an object in a virtual space and a reference line.

  In order to solve the above problems, a program according to a preferred aspect of the present invention is an image obtained by capturing a virtual space with a virtual camera whose direction is controlled in accordance with the direction of a head-mounted display, and the binocular parallax is reduced. The used stereoscopic image, a display control unit that displays on the display device of the head mounted display, a program that causes a computer to function, the display control unit in the virtual space according to the direction of the head mounted display The pointing image is displayed in the direction of the reference line whose direction changes, and the display mode of the pointing image is changed according to the positional relationship between the reference line and an object in the virtual space.

  An image display system according to a preferred aspect of the present invention is an image display system including a head-mounted display and an information processing device, wherein the direction of the information processing device is controlled according to the direction of the head-mounted display. An image captured in a virtual space by a virtual camera, a display control unit that displays a stereoscopic image using binocular parallax on a display device of the head-mounted display, the display control unit includes: The pointing image is displayed in the direction of the reference line whose direction in the virtual space changes according to the direction of the display, and the display mode of the pointing image changes according to the positional relationship between the reference line and the object in the virtual space. Let it.

FIG. 1 is a perspective view illustrating an appearance of an image display system according to a first embodiment of the present invention. FIG. 1 is a block diagram illustrating a configuration of an image display system. FIG. 2 is a block diagram illustrating a functional configuration of the image display system. FIG. 3 is an explanatory diagram of a virtual space. FIG. 4 is an explanatory diagram of an image displayed by the display device. 5 is a flowchart illustrating a content of a process executed by the control device. It is explanatory drawing of rotation of HMD in a roll direction. FIG. 4 is an explanatory diagram of an image displayed by the display device. FIG. 15 is an explanatory diagram of movement of a virtual camera in a modification example A4. FIG. 14 is an explanatory diagram of a change in an imaging range in Modification Example A5. It is a block diagram which illustrates the functional composition of the image display system in a 2nd embodiment. FIG. 14 is an explanatory diagram of an image displayed when a reference line does not intersect a character in the second embodiment. FIG. 14 is an explanatory diagram of an image displayed when a reference line crosses a character in the second embodiment. FIG. 9 is an explanatory diagram of a relationship between a character and an instruction image in a second embodiment. It is a flowchart which illustrates the content of the processing which the control device of 2nd Embodiment performs. FIG. 15 is an explanatory diagram of a change over time of an instruction image in a modified example B2. FIG. 15 is an explanatory diagram of an instruction image in a modified example B3. FIG. 15 is an explanatory diagram of an instruction image in a modified example B4. FIG. 15 is a perspective view illustrating an appearance of an image display system according to a modification example C5.

[A: First Embodiment]
FIG. 1 is a perspective view illustrating the appearance of an image display system 1A according to the first embodiment of the present invention. The image display system 1A is a video device for displaying an image, and includes a terminal device 10 and a mounting device 20. The terminal device 10 is a portable information terminal such as a smartphone or a tablet terminal. The terminal device 10 is installed detachably with respect to the attachment 20. The mounting device 20 is a device for mounting the terminal device 10 on the head of the user U. For example, as illustrated in FIG. 1, a goggle-type attachment including a belt wound around the head of the user U is suitable as the wearing tool 20.

  FIG. 2 is a block diagram illustrating the configuration of the terminal device 10. As illustrated in FIG. 2, the terminal device 10 is a computer system including a control device 11, a storage device 12, a display device 13, and a detection device 14. The control device 11 is a processor such as a CPU (Central Processing Unit), and controls each element of the image display system 1A. The storage device 12 stores a program executed by the control device 11 and various data used by the control device 11. For example, the storage device 12 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of types of recording media.

  The display device 13 displays an image under the control of the control device 11. For example, a flat display such as a liquid crystal display panel or an organic EL (Electroluminescence) display panel is used as the display device 13. As can be understood from FIG. 1, in a state where the wearing tool 20 is mounted on the head of the user U, a display device is provided in front of both eyes of the user U with the display surface facing the head of the user U. 13 are arranged. The display device 13 of the first embodiment displays a stereoscopic image in which the user U can perceive a stereoscopic effect. The stereoscopic image is composed of a right-eye image and a left-eye image to which binocular parallax is added. By allowing the right-eye image to be visually recognized by the right eye of the user U and the left-eye image to be visually recognized by the left eye of the user U, the user U perceives a three-dimensional effect.

  As illustrated in FIG. 2, the display device 13, the detection device 14, and the attachment 20 constitute a head-mounted display (hereinafter, referred to as “HMD”) 30, and the control device 11 and the storage device 12 form an image on the HMD 30. The information processing device 40 to be displayed is configured. That is, the image display system 1A includes the HMD 30 and the information processing device 40. As described above, in the first embodiment, the information processing device 40, the display device 13, and the detection device 14 are realized by the single terminal device 10. Note that the information processing device 40 may be grasped as an element of the HMD 30.

  The detection device 14 in FIG. 2 is a sensor that outputs a detection signal according to the attitude of the HMD 30. Specifically, the detection device 14 is selected from a plurality of types of sensors such as a gyro sensor for detecting angular velocity, an acceleration sensor for detecting acceleration, an inclination sensor for detecting an inclination angle, and a geomagnetic sensor for detecting a direction by geomagnetism. It consists of one or more types of sensors. By analyzing the detection signal output from the detection device 14, the posture of the HMD 30 or a change in the posture can be specified. The detection device 14 is also referred to as a sensor that outputs a detection signal according to the attitude of the display device 13 or the terminal device 10. The illustration of an amplifier for amplifying the detection signal and an A / D converter for converting the detection signal from analog to digital is omitted for convenience.

  As illustrated in FIG. 1, the posture of the HMD 30 is defined by three axes (X axis, Y axis, and Z axis) orthogonal to each other at the origin O. The X axis is an axis perpendicular to the display surface of the display device 13 and corresponds to the front-back direction of the user U. The Y axis and the Z axis are axes parallel to the display surface of the display device 13. The Y axis corresponds to the horizontal direction of the user U, and the Z axis corresponds to the vertical direction of the user U. Focusing on the display surface of the display device 13, the X axis corresponds to the depth direction of the display surface, the Y axis corresponds to the horizontal direction of the display surface, and the Z direction corresponds to the vertical direction of the display surface. The circumferential direction around the X axis is the roll direction, the circumferential direction around the Y axis is the pitch direction, and the circumferential direction around the Z axis is the yaw direction. The direction of the X axis is determined by the angle in the pitch direction and the angle in the yaw direction. The rotation in the roll direction corresponds to an operation of tilting the head to the left and right with the user U facing forward. In other words, for example, when the user U bows, the HMD 30 rotates in the roll direction. In the following description, as illustrated in FIG. 1, one side in the roll direction is referred to as “first side”, and the other side is referred to as “second side”. One of the clockwise and counterclockwise directions about the X axis is the first side, and the other is the second side.

  The control device 11 of FIG. 2 executes a program stored in the storage device 12 to function as the posture analysis unit 41, the display control unit 42A, and the operation control unit 43A, as illustrated in FIG. Note that some or all of the functions of the control device 11 may be realized by a dedicated electronic circuit.

  The posture analysis unit 41 specifies the posture of the HMD 30 by analyzing the detection signal output by the detection device 14. Specifically, the posture analysis unit 41 sequentially generates posture data on the posture of the HMD 30 at a predetermined cycle. The posture data is data representing the posture of the HMD 30 or a change in the posture.

  The display control unit 42A causes the display device 13 to display an image captured by the virtual camera E in the virtual space V. FIG. 4 is an explanatory diagram of the virtual space V. As illustrated in FIG. 4, the virtual camera E is installed in the virtual space V and captures an image of a specific range (hereinafter, referred to as an “imaging range”) R in the virtual space V. The imaging range R is a range that covers a predetermined angle in the vertical and horizontal directions with respect to the optical axis of the virtual camera E. The optical axis of the virtual camera E corresponds to a virtual line of sight of the user U in the virtual space V. The display control unit 42A of the first embodiment causes the display device 13 to display a stereoscopic image (a right-eye image and a left-eye image) using binocular parallax. Therefore, the virtual camera E is configured to include the first virtual camera that captures the image for the left eye and the second virtual camera that captures the image for the right eye. The image data representing the stereoscopic image captured by the virtual camera E is sequentially supplied from the display control unit 42A to the display device 13, so that the display device 13 displays the stereoscopic image.

  The display control unit 42A controls the direction of the virtual camera E in the virtual space V according to the direction of the HMD 30 specified by the posture analysis unit 41. Specifically, the direction of the optical axis of the virtual camera E (and the imaging range R) changes in conjunction with the rotation of the X axis about the origin O. For example, when the HMD 30 rotates in the yaw direction, the optical axis of the virtual camera E rotates left and right around the origin O, and when the HMD 30 rotates in the pitch direction, the optical axis of the virtual camera E changes to the origin O. Rotate up and down around. In the first embodiment, the attitude (angle about the optical axis) of the virtual camera E does not change even when the HMD 30 rotates in the roll direction. However, the inclination of the virtual camera E about the optical axis may be changed according to the rotation of the HMD 30 in the roll direction.

  As illustrated in FIG. 4, a character C is arranged in a virtual space V in which a virtual camera E is installed. The character C of the first embodiment is an object representing a virtual creature (for example, a human, an animal, or a monster) active in the virtual space V, and includes a head and a body. In the virtual space V, a game in which the character C is exchanged with the character C while being in contact (skinship) with the character C at any time is provided to the user U by the image display system 1A.

  A reference line Q is set in the virtual space V. The reference line Q is a straight line whose direction in the virtual space V changes according to the direction of the HMD 30 (the direction of the X axis). In the first embodiment, as illustrated in FIG. 4, the optical axis of the virtual camera E is illustrated as the reference line Q. For example, if the user U turns his head to the right, the reference line Q rotates to the right, and if the user U turns his head to the left, the reference line Q rotates to the left. As understood from the above description, the reference line Q is a virtual line of sight of the user U in the virtual space V.

  FIG. 5 is a schematic diagram of an image displayed on the display device 13. As illustrated in FIG. 5, the display control unit 42A causes the display device 13 to display a stereoscopic image captured by the virtual camera E in the virtual space V. When the character C is located within the imaging range R, the character C is displayed on the display device 13. Further, the display control unit 42A causes the display device 13 to display an image (hereinafter, referred to as an “instruction image”) G indicating the direction of the reference line Q. The instruction image G is arranged at a predetermined position on the reference line Q in the virtual space V. Since the reference line Q in the first embodiment is the optical axis of the virtual camera E, the pointing image G is displayed at a predetermined position on the display surface of the display device 13.

  The motion control unit 43A of FIG. 3 controls the motion of the character C in the virtual space V. The motion control unit 43A controls the motion of the character C in accordance with the progress of the game, and also causes the character C to execute an operation corresponding to the contact of the user U in the virtual space V (hereinafter, referred to as a “reaction operation”). Specifically, the motion control unit 43A determines whether or not the user U has touched the character C in the virtual space V, and causes the character C to perform a reaction motion in response to the touch by the user U. For example, various actions such as a change in posture, a change in facial expression, or the pronunciation of a specific line are typical examples of the reaction action.

  The motion control unit 43A determines whether the user U has touched the character C in the virtual space V according to the change in the posture of the HMD 30 specified by the posture analysis unit 41. The motion control unit 43A of the first embodiment determines that the user U has touched the character C in the virtual space V when the HMD 30 rotates in the roll direction. That is, the rotation of the HMD 30 in the roll direction corresponds to an instruction (operation instruction) for the user U to contact the character C in the virtual space V. Specifically, when the HMD 30 rotates in the roll direction, the motion control unit 43A contacts the point P where the reference line Q and the surface of the character C intersect (hereinafter, referred to as “contact point”). Is determined. The contact point P is located on the surface of the character C. When the reference line Q intersects the surface of the character C at a plurality of points, a point closest to the virtual camera E (that is, the user U) among the plurality of points is selected as the contact point P.

  As described above, in the first embodiment, the motion of the character C in the virtual space V is controlled according to the rotation of the HMD 30 in the roll direction. Therefore, the operation from the user U is reflected in the operation of the character C without the necessity of preparing an operating device for the user U to input an instruction for controlling the operation of the character C separately from the HMD 30. be able to. In the first embodiment, in particular, when the reference line Q and the character C intersect, it is determined that the user U has come into contact with the character C. It is possible to operate the character C. That is, there is an advantage that the user U can operate the character C more intuitively.

  FIG. 6 is a flowchart illustrating the specific contents of the processing (hereinafter, referred to as “first control processing”) executed by the operation control unit 43A. The operation control unit 43A repeatedly executes the first control process of FIG. 6 at a predetermined cycle. The storage device 12 stores contact determination data indicating whether or not the user U is in contact with the character C in the virtual space V (hereinafter, referred to as “contact state”). The contact determination data is, for example, a flag indicating one of a contact state and a non-contact state.

  When the first control process is started, the operation control unit 43A determines whether or not the contact determination data indicates a contact state (Sa1). When the contact determination data indicates a non-contact state (Sa1: NO), the operation control unit 43A determines whether the HMD 30 has rotated in the roll direction (Sa2). Specifically, the operation control unit 43A determines whether the HMD 30 has rotated to the first side (for example, clockwise) in the roll direction by referring to the posture data sequentially supplied from the posture analysis unit 41. I do.

  FIG. 7 is an explanatory diagram of a process of determining whether the HMD 30 has rotated in the roll direction. As illustrated in FIG. 7, the operation control unit 43A determines that the HMD 30 has rotated in the roll direction when the angle θ at which the HMD 30 has rotated to the first side in the roll direction exceeds a threshold value θt. Is a rotation angle of the Z axis or the Y axis about the X axis, and is defined, for example, with the vertical direction as a reference (θ = 0). On the other hand, when the rotation angle θ of the HMD 30 is smaller than the threshold value θt, the operation control unit 43A does not determine that the HMD 30 has rotated in the roll direction. As described above, in the first embodiment, it is not determined that the HMD 30 has rotated in the roll direction if the rotation is less than the threshold value θt. Therefore, it is possible to reduce the possibility that the HMD 30 is determined to rotate at an excessive frequency. Further, there is an advantage that the possibility that the user U does not intend to rotate the HMD 30 is determined to have rotated the HMD 30 (that is, the possibility of erroneous operation) can be reduced.

  In addition, not only when the rotation in the roll direction alone occurs on the HMD 30, but also when the rotation in the pitch direction or the yaw direction occurs together with the rotation in the roll direction, when the rotation angle in the roll direction exceeds the threshold value θt, Is determined that the HMD 30 has rotated in the roll direction.

  In the first control process of FIG. 6, when it is determined that the HMD 30 has rotated to the first side in the roll direction (Sa2: YES), the motion control unit 43A determines whether the reference line Q intersects the character C in the virtual space V. It is determined whether or not it is (Sa3). That is, it is determined whether or not the character C exists on the reference line Q. As illustrated in FIG. 8, when the reference line Q intersects the character C in the virtual space V (Sa3: YES), the operation control unit 43A sets the contact state where the user U has contacted the contact point P of the character C. It is determined that there is (Sa4). Specifically, the contact determination data is changed to a numerical value indicating the contact state. As understood from the above description, the operation control unit 43A determines that the user U has touched the contact point P of the character C in the virtual space V when the HMD 30 rotates in the roll direction. That is, the user U can instruct the character C to touch the contact point P by rotating the HMD 30 in the roll direction.

  When the state has transitioned to the contact state, the movement control unit 43A causes the character C to execute a reaction movement (Sa5). Specifically, the motion control unit 43A causes the character C to perform a reaction motion corresponding to the position of the contact point P where the reference line Q intersects among the characters C. That is, the reaction performed by the character C changes according to the position of the contact point P. For example, a table that defines the type of reaction action for each of a plurality of areas that divide the surface of the character C is stored in the storage device 12. The motion control unit 43A causes the character C to execute a reaction motion specified in the table for an area including the contact point P among a plurality of areas on the surface of the character C. For example, when the contact point P is located on the head of the character C, the operation control unit 43A performs a reaction operation (for example, a favorable operation such as rejoicing, laughing, or approaching) to accept the contact by the user U on the character C. Let C run. On the other hand, when the contact point P is located on the body of the character C, the operation control unit 43A performs a reaction operation (for example, a negative operation such as getting angry, sad, or moving away) to refuse contact by the user U. Cause character C to execute. According to the above configuration, it is possible to variously change the reaction motion of the character C according to the position of the contact point P.

  On the other hand, when the HMD 30 does not rotate to the first side in the roll direction (Sa2: NO), or when the reference line Q does not intersect with the character C (Sa3: NO), the motion control unit 43A sets the numerical value indicating the non-contact state. The contact determination data is maintained. That is, even when the HMD 30 rotates to the first side in the roll direction, when the reference line Q does not intersect the character C as illustrated in FIG. 5, it is determined that the user U is not in contact with the character C.

  When the user U contacts the character C in the virtual space V, it is determined that the contact determination data indicates a contact state in the subsequent step Sa1 of the first control process. When the contact determination data indicates a contact state (Sa1: YES), the operation control unit 43A refers to the posture data sequentially supplied from the posture analysis unit 41, and causes the HMD 30 to move the HMD 30 on the second side in the roll direction (for example, (Clockwise) is determined. Specifically, when the rotation angle θ with respect to the second side in the roll direction exceeds a predetermined threshold value θt, the operation control unit 43A determines that the HMD 30 has rotated to the second side in the roll direction. When the HMD 30 has rotated to the second side in the roll direction (Sa6: YES), the operation control unit 43A determines that the contact with the character C has been released (Sa7). Specifically, the operation control unit 43A changes the contact determination data to a numerical value indicating the non-contact state. On the other hand, when the HMD 30 does not rotate to the second side in the roll direction (Sa6: NO), the operation control unit 43A maintains the contact determination data at a numerical value indicating the contact state.

  It is preferable that, when the contact with the character C is released, the operation control unit 43A causes the character C to execute an operation responsive to the release of the contact. A configuration that is not executed is also assumed.

  As understood from the above description, the operation control unit 43A determines that the user U has come into contact with the character C when the HMD 30 has rotated to the first side in the roll direction (Sa2: YES) (Sa4). On the other hand, in a state where the user U is in contact with the character C (Sa1: YES), when the HMD 30 is rotated to the second side in the roll direction (Sa6: YES), it is determined that the contact with the character C is released. According to the above configuration, the user U can instruct generation and release of contact with the character C in the virtual space V according to the direction of rotation of the HMD 30 in the roll direction. Since the rotations opposite to each other in the roll direction correspond to the opposite operations of generating and releasing the contact with the character C, the user U can easily intuitively grasp the occurrence and release of the contact with the character C. is there.

  In the first embodiment, for example, a configuration exemplified below may be adopted.

[Modification A1]
The motion control unit 43A in the modification A1 causes the character C to execute a reaction motion according to the length of time during which the contact with the character C is maintained in the virtual space V. Specifically, the operation control unit 43A changes the reaction operation over time within a period in which the contact state continues (hereinafter, referred to as a “contact period”). The contact period is a period from when the HMD 30 rotates to the first side in the roll direction until it rotates to the second side. Note that a period from when the contact determination data is set to a numerical value indicating the contact state to when the contact determination data is changed to a numerical value indicating the non-contact state may be set as the contact period. The motion control unit 43A changes the reaction motion of the character C over time from the start point to the end point of the contact period. Therefore, the reaction behavior of the character C changes according to the length of the contact period. According to the above configuration, the reaction behavior of the character C can be variously changed according to the length of time the user U contacts the character C.

[Modification A2]
In the storage device 12 in the modification A2, parameters relating to the character C are stored. Specifically, the favorableness of the character C for the user U is stored in the storage device 12 as a parameter. Positiveness is a parameter indicating the degree of favorable emotion from the character C to the user U. The operation control unit 43A changes the liking of the character C to the user U according to the contact with the character C in addition to changing the liking according to the progress of the game. Specifically, as the degree of contact (for example, the number or time) of the user U with the character C increases, the favorable impression of the character C to the user U is set to a larger value. Note that, in the above example, the favorable sensibility of the character C for the user U has been described. However, the favorable sensibility stored in the storage device 12 is the favorable sensibility of the character C for the character (player character) used by the user U. May be. The user U can have a plurality of player characters in the virtual space V.

  In addition, the operation control unit 43A changes the reaction operation according to the favorable sensibility stored for the character C. That is, when the user U comes into contact with the character C (Sa2: YES, Sa3: YES), the type of reaction performed by the character C (Sa4) changes in accordance with the favorable sensibility of the character C. For example, in a state where the number of times or time of contact with the character C is insufficient and the favorable feeling is low, when the user U contacts the head, the character C executes a reaction operation of rejecting the contact. On the other hand, in a state where the number of times or time of contact with the character C is sufficiently secured and the favorable feeling is high, when the user U makes contact with the head, the character C performs a reaction operation of receiving the contact. According to the above configuration, the reaction motion of the character C can be variously changed according to the parameters related to the character C.

  Note that the parameters affecting the reaction motion of the character C are not limited to the favorable sensitivities exemplified above. For example, various parameters such as the degree of growth (level) of the character C or the degree of intimacy between the user U and the character C change in accordance with the contact of the character C by the user U, and change in accordance with the parameter. Thus, the reaction motion of the character C is controlled.

[Modification A3]
The motion control unit 43A in the modification A3 causes the character C to perform a reaction motion corresponding to the angle θ at which the HMD 30 rotates in the roll direction. For example, the rotation angle θ of the HMD 30 corresponds to a virtual pressure at which the user U contacts the character C in the virtual space V. For example, when the rotation angle θ is small, it means that the user U is in light contact with the character C. When the rotation angle θ is large, the user U strongly presses the character C. Means the state of being. For example, when the rotation angle θ is small, the motion control unit 43A causes the character C to execute a reaction operation of receiving the contact by the user U, and rejects the contact by the user U when the rotation angle θ is large. The character C is caused to execute the reaction action. According to the above configuration, the reaction motion of the character C can be variously changed according to the rotation angle θ of the HMD 30 (that is, the virtual pressure at which the user U contacts the character C).

[Modification A4]
The display control unit 42A in the modification A4 moves the virtual camera E (that is, the virtual viewpoint in the virtual space V) in the virtual space V according to the rotation of the HMD 30 in the roll direction. Specifically, the display control unit 42A changes the positional relationship between the virtual camera E and the character C in the virtual space V in conjunction with the rotation of the HMD 30. For example, the distance between the virtual camera E and the character C is controlled according to the rotation angle θ of the HMD 30. That is, the virtual camera E moves in the direction of the object in conjunction with the rotation of the HMD 30 in the roll direction.

  For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A moves the virtual camera E by a moving amount corresponding to the rotation angle θ (θ> 0) of the HMD 30 as shown by an arrow a1 in FIG. The character C is made to approach in the virtual space V. On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A moves the virtual camera E by the moving amount corresponding to the rotation angle θ (θ <0) of the HMD 30 as shown by an arrow a2 in FIG. It is separated from the character C in the virtual space V. According to the above configuration, the user U can move the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[Modification A5]
The display control unit 42A in the modification A5 changes the imaging range R (that is, the angle of view of the virtual camera E) by the virtual camera E according to the rotation of the HMD 30 in the roll direction. That is, the range displayed on the display device 13 in the virtual space V changes according to the rotation of the HMD 30 in the roll direction. For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A reduces the imaging range R at a ratio according to the rotation angle θ of the HMD 30, as indicated by an arrow b1 in FIG. Therefore, for example, the character C in the virtual space V is zoomed in (enlarged). On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A enlarges the imaging range R at a ratio according to the rotation angle θ of the HMD 30, as indicated by an arrow b2 in FIG. Therefore, for example, the character C in the virtual space V is zoomed out (reduced). According to the above configuration, the user U can change the imaging range R of the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[Modification A6]
In the first embodiment, when the HMD 30 has rotated to the second side in the roll direction (Sa6: YES), it is determined that the contact with the character C has been released. However, the condition for determining that the contact with the character C has been released is as follows. It is not limited to the above examples. For example, the motion control unit 43A may determine that the contact with the character C has been released when the HMD 30 has returned from the state in which the HMD 30 has rotated in the roll direction (Sa2: YES). Specifically, the motion control unit 43A sets the rotation angle θ of the HMD 30 to the initial angle (eg, θ = 0) at the time when the rotation for contact with the character C is started in step Sa6 in FIG. If it has changed, it is determined that the contact with the character C has been released (Sa7). On the other hand, in step Sa2, the operation control unit 43A determines that the user U has come into contact with the character C when the rotation angle θ changes regardless of the rotation direction (first side / second side) of the HMD 30. I do. As can be understood from the above description, by monitoring the rotation angle θ of the HMD 30, the occurrence and release of the contact with the character C can be performed without determining the rotation direction (first side / second side) of the HMD 30. Can be determined. That is, in determining the contact with the character C, the operation control unit 43A does not necessarily need to determine the direction of rotation of the HMD 30.

[Modification A7]
In the first embodiment, when the reference line Q intersects with the character C in the virtual space V (Sa3: YES), the character C is caused to execute a reaction operation to the contact by the user U. The condition relating to the positional relationship with C is not limited to the above example. For example, the reaction action may be performed by the character C on condition that the distance between the reference line Q and the character C is smaller than a predetermined threshold. The case where the distance between the reference line Q and the character C is less than the predetermined threshold includes the case where the reference line Q intersects the character C and the case where the reference line Q is separated from the character C within a range below the threshold. . Further, the reaction action may be performed by the character C on condition that the reference line Q intersects a specific part of the character C. When the reference line Q intersects a part other than the specific part of the character C, the movement control unit 43A does not cause the character C to execute the reaction movement. As can be understood from the above description, the motion control unit 43A of the first embodiment determines whether the character C is in accordance with the positional relationship between the reference line Q and the character C (for example, the relationship between the reference line Q and the character C). It is comprehensively expressed as an element that controls the operation.

[B: Second Embodiment]
A second embodiment of the present invention will be described. In the following examples, the same reference numerals are used for elements having the same functions as those in the first embodiment, and detailed descriptions thereof will be appropriately omitted.

  The image display system according to the second embodiment has the same configuration as the first embodiment illustrated in FIGS. 1 and 2. That is, the HMD 30 according to the second embodiment includes the display device 13 and the detection device 14 of the terminal device 10 and the mounting device 20, and the information processing device 40 includes the control device 11 and the storage device 12 of the terminal device 10. I do.

  FIG. 11 is a block diagram illustrating a functional configuration of the terminal device 10 according to the second embodiment. As illustrated in FIG. 11, the control device 11 of the second embodiment functions as a posture analysis unit 41, a display control unit 42B, and an operation control unit 43B by executing a program stored in the storage device 12. Note that part or all of the control device 11 may be realized by a dedicated electronic circuit. As in the first embodiment, the posture analysis unit 41 analyzes the detection signal output by the detection device 14 to sequentially generate posture data relating to the posture of the HMD 30.

  The display control unit 42B displays an image within the imaging range R of the virtual camera E in the virtual space V and an instruction image G indicating the direction of the reference line Q in the display device, similarly to the display control unit 42A of the first embodiment. 13 is displayed. As described above in the first embodiment, the reference line Q is a virtual straight line whose direction in the virtual space V changes according to the direction of the HMD 30.

  The display control unit 42B according to the second embodiment changes the display mode of the instruction image G according to the positional relationship between the reference line Q and the character C (an example of an object) in the virtual space V. Specifically, the display control unit 42B makes the display mode of the instruction image G different when the reference line Q intersects the character C and when the reference line Q does not intersect the character C. The display mode means a property of an image that can be visually distinguished by the user U. For example, in addition to the three attributes of color, hue (color tone), saturation and lightness (gradation), size and image content (for example, pattern or shape) are also included in the concept of the display mode.

  When the reference line Q does not cross the character C, the display control unit 42B causes the display device 13 to display the image G1 as the instruction image G as illustrated in FIG. The image G1 is a circular or point-like object arranged in the virtual space V, and is arranged on the reference line Q in the virtual space V. On the other hand, when the reference line Q intersects the character C, the display control unit 42B causes the display device 13 to display an image G2 different from the image G1 as the instruction image G, as illustrated in FIG. The image G2 is a planar object schematically representing the hand of the user U. As illustrated in FIG. 14, the display control unit 42B places the image G2 at a point on the surface of the character C in the virtual space V that intersects with the reference line Q (that is, the contact point P). That is, the image G2 contacts the contact point P on the surface of the character C in the virtual space V. The contact point P is also referred to as a point on the surface of the character C where the image G2 contacts.

  In the above description, switching between the image G1 and the image G2 has been described as an example. However, the change in the display mode of the instruction image G includes switching between display and non-display. That is, the display control unit 42B may display the image G2 as the instruction image G when the reference line Q intersects the character C, and may hide the instruction image G when the reference line Q does not intersect the character C. .

  As understood from the above description, in the second embodiment, the display mode of the instruction image G indicating the direction of the reference line Q in the virtual space V changes according to the positional relationship between the reference line Q and the character C. . Therefore, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V. Further, since the instruction image G (image G2) is arranged on the surface of the character C in the virtual space V, the user U can easily perceive a state in which the user is in contact with the surface of the character C in the virtual space V. There is. By arranging the instruction image G on the surface of the character C, there is also an advantage that the user U can easily and accurately grasp the contact point P visually.

  The motion control unit 43B of FIG. 11 controls the motion of the character C in the virtual space V, similarly to the motion control unit 43A of the first embodiment. Specifically, the motion control unit 43B causes the character C to perform a motion that responds to the contact of the user U in the virtual space V (that is, a reaction motion). A typical example of a reaction action is a change in posture, a change in facial expression, or the pronunciation of a particular line.

  Specifically, the motion control unit 43B causes the character C to execute a motion corresponding to the position of the contact point P where the reference line Q intersects among the characters C. For example, when the contact point P is located at the head of the character C, the motion control unit 43B causes the character C to execute a reaction operation of receiving the contact by the user U, and the contact point P is changed to the body of the character C. In this case, the character C is caused to execute an operation of rejecting the contact by the user U. According to the above configuration, it is possible to variously change the motion of the character C according to the position of the contact point P.

  FIG. 15 is a flowchart illustrating the specific contents of the processing (hereinafter, referred to as “second control processing”) executed by the display control unit 42B and the operation control unit 43B of the second embodiment. The second control process of FIG. 15 is repeatedly executed at a predetermined cycle.

  When the second control process is started, the display control unit 42B determines whether the reference line Q intersects the character C in the virtual space V (Sb1). When the reference line Q intersects the character C (Sb1: YES), the display control unit 42B uses the image G2 arranged at the contact point P where the reference line Q and the character C intersect as the instruction image G on the display device 13. It is displayed (Sb2). On the other hand, when the reference line Q does not intersect with the character C (Sb1: NO), the display control unit 42B causes the display device 13 to display the image G1 arranged on the reference line Q as the instruction image G (Sb3).

  In a state where the image G2 overlapping the character C is displayed, the operation control unit 43B determines whether or not an instruction to perform an operation on the character C (hereinafter, referred to as “operation instruction”) has been received from the user U (Sb4). The motion instruction is an instruction for generating an action on the character C in the virtual space V. The operation instruction of the second embodiment means an instruction that the user U contacts the character C in the virtual space V.

  The operation control unit 43B of the second embodiment receives a specific change regarding the posture of the HMD 30 as an operation instruction from the user U. Specifically, the operation control unit 43B determines that the operation instruction has been received when the change in the posture of the HMD 30 satisfies a predetermined condition (hereinafter, referred to as “instruction determination condition”). For example, similarly to the first embodiment, when the HMD 30 rotates in the roll direction, the operation control unit 43B determines that an operation instruction has been given from the user U. That is, the rotation of the HMD 30 in the roll direction is the instruction determination condition. On the other hand, when the change in the posture of the HMD 30 does not satisfy the instruction determination condition, the operation control unit 43B determines that the operation instruction has not been received. Note that, as described above with reference to FIG. 7, it is preferable that the configuration in which the HMD 30 is rotated is determined when the rotation angle θ of the HMD 30 exceeds the threshold θt.

  When an operation instruction is received from the user U (Sb4: YES), the operation control unit 43B determines that the user U has contacted the character C in the virtual space V, and performs an operation to react to the contact by the user U. The character C is executed (Sb5). Specifically, the motion control unit 43B causes the character C to perform a motion corresponding to the position of the contact point P where the reference line Q intersects among the characters C, as described above. As understood from the above description, in the second embodiment, the user U gives the virtual instruction by giving the operation instruction while moving the reference line Q so as to cross the desired position of the character C. It is possible to touch a desired position of the character C in the space V.

  In a configuration in which a change in the posture of the HMD 30 is recognized as an operation instruction, a direct operation such as a touch operation on the display surface is not required, and thus there is a problem in that the user U does not easily feel a sense of contact. In the second embodiment, the display mode of the instruction image G changes according to the positional relationship between the reference line Q and the character C (specifically, whether or not the reference line Q intersects the character C). Compared to a configuration in which G is displayed in a fixed manner, there is an advantage that the user U can easily feel the feeling of the user U touching the character C in the virtual space V.

  In the second embodiment, for example, a configuration exemplified below may be adopted.

[Modification B1]
In the second embodiment, the character C is caused to execute a reaction motion corresponding to the position of the contact point P. However, similarly to the first embodiment, the character C is controlled according to the amount of change in the posture of the HMD 30 (for example, the rotation angle θ in the roll direction). The character C may be caused to perform the reacting action. For example, it is assumed that the amount of change in the posture of the HMD 30 is a virtual pressure at which the user U contacts the character C in the virtual space V. The motion control unit 43B of the modified example B1 causes the character C to execute a reaction operation of receiving the contact by the user U when the change amount of the posture of the HMD 30 is small, and uses the reaction operation when the change amount of the posture is large. The character C is caused to execute a reaction operation of rejecting the contact by the person U. According to the above aspect, it is possible to cause the character C to perform various reaction actions according to the posture of the HMD 30.

[Modification B2]
The operation control unit 43B of the modified example B2 receives an operation instruction from the user U within a specific period on the time axis (hereinafter, referred to as “instruction reception period”). That is, the operation instruction given by the user U during the instruction reception period is effectively reflected in the operation of the character C, but the operation instruction given by the user U at the time after the elapse of the instruction reception period is ignored. The instruction receiving period is, for example, a period of a predetermined time length (for example, several seconds) starting from the time when the reference line Q starts to cross the character C. The start point or the end point of the instruction receiving period is not limited to the above example. For example, the point in time at which a specific event starts in the game may be set as the start point of the instruction reception period. By restricting the reception of the operation instruction to the instruction reception period as described above, the user U is given an appropriate sense of tension, and as a result, it is possible to suppress the tiredness of the game.

  The display control unit 42B changes the display mode of the instruction image G (image G2) over time during the instruction reception period. For example, as illustrated in FIG. 16, it is assumed that the image G2 displayed as the instruction image G when the reference line Q intersects the character C includes an image G21 and an image G22. The image G21 is an image representing the hand of the user U, like the image G2 of the first embodiment, and the image G22 is a circular image arranged behind the image G21. The display control unit 42B reduces the size of the image G22 over time from the start point to the end point of the instruction receiving period. The size of the image G21 does not change. According to the above configuration, there is an advantage that the user U can visually grasp the instruction reception period (for example, remaining time) in which the reception of the operation instruction is permitted from the display mode of the instruction image G.

[Modification B3]
The display control unit 42B of the modification B3 changes the display mode of the instruction image G according to the position of the contact point P where the reference line Q intersects among the characters C in the virtual space V. Specifically, as illustrated in FIG. 17, the pointing image is different between the case where the contact point P is located at the first position on the surface of the character C and the case where the contact point P is located at the second position on the surface of the character C in the virtual space V. The display mode of G is different. The first position is, for example, a position at which the character C accepts contact by the user U (for example, the body of the character C), and the second position is, for example, a position at which the character C rejects contact by the user U (for example, the character C). C head).

  Although the case where the contact point P is at the first position or the second position is illustrated in FIG. 17, the display control unit 42B displays the instruction image G in parallel with the change in the position of the contact point P in the character C. The embodiment may be changed over time. That is, when the user U moves the contact point P on the surface of the character C in accordance with the posture of the HMD 30, the display control unit 42B changes the display mode of the instruction image G over time in conjunction with the movement of the contact point P. To change. For example, during the period in which the contact point P moves from the first position to the second position in FIG. 17, the display control unit 42B changes the display mode of the instruction image G from the display mode at the first position to the display mode at the second position. Change continuously or step by step.

  In the modification B3, the display mode of the instruction image G changes according to the position of the contact point P. Therefore, by visually recognizing the display mode of the instruction image G, the user U can infer the action of the character C at the time of the contact. That is, by confirming the display mode of the instruction image G, the contact point P is gradually moved while estimating the action of the character C at the time of contact, and the contact point P is maintained at a position where a desired action is expected. It is possible to provide the user U with an interest of giving an operation instruction.

[Modification B4]
The display control unit 42B of the modified example B4 controls the character C so that the line of sight of the character C follows the reference line Q. For example, the character C is controlled so that the line of sight is directed to the contact point P where the reference line Q intersects the character C. Specifically, as illustrated in FIG. 18, the eyeball (for example, the pupil) and the head of the character C rotate so as to track the reference line Q. Note that only one of the eyeball and the head of the character C may track the reference line Q. According to the modified example B4, there is an advantage that the user U can easily grasp the feeling of the user U affecting the character C.

[Modification B5]
In the second embodiment, the display mode of the instruction image G is changed according to whether the reference line Q intersects the character C. However, the conditions for changing the display mode of the instruction image G are as described above. Not limited. For example, the display control unit 42B may change the display mode of the instruction image G continuously or stepwise according to the distance between the reference line Q and the character C. The intersection between the reference line Q and the character C is not an essential condition for changing the display mode of the instruction image G. Further, the display mode of the instruction image G may be changed depending on whether the reference line Q intersects a specific part of the character C. In a state where the reference line Q intersects a part other than the specific part of the character C, the instruction image G is displayed in the same display mode as when the reference line Q does not intersect with the character C. As understood from the above description, the operation control unit 43B of the second embodiment is comprehensively expressed as an element that changes the display mode of the instruction image G according to the positional relationship between the reference line Q and the character C. .

[Modification B6]
The configuration of the second embodiment in which the display mode of the instruction image G is changed according to the positional relationship between the reference line Q and the character C is used in a state where the user U holds the terminal device 10 (for example, a smartphone) by hand. The same applies to cases. As in the second embodiment, the imaging range R displayed on the display device 13 in the virtual space V changes according to the attitude of the terminal device 10.

  The terminal device 10 of the modified example B6 includes a touch panel that detects a contact of the user U with the display surface of the display device 13. In the second embodiment, the position of the contact point P in the virtual space V is changed according to the attitude of the HMD 30. In the modified example B6, the point at which the user U contacts the display surface of the display device 13 is determined. The instruction image G is displayed as the contact point P. For example, a reference line Q passing through a contact point P where the user U has contacted is set in the virtual space V. That is, a virtual straight line in the direction instructed by the user U by touching the display surface of the display device 13 (that is, touch operation) is set in the virtual space V as the reference line Q. When the reference line Q intersects the character C, the display control unit 42B causes the display device 13 to display the image G2 as the instruction image G, and displays the image G1 when the reference line Q does not intersect the character C. Is displayed on the display device 13. Even in the above configuration, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V (for example, a state where the user U is in contact with the character C in the virtual space V). There are advantages.

  As understood from the illustration of the modification B6, in the second embodiment, the configuration in which the display device 13 is mounted on the head of the user U can be omitted. The display control unit 42B in a configuration not premised on being mounted on the head of the user U displays an image obtained by capturing the virtual space V with the virtual camera E whose attitude is controlled according to the attitude of the display device 13. 13 is displayed as an element to be displayed. In addition, in the display device 13 used while the user U holds it in his hand, a virtual straight line in the direction instructed by an operation (for example, a touch operation) on the display device 13 is a “reference line”.

[C: Another modified example]
Specific modifications of the first embodiment or the second embodiment will be exemplified below. Two or more aspects arbitrarily selected from the following exemplifications may be appropriately combined within a mutually consistent range.

[Modification C1]
The reference line Q is not limited to the optical axis of the virtual camera E illustrated in each of the above embodiments. For example, a straight line perpendicular to the display surface of the display device 13 is used as the reference line Q. Further, a straight line that forms a predetermined angle with respect to the optical axis of the virtual camera E or a straight line perpendicular to the display surface may be used as the reference line Q. In the HMD 30 equipped with an eye tracking (line-of-sight measurement) function for estimating the direction of the line of sight of the user U, the line of sight estimated by the function may be used as the reference line Q. As understood from the above examples, the reference line Q is comprehensively represented as a virtual straight line set in the virtual space V.

[Modification C2]
In each of the above-described embodiments, the character C is illustrated as an object arranged in the virtual space V, but the object that the user U contacts in the virtual space V is not limited to the character C. For example, inanimate elements such as buildings and natural objects in the virtual space V are also included in the concept of the object. The “action” of a biological object (character C) is, for example, the behavior, action, appearance, or attitude of the object. The “action” for an inanimate object is, for example, a change in the form of the element. For example, an operation of opening a door of a building or an operation of deforming or moving a natural object such as a rock is exemplified as an operation of an object.

[Modification C3]
In each of the above-described embodiments, when the HMD 30 rotates in the roll direction, it is determined that the user U has touched the character C in the virtual space V, and the character C performs the reaction operation. That is, the operation instruction) is not limited to the above example. For example, the rotation of the HMD 30 in the pitch direction or the yaw direction may be used as the contact by the user U to cause the character C to perform the reaction operation. Further, for example, the character C reacts to the character C on condition that the state where the reference line Q intersects within a specific range of the character C (that is, when the user U stares in the range for a long time) continues for a predetermined time. The operation may be executed. The character C may perform a reaction operation on condition that the user U operates an operation device (not shown) connected to the HMD 30. As understood from the above examples, a configuration in which a change in the posture of the HMD 30 (particularly, rotation in the roll direction) as a trigger of the reaction operation of the character C may be omitted.

[Modification C4]
In each of the above-described embodiments, the rotation of the HMD 30 in the roll direction is received as a motion instruction (an instruction of contact with the character C). However, the change in the posture of the HMD 30 determined to be the motion instruction is not limited to the rotation in the roll direction. For example, the rotation of the HMD 30 in the pitch direction or the yaw direction may be received as an operation instruction.

[Modification C5]
The modifications (Modifications A1 to A7) illustrated for the first embodiment are similarly applied to the second embodiment. Further, the modifications (modifications B1 to B6) exemplified in the second embodiment are similarly applied to the first embodiment.

[Modification C6]
In each of the above-described embodiments, the configuration in which the display device 13 and the detection device 14 of the HMD 30 and the information processing device 40 that controls the HMD 30 are realized by a single terminal device 10 is illustrated. However, as in the image display system 1B of FIG. The HMD 30 and the information processing device 40 may be realized as separate devices. The HMD 30 and the information processing device 40 can communicate with each other by wire or wirelessly. The method of communication between the HMD 30 and the information processing device 40 is arbitrary, but short-range wireless communication such as bluetooth (registered trademark) is preferable.

  The HMD 30 includes the display device 13, the detection device 14, and the mounting device 20 as in the first embodiment, and is mounted on the head of the user U. The information processing device 40 is a control device that displays various images on the HMD 30 by communicating with the HMD 30, and includes a control device 11 and a storage device 12. For example, various information terminals such as a mobile phone, a smartphone, a tablet terminal, a personal computer, and a home game device are used as the information processing device 40. It does not matter whether the information processing device 40 is portable or stationary. The control device 11 executes the program stored in the storage device 12 to execute the posture analysis unit 41, the display control unit 42 (42A, 42B), and the operation control unit in the same manner as in the first embodiment or the second embodiment. 43 (43A, 43B). Therefore, the same effects as those of the first embodiment or the second embodiment can be realized in the configuration of FIG.

[Modification C7]
A preferred embodiment of the present invention is realized by cooperation between a computer (specifically, the control device 11) and a program, as exemplified in the above embodiments. The program according to each of the above-described embodiments is provided in a form stored in a computer-readable recording medium, and is installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, and a known arbitrary recording medium such as a semiconductor recording medium or a magnetic recording medium is used. In the form of a recording medium. Note that the non-transitory recording medium includes any recording medium except for a transient propagation signal (transitory, propagating signal), and does not exclude a volatile recording medium. In addition, the program can be provided to the computer in a form of distribution via a communication network.

[D: Appendix]
From the above description, preferable aspects of the present invention are grasped, for example, as follows. In the following, in order to facilitate understanding of each embodiment, reference numerals in the drawings are written in parentheses for convenience, and the present invention is not limited to the illustrated embodiment.

[Aspect 1]
A program according to a preferred aspect (aspect 1) of the present invention is an image obtained by capturing a virtual space (V) with a virtual camera (E) whose direction is controlled according to the direction of a head-mounted display (30), A program that causes a computer (11) to function as a display control unit (42B) that causes a display device (13) of the head-mounted display (30) to display a stereoscopic image using binocular parallax. The unit (42B) displays the pointing image (G) in the direction of a reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head-mounted display (30), and displays the reference image (G). The display mode of the instruction image (G) is changed according to the positional relationship between Q) and the object (C) in the virtual space (V). In the above configuration, the display mode of the instruction image (G) indicating the direction of the reference line (Q) in the virtual space (V) changes according to the positional relationship between the reference line (Q) and the object (C). Therefore, there is an advantage that the user (U) can easily grasp the positional relationship between the reference line (Q) and the object (C) in the virtual space (V).

  The “reference line” is a virtual straight line set in the virtual space (V). For example, the optical axis of the virtual camera (E), the center line of the display screen by the display device (13), or a straight line that forms a predetermined angle with respect to these straight lines is a preferable example of the reference line (Q). In a configuration in which the eye tracking function for estimating the line of sight of the user (U) can be used, the line of sight of the user (U) may be used as the reference line (Q) in addition to the straight lines exemplified above.

  An “object” is a virtual object installed in the virtual space (V). A typical example of the object (C) is a character (for example, a human, an animal, or a monster), but an inanimate element such as a building or a natural object in the virtual space (V) may be included in the concept of the object.

  The “display mode” means a property of an image that can be visually distinguished. For example, in addition to the three attributes of color, hue (color tone), saturation and lightness (gradation), size and image content (for example, pattern or shape) are also included in the concept of the display mode. Further, the change in the display mode includes switching between display and non-display.

[Aspect 2]
In a preferred example (Aspect 2) of Aspect 1, the display control unit (42B) places the instruction image (G) at a point (P) on the surface of the object (C) that intersects with the reference line (Q). Display. According to the above aspect, since the pointing image (G) is arranged on the surface of the object (C) in the virtual space (V), the state of contact with the surface of the object (C) is determined by the user ( There is an advantage that U) is easily perceived by the instruction image (G).

[Aspect 3]
A program according to a preferred example (aspect 3) of aspect 1 or aspect 2 includes an operation control unit (for controlling an operation of the object (C) in accordance with a positional relationship between the reference line (Q) and the object (C)). 43B) causes the computer (11) to function. According to the above aspect, it is possible to cause the object (C) to execute various operations reflecting the instruction from the user (U).

  The “object operation” is, for example, the behavior, action, appearance, or attitude of a character that is an example of the object (C). The “movement of an object” with respect to an inanimate element is, for example, a change in the form of the element (for example, a door or window of a building opens, a natural object such as rock deforms or moves, etc.).

[Aspect 4]
In a preferred example of the third aspect (fourth aspect), when the operation control unit (43B) receives an operation instruction from the user (U), the operation control unit (43B) determines a positional relationship between the reference line (Q) and the object (C). Controls the operation of the object (C). According to the above aspect, the user (U) gives an operation instruction while moving the reference line (Q) to a desired position, thereby performing an operation corresponding to a desired position on the object (C). Can be executed.

  The “operation instruction” is an instruction for generating an action (for example, contact) on the object (C) in the virtual space (V). This is also referred to as an instruction for determining the provisional direction represented by the instruction image (G) in the direction desired by the user (U). For example, when a specific change (for example, rotation in the roll direction) occurs in the direction of the head mounted display (30), it is determined that the operation instruction has been received. In addition, for example, when the state in which the reference line (Q) is stationary in the virtual space (V) continues for a predetermined time (for example, 2 seconds), it may be determined that the operation instruction has been received.

[Aspect 5]
In a preferred example (Aspect 5) of Aspect 4, the operation control unit (43B) determines that the operation instruction has been received when a change in the posture of the head mounted display (30) satisfies a predetermined condition, and The display control unit (42B) causes the object (C) to execute an operation according to the amount of change in the posture in accordance with the change in the posture of the head-mounted display (30) before receiving the operation instruction. The display mode of the instruction image (G) is changed. According to the above aspect, it is possible to cause the object (C) to execute various operations according to the posture of the head mounted display (30). In addition, since the display mode of the instruction image (G) changes according to the attitude of the head mounted display (30), the user (U) has a feeling of affecting the object (C) (for example, a feeling of touching). Is easy for the user (U) to grasp. The “predetermined condition” is a condition of a posture change determined to be an operation instruction by the user (U), and is, for example, rotation in a roll direction in the embodiment.

[Aspect 6]
In a preferred example (Aspect 6) of Aspect 4 or Aspect 5, the display control unit (42B) is configured to display the instruction image (G) within an instruction acceptance period in which the user (U) is allowed to accept the operation instruction. The display mode is changed over time. According to the above aspect, the user (U) can visually grasp the instruction reception period in which the reception of the operation instruction is permitted (for example, the remaining time of the instruction reception period) from the display mode of the instruction image (G). The “instruction reception period” is, for example, the remaining time during which a single contact with the character is possible or the remaining time in an operation mode in which contact with the character is permitted.

[Aspect 7]
In a preferred example (Aspect 7) of any one of Aspects 3 to 6, the display control unit (42B) determines a position of a point (P) of the object (C) where the reference line (Q) intersects. To change the display mode of the instruction image (G). According to the above aspect, the display mode of the instruction image (G) changes according to the position of the point (P) where the reference line (Q) intersects with the object (C). The operation of the object (C) can be estimated from the display mode of the instruction image (G).

[Aspect 8]
In a preferred example (Aspect 8) of Aspect 7, the display control unit (42B) performs the instruction in parallel with a change in the position of a point (P) of the object (C) where the reference line (Q) intersects. The display mode of the image (G) is changed over time. According to the above aspect, the display mode of the instruction image (G) changes over time in parallel with the change in the position of the point (P) where the reference line (Q) intersects with the object (C). It is possible to provide the user (U) with an interest of gradually changing the reference line (Q) while checking the display mode of the image (G). The “change in display mode over time” means that the display mode does not change only in a binary manner at a specific time, but the display mode changes gradually with the passage of time. It does not matter whether the display mode changes continuously or stepwise.

[Aspect 9]
In a preferred example (Aspect 9) of any of Aspects 1 to 8, the motion control unit (43B) controls the object (C) so that the line of sight of the object (C) tracks the reference line (Q). Control. According to the above aspect, the user (U) can easily grasp the sensation (for example, the touching sensation) affecting the object (C). Note that “the line of sight of the object tracks the reference line” refers to not only the case where the eyeball (eg, pupil) of the object (C) rotates so as to track the reference line (Q), but also the head of the object (C). This includes the case where the unit rotates according to the reference line (Q).

[Aspect 10]
An image display system (1A, 1B) according to a preferred embodiment (aspect 10) of the present invention is an image display system (1A, 1B) including a head-mounted display (30) and an information processing device (40). The information processing apparatus (40) is an image obtained by capturing a virtual space (V) with a virtual camera (E) whose direction is controlled according to the direction of a head-mounted display (30), and uses binocular parallax. A display controller (42B) for displaying the obtained stereoscopic image on the display device (13) of the head-mounted display (30), wherein the display controller (42B) is provided in the direction of the head-mounted display (30). The pointing image (G) is displayed in the direction of the reference line (Q) in which the direction in the virtual space (V) changes according to the reference line (Q) and the object in the virtual space (V) Depending on the positional relationship between C) changing the display mode of the instruction image (G). In the above configuration, the display mode of the instruction image (G) indicating the direction of the reference line (Q) in the virtual space (V) changes according to the positional relationship between the reference line (Q) and the object (C). Therefore, there is an advantage that the user (U) can easily grasp the positional relationship between the reference line (Q) and the object (C) in the virtual space (V). The head mounted display (30) and the information processing device (40) may be either integral or separate.

1A, 1B image display system, 10 terminal device, 11 control device, 12 storage device, 13 display device, 14 detection device, 20 mounting device, 30 HMD, 40 information processing device, 41 Posture analysis unit, 42A, 42B: display control unit, 43A, 43B: operation control unit, V: virtual space, E: virtual camera, C: character, R: imaging range, Q: reference line, G: instruction image, P ... the point of contact.

Claims (8)

A display control unit configured to display a stereoscopic image using binocular parallax on a display device of the head-mounted display, which is an image obtained by capturing a virtual space with a virtual camera whose direction is controlled in accordance with the direction of the head-mounted display. , And ,
A program that causes a computer to function as an operation control unit that controls the operation of the object according to the positional relationship between a reference line whose direction in the virtual space changes according to the direction of the head mounted display and the object in the virtual space. So,
The display control unit may display the instruction image in the direction of the reference line changes the display mode of the instruction image according to the positional relationship between the said reference line object,
The object is a character having a line of sight,
The operation control unit, the program line of sight of the character that controls the character to track the reference line. The program according to claim 1, wherein the display control unit displays the instruction image at a point on the surface of the object that intersects the reference line. The operation control unit, when receiving an operation instruction from a user, according to claim 1 or claim 2 program for controlling the operation of said object in accordance with the positional relationship between the said reference line object. The display controller according to claim 3 of the program over time to change the display mode of the instruction image in the instruction reception period in which the user by the operation instruction reception is permitted.   The operation control unit causes the object to execute an operation according to a change amount of a posture of the head mounted display.
  The program according to any one of claims 1 to 4. The display controller, one of the program of claims 1 to 5, wherein the reference line of the object changes the display mode of the instruction image in accordance with the position of the point of intersection. The program according to claim 6 , wherein the display control unit changes a display mode of the instruction image with time in parallel with a change in a position of a point where the reference line intersects among the objects. An image display system including a head-mounted display and an information processing device,
The information processing device,
An image obtained by imaging a virtual space with a virtual camera direction according to the direction of the head-mounted display is controlled, a stereoscopic image using binocular parallax, the display control to display on a display device of the head mounted display Department and
An operation control unit that controls an operation of the object according to a positional relationship between a reference line whose direction in the virtual space changes according to the direction of the head mounted display and an object in the virtual space ,
The display control unit may display the instruction image in the direction of the reference line changes the display mode of the instruction image according to the positional relationship between the said reference line object,
The object is a character having a line of sight,
The image display system wherein the operation control unit, the line of sight of the character that controls the character to track the reference line.

Images