JP6388270B1 - Information processing apparatus, information processing apparatus program, head mounted display, and display system - Google Patents

Abstract

An operation other than a change in attitude of a virtual camera in a virtual space is performed.
A control program PRG causes a processor 1000 to display an image obtained by capturing a virtual space SP-V with a virtual camera CM on a display unit 12 provided in the HMD1, and attitude information regarding the attitude of the HMD1. The display control unit 110 changes the position of the virtual camera CM based on the posture information B around the virtual point K, and changes the posture of the virtual camera CM to the posture information B. It is possible to display an image in a posture control mode that changes based on posture information and a position control mode that changes the position of the virtual point K based on posture information B and changes the position of the virtual camera CM based on posture information B. When the image is displayed in the posture control mode, when the posture change amount indicated by the posture information B exceeds the reference amount, the control mode is changed to the posture control mode. Switching from mode to position control mode.
[Selection] Figure 8

Description

  The present invention relates to an information processing apparatus, a program for the information processing apparatus, a head mounted display, and a display system.

  In recent years, a head mounted display (HMD) has been widely spread. The HMD is an image obtained by capturing a virtual space with a virtual camera, for example, a stereoscopic image using binocular parallax, etc., on a display unit that is mounted on the user's head and provided in front of the user's eyes. Display (for example, refer to Patent Document 1). In such an HMD, generally, the user can visually recognize various directions in the virtual space by changing the posture of the virtual camera in the virtual space based on the change in the posture of the HMD.

Japanese Patent Laid-Open No. 2006-115122

  By the way, since the user wearing the HMD is viewing the display unit provided in front of the user's eyes, it may be difficult to see a part other than the display unit. Therefore, for a user wearing the HMD, for example, an operation of a controller or the like that is operated with a hand may be a burden. For this reason, for example, in a game using the HMD, in order not to place an excessive burden on the user wearing the HMD, it is preferable to accept various instructions from the user due to a change in the attitude of the HMD. However, when an instruction from the user is received due to a change in the attitude of the HMD, it is difficult to perform an operation other than a change in the attitude of the virtual camera in the virtual space. For example, an operation such as a change in the position of the virtual camera in the virtual space is performed. It could be difficult.

  The present invention has been made in view of the above-described circumstances, and provides a technique that enables operations other than a change in the posture of a virtual camera in a virtual space due to a change in the posture of the HMD. One.

  In order to solve the above problems, a program for an information processing device according to one embodiment of the present invention is a program for an information processing device including a processor, and the processor is an image obtained by imaging a virtual space with a virtual camera. A display control unit that displays a stereoscopic image using binocular parallax on a display unit provided in a head-mounted display, and an acquisition unit that acquires posture information about the posture of the head-mounted display The display control unit changes the position of the virtual camera in the virtual space based on the posture information around a virtual point set in the virtual space, and the posture of the virtual camera in the virtual space The first control mode for changing the position based on the posture information and the position of the virtual point in the virtual space based on the posture information. And a second control mode that changes the position of the virtual camera in the virtual space based on the posture information, and can display an image in a plurality of control modes, according to the first control mode. In the case of displaying an image, the control mode is switched from the first control mode to the second control mode when a change amount of the posture indicated by the posture information becomes a reference amount or more. It is characterized by.

It is explanatory drawing which shows an example of the outline | summary of the head mounted display 1 which concerns on embodiment of this invention. It is explanatory drawing which shows the usage example of the head mounted display. It is explanatory drawing which shows an example of virtual space SP-V. It is explanatory drawing which shows an example of the virtual camera CM in virtual space SP-V. It is explanatory drawing which shows an example of the display image GH. It is explanatory drawing which shows an example of the visual recognition image GS. It is explanatory drawing which shows an example of the virtual camera CM in virtual space SP-V. 2 is a block diagram illustrating an example of a configuration of a terminal device 10. FIG. 2 is a block diagram illustrating an example of a hardware configuration of a terminal device 10. FIG. 4 is a flowchart illustrating an example of the operation of the terminal device 10. 4 is a flowchart illustrating an example of the operation of the terminal device 10. 4 is a flowchart illustrating an example of the operation of the terminal device 10. It is explanatory drawing which shows an example of operation | movement of virtual camera CM in normal control mode. It is explanatory drawing which shows an example of the visual recognition image GS. It is explanatory drawing which shows an example of operation | movement of virtual camera CM in attitude | position control mode. It is explanatory drawing which shows an example of the visual recognition image GS. It is explanatory drawing which shows an example of operation | movement of virtual camera CM in position control mode. It is explanatory drawing which shows an example of the visual recognition image GS. 10 is a flowchart illustrating an example of an operation of the terminal device 10 according to Modification 1. 10 is a flowchart illustrating an example of an operation of the terminal device 10 according to Modification 1. FIG. 16 is a block diagram illustrating an example of a configuration of a display system SYS according to Modification Example 6.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In each figure, the size and scale of each part are appropriately changed from the actual ones. Further, since the embodiments described below are preferable specific examples of the present invention, various technically preferable limitations are attached thereto. However, the scope of the present invention is particularly limited in the following description. Unless otherwise stated, the present invention is not limited to these forms.

[1. Embodiment]
Embodiments of the present invention will be described below.

[1.1. Overview of head-mounted display]
Hereinafter, an example of the outline of the head mounted display 1 (hereinafter referred to as “HMD1”) according to the present embodiment will be described with reference to FIGS. 1 to 7.

  FIG. 1 is an exploded perspective view for explaining an example of the outline of the HMD 1 according to the present embodiment. FIG. 2 is an explanatory diagram for explaining an example of a usage image of the HMD 1 according to the present embodiment.

As shown in FIG. 1, the HMD 1 according to the present embodiment includes a terminal device 10 and a wearing tool 90.
The terminal device 10 includes a display unit 12 for displaying an image. In this embodiment, the case where a smart phone is employ | adopted as the terminal device 10 is assumed as an example. However, the terminal device 10 may be a dedicated display device for being provided in the HMD 1.

The mounting tool 90 is a component for mounting the HMD 1 on the head of the user U as shown in FIG.
As shown in FIG. 1, the mounting tool 90 includes a pair of temples 91L and 91R for mounting the HMD 1 on the head of the user U, a mounting hole 92 for mounting the terminal device 10 to the HMD 1, and the user U using the HMD 1 Is mounted on the head, a pair of through holes 92L and 92R provided so as to correspond to the position where both eyes of the user U exist are provided. A lens may be provided in each of the through holes 92L and 92R. When the user U wears the HMD 1 on the head, the left eye of the user U is inserted into the mounting hole 92 through the through hole 92L or the lens provided in the mounting hole 92. The display unit 12 included in the terminal device 10 can be visually recognized, and the right eye of the user U is inserted into the attachment hole 92 through the through hole 92R or the lens provided in the through hole 92R. The display unit 12 included in the terminal device 10 can be viewed.

As illustrated in FIG. 2, the user U wearing the HMD 1 on the head can change the posture of the HMD 1 by changing the posture of the head. For convenience of explanation, introducing device coordinate system sigma _S is a coordinate system fixed to the HMD 1.
The device coordinate system sigma _S, for example, has an origin at a predetermined position of the HMD 1, _{X S} axis orthogonal to each other, _{Y S} axis, and an orthogonal coordinate system of the three axes having a _{Z S} axis. In the present embodiment, as shown in FIG. 2, when the user U wearing the HMD 1, + X _S direction is the forward direction viewed from the user U, it becomes left direction + Y _S direction as viewed from the user U, + Z _S direction such that the upward direction as viewed from the user U, the case where the apparatus coordinate system sigma _S is set, it is assumed as an example.

As shown in FIG. 2, the user U wearing the HMD1 the head, by changing the posture of the head, the direction of rotation about X _S axis, i.e., as HMD1 in the roll direction Q _X rotates, it is possible to change the attitude of HMD1, the direction of rotation about _{Y S} axis, i.e., as HMD1 the pitch direction _{Q Y} is rotated, it is possible to change the attitude of HMD1, also, _{Z S} direction of rotation about the axis, i.e., as HMD1 in the yaw direction Q _Z is rotated, it is possible to change the attitude of HMD1. That is, the user U who wears the HMD 1 on the head can change the posture of the head so that any rotation in which a part or all of the roll direction Q _X , the pitch direction Q _Y , and the yaw direction Q _Z is synthesized. direction, i.e., so HMD1 is rotated in the rotating direction _{Q W} around any rotational axis _{W S,} we are possible to change the attitude of HMD1.

  The terminal device 10 images the virtual space SP-V with the virtual camera CM, which is a virtual camera existing in the virtual space SP-V, and displays a display image GH that is an image indicating the imaging result on the display unit 12. Let

  FIG. 3 is an explanatory diagram for explaining the virtual space SP-V and the virtual camera CM.

In the present embodiment, as illustrated in FIG. 3, a case where a virtual character V (an example of “predetermined object”) exists in the virtual space SP-V is assumed as an example. Further, in the present embodiment, as illustrated in FIG. 3, it is assumed that the virtual camera CM includes a left-eye virtual camera CM-L and a right-eye virtual camera CM-R as an example.
In the following, for convenience of explanation, as shown in FIG. 3, to introduce a virtual space coordinate system sigma _V is a coordinate system fixed in the virtual space SP-V. A virtual space coordinate system sigma _V, for example, has an origin at a predetermined position in the virtual space SP-V, X _V axes perpendicular to one another, Y _V axis, and, in the orthogonal coordinate system of the three axes having a Z _V axis is there.

4, the virtual space SP-V, when viewed in plan from the + Z _V direction is an explanatory view for explaining the positional relationship between the virtual camera CM and the character V. FIG. 4 illustrates a case where the virtual camera CM images the character V from the front direction of the character V.

Hereinafter, as shown in FIG. 4, the position of the virtual camera CM-L in the virtual space SP-V is referred to as a position PC-L, and the position of the virtual camera CM-R in the virtual space SP-V is referred to as a position PC- The midpoint between the position PC-L and the position PC-R is referred to as R, and is referred to as position PC.
In the following, as shown in FIG. 4, the optical axis direction of the virtual camera CM-L is referred to as an imaging direction LC-L, and the optical axis direction of the virtual camera CM-R is referred to as an imaging direction LC-R. A direction indicated by the sum of a vector indicating the imaging direction LC-L and a vector indicating the imaging direction LC-R is referred to as an imaging direction LC. In the present embodiment, it is assumed that the imaging direction LC-L and the imaging direction LC-R are the same direction. For this reason, in the present embodiment, the imaging direction LC is the same direction as the imaging direction LC-L and the imaging direction LC-R.
A straight line that intersects the position PC and extends in the imaging direction LC is an example of a “reference straight line”.

  FIG. 5 is a diagram illustrating an example of a display image GH that is a result of the virtual camera CM capturing the virtual space SP-V. 5, it is assumed that the virtual camera CM images the character V from the front direction of the character V as shown in FIG.

As shown in FIG. 5, the left eye viewing area 12 -L visible through the through-hole 92 </ b> L in the display unit 12 is captured by the imaging result of the virtual camera CM-L, for example, the virtual camera CM-L. A character image GV-L that is the result of imaging the character V is displayed. In addition, in the display unit 12, in the right eye viewing area 12 -R visible through the through hole 92 R, the imaging result of the virtual camera CM-R, for example, the character V is captured by the virtual camera CM-R. The resulting character image GV-R is displayed.
That is, the user U can visually recognize the character image GV-L with the left eye and can visually recognize the character image GV-R with the right eye. Therefore, as illustrated in FIG. 6, the user U visually recognizes a virtual object existing in the virtual space SP-V such as the character V as a three-dimensional three-dimensional object on the display unit 12. The image GS can be visually recognized.

In the following, for convenience of explanation, as shown in FIG. 6, to introduce the screen coordinate system sigma _D is a fixed coordinate system on the display unit 12. Here, the screen coordinate system sigma _D, when the user U is viewing a viewed image GS, which is a coordinate system for representing positions in the display unit 12. More specifically, the screen coordinate system sigma _D, for example, has an origin at a predetermined position of the display unit 12, a two-axis orthogonal coordinate system having Y _D axis and Z _D axis orthogonal to each other. In the present embodiment, when the terminal device 10 is inserted into the mounting hole 92, Y _D axis and Y _S axis is parallel, also, it is assumed that Z _D axis and Z _S axis is parallel.
In the following, for convenience of explanation, as illustrated in FIG. 7, when the relative positional relationship between the virtual camera CM and the character V in the virtual space SP-V is described, two virtual cameras CM− are used. L and CM-R are expressed as one virtual camera CM. In the following description, the position of the virtual camera CM in the virtual space SP-V is the position PC, and the optical axis direction of the virtual camera CM is the imaging direction LC.
In the following, for convenience of explanation, as shown in FIG. 7, to introduce the camera coordinate system sigma _C is a coordinate system fixed to the virtual camera CM in the virtual space SP-V. The camera coordinate system sigma _C, for example, has an origin at the position PC of the virtual camera CM of the virtual space SP-V, X _C axes perpendicular to one another, Y _C-axis, and, the three axes having a Z _C axis Cartesian coordinate system. In the present embodiment, as viewed from the user U wearing the HMD 1, a _{X C-axis X S} axis in the same direction, _{Y C} axis is the same direction as the _{Y S} axis, _{Z C} axis _{Z S} axis As an example, the case where the direction is the same as is assumed. That is, in this embodiment, X _C-axis, a case extending in the optical axis direction of the virtual camera CM, it is assumed as an example. In FIG. 7, is shown as position different origin of the camera coordinate system sigma _C and position PC, this is a on for convenience of illustration, the same is the origin and the position PC of the camera coordinate system sigma _C It is a position (the same applies to FIGS. 13, 15, and 17 described later).

[1.2. Configuration of terminal device]
Hereinafter, an example of the configuration of the terminal device 10 will be described with reference to FIGS. 8 and 9.

  FIG. 8 is a functional block diagram illustrating an example of the configuration of the terminal device 10.

  As illustrated in FIG. 8, the terminal device 10 includes a control unit 11 that controls each unit of the terminal device 10, a display unit 12 that displays an image, and an operation unit that receives an operation performed by the user U of the terminal device 10. 13, a posture information generation unit 14 that detects posture change of the terminal device 10 and outputs posture information B indicating the detection result, and a storage unit 15 that stores various information including the control program PRG of the terminal device 10. Prepare.

In the present embodiment, for example, a triaxial angular velocity sensor 1002 (see FIG. 9) is employed as the posture information generation unit 14. Specifically, the posture information generation unit 14 includes an X-axis angular velocity sensor for detecting a change in the attitude of the roll direction Q _X per unit time, and Y-axis angular velocity sensor for detecting a posture change in the pitch direction Q _Y per unit time, It comprises a Z-axis angular velocity sensor for detecting a posture change in the yaw direction Q _Z in the unit time, the. The posture information generation unit 14 periodically outputs posture information B indicating detection results by the X-axis angular velocity sensor, the Y-axis angular velocity sensor, and the Z-axis angular velocity sensor.

  The control unit 11 includes a posture information acquisition unit 114 (an example of “acquisition unit”) that acquires posture information B, and a display control unit 110 that generates a display image GH based on the posture information B.

  The display control unit 110 includes a mode specifying unit 111, a virtual camera control unit 112, and a display information generation unit 113.

Based on the attitude information B, the virtual camera control unit 112 controls the virtual camera CM in the virtual space SP-V. In the present embodiment, the virtual camera control unit 112 can control the virtual camera CM in a plurality of control modes. Specifically, the virtual camera control unit 112 can control the virtual camera CM in the normal control mode, control the virtual camera CM in the attitude control mode, and control the virtual camera CM in the position control mode.
Here, the normal control mode is a control mode for controlling the virtual camera CM so that the posture of the virtual camera CM is changed while the position PC of the virtual camera CM in the virtual space SP-V is fixed.
In the attitude control mode, the virtual camera CM is changed so that the attitude of the virtual camera CM in the virtual space SP-V is changed by rotating the virtual camera CM around the virtual point K in the virtual space SP-V. It is a control mode to control.
The position control mode is a control mode for controlling the virtual camera CM so that the position PC of the virtual camera CM is changed while the posture of the virtual camera CM in the virtual space SP-V is fixed.

The mode specifying unit 111 specifies the control mode of the virtual camera control unit 112 based on the attitude information B.
The display information generation unit 113 generates display information DS representing the imaging result of the virtual space SP-V by the virtual camera CM, and supplies the display information DS to the display unit 12, thereby displaying the display image on the display unit 12. Display GH.

  FIG. 9 is a hardware configuration diagram illustrating an example of a hardware configuration of the terminal device 10.

  As illustrated in FIG. 9, the terminal device 10 detects a change in attitude of the processor 1000 (an example of “information processing device”) that controls each unit of the terminal device 10, a memory 1001 that stores various types of information, and the terminal device 10. Then, an angular velocity sensor 1002 that outputs posture information B indicating the detection result, a display device 1003 that can display various images, and an input device 1004 for receiving an operation by the user U of the terminal device 10 are provided.

The memory 1001 includes, for example, a volatile memory such as a RAM (Random Access Memory) that functions as a work area of the processor 1000 and an EEPROM (Electrically Erasable Programmable Read-Only Memory) that stores various information such as the control program PRG of the terminal device 10. ) And the like, and functions as the storage unit 15.
The processor 1000 is a CPU (Central Processing Unit), for example, and functions as the control unit 11 by executing the control program PRG stored in the memory 1001 and operating according to the control program PRG.
As described above, the angular velocity sensor 1002 includes the X-axis angular velocity sensor, the Y-axis angular velocity sensor, and the Z-axis angular velocity sensor, and functions as the posture information generation unit 14.
The display device 1003 and the input device 1004 are touch panels, for example, and function as the display unit 12 and the operation unit 13. Note that the display device 1003 and the input device 1004 may be configured separately. The input device 1004 may be configured by one or a plurality of devices including part or all of a pointing device such as a touch panel, operation buttons, a keyboard, a joystick, and a mouse.

  The processor 1000 includes hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to or instead of the CPU. It may be done. In this case, part or all of the control unit 11 realized by the processor 1000 may be realized by hardware such as a DSP.

[1.3. Operation of terminal device]
Hereinafter, an example of the operation of the terminal device 10 will be described with reference to FIGS. 10 to 18.

  FIGS. 10 to 12 are flowcharts illustrating an example of the operation of the terminal device 10 when the terminal device 10 performs a display process for displaying the display image GH on the display unit 12. In the present embodiment, when the user U inputs a predetermined start operation for starting the display process from the operation unit 13, the terminal device 10 starts the display process.

[1.3.1. Normal control mode]
As shown in FIG. 10, when the display process is started, the virtual camera control unit 112 initializes the attitude of the virtual camera CM in the virtual space SP-V (S100).
For example, in step S100, the virtual camera control unit 112 sets the imaging direction LC of the virtual camera CM in the virtual space SP-V to the + _XV direction as shown in FIG. In other words, the virtual camera control section 112, in step S100, + _{X C} as the direction and the + _{X V} direction is the same direction, to set the camera coordinate system sigma _C.
Note that the virtual camera control unit 112 may initialize the position PC of the virtual camera CM in the virtual space SP-V in step S100. For example, in step S100, the virtual camera control unit 112 displays the position PC of the virtual camera CM in the virtual space SP-V as viewed from the front of the character V, that is, as shown in FIG. You may set to the position of _V direction. In other words, the virtual camera control section 112, in step S100, as the camera coordinate system sigma _C of X _C axis on the character V is present, it may be set the camera coordinate system sigma _C.

  Next, the display information generation unit 113 generates display information DS indicating a result of imaging the virtual space SP-V by the virtual camera CM, and supplies the display information DS to the display unit 12, thereby causing the display unit 12 to display the display information DS. On the other hand, the display image GH is displayed (S102).

Next, the mode designating unit 111 sets the control mode to the normal control mode (S104).
Further, the virtual camera control unit 112 determines the attitude of the HMD 1 when the mode specifying unit 111 sets the control mode to the normal control mode as an “initial attitude” (S106).
In the following, the apparatus coordinate system Σ _S when the HMD 1 is in the initial posture is referred to as “initial coordinate system Σ _S0 ”. In the following, as shown in FIG. 13 to be described later, the position PC of the virtual camera CM when the HMD 1 is in the initial posture is referred to as “initial position PC 0”, and the imaging direction LC when the HMD 1 is in the initial posture. , referred to as "initial imaging direction LC0", the camera coordinate system sigma _C when HMD1 is the initial position, it referred to as "initial camera coordinate system sigma _C0".

Next, the posture information acquisition unit 114 acquires posture information B from the posture information generation unit 14 (S108).
Then, the virtual camera control unit 112 calculates the posture change dB0 from the initial posture of the HMD 1 based on the posture information B acquired by the posture information acquisition unit 114 in step S108 (S110).
In the present embodiment, the virtual camera control section 112, as an example, the posture change DB0, a rotary shaft W _S which defines the direction of rotation Q _W as seen from the initial coordinate system sigma _S0, around the rotation axis W _S a rotation angle theta _W, and be represented by. That is, the direction indicated in the present embodiment, if HMD1 viewed from the initial coordinate system sigma _S0 is rotated by the rotation angle theta _W about an axis of rotation W _S, the posture change DB0, the rotation axis W _S in the initial coordinate system sigma _S0 and comprise a vector, and a value indicating the rotation angle theta _W.
However, the posture change dB0 may be expressed by any other expression method. For example, the posture change dB0 shows the posture change from the initial coordinate system sigma _S0 to the apparatus coordinate system sigma _S, may be those represented by the posture converting a 3 × 3 matrix, the initial coordinate system sigma _S0 shows the change in the posture of the device coordinate system sigma _S from or may be expressed by the quaternion.
In the following, HMD 1 is, when rotated by the rotation angle theta _W about an axis of rotation W _S, the rotation angle theta _W, sometimes referred to as "the attitude change amount".

Next, the virtual camera control unit 112 determines the attitude of the virtual camera CM in the virtual space SP-V based on the attitude change dB0 calculated in step S110 (S112).
Specifically, in step S112, the virtual camera control unit 112 changes the imaging direction LC from the initial imaging direction LC0 by the posture change amount corresponding to the posture change dB0 in the virtual space SP-V. The attitude of the virtual camera CM in the space SP-V is determined. That is, in step S112, the virtual camera control section 112 in the initial camera coordinate system sigma _C0, the camera coordinate system sigma _C, by changing in response to a change in posture DB0, of the virtual camera CM in the virtual space SP-V Determine posture.
More specifically, as an example, the virtual camera control unit 112 sets a virtual rotation axis W _SC0 so as to intersect the initial position PC0 of the virtual camera CM in the virtual space SP-V, as shown in FIG. the camera coordinate system sigma _C, to determine the initial camera coordinate system sigma _C0 as a virtual rotation axis W coordinate system is rotated by the rotation angle theta _W around _SC0, the attitude of the virtual camera CM in the virtual space SP-V decide. Here, components of the vector and the virtual rotation axis W _SC0, which represents the direction of the initial position to a straight line in the virtual space SP-V which intersects the PC0, virtual rotation axis W _SC0 viewed from the initial camera coordinate system sigma _C0 When the components of the vector representing the direction of the axis of rotation W _S viewed from the initial coordinate system sigma _S0 is a straight line as the same.
For example, when the HMD 1 is rotated from the initial posture by the rotation angle θ _{W with} respect to the yaw direction Q _Z around the Z _S axis, the virtual camera control unit 112 is configured so that the initial camera coordinate system intersects the initial position PC0. after setting the virtual rotation axis W _SC0 parallel to Z _C axis of sigma _C0, the camera coordinate system sigma _C, it is rotated an initial camera coordinate system sigma _C0 virtual rotation axis W _SC0 about by the rotation angle theta _W posture Is determined as a coordinate system having the position of the virtual camera CM.
Further, for example, HMD 1 is from the initial position, it rotates by the rotation angle theta _W with respect to the pitch direction Q _Y around Y _S axis, the virtual camera control section 112, so as to cross to the initial position PC0, initial camera after setting the virtual rotation axis W _SC0 parallel to Y _C-axis of the coordinate system sigma _C0, the camera coordinate system sigma _C, rotates the initial camera coordinate system sigma _C0 virtual rotation axis W _SC0 about by the rotation angle theta _W By determining the coordinate system having the determined posture, the posture of the virtual camera CM is determined.
For example, as HMD1 found from the initial position, it rotates by the rotation angle theta _W with respect to the roll direction Q _X around X _S axis, the virtual camera control section 112, intersects the initial position PC0, initial camera after setting the virtual rotation axis W _SC0 parallel to X _C-axis of the coordinate system sigma _C0, the camera coordinate system sigma _C, rotates the initial camera coordinate system sigma _C0 virtual rotation axis W _SC0 about by the rotation angle theta _W By determining the coordinate system having the determined posture, the posture of the virtual camera CM is determined.

  Next, the display information generation unit 113 generates display information DS indicating a result of imaging the virtual space SP-V by the virtual camera CM, and supplies the display information DS to the display unit 12, thereby causing the display unit 12 to display the display information DS. On the other hand, the display image GH is displayed (S114).

FIG. 13 is an explanatory diagram for explaining an example of a change in posture of the virtual camera CM in the normal control mode. FIG. 14 is a diagram illustrating an example of a visual image GS displayed on the display unit 12 in the normal control mode.
In Figure 13, as an example, at time t1, HMD 1 becomes the initial position at time t2, the HMD 1, comprising the initial position, the rotation angle theta _W by the rotation position with respect to the yaw direction _{Q Z} around _{Z S} axis Assume a case.
Thus, for example, as shown in FIG. 13, the virtual camera CM is, at time t1, if the initial imaging direction LC0 imaging direction LC (t1) becomes the posture facing + X _V direction, the virtual camera CM is at time t2, the imaging direction LC (t1), and around the Z _C parallel imaginary rotation axis to axis W _SC0 initial camera coordinate system sigma _C0 intersects the initial position PC0 is rotated by the rotation angle theta _W, imaging The posture is such that it has a direction LC (t2). In other words, the camera coordinate system sigma _C at time t2, the initial camera coordinate system sigma _C0 is a camera coordinate system sigma _C at time t1, the initial camera coordinate system sigma _C0 intersects the origin of the initial camera coordinate system sigma _C0 It becomes the coordinate system is rotated by the rotation angle theta _W around the Z _C imaginary axis of rotation which is parallel to the axis W _SC0.
Therefore, when the character image GV is displayed at the center Ymid of the display unit 12 as shown in FIG. 6 at time t1, the character image GV is displayed at time t2 as shown in FIG. It will be displayed in than the central portion Ymid of the display section 12 + Y _D direction position Y (t2).

As illustrated in FIG. 10, the mode designating unit 111 determines whether to shift the control mode to the posture control mode based on the posture information B acquired in step S108 or the posture change dB0 calculated in step S110. Is determined (S116).
In the present embodiment, as an example, when the posture change amount of the HMD 1 in the determination period from a time before the current time to the current time is equal to or less than a predetermined threshold, the mode specifying unit 111 sets the control mode. Assume that the normal control mode is shifted to the attitude control mode. Therefore, in step S116, for example, the mode specifying unit 111 shifts the control mode to the posture control mode by determining whether or not the posture change amount of the HMD1 in the determination period is equal to or less than a predetermined threshold. It is determined whether or not. In step S116, for example, the mode designating unit 111 determines whether or not the difference between the attitude of the HMD1 at the start time of the determination period and the attitude of the HMD1 at an arbitrary time within the determination period is smaller than a predetermined threshold value. It may be determined whether or not to shift the control mode to the attitude control mode.
Then, if the result of determination in step S116 is affirmative, mode designating section 111 advances the process to step S120, and if the result of determination in step S116 is negative, advances the process to step S118.

  Thereafter, the control unit 11 determines whether or not the user U has input a predetermined end operation for ending the display process from the operation unit 13 (S118). And the control part 11 advances a process to step S108, when the result of determination in step S118 is negative, and complete | finishes a display process, when the result of determination in step S118 is affirmative.

[1.3.2. Attitude control mode]
As shown in FIG. 11, the mode specifying unit 111 sets the control mode to the attitude control mode (S120).

Next, the virtual camera control unit 112 determines the attitude of the HMD 1 when the mode specifying unit 111 sets the control mode to the attitude control mode as a “reference attitude” (S122).
In the following, the apparatus coordinate system Σ _S when the HMD 1 is in the reference posture is referred to as a “reference coordinate system Σ _S1 ”. In the following, as shown in FIG. 15 described later, the position PC of the virtual camera CM when the HMD1 is in the reference posture is referred to as “reference position PC1”, and the imaging direction LC when the HMD1 is in the reference posture. The camera coordinate system Σ _C when the HMD 1 is in the reference posture is referred to as “reference camera coordinate system Σ _C1 ”.

Next, the virtual camera control unit 112 sets a virtual point K in the virtual space SP-V based on the reference imaging direction LC1 of the virtual camera CM when the HMD1 is in the reference posture (S124).
Specifically, in step S124, the virtual camera control unit 112 is, for example, on a straight line that intersects the reference position PC1 of the virtual camera CM and extends in the reference imaging direction LC1 and exists in the virtual space SP-V. A virtual point K is set in a setting area corresponding to the existence area of the character V to be played.
Here, the existence area of the character V is, for example, an area in the virtual space SP-V and includes the surface of the character V and the inside of the character V. The setting area is, for example, a part or all of the existing area. However, the setting area may be an area including an area outside the existing area. For example, the setting area may be an area where the distance from the existing area is a predetermined distance or less, or may be an area where the distance from a point in the existing area is a predetermined distance or less.
Hereinafter, as shown in FIG. 15 described later, the position PK of the virtual point K in the virtual space SP-V set in step S124 is referred to as “reference position PK1”.
In the present embodiment, the virtual camera control unit 112 determines the virtual point K based on the reference imaging direction LC1, but the present invention is not limited to such an aspect. For example, the virtual camera control unit 112 may set the virtual point K at a predetermined position in the virtual space SP-V. Further, for example, the virtual camera control unit 112 sets a virtual point K at a position corresponding to a specific part of the character V such as the face of the character V in the existence area of the character V in the virtual space SP-V. May be.

Next, the posture information acquisition unit 114 acquires posture information B from the posture information generation unit 14 (S126).
Then, the virtual camera control unit 112 calculates a posture change dB1 from the reference posture of the HMD 1 based on the posture information B acquired by the posture information acquisition unit 114 in step S126 (S128).
In the present embodiment, when viewed in the reference coordinate system sigma _S1, rotation HMD1 is, from the reference position, it rotates by the rotation angle theta _W about an axis of rotation W _S, the posture change dB1, in the reference coordinate system sigma _S1 the direction vector showing the axial W _S, and include, a value indicating the rotation angle theta _W.

As illustrated in FIG. 11, the mode specifying unit 111 determines whether or not to shift the control mode to the position control mode based on the posture information B acquired in step S126 or the posture change dB1 calculated in step S128. Is determined (S130).
In the present embodiment, as an example, when the posture change amount from the reference posture of the HMD 1 is equal to or larger than the reference amount θth (an example of “reference angle”), the mode specifying unit 111 changes the control mode from the posture control mode. Assume a case of shifting to the position control mode. For this reason, the mode designating unit 111 shifts the control mode to the position control mode by determining whether or not the posture change amount from the reference posture of the HMD 1 is equal to or larger than the reference amount θth in step S130, for example. It is determined whether or not.
In step S130, the mode designation unit 111 determines whether or not the posture change amount related to a predetermined direction component of the posture change dB1 from the reference posture of the HMD1 is equal to or larger than the reference amount θth. It may be determined whether or not the mode is shifted to the position control mode. Specifically, the mode designation section 111, in step S130, for example, of the posture change dB1 from the reference posture of the HMD 1, the direction component in the pitch direction _{Q Y,} direction component in the yaw direction _{Q Z} or Pitch Q posture variation of the direction component of the _Y and the yaw direction Q _Z is, by determining whether a reference amount θth above, the control mode may determine whether to transition to the position control mode .
Then, if the result of determination in step S130 is affirmative, mode designating section 111 advances the process to step S140, and if the result of determination in step S130 is negative, advances the process to step S132.

As illustrated in FIG. 11, the mode designating unit 111 determines whether or not to shift the control mode to the normal control mode based on the posture information B acquired in step S126 or the posture change dB1 calculated in step S128. Is determined (S132).
In the present embodiment, as an example, when the posture change amount of the HMD 1 in the determination period from a time before the current time to the current time is equal to or less than a predetermined threshold, the mode specifying unit 111 sets the control mode. Assume that the posture control mode is shifted to the normal control mode. For this reason, the mode designating unit 111 shifts the control mode to the normal control mode in step S132 by, for example, determining whether the posture change amount of the HMD1 in the determination period is equal to or less than a predetermined threshold. It is determined whether or not. In step S132, for example, the mode specifying unit 111 determines whether or not the difference between the attitude of the HMD1 at the start time of the determination period and the attitude of the HMD1 at an arbitrary time within the determination period is smaller than a predetermined threshold value. It may be determined whether or not to shift the control mode to the normal control mode.
In step S132, the mode specifying unit 111 determines whether or not the posture change amount related to a specific direction component of the posture change dB1 from the reference posture of the HMD1 is greater than or equal to a predetermined value. It may be determined whether or not to shift to the normal control mode. Specifically, the mode designation section 111, in step S132, for example, whether of the posture change dB1 from the reference posture of the HMD 1, the posture variation of the direction component of the roll direction Q _X is equal to or more than a predetermined value It may be determined whether to shift the control mode to the normal control mode.

Next, the virtual camera control unit 112 determines the position and orientation of the virtual camera CM in the virtual space SP-V based on the orientation change dB1 calculated in step S128 (S134).
Specifically, in step S134, the virtual camera control unit 112 changes the position PC and the imaging direction LC in the virtual space SP-V from the reference position PC1 and the reference imaging direction LC1 to the attitude change amount corresponding to the attitude change dB1. By changing only the position, the position and orientation of the virtual camera CM in the virtual space SP-V are determined. That is, in step S134, the virtual camera control section 112, the reference camera coordinate system sigma _C1, the camera coordinate system sigma _C, by changing in response to a change in posture dB1, the virtual camera CM in the virtual space SP-V Determine position and orientation.
More specifically, as an example, the virtual camera control unit 112 sets the reference rotation axis _WSC1 so as to intersect the virtual point K in the virtual space SP-V, as shown in FIG. the sigma _C, to determine the reference camera coordinate system sigma _C1 as a coordinate system that has only rotated the rotation angle theta _W around the reference rotation axis W _SC1, to determine the position and orientation of a virtual camera CM in the virtual space SP-V . Here, the reference rotation axis W _SC1, a straight virtual space SP-V which intersects the reference position PK1 virtual point K, the direction of the reference rotation axis W _SC1 when viewed in the reference camera coordinate system sigma _C1 and components of the vector representing the components of the vector representing the direction of the axis of rotation W _S when viewed in the reference coordinate system sigma _S1 is a straight line as the same. That is, as an example, as shown in FIG. 15, the virtual camera control unit 112 moves the virtual camera CM located at the reference position PC1 around the reference rotation axis _WSC1 around the virtual point K in the virtual space SP-V. By rotating the rotation angle θ _W, the position PC of the virtual camera CM in the virtual space SP-V is determined.

  Next, the display information generation unit 113 generates display information DS indicating a result of imaging the virtual space SP-V by the virtual camera CM, and supplies the display information DS to the display unit 12, thereby causing the display unit 12 to display the display information DS. On the other hand, the display image GH is displayed (S136).

FIG. 15 is an explanatory diagram for explaining an example of changes in the position and posture of the virtual camera CM in the posture control mode. FIG. 16 is a diagram illustrating an example of a visual image GS displayed on the display unit 12 in the posture control mode.
In Figure 15, as an example, at time t3, becomes HMD1 reference posture at time t4, is HMD1, the reference posture, the rotation angle theta _W by the rotation position with respect to the yaw direction _{Q Z} around _{Z S} axis Assume a case.
Thus, for example, as shown in FIG. 15, when the virtual camera CM is, at time t3, the reference is an image pickup direction LC1 imaging direction LC (t3) is the posture facing + X _V direction, the virtual camera CM at time t4, the imaging direction LC (t3), is rotated around the Z _C reference rotation axis parallel to the axis W _SC1 of the base camera coordinate system sigma _C1 intersects the virtual point K by the rotation angle theta _W Then, the posture has an imaging direction LC (t4). In other words, the camera coordinate system sigma _C is at time t4, parallel reference camera coordinate system sigma _C1 is a camera coordinate system sigma _C at time t3, the Z _C axis of the base camera coordinate system sigma _C1 intersects the virtual point K It made around a reference rotation axis W _SC1 to the rotation angle theta _W only coordinate system is rotated.
As shown in FIG. 15, the position PC (t4) of the virtual camera CM at time t4 is, for example, the position PC (t3) of the virtual camera CM at time t3, that is, the reference position PC1 is centered on the virtual point K. a position around the reference rotation axis W _SC1 is rotated by the rotation angle theta _W to.
Therefore, when the character image GV represents the character V facing in the front direction in the central portion Ymid of the display unit 12 as shown in FIG. 6 at time t3, as shown in FIG. The character image GV represents the character V whose direction is changed by the rotation angle θ _W from the front direction in the central portion Ymid of the display unit 12.

  Thereafter, the control unit 11 determines whether or not the user U has input a predetermined end operation for ending the display process from the operation unit 13 (S138). And the control part 11 advances a process to step S126, when the result of determination in step S138 is negative, and complete | finishes a display process, when the result of determination in step S138 is affirmative.

[1.3.3. Position control mode]
As shown in FIG. 12, the mode designating unit 111 sets the control mode to the position control mode (S140).

Next, as shown in FIG. 17 to be described later, the virtual camera control unit 112 determines the position PC of the virtual camera CM when the mode specifying unit 111 sets the control mode to the position control mode as “starting position PC2”. (S142). In the following, the attitude of the HMD 1 when the mode specifying unit 111 sets the control mode to the position control mode is referred to as “starting attitude”, and the imaging direction LC when the HMD 1 is the starting attitude is referred to as “imaging direction LC 2. And the camera coordinate system Σ _C when the HMD 1 is in the starting posture is referred to as “starting camera coordinate system Σ _C2 ”.
When the mode specifying unit 111 sets the control mode to the position control mode, the virtual point K exists on a straight line that intersects the starting position PC2 of the virtual camera CM and extends in the imaging direction LC2. Hereinafter, as shown in FIG. 17 to be described later, the position PK of the virtual point K in the virtual space SP-V when the mode specifying unit 111 sets the control mode to the position control mode is referred to as “starting position PK2”. . The starting position PK2 is the same position as the reference position PK1.

Next, the posture information acquisition unit 114 acquires posture information B from the posture information generation unit 14 (S144).
Then, based on the posture information B acquired by the posture information acquisition unit 114 in step S126 and the posture information B acquired by the posture information acquisition unit 114 in step S144, the virtual camera control unit 112 determines from the reference posture of the HMD1. The posture change dB2 is calculated (S146).
In the present embodiment, when viewed in the reference coordinate system sigma _S1, rotation HMD1 is, from the reference position, it rotates by the rotation angle theta _W about an axis of rotation W _S, the posture change dB2, in the reference coordinate system sigma _S1 the direction vector showing the axial W _S, and include, a value indicating the rotation angle theta _W.

As shown in FIG. 12, the mode specifying unit 111 shifts the control mode to the attitude control mode based on the attitude information B acquired in step S126 and step S144 or the attitude change dB2 calculated in step S146. It is determined whether or not to perform (S148).
In the present embodiment, as an example, when the posture change amount from the reference posture of the HMD 1 is less than the reference amount θth, the mode specifying unit 111 shifts the control mode from the position control mode to the posture control mode. Is assumed. For this reason, the mode designating unit 111 shifts the control mode to the posture control mode by determining whether or not the posture change amount from the reference posture of the HMD1 is less than the reference amount θth in step S148, for example. It is determined whether or not.
In step S148, the mode designation unit 111 determines whether or not the posture change amount related to a predetermined direction component of the posture change dB2 from the reference posture of the HMD1 is less than the reference amount θth. It may be determined whether or not the mode is shifted to the attitude control mode. Specifically, the mode designation section 111, in step S148, for example, of the posture change dB2 from the reference posture of the HMD 1, the direction component in the pitch direction _{Q Y,} direction component in the yaw direction _{Q Z} or Pitch Q posture variation of the direction component of the _Y and the yaw direction Q _Z is, by determining whether or not less than the reference amount [theta] th, the control mode may determine whether or not to shift to the attitude control mode .
Then, if the result of determination in step S148 is affirmative, mode designating section 111 advances the process to step S120, and if the result of determination in step S148 is negative, advances the process to step S150.

Next, the virtual camera control unit 112 determines the position PK of the virtual point K in the virtual space SP-V based on the posture change dB2 calculated in step S146 (S150).
Specifically, in step S150, the virtual camera control unit 112 is, for example, a direction that intersects both the reference rotation axis _WSC1 and the imaging direction LC2 set in the virtual space SP-V, and the reference position PC1. The position PK of the virtual point K is changed in the direction away from the center. More specifically, for example, the virtual camera control unit 112 sets the position PK of the virtual point K in a direction orthogonal to both the reference rotation axis _WSC1 and the imaging direction LC2 and away from the reference position PC1. Change. In this case, the virtual camera control unit 112, for example, the rotation angle θ included in the posture change dB2 as shown in FIG. 17 described later, such as a change from the starting position PK2, that is, the position PK (t5) to the position PK (t6). The position PK of the virtual point K is changed by the distance Dis corresponding to the excess angle θov obtained by subtracting the reference amount θth from _W.

Next, the virtual camera control unit 112 determines the position PC of the virtual camera CM in the virtual space SP-V based on the posture change dB2 calculated in step S146 (S152). In the present embodiment, in the position control mode, the imaging direction LC of the virtual camera CM is maintained at the imaging direction LC2.
Specifically, in step S152, the virtual camera control unit 112 is, for example, a direction intersecting both the reference rotation axis _WSC1 and the imaging direction LC2 set in the virtual space SP-V, and the reference position PC1. The position PC of the virtual camera CM is changed in the direction away from the camera. More specifically, the virtual camera control unit 112, for example, sets the position PC of the virtual camera CM in a direction orthogonal to both the reference rotation axis _WSC1 and the imaging direction LC2 and away from the reference position PC1. Change. In this case, the virtual camera control unit 112, for example, only a distance Dis corresponding to the excess angle θov as shown in FIG. 17 described later, such as a change from the starting position PC2, that is, the position PC (t5) to the position PC (t6). The position PC of the virtual camera CM is changed.
That is, the position change of the virtual point K in step S150 and the position change of the virtual camera CM in step S152 have directions orthogonal to the reference rotation axis _WSC1 and the imaging direction LC2, as shown in FIG. And the same vector VI having the magnitude Dis. In other words, in the position control mode, the virtual camera control unit 112 has the virtual point K located in the imaging direction LC2 when viewed from the virtual camera CM, and the distance between the virtual camera CM and the virtual point K is a constant distance. So that the position PK of the virtual point K and the position PC of the virtual camera CM are changed. Thus, the virtual camera control unit 112 changes the position PK of the virtual point K in the virtual plane perpendicular to the reference rotation axis _WSC1 in the position control mode.
Note that the virtual camera control unit 112 may execute the process of step S150 and the process of step S152 at the same time, or may execute the process of step S152 before executing the process of step S150.

  Next, the display information generation unit 113 generates display information DS indicating a result of imaging the virtual space SP-V by the virtual camera CM, and supplies the display information DS to the display unit 12, thereby causing the display unit 12 to display the display information DS. On the other hand, the display image GH is displayed (S154).

FIG. 17 is an explanatory diagram for explaining an example of a change in the position and orientation of the virtual camera CM in the position control mode. FIG. 18 is a diagram illustrating an example of a visual image GS displayed on the display unit 12 in the position control mode.
In FIG. 17, as an example, at time t3, HMD1 becomes the reference posture, at time t5, HMD1 becomes the starting posture, and at time t6, HMD1 changes the reference posture with respect to the yaw direction Q _Z around the Z _S axis. , it is assumed that the only rotational posture large rotation angle theta _W than the reference amount [theta] th.
Thus, for example, as shown in FIG. 17, when the virtual camera CM is, at time t3, the reference is an image pickup direction LC1 imaging direction LC (t3) is the posture facing + X _V direction, the virtual camera CM, at time t5, the imaging direction LC (t3), is rotated by the reference amount θth around parallel reference rotation axis W _SC1 to Z _C axis of the base camera coordinate system sigma _C1 intersects the virtual point K, The posture has an imaging direction LC (t5) which is the imaging direction LC2, and the virtual camera CM has an orientation having an imaging direction LC (t6) which is the imaging direction LC2 at time t6. In other words, the starting point the camera coordinate system sigma _C2 is a camera coordinate system sigma _C at time t5 is the reference camera coordinate system sigma _C1 is a camera coordinate system sigma _C at time t3, the reference camera coordinate system intersects the virtual point K becomes the coordinate system is rotated by the reference amount θth around parallel reference rotation axis W _SC1 to Z _C axis of sigma _C1. The camera coordinate system sigma _C at time t6, the origin camera coordinate system sigma _C2, becomes the coordinate system is only translated direction and distance indicated by vector VI.
As shown in FIG. 17, the position PC (t5) of the virtual camera CM at time t5 is based on the virtual position K with the position PC (t3) of the virtual camera CM at time t3, that is, the reference position PC1. is a position rotated by reference quantity θth about the axis of rotation W _SC1, the starting point position PC2. As shown in FIG. 17, the position PC (t6) of the virtual camera CM at time t6 is the position PC (t5) of the virtual camera CM at time t5, that is, the position obtained by moving the starting position PC2 by the vector VI. It becomes. Further, as shown in FIG. 17, the position PK (t6) of the virtual point K at time t6 is the position PK (t5) of the virtual point K at time t5, that is, the position obtained by moving the starting position PK2 by the vector VI. It becomes.
Therefore, at time t3, as shown in FIG. 6, when the character image GV represents the character V facing in the front direction in the central portion Ymid of the display unit 12, the character image GV is shown in FIG. As shown in FIG. 18, the character V whose direction is changed by the reference amount θth from the front direction in the central portion Ymid of the display unit 12 is represented, and the character image GV is displayed at the time t6 as shown in FIG. in -Y _D direction position Y (t6) than the central portion Ymid of 12 representing the character V of changing the orientation by reference amount θth from the front.

  Thereafter, the control unit 11 determines whether or not the user U has input a predetermined end operation for ending the display process from the operation unit 13 (S156). And the control part 11 advances a process to step S144, when the result of determination in step S156 is negative, and complete | finishes a display process, when the result of determination in step S156 is affirmative.

In step S148, the mode designating unit 111 determines whether or not to shift the control mode to the posture control mode by determining whether or not the posture change amount from the reference posture of the HMD1 is less than the reference amount θth. However, the present invention is not limited to such an embodiment.
For example, in step S148, the mode specifying unit 111 determines whether or not the posture change amount related to a specific direction component of the posture change dB2 from the reference posture of the HMD1 is equal to or greater than a predetermined value. It may be determined whether or not to shift to the attitude control mode. Specifically, the mode designation section 111, in step S148, for example, whether of the posture change dB2 from the reference posture of the HMD 1, the posture variation of the direction component of the roll direction Q _X is equal to or more than a predetermined value It may be determined whether or not to shift the control mode to the attitude control mode.
Further, in step S148, the mode designation unit 111 determines whether or not to shift the control mode to the posture control mode by determining whether or not the posture change amount of the HMD1 in the determination period is equal to or less than a predetermined threshold. May be determined. Further, in step S148, for example, the mode specifying unit 111 determines whether or not the difference between the attitude of the HMD1 at the start time of the determination period and the attitude of the HMD1 at an arbitrary time within the determination period is smaller than a predetermined threshold value. It may be determined whether or not to shift the control mode to the attitude control mode.
In these cases, the virtual camera control unit 112 determines the position PK of the virtual point K and the virtual camera CM when the posture change amount from the reference posture of the HMD 1 becomes less than the reference amount θth in steps S150 and S152. The position PC may be changed in a direction orthogonal to both the reference rotation axis _WSC1 and the imaging direction LC2 and closer to the reference position PC1.

[1.4. Conclusion of the embodiment]
As described above, in the present embodiment, the display control unit 110 performs the normal control mode in which the posture of the virtual camera CM is changed based on the posture information B, and the position and posture of the virtual camera CM based on the posture information B. The visual image GS can be displayed in the attitude control mode for changing the position of the virtual camera CM and the position control mode for changing the position of the virtual camera CM based on the attitude information B. Therefore, according to the present embodiment, the user U can operate the position and posture of the virtual camera CM in the virtual space SP-V by changing the posture of the HMD 1.

  Further, in the present embodiment, the display control unit 110 changes the position and posture of the virtual camera CM around the virtual point K set in the setting region corresponding to the region where the character V exists, and is visually recognized in the posture control mode. The image GS can be displayed. Therefore, according to this embodiment, the user U wearing the HMD 1 can continuously display the character image GV on the display unit 12.

  In the present embodiment, the attitude control mode is an example of a “first control mode”, and the position control mode is an example of a “second control mode”.

[2. Modified example]
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined within a range that does not contradict each other. In addition, about the element in which an effect | action and a function are equivalent to embodiment in the modification illustrated below, the detailed description of each is abbreviate | omitted suitably using the code | symbol referred by the above description.

[Modification 1]
In the embodiment described above, the display control unit 110 can display the visual image GS in three control modes: the normal control mode, the attitude control mode, and the position control mode. It is not limited to. The display control unit 110 only needs to be able to display the visual image GS in at least two control modes among the three control modes of the normal control mode, the attitude control mode, and the position control mode. For example, the display control unit 110 may be capable of displaying the visual image GS in two control modes, a normal control mode and a position control mode.

19 and 20 are flowcharts illustrating an example of the operation of the terminal device 10 when the display process according to the present modification is executed. The flowcharts shown in FIGS. 19 and 20 are shown in FIGS. 10 to 12 except that the control unit 11 executes the process of step S160 and does not execute the processes of steps S116 and S120 to S138. This is the same as the flowchart.
As illustrated in FIG. 19, the mode specifying unit 111 determines whether to shift the control mode to the position control mode based on the posture information B acquired in step S108 or the posture change dB0 calculated in step S110. Is determined (S160).
Specifically, in step S160, the mode specifying unit 111 determines that the mode specifying unit 111 changes the control mode from the normal control mode when the posture change amount from the initial posture of the HMD1 is equal to or larger than the reference amount θth. Switch to control mode. That is, in step S160, for example, whether the mode designation unit 111 shifts the control mode to the position control mode by determining whether or not the posture change amount from the initial posture of the HMD1 is equal to or larger than the reference amount θth. Determine whether or not.
Then, if the result of determination in step S160 is affirmative, mode designating section 111 proceeds to step S140, and if the result of determination in step S160 is negative, the process proceeds to step S112.
As shown in FIG. 20, the virtual camera control unit 112 advances the process to step S104 when the result of determination in step S148 is affirmative, and performs processing when the result of determination in step S148 is negative. Advances to step S150.

  Also in this modification, the display control unit 110 can display the visual image GS in two control modes, the normal control mode and the position control mode. Therefore, the user U can operate the position and posture of the virtual camera CM in the virtual space SP-V by changing the posture of the HMD 1.

  In the present modification, the normal control mode is an example of a “first control mode”, and the position control mode is an example of a “second control mode”.

[Modification 2]
In the embodiment and the modification described above, the imaging direction LC and the direction of the straight line connecting the virtual camera CM and the virtual point K coincide with each other, but the present invention is not limited to such an aspect. For example, the angle formed by the imaging direction LC and a straight line connecting the virtual camera CM and the virtual point K may be a predetermined angle or less.

[Modification 3]
In the embodiment and the modification described above, the virtual point K can be moved outside the existence area and the setting area of the character V in the position control mode, but the present invention is not limited to such an aspect. . In the position control mode, the virtual camera control unit 112 may limit the change in the position PC of the virtual camera CM so that the virtual point K is located inside the existence area or the setting area.

[Modification 4]
In the embodiment and the modification described above, the virtual camera control unit 112 determines the distance Dis based on the excess angle θov, but the present invention is not limited to such an aspect. Virtual camera control section 112, for example, the distance Dis is, as a value corresponding to the time length rotation angle theta _W is equal to or greater than a reference amount θth contained in attitude change dB2, position PC of the virtual camera CM and The position PK of the virtual point K may be determined.
For example, the virtual camera control section 112, in the position control mode, when the rotation angle theta _W contained in the posture change dB2 is equal to or greater than the reference amount [theta] th, the position PK position PC and the virtual point K of the virtual camera CM Further, it may be moved at a constant speed in the vector VI direction.
Further, the virtual camera control section 112, in the position control mode, when the rotation angle theta _W contained in the posture change dB2 is equal to or greater than the reference amount [theta] th, the position PK position PC and the virtual point K of the virtual camera CM Further, it may be moved in the vector VI direction at a speed corresponding to the excess angle θov.

[Modification 5]
In the embodiment and the modification described above, the posture information B indicates the detection result of the posture change of the terminal device 10, but the present invention is not limited to such a mode. The posture information B may be information indicating the posture of the terminal device 10 viewed from a coordinate system fixed on the ground, for example.
In this case, the posture information generation unit 14 may be configured to include one or both of an angular velocity sensor and a geomagnetic sensor, for example. In this case, the posture information B may be, for example, image information provided outside the HMD 1 and output from an imaging device that captures the HMD 1.

[Modification 6]
In the embodiment and the modification described above, the information processing apparatus is provided in the HMD 1, but the information processing apparatus may be provided separately from the HMD 1.

FIG. 21 is a block diagram showing an example of the configuration of the display system SYS according to this modification.
As shown in FIG. 21, the display system SYS includes an information processing device 20 and a head mounted display 1 </ b> A that can communicate with the information processing device 20. Among these, the information processing apparatus 20 may include, for example, the control unit 11, the operation unit 13, and the storage unit 15. In addition to the display unit 12 and the posture information generation unit 14, the head mounted display 1A includes an operation unit 31 that receives an operation by the user U wearing the head mounted display 1A, a storage unit 32 that stores various types of information, May be provided.

[3. Addendum]
From the above description, the present invention is grasped as follows, for example. In order to facilitate understanding of each aspect, reference numerals in the drawings are appended in parentheses for convenience in the following, but the present invention is not limited to the illustrated aspects.

[Appendix 1]
A program for an information processing device according to an aspect of the present invention is a program for an information processing device including a processor, wherein the processor is an image obtained by capturing a virtual space with a virtual camera and uses binocular parallax. A display control unit that displays a stereoscopic image on a display unit provided in a head-mounted display, and an acquisition unit that acquires posture information related to the posture of the head-mounted display function, and the display control unit The position of the virtual camera in the virtual space is changed based on the posture information around a virtual point set in the virtual space, and the posture of the virtual camera in the virtual space is changed based on the posture information. A first control mode for changing the position of the virtual point in the virtual space based on the posture information, An image can be displayed in a plurality of control modes including a second control mode that changes the position of the virtual camera based on the posture information, and the image is displayed in the first control mode. In this case, the control mode is switched from the first control mode to the second control mode when a change amount of the posture indicated by the posture information is equal to or greater than a reference amount.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Moreover, according to this aspect, in the first control mode and the second control mode, the position of the virtual camera in the virtual space can be changed based on the posture information. That is, according to this aspect, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, compared with the case where the position of the virtual camera in the virtual space is changed by changing the position of the head mounted display in the real space, the user wearing the head mounted display can easily The position of the virtual camera in the virtual space can be changed.
Further, according to this aspect, in the first control mode, the position of the virtual camera in the virtual space is changed around the virtual point based on the posture information, and in the second control mode, based on the posture information, the virtual space is changed. The position of the virtual point in is changed, and the position of the virtual camera in the virtual space is changed based on the position of the virtual point. For this reason, according to this aspect, the user wearing the head mounted display can continue to see the vicinity of the virtual point in the virtual space. Thus, according to this aspect, for example, a user wearing a head-mounted display sees a predetermined object in a virtual space such as an opponent in a battle-type fighting game or a racing car in a race game. When it is necessary to continue, the user who wears the head mounted display controls the posture of the head mounted display so that the virtual point is set at the position corresponding to the predetermined object in the virtual space. It is possible to continuously display an area where a predetermined object exists in the virtual space on the display unit of the display.

  In the above aspect, the “virtual camera” refers to, for example, a first virtual camera that images the virtual space and a second virtual image that captures the virtual space from a position different from the first virtual camera in the virtual space. And a virtual camera. In this case, the “stereoscopic image” refers to, for example, a left-eye image obtained by capturing the virtual space with the first virtual camera and viewed by the user with the left eye, and the virtual space to the second virtual camera. And an image for the right eye for the user to visually recognize with the right eye.

  In the above aspect, the “head mounted display” may be, for example, a display device that can be mounted on the head. Specifically, the “head-mounted display” may be a goggle-type or glasses-type display device that can be attached to the head. In addition, the “head mounted display” may include, for example, a wearing device that can be worn on the head and a portable display device such as a smartphone that is attached to the wearing device. Good.

  In the above aspect, the “head mounted display attitude” may be, for example, the orientation of the head mounted display, the tilt of the head mounted display, or the orientation and inclination of the head mounted display. It may be a concept including both. Here, the “direction of the head-mounted display” may be, for example, a direction in which the head-mounted display is facing in a horizontal plane in real space, or an angle formed by the reference direction of the head-mounted display and the magnetic north direction. There may be. Further, the “tilt of the head mounted display” may be an angle formed by the reference direction of the head mounted display and the vertical direction, for example.

  In the above aspect, the “posture information” may be, for example, information indicating the posture of the head mounted display or information indicating the posture change of the head mounted display.

  In the above aspect, for example, the “acquisition unit” may acquire posture information from a head mounted display, or may acquire posture information from an imaging device that images the head mounted display. When the acquisition unit acquires the posture information from the head mounted display, the head mounted display may include a sensor for detecting information indicating the posture change of the head mounted display, and indicates the posture of the head mounted display. A sensor for detecting information may be provided. Here, the “sensor for detecting information indicating the posture change of the head mounted display” may be, for example, an angular velocity sensor. The “sensor for detecting information indicating the posture of the head mounted display” may be, for example, one or both of a geomagnetic sensor and an angular velocity sensor. In addition, when the acquisition unit acquires posture information from an imaging device that images a head-mounted display, the posture information may be an image indicating a result of imaging the head-mounted display by the imaging device, for example.

[Appendix 2]
An information processing apparatus program according to another aspect of the present invention is the information processing apparatus program according to attachment 1, wherein the display control unit displays the virtual camera and the virtual point in the first control mode. The position and orientation of the virtual camera in the virtual space such that an angle formed between a connecting line segment and a reference straight line in the virtual space determined based on an image displayed on the display unit is equal to or smaller than a predetermined angle. It is characterized by changing.

  According to this aspect, the angle formed by the direction captured by the virtual camera in the virtual space and the direction of the virtual point viewed from the virtual camera can be set to a predetermined angle or less, for example. Therefore, according to this aspect, for example, when a user wearing a head mounted display needs to continue to look at a predetermined object in the virtual space, the virtual point is located at a position corresponding to the predetermined object in the virtual space. By setting the position of the head mounted display, the user wearing the head mounted display can continuously display the area where the predetermined object exists in the virtual space on the display unit of the head mounted display. It can be displayed.

  In the above aspect, the “reference straight line” may be, for example, a straight line connecting a virtual camera and a specific portion in a virtual space displayed in a specific pixel in an image displayed on the display unit. Good. In this aspect, the “reference straight line” may be, for example, a direction in which the virtual camera faces. More specifically, the “reference straight line” may be, for example, the optical axis direction of the virtual camera.

[Appendix 3]
An information processing apparatus program according to another aspect of the present invention is the information processing apparatus program according to attachment 1 or 2, wherein the display control unit is configured to execute the virtual in the virtual space in the first control mode. The position and posture of the virtual camera in the virtual space are changed so that the distance between the point and the virtual camera is maintained at a constant distance.

  According to this aspect, the interval between the virtual camera and the virtual point can be kept constant in the virtual space. That is, according to this aspect, for example, by setting the virtual point based on the position of the predetermined object in the virtual space, the interval between the virtual camera and the predetermined object is maintained substantially constant in the virtual space. Can do. For this reason, according to this aspect, it is possible to display the result of imaging the predetermined object from various directions while maintaining the size of the predetermined object substantially constant on the display unit of the head mounted display. Note that “maintaining the interval substantially constant” is a concept including not only maintaining the interval completely constant but also maintaining the interval constant while including an error in a predetermined range.

[Appendix 4]
A program for an information processing device according to another aspect of the present invention is the program for the information processing device according to appendices 1 to 3, wherein the display control unit is configured to perform predetermined processing in the virtual space in the first control mode. The virtual point is set in a setting area corresponding to the existence area of the object, and the position of the virtual point is changed inside the setting area in the second control mode.

  According to this aspect, it is possible to continuously display the area where the predetermined object exists in the virtual space on the display unit of the head mounted display.

  In the above aspect, the “predetermined object” may be a virtual object existing in the virtual space. For example, the “predetermined object” may be an object having a certain shape, an object existing at a certain position in the virtual space, or a shape and an existing position in the virtual space. One or both may be variable. Specifically, the “predetermined object” may be, for example, a character such as a person, an animal, and a monster that exists in the virtual space, or a vehicle, a building, and the like that exist in the virtual space. Artifacts such as tools may be used, and environmental components such as rooms, forests, wildernesses, and stars that constitute a virtual space may be used.

[Appendix 5]
A program for an information processing device according to another aspect of the present invention is the program for the information processing device according to appendices 1 to 4, wherein the display control unit includes the virtual camera and the virtual in the second control mode. The position of the virtual point in the virtual space such that an angle formed by a line segment connecting the points and a reference straight line in the virtual space determined based on an image displayed on the display unit is equal to or smaller than a predetermined angle. And the position of the virtual camera in the virtual space.

  According to this aspect, the angle formed by the direction captured by the virtual camera in the virtual space and the direction of the virtual point viewed from the virtual camera can be set to a predetermined angle or less, for example. Therefore, according to this aspect, for example, when a user wearing a head mounted display needs to continue to look at a predetermined object in the virtual space, the virtual point is located at a position corresponding to the predetermined object in the virtual space. By setting the position of the head mounted display, the user wearing the head mounted display can continuously display the area where the predetermined object exists in the virtual space on the display unit of the head mounted display. It can be displayed.

[Appendix 6]
An information processing apparatus program according to another aspect of the present invention is the information processing apparatus program according to attachment 5, wherein the display control unit is configured to store the virtual camera in the virtual space in the second control mode. It is characterized by keeping the posture constant.

  According to this aspect, for example, when a user wearing a head-mounted display needs to continue to look at a predetermined object in the virtual space, a virtual point is set at a position corresponding to the predetermined object in the virtual space. As described above, the user wearing the head mounted display can continuously display the area where the predetermined object exists in the virtual space on the display unit of the head mounted display by controlling the posture of the head mounted display. Is possible.

[Appendix 7]
An information processing apparatus program according to another aspect of the present invention is the information processing apparatus program according to any one of appendices 1 to 6, wherein the display control unit is configured to execute the virtual in the virtual space in the second control mode. Changing the position of the point by a distance according to the amount of change in posture indicated by the posture information, and changing the position of the virtual camera in the virtual space by a distance according to the amount of change in posture indicated by the posture information; It is characterized by that.

According to this aspect, in the second control mode, the position of the virtual camera in the virtual space can be changed based on the amount of change in the posture of the head mounted display. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
According to this aspect, the amount of change in the position of the virtual camera is determined in accordance with the amount of change in the posture of the head mounted display. For this reason, according to this aspect, the user wearing the head mounted display can change the position of the virtual camera to a desired position.

[Appendix 8]
A program for an information processing device according to another aspect of the present invention is the program for the information processing device according to appendices 1 to 7, wherein the display control unit is configured so that the head mounted display is in the second control mode. When rotating around a rotation axis, the position of the virtual point in the virtual space is changed in a virtual plane perpendicular to a virtual straight line corresponding to the rotation axis, and the position of the virtual camera in the virtual space is It is characterized by changing in a virtual plane.

According to this aspect, in the second control mode, the direction of change in the position of the virtual camera in the virtual space can be determined based on the direction of change in the posture of the head mounted display. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
According to this aspect, the change direction of the position of the virtual camera is determined according to the change direction of the posture of the head mounted display. For this reason, according to this aspect, the user wearing the head mounted display can change the position of the virtual camera to a desired position.

  In the above aspect, the “virtual straight line” means, for example, a straight line parallel to the rotation axis by an operator for converting the position and orientation of the real space where the head-mounted display exists into the position and orientation in the virtual space. It may be a straight line obtained by converting.

[Appendix 9]
An information processing apparatus program according to another aspect of the present invention is the information processing apparatus program according to appendices 1 to 8, wherein the display control unit is configured such that, in the first control mode, the head mounted display is The control mode is changed from the first control mode when the head mount display is rotated around the rotation axis and the angle of rotation around the rotation axis is equal to or larger than a reference angle. The mode is switched to the second control mode.

  According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.

[Appendix 10]
An information processing apparatus according to one embodiment of the present invention provides a display control for displaying a stereoscopic image using binocular parallax on a display unit provided in a head-mounted display, which is an image obtained by capturing a virtual space with a virtual camera. And an acquisition unit that acquires posture information related to the posture of the head mounted display, and the display control unit centers the virtual point set in the virtual space on the position of the virtual camera in the virtual space. A first control mode for changing the posture of the virtual camera in the virtual space based on the posture information, and a position of the virtual point in the virtual space, the first control mode for changing the posture of the virtual camera in the virtual space. And a second control mode that changes the position of the virtual camera in the virtual space based on the posture information. When the image can be displayed in the control mode and the image is displayed in the first control mode, the control mode is changed when the change amount of the posture indicated by the posture information is equal to or larger than a reference amount. , Switching from the first control mode to the second control mode.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Moreover, according to this aspect, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

[Appendix 11]
A head-mounted display according to an aspect of the present invention is a head-mounted display including a display unit and an information processing device, and the information processing device is an image obtained by capturing a virtual space with a virtual camera, A display control unit configured to display a stereoscopic image using binocular parallax on the display unit; and an acquisition unit configured to acquire posture information relating to a posture of the head mounted display, wherein the display control unit includes the virtual space The position of the virtual camera in the virtual space is changed based on the posture information around a virtual point set in the virtual space, and the posture of the virtual camera in the virtual space is changed based on the posture information. 1 control mode, and the position of the virtual point in the virtual space is changed based on the posture information, and the virtual camera in the virtual space is changed. In the case where the image can be displayed in a plurality of control modes including the second control mode that changes the position based on the posture information, and when the image is displayed in the first control mode, The control mode is switched from the first control mode to the second control mode when a change amount of the posture indicated by the posture information is equal to or greater than a reference amount.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Moreover, according to this aspect, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

[Appendix 12]
A display system according to one embodiment of the present invention is a display system including a head-mounted display having a display unit and an information processing device, and the information processing device is an image obtained by imaging a virtual space with a virtual camera. A display control unit that displays a stereoscopic image using binocular parallax on the display unit, and an acquisition unit that acquires posture information about the posture of the head mounted display, and the display control unit includes: The position of the virtual camera in the virtual space is changed based on the posture information around a virtual point set in the virtual space, and the posture of the virtual camera in the virtual space is changed based on the posture information. The first control mode to be changed and the position of the virtual point in the virtual space are changed based on the posture information, and the temporary space in the virtual space is changed. In the case of displaying an image in a plurality of control modes including a second control mode that changes the position of the camera based on the posture information, and displaying the image in the first control mode, The control mode is switched from the first control mode to the second control mode when a change amount of the posture indicated by the posture information is equal to or greater than a reference amount.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Moreover, according to this aspect, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

[Appendix 13]
A program for an information processing device according to another aspect of the present invention is a program for an information processing device including a processor, wherein the processor is an image obtained by capturing a virtual space with a virtual camera and uses binocular parallax. The stereoscopic image is displayed on a display unit provided in a head mounted display, and the display control unit functions as an acquisition unit that acquires posture information regarding the posture of the head mounted display. A first control mode for changing a posture of the virtual camera in the virtual space based on the posture information without changing a position of the virtual camera in the virtual space; and a position of the virtual camera in the virtual space. In addition, it is possible to display an image in a plurality of control modes including a second control mode that is changed based on the posture information. When displaying an image in the first control mode, when the change amount of the posture indicated by the posture information is equal to or greater than a reference amount, the control mode is changed from the first control mode to the second control mode. The mode is switched to the mode.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Also, according to this aspect, in the second control mode, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

[Appendix 14]
A program for an information processing device according to another aspect of the present invention is a program for an information processing device including a processor, wherein the processor is an image obtained by capturing a virtual space with a virtual camera and uses binocular parallax. The stereoscopic image is displayed on a display unit provided in a head mounted display, and the display control unit functions as an acquisition unit that acquires posture information regarding the posture of the head mounted display. The position of the virtual camera in the virtual space is changed based on the posture information around a virtual point set in the virtual space, and the posture of the virtual camera in the virtual space is changed based on the posture information. A first control mode to be changed, and a position of the virtual point in the virtual space are changed based on the posture information, and the virtual space The position of definitive the virtual camera, and a second control mode for changing on the basis of the attitude information can be displayed in the image by, characterized in that.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Moreover, according to this aspect, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

[Appendix 15]
A program for an information processing device according to another aspect of the present invention is a program for an information processing device including a processor, wherein the processor is an image obtained by capturing a virtual space with a virtual camera and uses binocular parallax. The stereoscopic image is displayed on a display unit provided in a head mounted display, and the display control unit functions as an acquisition unit that acquires posture information regarding the posture of the head mounted display. A first control mode for changing a posture of the virtual camera in the virtual space based on the posture information without changing a position of the virtual camera in the virtual space; and a position of the virtual camera in the virtual space. The image can be displayed in the second control mode that is changed based on the posture information.

According to this aspect, the posture of the virtual camera in the virtual space can be changed based on the posture information in the first control mode, and the virtual space in the virtual space can be changed based on the posture information in the second control mode. The position of the camera can be changed. For this reason, according to this aspect, it becomes possible to perform operations other than the change of the posture of the virtual camera in the virtual space by the change of the posture of the head mounted display.
Also, according to this aspect, in the second control mode, the position of the virtual camera in the virtual space can be changed without changing the position of the head mounted display in the real space. For this reason, according to this aspect, the user wearing the head mounted display can easily change the position of the virtual camera in the virtual space.

DESCRIPTION OF SYMBOLS 1 ... Head mounted display, 10 ... Terminal device, 11 ... Control part, 12 ... Display part, 13 ... Operation part, 14 ... Attitude information generation part, 15 ... Memory | storage part, 90 ... Wearing tool, 110 ... Display control part, 111 ... Mode designation unit, 112 ... Virtual camera control unit, 113 ... Display information generation unit, 114 ... Attitude information acquisition unit, 1000 ... Processor, 1002 ... Angular velocity sensor.

Claims (12)

A program for an information processing apparatus comprising a processor,
The processor;
An image obtained by capturing a virtual space with a virtual camera, and a stereoscopic image using binocular parallax,
A display control unit for displaying on a display unit provided in the head mounted display;
An acquisition unit for acquiring posture information relating to the posture of the head mounted display;
To function,
The display control unit
The position of the virtual camera in the virtual space,
Centering on the virtual point set in the virtual space,
Change based on the posture information,
The attitude of the virtual camera in the virtual space,
A first control mode to be changed based on the posture information;
The position of the virtual point in the virtual space,
Change based on the posture information,
The position of the virtual camera in the virtual space,
A second control mode to be changed based on the posture information;
It is possible to display images in multiple control modes including
When displaying an image in the first control mode,
When the amount of change in posture indicated by the posture information is equal to or greater than a reference amount,
Switching the control mode from the first control mode to the second control mode;
A program for an information processing apparatus. The display control unit
In the first control mode,
A line segment connecting the virtual camera and the virtual point;
An angle formed with a reference straight line in the virtual space determined based on an image displayed on the display unit,
In order to be below a predetermined angle,
Changing the position and orientation of the virtual camera in the virtual space;
The information processing apparatus program according to claim 1, wherein: The display control unit
In the first control mode,
An interval between the virtual point and the virtual camera in the virtual space is
To maintain a certain distance,
Changing the position and orientation of the virtual camera in the virtual space;
The program of the information processing apparatus according to claim 1, wherein the program is an information processing apparatus. The display control unit
In the first control mode,
In the setting area corresponding to the existence area of the predetermined object in the virtual space,
Set the virtual point,
In the second control mode,
Inside the setting area,
Changing the position of the virtual point;
The information processing apparatus program according to any one of claims 1 to 3, wherein the program is an information processing apparatus. The display control unit
In the second control mode,
A line segment connecting the virtual camera and the virtual point;
An angle formed with a reference straight line in the virtual space determined based on an image displayed on the display unit,
In order to be below a predetermined angle,
The position of the virtual point in the virtual space;
Changing the position of the virtual camera in the virtual space;
The program of the information processing apparatus according to any one of claims 1 to 4, wherein the program is an information processing apparatus. The display control unit
In the second control mode,
Keeping the attitude of the virtual camera in the virtual space constant,
The program of the information processing apparatus according to claim 5, wherein The display control unit
In the second control mode,
The position of the virtual point in the virtual space,
Change the distance according to the amount of change in posture indicated by the posture information,
The position of the virtual camera in the virtual space,
The distance is changed according to the amount of change in posture indicated by the posture information.
The information processing apparatus program according to any one of claims 1 to 6, wherein the program is an information processing apparatus. The display control unit
In the second control mode,
When the head mounted display rotates around the rotation axis,
The position of the virtual point in the virtual space,
Changing in a virtual plane perpendicular to a virtual straight line corresponding to the rotation axis,
The position of the virtual camera in the virtual space,
Changing in the virtual plane,
The program of the information processing apparatus according to any one of claims 1 to 7, wherein the program is an information processing apparatus. The display control unit
In the first control mode,
The head-mounted display rotates about a rotation axis;
When the angle of rotation of the head mounted display around the rotation axis is equal to or greater than a reference angle,
Switching the control mode from the first control mode to the second control mode;
The information processing apparatus program according to any one of claims 1 to 8, wherein the program is an information processing apparatus. An image obtained by capturing a virtual space with a virtual camera, and a stereoscopic image using binocular parallax,
A display control unit for displaying on a display unit provided in the head mounted display;
An acquisition unit for acquiring posture information relating to the posture of the head mounted display;
With
The display control unit
The position of the virtual camera in the virtual space,
Centering on the virtual point set in the virtual space,
Change based on the posture information,
The attitude of the virtual camera in the virtual space,
A first control mode to be changed based on the posture information;
The position of the virtual point in the virtual space,
Change based on the posture information,
The position of the virtual camera in the virtual space,
A second control mode to be changed based on the posture information;
It is possible to display images in multiple control modes including
When displaying an image in the first control mode,
When the amount of change in posture indicated by the posture information is equal to or greater than a reference amount,
Switching the control mode from the first control mode to the second control mode;
An information processing apparatus. A head-mounted display comprising a display unit and an information processing device,
The information processing apparatus includes:
An image obtained by capturing a virtual space with a virtual camera, and a stereoscopic image using binocular parallax,
A display control unit to be displayed on the display unit;
An acquisition unit for acquiring posture information relating to the posture of the head mounted display;
With
The display control unit
The position of the virtual camera in the virtual space,
Centering on the virtual point set in the virtual space,
Change based on the posture information,
The attitude of the virtual camera in the virtual space,
A first control mode to be changed based on the posture information;
The position of the virtual point in the virtual space,
Change based on the posture information,
The position of the virtual camera in the virtual space,
A second control mode to be changed based on the posture information;
It is possible to display images in multiple control modes including
When displaying an image in the first control mode,
When the amount of change in posture indicated by the posture information is equal to or greater than a reference amount,
Switching the control mode from the first control mode to the second control mode;
A head mounted display. A display system comprising a head mounted display having a display unit and an information processing device,
The information processing apparatus includes:
An image obtained by capturing a virtual space with a virtual camera, and a stereoscopic image using binocular parallax,
A display control unit to be displayed on the display unit;
An acquisition unit for acquiring posture information relating to the posture of the head mounted display;
With
The display control unit
The position of the virtual camera in the virtual space,
Centering on the virtual point set in the virtual space,
Change based on the posture information,
The attitude of the virtual camera in the virtual space,
A first control mode to be changed based on the posture information;
The position of the virtual point in the virtual space,
Change based on the posture information,
The position of the virtual camera in the virtual space,
A second control mode to be changed based on the posture information;
It is possible to display images in multiple control modes including
When displaying an image in the first control mode,
When the amount of change in posture indicated by the posture information is equal to or greater than a reference amount,
Switching the control mode from the first control mode to the second control mode;
A display system characterized by that.

Images