WO2019230437A1

WO2019230437A1 - Information processing device, information processing method, and program

Info

Publication number: WO2019230437A1
Application number: PCT/JP2019/019622
Authority: WO
Inventors: 安田　亮平; 石川　毅; 高橋　慧; 孝悌清水; 惇一清水
Original assignee: ソニー株式会社
Priority date: 2018-05-31
Filing date: 2019-05-17
Publication date: 2019-12-05
Also published as: JP2021140196A

Abstract

The present feature pertains to an information processing device, an information processing method, and a program that are configured so as to enable a virtual space to be moved more smoothly. Provided is an information processing device provided with: a movement information acquisition unit for acquiring movement information for moving a user's line of sight in a virtual space; a direction information acquisition unit for acquiring direction information that corresponds to the direction of the user's line of sight; and a display control unit that, when the position of the user's light of sign is present in the peripheral region of an object arranged in the virtual space, moves the user's view point in the direction of the line of sight on the basis of at least one of the acquired movement information and direction information and changes the direction of the line of sight, thereby controlling a display device so that an image visually recognized by the user is presented to the user. The present feature can be applied, for example, to a head-mounted display.

Description

Information processing apparatus, information processing method, and program

The present technology relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program that can move a virtual space more smoothly.

In recent years, research and development of technologies for providing a virtual reality (VR) function using devices such as a head-mounted display (HMD) have been actively conducted (for example, patents) Reference 1).

Patent Document 1 discloses an image generation apparatus that can move a user's viewpoint with respect to a wide viewing angle image such as a panoramic image when a user moves a panoramic space (virtual space) with a head-mounted display. Is disclosed.

JP 2016-62486

By the way, when moving the user's viewpoint in the virtual space, there is a situation where the user wants to go around the obstacle object such as a wall. However, under the present situation, when moving around the obstacle object, it is necessary to further move and change the direction of the user after moving once near the edge of the obstacle object. Therefore, there is a demand for a technique for smoothly moving the virtual space.

This technology has been made in view of such a situation, and makes it possible to move the virtual space more smoothly.

An information processing apparatus according to an aspect of the present technology includes a movement information acquisition unit that acquires movement information that moves a user's viewpoint in a virtual space, and a direction information acquisition unit that acquires direction information corresponding to the direction of the user's line of sight. When the position of the user's line of sight is in the peripheral area of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. And a display control unit that controls the display device so as to present an image visually recognized by the user by changing the direction of the line of sight to the user.

The information processing method and program according to one aspect of the present technology are an information processing method and program corresponding to the information processing apparatus according to one aspect of the present technology described above.

In the information processing apparatus, the information processing method, and the program according to one aspect of the present technology, movement information for moving the viewpoint of the user in the virtual space is acquired, direction information corresponding to the direction of the line of sight of the user is acquired, When the position of the user's line of sight is in the peripheral area of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. The display device is controlled so as to present the user with an image visually recognized by the user by changing the direction of the line of sight.

The information processing apparatus according to one aspect of the present technology may be an independent apparatus, or may be an internal block constituting one apparatus.

According to one aspect of the present technology, the virtual space can be moved more smoothly.

It should be noted that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure which shows the example of the external appearance of the head mounted display to which this technique is applied. It is a block diagram showing an example of composition of a 1 embodiment of a head mount display to which this art is applied. It is a block diagram which shows the example of a functional structure of the control part of FIG. It is a figure explaining the outline | summary of this technique. It is a figure explaining the outline | summary of this technique. It is a flowchart explaining the flow of the process performed by a head mounted display. It is a flowchart explaining the flow of the 1st example of the surrounding determination process for single edges. It is a figure which shows the 1st example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the 1st example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the 1st example of the CG image | video of a model room. It is a figure which shows the 2nd example of the CG image | video of a model room. It is a figure which shows the 3rd example of the CG image | video of a model room. It is a figure which shows the 4th example of CG image of a model room. It is a flowchart explaining the flow of the 2nd example of the surrounding determination process for single edges. It is a figure which shows the 2nd example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the 2nd example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the 2nd example of the determination method of the position and direction of a wraparound destination. It is a flowchart explaining the flow of the 3rd example of the surrounding determination process for single edges. It is a figure which shows the example of allocation of the space attribute for every room. It is a figure which shows the example of adjustment of the position and direction of a wraparound destination according to a space attribute. It is a figure which shows the example of adjustment of the position and direction of a wraparound destination according to a space attribute. It is a flowchart explaining the flow of the 4th example of the wraparound destination determination process for single edges. It is a figure which shows the example of adjustment of the wraparound amount according to the inclination amount of a body. It is a flowchart explaining the flow of the 4th example of the wraparound destination determination process for single edges. It is a figure which shows the example in the case of making the state which opened the door, when gazing at the door periphery. It is a figure which shows the example in the case of displaying the image | video of an overhead viewpoint when gazing at a specific area | region. It is a figure which shows the example in the case of displaying the image | video of a wraparound destination. It is a flowchart explaining the flow of the wraparound destination determination process from an obstruction upper end point. It is a figure which shows the example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the example of the determination method of the position and direction of a wraparound destination. It is a figure which shows the example of clustering of a detailed object. It is a flowchart explaining the flow of the 1st example of the surrounding destination determination process for multiple edges. It is a figure which shows the example of a display of the candidate of a wraparound destination. It is a flowchart explaining the flow of the 2nd example of the wraparound destination determination process for multiple edges. It is a figure which shows the example of the deformation display of a specific obstruction object. It is a flowchart explaining the flow of the 3rd example of the surrounding determination process for multiple edges. It is a figure which shows the example in case the edge gravity center position is made into the position of a movement destination. It is a figure which shows the example in case the edge gravity center position is made into the position of a movement destination. It is a figure which shows the example of a structure of a computer.

Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order.

1. Embodiment 2 of the present technology Modification 3 Computer configuration

<1. Embodiment of the present technology>

(Example of HMD appearance)
FIG. 1 is a diagram illustrating an example of the appearance of a head mounted display to which the present technology is applied.

The head-mounted display 1 is an information processing apparatus that is worn on the head so as to cover both eyes of the user and that can be used for viewing still images, moving images, and the like displayed on a display screen provided in front of the user's eyes.

1, the head mounted display 1 has a main body portion 11 and a head contact portion 12. The main body 11 is provided with a display, various sensors, a communication module, and the like. The head contact portion 12 fixes the main body portion 11 so as to contact the user's head and cover both eyes.

(Example of HMD configuration)
FIG. 2 is a block diagram illustrating an example of a configuration of an embodiment of a head mounted display to which the present technology is applied.

2, the head mounted display 1 includes a control unit 100, a sensor unit 101, a storage unit 102, a display unit 103, an audio output unit 104, an input terminal 105, an output terminal 106, and a communication unit 107.

The control unit 100 includes, for example, a CPU (Central Processing Unit). The control unit 100 is a central processing device that controls the operation of each unit and performs various arithmetic processes. Here, a dedicated processor such as a GPU (Graphics Processing Unit) may be provided.

The sensor unit 101 includes, for example, various sensor devices. The sensor unit 101 performs sensing of the user and its surroundings, and supplies sensor data corresponding to the sensing result to the control unit 100.

Here, as the sensor unit 101, for example, a magnetic sensor that detects the magnitude and direction of a magnetic field (magnetic field), an acceleration sensor that detects acceleration, a gyro sensor that detects an angle (attitude), angular velocity, and angular acceleration, or a nearby sensor A proximity sensor for detecting Here, the user's gesture or the like may be detected by an inertial measurement device (IMU: Inertial Measurement Unit) that obtains a three-dimensional angular velocity and acceleration.

Further, a camera having an image sensor may be provided as the sensor unit 101, and image data obtained by imaging a subject may be supplied to the control unit 100. Further, the sensor unit 101 includes sensors for measuring the surrounding environment such as a temperature sensor for detecting temperature, a humidity sensor for detecting humidity, an ambient light sensor for detecting ambient brightness, breathing and pulse, A biosensor for detecting biometric information such as a fingerprint and an iris, and a sensor for detecting position information such as a GPS (Global Positioning System) signal can be included.

The storage unit 102 includes, for example, a semiconductor memory including a nonvolatile memory and a volatile memory. The storage unit 102 stores various data according to control from the control unit 100.

The display unit 103 includes, for example, a display device (display device) such as a liquid crystal display (LCD) or an organic EL display (OLED: Organic Light Emitting Diode). The display unit 103 displays a video (or image) corresponding to the video data (or image data) supplied from the control unit 100.

The audio output unit 104 includes an audio output device such as a speaker. The sound output unit 104 outputs sound (sound) corresponding to the sound data supplied from the control unit 100.

The input terminal 105 includes, for example, an input interface circuit, and is connected to an electronic device via a predetermined cable. For example, the input terminal 105 supplies the control unit 100 with video data, audio data, commands, and the like input from devices such as a game machine (dedicated console), a personal computer, and a playback machine, for example.

The output terminal 106 includes, for example, an output interface circuit and the like, and is connected to an electronic device via a predetermined cable. For example, the output terminal 106 outputs audio data supplied thereto to a device such as an earphone or a headphone via a cable.

The communication unit 107 is configured as a communication module that supports wireless communication such as Bluetooth (registered trademark), wireless LAN (Local Area Network), or cellular communication (for example, LTE-Advanced or 5G), or wired communication. Is done. The communication unit 107 communicates with an external device according to a predetermined communication method, and exchanges various data (for example, video data, audio data, commands, etc.). The external devices here include, for example, a game machine (dedicated console), a personal computer, a server, a player, a dedicated controller, a remote controller, and the like.

(Example of functional configuration of control unit)
FIG. 3 is a block diagram illustrating an example of a functional configuration of the control unit 100 of FIG.

3, the control unit 100 includes a movement information acquisition unit 151, a direction information acquisition unit 152, a wraparound destination control unit 153, a viewpoint camera position / posture control unit 154, and a display control unit 155.

The movement information acquisition unit 151 acquires movement information based on data supplied from the sensor unit 101, the input terminal 105, the communication unit 107, and the like, and supplies the movement information to the wraparound destination control unit 153.

Here, the movement information is information for moving the viewpoint of the user in the virtual space. That is, the movement information is obtained based on some input for determining the movement of the user in the virtual space. For example, the user's operation (for example, controller operation) or action (for example, walking or gesture), or the user The detection result of the dwell time of the line of sight (for example, the gaze time of a predetermined area) is included. The movement information may include information indicating that the user has stopped.

The direction information acquisition unit 152 acquires direction information based on data supplied from the sensor unit 101, the input terminal 105, the communication unit 107, or the like, and supplies the direction information to the wraparound destination control unit 153.

Here, the direction information is information corresponding to the direction of the user's line of sight. In other words, the direction information is obtained based on some input for determining the user's line of sight, for example, the user's line of sight or face orientation, or the user's operation (for example, a pointing device (such as a dedicated controller or remote controller). Operation) or action (for example, gestures such as pointing).

The wraparound destination control unit 153 determines the wraparound destination of the user's viewpoint (position) with respect to the obstacle object based on at least one of the movement information from the movement information acquisition unit 151 and the direction information from the direction information acquisition unit 152. Control.

The viewpoint camera position / posture control unit 154 controls the position and posture of the viewpoint camera according to the control of the wraparound destination by the wraparound destination control unit 153.

The display control unit 155 controls the display of the video (or image) and displays it on the display unit 103 according to the control of the position and orientation of the viewpoint camera by the viewpoint camera position / orientation control unit 154.

3, the control unit 100 is configured as one or a plurality of CPUs, and at least one CPU includes a movement information acquisition unit 151, a direction information acquisition unit 152, a wraparound destination control unit 153, and a viewpoint camera position / posture control. The function of the unit 154 or the display control unit 155 is included. In addition, these functions may be realized in a single device, or may be realized in cooperation with a plurality of devices (CPUs) without closing to a single device.

By the way, in a device (system) that moves the user's viewpoint in the virtual space, there is a situation where it is desired to go around the obstacle object such as a wall. However, if there is no means for directly moving from the current user's viewpoint (position) to the position around the obstacle object, after moving once near the edge of the obstacle object, It is necessary for the user to perform an operation of changing the direction of movement.

Here, in the virtual space, when the user tries to go to the other side of the obstacle object from the place where the user is, it is considered that the vicinity of the edge of the obstacle object is often seen naturally. Based on this idea, for example, when the user's line of sight is detected and the edge of the obstacle object is viewed (gazing), the present technology moves so that the position around the obstacle object is easy to see. And adjust the rotation.

Specifically, as shown in FIG. 4, it is assumed that the user 20 wearing the head mounted display 1 gazes at the edge of the obstacle object 30 in the virtual space and the edge becomes the gaze area 200. To do.

Hereinafter, an area including a gazing point that the user 20 is gazing at is described as a gazing area 200. In other words, it can be said that the gaze area 200 is a peripheral area of the obstacle object 30 at the position of the line of sight L _S of the user 20. Here, the case where the edge of the obstacle object 30 is included in the peripheral area is illustrated, but the edge may not be included.

At this time, since the edge of the obstacle object 30 has become the gaze area 200, the position (viewpoint) of the user 20 moves to the position around the obstacle object 30 (instantaneous movement) as shown in FIG. However, here, it is possible to adjust so that the surface folded in the depth direction from the edge and the direction of the line of sight of the user 20 are parallel (A in FIG. 5).

Further, adjustment may be made so that the direction of the line of sight of the user 20 is parallel to the front surface (the surface on the back side of B in FIG. 5) (B in FIG. 5). In particular, when the obstacle object 30 as shown in FIG. 5B is like a thin plate, there is a great advantage that the position (viewpoint) of the user 20 is made to reach the back surface of the obstacle object 30. Further, the position (viewpoint) of the user 20 can be adjusted to be a position separated by a predetermined distance from the parallel plane, for example.

Thus, in the present technology, when the position of the line of sight L _S of the user 20 is in the peripheral area (for example, an edge) of the obstacle object 30 arranged in the virtual space, the position of the user 20 in the direction of the line of sight L _S ( By moving the (viewpoint) and changing the direction of the line of sight, an image visually recognized by the user 20 is presented to the user 20.

For example, by moving the position (viewpoint) of the user 20 to a position in the vicinity of the side surface (A in FIG. 5) or the back surface (B in FIG. 5) with respect to the user-side surface of the obstacle object 30. Is moved to a position where the movement of the viewpoint of the user 20 can be reduced, so that the virtual space can be moved more smoothly.

Here, in FIG. 5, the position (viewpoint) of the user 20 after movement when the conventional method is used is indicated by a dotted line, but in the conventional method, the viewpoint of the user 20 is the obstacle object 30. Therefore, after moving once near the edge of the obstacle object 30, the user 20 needs to further perform an operation of changing the movement and direction. On the other hand, in the present technology, since the position (viewpoint) of the user 20 is moved to a position where the movement of the viewpoint of the user 20 can be reduced, the moving operation for turning around the obstacle object 30 as compared with the conventional method. The movement in the virtual space can be performed more smoothly.

In the following description, “position of user 20” includes the meaning of “viewpoint of user 20” in the virtual space. The virtual space includes a virtual space imitating reality constructed on a computer and a virtual space in reality.

In addition, when the position of the line of sight L _S of the user 20 is on another object (for example, an effect object) that emphasizes a specific position in the virtual space, the position (view point) of the user 20 is moved in the direction of the line of sight L _S. The image visually recognized by the user 20 while maintaining the direction of the line of sight can be presented to the user 20. Here, the effect object is not the obstacle object 30 such as the wall described above, but an object floating in the space, for example. This effect object is presented, for example, when the position of the line of sight L _S of the user 20 is not in the peripheral area (for example, edge) of the obstacle object 30 arranged in the virtual space.

(HMD processing flow)
Next, the flow of processing executed by the head mounted display 1 (the control unit 100) will be described with reference to the flowchart of FIG.

In step S11, the movement information acquisition unit 151 acquires movement information. In step S12, the direction information acquisition unit 152 acquires direction information.

Here, when acquiring the direction information, as the sensor unit 101, for example, a sensor for detecting the line of sight of the user 20, a pointing from a dedicated controller, or a sensor from various sensors for detecting the gesture of the user 20 Direction information is generated based on the data. As a method for detecting the line of sight of the user 20, for example, a line of sight estimation by capturing an eyeball and acquiring image data of a Purkinje image, or a method of estimating the direction of the apparatus as the direction of the line of sight can be used.

In addition, when acquiring movement information and direction information, instead of detecting the line of sight of the user 20, information corresponding to the direction of the line of sight, for example, by detecting the direction of the head of the user 20 or pointing and operating the pointing device. May be obtained.

In step S13, the wraparound destination control unit 153 determines whether there is a moving operation based on the acquired movement information.

If it is determined in step S13 that there is no moving operation, the process returns to step S11, and the processes of steps S11 to S13 are repeated. On the other hand, if it is determined in step S13 that there is a moving operation, the process proceeds to step S14.

In step S14, the wraparound destination control unit 153 determines whether or not the end (edge) of the obstacle object 30 exists in the vicinity of the gazing point by the user 20 based on the acquired direction information.

If it is determined in the determination process in step S14 that only one edge exists in the vicinity of the gazing point, the process proceeds to step S15.

In step S15, the wraparound destination control unit 153 executes a single edge wraparound destination determination process.

In this single edge wraparound destination determination process, a process for determining the wraparound destination of the position (viewpoint) of the user 20 with respect to the obstacle object 30 when one edge exists in the vicinity of the gazing point is executed. Details of the single edge wraparound destination determination process will be described later with reference to FIGS.

If it is determined in step S14 that there is no edge near the point of interest, the process proceeds to step S16.

In step S16, the wraparound destination control unit 153 performs a wraparound destination determination process from the obstacle upper end point.

In this wraparound destination determination process from the obstacle upper end point, a process for determining the wraparound destination of the position (viewpoint) of the user 20 with respect to the obstacle object 30 when there is no edge near the gazing point is executed. Details of the wraparound destination determination process from the obstacle upper end point will be described later with reference to FIGS.

Furthermore, when it is determined in step S14 that there are a plurality of edges in the vicinity of the gazing point, the process proceeds to step S17.

In step S17, the wraparound destination control unit 153 performs a wraparound destination determination process for multiple edges.

In this multiple edge wraparound destination determination process, a process for determining the wraparound destination of the position (viewpoint) of the user 20 with respect to the obstacle object 30 when there are a plurality of edges near the gazing point is executed. The details of the multi-edge wraparound destination determination process will be described later with reference to FIGS. 32 to 38.

When the process in any of steps S15 to S17 ends, the process proceeds to step S18.

In step S <b> 18, the viewpoint camera position / posture control unit 154 determines the position (viewpoint) of the user 20 based on the result of the wraparound destination determination process by the wraparound destination control unit 153 (the result of any one of steps S <b> 15 to S <b> 17). Move to the position where you wrap around and change its direction.

In step S19, the display control unit 155 displays the wraparound video on the display unit 103 based on the result of the process in step S18. That is, the display control unit 155 displays at least one of the movement information and the direction information when the position of the line of sight L _S of the user 20 is in a peripheral area (for example, an edge) of the obstacle object 30 arranged in the virtual space. Based on this, the position (viewpoint) of the user 20 is moved in the direction of the line of sight L _S , and an image visually recognized by the user 20 by changing the direction of the line of sight is presented to the user 20.

When the process in step S19 is completed, the process returns to step S11, and the subsequent processes are repeated.

The flow of processing executed by the head mounted display 1 has been described above.

(Single edge wraparound destination determination process)
Next, details of the single edge wraparound destination determination process corresponding to the process of step S15 of FIG. 6 will be described with reference to FIGS. Here, as examples of the single edge wraparound destination determination process, seven examples of the first to seventh examples are shown.

(First example)
First, referring to FIG. 7 to FIG. 13, as a first example of the single edge wraparound destination determination process, a method of determining the position and direction of the movement destination according to the quadrant including the current user 20 position. The case where it is used will be described.

FIG. 7 is a flowchart for explaining the flow of processing when determining the movement destination according to the quadrant including the current user position as a first example of the single edge wraparound destination determination processing.

In step S101, the wraparound destination control unit 153 generates an edge gaze detection virtual object.

This edge gaze detection virtual object is generated to detect the area being watched by the user 20 when the line of sight of the user 20 faces the end (edge) of the obstacle object 30 such as a wall. Virtual object.

In step S102, the wraparound destination control unit 153 includes the current position of the user 20 in any quadrant on the XZ plane (first quadrant to fourth quadrant on the XZ plane) of the generated edge gaze detection virtual object. To identify.

In step S103, the wraparound destination control unit 153 determines the position of the user 20 according to the quadrant (one quadrant of the first quadrant to the fourth quadrant on the XZ plane) including the identified current user 20 position. Determine the location and direction of the destination. The position (viewpoint) of the moving destination determined in this way is set as a position (around position) around the obstacle object 30 such as a wall.

When the process in step S103 is completed, the process returns to step S15 in FIG. 6, and the subsequent processes are executed.

In the above, the flow of processing when determining the movement destination according to the quadrant including the current user position has been described. Next, a specific example of the first example will be described with reference to FIGS.

FIG. 8 is a diagram showing the relationship between the position before movement of the user 20 and the position after movement when the edge of the obstacle object 30 such as a wall is used as a reference.

In FIG. 8, when there is an obstacle object 30 near the user 20, the edge gaze detection virtual object 300 is virtually displayed when the line of sight of the user 20 faces the edge of the obstacle object 30. The 3D coordinate system (left-handed coordinate system) XZ plane including the object (a view of the space where the obstacle object 30 is located is viewed from directly above) is generated.

Here, for example, among the four XZ planes shown in FIG. 9, in the upper right XZ plane, the position of the user 20 is in the first quadrant, so the destination position P _M is on the negative X axis. The direction _DM is an obliquely lower left direction. In the upper left XZ plane, since the position of the user 20 is in the second quadrant, the movement destination position P _M is on the positive X axis, and the direction D _M is an oblique lower right direction. The

In the lower left XZ plane, since the position of the user 20 exists in the third quadrant, the movement destination position P _M is on the positive Z axis, and the direction D _M is an upper right direction. The In the lower right XZ plane, since the position of the user 20 exists in the fourth quadrant, the movement destination position P _M is on the positive Z axis, and the direction D _M is the upper left diagonal direction. .

In this way, the destination position P _M and its direction D _M are determined depending on which quadrant on the XZ plane the user 20 is located. For example, in the case of the standing position of the user 20 shown in FIG. 8, the edge gaze detection virtual object 300 exists in the third quadrant on the XZ plane (S102 in FIG. 7). The position P _{M of} 20 (the position of the wraparound destination) is determined on the positive Z axis, and the direction D _M is an obliquely upper right direction (S103 in FIG. 7). An example in which an actual video is displayed using this wraparound destination determination method is shown in FIGS.

(Example of display image)
FIGS. 10 to 13 show a CG image of a virtual model room as an example of an image displayed on the display unit 103 of the head mounted display 1 mounted on the head of the user 20. In this virtual model room, you can walk around in the virtual space created by CG (Computer Graphics) and feel as if you were on the spot. In FIGS. 10 to 13, the walls that partition the rooms of the model room correspond to the obstacle object 30.

FIG. 10 is a diagram illustrating a first example of a CG image of a model room.

FIG. 10A shows a state where the user 20 is watching the edge of the wall separating the kitchen and the hallway when the user 20 is in the kitchen.

Thus, when the edge of the wall partitioning the kitchen and the corridor becomes the gaze area 200, the position of the user 20 is moved to a position around the wall as shown in FIG. An image is displayed in which the direction of the line of sight is directed toward the end of the corridor that has turned around. That is, when the gaze area 200 is directed to the edge of the wall by the user 20, the user 20 automatically moves (instantaneously moves) from the kitchen to a position around the wall.

FIG. 11 is a diagram showing a second example of the CG video of the model room. The hallway shown in FIG. 11 is the same as the hallway shown in FIG.

FIG. 11A shows a state in which, when the user 20 is in the hallway, the edge of the wall separating the hallway and the bathroom is watched, and the edge becomes the gaze area 200. At this time, as shown in FIG. 11B, the position of the user 20 is moved to a position around the wall, and an image in which the direction of the line of sight is directed to the surrounding bathroom is displayed. Will move to a position around the wall.

FIG. 12 is a diagram showing a third example of the CG video of the model room.

FIG. 12A shows a state in which, when the user 20 is in the kitchen, the edge of the wall separating the hallway and the adjacent room is watched, and the edge becomes the watched area 200. At this time, as shown in FIG. 12B, the position of the user 20 is moved to the position around the wall, and an image in which the direction of the line of sight is directed to the adjacent room around is displayed. , I will move to the position around the wall from the kitchen.

FIG. 13 is a diagram showing a fourth example of the CG video of the model room. The adjacent room shown in FIG. 13 is the same as the adjacent room shown in FIG.

FIG. 13A shows a state in which, when the user 20 is in the next room, the edge of the wall partitioning the next room and the bedroom is watched, and the edge becomes the gaze area 200. At this time, as shown in FIG. 13B, the position of the user 20 is moved to a position around the wall, and an image in which the direction of the line of sight is directed to the bedroom around is displayed. It will move instantly from the room to the position around the wall.

Thus, when the user 20 is looking at the edge of the obstacle object 30 such as a wall by using the method of determining the position and direction of the movement destination according to the quadrant including the current position of the user 20 ( When moving around the obstacle object 30, the virtual space moves more smoothly in a situation where it is desired to move around the obstacle object 30. Can do.

That is, in the conventional method, after the user 20 moves from the current position of the user 20 to a position in the vicinity of the obstacle object 30, the user 20 moves and moves to the position around the obstacle object 30. Although it is necessary to perform an operation of changing the direction, the technique of the present technology can move the obstacle object 30 to a position that wraps around directly from the current position of the user 20.

(Second example)
Next, referring to FIGS. 14 to 17, as a second example of the single edge wraparound destination determination process, the virtual viewpoint object generated virtually is moved by moving along the obstacle object 30. The case where the method for determining the previous position is used will be described.

FIG. 14 is a flowchart for explaining the flow of processing when determining the position of the movement destination by moving the virtual viewpoint object along the obstacle object as a second example of the single edge wraparound destination determination processing. is there.

In step S121, the wraparound destination control unit 153 specifies a gazing point on the obstacle object 30 according to the line of sight of the user 20. Here, the gazing point on the obstacle object 30 includes a portion that is not an edge.

In step S122, the wraparound destination control unit 153 generates a virtual viewpoint object with the identified gazing point as a contact point.

This virtual viewpoint object is a virtual object that is generated to determine the position of the movement destination when the line of sight of the user 20 is not facing the edge of the obstacle object 30 such as a wall.

In step S123, the wraparound destination control unit 153 calculates a vector along the obstacle object by applying a predetermined arithmetic expression to the normal line from the specified gazing point. Here, for example, when the obstacle object 30 is a wall, a vector along the wall is obtained.

In step S124, the wraparound destination control unit 153 moves the generated virtual viewpoint object while contacting the obstacle object 30 based on the calculated vector. Here, for example, when the obstacle object 30 is a wall, the virtual viewpoint object is moved while being in contact with the wall based on the vector along the wall, with the position in contact with the specified gazing point as the start point of the movement trajectory.

In step S125, the wraparound destination control unit 153 determines whether the length of the movement trajectory of the virtual viewpoint object has reached a predetermined value.

If it is determined in step S125 that the length of the movement locus is not the predetermined value, the process returns to step S124, and the above-described process is repeated. If it is determined that the length of the movement trajectory has reached the predetermined value by moving the virtual viewpoint object while contacting the obstacle object 30, the process proceeds to step S126.

In step S126, the wraparound destination control unit 153 determines the movement point at which the length of the movement locus has become a predetermined value as the position of the movement destination of the user 20. The position (viewpoint) of the moving destination determined in this way is set as a position (around position) around the obstacle object 30 such as a wall.

When the process in step S126 is completed, the process returns to step S15 in FIG. 6, and the subsequent processes are executed.

As described above, the flow of processing when the position of the movement destination is determined by moving the virtual viewpoint object along the obstacle object has been described. Next, a specific example of the second example will be described with reference to FIGS. 15 to 17.

FIG. 15 shows an example of a virtual viewpoint object 310 that is virtually generated for the obstacle object 30 that is a wall.

In Figure 15, the line of sight of the user 20, when facing the direction of the obstacle object 30 near, based on the line-of-sight vector s, gaze point P _G on obstacle object 30 is specified (in FIG. 14 S121), the virtual view object 310 the gaze point P _G in contact is generated (S122 of FIG. 14).

Further, in this example, since the obstacle object 30 becomes a wall, for example, the along-wall vector w is calculated by performing the calculation shown in FIG. 16 (S123 in FIG. 14). That is, in FIG. 16, the starting point of the wall along the vector w, so as to be positioned at the base (starting point) of the traveling vector f, the normal of the gaze point P _G on the surface of the obstacle object 30, the traveling vector f Try to match the destination (end point).

At this time, when the length of the normal vector n is adjusted to the length of a, the wall-side vector w can be obtained by the following equation (1).

... (1)

Here, the normal is normalized, and the coefficient a is the length when the inverse vector of the progression vector f is projected onto the normal. That is, the coefficient a can be obtained by the following equation (2).

... (2)

In equation (2), “Dot” represents the inner product.

From the above, the along-wall vector w can be obtained by the following equation (3) from the relationship between the equations (1) and (2).

... (3)

In this way, when the along-wall vector w is obtained, the virtual viewpoint object 310 is moved in contact with the obstacle object 30 (wall) based on the along-wall vector w (S124 in FIG. 14).

FIG. 17 shows an example of movement of the virtual viewpoint object 310.

In FIG. 17, the virtual viewpoint object 310 is moved while touching the obstacle object 30 (wall) with the position in contact with the gazing point P _G as the start point P _S of the movement locus, and the length L of the movement locus is reached. _{When M} reaches a predetermined value (“YES” in S125 of FIG. 14), the movement point P _M is determined as the movement destination position of the user 20 (S126 of FIG. 14).

In this way, by using the method of determining the position of the movement destination by moving the virtual viewpoint object 310 along the obstacle object 30, the user 20 can select a certain part of the obstacle object 30 (part excluding the edge). In a situation where it is desired to go around the obstacle object 30 in order to move (instantaneous movement) to the position around the obstacle object 30 when watching (including gazing). Space movement can be performed more smoothly.

(Third example)
Next, referring to FIGS. 18 to 21, as a third example of the single edge wraparound destination determination process, when determining the position of the movement destination, the wraparound is performed according to the attribute of the space behind the obstacle object 30. A case where a method of changing the direction is used will be described. In FIGS. 18 to 21, the walls that partition the rooms of the model room correspond to the obstacle object 30.

FIG. 18 is a flowchart for explaining the flow of processing when changing the wraparound method according to the space attribute on the back side of the obstacle object, as a third example of the wraparound destination determination processing for single edge.

In step S141, the wraparound destination control unit 153 determines a preset space attribute on the back side of the wall. Here, it is assumed that “room” and “corridor” in the model room are assigned as an example of the space attribute.

If it is determined in step S141 that the space attribute is “room”, the process proceeds to step S142.

In step S142, the wraparound destination control unit 153 determines the position and direction of the movement destination when the space attribute is “room” as the position that wraps around the side surface of the wall (near the entrance of the space) and the direction parallel to the side surface. Adjust to.

If it is determined in step S141 that the space attribute is “corridor”, the process proceeds to step S143.

In step S143, the wrap-around destination control unit 153 adjusts the position and direction of the movement destination when the space attribute is “corridor” to the position where the wrap-around is performed on the back side of the wall and the direction parallel to the wall.

When the processing in step S142 or S143 is completed, the processing returns to step S15 in FIG. 6 and the subsequent processing is executed. Note that the position and direction of the movement destination adjusted in the process of step S142 or S143 can be determined by the process of the first example or the second example described above, for example.

So far, the flow of processing when changing the wraparound method according to the space attribute behind the obstacle object has been described. Next, a specific example of the third example will be described with reference to FIGS.

FIG. 19 shows an example of the attributed space in the model room.

In FIG. 19, the same first space attribute is assigned to the LDK and the two Western rooms. The second space attribute is assigned to the Japanese-style room, and the third space attribute is assigned to the washroom, bathroom, and toilet. Furthermore, a fourth space attribute is assigned to the hallway.

For example, for a room such as an LDK or a Western-style room assigned with the first space attribute, the position and direction of the destination are parallel to the side of the wall (near the entrance of the space) and the side. The direction is determined (S142 in FIG. 18).

FIG. 20 shows a state in which the user 20 looks at the edge of the wall that partitions the room as the obstacle object 30 and the edge becomes the gaze area 200.

In FIG. 20, since the wraparound destination is a room, the position (viewpoint) of the user 20 is at a position wrapping around the side surface of the obstacle object 30 (wall) based on the first space attribute that is “room”. While being moved, the direction of the line of sight is parallel to the side surface of the obstacle object 30 (wall) (arrow 320 in the figure). At this time, the head-mounted display 1 displays an image that has instantaneously moved near the entrance of the room (next to the wall).

In addition, for example, for a corridor to which the fourth space attribute is assigned, the position and direction of the destination are determined as a position that turns back around the wall and the direction parallel to the wall (see FIG. 18 S143).

FIG. 21 shows a state in which the user 20 looks at the edge of the wall that partitions the corridor as the obstacle object 30 and the edge becomes the gaze area 200.

In FIG. 21, since the wraparound destination is a corridor, the position (viewpoint) of the user 20 is turned around on the back side of the obstacle object 30 (wall) based on the fourth space attribute of “corridor”. While moving to the position, the direction of the line of sight is parallel to the obstacle object 30 (wall) (arrow 320 in the figure). At this time, the head-mounted display 1 displays an image that has instantaneously moved in the corridor (a place that has gone around to the back side of the wall).

In addition, although the space attribute of the room and the hallway was illustrated here, the washroom, the bathroom, the toilet, etc. to which the second space attribute is assigned and the third space attribute are similarly applied. For each spatial attribute, processing for determining the position of the movement destination and the direction of the line of sight is executed. For example, the space attribute can be set for each room in advance. Furthermore, here, both the position (viewpoint) of the user 20 and the direction of the line of sight are adjusted according to the space attribute, but at least one of them may be adjusted.

In this way, by using a method of changing the way of wrapping according to the attributes of the space behind the obstacle object 30, the user 20 is looking at a certain part of the obstacle object 30 (when gazing). In order to move to the position around the obstacle object 30 according to the attribute of the target space (instantaneous movement), it is more appropriate for each target space in a situation where it is desired to go around the obstacle object 30. It is possible to move the virtual space.

(Fourth example)
Next, referring to FIG. 22 to FIG. 24, as a fourth example of the single edge wraparound destination determination process, when determining the position of the movement destination, the amount of inclination of the body of the user 20 or the gaze of the edge portion is determined. The case where the method of adjusting the amount of wraparound according to parameters such as time will be described.

FIG. 22 is a flowchart for explaining the flow of processing when adjusting the amount of wraparound according to the amount of inclination of the user's body as a fourth example of the wraparound destination determination processing for single edge.

In step S161, the wraparound destination control unit 153 calculates the amount of inclination of the body of the user 20 who is gazing at the edge of the obstacle object 30 based on the sensor data detected by the sensor unit 101.

Here, for example, the tilt amount of the body of the user 20 can be detected based on sensor data detected by an acceleration sensor or a gyro sensor. Note that as the amount of inclination of the body of the user 20, in addition to the amount of inclination of the entire body, for example, the amount of inclination of a part of the body such as the head may be used.

In step S162, the wraparound destination control unit 153 compares the calculated body tilt amount with a preset threshold value to determine whether the body tilt amount is larger than the threshold value.

If it is determined in step S162 that the amount of body tilt is greater than the threshold, the process proceeds to step S163.

In step S163, the wraparound destination control unit 153 adjusts the wraparound amount according to the inclination amount of the body of the user 20 when determining the position and direction of the movement destination. Here, for example, the amount of sneaking around the obstacle object 30 can be increased as the inclination of the body of the user 20 is increased. Further, the position of the movement destination adjusted in the process of step S163 can be determined by the processes of the first example and the second example described above, for example.

If it is determined in step S162 that the amount of body tilt is smaller than the threshold value, the process in step S163 is skipped. That is, in this case, when the position of the movement destination is determined, adjustment according to the amount of body tilt is not performed. When the process of step S163 ends or is skipped, the process returns to step S15 of FIG. 6, and the subsequent processes are executed.

The flow of processing when adjusting the amount of wraparound according to the inclination of the user's body has been described above. Next, a specific example of the fourth example will be described with reference to FIG.

FIG. 23 shows an example of adjustment of the amount of wraparound according to the amount of inclination of the body of the user 20.

23A shows a state where the user 20 has watched the edge of the obstacle object 30 and the edge has become the watch area 200. At this time, since the body (for example, the head) of the user 20 is not tilted (it is determined that the body tilt amount is smaller than the threshold (“NO” in S162 in FIG. 22)), The amount of wraparound is not adjusted.

On the other hand, FIG. 23B shows that when the edge of the obstacle object 30 becomes the gaze area 200, the body of the user 20 is tilted (because it is determined that the amount of body tilt is greater than the threshold ( 22 is adjusted according to the amount of body tilt (S163 in FIG. 22).

Here, the larger the amount of inclination of the body of the user 20 is, the larger the amount of wraparound at the position of the movement destination is, so that an image that is instantaneously moved is displayed on the back side of the obstacle object 30. An image that matches the psychology of the user 20 who is looking into the back side of the object 30 while tilting his / her body is displayed.

Further, in FIG. 23, an arrow 320 corresponding to the amount of wraparound is illustrated, but by presenting this arrow 320 (annotation display), the user 20 can move immediately before moving to the back side of the obstacle object 30. It is possible to predict in advance how much it will wrap around. In other words, it can be said that the arrow 320 is specific information for specifying at least one of the user's position (viewpoint) and the direction of the line of sight before moving the position of the user 20. Note that the arrow 320 may be hidden.

Further, the parameters for adjusting the amount of wraparound are not limited to the amount of inclination of the body of the user 20, but other parameters such as, for example, the gaze time of the edge portion of the obstacle object 30 by the user 20 and the gesture of the user 20 May be used. Hereinafter, the case where the gaze time of the edge part is used as an example of another parameter will be described.

FIG. 24 is a flowchart for explaining the flow of processing when the wraparound amount is adjusted according to the gaze time of the edge portion as a fourth example of the single edge wraparound destination determination processing.

In step S181, the wraparound destination control unit 153 calculates the gaze time of the edge portion of the obstacle object 30 by the user 20 based on the sensor data detected by the sensor unit 101.

In step S182, the wraparound destination control unit 153 compares the calculated gaze time with a preset threshold value and determines whether or not the gaze time is greater than the threshold value.

If it is determined in step S182 that the gaze time is greater than the threshold, the process proceeds to step S183.

In step S183, the wraparound destination control unit 153 adjusts the wraparound amount according to the gaze time when determining the position and direction of the movement destination. Here, for example, as the gaze time of the edge portion of the obstacle object 30 moves in a longer state, the amount of sneaking around the obstacle object 30 can be increased. Further, the position of the movement destination adjusted in the process of step S183 can be determined by the process of the first example or the second example described above, for example.

If it is determined in step S182 that the gaze time is smaller than the threshold value, the process in step S183 is skipped. That is, in this case, when determining the position of the movement destination, adjustment according to the gaze time is not performed. When the process of step S183 ends or is skipped, the process returns to step S15 of FIG. 6 and the subsequent processes are executed.

The flow of processing when adjusting the amount of wraparound according to the gaze time of the edge has been described above.

In the fourth example, the processing when the amount of inclination of the body of the user 20 and the gaze time of the edge portion of the obstacle object 30 are used as parameters for adjusting the amount of wraparound has been described as separate processing. However, processing using these parameters simultaneously may be executed. For example, when executing the processing, adjustment is made so that the amount of sneaking around the obstacle object 30 increases as the amount of inclination of the body of the user 20 is large and the gaze time of the edge portion is long. Can do.

In this way, by using a method of adjusting the amount of wraparound according to parameters such as the amount of body tilt of the user 20, the gesture, and the gaze time of the edge portion, the user 20 is viewing a portion where the obstacle object 30 is present. In the situation where the obstacle object 30 is moved (instantaneous movement) to the position where the obstacle object 30 is wrapped around with the amount of wraparound according to the parameter (when gazing), The virtual space that matches the psychology of the user 20 can be moved.

(Fifth example)
Next, referring to FIG. 25, as a fifth example of the single-edge wraparound destination determination process, a method of removing the target object as an obstacle is used as the preceding process for determining the position of the movement destination. Will be explained.

FIG. 25 shows an example in which the user 20 opens the door when gazing around the door. 25, A to C in FIG. 25 are arranged in time series in that order.

FIG. 25A shows a state where the user 20 in the hallway in the virtual space is looking at the closed door on the right side. At this time, the right door viewed by the user 20 is automatically opened (B in FIG. 25). Here, for example, when the control unit 100 determines that the gaze time of the right door by the user 20 is longer than the threshold based on the sensor data detected by the sensor unit 101, the door is opened. To be.

Thereafter, B in FIG. 25 shows a state in which the user 20 gazes near the entrance of the opened right door (near the edge of the wall) and the edge becomes the gaze area 200. At this time, as shown in FIG. 25C, the position (viewpoint) of the user 20 is moved to a position (around the entrance of the opened right door) around the wall, and the direction of the line of sight is moved. Be directed to the previous room.

As described above, as a first stage process for determining the position of the movement destination, the user 20 performs an operation of opening the door by opening the door when the user 20 is gazing around the door. Instead, it is possible to move to a position around the wall simply by turning the line of sight L _S. Here, the door has been described as an example of the obstacle object, but the object other than the door may be removed as the preceding process for determining the position of the movement destination.

(Sixth example)
Next, with reference to FIG. 26, a sixth example of the single-edge wraparound destination determination process will be described in which a method of displaying an overhead view video when a specific area is being watched is used. .

FIG. 26 shows an example of a case where the user 20 displays an image of an overhead view when watching the vicinity of the boundary between the wall and the ceiling.

FIG. 26A shows a state in which the user 20 in the room in the virtual space gazes near the edge of the boundary between the ceiling and the wall, and the edge becomes the gaze area 200. At this time, as shown in FIG. 26B, the viewpoint of the user 20 is switched to the overhead viewpoint, and an image of the model room including the room in which the user 20 is present viewed from obliquely above is displayed.

As described above, by displaying the video of the overhead view when the vicinity of the boundary between the wall and the ceiling is being watched, the user 20 can grasp the entire image of the model room at an arbitrary timing. Here, as an example of the condition for displaying the overhead view video, the vicinity of the boundary between the wall and the ceiling is exemplified. However, when the other specific area is watched, the overhead view video is displayed. May be. The model room is also an example, and an image of an overhead view can be displayed when a specific area is watched even in another virtual space.

(Seventh example)
Finally, with reference to FIG. 27, a case of using a method of displaying a wraparound destination image for the obstacle object 30 will be described as a seventh example of the single edge wraparound destination determination process.

FIG. 27 shows an example in the case of displaying a video image of the obstacle wrapping around the obstacle object 30.

In FIG. 27, when the user 20 is in a passage in a certain virtual space, the edge portion of the obstacle object 30 is watched, and the edge becomes the watch area 200. At this time, since the position of the wraparound destination with respect to the obstacle object 30 is a blind spot, it is not visible to the user 20, but the image of the wraparound destination is displayed in the small screen display area 400.

The user 20 can check the video of the wraparound destination and determine whether to move the obstacle object 30 to the wraparound destination. If it is determined to move to the destination, the above-described processing is performed, so that the position (viewpoint) of the user 20 is moved to the position around the obstacle object 30.

Here, the obstacle object 30 can include, for example, display shelves of retail stores such as department stores and supermarkets, in addition to the walls that partition the model room. In this case, for example, when the user 20 gazes at the edge portion of the display shelf, the small screen display area 400 pops up, and the image of the point around the shelf is displayed.

That is, in the head mounted display 1, when the position of the user's line of sight L _S is in a peripheral area (for example, an edge) of the obstacle object 30 arranged in the virtual space, the position of the user 20 in the direction of the line of sight L _S. By moving the (viewpoint) and changing the direction of the line of sight, the image visually recognized by the user 20 can be presented not only on the entire visual field area of the user 20 but also on only a part of the visual field area. . In other words, the edge of the obstacle object 30 becomes the gaze area 200, the position (viewpoint) and direction of the destination are determined, and the user 20 moves the viewpoint when presenting an image recognized by the user 20. It can be said that there are cases where the viewpoint of the user 20 is not moved and cases where the viewpoint of the user 20 is not moved.

Thus, by displaying the image of the wraparound destination, the user 20 can determine whether or not to move the obstacle object 30 to the wraparound destination. It is possible to make a more appropriate judgment in situations where it is desired to get around.

In the foregoing, the description has been given regarding the details of the single edge wraparound destination determination process. Note that the first to seventh examples of the single edge wraparound destination determination process described above are examples, and the single edge wraparound destination determination process is performed when one edge exists in the vicinity of the gazing point. Another process for determining the position of the wraparound destination with respect to the obstacle object 30 may be executed.

(Process to determine the wraparound destination from the upper end of the obstacle)
Next, the details of the wraparound destination determination process from the obstacle upper end point corresponding to the process of step S16 of FIG. 6 will be described with reference to FIGS.

FIG. 28 is a flowchart for explaining the flow of the wraparound destination determination process from the obstacle upper end point.

In step S201, the wraparound destination control unit 153 determines a wraparound point for an obstacle object having no edge. Here, for example, a wraparound point can be determined using a ray casting method.

In step S202, the wraparound destination control unit 153 determines the position (wraparound position) and direction of the movement destination according to the determined wraparound point.

When the process in step S202 is completed, the process returns to step S16 in FIG. 6, and the subsequent processes are executed.

The flow of the wraparound destination determination process from the obstacle upper end point has been described above. Next, a specific example of a method of going around to an end point of an obstacle object having no edge will be described with reference to FIGS.

FIG. 29 shows an example of a method for determining a wraparound point for an obstacle object 31 having no edge.

In FIG. 29, when the line of sight of the user 20 faces the direction of the obstacle object 31 having no edge, the point on the obstacle object 31 is the end point of the line-of-sight vector s corresponding to the direction of the line of sight. There is a gaze point P _G.

Here, a point used as a wraparound position for the obstacle object 31 is determined using the ray casting method. The ray casting method is a method in which a ray (ray) is emitted (cast) from the viewpoint of the user 20 and a distance to the closest object (object) is measured.

For example, a ray casting range R _RC corresponding to a predetermined range in the front direction of the face such as the viewing angle of the user 20 is set, and a plurality of ray castings are performed within the range of the ray casting range R _RC . In FIG. 29, five light rays Ray (solid line and dotted line arrows) are illustrated, but the outermost light ray Ray (light ray Ray indicated by the solid line arrow) hitting the obstacle object 31 in the left and right direction respectively. The points hit as the rays Ray-A and Ray-B are the end points EP _A and EP _B on the obstacle object 31, respectively.

As a method of selecting one end point EP from the two end points EP _A and EP _B thus obtained, for example, there are the following two methods.

That is, as a first method, as in the above-described FIGS. 15 to 17, the obstacle object 31 is searched in the direction of the vector w along the wall, so that it is first found among the end points EP _A and EP _B. This is a method of selecting the other end point EP. For example, in FIG. 29, when searching in the right-hand direction, the end point EP _B by the ray Ray-B is found first, so this end point EP _B is selected.

As a second method, the light ray Ray having a smaller angle is selected from the angles θ _A and θ _B which are angles formed by the light ray Ray-A and the light ray Ray-B, respectively. For example, in FIG. 29, since θ _A > θ _B , the end point EP _B by the ray Ray- _B is selected.

In this way, the end point EP _B by Ray Ray-B, is determined as the point coupling loop with respect to the obstacle object 31 is not an edge (S201 in FIG. 28).

FIG. 30 shows an example of a method for determining the position (viewpoint) of the movement destination as the wraparound position.

In FIG. 30, the end point EP _B based on the ray Ray- _A is not used because the end point EP _B based on the ray Ray- _B is determined as the wraparound point by the method for determining the wraparound point described above. .

That is, in FIG. 30, the end point EP _B by the ray Ray- _B is used as the position to which the user 20 moves, but here the end point EP _B , that is, the obstacle object 31 and the ray Ray-B, is used. A position away from the object surface of the collision point by a predetermined distance in the normal direction can be set as a position P _M (around position) of the movement destination of the user 20. At this time, the direction of the line of sight of the user 20 after the movement may be directed in another direction, for example, the direction of the ray Ray-B in addition to the same direction as the line of sight before the movement.

In this way, a position away from the determined wraparound point end point EP _B by a predetermined distance is determined as the position P _M (wraparound position) of the user 20 movement destination (S202 in FIG. 28).

Thus, when the user 20 is looking at a certain part of the obstacle object 31 by using the method of moving the user's position (viewpoint) to the position corresponding to the wraparound point with respect to the obstacle object 31 having no edge. Since the object is moved (instantaneous movement) to the position around the obstacle object 31 (when gazing), the virtual space can be moved more smoothly in a situation where it is desired to go around the obstacle object 31. be able to.

Incidentally, in FIG. 30, the position of the user 20 after movement when using the conventional method, are expressed by dotted lines, in the conventional method, only to move the aim position of the gaze point P _G Therefore, after moving from the current position of the user 20 to the vicinity of the obstacle object 31 once, it is necessary to move further to the position around the obstacle object 31. On the other hand, according to the technique of the present technology, it is possible to move directly from the current position of the user 20 to the position around the obstacle object 31 (movement destination position P _M ).

Further, as shown in FIG. 31, for a detailed object group having an arrangement that cannot be entered as the position of the viewpoint of the user 20 because one obstacle object (for example, a tree dense in the forest) is too fine, Clusters allow objects close to each other to be handled as a large block of obstacle objects. For example, in FIG. 31, a collection of a plurality of trees (obstacle objects) is handled as one obstacle object 32-1 and 32-2. Even in this case, the above-described method can be applied to a large group of obstacle objects 32-1 and 32-2 as obstacle objects 32-1 and 32-2 having no edge.

As above, the details of the wraparound destination determination process from the obstacle upper end point have been described. The wraparound destination determination process from the obstacle upper end point described above is an example, and the wraparound destination determination process from the obstacle upper end point is a wraparound destination determination process for the obstacle object 31 when no edge exists near the point of interest. Another process for determining the position of the may be executed.

(Multiple wraparound destination determination process)
Next, with reference to FIGS. 32 to 38, details of the multi-edge wraparound destination determination process corresponding to the process of step S17 of FIG. 6 will be described. Here, examples of the three processes of the first to third examples are shown as the wraparound destination determination process for multiple edges.

(First example)
First, with reference to FIGS. 32 and 33, a case where a method of displaying wraparound destination candidates is used as a first example of the wraparound destination determination process for multiple edges will be described.

FIG. 32 is a flowchart for explaining the flow of processing when a wraparound destination candidate is displayed as a first example of the wraparound destination determination process for multiple edges.

In step S301, the wraparound destination control unit 153 calculates the distance from the gazing point of each edge of the obstacle object 30.

In step S <b> 302, the wraparound destination control unit 153 selects one edge as the target edge among the edges for which the distance from the gazing point is calculated.

In step S303, the wraparound destination control unit 153 determines whether the selected target edge is the edge closest to the gazing point based on the calculated distance from the gazing point of each edge.

If it is determined in step S303 that the edge is closest to the gazing point, the process proceeds to step S304. In step S304, the wraparound destination control unit 153 highlights (for example, highlights) an arrow indicating the wraparound destination of the target edge.

On the other hand, if it is determined in step S303 that the edge is not closest to the gazing point, the process proceeds to step S305. In step S305, the wraparound destination control unit 153 non-highlights (for example, displays lightly) an arrow indicating the wraparound destination of the target edge.

When the process of step S304 or S305 is completed, the process proceeds to step S306. In step S306, the wraparound destination control unit 153 determines whether all edges have been selected.

If it is determined in step S306 that not all edges have been selected, the process returns to step S302, and the processes of steps S302 to S306 are repeated. That is, an unselected edge (other edge) is selected as the target edge (S302), and when the selected other edge is closest to the gazing point, the arrow is highlighted (S304). In this case, the arrow is not highlighted (S305).

Here, the case where the arrow indicating the wraparound destination of the target edge is highlighted is shown here, but any other display form can be used as long as the display form is highlighted. . Similarly, thin display is an example of a non-highlighted display form, and other display forms may be used.

If it is determined in step S306 that all edges have been selected, the process proceeds to step S307. In step S307, the wraparound destination control unit 153 determines whether or not the wraparound destination candidate indicated by the highlighted arrow matches the user's intention.

Here, when an arrow indicating a wraparound of one edge is highlighted (for example, highlighted) and an arrow indicating a wraparound of another edge other than the edge is not highlighted (for example, displayed lightly), It is determined whether the wraparound destination candidate indicated by the highlighted arrow is a wraparound destination that matches the intention of the user 20.

If it is determined in step S306 that the wraparound candidate matches the intention of the user 20, the process proceeds to step S308. In step S308, the wraparound destination control unit 153 determines the position corresponding to the wraparound destination candidate as the position of the movement destination.

If it is determined in step S307 that the wraparound candidate indicated by the highlighted arrow does not match the intention of the user 20, the process of step S308 is skipped. When the process in step S308 ends or is skipped, the process returns to step S17 in FIG. 6 and the subsequent processes are executed. Further, when the wraparound destination candidate does not match the intention of the user 20, for example, the arrow of the other edge is highlighted to determine again whether the wraparound destination candidate matches the intention of the user 20. You may make it do.

So far, the flow of processing when displaying wraparound candidates has been described. Next, a specific example of the first example will be described with reference to FIG.

FIG. 33 shows an example of the display of the wraparound candidate.

As shown in FIG. 33, in a virtual space where obstacle objects 30-1 to 30-3 such as walls are arranged adjacent to each other, the line of sight L _S of the user 20 is between the obstacle objects 30-1 to 30-3. The area indicated by the dotted circle in the figure is the gazing point vicinity area 340.

At this time, the edge 30E-1 of the obstacle object 30-1, the edge 30E-2 of the obstacle object 30-2, and the edge 30E-3 of the obstacle object 30-3 are within the gazing point vicinity region 340. Since it exists, the user 20 is gazing at one of the edges 30E.

Here, the distance from the gazing point of each edge is calculated, and since the edge 30E-3 is determined to be the closest edge from the gazing point among the edges 30E-1 to 30E-3, the obstacle object 30- The arrow 320-3 indicating the wraparound destination of 3 is highlighted (for example, highlighted) (S304 in FIG. 32), and the arrows 320-1 and 320-2 indicating the wraparound destination of the obstacle objects 30-1 and 30-2 are displayed. Is not highlighted (for example, lightly displayed) (S305 in FIG. 32).

In other words, it can be said that the arrow 320 is identification information for identifying the edge 30E closest to the line of sight of the user 20. Here, arrows 320-1 and 320-2 are presented together with the highlighted arrow 320-3, but the arrows 320-1 and 320-2 may be hidden.

If the highlighted arrow 320-3 matches the intention of the user 20, the position corresponding to the wraparound destination candidate is determined as the movement destination position (S308 in FIG. 32), and the user 20 Is moved (instantaneous movement) to a position around the obstacle object 30-3.

In this way, by using the method of displaying the wraparound destination candidates (annotation display), even when there are a plurality of wraparound destination candidates, the user 20 checks his / her wraparound destination with the highlighted arrow, You can move without making a mistake.

(Second example)
Next, a case where a method of deforming (deforming) a specific obstacle object 30 is used as a second example of the multi-edge wraparound destination determination process will be described with reference to FIGS. 34 and 35.

FIG. 34 is a flowchart for explaining the flow of processing when a specific obstacle object is deformed as a second example of the wraparound destination determination processing for multiple edges.

In step S321, the wraparound destination control unit 153 identifies the obstacle object 30 to be deformed. Here, for example, an obstacle object 30 is specified that makes it easy to determine the edge that the user 20 is gazing by deforming its shape or position.

In step S322, the wraparound destination control unit 153 deforms the identified obstacle object 30. In the deformation process, for example, a process such as changing the shape or position of the specified obstacle object 30 is performed so that the edge that the user 20 is gazing at can be easily determined.

In step S323, the wraparound destination control unit 153 determines whether or not the gaze area 200 has been determined. If it is determined in step S323 that the gaze area 200 has not been determined, the determination process in step S323 is repeated.

If it is determined in step S323 that the gaze area 200 has been determined, the process proceeds to step S324. In step S324, the wraparound destination control unit 153 determines the wraparound destination position corresponding to the gaze area 200 as the movement destination position.

When the process in step S324 is completed, the process returns to step S17 in FIG. 6, and the subsequent processes are executed.

So far, the flow of processing when deforming a specific obstacle object has been described. Next, a specific example of the second example will be described with reference to FIG.

FIG. 35 shows an example of a default display of a specific obstacle object 30.

As shown in FIG. 35A, in the virtual space where obstacle objects 30-1 to 30-3 such as walls are arranged adjacent to each other, the line of sight L _S of the user 20 is the obstacle objects 30-1 to 30-3. The space is a gazing point vicinity region 340.

At this time, since the edges 30E-1 to 30E-3 exist in the gazing point vicinity area 340, it is impossible to determine which edge 30E is being watched by the user 20. Therefore, here, the shape and position of the specific obstacle object 30 are deformed and deformed so that the edge 30E being watched by the user 20 can be easily identified (S322 in FIG. 34).

More specifically, as shown in FIG. 35B, among the obstacle objects 30-1 to 30-3, the lateral lengths of the obstacle objects 30-1 and 30-3 are shortened (or left side). ), The space between the obstacle objects 30-1 to 30-3 is widened, and the edge 30E at which the user 20 is gazing can be easily identified. As a result, in FIG. 35B, the back edge of the obstacle object 30-3 is set as the gaze area 200.

Note that after the viewpoint of the user 20 moves to the wraparound destination, the deformed obstacle objects 30-1 and 30-3 are restored to the original state.

In this way, by using the method of deforming the specific obstacle object 30, even when there are a plurality of wraparound points in the vicinity of the gazing point of the user 20, it is easy to determine the edge 30E that the user 20 is gazing at. Therefore, it is possible to move without making a mistake in the wraparound destination.

(Third example)
Finally, with reference to FIGS. 36 to 38, a case where a method of determining the edge centroid position as the movement destination position is described as a third example of the multi-edge wraparound destination determination process.

FIG. 36 is a flowchart for explaining the flow of processing when determining the edge centroid position as the position of the movement destination as a third example of the wraparound destination determination processing for multiple edges.

In step S341, the wraparound destination control unit 153 calculates the position of the center of gravity of the edge.

In step S342, the wraparound destination control unit 153 determines the calculated center of gravity position as the position of the movement destination.

When the process in step S342 is completed, the process returns to step S17 in FIG. 6, and the subsequent processes are executed.

In the above, the processing flow in the case of determining the edge gravity center position as the movement destination position has been described. Next, a specific example of the third example will be described with reference to FIGS.

FIG. 37 shows an example in which the edge barycenter position is set as the movement destination position (viewpoint).

As shown in FIG. 37, in a virtual space where obstacle objects 30-1 to 30-3 such as walls are arranged adjacent to each other, the line of sight L _S of the user 20 is between the obstacle objects 30-1 to 30-3. The space is a gazing point vicinity region 340.

At this time, since the edges 30E-1 to 30E-3 exist in the gazing point vicinity area 340, it is impossible to determine which edge 30E is being watched by the user 20, and the user 20 It is necessary not to go around in an unintended direction. Therefore, here, when a plurality of edges exist in the gazing point vicinity region 340 and it is difficult to discriminate, the position of the center of gravity of the edge 30E is calculated, and the position of the center of gravity is set as the movement destination position.

More specifically, as shown in FIG. 37, the edge 30E-1 of the obstacle object 30-1, the edge 30E-2 of the obstacle object 30-2, and the edge 30E-3 of the obstacle object 30-3. Is present in the gazing point vicinity region 340, for example, the barycentric position g of the edge 30E can be calculated as follows.

FIG. 38 shows a state where the space between the obstacle objects 30-1 to 30-3 is looked down from directly above. Here, as shown in FIG. 38, the gravity center position g of the figure formed by connecting the position P1 of the edge 30E-1, the position P2 of the edge 30E-2, and the position P3 of the edge 30E-3 is calculated. Then, the barycentric position g can be determined as the movement destination position (S342 in FIG. 36). Thereby, after moving to the gravity center position g, the user 20 can gaze at the edge 30E in the direction intended by the user.

Here, the centroid position in the case where the figure formed by connecting the edge positions (the figure corresponding to a plurality of edges) is a triangle is illustrated, but the centroid position g is similarly calculated and moved in other figures. It can be the previous position.

In this way, by using the method of determining the edge gravity center position as the position of the movement destination, when there are a plurality of wraparound points in the vicinity of the gazing point of the user 20, the user 20 wraps around in an unintended direction. In addition, after moving to the center of gravity of the edge, the user 20 can surely move to the wraparound destination intended by the user 20.

The details of the wraparound destination determination process for multiple edges have been described above. Note that the first to third examples of the multiple edge wraparound destination determination process described above are examples, and the multiple edge wraparound destination determination process is an obstacle when there are a plurality of edges in the vicinity of the gazing point. Another process for determining the position of the wraparound destination with respect to the object 30 may be executed.

<2. Modification>

In the above description, the head-mounted display 1 has been exemplified. However, the present technology is not limited to a head-mounted display in a narrow sense. For example, glasses, glasses-type display, glasses-type camera, headphones, headsets (headphones with microphones), earphones It can also be applied to earrings, ear-mounted cameras, hats, hats with cameras, hair bands, and the like. In addition, the present technology is applied not only to the head mounted display 1 configured as a single device, but also to an information processing device (virtual space experience system such as a game machine or a personal computer) connected to the head mounted display 1. The present invention may be applied to some devices among a plurality of devices. The system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing.

In the above description, the three-dimensional video (3D video) displayed on the display unit 103 of the head mounted display 1 has been described. However, the present technology may be a two-dimensional video (2D) displayed on a display device such as a monitor. It can also be applied to (video). Furthermore, in the above description, a virtual reality (virtual reality) is realized that allows a user 20 wearing the head mounted display 1 to display a three-dimensional image and experience a feeling as if they were there. However, the present technology is not limited to virtual reality (VR), and may be applied to augmented reality (AR) that expands the real world by displaying additional information in a real space. Further, the displayed video is not limited to a video in a VR (Virtual Reality) space, and may be another video such as a video in a real space.

<3. Computer configuration>

The series of processes described above (for example, the HMD process shown in FIG. 6) can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer of each device. FIG. 39 is a block diagram illustrating an example of a hardware configuration of a computer that executes the above-described series of processes using a program.

In the computer 1000, a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are connected to each other via a bus 1004. An input / output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a recording unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

The input unit 1006 includes a microphone, a keyboard, a mouse, and the like. The output unit 1007 includes a speaker, a display, and the like. The recording unit 1008 includes a hard disk, a nonvolatile memory, and the like. The communication unit 1009 includes a network interface or the like. The drive 1010 drives a removable recording medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer 1000 configured as described above, the CPU 1001 loads the program recorded in the ROM 1002 or the recording unit 1008 to the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program. A series of processing is performed.

The program executed by the computer 1000 (CPU 1001) can be provided by being recorded on a removable recording medium 1011 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

In the computer 1000, the program can be installed in the recording unit 1008 via the input / output interface 1005 by attaching the removable recording medium 1011 to the drive 1010. The program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the recording unit 1008. In addition, the program can be installed in the ROM 1002 or the recording unit 1008 in advance.

Here, in the present specification, the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing). The program may be processed by a single computer (processor) or may be distributedly processed by a plurality of computers.

Note that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology. For example, the configuration described in the above embodiment may be implemented alone, or may be implemented by combining a plurality of configurations.

Further, each step of the above-described processing can be executed by one apparatus or can be executed by a plurality of apparatuses. Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

In addition, this technique can take the following structures.

(1)
A movement information acquisition unit for acquiring movement information for moving the viewpoint of the user in the virtual space;
A direction information acquisition unit that acquires direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. An information processing apparatus comprising: a display control unit configured to control a display device so as to present an image visually recognized by the user by changing a direction of the line of sight to the user.
(2)
The information processing apparatus according to (1), wherein the viewpoint of the user is moved to a position where movement of the viewpoint of the user can be reduced.
(3)
The information processing apparatus according to (1) or (2), wherein the viewpoint of the user is moved to a position corresponding to the current position of the user with respect to the surrounding area, a position along the object, or a position of an overhead viewpoint. .
(4)
The information processing apparatus according to any one of (1) to (3), wherein the direction of the line of sight is changed to a predetermined direction according to the object or the peripheral area.
(5)
The information processing apparatus according to any one of (1) to (4), wherein the object is an obstacle arranged in the virtual space.
(6)
The information processing apparatus according to (5), wherein the object includes an edge.
(7)
The information processing apparatus according to (6), wherein the viewpoint of the user is moved to a position in the vicinity of a side surface or a back surface of the object with respect to the user-side surface.
(8)
The information processing apparatus according to any one of (1) to (7), wherein the peripheral area is an area including the object at the line of sight and includes an edge of the object.
(9)
The information processing apparatus according to any one of (1) to (8), wherein the movement information includes a detection result of an operation or action of the user or a dwell time of the user's line of sight.
(10)
The information processing apparatus according to any one of (1) to (9), wherein the direction information includes a detection result of the user's line of sight or face direction, or the user's operation or action.
(11)
At the time of moving the user's viewpoint, at least one of the user's viewpoint and the direction of the line of sight is adjusted according to an attribute of a space of the movement destination. (1) to (10) Information processing device.
(12)
The information processing apparatus according to any one of (1) to (11), wherein when the user's viewpoint is moved, the wraparound amount of the movement destination of the user is adjusted according to a predetermined parameter.
(13)
The information processing apparatus according to (5), wherein when the object does not include an edge, the viewpoint of the user is moved to a position corresponding to a predetermined end point on the object.
(14)
When the object includes a plurality of edges, present specific information for specifying an edge closest to the position of the line of sight, deform one or a plurality of the objects, or correspond to the plurality of edges The information processing apparatus according to (5), wherein the viewpoint of the user is moved to the position of the center of gravity of the figure.
(15)
The information processing apparatus according to any one of (1) to (14), wherein the display control unit presents the image only on all or a part of the visual field region of the user.
(16)
The display control unit is configured to move the user's viewpoint in the direction of the line of sight and maintain the direction of the line of sight when the position of the line of sight is in another object that emphasizes a specific position in the virtual space. The information processing apparatus according to any one of (1) to (15), wherein an image visually recognized by the user is presented.
(17)
The display control unit presents specific information for specifying at least one of the user's viewpoint and the direction of the line of sight before moving the user's viewpoint. Any one of (1) to (16) The information processing apparatus described in 1.
(18)
The information processing apparatus according to any one of (1) to (17), configured as a head mounted display.
(19)
Information processing device
Get movement information to move the user's viewpoint in virtual space,
Obtaining direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. An information processing method for controlling a display device so as to present an image visually recognized by the user to the user by changing a direction of the line of sight.
(20)
Computer
A movement information acquisition unit for acquiring movement information for moving the viewpoint of the user in the virtual space;
A direction information acquisition unit that acquires direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. A program for causing an image visually recognized by the user by changing the direction of the line of sight to function as a display control unit that controls a display device to present the image to the user.

1 head mounted display, 11 body unit, 12 head contact unit, 100 control unit, 101 sensor unit, 102 storage unit, 103 display unit, 104 speaker, 105 input terminal, 106 output terminal, 107 communication unit, 151 movement information acquisition Unit, 152 direction information acquisition unit, 153 wraparound destination control unit, 154 viewpoint camera position / posture control unit, 155 display control unit, 1000 computer, 1001 CPU

Claims

A movement information acquisition unit for acquiring movement information for moving the viewpoint of the user in the virtual space;
A direction information acquisition unit that acquires direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. An information processing apparatus comprising: a display control unit configured to control a display device so as to present an image visually recognized by the user by changing a direction of the line of sight to the user.
The information processing apparatus according to claim 1, wherein the viewpoint of the user is moved to a position where movement of the viewpoint of the user can be reduced.
The information processing apparatus according to claim 2, wherein the viewpoint of the user is moved to a position corresponding to the current position of the user with respect to the peripheral area, a position along the object, or a position of an overhead viewpoint.
The information processing apparatus according to claim 1, wherein the direction of the line of sight is changed to a predetermined direction according to the object or the peripheral area.
The information processing apparatus according to claim 1, wherein the object is an obstacle arranged in the virtual space.
The information processing apparatus according to claim 5, wherein the object includes an edge.
The information processing apparatus according to claim 6, wherein the viewpoint of the user is moved to a position in the vicinity of a side surface or a back surface of the object with respect to a user-side surface.
The information processing apparatus according to claim 1, wherein the peripheral area is an area including the object at the position of the line of sight and includes an edge of the object.
The information processing apparatus according to claim 1, wherein the movement information includes a detection result of an operation or action of the user or a dwell time of the user's line of sight.
The information processing apparatus according to claim 1, wherein the direction information includes a detection result of the user's line of sight or face direction, or an operation or action of the user.
The information processing apparatus according to claim 1, wherein at the time of moving the user's viewpoint, at least one of the user's viewpoint and the direction of the line of sight is adjusted according to an attribute of the destination space.
The information processing apparatus according to claim 1, wherein when moving the user's viewpoint, the amount of wraparound of the destination of the user is adjusted according to a predetermined parameter.
The information processing apparatus according to claim 5, wherein when the object does not include an edge, the viewpoint of the user is moved to a position corresponding to a predetermined end point on the object.
When the object includes a plurality of edges, present specific information for specifying an edge closest to the position of the line of sight, deform one or a plurality of the objects, or correspond to the plurality of edges The information processing apparatus according to claim 5, wherein the viewpoint of the user is moved to the position of the center of gravity of the figure.
The information processing apparatus according to claim 1, wherein the display control unit presents the image only on all or only part of the visual field region of the user.
The display control unit is configured to move the user's viewpoint in the direction of the line of sight and maintain the direction of the line of sight when the position of the line of sight is in another object that emphasizes a specific position in the virtual space. The information processing apparatus according to claim 1, wherein an image visually recognized by the user is presented.
The information processing apparatus according to claim 1, wherein the display control unit presents specific information for specifying at least one of the viewpoint of the user and the direction of the line of sight before moving the viewpoint of the user.
The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as a head-mounted display.
Information processing device
Get movement information to move the user's viewpoint in virtual space,
Obtaining direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. An information processing method for controlling a display device so as to present an image visually recognized by the user to the user by changing a direction of the line of sight.
Computer
A movement information acquisition unit for acquiring movement information for moving the viewpoint of the user in the virtual space;
A direction information acquisition unit that acquires direction information corresponding to the direction of the user's line of sight;
When the position of the user's line of sight is in the peripheral region of the object arranged in the virtual space, the user's viewpoint is moved in the direction of the line of sight based on at least one of the acquired movement information and the direction information. A program for causing an image visually recognized by the user by changing the direction of the line of sight to function as a display control unit that controls a display device to present the image to the user.