JP2020170501A - Information processing device, information processing method, and program - Google Patents

Abstract

To reduce deviation of a display position of a portion of the body of a protagonist from a position of a portion of the body of a user corresponding thereto.SOLUTION: An information processing device has: first input means (an HMD motion detection unit 14) for inputting detection information from a sensor which detects motion of the head of a user; generation means (a VR space image generation unit 13) for generating a virtual space which includes a body image indicating at least a part of the body of the user in accordance with the detection information inputted by the first input means; second input means (an input unit 12) for inputting correction information from a device on which an operation for correcting the angle of the head of the user in a pitch direction is performed; correction means (a pitch angle correction amount setting unit 11 and the VR space image generation unit 13) for correcting, on the basis of the correction information inputted by the second input means, the angle in the pitch direction when the virtual space is presented on a display; and presentation means (the VR space image generation unit 13) for presenting, to the display, the virtual space for which the angle in the pitch direction has been corrected by the correction means.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, technology for providing a virtual space using a head-mounted display has rapidly become widespread. It has been pointed out that with such technology, whether or not the user can get an "immersive feeling" that makes the experience feel as if it were a reality is an extremely important factor in providing emotion and satisfaction. ing.

In Patent Document 1, when a posture angle detecting device is attached to a target part of a human body and the posture angle of the target part is measured, the mounting angle of the posture angle detecting device with respect to the target part is stored in advance as an initial posture angle. When detecting the posture angle of the human body, the angle velocity and acceleration of the posture angle detection device and the initial posture angle are used for arithmetic processing to correct the mounting angle error of the posture angle detection device with respect to the target part, and the posture related to the target part. A technique for detecting an angle with high accuracy is disclosed.

According to such a technology, the actual movement of the user wearing the head-mounted display on the head matches the output attitude angle of the head-mounted display, so that the image displayed on the head-mounted display can be used. There is no sense of discomfort with the user's feeling, and the user can view the image on the head-mounted display with a natural feeling.

Japanese Unexamined Patent Publication No. 2004-150900

By the way, it is known that the display of parts of the body such as hands and feet may enhance the immersive feeling, and in the future, it will be more positively in the virtual space from the first person perspective. It is assumed that a part of the body of the main character will be displayed.

In this way, when displaying a part of the body of the main character from the first person perspective, the position of the part of the body of the main character expressed on the screen is large with the position of the corresponding part of the body of the actual user. If they are different, the immersive feeling will be significantly impaired.

However, Patent Document 1 cannot solve such a problem.

Therefore, the present invention provides an information processing device, an information processing method, and a program capable of suppressing the deviation between the display position of a part of the body of the main character and the position of a part of the body of the corresponding user. It is intended to be provided.

In order to solve the above problems, the present invention inputs detection information from a sensor that detects the movement of the user's head in an information processing device that presents a virtual space on a display mounted on the user's head. A first input means, a generation means for generating the virtual space including a body image showing at least a part of the user's body according to the detection information input by the first input means, and a generation means of the user's head. When presenting the virtual space on the display based on the second input means for inputting the correction information from the device in which the operation for correcting the angle in the pitch direction is performed and the correction information input by the second input means. It is characterized by having a correction means for correcting the angle in the pitch direction of the above, and a presentation means for presenting the virtual space in which the angle in the pitch direction is corrected by the correction means to the display.
According to such a configuration, it is possible to suppress the deviation between the display position of a part of the body of the main character and the position of a part of the body of the corresponding user.

Further, the present invention has a storage means for storing the correction information input from the second input means, and the correction means has a pitch of the virtual space based on the correction information stored in the storage means. It is characterized by correcting the angle of direction.
According to such a configuration, it is possible to continuously correct the angle in the pitch direction based on the stored correction information.

Further, in the present invention, when the correction means newly specifies the correction angle in the pitch direction, the pitch direction of the virtual space is set to the previous value of the correction information stored in the storage means as an initial value. It is characterized in that the correction value of the angle of is specified.
With such a configuration, re-correction can be easily performed.

Further, in the present invention, the device has an operation unit capable of operating in a direction corresponding to the front-back direction or the up-down direction of the user, and the correction means corresponds to the front-back direction or the up-down direction of the operation unit. It is characterized in that the angle in the pitch direction when presenting the virtual space on the display is corrected with reference to the operation amount related to the direction of operation.
According to such a configuration, adjustment can be performed intuitively and easily by operating in the front-rear direction or the up-down direction.

Further, in the present invention, the device can specify at least one point in the virtual space, and the correction means refers to a position designated by the device and presents the virtual space on the display. It is characterized in that the angle in the pitch direction of is corrected.
According to such a configuration, the angle in the pitch direction can be easily adjusted by using the pointing device.

Further, the present invention has an acquisition means for acquiring an angle in the pitch direction of the body image in the virtual space, and the correction means refers to the angle in the pitch direction acquired by the acquisition means to display the display. It is characterized in that the angle in the pitch direction when presenting the virtual space is corrected.
According to such a configuration, the angle in the pitch direction can be automatically set.

Further, the present invention is characterized in that the correction means displays a gauge indicating a correction amount of an angle in the pitch direction in a part of the virtual space based on the correction information input from the second input means. And.
According to such a configuration, the correction amount can be accurately grasped by referring to the gauge.

Further, the present invention comprises a first input step of inputting detection information from a sensor that detects the movement of the user's head in an information processing method for presenting a virtual space on a display mounted on the user's head. According to the detection information input in the first input step, the generation step of generating the virtual space including the body image showing at least a part of the user's body and the pitch direction angle of the user's head are modified. Based on the second input step for inputting the correction information from the device on which the operation is performed and the correction information input in the second input step, the angle in the pitch direction when presenting the virtual space on the display is determined. It is characterized by having a correction step for correction and a presentation step for presenting the virtual space in which the angle in the pitch direction is corrected in the correction step to the display.
According to such a method, it is possible to suppress the deviation between the display position of a part of the body of the main character and the position of a part of the body of the corresponding user.

Further, the present invention detects the movement of the user's head in the computer-readable program that causes the computer to function information processing that presents a virtual space on a display mounted on the user's head. A first input means for inputting detection information from a computer, and a generation means for generating the virtual space including a body image showing at least a part of the user's body according to the detection information input by the first input means. A second input means for inputting correction information from a device on which an operation for correcting the pitch direction angle of the user's head is performed, and the display on the display based on the correction information input by the second input means. It is characterized in that it functions as a correction means for correcting an angle in the pitch direction when presenting a virtual space, and a presenting means for presenting the virtual space whose angle in the pitch direction is corrected by the correction means to the display.
According to such a program, it is possible to suppress the deviation between the display position of a part of the body of the main character and the position of a part of the body of the corresponding user.

According to the present invention, it is provided an information processing device, an information processing method, and a program capable of suppressing a deviation between a display position of a part of the body of the main character and a position of a part of the body of a corresponding user. Can be done.

This is a configuration example of a system including an information processing device according to an embodiment of the present invention. It is a figure which shows the configuration example of the information processing apparatus shown in FIG. It is a figure which shows the structural example of the HMD shown in FIG. It is a figure which shows the three axial directions which the HMD shown in FIG. 3 has. It is an external perspective view of the operation device shown in FIG. It is a figure which shows the electric configuration example of the operation device shown in FIG. It is a figure which shows the positional relationship of the VR space image intended by the creator and the like. It is a figure which shows the positional relationship of the actual VR space image. It is a figure which shows the positional relationship of the VR space image after pitch angle correction. It is a figure which shows the positional relationship of the VR space image intended by the creator and the like. It is a figure which shows the positional relationship of the actual VR space image. It is a figure which shows the positional relationship of the VR space image after pitch angle correction. It is a flowchart for demonstrating an example of the process executed in embodiment shown in FIG. It is a figure which shows the other configuration example of the operation device shown in FIG. It is a figure which shows the other structural example of the operation device shown in FIG. It is a figure which shows an example of the display screen at the time of adjusting a pitch angle using a pointing device. It is a flowchart for demonstrating an example of other processing executed in embodiment shown in FIG. It is a flowchart for demonstrating an example of still more processing executed in embodiment shown in FIG.

Next, an embodiment of the present invention will be described.

(A) Explanation of Configuration of Embodiment of the Present Invention FIG. 1 is a diagram showing a configuration example of an information processing system including an information processing apparatus according to the embodiment of the present invention. As shown in FIG. 1, the information processing device system according to the embodiment of the present invention includes an information processing device 10, an HMD (Head Mount Display) 30, an operating device 50, and a network 70.

Here, as will be described later, the information processing device 10 is configured by a personal computer or the like, and is VR (Virtual Reality) based on, for example, a program or data supplied from a server (not shown) connected to the network 70. A spatial image is generated and supplied to the HMD 30, and the VR spatial image is updated according to the movement of the head of the user wearing the HMD 30. Further, when the operation device 50 is operated by the user, the information processing device 10 acquires the operation amount and executes a process (described later) according to the operation amount.

The HMD 30 is worn by the user on the head and displays the VR spatial image supplied from the information processing device 10 on the built-in display, and detects the movement of the user's head and supplies the VR space image to the information processing device 10. ..

The operation device 50 is operated by the user to generate and output information according to the amount of operation.

The network 70 is configured by, for example, the Internet or the like, and transmits information as an IP packet between a server (not shown) and the information processing device 10.

FIG. 2 is a block diagram showing an example of an electrical configuration of the information processing device 10. As shown in FIG. 2, the information processing device 10 includes a pitch angle correction amount setting unit 11, an input unit 12, a VR space image generation unit 13, an HMD motion detection unit 14, an avatar generation unit 15, and a communication unit 16. are doing.

Here, the pitch angle correction amount setting unit 11 sets the amount of correction of the pitch angle of the VR spatial image based on the information input from the operation device 50.

When the operation device 50 is operated, the input unit 12 inputs information indicating the operation amount from the operation device 50.

The VR space image generation unit 13 generates a VR space image to be constructed around the user and supplies it to the HMD 30.

The HMD motion detection unit 14 inputs information about the motion of the HMD 30 output from an acceleration sensor (described later) included in the HMD 30, and detects the motion of the HMD 30 to detect the motion of the user's head.

The avatar generation unit 15 generates an image of at least a part of the body of the avatar as an incarnation, which is the main character of the first person viewpoint.

The communication unit 16 accesses a server (not shown) via the network 70, acquires information stored in the server, and transmits information to the server.

FIG. 3 is a block diagram showing an example of electrical configuration of the HMD 30. As shown in FIG. 3, the HMD 30 includes a processor 31, a memory 32, a set of displays 33, a plurality of acceleration sensors 34, a plurality of LEDs (Light Emitting Diodes) 35, a communication unit 36, a speaker 37, and a microphone (hereinafter referred to as a microphone). , Simply referred to as a "microphone") 38.

The HMD 30 is used by being worn on the user's head. The HMD 30 has a pair of left and right displays 33 for displaying the VR space image supplied from the information processing device 10 to the left and right eyes of the user, and is located between the display 33 and the eyes of the user. It has a pair of left and right optical lenses for expanding the viewing angle, and an acceleration sensor 34 for detecting the movement of the user's head.

Here, the processor 31 controls each part of the device based on the program and the data stored in the memory 32.

The memory 32 is composed of, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), or the like, and stores programs and data executed by the processor 31.

The display 33 presents the VR spatial image supplied from the information processing device 10 to the left and right eyes of the user.

The acceleration sensor 34 is composed of, for example, a gyro sensor. As shown in FIG. 4, the acceleration sensor 34 has a roll angle θr as an angle in the roll direction, which is a direction in which the user's head is bent left and right, and an angle in the yaw direction, which is a direction in which the user swings his face left and right. It is detected and output as the respective accelerations of the yaw angle θy as and the pitch angle θp as the angle in the pitch direction which is the raising and lowering direction of the jaw.

Return to FIG. The LED 35 constitutes, for example, a light emitting marker, and is configured to be arranged so that the position of the HMD 30 can be detected with respect to the information processing device 10.

The communication unit 36 is wirelessly or wiredly connected to the information processing device 10, receives a VR space image from the information processing device 10, and processes information indicating acceleration in the three axes of the HMD 30 detected by the acceleration sensor 34. It is transmitted to the device 10.

The speaker 37 converts the voice information supplied from the information processing device 10 via the communication unit 36 into voice and emits the sound. Headphones may be used instead of the speaker 37.

The microphone 38 converts the voice emitted by the user into an electric signal and outputs it.

FIG. 5 is an external perspective view showing a configuration example of the operation device 50. As shown in FIG. 5, the operation device 50 has, for example, a main body portion 501 made of resin, and the user grips and uses the main body portion 501. Further, a flat surface portion 502 on which the operation portion 503 is arranged is formed at the end portion of the main body portion 501. By rotating the operation unit 503, the user can adjust the pitch angle when presenting the VR space image as described later.

FIG. 6 is a diagram showing an example of an electrical configuration of the operation device 50. As shown in FIG. 6, the operation device 50 includes a processor 51, a memory 52, an operation amount detection unit 53, a sensor group 54, a communication unit 55, and a vibration unit 56.

Here, the processor 51 controls each part of the device based on the program and the data stored in the memory 52.

The memory 52 is composed of RAM, ROM, and the like, and stores programs and data executed by the processor 51.

The operation amount detection unit 53 detects and outputs, for example, the operation amount of the operation unit 503 shown in FIG. More specifically, in the example of FIG. 5, the operation unit 503 has a dial-type structure, and the operation amount detection unit 53 is configured by a rotary encoder that detects the rotation angle of the dial-type operation unit 503. Generates and outputs information indicating the amount of operation.

The sensor group 54 is composed of, for example, a tilt sensor or the like, and detects and outputs the angle of the operating device 50.

The communication unit 55 is wirelessly or wiredly connected to the information processing device 10, for example, receives information for controlling the vibration unit 56 from the information processing device 10, and is detected by the operation amount detection unit 53 and the sensor group 54. The information is transmitted to the information processing device 10.

The vibrating unit 56 is composed of, for example, a vibration motor, and is controlled by the processor 51 to generate vibration to vibrate the main body 501 of the operating device 50.

(B) Description of Operation of Embodiment of the Present Invention Next, the operation of the embodiment of the present invention will be described. In the following, after explaining the outline of the operation of the embodiment of the present invention, the details of the operation will be described with reference to the flowchart.

7 to 12 are diagrams for explaining the outline of the operation of the present invention. FIG. 7 shows a situation in which the environment in the sea is presented as a VR space image, and is assumed by the creator or photographer of the VR space image (hereinafter, referred to as “creator or the like”). It is a figure. In the example of FIG. 7, the environment in the sea (details are omitted for simplification of the drawing) is presented to the user 90 as a VR space image. Further, in the example of FIG. 7, the fin 100 as a part of the body of the avatar (incarnation) of the user 90 is presented at a position corresponding to the foot of the user 90. Further, in the example of FIG. 7, the shark 110 is attacking the fin 100. That is, the creator or the like assumes that the user 90 will view the image in a sitting position, for example, and that the user 90's feet are located about 45 degrees below the front direction of the virtual camera. Produces or photographs spatial images (hereinafter referred to as "production, etc.").

FIG. 8 shows the actual viewing situation of the user 90. In the example of FIG. 8, the user 90 is viewing the VR space image not in the sitting position but in the supine position lying on the sofa 92 or the like, for example. In such a case, the conventional information processing device that presents the VR space image does not consider the posture of the user 90. Therefore, as shown in FIG. 8, the fin 100 is located at a position away from the foot of the user 90. Is displayed. In that case, since it is difficult for the user 90 to recognize the fin 100 as a part of his / her body, even if the shark 110 attacks such a fin 100, he / she does not feel that it is an attack against himself / herself. Have difficulty.

One of the hero's bodies shown in the VR space image when another person such as an enemy, monster, or friend tries to approach or touch the body of the hero from the first-person perspective in the VR space. Conventionally, there has been a problem that the immersive feeling is significantly impaired when the position of the portion is different from the position of the actual user 90.

Therefore, in the embodiment of the present invention, the user 90 operates the operation unit 503 of the operation device 50 at the start of reproduction of the VR space image or during the reproduction of the VR space image, so that the pitch angle is as shown in FIG. Correct θp and adjust so that the image of a part of the body of the user 90's avatar or another person who is trying to interfere with the part of the body is presented at the position assumed by the creator etc. It is characterized by performing.

10 to 12 show the case of different contents. FIG. 10 shows the relationship between the VR spatial image intended by the creator and the like and the user 90. That is, in the example of FIG. 10, the user 90 is in a standing position, and the dog 120 is present under the feet of the user 90.

FIG. 11 shows a state at the time of actual viewing. In the example of FIG. 11, the user 90 is in a sitting position in which the upper body is tilted from the vertical direction. In such a case, the dog 120 is displayed floating in the air, not at the feet of the user 90. If a hand that is part of the user 90's avatar is displayed in the VR space image at the corresponding position and the dog 120 is stroked by the avatar's hand, it is difficult to recognize this hand as its own hand. Is.

FIG. 12 shows a display example when the display position is adjusted according to the present embodiment. In the present embodiment, the pitch angle θp is adjusted as shown in FIG. 12 by the user 90 operating the operation unit 503 of the operation device 50 at the start of reproduction of the VR space image or during the reproduction of the VR space image. Since the VR space image can be modified so as to be presented at the position assumed by the creator or the like, the dog 120 can be displayed at the feet of the user 90. In this case, a hand that is a part of the user 90's avatar is displayed, and when the dog 120 is stroked by the avatar's hand, this hand can be recognized as its own hand.

By making adjustments in this way to match a part of the body of the avatar with a part of the body of the user 90, it is possible to enhance the sense of possession of the avatar and thereby enhance the immersive feeling.

The following papers are known regarding the feeling of physical ownership.

For example, in Article 1 (Botvinick M., and Cohen J .: "Rubber hands'feel' touch that eyes see", Nature, 391 (6669): 756, (1998)), in the field of view presented to the subject. If you put a rubber prosthesis on the surface and perform prior training to touch the rubber prosthesis at the same timing as touching the actual hand of the subject in a place hidden from the visual field, it will be made of rubber due to the illusion of the brain. It is described that the visual information that the artificial limb is touched causes a feeling of being touched (RHI: Rubber Hand Illusion). A series of phenomena similar to this is called bodily sensation transfer or physical possession, and has been studied extremely enthusiastically in recent years.

Also, in Paper 2 (Slater M., Spanlang B., Sanchez-Vives MV, and Blanke O .: “First Person Experience of Body Transfer in Virtual Reality,” PLoS ONE, 5 (5): e10564, (2010)) It is stated that a sense of physical possession can be obtained not only from the first-person perspective but also from the third-person perspective.

In addition, Paper 3 (Peck TC, Seinfeld S., Aglioti SM, and Slater M .: "Putting yourself in the skin of a black avatar reduces implicit racial bias", Consciousness and cognition, 22 (3): 779-787, ( In 2013)), it is stated that a feeling of physical possession can be obtained regardless of the difference in the skin color of the displayed avatar.

In addition, Paper 4 (Kondo R., Sugimoto M., Minamizawa K., Hoshi T., Inami M., and Kitazaki M .: "Illusory body ownership of an invisible body interpolated between virtual hands and feet via visual-motor synchronicity" , Scientific reports, 8: 7541 ((2018)), only the hands and feet are displayed in synchronization with the movement of the subject, even when the human body that should be connected between them is displayed as if it were a transparent human. , The fact that the transparent part gives a feeling of physical possession is shown.

On the other hand, in Paper 5 (Pavani F .: "Visual capture of touch: Out-of-the-body experiences with rubber gloves", Psychological Science, 11 (5): 353-359, (2000)), the prosthetic hand was placed. It is stated that the smaller the difference between the angle of and the angle at which the actual hand should be seen, the stronger the sense of physical possession. That is, as a natural property of human cognition, the display of the main character's body in the virtual space causes a transfer of bodily sensation as close as possible to the position where the user's body is placed in the real space. It is suggested that it is easy and therefore tends to be immersive.

That is, the present embodiment pays attention to various properties shared by many contents related to virtual reality, based on these scientific properties naturally possessed by the cognitive mechanism of the human brain. Therefore, for example, it is intended to provide a technical solution to the situation shown in FIG.

Further, the present embodiment is characterized in that the pitch angle θp is particularly paid attention to among the three angles (θr, θy, θp) detected from the movement of the head of the user 90 and used for adjusting the line of sight. Is. In other words, the posture of Avatar as the main character from the first-person perspective may change in various ways as the story progresses, but when the human eye sees his body in sight, he bends his head. Being (movement in the roll axis direction) and looking back to the left and right (movement in the yaw axis direction) are relatively rare events. Assuming that the protagonist is not angry and is facing almost the front of the object of interest without shifting to the left or right, in many cases there is no problem.

Your body in the field of view is often near the front center of the field of view, and you can often expect it to be within your shoulder width. At this time, the appearance changes greatly depending on the raising and lowering of the jaw (movement in the pitch axis direction). The shape of the body in the field of view changes according to human movements such as standing, sitting, bending, and lying down, but these changes are mainly adjusted by adjusting the pitch angle. Can be done.

Therefore, in the present invention, in addition to the HMD motion detection unit 14 that detects the movement of the line of sight based on the roll angle θr, the yaw angle θy, and the pitch angle θp from the sensor that detects the movement of the user's head, in particular, the vertical direction of the line of sight. By providing the pitch angle correction amount setting unit 11 for enabling forced movement (pitch up, pitch down) based on the intention of the user 90 with respect to the pitch angle that controls the change of the avatar as the main character of the first person viewpoint. It is possible to reduce the discrepancy between the physical expression of the user and the actual posture of the user.

Since the human brain has a great ability to adjust, it is expected that a great psychological effect that can maintain an immersive feeling can be obtained simply by adjusting the rough overlap of the positions of the limbs. Therefore, in the present embodiment, the body of the avatar, which is the main character of the first-person viewpoint, and the actual posture of the body of the user 90 are roughly aligned by the operation device 50, and the user 90 makes his / her own intention. By operating the operation device 50 accordingly, it is possible to provide a virtual space with a higher immersive feeling.

FIG. 13 is a flowchart for explaining an example of the process executed in the embodiment of the present invention. Hereinafter, the detailed operation of the present invention will be described with reference to the flowchart shown in FIG. When the processing of the flowchart shown in FIG. 13 is started, the following steps are executed.

In step S10, the HMD motion detection unit 14 of the information processing device 10 detects the initial position of the HMD 30 with reference to the output of the acceleration sensor 34 of the HMD 30. That is, for example, the initial roll angle θr0, the initial yaw angle θy0, and the initial pitch angle θp0 are detected as the initial positions of the HMD 30 in the triaxial direction. For example, when θr0 = 0, θy0 = 0, and θp0 = 0, the roll angle θr, yaw angle θy, and pitch angle θp, which are the reference for drawing, are θr = 0 and θy = 0, respectively. , And θp = 0. Instead of the output of the acceleration sensor 34, the LED 35 provided in the HMD 30 may be turned on, and these may be used as markers to detect the initial position by image recognition.

In step S11, the pitch angle correction amount setting unit 11 substitutes “0” as the initial value of the pitch angle correction amount θpc.

In step S12, the avatar generation unit 15 executes a process of generating an avatar image of the user 90 based on the roll angle θr, the yaw angle θy, and the pitch angle θp as the initial positions detected in step S10. For example, in the example of FIG. 7, the avatar generation unit 15 generates an image of the fin 100 attached to a part of the body of the avatar of the user 90.

In step S13, the VR spatial image generation unit 13 executes a process of generating a VR spatial image based on the roll angle θr, the yaw angle θy, and the pitch angle θp as the initial positions detected in step S10. For example, in the example of FIG. 7, an image showing the environment under the sea is generated, and the avatar image generated in step S12 is superimposed to generate a VR space image.

In step S14, the VR space image generation unit 13 outputs the VR space image generated in step S13 to the HMD 30. As a result, in the HMD 30, for example, the communication unit 36 receives the VR space image and supplies it to the processor 31. The processor 31 displays the VR spatial image supplied from the communication unit 36 on the display 33.

In step S15, the HMD motion detection unit 14 detects the position of the HMD 30 with reference to the information supplied from the acceleration sensor 34 of the HMD 30. More specifically, the HMD motion detection unit 14 detects motion in the roll axis direction, the yaw axis direction, and the pitch axis direction shown in FIG.

In step S16, the pitch angle correction amount setting unit 11 refers to the information input from the input unit 12 and determines whether or not the operation unit 503 of the operation device 50 has been operated for pitch up or pitch down. If it is determined that the operation has been performed (step S16: Y), the process proceeds to step S17, and in other cases (step S16: N), the process proceeds to step S18. For example, when the operation unit 503 is rotated by the user's thumb or the like, it is determined as Y and the process proceeds to step S17.

In step S17, the pitch angle correction amount setting unit 11 executes a process of updating the pitch angle correction amount θpc according to the operation amount of the operation device 50 detected in step S16 with respect to the operation unit 503. For example, in the example of FIG. 9, as an example, “−35 degrees” corresponding to the operation amount of the operation unit 503 is input, so the pitch angle correction amount θpc = −35 is set. Regarding the conversion from the operation amount of the operation unit 503 to the pitch angle correction amount θpc, it is desirable that the sensitivity can be adjusted, for example, on the option screen of the system.

In step S18, the VR spatial image generation unit 13 executes a process of correcting the pitch angle according to the pitch angle correction amount updated in step S17. For example, in the example of FIG. 9 described above, since the pitch angle correction amount θpc = −35, the VR spatial image generation unit 13 executes a process of correcting the pitch angle θp by −35 degrees (a process of setting θp ← θp + θpc). To do.

In step S19, the avatar generation unit 15 executes a process of generating an avatar image of the user 90 based on the pitch angle modified in step S18. For example, in the example of FIG. 9 described above, the avatar generation unit 15 generates an image of the fins 100 attached to a part of the body of the avatar of the user 90 based on the pitch angle θp modified in step S18.

In step S20, the VR space image generation unit 13 executes a process of generating a VR space image. For example, in the example of FIG. 9 described above, an image showing the underwater environment is generated and an image of the shark 110 is generated based on the pitch angle θp corrected in step S18. Then, the avatar image generated in step S12 is superimposed on these images to generate a VR space image.

In step S21, the VR space image generation unit 13 outputs the VR space image generated in step S20 to the HMD 30. As a result, in the HMD 30, for example, the communication unit 36 receives the VR space image and supplies it to the processor 31. The processor 31 displays the VR spatial image supplied from the communication unit 36 on the display 33. As a result, as shown in FIG. 9, the fin 100, which is an avatar image, and the VR space image in which the positions of the shark 110 are appropriately adjusted are displayed on the display 33 of the HMD 30. As described above, it is known that the human brain causes a transfer of bodily sensation between the main character and the self in the VR space, and has a sense of physical possession. At that time, the visual expression about the hero's body provided in the virtual space has the property that the closer the body is to the position where it should be, the easier it is to get that feeling. Therefore, by modifying the pitch angle θp, for example, in the example of FIG. 9, the feeling of physical possession of the fin 100 can be strengthened. As a result, the attack of the shark 110 can be strongly recognized as an attack on itself, so that the feeling of immersion in the VR space can be enhanced.

In step S22, the VR spatial image generation unit 13 determines whether or not to end the processing, ends the processing when it is determined to end the processing (step S22: Y), and ends the processing in other cases (step S22). : N) returns to step S15 and repeats the same process as described above.

According to the above processing, for example, as shown in FIG. 8, when the position of the fin 100 which is a part of the avatar image and the position of the foot of the user 90 do not match, the operation of the operation device 50 is performed. By manipulating the unit 503 to correct the pitch angle θp and matching them as shown in FIG. 9, the immersive feeling in the VR space can be enhanced.

(C) Description of Modified Embodiment The above embodiment is an example, and it goes without saying that the present invention is not limited to the cases described above. For example, in the above-described embodiment, the case where the operation device 50 as shown in FIG. 5 is used has been described as an example, but for example, the operation devices 50A and 50B shown in FIG. 14 or 15 may be used. .. FIG. 14 is an example of an operation device 50A having a lever-type operation unit 503A. That is, in the example of FIG. 14, the pitch angle can be corrected by operating the operation unit 503A with the thumb or the like of the user 90. When the operation unit 503A is continuously operated, the pitch angle changes at a predetermined speed (for example, 5 degrees / sec when the maximum operation is performed) according to the operation amount (tilt) of the operation unit 503A. Can be done.

FIG. 15 is a configuration example of an operation device 50B having button-type operation units 503B and 503C. That is, in the configuration example of FIG. 15, the desired angle can be adjusted by operating the operation unit 503B when pitching up and operating the operation unit 503C when pitching down. When the operation units 503B and 503C are continuously operated, the pitch angle can be changed at a predetermined speed (for example, 5 degrees / sec). In the operation devices shown in FIGS. 5, 14, and 15, the front-back direction is associated with the pitch angle for adjustment, but the user's vertical direction is associated with the pitch angle for adjustment. You may do so.

Further, in order to avoid loss of immersion in the VR space, it is desirable to use the operation devices 50, 50A, 50B described above, but when the operation device is not used for the purpose of simplifying the system configuration or the like. As an alternative means, for example, an operation screen as shown in FIG. 16 may be displayed in the VR space, and the pitch angle correction amount may be input from the operation screen.

More specifically, the operation screen shown in FIG. 16 may be operated by using a pointing device attached to the HMD 30 or the information processing device 10 for operating or designating an object in the VR space.

In the display example of FIG. 16, the pitch angle adjustment gauge is set for a certain period of time (for example, 10 seconds) by clicking within a band having a certain width at the left and right edges of the screen (hereinafter referred to as “gauge display area”) with a pointing device. It is displayed, and you can specify the pitch amount (increase / decrease amount) by clicking one point on the gauge. It is desirable that the upper part of the gauge is pitched up and the lower part is pitched down. The pitch amount represented by the number line of the gauge may be, for example, in the range of plus 45 degrees (pitch up) to minus 45 degrees (pitch down).

In the case of FIG. 16, it is desirable that the sensitivity (the magnitude of the pitch angle correction amount represented by the gauge) can be adjusted on the option screen of the system as in the case of the operation device 50. When a point on the gauge is clicked, the gauge display and click acceptance are continued for the same time (10 seconds in the above example) after reflecting the pitch angle adjustment. As a result, the user can repeat the pitch angle adjustment several times to make fine adjustments to a suitable angle.

The pitch angle adjustment gauge is preferably drawn in a non-transparent color so as not to be affected by the brightness of the virtual space. In addition, although gauge display areas are provided on both the left and right ends as standard, it is desirable that the option setting allows the user to select whether to install only on the left or right side. Further, even if a point outside the range of the pitch angle adjustment gauge is clicked, it can be treated as an effective input and the pitch angle can be adjusted. This makes it possible to click on an object of interest in the virtual space and adjust the pitch angle so that it is in front of the line of sight.

FIG. 17 is an example of a flowchart for setting the pitch angle correction amount based on the display screen shown in FIG. In FIG. 17, the same reference numerals are given to the portions corresponding to the flowchart shown in FIG. 13, and the description thereof will be omitted. In FIG. 17, as compared with FIG. 13, step S16 is excluded, and steps S30 to S32 are newly added. Therefore, in the following, steps S30 to S32 will be mainly described.

In step S30, the pitch angle correction amount setting unit 11 determines whether or not one point in the VR space has been designated by the pointing device, and if it is determined that the point has been designated (step S30: Y), the process proceeds to step S31. In other cases (step S30: N), the process proceeds to step S18. For example, in FIG. 16, when a gauge display area at the left and right edges of the screen is clicked by the pointing device and then one point on the gauge is clicked, it is determined as Y and the process proceeds to step S31.

In step S31, the pitch angle correction amount setting unit 11 identifies the clicked point. For example, in FIG. 16, a point on the gauge clicked by the pointing device is identified.

In step S32, the pitch angle correction amount setting unit 11 calculates the pitch angle correction amount θpc from the designated point. For example, in FIG. 16, the pitch angle correction amount θpc is calculated from the position of one point on the gauge clicked by the pointing device.

In step S17, the pitch angle correction amount setting unit 11 updates the existing pitch angle correction amount θpc with the pitch angle correction amount θpc calculated in step S32.

By the above processing, the pitch angle correction amount can be updated using the display screen shown in FIG. According to such a modified embodiment, the pitch angle correction amount can be set by using the existing pointing device without adding a new operating device.

Further, in the above embodiment, the pitch angle correction amount is manually input by the user, but is automatically adjusted when the posture of the presented avatar changes based on the intention of the creator or the like. You may try to do it. Specifically, information indicating the posture of the avatar is stored in the data for generating the VR space image, and the pitch angle correction amount is automatically updated based on the information indicating the posture of the avatar. You may.

FIG. 18 is a flowchart for realizing such a process. In FIG. 18, the parts corresponding to those in FIG. 13 are designated by the same reference numerals, and the description thereof will be omitted. In FIG. 18, the processes of steps S50 and S51 are added as compared with FIG. Other than these, the same as in FIG. 13, and therefore, the processing of step S50 and step S51 will be mainly described below. In order to realize the following processing, it is necessary to embed, for example, a pitch angle correction amount corresponding to the posture of the avatar in advance in the data for generating the VR space image.

In step S50, the pitch angle correction amount setting unit 11 acquires the pitch angle correction amount θpc corresponding to the currently presented VR space image scene from, for example, the VR space image generation unit 13. For example, the pitch angle correction amount θpc embedded in the data for generating the VR space image is acquired according to the posture of the avatar presented in the scene. When the scene of the VR space image changes and the posture of the avatar changes based on the intention of the creator or the like, the pitch angle correction amount θpc changes. Therefore, when the pitch angle correction amount θpc changes, a new pitch angle correction amount θpc is acquired from, for example, the VR spatial image generation unit 13 or the avatar generation unit 15.

In step S51, the pitch angle correction amount setting unit 11 updates the existing pitch angle correction amount θpc with the pitch angle correction amount θpc acquired in step S50. For example, when the pitch angle correction amount of the immediately preceding scene is θpc1 and the pitch angle correction amount of the new scene is θpc2, the pitch angle correction amount θpc2 is newly set.

In step S16 and step S17, for example, the pitch angle correction amount is set according to the operation of the operation device 50 described above. In step S17, the pitch angle correction amount θpc can be set by increasing or decreasing the value according to the operation of the operation device 50 with respect to the pitch angle correction amount θpc updated in step S51.

According to the above processing, the pitch angle can be automatically corrected every time the scene changes, so that the user 90 can roughly save the trouble of manually setting the pitch angle correction amount, and the user 90 can use the operating device 50. It is possible to make only minor corrections by operation.

In the above-described embodiment, the case where the VR space image is drawn by computer graphics has been described as an example, but the present invention is not limited to computer graphics, for example, the whole sky. A live-action image captured by a camera capable of capturing an image may be used. That is, the present invention may be applied to image-based rendering or image-based modeling. When the live-action image is used, the avatar is included in the live-action image. In that case, the avatar generation unit 15 can be excluded from the configuration shown in FIG. Of course, a 180 ° image may be used instead of the entire sky (360 °) image.

Further, in the modified embodiment shown in FIG. 18, the pitch angle correction amount according to the scene is embedded in the data for generating the VR space image. For example, the presented VR space image is expressed in computer graphics. In the case of a captured image, instead of embedding the pitch angle correction amount of the main character image included in the image, an image analysis processing unit is added to the configuration shown in FIG. 3, and the analysis processing by the image analysis processing unit is performed. The appropriate pitch angle correction amount may be specified from the image, and the specified pitch angle correction amount may be set.

Further, in the above embodiment, the process shown in FIG. 13 and the like is executed by the information processing device 10, but the configuration of the information processing device 10 is incorporated in the HMD 30 and the process shown in FIG. 13 and the like is performed by the HMD 30. You may want to do it.

Further, in the configuration example shown in FIG. 1, the network 70 and the server are provided, but the configuration may not include the network 70. Further, the information stored in the server may be stored in the information processing device 10 or the HMD 30.

Further, in the above embodiment, the pitch angle is adjusted, but the yaw angle and the roll angle may be adjusted as needed.

10 Information processing device 11 Pitch angle correction amount setting unit 12 Input unit 13 VR spatial image generation unit 14 HMD motion detection unit 15 Avatar generation unit 16 Communication unit 30 HMD
31 Processor 32 Memory 33 Display 34 Accelerometer 35 LED
36 Communication unit 37 Speaker 38 Microphone 50, 50A, 50B Operation device 51 Processor 52 Memory 53 Operation amount detection unit 54 Sensor group 55 Communication unit 56 Vibration unit 70 Network 90 User 92 Sofa 100 Fin 110 Same 120 Dog 501 Main unit 502 Flat unit 503, 503A, 503B, 503C operation unit

Claims (9)

In an information processing device that presents a virtual space on a display mounted on the user's head
A first input means for inputting detection information from the sensor that detects the movement of the user's head, and
A generation means for generating the virtual space including a body image showing at least a part of the user's body according to the detection information input by the first input means.
A second input means for inputting correction information from the device on which the operation of correcting the angle of the user's head in the pitch direction is performed, and
Based on the correction information input by the second input means, the correction means for correcting the angle in the pitch direction when presenting the virtual space on the display, and the correction means.
A presentation means for presenting the virtual space whose angle in the pitch direction has been corrected by the correction means to the display, and
An information processing device characterized by having. It has a storage means for storing the correction information input from the second input means.
The correction means corrects the angle in the pitch direction of the virtual space based on the correction information stored in the storage means.
The information processing apparatus according to claim 1. When the correction means newly specifies the correction angle in the pitch direction, the correction means sets the correction value of the angle in the pitch direction of the virtual space with the previous value of the correction information stored in the storage means as an initial value. The information processing apparatus according to claim 2, wherein the information processing apparatus is designated. The device has an operation unit capable of operating in a direction corresponding to the front-back direction or the up-down direction of the user.
The correction means corrects the angle in the pitch direction when presenting the virtual space on the display with reference to the operation amount of the operation unit in the direction corresponding to the front-back direction or the up-down direction.
The information processing device according to any one of claims 1 to 3, wherein the information processing device is characterized by the above. The device can specify at least one point in the virtual space.
The correction means refers to a position designated by the device and corrects an angle in the pitch direction when presenting the virtual space to the display.
The information processing device according to any one of claims 1 to 3, wherein the information processing device is characterized by the above. It has an acquisition means for acquiring an angle in the pitch direction of the body image in the virtual space.
The correction means corrects the pitch direction angle when presenting the virtual space on the display with reference to the pitch direction angle acquired by the acquisition means.
The information processing apparatus according to any one of claims 1 to 5, characterized in that. Claims 1 to 1, wherein the correction means displays a gauge indicating a correction amount of an angle in the pitch direction in a part of the virtual space based on the correction information input from the second input means. The information processing apparatus according to any one of 6. In an information processing method that presents a virtual space on a display mounted on the user's head
The first input step of inputting the detection information from the sensor that detects the movement of the user's head, and
A generation step of generating the virtual space including a body image showing at least a part of the user's body according to the detection information input in the first input step.
A second input step of inputting correction information from the device on which the operation of correcting the angle of the user's head in the pitch direction is performed, and
Based on the correction information input in the second input step, a correction step of correcting the angle in the pitch direction when presenting the virtual space on the display, and a correction step.
A presentation step of presenting the virtual space in which the angle in the pitch direction is corrected in the correction step to the display,
An information processing method characterized by having. In a computer-readable program that causes a computer to function information processing that presents virtual space on a display mounted on the user's head.
The computer
A first input means for inputting detection information from a sensor that detects the movement of the user's head,
A generation means for generating the virtual space including a body image showing at least a part of the user's body according to the detection information input by the first input means.
A second input means for inputting correction information from the device on which the operation of correcting the angle of the user's head in the pitch direction is performed.
A correction means for correcting an angle in the pitch direction when presenting the virtual space on the display based on the correction information input by the second input means.
A presentation means for presenting the virtual space whose angle in the pitch direction has been corrected by the correction means to the display.
A program characterized by functioning as.