JP2019220185A - Information processing device and image forming method - Google Patents

Abstract

To provide a technique for implementing display control suitable for a head mounted display.SOLUTION: A motion detecting unit 30 detects posture of a head mounted display mounted on a head of a user, and a line of sight direction determining unit 32 determines a direction of the line of sight according to the detected posture of the head mounted display. An image generating unit 34 generates an image based on the determined direction of the line of sight, and an image providing unit 36 provides the generated image to the head mounted display. The line-of-sight direction determining unit 32 defines the direction of the line of sight in which a rotation angle with respect to a detected horizontal reference direction of the head mounted display is reversed.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a technique for generating an image to be displayed on a head mounted display.

  A head mounted display (HMD) is mounted on the user's head to provide the user with a virtual reality (VR) world. Recently, an application that allows a user to play a game while viewing a screen displayed on the HMD has appeared. In a conventional stationary display such as a television, since the user's field of view extends to the outside of the screen, the user may not be able to concentrate on the screen or lack a sense of immersion in the game. On the other hand, when the user wears the HMD, only the video displayed on the HMD is seen by the user, so that the user feels more immersive in the video world and has an effect of further enhancing the entertainment of the game. If the HMD is provided with a head tracking function and the display screen is updated in conjunction with the posture of the user's head, the feeling of immersion in the video world is further improved.

  2. Description of the Related Art In recent years, omnidirectional cameras (omnidirectional cameras) that take 360-degree panoramic photographs in all directions (up, down, left, and right) have become widespread. Development of unmanned flying objects that can be remotely controlled is also underway, and by mounting multiple cameras on such flying objects, panoramic photography in all directions from top to bottom, left and right can be taken from the air. The omnidirectional panoramic image captured in this manner is displayed on the HMD, and the head tracking function updates the display screen in conjunction with the posture of the user's head, so that the user can feel as if they are in a real place. Expected to be given.

JP 2015-95045 A

  As a recent trend to enhance the sense of reality of an image, development for widening the viewing angle of the HMD has been activated. HMDs with a viewing angle exceeding 80 degrees are already on the market, and HMDs with a wide viewing angle provide users with a video world that is comparable to the real world.

  The inventor of the present invention has made various attempts to display an image on an HMD having a wide viewing angle, and as a result, has recognized that depending on how the image is displayed, the user may fall into a sense of being intoxicated. In this specification, such a sensation is referred to as “motion sickness (simulator sickness)”. When the same image is viewed on a television screen, even if no image sickness occurs, the image sickness may occur on the HMD screen. Also, it has been recognized that when an information element such as a menu is included in the display image, the user may feel uncomfortable depending on how the information element is presented. The present inventor has come up with a display control suitable for the HMD based on knowledge obtained through various trials.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide a technique for realizing display control suitable for an HMD.

  In order to solve the above-described problem, an information processing apparatus according to an aspect of the present invention includes a detecting unit that detects a posture of a head-mounted display mounted on a user's head, and a posture of the head-mounted display detected by the detecting unit. A gaze direction determining unit that determines the gaze direction, an image generation unit that generates an image based on the determined gaze direction, and an image providing unit that provides the generated image to the head mounted display. The line-of-sight direction determination unit determines a line-of-sight direction in which the detected rotation angle of the head-mounted display with respect to the horizontal reference direction is inverted.

  Another embodiment of the present invention relates to an image generating method. The method includes the steps of: detecting a posture of a head-mounted display mounted on a user's head; determining a gaze direction according to the detected posture of the head-mounted display; and generating an image based on the determined gaze direction. And The step of determining the line-of-sight direction determines a line-of-sight direction in which the detected rotation angle of the head-mounted display with respect to the horizontal reference direction is inverted.

  It should be noted that any combination of the above-described components, the expression of the present invention, a method, an apparatus, a system, a computer program, a recording medium on which the computer program is readable recorded, a data structure, and the like are also included in the present invention. This is effective as an embodiment.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which implement | achieves the display control suitable for HMD can be provided.

FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment. FIG. 11 is an explanatory diagram of a rendering process in the information processing device. It is a figure showing an example of appearance shape of HMD. FIG. 3 is a diagram illustrating functional blocks of the HMD. FIG. 3 is a diagram illustrating functional blocks of the information processing apparatus. FIG. 4 is a diagram for describing panoramic image data stored in an image storage unit. It is a figure showing a display image. It is a figure showing a display image. It is a figure showing a display image. It is a figure showing the upper surface and the back side of an input device. It is a figure showing a display image. It is a figure which shows the display image before and after switching. It is a figure showing the example which superimposed and displayed the information element on the display image. It is a figure showing signs that an information element moves with a panorama image. It is a figure showing the information element superimposed and displayed on the panorama image. It is a figure showing the information element superimposed and displayed on the panorama image.

  FIG. 1 illustrates a configuration example of an information processing system 1 according to the embodiment. The information processing system 1 includes an information processing device 10, a head-mounted display device (HMD) 100 that the user wears on the head, an input device 6 that the user operates with his / her fingers, and an imaging device that captures the user wearing the HMD 100. 7 and an output device 4 for displaying an image.

  In the embodiment, the information processing device 10 includes a processing device 12 and an output control device 14. Here, the processing device 12 is a terminal device that receives operation information of the input device 6 operated by the user and executes various applications such as a game. The processing device 12 and the input device 6 may be connected by a cable or may be connected by a known wireless communication technology. The output control device 14 is a processing unit that outputs image data to the HMD 100. The output control device 14 and the HMD 100 may be connected by a cable or may be connected by a known wireless communication technology.

  The imaging device 7 captures an image of the user wearing the HMD 100 and provides the captured image to the processing device 12. The imaging device 7 may be a stereo camera. As will be described later, the HMD 100 is provided with a marker (tracking LED) for realizing tracking of the user's head, and the processing device 12 detects the movement of the HMD 100 based on the position of the captured marker. Note that the HMD 100 is also equipped with a posture sensor (acceleration sensor, gyro sensor), and the processing device 12 acquires sensor information detected by the posture sensor from the HMD 100, thereby achieving high accuracy in conjunction with the use of the captured image of the marker. To implement the tracking process.

  In the information processing system 1, the user does not necessarily need the output device 4 to view the image displayed on the HMD 100, but the output control device 14 or the processing device 12 outputs the same image as the image displayed on the HMD 100 to the output device. 4 may be output. This allows another user to view the image that the user is viewing on the HMD 100 on the output device 4. As will be described later, since an image displayed on the HMD 100 has been subjected to distortion correction of the optical lens, an image that has not been subjected to distortion correction needs to be output from the output device 4.

  In the information processing system 1, the processing device 12, the output device 4, the input device 6, and the imaging device 7 may construct a conventional game system. In this case, the processing device 12 is a game device that executes an application such as a game, and the input device 6 may be a device that supplies user processing information to the processing device 12 such as a game controller, a keyboard, a mouse, and a joystick. By adding the output control device 14 and the HMD 100 to the components of the game system, the information processing system 1 that executes a virtual reality (VR) application is configured.

  The function of the output control device 14 may be incorporated in the processing device 12 as a part of the function of the VR application. That is, the information processing device 10 may be configured by one processing device 12 or may be configured by the processing device 12 and the output control device 14. In the following, the functions of the processing device 12 and the output control device 14 necessary for realizing the VR application will be collectively described as functions of the information processing device 10.

  The information processing device 10 generates image data to be displayed on the HMD 100. In the embodiment, the information processing apparatus 10 prepares a 360-degree panoramic image in all directions, up, down, left, and right, photographed by a spherical camera, and generates the panoramic image based on the gaze direction determined from the posture of the HMD 100 mounted on the user's head. The image is displayed on the HMD 100. The display content may be a still image or a moving image. In addition, the image is not limited to an actually photographed image, and may be an image drawn in real time by a game application.

  The HMD 100 is a display device that displays an image through an optical lens on a display panel positioned in front of the user when the user wears the HMD 100 on his / her head. The HMD 100 independently displays a left-eye image on the left half of the display panel and a right-eye image on the right half of the display panel. These images form a parallax image as viewed from the left and right viewpoints, and are displayed in left and right regions obtained by dividing the display panel into two, so that the images can be stereoscopically viewed. In addition, in order for the user to view the display panel through the optical lens, the information processing apparatus 10 supplies the HMD 100 with image data in which optical distortion due to the lens has been corrected in advance. In the information processing apparatus 10, the correction processing of the optical distortion may be performed by any of the processing apparatus 12 and the output control apparatus 14.

  FIG. 2 is an explanatory diagram of the rendering process in the information processing device 10. In the VR application of the embodiment, a virtual environment in which a user is located at the center of a sphere and changes the direction of a line of sight to change a visible image is realized. The content image as the image material is pasted on the inner peripheral surface of the virtual sphere centered on the center point 9 where the user is located. Here, the content image is a 360-degree panoramic image in all directions of up, down, left, and right taken by the omnidirectional camera, and is pasted on the inner peripheral surface of the virtual sphere such that the top of the content image matches the top of the virtual sphere. Can be As a result, the user's real world top and bottom are aligned with the video world provided to the HMD 100, and a VR application that reproduces a real video world is realized.

  The information processing apparatus 10 detects a rotation angle and a tilt of the user's head (actually, the HMD 100) by performing a head tracking process of the user. Here, the rotation angle of the HMD 100 is a rotation angle with respect to a reference direction of a horizontal plane, and the reference direction may be set, for example, as a direction in which the power of the HMD 100 is turned on. The inclination of the HMD 100 is an inclination angle with respect to a horizontal plane. A known technique may be used as the head tracking process. The information processing apparatus 10 can detect the rotation angle and the tilt of the HMD 100 only from the sensor information detected by the attitude sensor of the HMD 100, By analyzing the image of the marker (LED for tracking), the rotation angle and the inclination of the HMD 100 can be detected with high accuracy.

  The information processing device 10 determines the orientation of the virtual camera 8 in the virtual sphere according to the detected rotation angle and tilt of the HMD 100. The virtual camera 8 is arranged so as to photograph the inner peripheral surface of the virtual sphere from the center point 9 of the virtual sphere, and the information processing device 10 determines the detected rotation angle and tilt and the light of the virtual camera 8 on the virtual sphere. Match the rotation angle and inclination of the shaft. The information processing device 10 acquires the captured image 5 of the virtual camera 8, that is, performs a rendering process, performs optical distortion correction for an optical lens, and supplies image data to the HMD 100. Although one virtual camera 8 is shown in FIG. 2, two virtual cameras 8 for the left eye and the right eye are actually arranged to generate respective image data.

  FIG. 3 shows an example of the external shape of the HMD 100. In this example, the HMD 100 includes an output mechanism 102 and a mounting mechanism 104. The mounting mechanism section 104 includes a mounting band 106 for fixing the HMD 100 to the head by making a round around the head when the user wears the head. The wearing band 106 is made of a material or a structure whose length can be adjusted according to the user's head circumference.

  The output mechanism unit 102 includes a housing 108 shaped to cover left and right eyes when the user wears the HMD 100, and includes a display panel at a position facing the eyes when the HMD 100 is worn. The display panel is realized by a liquid crystal panel, an organic EL panel, or the like. The housing 108 further includes a pair of left and right optical lenses that are located between the display panel and the eyes of the user when the HMD 100 is mounted and that increase the viewing angle of the user. The HMD 100 may further include a speaker or an earphone at a position corresponding to the user's ear when worn.

  Light-emitting markers 110a, 110b, 110c, and 110d are provided on the outer surface of the housing 108. In this example, the tracking LED constitutes the light emitting marker 110, but may be another type of marker. In any case, the marker is photographed by the imaging device 7 and the information processing device 10 can perform image analysis. Good. Although the number and arrangement of the light emitting markers 110 are not particularly limited, the number and arrangement are required to be such that the posture (rotation angle and inclination) of the HMD 100 can be detected by photographing and image analysis. Are provided at four corners on the front surface of the housing 108. Further, the light emitting marker 110 may be provided on a side portion or a rear portion of the wearing band 106 so that the image can be captured even when the user turns his / her back to the image capturing device 7.

  The HMD 100 may be connected to the information processing device 10 by a cable or may be connected by a known wireless communication technology. The HMD 100 transmits the sensor information detected by the attitude sensor to the information processing device 10, receives the image data generated by the information processing device 10, and displays the image data on the display panel.

  Note that the HMD 100 shown in FIG. 3 is an immersive (non-transmissive) display device that completely covers both eyes, but may be a transmissive display device. The shape may be a hat type as shown in the figure, or may be an eyeglass type.

  FIG. 4 shows functional blocks of the HMD 100. The control unit 120 is a main processor that processes and outputs various data such as image data, audio data, and sensor information, and instructions. The storage unit 122 temporarily stores data and instructions to be processed by the control unit 120. The posture sensor 124 detects posture information such as a rotation angle and a tilt of the HMD 100. The posture sensor 124 includes at least a three-axis acceleration sensor and a three-axis gyro sensor. The microphone 126 converts a user's voice into an electric signal. The light emitting markers 110 are LEDs, and are attached to the mounting band 106 and the housing 108 of the HMD 100 in plural numbers.

  The communication control unit 128 transmits data input from the control unit 120 to the external information processing device 10 by wire or wireless communication via a network adapter or an antenna. In addition, the communication control unit 128 receives data from the information processing apparatus 10 by wire or wireless communication via a network adapter or an antenna, and outputs the data to the control unit 120.

  When receiving the image data and the audio data from the information processing device 10, the control unit 120 supplies the image data and the audio data to the display panel 130 to display the image data and the audio data, and supplies the audio output unit 132 to output the audio. Further, the control unit 120 causes the communication control unit 128 to transmit the sensor information from the attitude sensor 124 and the voice data from the microphone 126 to the information processing device 10.

  FIG. 5 shows functional blocks of the information processing apparatus 10. The information processing apparatus 10 includes a sensor information acquisition unit 20, a captured image acquisition unit 22, and an instruction acquisition unit 24 as an input interface with the outside. The sensor information acquisition unit 20 acquires sensor information from the attitude sensor 124 of the HMD 100 at a predetermined cycle. The captured image acquisition unit 22 acquires a captured image of the HMD 100 from the imaging device 7 at a predetermined cycle. For example, the imaging device 7 captures an image every (1/60) second, and the captured image obtaining unit 22 obtains a captured image every (1/60) second. The instruction obtaining unit 24 obtains, from the input device 6, an instruction input by the user.

  The information processing device 10 further includes a motion detecting unit 30, a gaze direction determining unit 32, an image generating unit 34, and an image providing unit 36. The motion detection unit 30 detects the posture of the HMD 100 mounted on the user's head. The gaze direction determination unit 32 determines the gaze direction according to the attitude of the HMD 100 detected by the motion detection unit 30. The image generation unit 34 generates an image in accordance with the detected posture of the HMD 100, and specifically, generates an image based on the gaze direction determined by the gaze direction determination unit 32. The image providing unit 36 provides the generated image to the HMD 100.

  In FIG. 5, each element described as a functional block that performs various processes can be configured by a circuit block, a memory, or another LSI in terms of hardware, and is loaded in the memory in terms of software. It is realized by a program or the like. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof, and the present invention is not limited to any of them.

  The image storage unit 40 stores 360-degree panoramic image data shot in advance. The image storage unit 40 may store a plurality of content images, and the content images may be still images or moving images. The image storage unit 40 of the embodiment stores panoramic image data in all directions, up, down, left, and right, and the information processing apparatus 10 provides a panoramic image in all directions to a user wearing the HMD 100. Therefore, the user turns the head to the left or right (the user may rotate the body counterclockwise or clockwise) to rotate the horizontal line of sight left or right, so that the left or right panoramic image is obtained. The panoramic image displayed on the display panel 130 of the HMD 100 is displayed on the display panel 130 of the HMD 100 when the user tilts his or her head up or down and tilts his / her gaze in the vertical direction. Become.

  The information element storage unit 42 stores information elements to be included in the panoramic image. This information element may be, for example, a menu item for changing the brightness or the like of the content image, or a menu item for selecting a content image which is an image material. When the user inputs an information element display instruction to the input device 6, the image generation unit 34 includes the information element in the generated image and displays the information element superimposed on the panoramic image on the display panel 130 of the HMD 100. Note that the information element may be notification information to the user, and in this case, the information element may be superimposed on the panoramic image without operating the input device 6 by the user.

  FIG. 6 is a diagram for describing panoramic image data stored in the image storage unit 40. For convenience of explanation, FIG. 6 shows a part of panoramic image data in all directions of up, down, left, and right, and omits a part of an image downward from a horizontal plane and a part of an image in a left and right direction. As described with reference to FIG. 2, the display panel 130 of the HMD 100 displays an image obtained by rendering a panoramic image pasted on the inner peripheral surface of the virtual sphere, and the user moves the rotation angle and inclination of the HMD 100 to change the line of sight. Is changed, the displayed panoramic image is moved according to the viewing direction.

  In the HMD 100, the communication control unit 128 transmits the sensor information acquired by the posture sensor 124 to the information processing device 10 at a predetermined cycle. The imaging device 7 captures an image of the HMD 100 at a predetermined cycle, and transmits a captured image to the information processing device 10. Referring to FIG. 5, sensor information acquiring section 20 acquires sensor information of attitude sensor 124 and supplies the information to motion detecting section 30. The photographed image acquiring unit 22 acquires a photographed image and supplies the photographed image to the motion detecting unit 30.

  The motion detection unit 30 performs a head tracking process of detecting the posture of the HMD 100 to detect the posture of the head of the user wearing the HMD 100. This head tracking process is performed in order to link the visual field displayed on the display panel 130 of the HMD 100 with the posture of the user's head. In the head tracking processing of the embodiment, the rotation angle of the HMD 100 with respect to the horizontal reference direction and the inclination angle with respect to the horizontal plane are detected. The horizontal reference direction may be set, for example, as the direction in which the power of the HMD 100 is turned on.

  The head tracking process uses a known method, and the motion detection unit 30 may detect the rotation angle of the HMD 100 with respect to the horizontal reference direction and the inclination angle of the HMD 100 with respect to the horizontal plane only from the sensor information of the posture sensor 124. It is preferable to further utilize the photographing result of the light emitting marker 110 to enhance the detection accuracy. The motion detection unit 30 detects a rotation angle and a tilt angle at a predetermined cycle. For example, if the image supplied to the HMD 100 is 60 fps, it is preferable that the detection process of the motion detection unit 30 is also executed at a period of (1/60) second.

  The gaze direction determination unit 32 determines the gaze direction according to the attitude of the HMD 100 detected by the motion detection unit 30. This gaze direction is the gaze direction of the user, and thus the gaze direction (optical axis direction) of the virtual camera 8 disposed at the center point 9 of the virtual sphere (see FIG. 2). Here, the line-of-sight direction determination unit 32 may determine the rotation angle and the tilt angle detected by the motion detection unit 30 as the line-of-sight direction (optical axis direction) of the virtual camera 8 as it is, or perform some correction processing. The gaze direction of the virtual camera 8 may be determined. For example, when stable sensor information is not provided to the motion detection unit 30 due to noise on the sensor information, the motion detection unit 30 performs a motion that vibrates even though the user's head is not moving. May be detected. In such a case, the line-of-sight direction determination unit 32 may determine the line-of-sight direction by performing smooth correction on the motion detected by the motion detection unit 30.

  In addition, the visual field of a human is vertically asymmetric, and the lower side is slightly wider than the upper side of the line of sight. Therefore, the gaze direction determination unit 32 may determine the gaze direction of the virtual camera 8 by slightly tilting the inclination angle detected by the motion detection unit 30 downward.

  The image generation unit 34 generates an image according to the attitude of the HMD 100 detected by the motion detection unit 30, and specifically generates an image based on the gaze direction of the virtual camera 8 determined by the gaze direction determination unit 32. The image generator 34 defines a left-eye field of view and a right-eye field of view specified by the line-of-sight direction, and renders and generates left-eye and right-eye images, respectively. At this time, the image generation unit 34 generates a panoramic image in which distortion due to the image light from the display panel passing through the optical lens has been corrected.

  The example HMD 100 provides a user with a field of view of about 100 degrees horizontally and about 100 degrees vertically. Referring to FIG. 2, captured image 5 of the virtual sphere is captured at an angle of view of about 100 degrees in the horizontal direction and about 100 degrees in the vertical direction, and displayed on display panel 130 of HMD 100. As described above, since the visual field of a person is slightly wider on the lower side than the upper side of the line of sight, the optical lens and the display panel 130 in the HMD 100 are tilted 5 degrees with respect to the direction directly facing the eyes, and A vertical field of view of 55 degrees on the lower side may be realized by the arrangement of the optical lens and the display panel 130.

  FIG. 7 shows a display image 200a generated by the image generation unit 34. In the following drawings, the display image is expressed as an image cut out from the panoramic image in order to easily understand the positional relationship of the display image in the entire panoramic image.

  The image generation unit 34 generates an image based on the gaze direction 202a determined by the gaze direction determination unit 32. Note that the image generation unit 34 actually renders and generates the left-eye display image and the right-eye display image, and these display images are different images including parallax. Is not specifically described. The image providing unit 36 provides the HMD 100 with the display image 200a generated by the image generating unit 34. In the HMD 100, the control unit 120 displays the display image 200a on the display panel 130, so that the user can see the display image 200a displayed on the display panel 130.

  FIG. 8 shows a display image 200b generated by the image generation unit 34. The gaze direction determination unit 32 determines the gaze direction according to the attitude of the HMD 100, and the image generation unit 34 generates an image based on the determined gaze direction. In this example, the user turns his / her head counterclockwise in the horizontal direction, and the line of sight changes continuously from the line of sight 202a to the line of sight 202b. Here, the user rotates the head approximately 60 degrees counterclockwise, and by this rotation operation, the image generation unit 34 causes the image in which the panorama image is continuously moved in the counterclockwise direction from the display image 200a to be moved for (1/60) second. Is generated in the cycle of The image providing unit 36 provides the generated image to the HMD 100 at a cycle of (1/60) second.

  FIG. 9 shows a display image 200c generated by the image generation unit 34. The gaze direction determination unit 32 determines the gaze direction according to the attitude of the HMD 100, and the image generation unit 34 generates an image based on the determined gaze direction. This example shows a state in which the user tilts his or her head upward from the state in which the display image 200a is displayed on the display panel 130, and the line of sight changes continuously from the line of sight 202a to the line of sight 202c. Here, the user tilts the head upward by about 30 degrees, and by this tilting operation, the image generation unit 34 causes the image in which the panorama image is continuously moved upward from the display image 200a to be displayed at a period of (1/60) second. Generate. The image providing unit 36 provides the generated image to the HMD 100 at a cycle of (1/60) second.

  As described above, the user changes the line-of-sight direction by moving the head, and the information processing apparatus 10 provides the HMD 100 with a panoramic image in a desired viewing direction and causes the display panel 130 to display the panoramic image. Changing the gaze direction by moving the head is the same as the movement in the real world, and matches the user's sense. At this time, the HMD 100 provides the user with a wide viewing angle, so that the sense of immersion in the panoramic image can be further enhanced.

  On the other hand, in order to see a panoramic image right behind, for example, the user must naturally turn right behind. It is not enough to turn your neck to look directly behind, so you need to change your body direction. Therefore, when the user is sitting on a non-rotating chair such as a sofa, the user must stand up and change his body direction. When the HMD 100 is a non-transmissive display device, the user has both eyes covered with the housing 108 and cannot see the surrounding environment. You may feel resistance.

  Therefore, in this embodiment, the user is allowed to input a gaze direction switching instruction from the input device 6 so that the gaze direction can be changed by operating the input device 6 without moving the head. In the embodiment, the input device 6 is shown as a device different from the HMD 100. However, the input device 6 may be provided in the HMD 100, that is, formed as an operation input unit such as a button on the wearing band 106 or the housing 108 of the HMD 100. May be done.

  FIG. 10A shows the upper surface of the input device 6. The user operates the input device 6 by gripping the left grip 78b with the left hand and the right grip 78a with the right hand. On the upper surface of the housing of the input device 6, a direction button 71 as an input unit, analog sticks 77a and 77b, and operation buttons 76 are provided. The direction buttons 71 include an up button 71a, a left button 71b, a down button 71c, and a right button 71d. The right analog stick 77a and the left analog stick 77b are used to input a direction and a tilt amount by being tilted. Note that the right analog stick 77a and the left analog stick 77b also function as push-down buttons that sink down when pressed by the user and return to the original position when the user releases the hand. A touch pad 79 is provided in a flat area between the direction button 71 and the operation button 76 on the upper surface of the housing. The touch pad 79 also functions as a push button that sinks down when pressed by the user and returns to the original position when the user releases his hand.

  A home button 80 is provided between the right analog stick 77a and the left analog stick 77b. The home button 80 is used to turn on the power of the input device 6 and at the same time activate a communication function for wirelessly connecting to the information processing device 10. The SHARE button 81 is provided on the left side of the touch pad 79. The SHARE button 81 is used to input an instruction from the user for the OS or the system software in the information processing device 10. The OPTIONS button 82 is provided on the right side of the touch pad 79. The OPTIONS button 82 is used to input an instruction from a user for an application (game) executed in the information processing apparatus 10. Both the SHARE button 81 and the OPTIONS button 82 may be formed as push buttons.

  FIG. 10B illustrates a rear side surface of the input device 6. A touch pad 79 is bent and extended from the upper surface of the housing on the upper side of the rear side of the housing of the input device 6, and a horizontally long light emitting section 85 is provided on the lower side of the rear side of the housing. The light emitting section 85 has red (R), green (G), and blue (B) LEDs, and lights up according to the emission color information transmitted from the information processing device 10. The upper R1 button 83a and L1 button 83b may be configured as push buttons, and the lower R2 button 84a and L2 button 84b may be configured as a pivotally supported trigger type button.

  In the embodiment, the left analog stick 77b of the operation members of the input device 6 is used for inputting a line-of-sight direction switching instruction. When the left analog stick 77b is tilted to the left, the line of sight moves to the left, and when tilted to the right, the line of sight moves to the right. Thus, the left analog stick 77b is suitable for the user to intuitively change the line of sight. Note that another operation member, for example, the right analog stick 77a, the direction button 71, or the like may be used for inputting a line-of-sight direction switching instruction.

  The inventor has tried various processes of changing the line-of-sight direction with respect to the operation of the left analog stick 77b. First, as one method of the change process, the direction of the line of sight is continuously moved according to the tilt direction of the left analog stick 77b, and an image that moves so that a panoramic image flows is generated. This is a general method of changing the line of sight of a game displayed on a television screen, and is considered to match the user's sense. In addition, various attempts were made for the moving speed of the image at that time.

  As a result, it has been found that, when the panoramic image is continuously moved by operating the left analog stick 77b, a user who looks at the panoramic image feels intoxicated. Regardless of the moving speed of the image, the result of image sickness was obtained, depending on the individual, whether moving slowly or quickly, depending on the individual.

  An effect of continuously moving a display image in one direction on a television screen is often performed, and at this time, the user is not usually sick of an image. After examining this point, the present inventors have found that one of the causes is a difference in viewing angle.

  As for the television screen, the user views the television screen from a certain distance, so that the horizontal viewing angle of the user with respect to the television screen is at most several tens of degrees. On the other hand, in the HMD having a wide viewing angle, the viewing angle of the user is close to 90 degrees or 90 degrees or more (hereinafter, the horizontal viewing angle of the HMD 100 is about 100 degrees). That is, in the HMD 100, the image moves continuously in the range of about 100 degrees (further in the entire field of view), so that the user tends to suffer from image sickness, which is a problem peculiar to the HMD 100. Then, the inventor tried to move the panoramic image discontinuously, not continuously, by operating the left analog stick 77b, and as a result, found that the occurrence of image sickness can be prevented. Therefore, in the embodiment, when the information processing apparatus 10 obtains the operation of the left analog stick 77b as the switching instruction of the line of sight, the information processing apparatus 10 displays the panoramic image discontinuously, that is, the panoramic image is moved by a predetermined angle by switching once (one step). Control.

  When the user tilts the left analog stick 77b of the input device 6, the instruction obtaining unit 24 obtains, from the input device 6, an instruction to switch the line of sight in the tilted direction. For example, when the left analog stick 77b is once tilted down and returned immediately, the instruction obtaining unit 24 obtains one switching instruction in the tilted direction. When the left analog stick 77b continues to be tilted, the instruction obtaining unit 24 continuously obtains a switching instruction in the tilted direction.

  Whether the instruction acquiring unit 24 acquires the tilting operation of the left analog stick 77b as one switching instruction or as a continuous switching instruction depends on the tilted time (from the time of tilting to the time of returning to the original position). Time) exceeds a predetermined time. Specifically, the instruction obtaining unit 24 obtains a single switching instruction when the tilting time is shorter than the predetermined time, and as a switching instruction again when the tilting time reaches the predetermined time when the tilting time reaches the predetermined time. get. Thereafter, when the tilting operation is further continued, a switching instruction may be acquired each time a time shorter than the predetermined time elapses.

  When the instruction acquisition unit 24 acquires an instruction to switch the gaze direction, the gaze direction determination unit 32 changes the gaze direction by a predetermined angle. For example, when the left analog stick 77b is tilted to the left, the instruction obtaining unit 24 obtains a switching instruction to rotate the line of sight to the left, and the line of sight direction determining unit 32 changes the line of sight to a predetermined angle counterclockwise. Hereinafter, an example in which the user inputs a switching instruction to rotate the line of sight left to the input device 6 while the display image 200a illustrated in FIG. 7 is displayed on the display panel 130 of the HMD 100 will be described.

  FIG. 11 shows a display image 200d generated by the image generation unit 34. When the instruction acquisition unit 24 acquires a switching instruction to rotate the line of sight left, the line of sight direction determination unit 32 changes the line of sight direction by a predetermined angle counterclockwise. Here, the predetermined angle is set to an angle larger than 10 degrees and smaller than the horizontal viewing angle of the HMD 100 (100 degrees).

  If the angle to be changed in one step is set to 10 degrees or less, for example, at least 18 steps are required in order to view an image right behind (rotated by 180 degrees), and the number of steps is large, and the user is slightly relieved. It will make you feel. On the other hand, when the change angle in one step is equal to or larger than the viewing angle of the HMD 100, a completely new image (an image having no overlapping portion with that before the switching) is suddenly displayed on the display panel 130, and the image with the image before the switching is not displayed. Continuity cannot be guaranteed. Therefore, it is preferable that the change angle in one step is set to an angle larger than 10 degrees and smaller than the viewing angle of the HMD 100.

  More preferably, the change angle in one step is set to half or less of the viewing angle of the HMD 100. As a result, at least half or more of the image before switching is included in the image after switching, and the user can recognize the continuity of the image before and after switching. In order to make the continuity of the image more recognizable, it is preferable that the change angle in one step is smaller. The inventor tried various change angles per one step, and when the change angle per one step was set to 15 degrees or more and 30 degrees or less, the desired angle was reached when switching continuously. From the viewpoint of time and image continuity, it was recognized that it was most preferable. The example shown in FIG. 11 shows a state in which the change angle in one step is 22.5 degrees.

  It is generally said that a human has a central visual field of about 40 to 45 degrees and a peripheral visual field of about 200 degrees. By paying attention to the central visual field, the change angle per step may be set smaller than the central visual field so that the same image remains before and after switching in the central visual field. From this point, it is significant to set the change angle per step to 30 degrees or less. The change angle per step is set to a value obtained by dividing 360 degrees by an integer N, and the user can return to the original display position by inputting an integer number of switching instructions from the input device 6. Is preferred.

  FIG. 12 shows display images before and after switching displayed on the display panel 130. FIG. 12A shows a display image 200a, and FIG. 12B shows a display image 200d. The display image 200d is obtained by rotating the display image 200a counterclockwise with the change angle of one step being 22.5 degrees. By setting the change angle to 22.5 degrees in the viewing angle of 100 degrees, 22.5% of the images are changed before and after switching, but 77.5% exist in the images before and after switching. Therefore, the user can easily recognize the continuity of the image.

  When the instruction acquiring unit 24 continuously acquires the switching instruction, the line-of-sight direction determining unit 32 changes the line-of-sight direction by a predetermined angle. This change cycle may be a fixed value such as one second, or may be dynamically determined according to the amount of tilt of the left analog stick 77b. That is, the change cycle may be set shorter as the amount of tilt is larger.

  In the embodiment, the switching instruction using the input device 6 is valid only in the horizontal direction, and invalid in the vertical direction. This is because a human is sitting in a chair or standing in an upright position, facing in the horizontal direction, and usually does not always face upward or downward, so that such a sense of reality is also observed in the VR application. . Note that it is also possible to make the switching instruction in the vertical direction valid, and it may be possible to set whether to make the switching instruction in the vertical direction valid or invalid according to the content image. For a content image in which the user does not care about the top and bottom, for example, a captured image of a starry sky in the sky, a vertical switching instruction may be valid.

  FIG. 12 shows the display images 200a and 200d before and after the switching when the change angle in one step is 22.5 degrees, but the change angle may be larger or smaller than 22.5 degrees depending on the user. Sometimes you want. Therefore, an option for selecting a change angle per step may be displayed on the display panel 130, and the user may operate the input device 6 to select an option displayed on the display panel 130. . In an embodiment, the user is provided with 15 and 30 degree options other than 22.5 degrees.

In the following, a description will be given of a procedure for displaying options of a change angle per one step in a state where the display image 200a shown in FIG. 12A is displayed on the display panel 130 of the HMD 100.
The user can display a menu item on the display panel 130 by operating a predetermined input unit of the input device 6. For example, the △ button 75 (see FIG. 10A) may be assigned to the menu display operation. When the user presses the △ button 75, the input device 6 transmits to the information processing device 10 operation information indicating that the △ button 75 has been pressed. In the information processing apparatus 10, the instruction obtaining unit 24 obtains press information of the △ button 75 as a menu item display instruction.

  When the instruction obtaining unit 24 obtains a menu item display instruction, the image generating unit 34 performs a process of including the menu item in the generated panoramic image. Note that the menu item is an example of an information element to be presented to the user, and the information element may be another item or notification information. By including information elements such as menus in the panoramic image by the image generating unit 34, the user can see the presented information element on the display panel 130 of the HMD 100 while viewing the panoramic image based on the viewing direction.

  The information element storage unit 42 stores information elements to be included in the panoramic image. The image generation unit 34 reads information elements according to the display instruction from the information element storage unit 42, and superimposes a window in which the information elements are arranged on a panoramic image. The image providing unit 36 provides the HMD 100 with a panoramic image including menu items.

  FIG. 13A shows an example in which the menu window 204 is superimposed on the display image 200a. In FIG. 13A, the panorama image behind the menu window 204 is hidden by the menu window 204. However, the menu window 204 is actually displayed transparently, and the user can also see the panorama image behind the menu window 204. It is preferable to do so.

  In the menu window 204, a selection frame surrounding one item is displayed, and the user can move the selection frame by pressing the upper button 71a or the lower button 71c of the input device 6. By arranging a selection frame on a desired item and pressing an enter button (for example, a button 72), an information element of a lower layer corresponding to the selected item is displayed on the display panel 130.

  In the information processing apparatus 10, the instruction acquisition unit 24 acquires operation information of the direction button 71 as an item selection instruction, and acquires operation information of the O button 72 as an item determination instruction. The image generation unit 34 moves the selection frame according to the item selection instruction, and reads the corresponding information element from the information element storage unit 42 according to the item determination instruction, and includes the information element in the panoramic image.

  FIG. 13B shows an example in which the selection window 206 is superimposed on the display image 200a. When “change the change angle of one step” is selected in the menu window 204 of FIG. 13A, a selection window 206 that presents options of the change angle per step is displayed. Like the menu window 204, it is preferable that the selection window 206 is also transparently displayed so that the user can see the panoramic image behind. When the user selects and determines any angle in the selection window 206, the change angle per step is changed.

  Thus, the information element is superimposed on the panoramic image and displayed on the display panel 130. In the information processing system 1, the panoramic image on the display panel 130 is changed according to a change in the posture of the HMD 100 (that is, a movement in the line of sight) or the operation of the left analog stick 77b. It was examined how to control the display of the information elements to be displayed is suitable for the HMD 100.

  First, the inventor tried a method of always displaying information elements at the center of the screen of the display panel 130. In this case, even if the gaze direction is changed and the panoramic image of the background of the information element changes, the display position of the information element on the screen does not move and is always displayed at a fixed position in the center of the screen. It has been found that this display method gives an impression as if the information element is moving in the panoramic image, and causes an uncomfortable feeling.

  Then, the present inventor tried a method of not changing the arrangement position when the information element is included in the panoramic image. That is, when the arrangement position of the information element in the panoramic image is fixed, and thus the user changes the line of sight, the information element is moved together with the panoramic image. By fixing the relative positional relationship between the information element and the panorama image, it was found that the information element gave an impression that the information element forms a part of the panorama image, and did not cause the user to feel uncomfortable.

  However, in this case, since the information element moves together with the panoramic image, if the movement in the line-of-sight direction is large, the information element comes off the display screen, and the user loses the information element. In particular, since the input device 6 is assigned to the menu operation during the menu display, the user changes the line-of-sight direction even when the menu operation is continued or when the menu operation is ended, You need to find the missing menu window.

  To solve this problem, the inventor fixed the relative positional relationship between the information element and the panoramic image in principle, and moved the information element together with the panoramic image, while the information element was likely to be off the screen. Then, before deviating, a method for changing the relative positional relationship between the information element and the panoramic image was devised.

  In this method, based on the time point when the information element is displayed, when the posture of the HMD 100 changes from that time point, the image generation unit 34 moves together with the image if the amount of change in the posture is smaller than a predetermined first angle. When the information element is displayed as described above and the amount of change in the posture reaches a predetermined first angle, an image is generated such that the information element is displayed at a position shifted by a predetermined second angle in the image. The amount of change in the posture of the HMD 100 corresponds to the amount of change in the movement in the line-of-sight direction.

  Specifically, the gaze direction determination unit 32 monitors a change in the movement of the gaze direction from the time when the information element is displayed, and if the movement in the gaze direction is smaller than a predetermined first angle, the image generation unit 34 When the information element is displayed so as to move with the panoramic image, while the movement in the line-of-sight direction reaches a predetermined first angle, the image generating unit 34 moves the information element by a predetermined second angle in the panoramic image. Display at the position In the following, an example will be described in which the gaze direction determination unit 32 monitors the rotation angle of the gaze in the horizontal direction as a change in the movement of the gaze direction, and the image generation unit 34 controls the display of information elements.

  For example, consider the case where the user turns his / her head counterclockwise in the horizontal direction from the time when the display of the information element is started, and the line of sight changes to the left. At this time, the panoramic image is displayed so as to flow rightward, and the displayed information element also flows rightward in the display panel 130. When the movement angle of the information element, that is, the rotation angle of the line of sight in the horizontal direction reaches a predetermined first angle, the image generating unit 34 rotates the information element to the left by a predetermined second angle in the panoramic image. To display. The first angle is set to an angle at which all of the information elements do not completely deviate from the user's field of view, so that the information elements are maintained at least partially displayed on the display panel 130 at all times. Hereinafter, the first angle may be referred to as a “position change reference angle”, and the second angle may be referred to as a “return angle”.

Hereinafter, FIG. 13A illustrates a state in which the menu window 204 is displayed first, and an example in which the user rotates the line of sight left from the initial display state will be described.
FIG. 14 shows how the menu window 204 as an information element moves together with the panoramic image. When the user turns his / her head counterclockwise in the horizontal direction, the motion detection unit 30 detects the posture of the HMD 100, and the gaze direction determination unit 32 rotates the gaze direction to the left according to the detected posture of the HMD 100. FIG. 14A shows a display image 200d rotated left by 22.5 degrees from the initial display state. As described above, the rotation of 22.5 degrees is also realized by operating the left analog stick 77b, and the display image 200d is displayed on the display panel by tilting the left analog stick 77b to the left without moving the neck. 130 may be displayed.

  Since the relative positional relationship between the menu window 204 and the panoramic image is fixed, the menu window 204 moves together with the panoramic image, and therefore, on the display panel 130, the user sees the menu window 204 moving rightward.

  FIG. 14B shows a display image 200e rotated left by 45 degrees from the initial display state. In order to display the display image 200e, the user may turn his / her head 45 degrees counterclockwise. However, the left rotation of 45 degrees is also realized by operating the left analog stick 77b twice to the left. It is also realized by an operation in which the user turns the neck 22.5 degrees counterclockwise and tilts the left analog stick 77b once to the left. FIG. 14A shows that the menu window 204 is further moved rightward.

  FIG. 15A shows a display image 200f rotated leftward from the initial display state to immediately before the position change reference angle. Here, the position change reference angle is set to an angle at which all of the information elements do not completely deviate from the screen.

  Therefore, the position change reference angle needs to be set to an angle smaller than the viewing angle of the HMD 100. This is because if the position change reference angle is equal to or larger than the viewing angle, a situation occurs in which all of the information elements are completely off the screen. Considering the horizontal position change reference angle, since the horizontal viewing angle of the HMD 100 is 100 degrees, the position change reference angle must be set to an angle smaller than 100 degrees.

  In addition, the inventor of the present invention tested the appearance by variously setting the position change reference angle. When the position change reference angle was set very small with respect to the viewing angle of the HMD 100, the display element eventually followed the line of sight. It turned out to give the impression of doing. As a result of various trials, by setting the position change reference angle to about half the viewing angle of the HMD 100, it is possible to recognize the integration between the display element and the panoramic image of the background, and the display element can be viewed. It was obtained from the knowledge that the user interface can be realized. In the embodiment, the position change reference angle (first angle) is set to 60 degrees, which is about half (50 degrees) the viewing angle of the HMD 100.

  FIG. 15B shows a display image 200g in which the information element is rotated leftward by the second angle (60 degrees), which is the return angle, in the panoramic image from the initial display state. As described above, when the change of the line of sight in the predetermined direction reaches the first angle (60 degrees) from the initial display state, the image generating unit 34 sets the relative position between the information element and the panoramic image in the predetermined direction by the second angle. Relocate to the position changed by minutes. Here, the second angle is set to the same angle as the first angle. After the rearrangement, the gaze direction determination unit 32 monitors a change in the movement of the gaze direction based on the gaze direction at the time of the rearrangement.

  In FIG. 15A, only a part of the menu window 204 is displayed. However, in FIG. 15B, the entire menu window is displayed again. Easier to do. Also, as shown in FIG. 15A, by performing display control so that the entire menu window 204 does not go out of display, the user can enjoy the panoramic image without losing sight of the menu window 204 and at any time. The menu window 204 can be operated.

  The example in which the first angle that is the position change reference angle is equal to the second angle that is the return angle has been described above. When the first angle is equal to the second angle, when the line-of-sight direction changes by the first angle from the initial display state, as shown in FIG. 15B, the information element is displayed at the center displayed in the initial display state. Relocated to a location. Therefore, when the user turns his / her head further counterclockwise in the horizontal direction, the menu window 204 moves rightward from the center. Therefore, the menu window 204 is displayed on the right side with respect to the center, making it slightly difficult to see.

  Therefore, an example in which the second angle as the return angle is set to be larger than the first angle as the position change reference angle will be described below. The second angle is 60 degrees, and the first angle is 45 degrees.

  Here, it is assumed that FIG. 14B shows the display image 200e rotated leftward from the initial display state to just before the first angle (45 degrees). When the line-of-sight direction further moves to the left and the change in the line-of-sight direction in the predetermined direction reaches the first angle (45 degrees), the image generation unit 34 sets the relative position between the information element and the panorama image in the predetermined direction to the second angle ( (60 degrees) and relocated to the position changed by the distance. Here, since the second angle is set to be larger than the first angle, the information element to be rearranged is returned beyond the center position in the initial display state, and is thus displayed to the left of the center position. After the rearrangement, the gaze direction determination unit 32 monitors a change in the movement of the gaze direction based on the gaze direction at the time of the rearrangement.

  FIG. 16 shows a display image 200h in which the information element is rotated left by a second angle (60 degrees) in the panoramic image from the initial display state. As described above, when the change of the line of sight in the predetermined direction reaches the first angle (45 degrees) from the initial display state, the image generating unit 34 converts the information element into the second direction (60 degrees) in the predetermined direction in the panoramic image. Relocate to the position changed in the direction to return by minutes.

  By setting the second angle larger than the first angle, when the user continuously moves the line of sight left, the menu window 204 gradually moves rightward toward the center. Therefore, it is possible to enhance the visibility of the information element while allowing the user to recognize the integration between the display element and the panoramic image of the background. By setting the first angle to be larger than half of the second angle, it is possible to prevent the movement amount in the panoramic image from increasing when rearranging the display elements.

  As described above, when the horizontal line-of-sight change reaches the position change reference angle (first angle), the horizontal arrangement of the information elements is changed by the return angle (second angle). The same applies to the vertical direction.When the visual line change in the vertical direction reaches the vertical position change reference angle, display control is performed to change the vertical arrangement of the information elements by the return angle in the vertical direction. You may. Note that the position change reference angle in the vertical direction may be set smaller than the position change reference angle in the horizontal direction, and may be set to, for example, 30 degrees.

  Next, the position at which the information element is initially displayed is examined. One method is to display the information element near the center of the displayed image at the timing of displaying the information element, as shown in FIG. This method is effective and easy for the user to understand.

  As another method, in the virtual sphere, candidates for the display position of the information element may be set in advance. In the coordinate system of the virtual sphere, when the horizontal reference direction in the horizontal plane is 0 degree, 60 degrees, 120 degrees, 180 degrees, 240 degrees, and 300 degrees in the horizontal direction are candidate rotation angles, and 0 degrees in the vertical direction, ± 30 degrees and ± 60 degrees are candidate inclination angles. Here, when displaying the information element, a candidate rotation angle and a candidate inclination angle with the closest gaze direction may be extracted, and the information element may be displayed centering on the extracted rotation angle and inclination angle. By determining the candidates for the initial position in advance in this way, the image generation unit 34 only has to select the initial position from the candidate positions, and the display control can be simplified.

  The present invention has been described based on the embodiments. It should be understood by those skilled in the art that the embodiments are exemplifications, and that various modifications can be made to the combination of each component and each processing process, and that such modifications are also within the scope of the present invention.

  FIG. 14A shows the display image 200d and the menu window 204 rotated left by 22.5 degrees from the initial display state, and the left rotation of 22.5 degrees may be realized by turning the user's neck. Also, it has been described that the operation may be realized by operating the left analog stick 77b. In the modification, when the left analog stick 77b is operated, the relative positional relationship between the information element and the panoramic image is released, and the information element in the image after switching by the operation of the left analog stick 77b is: It may be displayed at the center as shown in FIG. This is because when the image is switched by operating the left analog stick 77b, the displayed image jumps and the continuity of the image is interrupted. In such a case, the relative relationship between the information element and the panoramic image is reduced. It is also possible to release the fixing of the positional relationship and control the display of the information elements.

  In the embodiment, the display control of the information element is performed using the position change reference angle and the return angle. In a modification, the display control of the information element may be performed according to the type of the information element. For example, when an important warning is issued from the system to the user, the image generation unit 34 may always place a warning as an information element in the center of the display image.

  As one function of the VR application, the image generation unit 34 may realize a mirror function. Here, the mirror function is a function of including an image in the opposite direction of the line of sight in the display image. Here, instead of including the image of the opposite direction of the line of sight, only the rotation angle of the line of sight is inverted, that is, the rotation angle of the line of sight is rotated 180 degrees, and the image of the line of sight whose inclination is not changed is displayed as the display image. Include it. This mirror function allows the user to see an image of the back side at the same height.

DESCRIPTION OF SYMBOLS 1 ... Information processing system, 6 ... Input device, 8 ... Virtual camera, 10 ... Information processing device, 12 ... Processing device, 14 ... Output control device, 20 ... Sensor Information acquisition unit, 22: photographed image acquisition unit, 24: instruction acquisition unit, 30: motion detection unit, 32: gaze direction determination unit, 34: image generation unit, 36 ... Image providing unit, 40 image storage unit, 42 information element storage unit, 100 HMD, 102 output mechanism unit, 104 mounting mechanism unit, 106 mounting band, 108 housing, 110 light emitting marker, 120 control unit, 122 storage unit, 124 attitude sensor, 126 microphone, 128 communication control unit, 130 ... display panel, 132 ... audio output unit.

Claims (5)

A detection unit that detects the posture of the head mounted display mounted on the user's head,
A gaze direction determining unit that determines a gaze direction according to the attitude of the head mounted display detected by the detection unit,
An image generation unit that generates an image based on the determined line-of-sight direction,
An image providing unit that provides the generated image to the head mounted display,
The line-of-sight direction determining unit determines a line-of-sight direction obtained by inverting the rotation angle of the detected head mounted display with respect to the horizontal reference direction,
An information processing apparatus characterized by the above-mentioned. The line-of-sight direction determination unit determines a line-of-sight direction in which the detected rotation angle with respect to the horizontal reference direction of the head mounted display is rotated by 180 degrees.
The information processing apparatus according to claim 1, wherein: The detection unit detects a rotation angle of the head mounted display with respect to a horizontal reference direction and an inclination angle with respect to a horizontal plane,
The gaze direction determination unit adjusts the inclination angle of the gaze direction to the detected inclination angle.
The information processing apparatus according to claim 1, wherein: Detecting the attitude of the head mounted display mounted on the user's head;
Determining a line-of-sight direction according to the detected orientation of the head-mounted display;
Generating an image based on the determined line-of-sight direction, comprising:
The step of determining the line-of-sight direction determines a line-of-sight direction obtained by inverting the rotation angle of the detected head mounted display with respect to the horizontal reference direction,
An image generation method characterized by the above-mentioned. On the computer,
A function of detecting a posture of a head mounted display mounted on a user's head,
A function to determine the line of sight according to the detected orientation of the head mounted display,
A function of generating an image based on the determined line-of-sight direction, and a program for realizing the function,
The program, wherein the function of determining the line-of-sight direction includes a function of determining the line-of-sight direction in which the detected rotation angle of the head-mounted display with respect to the horizontal reference direction is inverted.

Images