JP2022164728A - Information processing apparatus and image generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technoligy of implementing display control suitable for a head-mounted display.

SOLUTION: A displacement detection unit 30 detects an attitude of a head-mounted display mounted on the head of a user. A line-of-sight direction determination unit 32 determines a line-of-sight direction in accordance with the detected attitude of the head-mounted display. An image generation unit 34 generates an image based on the determined line-of-sight direction. An image providing unit 36 provides the head-mounted display with the generated image. An instruction acquisition unit 24 acquires an instruction to change the line-of-sight direction from an input device 6. When the instruction acquisition unit 24 acquires the change instruction, the line-of-sight direction determination unit 32 changes the line-of-sight direction at a predetermined angle.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to technology for generating an image to be displayed on a head mounted display.

A head-mounted display (HMD) is worn on the user's head to provide the user with a virtual reality (VR) world. Recently, an application has appeared that allows a user to play a game while viewing a screen displayed on an HMD. With conventional stationary displays such as televisions, the user's field of vision extends to the outside of the screen. On the other hand, when the user wears the HMD, the user only sees the image displayed on the HMD, which increases the sense of immersion in the image world and further enhances the entertainment value of the game. If the HMD is provided with a head tracking function and the display screen is updated in conjunction with the posture of the user's head, the sense of immersion in the image world will be further improved.

In recent years, an omnidirectional camera (omnidirectional camera) that captures a 360-degree panorama photograph in all directions up, down, left, and right has become widespread. In addition, the development of unmanned flying objects that can be controlled remotely is underway, and by mounting multiple cameras on such flying objects, it is possible to take panoramic photographs in all directions from the air. The omnidirectional panorama image captured in this way is displayed on the HMD, and by updating the display screen in conjunction with the user's head posture using the head tracking function, the user can feel as if they are actually there. expected to be given.

JP 2015-95045 A

As a recent trend for enhancing the presence of images, the development of widening the viewing angle of HMDs is becoming active. HMDs with a viewing angle of more than 80 degrees are already on the market, and HMDs with a wide viewing angle provide users with an image world comparable to the real world.

As a result of various trials of image display on HMDs with wide viewing angles, the inventors of the present invention have recognized that depending on how the image is displayed, the user may feel intoxicated. In this specification, such a sensation is referred to as "motion sickness (simulator sickness)". Even if the same image does not occur when viewing the same image on a television screen, image sickness may occur on the screen of the HMD. In addition, the inventors have also recognized that when information elements such as a menu are included in a display image, the user may feel uncomfortable depending on how the information elements are presented. The inventor of the present invention has arrived at the idea of display control suitable for HMDs based on knowledge obtained through various trials.

The present invention has been made in view of these problems, and an object thereof is to provide a technique for realizing display control suitable for HMDs.

In order to solve the above problems, an information processing apparatus according to one aspect of the present invention includes a detection unit that detects an orientation of the apparatus from a photographed image of the apparatus and/or sensor information detected by an orientation sensor of the apparatus; A line-of-sight direction determination unit that determines a line-of-sight direction according to the detected orientation of the device, an image generation unit that generates an image based on the determined line-of-sight direction, and an instruction acquisition unit that acquires an instruction to switch the line-of-sight direction. When the instruction acquisition unit acquires the switching instruction, the line-of-sight direction determination unit changes the line-of-sight direction discontinuously or changes the line-of-sight direction by a predetermined angle.

According to another aspect of the present invention, there is provided an image generation method comprising the steps of: detecting an orientation of a device from a photographed image of the device and/or sensor information detected by an orientation sensor of the device; , a step of determining a viewing direction, a step of generating an image based on the determined viewing direction, and a step of obtaining a switching instruction of the viewing direction, wherein the step of determining the viewing direction includes the switching instruction. is obtained, the line-of-sight direction is discontinuously changed or the line-of-sight direction is changed by a predetermined angle.

Any combination of the above constituent elements, and any conversion of the expression of the present invention between a method, an apparatus, a system, a computer program, a recording medium on which the computer program is readable, a data structure, etc., are also included in the present invention. It is effective as an aspect of

ADVANTAGE OF THE INVENTION According to this invention, the technique which implement|achieves the display control suitable for HMD can be provided.

It is a figure which shows the structural example of the information processing system in an Example. FIG. 4 is an explanatory diagram of rendering processing in an information processing apparatus; It is a figure which shows the example of the external appearance shape of HMD. It is a figure which shows the functional block of HMD. It is a figure which shows the functional block of an information processing apparatus. FIG. 3 is a diagram for explaining panorama image data stored in an image storage unit; FIG. FIG. 10 is a diagram showing a display image; FIG. 10 is a diagram showing a display image; FIG. 10 is a diagram showing a display image; It is a figure which shows the upper surface of an input device, and a back|inner side surface. FIG. 10 is a diagram showing a display image; It is a figure which shows the display image before and after switching. FIG. 10 is a diagram showing an example in which information elements are superimposed on a display image; FIG. 10 is a diagram showing how an information element moves with a panoramic image; FIG. 10 is a diagram showing information elements superimposed on a panorama image; FIG. 10 is a diagram showing information elements superimposed on a panorama image;

FIG. 1 shows a configuration example of an information processing system 1 in an embodiment. An information processing system 1 includes an information processing device 10, a head mounted display device (HMD) 100 worn by a user on the head, an input device 6 operated by the user with fingers, and an imaging device for capturing an image of the user wearing the HMD 100. 7 and an output device 4 for displaying images.

In the embodiment, the information processing device 10 includes a processing device 12 and an output control device 14 . Here, the processing device 12 is a terminal device that receives operation information of the input device 6 operated by the user and executes various applications such as games. The processing device 12 and the input device 6 may be connected by a cable or by known wireless communication technology. The output control device 14 is a processing unit that outputs image data to the HMD 100 . The output control device 14 and the HMD 100 may be connected by a cable or by known wireless communication technology.

The imaging device 7 takes an image of the user wearing the HMD 100 and provides the image to the processing device 12 . The imaging device 7 may be a stereo camera. As will be described later, the HMD 100 is equipped with a marker (tracking LED) for realizing tracking of the user's head, and the processing device 12 detects the movement of the HMD 100 based on the position of the photographed marker. The HMD 100 is also equipped with an orientation sensor (acceleration sensor, gyro sensor). tracking process.

In the information processing system 1, since the user sees the image displayed on the HMD 100, the output device 4 is not necessarily required. 4 may be output. This allows another user to view the image viewed by the user on the HMD 100 on the output device 4 . As will be described later, the image to be displayed on the HMD 100 is corrected for distortion of the optical lens.

In the information processing system 1, the processing device 12, the output device 4, the input device 6, and the imaging device 7 may construct a conventional game system. In this case, the processing device 12 may be a game device that executes an application such as a game, and the input device 6 may be a device such as a game controller, keyboard, mouse, or joystick that supplies user operation information to the processing device 12 . By adding the output control device 14 and the HMD 100 to the components of this game system, an information processing system 1 that executes a virtual reality (VR) application is configured.

Note that the functions of the output control device 14 may be incorporated into the processing device 12 as part of the functions of the VR application. In other words, the information processing device 10 may be composed of one processing device 12 or may be composed of the processing device 12 and the output control device 14 . Hereinafter, the functions of the processing device 12 and the output control device 14 necessary for realizing the VR application will be collectively described as the functions of the information processing device 10 .

The information processing device 10 generates image data to be displayed on the HMD 100 . In the embodiment, the information processing apparatus 10 prepares a 360-degree panorama image in all directions up, down, left, and right captured by an omnidirectional camera, and generates it based on the line-of-sight direction determined from the posture of the HMD 100 worn on the user's head. An image is displayed on the HMD 100 . Note that the display content may be either a still image or a moving image. Also, the image is not limited to an image actually shot, and may be drawn in real time by a game application.

The HMD 100 is a display device that displays an image through an optical lens on a display panel positioned in front of the user's head when worn by the user. The HMD 100 independently displays an image for the left eye on the left half of the display panel and an image for the right eye on the right half. These images form parallax images viewed from left and right viewpoints, and are displayed in left and right regions obtained by dividing the display panel into two, so that the images can be stereoscopically viewed. Since the user views the display panel through an optical lens, the information processing apparatus 10 supplies the HMD 100 with image data in which optical distortion caused by the lens has been corrected in advance. In the information processing device 10 , this optical distortion correction processing may be performed by either the processing device 12 or the output control device 14 .

FIG. 2 is an explanatory diagram of rendering processing in the information processing apparatus 10. As shown in FIG. The VR application of the embodiment implements a virtual environment in which the user is positioned at the center of the sphere and changes the direction of the line of sight to change the visible image. A content image, which is an image material, is pasted on the inner peripheral surface of a virtual sphere centered on the center point 9 where the user is positioned. Here, the content image is a 360-degree panorama image taken by an omnidirectional camera, and pasted on the inner peripheral surface of the virtual sphere so that the top and bottom of the content image and the top and bottom of the virtual sphere match. be done. As a result, the top and bottom of the user's real world and the top and bottom of the image world provided to the HMD 100 are aligned, and a VR application that reproduces a realistic image world is realized.

The information processing apparatus 10 detects the rotation angle and inclination of the user's head (actually, the HMD 100) by performing the user's head tracking process. Here, the rotation angle of the HMD 100 is the rotation angle of the horizontal plane with respect to the reference direction, and the reference direction may be set as the direction in which the HMD 100 is turned on, for example. The tilt of the HMD 100 is the tilt angle with respect to the horizontal plane. A known technique may be used as the head tracking process, and the information processing apparatus 10 can detect the rotation angle and tilt of the HMD 100 only from the sensor information detected by the orientation sensor of the HMD 100, and furthermore, the image of the HMD 100 photographed by the imaging device 7 can be detected. By image-analyzing the marker (tracking LED), the rotation angle and tilt of the HMD 100 can be detected with high accuracy.

The information processing apparatus 10 determines the posture of the virtual camera 8 on the virtual sphere according to the detected rotation angle and tilt of the HMD 100 . The virtual camera 8 is arranged so as to photograph the inner peripheral surface of the virtual sphere from the center point 9 of the virtual sphere. Match the rotation angle and tilt of the axis. The information processing apparatus 10 acquires the captured image 5 of the virtual camera 8 , that is, performs rendering processing, performs optical distortion correction for the optical lens, and supplies image data to the HMD 100 . Although one virtual camera 8 is shown in FIG. 2, two virtual cameras 8 for the left eye and the right eye are actually arranged to generate respective image data.

FIG. 3 shows an example of the external shape of the HMD 100. As shown in FIG. In this example, the HMD 100 is composed of an output mechanism section 102 and a mounting mechanism section 104 . The mounting mechanism unit 104 includes a mounting band 106 that is worn by the user and wraps around the head to fix the HMD 100 to the head. The mounting band 106 is made of a material or structure whose length can be adjusted according to the head circumference of the user.

The output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the HMD 100 is worn by the user, and has a display panel inside that faces the eyes when the HMD 100 is worn. The display panel is realized by a liquid crystal panel, an organic EL panel, or the like. The housing 108 further includes a pair of left and right optical lenses positioned between the display panel and the user's eyes when the HMD 100 is worn, to expand the viewing angle of the user. The HMD 100 may further include speakers and earphones at positions corresponding to the user's ears when worn.

The outer surface of housing 108 is provided with luminous markers 110a, 110b, 110c, and 110d. In this example, the tracking LED constitutes the light-emitting marker 110, but other types of markers may be used as long as they are photographed by the imaging device 7 and can be image-analyzed by the information processing device 10. good. The number and arrangement of the luminous markers 110 are not particularly limited, but the number and arrangement should be such that the posture (rotational angle and tilt) of the HMD 100 can be detected by photographing and analyzing the image. , they are provided at the four corners of the front surface of the housing 108 . Furthermore, the light-emitting marker 110 may be provided on the side or rear portion of the wearing band 106 so that the image can be captured even when the user turns his/her back to the imaging device 7 .

The HMD 100 may be connected to the information processing apparatus 10 by a cable or by a known wireless communication technique. The HMD 100 transmits sensor information detected by the orientation sensor to the information processing apparatus 10, receives image data generated by the information processing apparatus 10, and displays the data on the display panel.

Although the HMD 100 shown in FIG. 3 is an immersive (non-transmissive) display device that completely covers both eyes, it may be a transmissive display device. As for the shape, it may be a hat type as shown in the figure, or may be a spectacle type.

FIG. 4 shows functional blocks of the HMD 100. As shown in FIG. The control unit 120 is a main processor that processes and outputs various data such as image data, audio data, sensor information, and commands. Storage unit 122 temporarily stores data, instructions, and the like processed by control unit 120 . The orientation sensor 124 detects orientation information such as the rotation angle and tilt of the HMD 100 . The attitude sensor 124 includes at least a triaxial acceleration sensor and a triaxial gyro sensor. A microphone 126 converts the user's voice into an electrical signal. The light-emitting markers 110 are LEDs, and a plurality of the light-emitting markers 110 are attached to the mounting band 106 and the housing 108 of the HMD 100 .

The communication control unit 128 transmits data input from the control unit 120 to the external information processing device 10 by wired or wireless communication via a network adapter or an antenna. Further, the communication control unit 128 receives data from the information processing device 10 by wired or wireless communication via a network adapter or an antenna, and outputs the data to the control unit 120 .

When the control unit 120 receives image data and audio data from the information processing apparatus 10, the control unit 120 supplies the data to the display panel 130 for display, and supplies the data to the audio output unit 132 for audio output. The control unit 120 also causes the communication control unit 128 to transmit sensor information from the posture sensor 124 and audio data from the microphone 126 to the information processing apparatus 10 .

FIG. 5 shows functional blocks of the information processing apparatus 10. As shown in FIG. The information processing apparatus 10 includes a sensor information acquisition unit 20, a captured image acquisition unit 22, and an instruction acquisition unit 24 as an input interface with the outside. The sensor information acquisition unit 20 acquires sensor information from the orientation sensor 124 of the HMD 100 at predetermined intervals. The captured image acquisition unit 22 acquires captured images of the HMD 100 from the imaging device 7 at predetermined intervals. For example, the imaging device 7 captures an image every (1/60) second, and the captured image acquisition unit 22 acquires a captured image every (1/60) second. The instruction acquisition unit 24 acquires an instruction input by the user from the input device 6 .

The information processing apparatus 10 further includes a motion detection section 30 , a line-of-sight direction determination section 32 , an image generation section 34 and an image provision section 36 . The motion detection unit 30 detects the posture of the HMD 100 worn on the user's head. The line-of-sight direction determination unit 32 determines the line-of-sight direction according to the posture of the HMD 100 detected by the motion detection unit 30 . The image generation unit 34 generates an image according to the detected posture of the HMD 100 , specifically, based on the line-of-sight direction determined by the line-of-sight direction determination unit 32 . The image providing unit 36 provides the HMD 100 with the generated image.

In FIG. 5, each element described as a functional block that performs various processes can be configured by a circuit block, memory, and other LSI in terms of hardware, and is loaded in memory in terms of software. It is realized by a program or the like. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and are not limited to either one.

The image storage unit 40 stores 360-degree panoramic image data captured in advance. The image storage unit 40 may store a plurality of content images, and the content images may be still images or moving images. The image storage unit 40 of the embodiment stores omnidirectional panoramic image data, and the information processing apparatus 10 provides the omnidirectional panoramic image to the user wearing the HMD 100 . Therefore, by turning the user's neck to the left or right (the user may rotate the whole body to the left or right) to rotate the horizontal line of sight to the left or right, a panoramic image in the left or right direction can be obtained. Displayed on the display panel 130 of the HMD 100, and by tilting the user's head up or down and tilting the line of sight in the vertical direction, an upward or downward panoramic image is displayed on the display panel 130 of the HMD 100. Become.

The information element storage unit 42 stores information elements to be included in the panorama image. This information element may be, for example, a menu item for changing the brightness of a content image, or a menu item for selecting a content image that is an image material. When the user inputs an information element display instruction to the input device 6 , the image generator 34 includes the information element in the generated image, and the information element superimposed on the panoramic image is displayed on the display panel 130 of the HMD 100 . Note that the information element may be notification information for the user, and in this case, the information element may be superimposed on the panorama image without the operation of the input device 6 by the user.

FIG. 6 is a diagram for explaining panoramic image data stored in the image storage unit 40. As shown in FIG. For convenience of explanation, FIG. 6 shows part of the panorama image data in all directions, omitting part of the image downward from the horizontal plane and part of the image in the horizontal direction. As described with reference to FIG. 2, the display panel 130 of the HMD 100 displays an image obtained by rendering a panoramic image pasted on the inner peripheral surface of the virtual sphere, and the user moves the rotation angle and tilt of the HMD 100 to change the line-of-sight direction. By changing , the displayed panorama image is moved according to the line-of-sight direction.

In the HMD 100, the communication control unit 128 transmits sensor information acquired by the orientation sensor 124 to the information processing apparatus 10 at predetermined intervals. The imaging device 7 also images the HMD 100 at a predetermined cycle and transmits the captured image to the information processing device 10 . Referring to FIG. 5 , sensor information acquisition unit 20 acquires sensor information from orientation sensor 124 and supplies it to motion detection unit 30 . Also, the captured image acquisition unit 22 acquires a captured image and supplies it to the motion detection unit 30 .

The motion detection unit 30 detects the posture of the HMD 100 to perform head tracking processing for detecting the posture of the head of the user wearing the HMD 100 . This head tracking processing is performed to link the field of view displayed on the display panel 130 of the HMD 100 with the posture of the user's head. In the head tracking process of the embodiment, the rotation angle of the HMD 100 with respect to the horizontal reference direction and the tilt angle with respect to the horizontal plane are detected. The horizontal reference direction may be set, for example, as the direction in which the HMD 100 is turned on.

This head tracking processing uses a known method, and the motion detection unit 30 may detect the rotation angle of the HMD 100 with respect to the horizontal reference direction and the inclination angle with respect to the horizontal plane from only the sensor information of the orientation sensor 124. It is preferable to improve the detection accuracy by further using the imaging result of the luminous marker 110 of . The motion detector 30 detects the rotation angle and the tilt angle at a predetermined cycle. For example, if the image to be supplied to the HMD 100 is 60 fps, it is preferable that the detection processing of the motion detection section 30 is also executed at a cycle of (1/60) seconds.

The line-of-sight direction determination unit 32 determines the line-of-sight direction according to the posture of the HMD 100 detected by the motion detection unit 30 . This line-of-sight direction is the line-of-sight direction of the user, and is also the line-of-sight direction (optical axis direction) of the virtual camera 8 arranged at the center point 9 of the virtual sphere (see FIG. 2). Here, the line-of-sight direction determination unit 32 may determine the rotation angle and the tilt angle detected by the motion detection unit 30 as they are as the line-of-sight direction (optical axis direction) of the virtual camera 8, or may perform some correction processing. A line-of-sight direction of the virtual camera 8 may be determined. For example, when stable sensor information is not provided to the motion detection unit 30 due to noise in the sensor information, the motion detection unit 30 detects a vibrating motion even though the user's head is not moving. may detect. In such a case, the line-of-sight direction determination unit 32 may determine the line-of-sight direction by smoothing and correcting the motion detected by the motion detection unit 30 .

In addition, the human visual field is vertically asymmetrical, and the lower part of the line of sight is slightly wider than the upper part. Therefore, the line-of-sight direction determination unit 32 may determine the line-of-sight direction of the virtual camera 8 by tilting the tilt angle detected by the motion detection unit 30 slightly downward.

The image generation unit 34 generates an image according to the orientation of the HMD 100 detected by the motion detection unit 30 , specifically based on the line-of-sight direction of the virtual camera 8 determined by the line-of-sight direction determination unit 32 . The image generator 34 defines a left-eye field of view and a right-eye field of view specified by the line-of-sight direction, and renders and generates left-eye and right-eye images, respectively. At this time, the image generator 34 generates a panoramic image in which distortion due to the image light from the display panel passing through the optical lens is corrected.

The example HMD 100 provides the user with a field of view of approximately 100 degrees horizontally and approximately 100 degrees vertically. Referring to FIG. 2 , captured image 5 on the virtual sphere is captured at an angle of view of approximately 100 degrees in the horizontal direction and approximately 100 degrees in the vertical direction, and displayed on display panel 130 of HMD 100 . As described above, the human visual field is slightly wider at the lower side than the upper side of the line of sight. A vertical field of view of 55 degrees down may be achieved by the arrangement of the optical lens and display panel 130 .

FIG. 7 shows a display image 200a generated by the image generator 34. As shown in FIG. Note that in the following drawings, the display image is represented as an image cut out from the panorama image in order to facilitate understanding of the positional relationship of the display image in the entire panorama image.

The image generation unit 34 generates an image based on the line-of-sight direction 202a determined by the line-of-sight direction determination unit 32 . In practice, the image generation unit 34 renders and generates a display image for the left eye and a display image for the right eye, and these display images are different images including parallax. , respectively, is not described. The image providing unit 36 provides the HMD 100 with the display image 200a generated by the image generating unit 34 . In the HMD 100, the control unit 120 causes the display panel 130 to display the display image 200a, so that the user can see the display image 200a displayed on the display panel 130. FIG.

FIG. 8 shows a display image 200b generated by the image generator 34. As shown in FIG. The line-of-sight direction determination unit 32 determines the line-of-sight direction according to the posture of the HMD 100, and the image generation unit 34 generates an image based on the determined line-of-sight direction. In this example, the user horizontally rotates his or her neck counterclockwise to show how the line of sight continuously changes from line-of-sight direction 202a to line-of-sight direction 202b. Here, the user rotates the head counterclockwise by about 60 degrees, and by this rotating operation, the image generating unit 34 generates an image in which the panorama image is continuously moved in the counterclockwise direction from the display image 200a (1/60) seconds. is generated with a period of The image providing unit 36 provides the generated image to the HMD 100 at a cycle of (1/60) seconds.

FIG. 9 shows a display image 200c generated by the image generator 34. As shown in FIG. The line-of-sight direction determination unit 32 determines the line-of-sight direction according to the posture of the HMD 100, and the image generation unit 34 generates an image based on the determined line-of-sight direction. In this example, the user tilts his/her head upward from the state where the display image 200a is displayed on the display panel 130, and the line of sight continuously changes from the line-of-sight direction 202a to the line-of-sight direction 202c. Here, the user tilts his/her head upward by about 30 degrees, and by this tilting motion, the image generation unit 34 continuously moves the panorama image upward from the displayed image 200a at a cycle of (1/60) second. Generate. The image providing unit 36 provides the generated image to the HMD 100 at a cycle of (1/60) second.

In this way, the user changes the line-of-sight direction by moving the head, and the information processing apparatus 10 provides the HMD 100 with a panoramic image in the desired viewing direction and causes the display panel 130 to display the panorama image. Changing the line-of-sight direction by moving the head is the same as the action in the real world, and matches the sense of the user. At this time, the HMD 100 provides the user with a wide viewing angle, thereby further enhancing the sense of immersion in the panoramic image.

On the other hand, for example, in order to see a panorama image directly behind, the user naturally has to face directly behind. In order to face directly behind, it is not enough to turn the neck, so it is necessary to change the direction of the body. Therefore, if the user is sitting on a non-rotating chair such as a sofa, the user must stand up and turn. If the HMD 100 is a non-transmissive display device, the user's eyes are covered by the housing 108 and the user cannot see the surrounding environment. You may feel resistance.

Therefore, in the embodiment, the user can input an instruction to switch the line-of-sight direction from the input device 6 so that the line-of-sight direction can be changed by operating the input device 6 without moving the user's head. In the embodiment, the input device 6 is shown as a device different from the HMD 100, but the input device 6 may be provided in the HMD 100, that is, formed as an operation input unit such as a button on the mounting band 106 or the housing 108 of the HMD 100. may be

FIG. 10(a) shows the top surface of the input device 6. FIG. The user operates the input device 6 by gripping the left grip portion 78b with the left hand and gripping the right grip portion 78a with the right hand. Direction buttons 71, analog sticks 77a and 77b, and operation buttons 76, which are input units, are provided on the upper surface of the housing of the input device 6. As shown in FIG. The directional buttons 71 include an up button 71a, a left button 71b, a down button 71c and a right button 71d. The right analog stick 77a and the left analog stick 77b are tilted and used to input the direction and tilt amount. The right analog stick 77a and the left analog stick 77b also function as push buttons that sink downward when pressed by the user and return to their original positions when released by the user. A touch pad 79 is provided in a flat area between the direction buttons 71 and the operation buttons 76 on the upper surface of the housing. The touch pad 79 also functions as a push button that sinks downward when pressed by the user and returns to its original position when released by the user.

A home button 80 is provided between the right analog stick 77a and the left analog stick 77b. The home button 80 is used to turn on the power of the input device 6 and at the same time activate the communication function for wireless connection with the information processing device 10 . A SHARE button 81 is provided on the left side of the touch pad 79 . The SHARE button 81 is used to input instructions from the user to the OS or system software in the information processing device 10 . An OPTIONS button 82 is provided on the right side of the touchpad 79 . The OPTIONS button 82 is used to input an instruction from the user to the application (game) executed on the information processing device 10 . Both the SHARE button 81 and the OPTIONS button 82 may be formed as push buttons.

FIG. 10(b) shows the rear side of the input device 6. FIG. A touch pad 79 is bent and extended from the top surface of the housing on the upper side of the rear side of the housing of the input device 6, and a horizontally long light emitting section 85 is provided on the lower side of the rear side of the housing. The light emitting unit 85 has LEDs of red (R), green (G), and blue (B), and lights according to the emitted light color information transmitted from the information processing device 10 . The upper R1 button 83a and L1 button 83b may be configured as push buttons, and the lower R2 button 84a and L2 button 84b may be configured as pivotally supported trigger type buttons.

In the embodiment, of the operation members of the input device 6, the left analog stick 77b is used for inputting an instruction to switch the line-of-sight direction. Tilting the left analog stick 77b to the left moves the line-of-sight direction to the left, and tilting it to the right moves the line-of-sight direction to the right. Thus, the left analog stick 77b is suitable for the user to intuitively change the line-of-sight direction. Note that other operation members, such as the right analog stick 77a, the direction button 71, etc., may be used to input the line-of-sight direction switching instruction.

The inventors have tried various ways of changing the line-of-sight direction with respect to the operation of the left analog stick 77b. First, as one change processing method, the line-of-sight direction is continuously moved in accordance with the tilting direction of the left analog stick 77b to generate an image in which the panorama image moves in a flowing manner. This is a common line-of-sight change method for games displayed on a television screen, and is considered to match the user's sense. In addition, various experiments were also conducted on the moving speed of the image at that time.

As a result, it was found that when the panoramic image is continuously moved by operating the left analog stick 77b, the user who sees it feels intoxicated. Regardless of the moving speed of the image, regardless of whether the image was moved slowly or quickly, the result was that the degree of motion sickness was different depending on the individual.

On television screens, it is often the case that a displayed image is continuously moved in one direction. After examining this point, the inventor found that one of the causes is the difference in viewing angle.

As for the television screen, since the user sees it from a certain distance, the horizontal viewing angle of the user with respect to the television screen is several tens of degrees at most. On the other hand, in a wide viewing angle HMD, the viewing angle of the user is close to 90 degrees or more than 90 degrees (hereinafter, the horizontal viewing angle of the HMD 100 is assumed to be about 100 degrees). In other words, with the HMD 100, the continuous movement of the image within a range of approximately 100 degrees (and over the entire field of view) tends to cause motion sickness in the user. Therefore, the inventor tried to move the panorama image discontinuously by operating the left analog stick 77b instead of moving it continuously, and as a result, obtained knowledge that the occurrence of image sickness can be prevented. Therefore, in the embodiment, when the information processing apparatus 10 acquires an operation of the left analog stick 77b as an instruction to switch the line-of-sight direction, the panorama image is displayed discontinuously, that is, moved by a predetermined angle in one switching (one step). Control.

When the user tilts the left analog stick 77 b of the input device 6 , the instruction obtaining unit 24 obtains from the input device 6 an instruction to switch the line-of-sight direction to the tilted direction. For example, when the left analog stick 77b is pushed down once and then immediately returned, the instruction obtaining unit 24 obtains one switching instruction in the pushed direction. Further, when the left analog stick 77b continues to be pushed down, the instruction obtaining unit 24 continuously obtains switching instructions in the pushed direction.

Whether the instruction acquisition unit 24 acquires the tilting operation of the left analog stick 77b as a single switching instruction or as a continuous switching instruction depends on the tilting time (the time from tilting to returning to the original position). time) exceeds a predetermined time. Specifically, when the tilting time is less than the predetermined time, the instruction obtaining unit 24 obtains it as a single switching instruction. get. After that, when the tilting operation continues, the switching instruction may be acquired every time a period of time shorter than the predetermined period of time elapses.

When the instruction acquisition unit 24 acquires the line-of-sight direction switching instruction, the line-of-sight direction determination unit 32 changes the line-of-sight direction by a predetermined angle. For example, when the left analog stick 77b is tilted leftward, the instruction acquisition unit 24 acquires a switching instruction to rotate the line-of-sight direction to the left, and the line-of-sight direction determination unit 32 changes the line-of-sight direction counterclockwise by a predetermined angle. An example will be described below in which the user inputs a switching instruction to the input device 6 to rotate the line-of-sight direction to the left while the display image 200a shown in FIG. 7 is displayed on the display panel 130 of the HMD 100 .

FIG. 11 shows a display image 200d generated by the image generator 34. As shown in FIG. When the instruction acquisition unit 24 acquires a switching instruction to rotate the line-of-sight direction to the left, the line-of-sight direction determination unit 32 changes the line-of-sight direction counterclockwise by a predetermined angle. Here, the predetermined angle is set to an angle larger than 10 degrees and smaller than the viewing angle (100 degrees) of the HMD 100 in the horizontal direction.

If the angle to be changed in one step is set to 10 degrees or less, for example, in order to see the image directly behind (180 degrees rotated), at least 18 steps are required, and the number of steps is large, and the user is a little frustrated. It will make you feel ugly. On the other hand, when the change angle in one step is equal to or greater than the viewing angle of the HMD 100, a completely new image (an image that does not overlap with the image before switching) is suddenly displayed on the display panel 130, and the image before switching is displayed. Continuity cannot be guaranteed. Therefore, the change angle in one step is preferably set to an angle larger than 10 degrees and smaller than the viewing angle of the HMD 100 .

More preferably, the change angle in one step is set to half or less of the viewing angle of HMD 100 . As a result, at least half or more of the image before switching is included in the image after switching, and the user can recognize the continuity of the images before and after switching. In order to make the continuity of images more recognizable, it is preferable that the change angle for one step is small. When the present inventor tried various change angles per step, when the change angle per step was set to 15 degrees or more and 30 degrees or less, the desired angle was reached when continuously switched. It has been recognized that this is the most preferable from the viewpoint of time and image continuity. The example shown in FIG. 11 shows a state where the change angle in one step is 22.5 degrees.

In general, it is said that the human central visual field is 40 to 45 degrees and the peripheral visual field is about 200 degrees. Focusing on this central visual field, the change angle per step may be set smaller than the central visual field so that the same image remains in the central visual field before and after switching. From this point of view as well, it makes sense to set the change angle per step to 30 degrees or less. The change angle per step is set to a value obtained by dividing 360 degrees by an integer N, and the user can return to the original display position by inputting an integer number of switching instructions from the input device 6. is preferred.

FIG. 12 shows display images before and after switching displayed on the display panel 130 . FIG. 12(a) shows a display image 200a, and FIG. 12(b) shows a display image 200d. A display image 200d is obtained by left-rotating the display image 200a with a change angle of 22.5 degrees per step. By setting the change angle to 22.5 degrees out of the viewing angle of 100 degrees, 22.5% of the image is changed before and after switching, but 77.5% of the image is present before and after switching. Therefore, the user can easily recognize the continuity of the images.

When the instruction acquisition unit 24 successively acquires switching instructions, the line-of-sight direction determination unit 32 changes the line-of-sight direction by a predetermined angle. This change period may be a fixed value such as one second, or may be dynamically determined according to the amount of tilting of the left analog stick 77b. That is, the larger the tilting amount, the shorter the change period may be set.

In the embodiment, the switching instruction using the input device 6 is valid only in the horizontal direction and invalid in the vertical direction. This is because when a human being sits on a chair or stands upright, they face horizontally, and they usually do not always face upwards or downwards. . Note that it is also possible to validate the switching instruction in the vertical direction, and it may be possible to set whether to validate or invalidate the switching instruction in the vertical direction according to the content image. For content images in which the top and bottom do not matter, for example, a photographed image of a starry sky, the vertical direction switching instruction may be valid.

FIG. 12 shows the display images 200a and 200d before and after switching when the change angle for one step is 22.5 degrees. may also wish to Therefore, options for selecting the change angle per step may be displayed on the display panel 130, and the user may select the options displayed on the display panel 130 by operating the input device 6. . In one embodiment, the user is provided with options of 15 degrees and 30 degrees, in addition to 22.5 degrees.

In the following, a procedure for displaying options for changing angles for each step in a state where the display image 200a shown in FIG. 12A is displayed on the display panel 130 of the HMD 100 will be described.
The user can display menu items on the display panel 130 by operating a predetermined input unit of the input device 6 . For example, the △ button 75 (see FIG. 10(a)) may be assigned to the menu display operation. When the user presses the Δ button 75 , the input device 6 transmits operation information indicating that the Δ button 75 has been pressed to the information processing device 10 . In the information processing apparatus 10, the instruction acquisition unit 24 acquires the press information of the Δ button 75 as an instruction to display the menu item.

When the instruction acquisition unit 24 acquires the display instruction of the menu item, the image generation unit 34 performs processing for including the menu item in the panorama image to be generated. Note that the menu items are an example of information elements presented to the user, and the information elements may be other items or notification information. The image generator 34 includes information elements such as a menu in the panoramic image, so that the user can see the presented information elements on the display panel 130 of the HMD 100 while viewing the panoramic image based on the line-of-sight direction.

The information element storage unit 42 stores information elements to be included in the panorama image. The image generation unit 34 reads information elements according to the display instruction from the information element storage unit 42, and superimposes a window in which the information elements are arranged on the panoramic image. The image providing unit 36 provides the HMD 100 with a panorama image including menu items.

FIG. 13A shows an example in which a menu window 204 is superimposed on the display image 200a. In FIG. 13A, the menu window 204 hides the panoramic image behind it, but in reality the menu window 204 is transparently displayed so that the user can also see the panoramic image behind the menu window 204. It is preferable to

A selection frame surrounding one item is displayed in the menu window 204 , and the user can move the selection frame by pressing the up button 71 a or the down button 71 c of the input device 6 . By arranging a selection frame on a desired item and pressing a decision button (for example, circle button 72), the information element in the lower layer corresponding to the selected item is displayed on display panel 130. FIG.

In the information processing apparatus 10, the instruction acquisition unit 24 acquires operation information of the direction button 71 as an item selection instruction, and acquires operation information of the circle button 72 as an item determination instruction. The image generation unit 34 moves the selection frame according to the item selection instruction, and according to the item determination instruction, reads the corresponding information element from the information element storage unit 42 and includes it in the panorama image.

FIG. 13B shows an example in which a selection window 206 is superimposed on the display image 200a. In the menu window 204 of FIG. 13(a), when "change the change angle of 1 step" is selected, a selection window 206 presenting options of the change angle per step is displayed. Like the menu window 204, the selection window 206 is preferably transparent so that the user can see the panoramic image behind it. When the user selects and determines one of the angles in the selection window 206, the change angle per step is changed.

The information elements are thus superimposed on the panoramic image and displayed on the display panel 130 . In the information processing system 1, the panorama image on the display panel 130 is changed according to the change in the attitude of the HMD 100 (that is, movement in the line-of-sight direction) and the operation of the left analog stick 77b. The HMD 100 was examined as to how to control the display of information elements to be displayed.

First, the inventor tried a method of always displaying an information element in the center of the screen of the display panel 130 . In this case, even if the viewing direction is changed and the background panorama image of the information element changes, the display position of the information element within the screen does not move, and the information element is always displayed at a fixed position in the center of the screen. It has been found that this display method gives the impression that the information elements are moving in the panorama image, causing a sense of incongruity.

Therefore, the inventor of the present invention has tried a method of not changing the arrangement position of the information element once the information element is included in the panorama image. In other words, the arrangement position of the information element in the panorama image is fixed, so that when the user changes the line-of-sight direction, the information element is moved together with the panorama image. It has been found that fixing the relative positional relationship between the information element and the panoramic image gives the impression that the information element constitutes a part of the panoramic image, and does not make the user feel uncomfortable.

However, in this case, the information element moves together with the panoramic image, and if the movement in the line-of-sight direction is large, the information element is out of the display screen, and the user loses sight of the information element. In particular, since the input device 6 is assigned to the menu operation while the menu is being displayed, the user can change the line-of-sight direction even if the menu operation is continued or terminated. I need to find the lost menu window.

In order to solve this problem, the present inventor fixed the relative positional relationship between the information element and the panoramic image in principle, and moved the information element together with the panoramic image, while Then, I came up with a method to change the relative positional relationship between the information element and the panoramic image before it comes off.

In this method, the point in time when the information element is displayed is used as a reference, and when the attitude of the HMD 100 changes from that point in time, the image generator 34 moves with the image if the amount of change in attitude is smaller than a predetermined first angle. When the amount of change in posture reaches a predetermined first angle, the image is generated so that the information element is displayed at a position moved by a predetermined second angle in the image. The amount of change in posture of the HMD 100 corresponds to the amount of change in movement in the line-of-sight direction.

Specifically, the line-of-sight direction determination unit 32 monitors a change in the line-of-sight direction movement from the time when the information element is displayed, and if the line-of-sight direction movement is smaller than a predetermined first angle, the image generation unit 34 , the information element is displayed so as to move with the panoramic image, while when the movement in the line-of-sight direction reaches a predetermined first angle, the image generator 34 moves the information element by a predetermined second angle in the panoramic image. position. In the following, an example will be described in which the image generation unit 34 controls the display of information elements by the sight line direction determination unit 32 monitoring the rotation angle of the sight line in the horizontal direction as a change in the movement of the sight line direction.

For example, consider a case where the user turns his or her head horizontally to the left after the display of the information element is started, and the line of sight changes to the left. At this time, the panorama image is displayed so as to flow rightward, so that the displayed information elements also flow rightward within the display panel 130 . When the movement angle of the information element, that is, the rotation angle of the line of sight in the horizontal direction reaches a predetermined first angle, the image generation unit 34 rotates the information element to a position rotated counterclockwise by a predetermined second angle in the panorama image. to display. This first angle is set to an angle at which all of the information elements do not completely fall out of the user's field of vision, so that at least a portion of the information elements is always displayed on the display panel 130 . Hereinafter, this first angle may be referred to as the "position change reference angle", and the second angle may be referred to as the "return angle".

FIG. 13A shows the state in which the menu window 204 is first displayed, and an example in which the user rotates the line of sight to the left from this initial display state will be described below.
FIG. 14 shows how the menu window 204, which is an information element, moves with the panoramic image. When the user horizontally rotates the neck counterclockwise, the motion detection unit 30 detects the posture of the HMD 100 , and the line-of-sight direction determination unit 32 rotates the line-of-sight direction to the left according to the detected posture of the HMD 100 . FIG. 14(a) shows a display image 200d rotated counterclockwise by 22.5 degrees from the initial display state. As described above, the rotation of 22.5 degrees can also be realized by operating the left analog stick 77b, and by tilting the left analog stick 77b to the left without moving the user's neck, the display image 200d is displayed on the display panel. 130 may be displayed.

Since the relative positional relationship between the menu window 204 and the panoramic image is fixed, the menu window 204 moves with the panoramic image, so the user on the display panel 130 sees the menu window 204 moving rightward.

FIG. 14(b) shows a display image 200e rotated counterclockwise by 45 degrees from the initial display state. In order to display the display image 200e, the user may rotate his/her neck counterclockwise by 45 degrees. It can also be realized by the user turning his/her neck counterclockwise 22.5 degrees and tilting the left analog stick 77b to the left once. A comparison with FIG. 14A shows that the menu window 204 has moved further to the right.

FIG. 15(a) shows a display image 200f rotated left from the initial display state to just before the position change reference angle. Here, the position change reference angle is set to an angle at which all of the information elements are not completely out of the screen.

Therefore, the position change reference angle needs to be set to an angle smaller than the viewing angle of the HMD 100 . This is because if the position change reference angle is greater than or equal to the viewing angle, a situation will occur in which all of the information elements are completely out of the screen. Considering the horizontal position change reference angle, the horizontal viewing angle of the HMD 100 is 100 degrees, so the position change reference angle must be set to an angle smaller than 100 degrees.

The inventor of the present invention set various reference angles for position change to test the appearance. It turned out to give the impression that it does. As a result of various trials, it was found that by setting the position change reference angle to about half the viewing angle of the HMD 100, it was possible to recognize the unity between the display element and the panoramic image of the background, and also to make the display element visible. We have learned that we can realize a user interface that In the embodiment, the position change reference angle (first angle) is set to 60 degrees, which is about half (50 degrees) of the viewing angle of the HMD 100 .

FIG. 15(b) shows a display image 200g in which the information element is rotated counterclockwise by the second angle (60 degrees), which is the return angle, in the panoramic image from the initial display state. In this way, when the change in the predetermined direction of the line of sight from the initial display state reaches the first angle (60 degrees), the image generator 34 shifts the relative positions of the information element and the panoramic image to the predetermined direction by the second angle. Relocate to the position changed by minutes. Here, the second angle is set to the same angle as the first angle. After the rearrangement, the line-of-sight direction determination unit 32 monitors changes in the movement of the line-of-sight direction based on the line-of-sight direction at the time of rearrangement.

In FIG. 15(a), only a portion of the menu window 204 is displayed, but in FIG. 15(b), the entire menu window 204 is displayed again, allowing the user to operate the menu window 204. easier to do. Further, as shown in FIG. 15A, display control is performed so that the entire menu window 204 is not out of the display, so that the user can enjoy the panorama image without losing sight of the menu window 204. The menu window 204 becomes operable.

An example in which the first angle, which is the position change reference angle, and the second angle, which is the return angle, are equal has been described above. When the first angle is equal to the second angle, when the viewing direction changes from the initial display state by the first angle, as shown in FIG. position. Therefore, when the user rotates the neck further counterclockwise in the horizontal direction, the menu window 204 moves from the center to the right. Therefore, the menu window 204 is displayed to the right of the center, making it somewhat difficult to see.

Therefore, an example in which the second angle, which is the return angle, is set larger than the first angle, which is the position change reference angle, will be described below. Assume that the second angle is 60 degrees and the first angle is 45 degrees.

Here, it is assumed that FIG. 14B shows the display image 200e rotated to the left from the initial display state to just before the first angle (45 degrees). When the line-of-sight direction moves further to the left and the change in the predetermined direction of the line of sight reaches the first angle (45 degrees), the image generator 34 shifts the relative positions of the information elements and the panoramic image to the second angle ( 60 degrees). Here, since the second angle is set larger than the first angle, the rearranged information elements are returned past the center position in the initial display state, and are therefore displayed leftward from the center position. After the rearrangement, the line-of-sight direction determination unit 32 monitors changes in the movement of the line-of-sight direction based on the line-of-sight direction at the time of rearrangement.

FIG. 16 shows a display image 200h in which the information element is rotated counterclockwise by a second angle (60 degrees) in the panorama image from the initial display state. In this way, when the change in the predetermined direction of the line of sight from the initial display state reaches the first angle (45 degrees), the image generator 34 moves the information element to the second angle (60 degrees) in the predetermined direction in the panorama image. Relocate to the changed position in the direction backward by the minute.

By setting the second angle to be larger than the first angle, the menu window 204 gradually moves rightward toward the center as the user continues to move the viewing direction to the left. Therefore, it is possible to improve the visibility of the information elements while allowing the user to recognize the unity between the display elements and the background panoramic image. By setting the first angle larger than half the second angle, it is possible to prevent the amount of movement in the panorama image from becoming large when rearranging the display elements.

As described above, when the change in the line of sight in the horizontal direction reaches the position change reference angle (first angle), the horizontal arrangement of the information elements is changed by the return angle (second angle). The same is true for the vertical direction, and when the line of sight change in the vertical direction reaches the position change reference angle in the vertical direction, display control is performed to change the vertical arrangement of the information element by the return angle in the vertical direction. may Note that the position change reference angle in the vertical direction may be set smaller than the position change reference angle in the horizontal direction, and may be set to 30 degrees, for example.

Next, consider where to initially display the information element. One method, as shown in FIG. 13A, is to display the information element near the center of the displayed image at the timing of displaying the information element. This method is intuitively understandable and effective for the user.

As another method, candidates for the display positions of the information elements may be set in advance in the virtual sphere. In the virtual spherical coordinate system, when the horizontal reference direction on the horizontal plane is 0 degrees, the candidate rotation angles are 60 degrees, 120 degrees, 180 degrees, 240 degrees, and 300 degrees in the horizontal direction. The candidate tilt angles are 30 degrees and ±60 degrees. Here, when displaying the information elements, the candidate rotation angle and the candidate tilt angle that are closest in line-of-sight direction may be extracted, and the information elements may be displayed around the extracted rotation angle and tilt angle. By predetermining candidates for the initial position in this way, the image generator 34 can simply select an initial position from among the candidate positions, thereby simplifying display control.

The present invention has been described above based on the examples. Those skilled in the art will understand that the embodiments are illustrative, and that various modifications can be made to combinations of each component and each treatment process, and such modifications are also within the scope of the present invention.

FIG. 14A shows a display image 200d and a menu window 204 that have been rotated counterclockwise by 22.5 degrees from the initial display state, and the counterclockwise rotation of 22.5 degrees may be achieved by turning the user's neck. , and may be realized by operating the left analog stick 77b. In the modified example, when the left analog stick 77b is operated, the relative positional relationship between the information element and the panoramic image is cancelled, and the information element in the image after switching by the operation of the left analog stick 77b is It may be displayed in the center as shown in FIG. 13(a). This is because when the image is switched by operating the left analog stick 77b, the displayed image jumps and the continuity of the image is interrupted. It is also possible to release the fixation of the positional relationship and control the display of the information elements.

Further, in the embodiment, display control of information elements is performed using the position change reference angle and the return angle. In a modified example, display control of information elements may be performed according to the type of information elements. For example, when the system issues an important warning to the user, the image generator 34 may always place the warning, which is an information element, in the center of the displayed image.

As one function of the VR application, the image generator 34 may realize a mirror function. Here, the mirror function is a function of including an image in the opposite direction to the viewing direction in the display image. Here, instead of including an image opposite to the line-of-sight direction, only the rotation angle of the line-of-sight direction is reversed. include. This mirror function allows the user to see the back side image at the same height.

Reference Signs List 1 information processing system 6 input device 8 virtual camera 10 information processing device 12 processing device 14 output control device 20 sensor Information acquisition unit 22 Photographed image acquisition unit 24 Instruction acquisition unit 30 Motion detection unit 32 Line of sight direction determination unit 34 Image generation unit 36 Image providing unit 40 Image storage unit 42 Information element storage unit 100 HMD 102 Output mechanism unit 104 Mounting mechanism unit 106 Mounting band DESCRIPTION OF SYMBOLS 108... Housing, 110... Light-emitting marker, 120... Control part, 122... Storage part, 124... Attitude sensor, 126... Microphone, 128... Communication control part, 130 ... display panel, 132 ... audio output section.

Claims (6)

a detection unit that detects the orientation of the device from a photographed image of the device and/or sensor information detected by the orientation sensor of the device;
a line-of-sight direction determination unit that determines a line-of-sight direction according to the detected orientation of the device;
an image generation unit that generates an image based on the determined line-of-sight direction;
an instruction acquisition unit that acquires an instruction to switch the line-of-sight direction,
The information processing apparatus according to claim 1, wherein when the instruction acquisition unit acquires the switching instruction, the line-of-sight direction determination unit changes the line-of-sight direction discontinuously or changes the line-of-sight direction by a predetermined angle. When the instruction acquisition unit continuously acquires switching instructions, the line-of-sight direction determination unit changes the line-of-sight direction discontinuously in the horizontal direction, or changes the direction in the horizontal direction by a predetermined angle.
The information processing apparatus according to claim 1, characterized by: The line-of-sight direction determination unit changes the line-of-sight direction at an angle greater than 10 degrees and 30 degrees or less.
The information processing apparatus according to claim 1, characterized by: The angle at which the line-of-sight direction determination unit changes the line-of-sight direction is set to a value obtained by dividing 360 degrees by an integer N,
The information processing apparatus according to claim 1, characterized by: a step of detecting the posture of the device from a photographed image of the device and/or sensor information detected by a posture sensor of the device;
determining the line-of-sight direction according to the detected orientation of the device;
generating an image based on the defined viewing direction;
obtaining an instruction to switch the line-of-sight direction, comprising:
The step of determining the line-of-sight direction changes the line-of-sight direction discontinuously or changes the line-of-sight direction by a predetermined angle when the switching instruction is acquired.
An image generation method characterized by: to the computer,
A function of detecting the orientation of the device from a photographed image of the device and/or sensor information detected by the orientation sensor of the device;
A function that determines the line-of-sight direction according to the detected posture of the device;
A function to generate an image based on a defined viewing direction;
A program for realizing a function of acquiring a line-of-sight direction switching instruction,
A program, wherein the function of determining the line-of-sight direction includes a function of changing the line-of-sight direction discontinuously or changing the line-of-sight direction by a predetermined angle when a switching instruction is acquired.

Images