JP2017146651A - Image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide, in a virtual reality space, a virtual object linked to a relative position between a user wearing a head-mounted display and a terminal gripped by the user.SOLUTION: A system according to one form of the present technology comprises a terminal information acquisition part 221, an object control part 222, and a visual field image creation part 223. The terminal information acquisition part 221 acquires information acquired from a terminal including a relative position of the terminal and a head-mounted display and the inclination of the terminal. The object control part 222 arranges a virtual object at the inclination of the terminal at a position in a virtual reality space specified on the basis of the position of a virtual camera in the virtual reality space and the relative position. The visual field image creation part 223 creates a visual field image in which the virtual object is arranged and outputs the visual field image to the head-mounted display.SELECTED DRAWING: Figure 3

Description

  The present technology relates to an image processing method and an image processing program. In particular, the present technology relates to an image processing method and an image processing program for displaying a virtual object arranged in a virtual reality space that is linked to a terminal held by a user.

  In a system using a head-mounted display, a technique for displaying an input operation surface of an input device on a head-mounted display in a user's visual field range is known. For example, in the technique of Patent Document 1, an input operation surface is arranged in the pedestrian's visual field range so as to secure the pedestrian's visual field.

JP 2013-125247 A

Even in a virtual reality space where a user wearing a head-mounted display is immersive, there is a need for a method that can operate the input device with a feeling of operation in the real space.
The present technology provides a method that can improve the immersion in the virtual reality space.

  In order to achieve the above object, one aspect of the present technology is a method for providing a virtual reality space into which a user immerses a head mounted display, the step of defining the virtual reality space, and the virtual reality space A step of specifying a position in the virtual reality space of a virtual camera defining a view image visually recognized by the user, a step of acquiring a tilt of a terminal held by the user in the real space, and the head mounted display And a step of acquiring a relative position between the terminal and the terminal, and a virtual touch panel of the first virtual object corresponding to the tilt at a position in the virtual reality space specified based on the position and the relative position of the virtual camera. Arranging the step, generating the field-of-view image including the first virtual object, and the head And outputting the count display to a method comprising a.

  According to the present technology, it is possible to improve an immersion feeling in the virtual reality space.

1 is a diagram illustrating an HMD system according to a first embodiment of the present technology. FIG. It is an XYZ space figure which shows an example of virtual space. It is a block diagram showing a function of an image processing device for realizing a function of an HMD system concerning a 1st embodiment of this art. FIG. 4A shows a user wearing an HMD that holds the terminal. FIG. 4B shows an example of a visual field image presented to the user of FIG. 4A via the HMD display. 6 is a flowchart illustrating an operation example of the image processing apparatus according to the first embodiment of the present technology. It is a figure which shows the HMD system by 2nd Embodiment of this technique. It is a block diagram which shows the function of an image processing apparatus for implement | achieving the function of the HMD system which concerns on 2nd Embodiment of this technique. FIG. 8A shows a user wearing an HMD that holds the terminal. FIG. 8B shows an example of a visual field image presented to the user of FIG. 8A via the HMD display. 14 is a flowchart illustrating an operation example of the image processing apparatus 200 according to the second embodiment of the present technology. The example of composition of the contents information concerning a 2nd embodiment of this art is shown.

  First, the contents of the embodiments of the present technology will be listed and described. An information processing method and program according to an embodiment of the present technology have the following configuration.

(Item 1)
A method for providing a virtual reality space for a user to immerse in a head-mounted display,
Defining the virtual reality space;
Identifying a position in the virtual reality space of a virtual camera defining a view image that the user visually recognizes in the virtual reality space;
Obtaining the tilt of the terminal held by the user in real space;
Obtaining a relative position between the head mounted display and the terminal;
Placing a virtual touch panel of a first virtual object at a position in a virtual reality space specified based on the position and the relative position of the virtual camera so as to correspond to the inclination;
Generating the field-of-view image including the first virtual object and outputting it to the head mounted display;
A method comprising:

(Item 2)
The method according to item 1, wherein the relative position is specified based on an image including the head mounted display taken by a camera of the terminal.

(Item 3)
The method according to item 1 or 2, wherein an image displayed on the virtual touch panel is rendered based on an image displayed on the display of the terminal.

(Item 4)
Identifying a user's touch input position on the touch panel of the terminal;
The step of specifying the position of the pointer with respect to the virtual touch panel based on the touch input position, The field-of-view image further includes the pointer at the position of the pointer with respect to the virtual touch panel. The method according to any one.

(Item 5)
Acquiring content information related to an image displayed on the display of the terminal from a server communicably connected to the system;
Generating a second virtual object based on the content information;
Generating a field-of-view image including the second virtual object and outputting to the head-mounted display;
The method according to any one of items 1 to 4, further comprising:

(Item 6)
The content information includes image information based on an image displayed on the display of the terminal, and mode information specifying a mode in which an image based on the image displayed on the display of the terminal is arranged in the virtual reality space,
6. The method of item 5, wherein the second virtual object is rendered based on the image information and aspect information.

(Item 7)
Identifying a user's touch input position on the touch panel of the terminal;
Further comprising: specifying a position of an input reception area with respect to the virtual touch panel based on the touch input position, wherein the second virtual object is rendered in the virtual reality space in association with the input reception area The method according to 5 or 6.

(Item 8)
A program for causing a computer to execute the method according to any one of items 1 to 7.

[Description of First Embodiment]
Hereinafter, a first embodiment of the present technology will be described with reference to the drawings. In addition, this technique is not limited to these illustrations, is shown by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and redundant descriptions are omitted.

  FIG. 1 is a hardware configuration diagram of an HMD system 100 according to an embodiment of the present technology. The HMD system 100 includes an HMD 110, an image processing device 200, and a terminal 300. For example, the HMD 110 and the image processing apparatus 200 are electrically connected by a wired cable 140 and can communicate with each other. Instead of the wired cable 140, a wireless connection (such as a wireless LAN or Bluetooth (registered trademark)) may be used.

[Overview of HMD system 100]
[HMD110]
The HMD 110 is a display device that is used by being worn on the head of the user 150. The HMD 110 includes a non-transmissive display 112 and a sensor 114. The HMD 110 may be mounted with any device according to other uses, and includes, for example, an eye tracking device (hereinafter referred to as ETD) 116.

  The display 112 displays a view image visually recognized by the user 150 in the virtual reality space in which the user 150 wearing the HMD 110 is immersed. The memory 204 stores various types of information (information on background images and information on images and positions of various objects arranged in the virtual reality space) that define the virtual reality space in which the user is immersed. Information stored in the memory 204 is executed by being developed by the processor 202, and a view image is rendered on the display 112.

  The display 112 may include a right-eye sub-display that provides a right-eye image and a left-eye sub-display that provides a left-eye image. Thereby, the user 150 can visually recognize the three-dimensional virtual reality space. Moreover, as long as the image for right eyes and the image for left eyes can be provided, you may be comprised by one display apparatus.

  The sensor 114 detects the inclination and / or position of the HMD 110. The sensor 114 is, for example, any one of a magnetic sensor, an angular velocity sensor, an acceleration sensor, or a combination thereof. The sensor 114 is built in the HMD 110 and outputs data (magnetism, angular velocity, or acceleration value) that specifies the inclination and position of the HMD 110. Based on the output data from the sensor 114, the inclination and position of the HMD 110 are specified, and the movement of the HMD 110 is detected. The view image displayed on the display 112 is preferably updated in conjunction with the inclination and position of the HMD 110. For example, when the user 150 turns his head to the right (or left, up, down), a visual field image in the right (or left, up, down) direction of the user is displayed on the display 112 in the virtual reality space. Thereby, the user 150 can be immersed in the virtual reality space.

  As the sensor 114, a sensor provided separately from the HMD 110 may be applied. For example, the sensor 114 is a position tracking camera, and is an infrared sensor that detects an infrared light emitter or infrared reflection marker provided on the surface of the HMD 110, and detects the movement of the HMD 110 by being installed at a predetermined position in the real space. To do.

  The ETD 116 is configured to track the eyeball movement of the user 150 and detect in which direction the user 150's line of sight is directed. For example, the ETD 116 includes an infrared light source and an infrared camera. The infrared light source irradiates infrared rays toward the eyes of the user 150 wearing the HMD 110. The infrared camera captures an image of the eyes of the user 150 irradiated with infrared rays. Infrared rays are reflected on the surface of the eye of the user 150, but the reflectance of the infrared rays differs between the pupil and a portion other than the pupil. In the image of the eyes of the user 150 captured by the infrared camera, the difference in the reflectance of the infrared appears as the brightness of the image. Based on this contrast, the pupil is identified in the image of the eye of the user 150, and the direction of the line of sight of the user 150 is detected based on the position of the identified pupil.

The HMD 110 may be provided with a speaker, a microphone, and the like.
[Image processing apparatus 200]
The image processing apparatus 200 is a computer connected to the HMD 110. The image processing apparatus 200 may be mounted on the HMD 110 or may be configured as another hardware (for example, a known personal computer or a server computer via a network). In addition, the image processing apparatus 200 may implement some functions in the HMD 110 and implement the remaining functions in different hardware. As shown in FIG. 1, the image processing apparatus 200 includes a processor 202, a memory 204, an input / output interface 206, and a communication interface 208.

  The processor 202 is configured to read a program stored in the memory 204 and execute processing according to the program. When the processor 202 executes an image processing program stored in the memory 204, each function of the image processing apparatus 200 described later is realized. The processor 202 includes arithmetic devices such as a CPU (Central Processing Unit), an MPU (Micro-processing unit), and a GPU (Graphics Processing Unit).

  The memory 204 stores a program for causing a realization computer to execute each function and method of the image processing apparatus 200 described later. The memory 204 can also temporarily store data generated by the operation of the image processing apparatus 200. The memory 204 includes a volatile storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and a nonvolatile storage device such as a hard disk, a flash memory, and an optical disk.

  The input / output interface 206 receives an input for operating the image processing apparatus 200 from the terminal 300 or the HMD 110. Further, the input / output interface 206 transmits the output from the image processing apparatus 200 to the terminal 300 and the HMD 110.

  The communication interface 208 includes various wired connection terminals for communicating with an external device via a network and various processing circuits for wireless connection. The communication interface 208 is configured to conform to various communication standards and protocols for communicating via a LAN (Local Area Network) or the Internet.

  FIG. 2 is an XYZ space diagram illustrating an example of a virtual reality space according to an embodiment of the present technology. The XZ plane represents a horizontal plane, and the Y axis extends in the vertical direction. The virtual reality space 6 is formed in, for example, a spherical shape with the center 3 as the center. Information that defines the virtual reality space 6 (information that forms a background image by being rendered on the celestial sphere, and aspects and position information of various objects arranged in the space) is stored in the memory 204 of the image processing apparatus. Yes. By reading the information, the processor 202 defines objects related to the background information and contents of the virtual reality space 6, and identifies the view image. The information specifies the virtual camera 1 that defines the view image visually recognized by the user in the virtual reality space 6 so as to be linked to the movement of the HMD 110. The movement of the virtual camera 1 changes following the movement of the head of the user 150 (movement of the HMD 110) and the movement of the line of sight of the user 150. The virtual camera 1 may always be arranged at the center 3 or may move in conjunction with the movement of the HMD.

The field-of-view area 5 is set to a range including a predetermined polar angle centered on the reference line of sight 4 and a range including a predetermined azimuth angle, as shown in FIG. It is preferable that the reference line-of-sight 4 is specified based on the viewing direction specified by the inclination and / or position of the HMD 110 and / or the line-of-sight direction of the user 150 detected by the ETD 116.
[Terminal 300]
The terminal 300 is an apparatus that is held by the user 150 and used by the user 150 to control the movement of a virtual object in the virtual reality space. The terminal 300 is a game terminal, a touch pad, a smartphone, a PDA, or the like, for example.

  The terminal 300 mainly includes an operation input unit 302, a processor 304, a memory 306, a display 308, a communication interface 310, a camera 312 and a sensor 314, which are connected to each other via a bus line. .

  The operation input unit 302 receives a user operation input and outputs an input signal corresponding to the operation input to the processor 304. The operation input unit 302 is a physical button or a touch panel provided on the main body of the terminal 300. The touch panel detects coordinate data corresponding to the touch input position at a predetermined sampling interval, and outputs the coordinate data.

The processor 304 is configured to read a program stored in the memory 306 and execute processing according to the program.
The image rendered by the processor 304 is displayed on the display 308. As the display 308, various display devices such as a liquid crystal display can be employed.

  The communication interface 310 connects to the image processing apparatus 200 via a network such as a wireless LAN connection function, an Internet connection function via a wireless LAN or a cellular phone network, and a short-range wireless communication function such as Bluetooth (registered trademark). Establish. The communication interface 310 transmits and receives various types of information between the terminal 300 and a communication partner (image processing apparatus 200 or server (not shown)).

  The camera 312 is an imaging device including an image sensor and the like. The camera 312 images a real space including part or all of the HMD 110 and stores the camera image in the memory 306. The processor 304 identifies the relative position between the camera 312 and the HMD 110 by analyzing how the HMD 110 appears in the camera image. In the present embodiment, the relative position between the camera 312 and the HMD 110 and the tilt of the camera 312 with respect to the HMD 110 can be specified. For calculating the relative position, a technique relating to so-called markerless AR (Augumented Reality) can be used.

The terminal 300 may detect the position and tilt of the terminal 300 using a built-in sensor 314 (for example, a magnetic sensor, an angular velocity sensor, or an acceleration sensor).
FIG. 3 is a block diagram showing a functional configuration of the image processing apparatus 200 according to the present embodiment. The image processing apparatus 200 includes a storage unit 210 and a processing unit 220. The storage unit 210 corresponds to the memory 204, and the processing unit 220 corresponds to the processor 202. The virtual reality space configuration information 211 is stored. The virtual reality space configuration information 211 includes information regarding the three-dimensional space image of the virtual reality space 6. The processing unit 220 includes a terminal information acquisition unit 221, an object control unit 222, and a view field image generation unit 223.

  In the present embodiment, the first virtual object is displayed in the virtual reality space based on the relative positional relationship in the real space between the terminal 300 held by the user 150 and the HMD 110 worn by the user 150. The first virtual object moves in the virtual reality space 6 in conjunction with the movement of the terminal 300 in the real space.

A processing flow of the image processing apparatus 200 will be described with reference to FIGS.
In step S501, the terminal information acquisition unit 221 acquires the relative position information a between the terminal 300 and the HMD in the real space.

  In step S <b> 502, the terminal information acquisition unit 221 acquires display image information b of the terminal 300. The display image information b is information including a two-dimensional image (for example, an image including a banner advertisement, a still image, a moving image, and text) displayed on the display 308 of the terminal 300.

  In step S <b> 503, the terminal information acquisition unit 221 acquires the inclination information c of the terminal 300. The terminal information acquisition unit 221 may further acquire operation input information d by the user. The operation input information d includes a touch input coordinate position on the touch panel (operation input unit 302), a moving direction, a moving speed, a moving amount, input signals to various operation buttons, and the like.

FIG. 4A shows a state in which the user 150 wearing the HMD 110 operates the terminal 300. The user 150 operates by bringing the index finger into contact with the touch panel.
FIG. 4B shows an example of a view field image displayed on the display 112 of the HMD 110 shown in FIG. 4A. A three-dimensional image in which the first virtual object 402 is arranged is rendered in the view field image.

  In step S504, the object control unit 222 identifies the position of the first virtual object 402 in the virtual reality space 6 based on the position of the virtual camera 1 and the relative position information a.

  In step S <b> 505, the object control unit 222 generates the first virtual object 402 and places it at the position of the specified virtual reality space 6. For example, the first virtual object 402 is a substantially rectangular parallelepiped object having a virtual touch panel 406 as shown in FIG. 4B. The object control unit 222 may arrange the first virtual object 402 in the virtual reality space 6 based on the inclination information c. Thereby, the movement (position, inclination) of the first virtual object 402 is linked to the movement (position, inclination) of the terminal 300 in the real space.

  The object control unit 222 may associate the coordinates (xt, yt) on the touch panel (operation input unit 302) of the terminal 300 with the coordinates (xvt, yvt) on the virtual touch panel 406 of the first virtual object 402. Further, the object control unit 222 may render an image displayed on the virtual touch panel 406 of the first virtual object 402 based on the display image information b. As a result, the image displayed on the virtual touch panel 406 is linked with the display image on the display 308 in the real space.

In step S <b> 506, the visual field image generation unit 223 refers to the virtual reality space configuration information 211 stored in the storage unit 210 and generates a visual field image corresponding to the visual field region 5. The view field image includes a first virtual object 402 and a pointer 404 that is linked to the touch input position A. When the touch input position A of the user with respect to the touch panel (operation input unit 302) is specified by the object control unit 222, the visual field image generation unit 223 determines the position B of the pointer 404 with respect to the virtual touch panel 406 based on the touch input position A. Is identified. As shown in FIG. 4B, the pointer 404 is displayed at a position B on the virtual touch panel 406 so as to correspond to the touch input position A on the touch panel. The shape of the pointer is arbitrary, and has, for example, the shape of a hand, a finger, or an arrow. In response to the change in the touch input position A, the position B of the pointer 404 with respect to the virtual touch panel changes.
[Description of Second Embodiment]
In the present embodiment, in addition to the first virtual object 402 shown in the first embodiment, a second virtual object is arranged in the virtual reality space 6. The second virtual object is generated based on the display image information b on the display 308 of the terminal 300. The second virtual object is, for example, an advertising object related to the display image. The first virtual object 402 of the first embodiment is an object that is linked to the movement of the terminal 300, whereas the second virtual object may not be linked to the movement of the terminal 300.

  FIG. 6 is a hardware configuration diagram of the HMD system 100 according to the second embodiment of the present technology. The second embodiment is different from the first embodiment described above in that the image processing apparatus 200 is connected to the server 400 via the network 500 so that they can communicate with each other. The network 500 may be any type of network such as the Internet or a LAN.

  FIG. 7 is a block diagram illustrating a functional configuration of the image processing apparatus 200 according to the second embodiment of the present technology. The second embodiment is different from the first embodiment in that the processing unit 220 further includes a content information acquisition unit 224 and a second virtual object generation unit 225.

  FIG. 9 is a flowchart illustrating a process of the image processing device 200 according to the second embodiment of the present disclosure. The process of step S901 to step S905 shown in FIG. 9 is the same as the process of step S501 to step S505 of FIG. Hereinafter, in this embodiment, the description of the same part as the structure of the first embodiment is omitted or simplified, and the part different from the first embodiment will be mainly described with reference to FIGS.

  In step S <b> 906, the content information acquisition unit 224 acquires content information related to an image displayed on the display 308 of the terminal 300 from the server 400 via the network 500 and stores the content information in the memory 204 of the image processing apparatus 200.

  The content information is various information related to the image displayed on the display 308, and includes information different from the two-dimensional image constituting the image displayed on the display 308. For example, when a part of the display image information b is a two-dimensional banner advertisement image, the content information is completely different from the two-dimensional banner advertisement image and the two-dimensional banner advertisement image. Includes three-dimensional advertising images with different shapes and sizes. For example, when a part of the display image information b of the terminal 300 is a still image, the content information may be a moving image corresponding to the still image. As described above, the content information can include a VR (Virtual Reality) image (for example, a three-dimensional image or a moving image) in which the display image on the terminal display is associated with the virtual reality space.

  The content information further includes mode information for specifying a mode in which the VR image based on the image displayed on the display 308 of the terminal 300 is arranged in the virtual reality space. The aspect information includes, for example, information on the arrangement coordinates of the VR image in the three-dimensional virtual reality space 6, the format of the VR image, the capacity of the VR image, and the like. The arrangement coordinates may be any position in the virtual reality space 6 and may not be linked to the movement of the terminal 300 in the real space. For example, the arrangement coordinates of the content information may always move so as to be away from the user's gazing point by a certain distance so as not to disturb the user's field of view, or may be fixed. FIG. 10 shows an example of content information stored in the memory 204.

  In step S907, the second virtual object generation unit 225 reads the content information from the memory 204, and generates the second virtual object 802 in the arrangement coordinates based on the image information and the aspect information included in the content information.

The view image generation unit 223 generates a view image including the second virtual object 802 and outputs the view image to the HMD 110. (Step S908)
FIG. 8A shows how the user 150 operates the terminal 300. An image based on display image information b including a two-dimensional advertisement image 806 is displayed on the display 308 of the terminal 300.

  FIG. 8B shows an example of a view field image displayed on the display 112 of the HMD 110 shown in FIG. 8A. In the view image, a three-dimensional image in which a balloon-shaped second virtual object 802 is arranged in addition to the sword-shaped first virtual object 402 and the hand-shaped pointer 404 is rendered. The image displayed on the virtual touch panel 406 of the first virtual object 402 is rendered based on the display image information b. The second virtual object 802 is rendered based on image information and aspect information related to the advertisement image 806.

  As described above, the image displayed on the virtual touch panel 406 is rendered based on the display image information b. However, the image displayed on the virtual touch panel 406 in the view field image may be small and difficult to view. According to this embodiment, based on the content information related to the display image information b on the display 308, the second virtual object 802 having a different form from the virtual touch panel 406 (for example, a large three-dimensional image) is displayed in the view image. Or displayed on the second virtual object 802 in a different form (for example, with large moving characters) in the view field image. By displaying the second virtual object 802 different from the first virtual object 402 based on the content information, it is possible to provide the user 150 with a visual field image that is easier to visually recognize.

  Further, the second virtual object 802 is rendered in the virtual reality space in association with the input reception area 804 on the virtual touch panel 406. The input receiving area 804 is an area where an input from the user 150 can be received via the virtual touch panel, and is associated with a part of the display image on the display 308 (the advertisement image 806 in FIG. 8A). The position of the input reception area 804 with respect to the virtual touch panel 406 is specified based on the touch input position A of the user with respect to the touch panel (operation input unit 302). The user input for the input reception area 804 is actually a touch input for a part of the display image on the display 308.

The terminal information acquisition unit 221 detects a touch input (for example, a tap input) for a position on the touch panel (operation input unit 302) corresponding to the input reception area 804 on the virtual touch panel 406. (Step S909)
When a touch input corresponding to the position A on the touch panel (operation input unit 302) corresponding to the input reception area 804 is detected, the process proceeds to step S910 (“Yes” in step S909).

  The view image generation unit 223 generates a view image based on a new content image linked to a part of the display image b of the terminal 300 (the advertisement image 806 in FIG. 8A) associated with the input reception area 804. (Step S910). The view image generated by the view image generation unit 223 is output to the HMD 110 and displayed on the display 112.

  In the present embodiment, the second virtual object 802 generated based on the two-dimensional display image of the terminal 300 can be generated based on a VR image that is different from the original display image of the terminal 300. Therefore, a VR image that gives a higher impact than a two-dimensional display image can be arranged in the view field image. In the present embodiment, the probability of guiding the user 150 to the advertisement can be increased by displaying the advertisement image that gives a high impact in the view field image.

  As mentioned above, although embodiment of this technique was described, this technique is not limited to the said embodiment. Those skilled in the art will appreciate that various modifications of the embodiments can be made without departing from the spirit and scope of the invention as set forth in the appended claims.

  DESCRIPTION OF SYMBOLS 100 ... Head mounted display (HMD) system, 110 ... HMD, 112 ... Display, 114 ... Sensor part, 116 ... ETD, 200 ... Image processing apparatus, 210 ... Memory | storage part, 220 ... Processing part, 1 ... Virtual camera, 4 ... Reference line of sight, 5 ... field of view, 6 ... virtual reality space, 308 ... display, 402 ... first virtual object, 404 ... pointer, 406 ... virtual touch panel, 802 ... second virtual object

Claims (8)

A method for providing a virtual reality space for a user to immerse in a head-mounted display,
Defining the virtual reality space;
Identifying a position in the virtual reality space of a virtual camera defining a view image that the user visually recognizes in the virtual reality space;
Obtaining the tilt of the terminal held by the user in real space;
Obtaining a relative position between the head mounted display and the terminal;
Placing a virtual touch panel of a first virtual object at a position in a virtual reality space specified based on the position and the relative position of the virtual camera so as to correspond to the inclination;
Generating the field-of-view image including the first virtual object and outputting it to the head mounted display;
A method comprising:   The method according to claim 1, wherein the relative position is determined based on an image including the head mounted display taken by a camera of the terminal.   The method according to claim 1, wherein the image displayed on the virtual touch panel is rendered based on information related to an image displayed on the display of the terminal. Identifying a user's touch input position on the touch panel of the terminal;
The step of specifying the position of the pointer with respect to the virtual touch panel based on the touch input position, further comprising: the view image further including the pointer at the position of the pointer with respect to the virtual touch panel. The method in any one of. Acquiring content information related to an image displayed on the display of the terminal from a server communicably connected to the system;
Generating a second virtual object based on the content information;
Generating a field-of-view image including the second virtual object and outputting to the head-mounted display;
The method according to claim 1, further comprising: The content information includes image information based on an image displayed on the display of the terminal, and mode information specifying a mode in which an image based on the image displayed on the display of the terminal is arranged in the virtual reality space,
The method of claim 5, wherein the second virtual object is rendered based on the image information and aspect information. Identifying a user's touch input position on the touch panel of the terminal;
Further comprising: specifying a position of an input reception area with respect to the virtual touch panel based on the touch input position, wherein the second virtual object is rendered in the virtual reality space in association with the input reception area. Item 7. The method according to Item 5 or 6. The program for making a computer perform the method of any one of Claims 1-7.