JP7275480B2 - Multiplayer Simultaneous Operation System, Method, and Program in VR - Google Patents

Description

The present invention relates to a technology that allows a plurality of people to simultaneously experience virtual reality (VR) while moving their viewpoints in a virtual space.

Conventionally, by viewing three-dimensional computer graphics (hereinafter also referred to as 3DCG) or live-action images projected on a giant screen or head-mounted display, it is possible to feel as if one is actually in the virtual space represented by those images. A virtual reality (hereinafter also referred to as VR) technology has been developed that can provide such a feeling.

In a method in which a large number of people can experience VR by watching images projected on a huge screen (hereinafter also referred to as theater type VR), the people who experience VR (hereinafter also referred to as spectators) are watching the same screen. ing. A person who introduces the virtual space to the audience (hereinafter also referred to as a navigator) operates a controller such as a game pad to change the position and orientation of the viewpoint in the virtual space for rendering.

By the way, in theater-type VR, the controller is not operated by the navigator, but the audience is made to operate the controller and experience moving freely in the virtual space. The spectator can operate the controller to move the position of the viewpoint in the virtual space to the place he or she wants to go, or move the direction of the viewpoint in the virtual space to the direction in which the spectator wants to see.

However, images rendered based on the viewpoint in the virtual space that the spectator is moving are displayed on a giant screen (ie, one screen). Since there is only one screen on which images can be projected while the spectators are operating the controllers, the spectators can only operate the controllers one by one. Therefore, it takes time for all spectators to experience the operation of the controller, and the number of spectators who can experience the operation of the controller has to be limited.

Therefore, the present invention has been made in view of such problems, and its purpose is to allow a large number of spectators to experience VR while simultaneously moving the positions and directions of their respective viewpoints in a virtual space. It is to provide a system, method, and program capable of

To achieve these ends, a first aspect of the invention is an apparatus. This device has a first viewpoint acquisition unit that acquires at least one of information on the position and orientation of a first viewpoint in the virtual space from the first operation device; a second viewpoint acquisition unit that acquires information on at least one of a position and an orientation of a second viewpoint; and an image rendered based on at least part of the information about the first viewpoint; and a rendering unit that superimposes and displays an image rendered based on at least part of the information about.

A second aspect of the invention is a method. This method comprises the steps of: acquiring information on at least one of the position and orientation of a second viewpoint in virtual space for the second display device from the second operating device; an image rendered based on at least part of information about at least one of the position and orientation of a first viewpoint in virtual space, and an image rendered based on at least part of information about the second viewpoint superimposed thereon and displayed on the second display device.

A third aspect of the present invention is a program. This program is a program that causes a computer to execute the method of the second aspect.

According to the present invention, each spectator experiences VR by viewing images rendered on a tablet possessed by each spectator based on the position and orientation of the viewpoint in the virtual space being moved by each spectator. be able to.

1 is an overall schematic diagram including a multi-person simultaneous operation device according to an embodiment of the present invention; FIG. 1 is a functional block diagram of a multi-person simultaneous operation device according to an embodiment of the present invention; FIG. It is an example of an image displayed on the screen of the multi-person simultaneous operation device according to one embodiment of the present invention. FIG. 4 is a sequence diagram showing the flow of photographing processing in the multi-person simultaneous operation device according to the embodiment of the present invention; FIG. 5 is a sequence diagram showing the flow of video relay processing in the multi-user simultaneous operation device according to the embodiment of the present invention; FIG. 10 is a sequence diagram showing the flow of drone icon display processing in the multi-person simultaneous operation device according to the embodiment of the present invention;

First, terms used in this specification will be explained. A “virtual space” is a virtual space represented by an image projected on a giant screen or the like. "Rendering" refers to creating an image based on the position and orientation of a viewpoint (also called a virtual camera) in virtual space, light source information in virtual space, and object information in virtual space. To tell. That is, an image captured by a virtual camera placed in the virtual space is rendered. Also, in this specification, "image" includes both "still image" and "video (also referred to as moving image)".

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an overall schematic diagram including a multi-person simultaneous operation device 100 according to one embodiment of the present invention. The overall schematic diagram includes a multi-person simultaneous operation device 100, one screen 111 viewed by spectators, a controller 112 operated by a navigator, and a plurality of tablets 120 held by each spectator. The screen 111 and the controller 112 are collectively called a multi-person viewing device. The tablet 120 is also called a personal viewing device. The multi-person simultaneous operation device 100, the screen 111, the controller 112, and the tablet 120 are collectively referred to as a multi-person simultaneous operation system. The multi-person simultaneous operation device 100 is a device that allows a plurality of spectators to simultaneously experience VR while moving the positions and orientations of their respective viewpoints in a virtual space. Specifically, the multi-user simultaneous operation device 100 performs rendering based on information about the position and orientation of the viewpoint in the virtual space acquired from the controller 112 and the tablet 120, and projects the rendered image on the screen 111. , or to the tablet 120 (which can also be rendered by the tablet 120, as described below). Details of the multi-person simultaneous operation device 100 will be described later with reference to FIG. Details of the screen 111, the controller 112, and the tablet 120 will be described below.

In the present invention, rendering may be performed by the tablet 120, or rendering may be performed by the multi-user simultaneous operation device 100. FIG.

A screen (main screen) 111 is a display device for displaying video for VR generated by the multi-user simultaneous operating device 100 (ie, an image rendered by the multi-user simultaneous operating device 100). For example, spectators in the audience seats can experience VR by looking at one huge screen 111 installed in front. For example, screen 111 is a projector screen. That is, a projector (not shown) acquires an image from the multi-user simultaneous operation device 100 and projects the image on the screen 111 . Note that the screen 111 can be any display device such as a display or a monitor.

The controller 112 is an operation device for instructing the multi-person simultaneous operation device 100 at least one of the position and orientation of the viewpoint in the virtual space. For example, a navigator can manipulate the controller 112 to move the position and/or orientation of a viewpoint in virtual space for rendering an image projected on the screen 111 . The controller 112 can send information on at least one of the position and direction of the viewpoint input by the navigator to the multi-user simultaneous operation device 100 via wireless communication or the like. For example, controller 112 is a gamepad. Note that the controller 112 can be any operating device such as a mouse and a keyboard. Alternatively, controller 112 can be a tablet as described below.

The tablet (second screen) 120 displays an image for VR generated by the tablet 120 (that is, an image rendered by the tablet 120 itself) or an image for VR generated by the multi-user simultaneous operation device 100 (that is, a multi-user simultaneous display). and an operating device function of instructing at least one of the viewpoint position and orientation to the tablet 120 itself or to the multi-user simultaneous operating device 100. It's a computer. For example, spectators can use tablets 120 distributed by navigators or the like. The tablet 120 can display on the screen of the tablet 120 an image rendered by the tablet 120 based on information on at least one of the viewpoint position and orientation input to the tablet 120 by the spectator. Alternatively, the tablet 120 can receive an image rendered by the multi-user simultaneous manipulation device 100 from the multi-user simultaneous manipulation device 100 via wireless communication or the like, and display the image on the screen of the tablet 120 . In addition, the tablet 120 can transmit information on at least one of the viewpoint position and orientation input by the audience to the multi-user simultaneous operation device 100 via wireless communication or the like. For example, the spectator operates a controller using a GUI (graphical user interface) displayed on the screen of the tablet 120, uses an acceleration sensor or gyroscope built in the tablet 120 (that is, moves the tablet 120 itself). ), the position and orientation of the viewpoint in the virtual space for rendering the image projected on the screen of the tablet 120 can be changed. Note that the tablet 120 can be a head-mounted display. The head-mounted display may be any head-mounted display such as binocular, monocular, non-transmissive, and transmissive.

FIG. 2 is a functional block diagram of a multi-person simultaneous operation device according to one embodiment of the present invention. The multi-person simultaneous operation device 100 includes a main screen viewpoint acquisition unit 201, a second screen viewpoint acquisition unit 202, a main screen rendering unit 203, a second screen rendering unit 204, a photographing unit 205, a photograph display unit 206, an image display unit 207, and a drone. It includes a display unit 208 , a shooting information storage unit 209 and a photo storage unit 210 . The multi-manipulator 100 is one or more computers including processors and memory. The multi-person simultaneous operation device 100 also includes a main screen viewpoint acquisition unit 201 , a second screen viewpoint acquisition unit 202 , a main screen rendering unit 203 , a second screen rendering unit 204 , a photographing unit 205 , a photograph display unit 206 , and an image display unit 207 . , a storage medium storing a program for operating the drone display unit 208 or a program for executing a processing flow to be described later.

The main screen viewpoint acquisition unit 201 can acquire information on at least one of the position and direction of the viewpoint input by the navigator from the controller 112 . Also, the main screen viewpoint acquisition unit 201 can acquire information other than the viewpoint indicating the situation in the virtual space (for example, lighting, time, weather, etc.) input by the navigator from the controller 112 . In other words, the navigator can use the controller 112 to change the situation in the virtual space. The main screen viewpoint acquisition unit 201 can store the acquired information in a memory (not shown).

The second screen viewpoint acquisition unit 202 can acquire information on at least one of the position and orientation of the viewpoint input by the audience from the tablet 120 . Also, the second screen viewpoint acquisition unit 202 can acquire information other than the viewpoint indicating the situation in the virtual space (for example, lighting, time, weather, etc.) input by the audience from the tablet 120 . In other words, the audience can use the tablet 120 to change the situation in the virtual space. The second screen viewpoint acquisition unit 202 can store the acquired information in a memory (not shown).

The main screen rendering unit 203 creates an image based on the viewpoint position and orientation information acquired by the main screen viewpoint acquisition unit 201 and information other than the viewpoint in the virtual space (eg, lighting, time, weather, etc.). (i.e., render) to generate images for VR. The main screen rendering unit 203 can send the generated video for VR to, for example, a projector. Then, the projector projects the image acquired from the main screen rendering unit 203 onto the screen 111 . It should be noted that the main screen rendering unit 203 may use both of the acquired viewpoint position and orientation information, or may use only one of them.

The second screen rendering unit 204 creates an image based on the viewpoint position and orientation information acquired by the second screen viewpoint acquisition unit 202 and information other than the viewpoint in the virtual space (for example, lighting, time, weather, etc.). (i.e., render) to generate images for VR. The second screen rendering unit 204 can transmit the generated video for VR to the tablet 120 . As noted above, tablet 120 may render, in which case second screen renderer 204 need not render. It should be noted that the second screen rendering unit 204 may use both the obtained viewpoint position and orientation information, or may use only one of them.

Here, the relationship between the virtual space represented by the image projected on the main screen (screen 111) and the virtual space represented by the image projected on the second screen (tablet 120) will be described. The virtual space represented by the image projected on the main screen (screen 111) and the virtual space represented by the image projected on the second screen (tablet 120) represent the same space. For example, while the navigator is introducing the inside of the virtual space to the audience on the main screen (screen 111), the same space as the virtual space projected on the main screen (screen 111) at that time is displayed in the same situation. (eg, lighting, time of day, weather, etc.) can also be projected on the second screen (tablet 120). Therefore, on the main screen (screen 111), the audience can see the virtual space based on the viewpoint moved by the navigator, and on the second screen (tablet 120), the same virtual space as the main screen (screen 111) can be viewed. can be seen based on the viewpoint that the spectator himself moves. In other words, the audience can experience moving freely in the virtual space viewed on the main screen (screen 111).

Return to FIG. The photographing unit 205 can allow the audience to photograph the virtual space with the tablet 120 . That is, it is possible to shoot a scene in a virtual space instead of shooting a scene in a real space. Specifically, the photographing unit 205 captures the scene in the virtual space seen from the viewpoint desired by the spectator while the spectator experiences moving freely in the virtual space on the second screen (tablet 120). It can be taken as a snapshot.

<Accumulation of shooting information>
The photographing unit 205 can receive an instruction from the tablet 120 to take a photograph in the virtual space. For example, the spectator presses a shutter button displayed on the screen of tablet 120 . Then, tablet 120 can transmit an instruction to take a picture of the virtual space to photographing unit 205 . Further, when the photographing unit 205 receives an instruction to take a photograph in the virtual space, the photographing unit 205 provides information on at least one of the position and orientation of the viewpoint in the virtual space for rendering the image displayed on the tablet 120. (That is, at least one of the position and direction of the viewpoint at that point in time obtained by the second screen viewpoint obtaining unit 202 from the tablet 120 ) can be stored in the shooting information storage unit 209 . In addition, when the photographing unit 205 receives an instruction to take a photograph in the virtual space, the photographing unit 205 provides information other than the viewpoint of the virtual space for rendering the image displayed on the tablet 120 (that is, acquisition of the second screen viewpoint). (information other than the point of view of the virtual space at that point in time acquired by the unit 202 from the tablet 120 ) can be stored in the imaging information storage unit 209 . For example, information other than the viewpoint of the virtual space for rendering is information that affects light sources and objects in the virtual space, such as illumination, time of day, and weather.

<Photo generation>
The photographing unit 205 performs rendering based on the information on the position and direction of the viewpoint stored in the photographing information storage unit 209, and the information that affects the light source and objects such as lighting, time, and weather in the virtual space. can be done. Note that the photographing unit 205 may use both the stored viewpoint position and orientation information, or may use only one of them. The photographing unit 205 can store the rendered result in the photograph storage unit 210 as an image (for example, a texture image). In this way, the photographing unit 205 can generate a still image (photograph) of the image displayed on the tablet 120 when the audience presses the shutter button.

As described above, the photographing information storage unit 209 stores information about the position and orientation of the viewpoint in the virtual space when the audience presses the shutter button, and light sources such as lighting, time, and weather in the virtual space. Contains information that affects an object.

As described above, the photograph storage unit 210 stores a still image (photograph) of an image displayed on the tablet 120 when the audience presses the shutter button, for example, as a texture image.

The photo display unit 206 can display photos stored in the photo storage unit 210 on the main screen (screen 111). Specifically, the photo display unit 206 displays one or a plurality of photos stored in the photo storage unit 210 so as to be superimposed on part of the video for VR projected on the main screen (screen 111). can do. In addition, the photo display unit 206 superimposes information such as the name and ID of the spectator holding the tablet 120 who took the photo on a part of the video for VR displayed on the main screen (screen 111). can be displayed. In this case, the multi-user simultaneous operation device 100 acquires information such as the names and IDs of the spectators possessing each tablet 120 in advance.

In this way, a photograph taken while the spectator is experiencing free movement in the virtual space on the second screen (tablet 120) is displayed on the main screen (screen 111). Therefore, the spectators can know what kind of scene other spectators have photographed in the virtual space. Note that the multi-person simultaneous operation device 100 can transmit still images (photographs) in the photo storage unit 210 to arbitrary computers (for example, spectator's smart phones, personal computers, etc.). Alternatively, the multi-user simultaneous operation device 100 can print and output the still images (photographs) in the photo storage section 210 with an arbitrary printer.

The video display unit 207 can display the video that one or more spectators are watching on the second screen (tablet 120) on the main screen (screen 111). Specifically, the image display unit 207 displays the image for VR generated by the second screen rendering unit 204 (that is, the image rendered by the multi-user simultaneous operation device 100) or the image for VR generated by the tablet 120 ( That is, the image rendered by the tablet 120) can be superimposed and displayed on part of the video for VR projected on the main screen (screen 111). In addition, the video display unit 207 superimposes information such as the name and ID of the spectator holding the tablet 120 displaying the video on a portion of the VR video displayed on the main screen (screen 111). can be displayed. In this case, the multi-user simultaneous operation device 100 acquires information such as the names and IDs of the spectators possessing each tablet 120 in advance. The reverse is also possible, and the image display unit 207 superimposes the VR image generated by the main screen rendering unit 203 on a part of the VR image displayed on the second screen (tablet 120). can be displayed.

In this way, the video that the audience sees while experiencing free movement in the virtual space on the second screen (tablet 120) is relayed on the main screen (screen 111). Therefore, the audience can know what kind of images other audiences are watching.

The drone display unit 208 can display on the main screen (screen 111) an icon showing how each spectator is moving in the virtual space. Specifically, the drone display unit 208 can place an icon (for example, a virtual drone) at the position of the viewpoint indicated by the tablet 120 in the virtual space. Further, the drone display unit 208 can match the orientation of the arranged icon (for example, virtual drone) with the orientation of the viewpoint indicated by the tablet 120 . Then, the main screen rendering unit 203 can render a virtual space in which icons (for example, virtual drones) are arranged. In addition, the drone display unit 208 displays information such as the name and ID of the spectator who owns the tablet 120 corresponding to the icon (for example, virtual drone), and displays the video for VR displayed on the main screen (screen 111). can be displayed superimposed on a part of In this case, the multi-user simultaneous operation device 100 acquires information such as the names and IDs of the spectators possessing each tablet 120 in advance. Note that the drone display unit 208 can also display an icon (for example, a virtual drone) displayed on the main screen (screen 111) on the second screen (tablet 120). Also, the drone display unit 208 can display an icon (eg, a virtual drone) indicating the position and orientation of the viewpoint indicated by the controller 112 on the second screen (tablet 120).

In this way, a drone icon is displayed on the main screen (screen 111) that indicates the position and orientation of the viewpoint when the audience is experiencing free movement in the virtual space on the second screen (tablet 120). be. Therefore, the spectator can know where the other spectators are in the virtual space. Furthermore, the spectator can know where other spectators took pictures in the virtual space.

FIG. 3 is an example of an image displayed on the screen 111 of the multi-person simultaneous operation device according to one embodiment of the present invention. An image for VR generated by the main screen rendering unit 203 is displayed on the main screen 111 in FIG. Also, on the main screen of FIG. 3, one or more photos 301 stored in the photo storage unit 210 may be superimposed on a portion of the video for VR generated by the main screen rendering unit 203 and displayed. In the main screen of FIG. 3, one or a plurality of images 302 for VR generated by the tablet 120 or the second screen rendering unit 204 are superimposed on a part of the images for VR generated by the main screen rendering unit 203. can be displayed as Also, one or more drone icons 303 may be displayed on the main screen of FIG.

FIG. 4 is a sequence diagram showing the flow of photographing processing in the multi-person simultaneous operation device according to one embodiment of the present invention. It is assumed that the tablet 120 continues to transmit information on at least one of the viewpoint position and orientation input by the audience to the multi-user simultaneous operation device 100 . Note that the tablet 120 may continue to transmit information other than the viewpoint of the virtual space input by the audience (for example, lighting, time, weather, etc.) to the multi-user simultaneous operation device 100 . It is also assumed that the controller 112 continues to transmit information on at least one of the position and direction of the viewpoint input by the navigator to the multi-user simultaneous operation device 100 . Note that the controller 112 may continue to transmit information other than the viewpoint of the virtual space input by the navigator (for example, lighting, time of day, weather, etc.) to the multi-user simultaneous operation device 100 .

At step 401 (S401), the tablet 120 transmits an instruction to the multi-user simultaneous operation device 100 to take a picture in the virtual space.
At step 402 (S402), the multi-user simultaneous operation device 100 stores information on the position and orientation of the viewpoint in the virtual space for rendering the image displayed on the tablet 120 at the time of S401. Note that the multi-user simultaneous operation device 100 stores information (for example, lighting, time, weather, etc.) other than the viewpoint of the virtual space for rendering the image displayed on the tablet 120 at the time of S401. you can
At step 403 (S403), the multi-user simultaneous operation device 100 performs rendering based on the viewpoint position and orientation stored at S402. After that, the multi-user simultaneous manipulation device 100 stores the rendering result as an image (for example, a texture image).
At step 404 ( S<b>404 ), the multi-player simultaneous control device 100 renders based on the viewpoint position and orientation that it continues to receive from the controller 112 . Then, the multi-user simultaneous operation device 100 superimposes and displays the image (for example, texture image) stored in S403 on the rendered image.

FIG. 5 is a sequence diagram showing the flow of video relay processing in the multi-user simultaneous operation device according to the embodiment of the present invention. It is assumed that the tablet 120 continues to transmit information on at least one of the viewpoint position and orientation input by the audience to the multi-user simultaneous operation device 100 . It is also assumed that the controller 112 continues to transmit information on at least one of the position and direction of the viewpoint input by the navigator to the multi-user simultaneous operation device 100 .

At step 501 ( S<b>501 ), the multi-user simultaneous manipulation device 100 renders based on the viewpoint position and orientation that it continues to receive from the tablet 120 . Alternatively, multi-user simultaneous operation device 100 receives images rendered by tablet 120 from tablet 120 . Also, the multi-user simultaneous manipulation device 100 renders based on the viewpoint position and orientation that it continues to receive from the controller 112 .
At step 502 (S502), the multi-user simultaneous operation device 100 renders the rendered image based on the position and orientation of the viewpoint from the controller 112 at S501, and renders the image based on the position and orientation of the viewpoint from the tablet 120 at S501. The rendered image or the image rendered by the tablet 120 is superimposed and displayed. At the same time, the multi-user simultaneous operation device 100 projects the rendered image on the tablet 120 in S501 based on the position and orientation of the viewpoint from the tablet 120 (this process is not necessary when the tablet 120 renders).

FIG. 6 is a sequence diagram showing the flow of drone icon display processing in the multi-user simultaneous operation device according to the embodiment of the present invention. It is assumed that the tablet 120 continues to transmit information on at least one of the viewpoint position and orientation input by the audience to the multi-user simultaneous operation device 100 . It is also assumed that the controller 112 continues to transmit information on at least one of the position and direction of the viewpoint input by the navigator to the multi-user simultaneous operation device 100 .

At step 601 (S601), the multi-user simultaneous operation device 100 arranges an icon (for example, a virtual drone) at the position of the viewpoint indicated by the tablet 120 in the virtual space. Also, the multi-user simultaneous operation device 100 matches the orientation of the arranged icon (for example, virtual drone) with the orientation of the viewpoint indicated by the tablet 120 .
At step 602 (S602), the multi-user simultaneous operating device 100 renders the virtual space in which the icon (for example, virtual drone) is placed at S601 based on the position and orientation of the viewpoint from the controller 112. FIG.
At step 603 (S603), the multi-user simultaneous operation device 100 displays the image rendered at S602 on the screen 111. FIG. In addition to or instead of S603, the multi-user simultaneous operation device 100 displays an icon (for example, virtual drone ) may be displayed.

<Linkage of viewpoint position>
Instead of using the tablet 120 to move both the position and orientation of the viewpoint, the spectator may be able to move only the orientation of the viewpoint. For example, the tablet 120 can generate an image for VR to be displayed on the tablet 120 by performing rendering based on viewpoint position information obtained from the controller 112 and viewpoint orientation information obtained from the tablet 120 . can. Alternatively, the multi-user simultaneous operation device 100 renders based on the viewpoint position information obtained from the controller 112 and the viewpoint direction information obtained from the tablet 120, and displays an image for VR on the tablet 120. can be generated. Therefore, spectators can take pictures from the same position facing different directions.

<Use of tablets>
Note that the tablets 120 distributed to the audience can also be utilized as follows. The multi-person simultaneous operation device 100 can display subtitles of explanations spoken by the navigator on the tablet 120 based on the attributes of the audience acquired in advance (for example, native language). In addition, the multi-user simultaneous operation device 100 can display supplementary explanations and images on the tablet 120 . The multi-user simultaneous operation device 100 can prompt the audience to look at the tablet 120 by causing the tablet 120 to emit an alert sound.

Note that, in the present invention, the tablet 120 may have at least some of the functions of the multi-user simultaneous operation device 100 . As described above, the tablet 120 may have the functionality of the second screen viewpoint acquisition unit 202 and the second screen rendering unit 204 of the multi-user simultaneous control device 100 so that the tablet 120 renders. Alternatively, the tablet 120 may have the functions of the main screen viewpoint acquisition unit 201 and the main screen rendering unit 203 of the multi-user simultaneous operation device 100, and the tablet 120 may render. Further, the tablet 120 has the functions of a photographing unit 205, a photograph display unit 206, a video display unit 207, a drone display unit 208, a photographing information storage unit 209, and a photograph storage unit 210. It is also possible to execute processing for relaying and displaying a drone icon.

Thus, according to the present invention, a plurality of spectators can simultaneously experience VR while moving the positions and directions of their viewpoints in the virtual space.

Although the embodiments of the present invention have been described so far, the above-described embodiments are merely examples, and the present invention is not limited to the above-described embodiments, and can be implemented in various forms within the scope of the technical idea thereof. It goes without saying that it is good.

Moreover, the scope of the invention is not limited to the illustrated and described exemplary embodiments, but includes all embodiments that achieve equivalent effects for which the invention is intended. Furthermore, the scope of the invention is not limited to the combination of inventive features defined by each claim, but may be defined by any desired combination of the particular features of each of the disclosed features. .

100 multi-person simultaneous operation device 111 screen (main screen)
112 controller 120 tablet (second screen)
201 Main screen viewpoint acquisition unit 202 Second screen viewpoint acquisition unit 203 Main screen rendering unit 204 Second screen rendering unit 205 Photo shooting unit 206 Photo display unit 207 Video display unit 208 Drone display unit 209 Shooting information storage unit 210 Photo storage unit 301 Photo 302 Image 303 Drone icon

Claims (6)

information on at least one of the position and orientation of the first viewpoint in the virtual space from each first operating device used by a plurality of spectators introduced in the virtual space projected on the main screen and the second screen; a first viewpoint acquisition unit that acquires first information other than a viewpoint indicating a situation in the virtual space ;
Information on at least one of the position and orientation of a second viewpoint in the virtual space and the situation in the virtual space are indicated from a second operating device used by a person who introduces the virtual space to the plurality of spectators. a second viewpoint acquisition unit that acquires second information other than the viewpoint ;
an image rendered based on at least part of the information about the first viewpoint and the first information , based on at least part of the information about the second viewpoint and the second information and a display unit for superimposing a rendered image and displaying it on the main screen. receiving an instruction to take a picture in the virtual space from the first operation device;
storing information of at least one of the position and orientation of the first viewpoint in the virtual space and the first information when an instruction to take a picture in the virtual space is received;
further configured to store as a photograph a still image rendered based on at least a portion of the stored information about the first viewpoint and the first information ;
2. The apparatus of claim 1, wherein the image rendered based on at least part of the information about the first viewpoint and the first information is the photograph. further configured to place an icon in the virtual space based on at least part of the obtained information about the first viewpoint;
An image in the virtual space in which the icon is superimposed on an image rendered based on at least part of the information about the second viewpoint and the second information is displayed on the main screen. 3. Apparatus according to claim 1 or 2, characterized in that: A method performed by an apparatus comprising:
Information on at least one of the position and orientation of the second viewpoint in the virtual space for the second display device and the virtual a step of obtaining second information other than the viewpoint indicating the situation in the space ;
at least part of information of at least one of the position and orientation of a first viewpoint in the virtual space for each of the first display devices used by the plurality of spectators introduced in the virtual space and the virtual space; An image rendered based on first information other than a viewpoint indicating a situation in space is superimposed on an image rendered based on at least part of the information about the second viewpoint and the second information. and displaying on the second display device, the second display device comprising a single screen. A program for causing a computer to execute the method according to claim 4. A multi-person simultaneous operation system comprising a first device, a second device, a main screen, and a multi-person simultaneous operation device,
The multi-person simultaneous operation system includes:
Information on at least one of the position and orientation of the first viewpoint in the virtual space from each of the first devices used by the plurality of spectators introduced in the virtual space projected on the main screen and the second screen. and first information other than a viewpoint indicating a situation in the virtual space , the first device including the second screen,
Information on at least one of the position and orientation of a second viewpoint in the virtual space and the situation in the virtual space are displayed from the second device used by the person who introduces the virtual space to the plurality of spectators. Acquiring second information other than the viewpoint ,
an image rendered based on at least part of the information about the first viewpoint and the first information , based on at least part of the information about the second viewpoint and the second information A multi-person simultaneous operation system characterized by being constructed so as to be superimposed on a rendered image and displayed on the main screen.

Images