JP6427298B1 - Information processing system - Google Patents

Abstract

To provide a screen sharing technology that achieves high-speed processing while reducing the amount of data to be transmitted. The position angle specifying unit 121 specifies the standing position (coordinates in the map) of the moving object in the virtual space as the predetermined position based on the information such as the command of the moving operation transmitted from the controller 2. The position angle specifying unit 121 specifies, as a predetermined angle, an angle indicating the direction in which the movement target is facing in the virtual space, based on the various sensing information of the sensor unit 17. The image acquisition unit 104 acquires data of an image corresponding to a predetermined position and a predetermined angle from the image DB 600. The HMD display control unit 105 causes the display unit 16 to display an image showing a landscape viewed when the moving object such as a user stands at a predetermined position and faces in the direction of a predetermined angle in the virtual space. The communication control unit 103 transmits position information, angle information, and the like to the display terminal 3 to realize sharing of the displayed image.

Description

  The present invention relates to an information processing system.

In recent years, with the development of information technology, for example, a screen sharing technology for sharing a screen between a plurality of terminals such as a video conference and a large-scale symposium has attracted attention.
Also, for example, when men and women who are thinking of getting married choose a wedding hall in the wedding salon, instead of just looking at the brochure, they can experience the image of a real wedding hall or simulate it Receiving explanations from the salon staff is extremely effective for knowing the image of the actual wedding hall.
In this regard, a screen sharing technique for sharing a screen among a plurality of terminals has been proposed (see, for example, Patent Documents 1 and 2).
For example, according to the technique described in Patent Document 1, the data of the image of each frame is divided into a plurality of blocks, and only the block in which the data of the image is changed is shared with the other terminal. Furthermore, according to the technology described in Patent Document 2, the transmission control apparatus determines the block size in accordance with the processing capability of the terminal that receives the image data.

JP, 2005-269272, A Patent 2009-245254 gazette

However, in either case of the technique described in Patent Document 1 or the technique described in Patent Document 2, when there is a change in the data of the image, the terminal on the side that receives the block in which the content has changed Need to send.
Therefore, in either case of the technology described in Patent Document 1 or the technology described in Patent Document 2, the amount of data (image data) to be transmitted from the transmitting terminal to the receiving terminal is large. The

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a screen sharing technology for realizing faster processing while suppressing the amount of data to be transmitted.

In order to achieve the above object, the first terminal (the HMD of the client in the first and second embodiments, and the HMD on the female side in the third embodiment) and the second terminal (the first and second embodiments) In the information processing system including the staff terminal for the wedding salon in the third embodiment and the HMD on the male side in the third embodiment,
For each of a plurality of positions in a virtual space (for example, a virtual space corresponding to an actual wedding place), data groups of a plurality of images showing the state of the virtual space when the viewpoint is changed at that position are stored With one or more storage devices
The first terminal is
Specifying means for specifying a predetermined position of a predetermined movement target (such as a user or a character) and a predetermined angle with respect to the predetermined position in the virtual space;
First image acquisition means for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the specifying means When,
First communication control means for executing control to transmit the predetermined position and the predetermined angle specified by the specifying means to the second terminal at least once;
Equipped with
The second terminal is
A second communication control unit that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means Second image acquisition means for acquiring from at least one;
Equipped with

  An information processing method and program according to an aspect of the present invention correspond to the information processing system according to an aspect of the present invention.

  According to the present invention, it is possible to provide a screen sharing technology that achieves high-speed processing while reducing the amount of data to be transmitted.

It is a figure showing composition of an information processing system concerning one embodiment of the present invention. It is a block diagram which shows an example of the hardware constitutions of HMD among the information processing systems of FIG. It is a block diagram which shows an example of the hardware constitutions of a display terminal among the information processing systems of FIG. FIG. 5 is a functional block diagram showing an example of a functional configuration for realizing screen sharing processing among the functional configurations of the HMD of FIG. 2 and the display terminal of FIG. 3. It is a figure which shows an example of the map of virtual space utilized in HMD of FIG. 4, and a display terminal. It is a flowchart explaining the flow of the screen sharing process performed by HMD of FIG. It is a flowchart explaining the flow of the screen sharing process performed by the display terminal of FIG. FIG. 5 is a functional block diagram showing an example of a functional configuration for realizing screen sharing processing among the functional configurations of the HMD of FIG. 2 and the display terminal of FIG. 3 and an example different from the example of FIG. 4; It is a figure which is an example of a structure of the information processing system which concerns on one Embodiment of this invention, and shows an example different from the example of FIG.

First Embodiment
Hereinafter, a first embodiment of the present invention will be described using the drawings.
FIG. 1 is a diagram showing an outline of an information processing system according to a first embodiment of the present invention.

First, prior to describing an embodiment of the present invention, a service to which the information processing system according to an embodiment of the present invention is applied (hereinafter, referred to as "the service") will be briefly described.
That is, this service is, for example, a service provided to the client U at the wedding salon, and while the client U makes a pseudo experience of moving or the like in the selectable wedding hall, the staff S appropriately It is a service that can receive explanations.

Specifically, for example, by mounting the head mounted display (HMD) 1 on the head, the client U can see the scene viewed when the user U stands at a predetermined position in a desired wedding hall and faces in a predetermined direction. Can visually recognize an image such as a picture of the scene.
In addition, the said image is not restricted to the photograph actually imaged, The image produced | generated, such as CG image and AR image, may be included.
Moreover, in the following, when only calling it a "image", it shall contain both a "moving image" and a "still image."
Furthermore, when the customer U wants to move in the wedding hall, more precisely, he wants to move in the virtual space corresponding to the wedding hall, a predetermined operation on the controller 2 (hereinafter referred to as "moving operation") Can change its standing position.
Further, when the customer U wants to change the direction of the direction (direction of sight line), the direction can be changed by actually moving his or her head.
That is, by moving the head (HMD 1) of the client U while moving the controller 2, the client U looks in the wedding hall when viewed in the desired direction at the desired standing position. You can see the view.

  Here, the staff S sees the view (viewed by the consultant U) in order to appropriately explain the client U about the inside of the wedding hall where the consultant U experiences a simulated movement etc. It is necessary to grasp a picture of the landscape. Therefore, the display terminal 3 displays a view (photograph showing the view) that the customer U is viewing.

As described above, the information processing system of FIG. 1 according to an embodiment of the present invention includes the HMD 1 and the controller 2 used by the consultant U, and the display terminal 3 used by the staff S.
The HMD 1, the controller 2, and the display terminal 3 are mutually connected via a predetermined network N. The form of the network N is not particularly limited. For example, Bluetooth (registered trademark), Wi-Fi, LAN (Local Area Network), the Internet, or the like can be adopted.

  The information processing system of FIG. 1 having such a configuration is, as described above, an image viewed by the customer U (a scene in the wedding hall where the customer U is experiencing a simulated experience such as movement etc. A photograph can be shared between the HMD 1 and the display terminal 3.

  Here, as a method of sharing an image, a method may be adopted in which the image is shared by transmitting data of the image viewed by the customer U from the HMD 1 to the display terminal 3, but the method In this case, the amount of data to be transmitted increases, making high-speed processing difficult. For this reason, adoption of HMD1 may become difficult.

Therefore, in the present embodiment, the following method is adopted as a method of sharing an image.
That is, although not shown in FIG. 1, for each of a plurality of positions in the virtual space corresponding to an actual wedding place, a plurality of images showing the state of each of the virtual spaces when the viewpoint is changed at that position. It is assumed that there is one or more storage devices in which data groups are stored.
Then, it is premised that each of the HMD 1 and the display terminal 3 can acquire image data from at least one of these one or more storage devices. Although details will be described later, the storage unit 18 (image DB 600) in the HMD 1 of FIG. 4 and the storage unit 49 (image DB 900) of the display terminal 3 of FIG. There is.
Under such a premise, the HMD 1 specifies the standing position and the angle of the client U in the virtual space.
Here, the angle is an angle representing the direction of a vector indicating the viewpoint of the customer U, which starts from the standing position of the customer U.
In the present embodiment, the angle is acquired by an angle sensor incorporated in the HMD 1, various sensors such as a viewpoint sensor (for example, a sensor unit 17 described later). Therefore, in the present embodiment, the posture of the head of the user specified by the angle sensor and the direction (viewpoint direction) of the line of sight of the user specified by the viewpoint sensor are “angles”.
The relationship between various image data stored in the image DB 600 and the image DB 900 and the “angle” will be described in detail with reference to FIG.
Based on the standing position and angle specified in this manner, the HMD 1 generates image data to be displayed on the HMD 1 side from at least one of the one or more storage devices described above (image DB 600 in FIG. 4 described later) get.
Further, the HMD 1 transmits, to the display terminal 3, information that can specify the standing position and angle thus specified. The display terminal 3 specifies the standing position and the angle based on the information transmitted from the HMD 1. Then, the display terminal 3 performs at least one of the one or more storage devices described above on the data of the image to be displayed on the display terminal 3 side based on the standing position and the angle (image DB 900 in FIG. 4 described later). Get from

In this embodiment, such a method is adopted as a method of sharing an image. Therefore, the amount of data to be transmitted to the display terminal 3 to be shared becomes small, and high-speed processing of the entire information processing system becomes possible. Thereby, adoption of HMD1 becomes possible easily.
That is, since the provider of this service can easily adopt the HMD 1, the client U can experience a more realistic image of the wedding hall.
Such an image sharing method is particularly suitable for image sharing in VR (Virtual Reality) display in the HMD 1 and AR (Augmented Reality) display.

  FIG. 2 is a block diagram showing an example of the hardware configuration of the HMD in the information processing system of FIG.

  The HMD 1 includes a central processing unit (CPU) 11, a read only memory (ROM) 12, a random access memory (RAM) 13, a bus 14, an input / output interface 15, a display unit 16, and a sensor unit 17. A storage unit 18, a communication unit 19, and a drive 20 are provided.

The CPU 11 executes various processes in accordance with a program stored in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13.
The RAM 13 appropriately stores information necessary for the CPU 11 to execute various processes.

  The CPU 11, the ROM 12 and the RAM 13 are connected to one another via a bus 14. An input / output interface 15 is also connected to the bus 14. The display unit 16, the sensor unit 17, the storage unit 18, the communication unit 19, and the drive 20 are connected to the input / output interface 15.

The display unit 16 is composed of various liquid crystal displays and the like, and displays various images such as photographs in the wedding hall.
The sensor unit 17 includes various sensors such as an inclination sensor, an acceleration sensor, and an angular velocity sensor, and acquires, for example, various detection information regarding an angle (direction) in the virtual space.
The storage unit 18 is a group of data of a plurality of images showing the state of the virtual space when changing the viewpoint at each position in the virtual space corresponding to the actual wedding place. Store information
The communication unit 19 communicates with another device (for example, the display terminal 3 in FIG. 1) via the network N, for example.

The controller 2 is also connected to the input / output interface 15.
The controller 2 is an input device that receives an instruction operation (for example, a move operation) by a user (such as the consultant U in FIG. 1) when the CPU 11 of the HMD 1 executes various processing. Connected Thus, for example, the CPU 11 receives an input from the controller 2 when a move operation is performed by a user (such as the consultant U in FIG. 1), and acquires the standing position in the virtual space that the user desires to share. be able to.

  The drive 20 is provided as needed. The drive 20 is made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, and a removable medium 21 is appropriately mounted. The program read from the removable media 21 by the drive 20 is installed in the storage unit 18 as necessary. The removable media 21 can also store various information stored in the storage unit 18 in the same manner as the storage unit 18.

  FIG. 3 is a block diagram showing an example of the hardware configuration of the display terminal 3 in the information processing system of FIG.

The display terminal 3 is configured of a tablet terminal or the like.
The display terminal 3 includes a CPU 41, a ROM 42, a RAM 43, a bus 44, an input / output interface 45, a touch operation input unit 46, a display unit 47, an input unit 48, a storage unit 49, and a communication unit 50. , And a drive 51.

The CPU 41 executes various processes in accordance with a program stored in the ROM 42 or a program loaded from the storage unit 49 into the RAM 43.
The RAM 43 appropriately stores information and the like necessary for the CPU 41 to execute various processes.

  The CPU 41, the ROM 42 and the RAM 43 are mutually connected via a bus 44. An input / output interface 45 is also connected to the bus 44. A touch operation input unit 46, a display unit 47, an input unit 48, a storage unit 49, a communication unit 50, and a drive 51 are connected to the input / output interface 45.

The touch operation input unit 46 is formed of, for example, a capacitive or resistive (pressure-sensitive) position input sensor stacked on the display unit 47, and detects the coordinates of the position at which the touch operation is performed.
The display unit 27 is composed of various liquid crystal displays and the like, and in the present embodiment, displays the same image as the image displayed on the display unit 16 (FIG. 2) of the HMD 1 such as various images in the wedding hall. .
As described above, in the present embodiment, the touch operation input unit 46 and the display unit 47 constitute a touch panel.
The storage unit 49 is configured by a hard disk or the like, and a plurality of images showing the state of each virtual space when the viewpoint is changed at each position in the virtual space corresponding to the actual wedding place. It stores various information such as data groups of That is, in the present embodiment, the storage unit 18 (FIG. 2) of the HMD 1 and the storage unit 49 of the display terminal 3 change the viewpoint at each of a plurality of positions in the virtual space corresponding to the actual wedding place. Data groups of a plurality of images showing the state of the respective virtual spaces in each case are shared.
The communication unit 50 communicates with another device (for example, the HMD 1 in FIG. 1) via the network N.

  The drive 51 is provided as needed. The drive 51 is made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, and a removable medium 52 is appropriately mounted. The program read from the removable medium 52 by the drive 51 is installed in the storage unit 49 as necessary. Further, the removable media 52 can store various information stored in the storage unit 49 in the same manner as the storage unit 49.

The cooperation of the various hardware and software of the HMD 1 and the display terminal 3 makes it possible to execute the screen sharing process.
The screen sharing process is data of one or more images (for example, images of the virtual space when the viewpoint is changed at each position at a plurality of positions in the virtual space corresponding to an actual wedding place) Assuming that there is one or more storage devices in which data of one omnidirectional image or data group of a plurality of images is stored, between a plurality of hardware (for example, HMD 1 and display terminal 3), By sharing position information (standing position) about a predetermined moving object and angle information based on the predetermined position, the specified position of the predetermined moving object and the angle from the position are specified, A series of processes to share an image to be displayed on hardware.
In the first embodiment, the position information and the angle information are transmitted from the HMD 1 to the display terminal 3, but the present invention is not particularly limited thereto.
That is, the HMD 1 and the display terminal 3 have the functional configuration as shown in FIG. 4 when executing the screen sharing process.
FIG. 4 is a functional block diagram showing an example of a functional configuration for realizing screen sharing processing among the functional configurations of the HMD 1 of FIG. 2 and the display terminal 3 of FIG. 3.

As shown in FIG. 4, in the CPU 11 of the HMD 1, the map management unit 101, the icon unit 102, the communication control unit 103, the image acquisition unit 104, and the HMD display control unit 105 function.
A map DB 500 and an image DB 600 are provided in an area of the storage unit 18 of the HMD 1.

The map management unit 101 of the HMD 1 uses the map of the virtual space (for example, the virtual space of the wedding hall in FIG. 5) presented as an image to the user such as the consultant U in FIG. Manage the standing position (coordinates in the map) and the like of the user or character of the customer U or the like in 1) in the virtual space.
Map information (information capable of specifying each position in the virtual space) is stored in the map DB 500, and is used by the map management unit 101. Map information will be described later with reference to FIG.

The map management unit 101 is provided with a position angle identification unit 121.
The position angle specifying unit 121 specifies a predetermined position (standing position) of a predetermined moving target in a virtual space corresponding to a wedding hall and a predetermined angle based on the predetermined position.
That is, the position angle specifying unit 121 specifies the standing position (coordinates in the map) of the movement target in the virtual space based on the information such as the movement operation command transmitted from the controller 2. The position angle specifying unit 121 outputs information (hereinafter, referred to as “position information”) indicating the standing position in the virtual space of the movement target, which is specified in this manner.
Further, based on the various sensing information of the sensor unit 17, the position angle specifying unit 121 specifies an angle indicating the direction in which the movement target in the virtual space is facing. The position angle specifying unit 121 outputs information (hereinafter referred to as “angle information”) indicating the angle in the virtual space specified in this manner.

The icon unit 102 manages icons displayed superimposed on an image showing how each direction is viewed from each position in the map managed by the map management unit 101.
Here, an icon means what is displayed superimposed on an image. For example, a symbol such as an arrow (symbol or character) is an example of an icon, and a message such as "please turn right" is also an example of an icon (which does not actually exist but exists only in the virtual space) ) Is also an example of the icon.

The image acquisition unit 104 acquires, from the image DB 600, data of an image to be displayed on the HMD 1 side based on the predetermined position and the predetermined angle specified by the position angle specification unit 121, that is, based on the position information and angle information described above. Do.
The HMD display control unit 105 executes control to cause the display unit 16 to display an image acquired as data by the image acquisition unit 104. That is, in the virtual space, the moving object such as the user stands at the standing position specified by the position information described above, and the image showing the scenery seen when the user looks in the direction of the angle specified by the angle information described above is displayed It is displayed on the part 16.
Furthermore, when the icon or information capable of specifying the icon is provided from the icon unit 102, the HMD display control unit 105 performs control of causing the display unit 16 to display the icon so as to be superimposed on the image to be displayed.

The communication control unit 103 transmits the information capable of specifying the predetermined position and the predetermined angle specified by the position angle specifying unit 121, that is, the position information and the angle information described above to the display terminal 3 via the communication unit 19 at least once. Execute control to send.
Although details will be described later, even on the display terminal 3 side, in the image displayed on the HMD 1 side, that is, in the virtual space, the moving object such as the user stands at the standing position specified by the position information described above It is possible to share the display of the image showing the landscape viewed when the user is facing the direction of the angle specified by the angle information.
Here, the reason for transmitting position information and angle information at least once may be transmitting position information and angle information each time, but transmitting position information and angle information once, and thereafter, Information indicating the relative position to the predetermined position specified by the position information (that is, difference information and hereinafter referred to as "relative position information") and information indicating the relative angle to the relative angle specified by the angle information (that is, difference information This is because, by transmitting “relative angle information” hereinafter, the display terminal 3 can specify the standing position and the angle of the moving object in the virtual space. As described above, by transmitting the relative position information and the relative angle information, it is possible to reduce the amount of data at the time of transmission as compared with the case where the position information and the angle information are transmitted each time.
However, if the relative position and the relative angle are transmitted many times, the error becomes large, so actually, when an appropriate condition is satisfied, the position information and the angle information are transmitted to the display terminal 3 You should do it.
Here, the appropriate conditions are not particularly limited, and for example, conditions based on the detection result of the sensor unit 17 of the HMD 1, specifically, for example, the following conditions can be adopted.
That is, for example, when the position information or the angle information is transmitted, when the acceleration or the angular acceleration is not zero or when the speed or the angular velocity is not zero can be adopted, respectively.
Also, for example, when the speed is 0, the position information and the relative position information may not be transmitted to the display terminal 3. Similarly, when the angular velocity is zero, the angle information and the relative angle information may not be transmitted to the display terminal 3.

With respect to such a functional configuration as the HMD 1, in the CPU 41 of the display terminal 3, the communication control unit 201, the map management unit 202, the icon unit 203, the image acquisition unit 204, and the display display control unit 205. And will work.
A map DB 800 and an image DB 900 are provided in an area of the storage unit 49 of the display terminal 3.

When the communication control unit 201 receives information that can specify the predetermined position and the predetermined angle, that is, the above-described position information and angle information, relative position information, or relative angle information that has been transmitted from the HMD 1, the communication control unit 201 can To get through.
Further, when an icon or information that can specify the icon is transmitted from the HMD 1, the communication control unit 201 acquires this via the communication unit 50.

The map management unit 202 uses the same map as the HMD 1 side, that is, the map of a virtual space (for example, the virtual space of the wedding hall in FIG. 5) presented and displayed as an image to the user such as the consultant U in FIG. And managing a standing position (coordinates in the map) of the predetermined movement target (a user or character of the consultant U or the like in FIG. 1) or the like in the virtual space.
Map information (information capable of specifying each position in the virtual space) is stored in the map DB 800 and used by the map management unit 202. Map information will be described later with reference to FIG.

The map management unit 202 is provided with a position angle identification unit 211.
The position angle specifying unit 211 specifies a predetermined position (standing position) of a predetermined moving target in a virtual space corresponding to a wedding hall and a predetermined angle based on the predetermined position.
That is, based on the information transmitted from the HMD 1 and acquired by the communication control unit 201 via the communication unit 50, the position angle specifying unit 211, based on the above-described position information and angle information or relative position information or relative angle information, The standing position (coordinates in the map) of the moving object in the virtual space and the angle indicating the direction in which the moving object in the virtual space is directed are specified.

  The icon unit 203 transmits, from the HMD 1, an icon to be displayed superimposed on an image showing a state in which each direction is viewed from each position in the map managed by the map managing unit 202, or information that can specify the icon. When acquired by the communication control unit 201 via the communication unit 50, the icon or information capable of specifying the icon is acquired and managed.

The image acquisition unit 204 generates image data to be displayed on the display terminal 3 side, that is, data of an image to be shared with the HMD 1 side from the image DB 900 based on the predetermined position and the predetermined angle specified by the position angle specification unit 211. get.
The display display control unit 205 controls the display unit 47 to display an image acquired as data by the image acquisition unit 204. That is, in the same image as the image displayed on the HMD 1 side, specifically, for example, in the virtual space, the moving object such as the user stands at the above-mentioned predetermined position (standing position) and moves in the direction of the above-mentioned predetermined angle. An image showing the scenery seen when the user turns is displayed on the display unit 47.
Furthermore, when the icon or the information that can specify the icon is managed in the icon unit 203, the display display control unit 205 performs control of causing the display unit 47 to display the icon by superimposing the icon on the image to be displayed. .

Next, details of information shared and stored by the HMD 1 and the display terminal 3 will be described with reference to FIG.
FIG. 5 is a diagram showing an example of a map of a virtual space used in the HMD 1 and the display terminal 3 of FIG.

In the example of FIG. 5, a wedding hall is employed as an example of the virtual space. That is, the virtual space is a virtual space imitating a wedding hall which staff members of the wedding hall (for example, the staff S in FIG. 1) introduce to a client (for example, the client U in FIG. 1). A client (for example, the client U in FIG. 1) can appropriately receive explanations from the staff of the wedding hall while simulating experiencing moving within the wedding hall by wearing the HMD 1 and operating the controller 2 Can.
Here, according to the map of FIG. 5, four positions “P1”, “P2”, “P3” and “P4” are provided as positions to which the movement object such as a consultant can move within the virtual space. Position is set. That is, the map management unit 101 and the map management unit 202 manage the coordinates on the map respectively indicating the four positions “P1”, “P2”, “P3”, and “P4”.
Then, in the image DB 600 of the HMD 1 and the image DB 900 of the display terminal 3, when the viewpoint is changed at each position of “P1”, “P2”, “P3”, and “P4”, respectively. A plurality of image data groups indicating the state of the virtual space are stored.

More specifically, in the image DB 600 and the image DB 900, the client (for example, the client U in FIG. 1) is placed at each position of "P1", "P2", "P3" and "P4" at the wedding hall. A plurality of image data groups are stored which show the situation when the 360-degree viewpoint is changed.
Note that data groups of these images are stored in the image DB 600 and the image DB 900 in association with the above-described position (for example, “P1”). Then, at each position (for example, “P1”), it is set for each angle whether the image corresponding to which data of the data group of the image is displayed (acquired by the image acquisition unit 204) . That is, the data group of the image is associated with the angle.

Here, the correspondence between the image data group and the angle will be described. As described above, the angle is an angle representing the direction of a vector indicating the viewpoint of the moving object, starting from the standing position of the moving object (for example, the customer U in FIG. 1) in the virtual space.
As an angle used for correlating the image data stored in the image DB 600 and the image DB 900, for example, at a predetermined position (for example, "P1") at a predetermined height (for example, 170 cm) from the moving object It is assumed that there is a starting point of a vector representing a viewpoint (hereinafter referred to as “viewpoint vector”), and the viewpoint vector at the time of reference posture of moving object is taken as a reference vector. An angle is adopted. Further, data of an image seen by the moving object is set for each posture of the moving object. In this way, the angle and the data of the image are associated.
Such data groups of images associated with respective positions and angles are present in advance in HMD 1 and display terminal 3 (image DB 600 of HMD 1 and image DB 900 of display terminal 3 in the example of FIG. 4) It is assumed that it is memorized in. Under such a premise, sharing of images between the HMD 1 and the display terminal 3 is realized only by sharing position information and angle information of the HMD 1 and the display terminal 3.

Next, with reference to FIG. 6, the screen sharing process executed by the HMD 1 of FIG. 4 will be described.
That is, FIG. 6 is a flowchart for explaining the flow of screen sharing processing on the HMD 1 side of FIG.

In step S1, the position angle identification unit 121 identifies a predetermined position of a predetermined movement target (a user such as the consultant U in FIG. 1, a character or the like) in the virtual space corresponding to the wedding hall.
That is, the position angle specifying unit 121 specifies the standing position (coordinates in the map) of the movement target in the virtual space as the predetermined position based on the information such as the movement operation command transmitted from the controller 2. In addition, the position angle identification unit 121 outputs position information indicating the predetermined position identified in this manner.

In step S2, the position angle specifying unit 121 specifies a predetermined angle based on the predetermined position in the virtual space corresponding to the wedding hall specified in step S2.
That is, based on the various sensing information of the sensor unit 17, the position angle specifying unit 121 specifies an angle indicating the direction in which the movement target in the virtual space is facing as a predetermined angle. The position angle identification unit 121 outputs angle information indicating the predetermined angle thus identified.

  In step S3, the image acquisition unit 104 acquires, from the image DB 600, data of an image corresponding to the predetermined position identified in step S1 and the predetermined angle identified in step S2.

In step S4, the HMD display control unit 105 causes the display unit 16 to display the image acquired as data in step S3.
That is, in the virtual space, the display unit 16 displays an image indicating a landscape viewed when the moving object such as the user stands at the above-described predetermined position and faces in the direction of the above-described predetermined angle.
Note that the HMD display control unit 105 causes the display unit 16 to display an icon so as to be superimposed on the image to be displayed, as necessary.

In step S5, the communication control unit 103 can identify the predetermined position identified in step S1 and the predetermined angle identified in step S2, that is, position information and angle information or relative position information and relative angle information (hereinafter referred to as The “position information, angle information, etc.” is transmitted to the display terminal 3 via the communication unit 19.
Among the position information and the angle information, the position information or the relative position information is hereinafter referred to as "position information or the like", and the angle information or the relative angle information is hereinafter referred to as "the angle information or the like".

In step S6, the CPU 11 of the HMD 1 determines whether an instruction to end the process has been issued.
Here, the end instruction of the process is not particularly limited, but in this embodiment, an instruction to shift the HMD 1 to a so-called sleep state or the like is adopted. That is, unless an instruction to shift to the sleep state or the like is issued in the HMD 1, NO is determined in step S 6, the process is returned to step S 1, and the subsequent processes are repeated.
On the other hand, when an instruction to shift to the sleep state or the like is issued in the HMD 1, YES is determined in step S6, and the screen sharing process ends.

Next, with reference to FIG. 7, the screen sharing process performed by the display terminal 3 will be described.
That is, FIG. 7 is a flow chart for explaining the flow of the screen sharing process on the side of the display terminal 3 of FIG.
In the present embodiment, information is delivered in one direction from the HMD 1 to the display terminal 3. In the screen sharing process shown in FIG. 7, the display terminal 3 acquires position information, angle information, etc. transmitted from the HMD 1 in the screen sharing process shown in FIG. 6, and based on the position information, the angle information, etc. Do sharing.

  In step S21, the communication control unit 201 of the display terminal 3 acquires the position information, the angle information, and the like transmitted from the HMD 1 via the communication unit 50.

  In step S22, the position angle specifying unit 211 specifies the standing position (coordinates in the map) of the moving object in the virtual space as the predetermined position based on the position information and the like acquired in step S21.

  In step S23, based on the angle information and the like acquired in step S21, the position angle specifying unit 211 specifies an angle indicating the direction in which the moving object in the virtual space is facing as a predetermined angle.

  In step S24, the image acquisition unit 204 acquires, from the image DB 900, data of an image corresponding to the predetermined position identified in step S22 and the predetermined angle identified in step S23.

In step S25, the display display control unit 205 causes the display unit 47 to display the image acquired as data in step S24.
That is, a moving object such as a user who uses the HMD 1 stands at the above-described predetermined position, and an image showing a landscape viewed when facing in the direction of the above-described predetermined angle, in other words, an image displayed on the HMD 1 side , And displayed on the display unit 47 of the display terminal 3.
In addition, the display control unit 205 causes the display unit 47 to display an icon so as to be superimposed on the image to be displayed, as necessary.

In step S26, the CPU 41 of the display terminal 3 determines whether an instruction to end the process has been issued. Here, the process end instruction is not particularly limited, but in the present embodiment, an instruction to shift the display terminal 3 to a so-called sleep state or the like is employed. That is, unless an instruction to shift to the sleep state or the like is made in the display terminal 3, the determination in step S26 is NO, the process is returned to step S21, and the subsequent processes are repeated.
On the other hand, when an instruction to shift to the sleep state or the like is issued in the display terminal 3, it is determined as YES in step S26, and the screen sharing process ends.

Second Embodiment
In the first embodiment described above, image sharing is realized by transmitting position information, angle information, and the like from the HMD 1 to the display terminal 3.
However, in order to realize sharing of an image, it is not necessary to transmit position information, angle information, and the like from the HMD 1 to the display terminal 3. For example, it is also possible to transmit position information and the like from the display terminal 3 to the HMD 1. That is, the second embodiment may be employed in which the HMD 1 and the display terminal 3 transmit and receive position information and the like in both directions.
By adopting this second embodiment, the provider of this service can provide a wider range of screen sharing services.

  Here, the system configuration of the second embodiment is the same as the system configuration of the first embodiment of FIG. That is, the information processing system of the second embodiment also has the HMD 1, the controller 2, and the display terminal 3. The hardware configuration of the HMD 1 and the display terminal 3 in the second embodiment is the same as that of the first embodiment shown in FIGS. 2 and 3 respectively.

  FIG. 8 is an example of a functional configuration for realizing screen sharing processing among the functional configurations of the HMD 1 of FIG. 2 and the display terminal 3 of FIG. 3, and a function showing an example different from the example of FIG. It is a block diagram.

As shown in FIG. 8, in the CPU 11 of the HMD 1, the map management unit 101, the icon unit 102, the communication control unit 103, the image acquisition unit 104, and the HMD display control unit 105 function.
A map DB 500 and an image DB 600 are provided in an area of the storage unit 18 of the HMD 1.
Furthermore, in the CPU 41 of the display terminal 3, the communication control unit 201, the map management unit 202, the icon unit 203, the image acquisition unit 204, and the display display control unit 205 function.
A map DB 800 and an image DB 900 are provided in an area of the storage unit 49 of the display terminal 3.
The various functional configurations other than the communication control unit 103 and the communication control unit 201 are the same as in the first embodiment. Therefore, only the communication control unit 103 and the communication control unit 201 will be described here, and the description of the other functional blocks will be omitted here.

The communication control unit 103 of the HMD 1 executes control to transmit position information, angle information, and the like to the display terminal 3 as in the first embodiment, and in the second embodiment, it is transmitted from the display terminal 3 side. Control is also performed to receive the received position information and the like.
Similarly, the communication control unit 201 of the display terminal 3 executes control to receive position information, angle information, and the like transmitted from the HMD 1 in the same manner as in the first embodiment, and further to the HMD 1 in the second embodiment. Also, control for transmitting position information etc. is executed.

Here, instead of transmitting position information and angle information to the HMD 1 from the display terminal 3, transmission of only position information and the like is permitted and transmission of angle information is prohibited.
This is because when the display of the image on the HMD 1 side is controlled based on angle information from the display terminal 3, the display image changes even though the user of the HMD 1 (consultant etc.) does not move his head. Or, the display image may change in a direction different from the direction in which one's head is moved, and the user may get drunk.
That is, when there is a possibility that the user of the HMD 1 may get drunk, only the position information from the display terminal 3 is shared on the HMD 1 side and the angle information is not shared in order to reduce the drunkenness. In other words, position information is shared in both directions between the display terminal 3 and the HMD 1, and angle information is shared only in one direction from the HMD 1 to the display terminal 3.

In addition, sharing of angle information from the display terminal 3 to the HMD 1 side may be performed if the wearer of the HMD 1 is in a state of being intoxicated. Here, the state where the user of the HMD 1 is difficult to get drunk means that the head of the wearer of the HMD 1 is not moving but the display image changes (a display image moving in a predetermined direction is sequentially displayed), or The angle (screen angle) of the display image on the display terminal 3 is in a moving state.
Therefore, if it is determined that the head angle of the user of the HMD 1 does not move based on the detection result of the sensor unit 17 of the HMD 1 or if the angle of the display image on the display terminal 3 side does not move When it is determined, it is also possible to transmit position information, angle information, and the like from the display terminal 3 to the HMD 1 (sharing of angle information).

As described above, in the second embodiment, the HMD 1 and the display terminal 3 have a functional configuration capable of mutually transmitting and receiving position information, angle information, and the like. However, in the communication from the display terminal 3 to the HMD 1, transmission of angle information is basically prohibited in order to prevent a sickness of the wearer of the HMD 1.
Thus, in the image displayed on either the HMD 1 side or the display terminal 3 side, that is, in the virtual space, the moving object such as the user stands at the standing position specified by the position information etc. It is possible to share the display of an image showing the landscape seen when facing in the direction of the specified angle.
Third Embodiment
In the first embodiment described above, sharing of an image is realized by transmitting position information, angle information, and the like in one direction from the HMD 1 to the display terminal 3. Furthermore, in the second embodiment described above, the HMD 1 and the display terminal 3 realize sharing of images by transmitting and receiving position information, angle information, and the like in both directions.
However, such sharing of images does not necessarily have to be performed between only one HMD 1 and one display terminal 3.
That is, in the third embodiment, between P (H is an arbitrary integer of 1 or more) HMD 1 and Q (Q is an arbitrary integer of 1 or more independent of P) display terminals 3 Image sharing is performed.

For example, in the example of FIG. 9, HMD1-M used by a predetermined client M (for example, a man who will be a groom) and HMD1-M used by other clients W (for example, a woman who will be a bride). The image sharing is performed between W and the display terminal 3.
FIG. 9 is an example of the configuration of an information processing system according to an embodiment of the present invention, and is a diagram showing an example different from the example of FIG.

The information processing system of FIG. 9 according to an embodiment of the present invention includes an HMD 1-M and a controller 2-M used by a client M, an HMD 1-W and a controller 2-W used by a client W, The display terminal 3 used for the staff S is provided.
As in FIG. 1, each of the HMD 1 -M, the controller 2 -M, the HMD 1 -W, the controller 2 -W, and the display terminal 3 is mutually connected via a predetermined network N.

  The information processing system of FIG. 9 having such a configuration is an image described by the staff S of the wedding salon (an image showing a scene in a wedding hall where the consultants M and W are experiencing a simulated experience such as movement) Can be shared between the HMD 1 -M, the HMD 1 -W, and the display terminal 3.

Here, the image sharing between the HMD 1-M and the display terminal 3 and the image sharing between the HMD 1-W and the display terminal 3 are the same as in the second embodiment, so the description thereof is omitted here. Do.
When sharing an image between the HMD 1 -M and the HMD 1 -W, the image can also be shared by transmitting position information and angle information and the like from one to the other. However, since the HMD 1 is usually worn directly on the head, if the display image changes rapidly, the user of the HMD 1 (for example, the consultant M in FIG. 9) may get drunk. That is, when the display image changes in the HMD1-W by moving the head of the client W, the head of the client M does not move the head if the change in the display image is shared on the HMD1-M side as well. Nevertheless, the displayed image may change rapidly and may get drunk.
Therefore, sharing of angle information and the like may be prohibited between HMD1-M and HMD1-W, and only position information and the like may be shared in order to prevent drunkenness of the client M.

In summary, in the first and second embodiments, one HMD 1 functions as a first terminal described below, and one display terminal 3 functions as a second terminal described below.
That is, for each of a plurality of positions in a virtual space (for example, a virtual space corresponding to an actual wedding hall), data of one or more images showing the state of the virtual space when changing the viewpoint at that position is An embodiment according to an information processing system including a first terminal and a second terminal capable of acquiring image data from at least one of one or more stored storage devices is the first embodiment and second embodiment. It is each of embodiment.

  On the other hand, in the third embodiment (for example, in the example of FIG. 9), the HMD1-W of the client W functions as the first terminal, and the HMD1-M of the client M functions as the second terminal. In this case, although not shown, the first terminal and the second terminal have the same functional configuration as the HMD 1 of FIG.

  Here, in the third embodiment, sharing of position information and the like is essential for sharing images between the first terminal and the second terminal, but sharing of angle information and the like may be arbitrary in order to prevent sickness.

For example, the communication control unit 103 of the first terminal permits transmission to the second terminal at least once for information that can specify the predetermined position specified by the position angle specifying unit 121, that is, position information and the like. . On the other hand, the communication control unit 103 of the first terminal executes control to prohibit transmission to the second terminal for information that can specify the predetermined angle specified by the position angle specifying unit 121, that is, angle information and the like. .
This prevents the user of the second terminal (the consultant M in the example of FIG. 9) from changing the display image and getting drunk even though the user's head is not moved.

Alternatively, for example, the communication control unit 103 of the first terminal in the third embodiment satisfies the predetermined condition for information capable of specifying the predetermined angle specified by the position angle specifying unit 121, that is, angle information, etc. In such a case, control may be performed to permit transmission to the second terminal, and to prohibit transmission to the second terminal in other cases.
That is, between the first terminal and the second terminal, the angle is shared but not continuously but intermittently. Also in this case, it is prevented that the user of the second terminal (in the example of FIG. 9, the client M) does not move his or her head and the displayed image changes to get drunk.

  Here, if the angle of the second terminal does not move, that is, if the head of the user (the client M in the example of FIG. 9) to which the second terminal is attached does not move, the display image due to the angle change Even if it changes, the user is not drunk. Therefore, as the above-described predetermined condition for permitting transmission of angle information and the like to the second terminal, a condition determined that the second terminal is not moving may be adopted. That is, the angle will not be shared only when the second terminal (HMD1-M in the example of FIG. 9) on the side where the information of the first terminal (HMD1-W in the example of FIG. 9) is shared is moving. .

  The various techniques that do not share the angle for the prevention of sickness described above have been described by taking the case where both the first terminal and the second terminal in the third embodiment are both HMD 1 as an example, but the application destination is limited to this As described above, the present invention is also applicable to the case where the first terminal in the second embodiment is the HMD 1 and the second terminal is the display terminal 3 and position information etc. is transmitted from the display terminal 3 to the HMD 1 It is.

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and any modification, improvement, etc. within the range in which the object of the present invention can be achieved is included in the present invention. is there.

  For example, in the above-mentioned embodiment, although an example of a wedding salon is mentioned as an example and explained as this service, it is not limited in particular to this. That is, the information processing system according to an embodiment of the present invention is applied to any scene to which a technology for sharing an image viewed by a movable object movable in a virtual space, such as a video conference or a large symposium, is applicable. It is possible.

Further, for example, in the third embodiment, two HMDs (the first terminal and the second terminal) do not share angle information at least partially for the purpose of preventing sickness.
The method of sharing the image between the two HMDs in this case is not particularly limited. For example, for the user of the second terminal (for example, the client M in FIG. 9), the user of the first terminal For example, a method may be adopted in which the client W) in FIG. 9 makes a voice and urges the client M to turn in the direction of the angle.
Further, for example, a method may be adopted in which an icon or the like indicating an angle desired to be directed to the user is displayed on the second terminal side.
Furthermore, for example, in order to share the image displayed on the first terminal side on the second terminal side, if the user turns in the wrong direction, the second terminal is shaken or a direction in which the user of the second terminal does not want to go. Various guidance such as displaying in dark or displaying the direction that the user of the second terminal wants to go to when the user of the second terminal moves to the next place (for example, the position P2 next to the current position P1 in the example of FIG. 5) It is also possible to adopt a method of setting functions.
Further, regarding such various guidance functions, the user of the second terminal (for example, the customer M in the example of FIG. 9) displays the image displayed on the first terminal side according to the instruction of the guidance function (image to be shared It is also possible to provide a function to feed back whether or not it is facing the direction of. Thereby, the other person (in the example of FIG. 9, the consultant W or the staff S) recognizes whether the user of the second terminal (in the example of FIG. 9, the consultant M) firmly follows the instruction of the guidance function. It is easy to do.

Also, for example, if the provider of this service can transmit and receive bidirectionally position information and angle information, etc., to share an image between the first terminal and the second terminal, the first terminal and the second terminal can be used. The priority of which of the display images is shared can also be set.
For example, when a male (in the example of FIG. 9, the client M) and a woman (in the example of FIG. 9, the client W) instruct to change the display image simultaneously, priority may be given to the instruction on the female side. Also, for example, when a plurality of persons instruct to change the display image at the same time, the controller or the like may be provided with a button for becoming the first priority. That is, the instruction of the person who pressed the button is adopted as the first priority, and the display image to be shared by a plurality of people changes.

Further, for example, in the above embodiment, the storage device (for example, the storage unit 18 in FIG. 2) for storing data of various images has been described as being built in the hardware of the HMD or the display terminal. It is not limited to this.
That is, a storage device for storing data of various images may be provided separately from the hardware of the HMD or the display terminal.

  Further, for example, in the above-described embodiment, the provider of this service freely sets the intervals at which the display status of the position information, the angle information, etc. and the icons are transmitted between various hardware (for example, the HMD 1 to the display terminal 3). be able to. For example, information may be exchanged about 30 times per second, or it is also possible to use relative information (difference information) as described above.

Further, for example, in the above-described embodiment, although the image is shared between the first terminal and the second terminal, the term “image sharing” as used herein means that the subject (object) shares the same image. It is. That is, it is sufficient that the user of the first terminal and the user of the second terminal can visually recognize the same scenery in the virtual space.
That is, for example, the resolutions of the image displayed on the first terminal and the image displayed on the second terminal may be different.
Also, for example, the icon displayed superimposed on the image in the first terminal may be different from the icon displayed superimposed on the image in the second terminal. In general, the amount of information is larger in the case where the staff who explains to the client visually recognizes that the client who moves in the virtual space (the user of the first terminal) sees, that is, the icon etc. It is suitable if there are many.
In summary, the first terminal executes control to cause the display device on the first terminal side to display the image based on the data of the image of the first data amount, while the second terminal determines that the first data amount is the first data amount and Alternatively, based on the data of the image of the second independent data amount, control may be performed to display the image on the display device on the second terminal side.

Also, for example, in the above-described embodiment, the icon is superimposed and displayed on the image shared between the first terminal and the second terminal, but what is displayed together with the image is particularly limited to the icon Also, any symbol can be adopted.
Furthermore, in order to share the display of the symbol such as the icon between the first terminal and the second terminal, in the above embodiment, the symbol or the information capable of specifying the symbol is unidirectionally communicated from the first terminal to the second terminal However, the present invention is not particularly limited thereto. One-way communication from the second terminal to the first terminal may be used, or two-way communication between the first terminal and the second terminal may be used.

Also, for example, the series of processes described above can be performed by hardware or software.
In other words, the functional configurations of FIG. 4 and FIG. 8 are merely examples and are not particularly limited.
That is, it is sufficient if the information processing system is provided with a function capable of executing the above-described series of processes as a whole, and what function blocks are used to realize this function is particularly shown in the examples of FIGS. It is not limited. Also, the location of the functional block is not particularly limited to FIGS. 4 and 8, and may be arbitrary.
Further, one functional block may be configured by hardware alone, may be configured by software alone, or may be configured by a combination of them.

Further, for example, in the case of executing a series of processes by software, a program configuring the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware.
The computer may be a computer capable of executing various functions by installing various programs, for example, a server, a smartphone, a personal computer, or various devices.

  Also, for example, a recording medium including such a program is not only configured by a removable medium (not shown) distributed separately from the apparatus main body to provide the program to the user, but is also incorporated in the apparatus main body in advance. And a recording medium provided to the user.

In the present specification, in the step of describing the program to be recorded on the recording medium, the processing performed chronologically along the order is, of course, parallel or individually not necessarily necessarily chronologically processing. It also includes the processing to be performed.
Also, in the present specification, the term "system" is intended to mean an overall device composed of a plurality of devices, a plurality of means, and the like.

In other words, the information processing system to which the present invention is applied can take various embodiments having the following configuration.
That is, the information processing system to which the present invention is applied is
The first terminal (the HMD of the client in the first or second embodiment. The HMD on the female side in the third embodiment) and the second terminal (in the first or second embodiment, a terminal for staff in a wedding salon In the third embodiment, in the information processing system including the male HMD),
For each of a plurality of positions in a virtual space (for example, a virtual space corresponding to an actual wedding place), data groups of a plurality of images showing the state of the virtual space when the viewpoint is changed at that position are stored One or more storage devices (for example, the storage unit 18 in FIG. 2 or the storage unit 49 in FIG. 3);
The first terminal is
Specifying means (for example, a position angle specifying unit 121 in FIG. 4) for specifying a predetermined position of a predetermined movement target (a user, a character or the like) in the virtual space and a predetermined angle based on the predetermined position;
First image acquisition means for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the specifying means (For example, the image acquisition unit 104 of FIG. 4),
The first communication control unit (for example, the communication control unit 103 in FIG. 4) executes control to transmit the predetermined position and the predetermined angle specified by the specifying unit to the second terminal at least once. )When,
Equipped with
The second terminal is
A second communication control unit (for example, the communication control unit 201 in FIG. 4) that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means A second image acquisition unit (for example, the image acquisition unit 204 in FIG. 4) to acquire from at least one is sufficient.

As a result, it is possible to provide a screen sharing technology that can reduce the amount of data to be transmitted and realize faster processing.
That is, the image data itself is not transmitted from the first terminal (for example, the HMD 1 in FIG. 1) to the second terminal (for example, the display terminal 3 in FIG. 1). Speeding up of the process can be realized by sharing the image by sharing only limited data.

  1 ... HMD, 2 ... controller, 3 ... display terminal, 11 ... CPU, 17 ... sensor unit, 41 ... CPU, 101 ... map management unit, 102 ... Icon unit, 103 ... communication control unit, 104 ... image acquisition unit, 105 ... HMD display control unit, 121 ... position and angle determination unit, 201 ... communication control unit, 202 ... Map management unit 203 ... icon unit 204 ... image acquisition unit 205 ... display display control unit 211 ... position and angle determination unit 500 ... map DB 600 ... image DB, 800 ... Map DB, 900 ... Image DB

Claims (16)

In an information processing system including a first terminal and a second terminal,
One or more storage devices storing data of one or more images showing the state of the virtual space when changing the viewpoint at each position at a plurality of positions in the virtual space;
The first terminal is
First specifying means for specifying a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
A first image for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the first specifying means Acquisition means,
First communication control means for executing control to transmit the predetermined position specified by the first specifying means and the information capable of specifying the predetermined angle to the second terminal at least once;
Equipped with
The second terminal is
A second communication control unit that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means Second image acquisition means for acquiring from at least one;
Equipped with
The first communication control means of the first terminal is
Control to transmit the predetermined position and the predetermined angle specified by the first specifying means to the second terminal at timing when a first condition is satisfied;
The predetermined position and the predetermined angle transmitted to the second terminal before the predetermined position and the predetermined angle specified by the first specifying unit at the timing satisfying the second condition and the timing satisfying the second condition Execute control to transmit the relative position and the relative angle, which are the difference between
Information processing system. In an information processing system including a first terminal and a second terminal,
One or more storage devices storing data of one or more images showing the state of the virtual space when changing the viewpoint at each position at a plurality of positions in the virtual space;
The first terminal is
First specifying means for specifying a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
A first image for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the first specifying means Acquisition means,
First communication control means for executing control to transmit the predetermined position specified by the first specifying means and the information capable of specifying the predetermined angle to the second terminal at least once;
Equipped with
The second terminal is
A second communication control unit that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means Second image acquisition means for acquiring from at least one;
Second specifying means for specifying a predetermined position of another movement target different from the predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
Equipped with
The first terminal is an HMD,
The first communication control means of the first terminal further executes control to receive the information transmitted from the second terminal,
The first image acquisition unit of the first terminal is further configured on the first terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the first communication control unit. Obtaining data of an image to be displayed from at least one of the one or more storage devices;
The second image acquisition unit of the second terminal further includes data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the second specification unit. Obtain from at least one of the above storage devices,
The second communication control means of the second terminal is further controlled to transmit information capable of specifying the predetermined position and the predetermined angle specified by the second specifying means to the first terminal at least once. Run
The second communication control means of the second terminal transmits to the first terminal at least once the information capable of specifying the predetermined position among the information specified by the second specifying means. Control for prohibiting transmission to the first terminal is further executed for the information that permits the identification of the predetermined angle among the information identified by the second identification means.
Information processing system. The first communication control means of the first terminal is
Control to transmit the predetermined position and the predetermined angle specified by the first specifying means to the second terminal at timing when a first condition is satisfied;
The difference between the predetermined position and the predetermined angle specified by the first specifying means at timing when a second condition is satisfied, and the predetermined position and the predetermined angle transmitted to the second terminal in the past is the second Execute control to send to the terminal,
The information processing system according to claim 2. The first terminal further comprises detection means for outputting at least one detection amount among velocity, acceleration, angular velocity and angular acceleration;
The first condition is a condition that the detected amount is not 0,
The second condition is a condition that the detected amount is 0,
The information processing system according to claim 3. The first terminal further comprises detection means for outputting at least one detection amount among velocity, acceleration, angular velocity and angular acceleration;
Each of the first condition and the second condition at least includes a condition that the detected amount is not zero.
The information processing system according to claim 3. The second terminal is
The virtual space further includes second specifying means for specifying a predetermined position of another movement target different from the predetermined movement target and a predetermined angle based on the predetermined position.
The second image acquisition unit of the second terminal further includes data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the second specification unit. Obtain from at least one of the above storage devices,
The second communication control means of the second terminal is further controlled to transmit information capable of specifying the predetermined position and the predetermined angle specified by the second specifying means to the first terminal at least once. Run
The first communication control means of the first terminal further executes control to receive the information transmitted from the second terminal,
The first image acquisition unit of the first terminal is further configured on the first terminal side based on the predetermined position and the predetermined angle specified by the information received by the control of the first communication control unit. Acquiring data of an image to be displayed from at least one of the one or more storage devices;
The information processing system according to any one of claims 1 to 5. The first terminal is an HMD,
The second communication control means of the second terminal permits transmission to the first terminal at least once for information capable of specifying the predetermined position specified by the second specifying means, and Control is further performed to prohibit transmission to the first terminal for information that can specify the predetermined angle specified by the second specifying means.
The information processing system according to claim 6. The second communication control means of the second terminal transmits information that can specify the predetermined angle specified by the second specifying means to the first terminal when a predetermined condition is satisfied. Further execute control for prohibiting transmission to the first terminal otherwise.
The information processing system according to claim 7. The predetermined condition is a condition determined that the first terminal has not moved.
The information processing system according to claim 8. The first terminal and the second terminal are
Control to add a predetermined symbol to the image acquired as data by the first image acquisition means or the second image acquisition means, and display the image on the display device at the first terminal or the second terminal side Further comprising display control means for executing
The first communication control means of the first terminal and the second communication control means of the second terminal further execute control of transmission and reception of the predetermined symbol.
The information processing system according to any one of claims 1 to 9. The display control means of the first terminal executes control to cause the display device on the first terminal side to display the image based on data of the image of the first data amount,
The display control means of the second terminal executes control to cause the display device on the second terminal side to display the image based on data of the image of the second data amount independent of the first data amount. ,
The information processing system according to claim 10. Each of the first terminal and the second terminal further comprises at least one of the one or more storage devices independently of one another.
Information processing system according to any one of claims 1 to 11. An information processing system including a first terminal and a second terminal, the image processing system comprising: at least one image showing the state of the virtual space when changing the viewpoint at each position in the virtual space; An information processing apparatus functioning as the first terminal in an information processing system including one or more storage devices storing data of
First specifying means for specifying a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
A first image for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the first specifying means Acquisition means,
First communication control means for executing control to transmit the predetermined position specified by the first specifying means and the information capable of specifying the predetermined angle to the second terminal at least once;
Equipped with
Data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information transmitted from the information processing apparatus by the second terminal is the one or more storage devices When obtained from at least one of
The first communication control means
Control to transmit the predetermined position and the predetermined angle specified by the first specifying means to the second terminal at timing when a first condition is satisfied;
The difference between the predetermined position and the predetermined angle specified by the first specifying means at timing when a second condition is satisfied, and the predetermined position and the predetermined angle transmitted to the second terminal in the past is the second Execute control to send to the terminal,
Information processing device. An information processing system including a first terminal and a second terminal, the image processing system comprising: at least one image showing the state of the virtual space when changing the viewpoint at each position in the virtual space; An information processing apparatus as an HMD that functions as the first terminal in an information processing system including one or more storage devices in which data of
First specifying means for specifying a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
A first image for acquiring data of an image to be displayed on the first terminal side from at least one of the one or more storage devices based on the predetermined position and the predetermined angle specified by the first specifying means Acquisition means,
First communication control means for executing control to transmit the predetermined position specified by the first specifying means and the information capable of specifying the predetermined angle to the second terminal at least once;
Equipped with
The information transmitted from the information processing apparatus is received by the second terminal, and data of an image to be displayed on the second terminal side based on the predetermined position and the predetermined angle specified by the information is the first data. In a case where a predetermined position of another moving target different from the predetermined moving target in the virtual space and a predetermined angle based on the predetermined position are specified, acquired from at least one of the storage devices described above,
The first communication control means further executes control to receive the information transmitted from the second terminal,
The first image acquisition means further includes data of an image to be displayed on the information processing apparatus side based on the predetermined position and the predetermined angle specified by the information received by the control of the first communication control means. Are obtained from at least one of the one or more storage devices,
As a result, in the second terminal, among the one or more storage devices based on the predetermined position and the predetermined angle specified by the second terminal, data of an image displayed on the second terminal side The information which is acquired from at least one and which can specify the predetermined position and the predetermined angle is specified, is transmitted to the information processing apparatus at least once by the second terminal, and the predetermined position among the specified information Information that can be specified can be transmitted to the information processing apparatus at least once, and information that can specify the predetermined angle among the specified information can be transmitted to the information processing apparatus Prohibited,
Further, as a result, when the transmission of the information capable of specifying the predetermined angle is prohibited in the second terminal, the first image acquisition unit can specify the predetermined position transmitted from the second terminal. Acquiring data of an image to be displayed on the information processing apparatus side based on the basic information;
Information processing device. An information processing system including a first terminal and a second terminal, the image processing system comprising: at least one image showing the state of the virtual space when changing the viewpoint at each position in the virtual space; An information processing apparatus functioning as the second terminal in the information processing system including one or more storage devices in which data of
The first terminal identifies a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position, and the first terminal is determined based on the identified predetermined position and the predetermined angle. Data of an image to be displayed on the side is acquired from at least one of the one or more storage devices, and information capable of specifying the specified predetermined position and the predetermined angle is processed at least once from the first terminal If sent to the device,
The information processing apparatus is
A second communication control unit that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the information processing apparatus side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means Second image acquisition means for acquiring from at least one;
Equipped with
In the first terminal, the predetermined position and the predetermined angle specified at the timing satisfying the first condition are transmitted to the information processing apparatus, and the predetermined position and the predetermined angle identified at the timing satisfying the second condition; The difference between the predetermined position and the predetermined angle transmitted to the second terminal in the past is transmitted to the information processing apparatus.
As a result, the second image acquisition unit transmits, from the first terminal, the predetermined position and the predetermined angle at the timing that satisfies the first condition, and the predetermined image transmitted to the second terminal in the past. Data of an image to be displayed on the information processing apparatus side is acquired based on the position and the difference between the predetermined angle and the position.
Information processing device. An information processing system including a first terminal and a second terminal, the image processing system comprising: at least one image showing the state of the virtual space when changing the viewpoint at each position in the virtual space; An information processing apparatus functioning as the second terminal in an information processing system including one or more storage devices storing data of
The first terminal as the HMD identifies a predetermined position of a predetermined movement target in the virtual space and a predetermined angle based on the predetermined position, and is based on the specified predetermined position and the predetermined angle. The data of the image to be displayed on the first terminal side is acquired from at least one of the one or more storage devices, and the specified predetermined position and the information capable of specifying the predetermined angle are at least once When transmitted to the information processing apparatus,
The information processing apparatus is
A second communication control unit that executes control to receive the information transmitted from the first terminal;
Among the one or more storage devices, data of an image to be displayed on the information processing apparatus side based on the predetermined position and the predetermined angle specified by the information received by the control of the second communication control means Second image acquisition means for acquiring from at least one;
Second specifying means for specifying a predetermined position of another movement target different from the predetermined movement target in the virtual space and a predetermined angle based on the predetermined position;
Equipped with
Based on the predetermined position and the predetermined angle specified by the information transmitted from the information processing apparatus by the first terminal, data of an image to be displayed on the first terminal side is the one or more Obtained from at least one of the storage devices,
The second image acquisition unit further includes, on the one or more storage devices, data of an image to be displayed on the information processing apparatus side based on the predetermined position and the predetermined angle specified by the second specification unit. Get from at least one of them,
The second communication control means further executes control to transmit information capable of specifying the predetermined position and the predetermined angle specified by the second specifying means to the first terminal at least once.
The second communication control means permits transmission to the first terminal at least once for information capable of specifying the predetermined position among the information specified by the second specifying means, and (2) With respect to the information that can specify the predetermined angle among the information specified by the specifying means, control is further executed to prohibit transmission to the first terminal,
Information processing device.

Images