JP6722786B1 - Spatial information management device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a spatial information management device capable of easily transmitting spatial information. A spatial information management device includes first motion information indicating an instruction of a first user P1 located in a first space RS1 and second motion information indicating a motion of a second user P2 located in a second space RS2. And get. Further, the spatial information management device constructs an integrated space VS in which the first object Ob1 moved according to the above instruction and the second object Ob2 moved corresponding to the movement of the second user P2 are arranged in one virtual space. To do. Then, the spatial information management device generates and outputs data for displaying the constituent elements of the integrated space VS on the first display device located in the first space RS1 from the viewpoint of the first object Ob1, and outputs the second data. The second display device located in the space RS2 generates and outputs data for displaying the first object Ob1 in the real environment of the second space RS2 from the viewpoint of the second object Ob2. [Selection diagram] Figure 1

Description

The present invention relates to a spatial information management device used for remote communication.

Conventionally, for example, as a system used for a teleconference, a system has been proposed for allowing a person in a first space to communicate with a person in a second space distant from the first space. Specifically, by displaying an image of the first space on a display device located in the second space and displaying an image of the second space on a display device located in the first space, The situation of the other space is transmitted to the person (see, for example, Patent Document 1).

JP, 2018-11273, A

However, displaying an image of the other space on the display device has a limit in transmitting spatial information. For example, when a person in the first space wants to explain to a person in the second space while pointing to an article in the second space, or when he or she wants to convey information through a motion such as a gesture, communication is performed. It is hard to say that it will proceed smoothly.

An object of the present invention is to provide a spatial information management device capable of easily transmitting spatial information.

A spatial information management device that solves the above problem acquires first motion information indicating an instruction from a first user located in a first space and second motion information indicating a motion of a second user located in a second space. An acquisition unit, a first object that is moved according to the instruction based on the first motion information, and a second object that is moved corresponding to the motion of the second user based on the second motion information, An integrated space generation unit that constructs an integrated space in which the first object is visibly arranged from the second object in one virtual space that reflects the physical environment of the space, and a first display device located in the first space The first display data for displaying the constituent elements of the integrated space from the viewpoint of the first object and the second display device located in the second space, and the second display device in the viewpoint of the second object. An output data generation unit for generating second display data for displaying one object in the real environment of the second space in an overlapping manner, and outputting the first display data to the first display device to perform the second display. An output unit for outputting data to the second display device.

According to the above configuration, the first object substituting for the first user and the second object substituting for the second user share one virtual space in the integrated space, and images from respective viewpoints in this integrated space Are provided to each user through the display device. Therefore, the positional relationship, movement, etc. of the constituent elements that reflect the actual space are shared in the integrated space, so that spatial information can be easily transmitted.

In the above configuration, the virtual space in which the first object and the second object are arranged is a virtual space having an arrangement of structures imitating the second space, and the integrated space generation unit is configured to generate the second space. The object may be arranged at a position and orientation in the virtual space corresponding to the position and orientation of the second user in the second space.
According to the above configuration, it is possible to accurately share the positional relationship of the constituent elements that reflects the actual space in the integrated space.

In the above configuration, the first display data is data for displaying a component of the integrated space on the first display device as VR content, and the second display data is AR content as the first content. It may be data for displaying the object in an overlapping manner on the real environment of the second space.

According to the above configuration, the first user can view the status of the second space through the VR content provided by the first display device, even when the first user is in the first space where the arrangement of the structures is different from that of the second space. Can be grasped. On the other hand, the second user can grasp the intention of the first user through the first object while viewing the actual image of the second space through the AR content provided by the second display device. With such a configuration, the realism of each user in communication between the first user and the second user is enhanced.

In the above configuration, the first motion information may indicate at least one of an instruction given by the movement of the first user and an instruction given by the operation of the first user on an operating device.
According to the above configuration, when the first motion information indicates an instruction by the movement of the first user itself, the movement of the first user is reflected in the movement of the first object. Therefore, the first user feels as if he/she is actually operating in the integrated space, and the sense of presence is enhanced. On the other hand, when the first motion information indicates an instruction by the operation of the first user on the operating device, it is possible to reduce the space required for the movement of the first user to move the first object. Therefore, even when the first space is smaller than the second space, it is possible to move the first object to the entire area of the integrated space.

In the above configuration, the first motion information includes a position, a direction, and a gesture instruction, the second motion information includes a position and a direction of the second user, and the gesture is excluded. You may be taken.

According to the above configuration, it is possible to transmit a detailed motion as spatial information to the second user through the motion of the first object. On the other hand, since only the position and orientation of the second user are reflected in the second object, a spatially highly useful spatial relationship such as the positional relationship between the first object or the structure and the second object is provided to the first user. It is possible to reduce the load required for obtaining and transmitting the second motion information and the calculation load required for arranging the second object, while enabling transmission of various information.

In the above configuration, the first object may have an appearance capable of expressing a change in position and orientation and a gesture, and the second object may have an appearance capable of expressing only a change in position and orientation. ..
According to the above configuration, the first object and the second object have the appearances suitable for the reflected information, so that the efficiency of the processing for drawing the object can be improved.

In the above configuration, the output data generation unit may generate data for displaying a guide for the first user on the first display device and include the data in the data output from the output unit to the first display device. , And at least one of generating data for displaying the guidance for the second user on the second display device and including the data in the data output from the output unit to the second display device. ..

According to the above configuration, smoother communication between the first user and the second user by using the spatial information management device is possible.
In the above configuration, the space information management device may be used for guiding the second space by the first user to the second user.

The guidance of the second space includes spatial relations between the structure and the second user in the second space, the relation between the structure and the guide content such as which part of the second space is to be guided, and the transmission of spatial information. Highly important. Therefore, by using the spatial information management device for guiding the second space, high benefit can be obtained.

According to the present invention, spatial information can be easily transmitted.

The figure which shows the structure of the space sharing system containing the said apparatus about one Embodiment of a spatial information management apparatus. The figure which shows the processing concept of the space sharing system of one Embodiment. The figure which shows the process concept regarding the movement of the 1st object in the space sharing system of one Embodiment. The sequence diagram which shows the processing procedure in the space sharing system of one embodiment. The figure which shows an example of the image displayed on the 1st display apparatus in the space sharing system of one Embodiment. The figure which shows an example of the image displayed on the 2nd display device in the space sharing system of one Embodiment. The figure which shows an example of the image displayed on the 1st display apparatus of a modification. The figure which shows an example of the image displayed on the 2nd display device of a modification.

One embodiment of the spatial information management device will be described with reference to FIGS.
[Overall configuration of space sharing system]
An overall configuration of a space sharing system using a space information management device will be described with reference to FIG. The space sharing system of the present embodiment is a system for remotely communicating between a first user P1 who is a person located in a first space RS1 and a second user P2 who is a person located in a second space RS2. Is. In the present embodiment, the space sharing system will be described by taking as an example the case where the first user P1 guides the second user P2 in the second space RS2.

The first space RS1 and the second space RS2 are spaces separated from each other. The second space RS2 is, for example, a museum, a museum, an exhibition hall, or the like, and the second user P2 is a visitor to an event held in the second space RS2. The first space RS1 is not particularly limited, but is, for example, a conference room, an operation room, a call center, or the like managed by the event management organization, and the first user P1 is the event staff.

As shown in FIG. 1, the space sharing system includes a first information acquisition device 100 and a first display device 110 arranged in a first space RS1, and a second information acquisition device 200 and a first information acquisition device 200 arranged in a second space RS2. 2 The display device 210 and the information management server 10 which is an example of the spatial information management device are included.

In the space sharing system, the first information Ir1 that is information such as the position of the first user P1 is acquired by the first information acquisition device 100, and the second information Ir2 that is information such as the position of the second user P2. Are acquired by the second information acquisition device 200. The information management server 10 is a virtual space in which objects are arranged as if the first user P1 and the second user P2 are located in the same space based on the first information Ir1 and the second information Ir2. Build the space VS. Then, an image based on the information extracted from the integrated space VS is displayed on the first display device 110, is visually recognized by the first user P1, and is also extracted from the integrated space VS on the second display device 210. An image based on the information is displayed and visually recognized by the second user P2.

Hereinafter, the configuration of each device will be described in detail.
The first information acquisition device 100, the first display device 110, and the information management server 10 mutually transmit and receive data via a network. Similarly, the second information acquisition device 200, the second display device 210, and the information management server 10 mutually transmit and receive data via the network. A network used for communication between each of the first information acquisition device 100, the first display device 110, the second information acquisition device 200, and the second display device 210 and the information management server 10 is a general-purpose communication such as the Internet. It may be a line or a dedicated communication line for communication of each device. Further, the network used for communication between each device and the information management server 10 may include a plurality of independent networks, or may be one common network.

The first information acquisition device 100 acquires the first information Ir1. The first information Ir1 includes first motion information Im1 indicating the position, orientation, and gesture of the first user P1 in the first space RS1, and first voice information Iv1 indicating voice.

The position of the first user P1 indicated by the first motion information Im1 may be an absolute position using latitude and longitude, or may be relative to the reference position in the first space RS1 or the initial position of the first user P1. It may be a specific position. Examples of the device for detecting the position include a GPS receiver, a motion sensor including a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like. The first information acquisition device 100 determines the position of the first user P1. Includes equipment that can be detected in any desired manner.

The orientation of the first user P1 indicated by the first motion information Im1 is, for example, the orientation of the head of the first user P1, in other words, the orientation that defines the direction of the line of sight of the first user P1. The orientation of the first user P1 is calculated with reference to each axis in the three-dimensional orthogonal coordinate system, for example. Examples of the device for detecting such a direction include the motion sensor described above, a device that configures a head tracking or face tracking system that uses an optical device or various sensors, and the like. , A device capable of detecting the orientation of the first user P1 in a desired manner.

The gesture of the first user P1 indicated by the first motion information Im1 is the movement of the entire body of the first user P1 or a part of the body such as the head, hands, or feet. Examples of the device for detecting such a gesture include various motion captures such as an optical type, an inertial sensor type, and a magnetic type, and a device configuring the head tracking or face tracking system. The device 100 includes a device that can detect the gesture of the first user P1 in a desired manner.

The first voice information Iv1 indicates a voice such as a voice uttered by the first user P1. As a device for detecting such a voice, the first information acquisition device 100 includes a microphone.
Furthermore, the first information acquisition device 100 includes a communication unit that performs communication processing with the information management server 10, such as connection processing to a network, and acquisition of the first information Ir1 by each of the above-described devices, and first communication by the communication unit. A control unit that controls transmission of the information Ir1 to the information management server 10 is provided.

The respective devices configuring the first information acquisition device 100, that is, the devices for acquiring the first information Ir1 and the devices configuring the communication unit and the control unit are provided to the first user P1 according to the function of each device. It is worn or carried, or installed in the first space RS1. Devices that need to exchange information with each other are connected wirelessly or by wire.

It should be noted that the first motion information Im1 may be information indicating an instruction regarding the movement by the first user P1. Specifically, as described above, when the first motion information Im1 is information indicating the position, orientation, and gesture of the first user P1, the first motion information Im1 is the position based on the movement of the first user P1 itself. It is information indicating a direction, a direction, and a gesture instruction. In this case, the actual change amount of the position, orientation, and gesture of the first user P1 is detected and reflected in the movement of the object in the integrated space VS.

Instead of this, the first motion information Im1 may be information indicating an instruction by the operation of the first user P1 on the operating device. In this case, the first information acquisition device 100 includes a controller-type device that functions as an operating device, and the first user P1 operates a stick-shaped or button-shaped operation unit, so that the operation is performed in the position, direction, It also functions as a gesture instruction and is reflected in the movement of the object in the integrated space VS.

The first motion information Im1 may include both an instruction by the movement of the first user P1 itself and an instruction by an operation on the operating device. For example, the change of the position and the direction may be instructed by the operation, and the gesture may be instructed by the movement of the first user P1 itself.

The first display device 110 is a device that provides VR (Virtual Reality) content to the first user P1, and is embodied as, for example, a head-mounted display or a tablet terminal including a display unit such as a liquid crystal panel.

The first display device 110 includes a communication unit that performs a communication process with the information management server 10, such as a connection process to a network, a display unit that displays an image, a voice output unit that outputs a voice, and processes of these units. And a control unit for controlling. The first display device 110 receives, from the information management server 10, the first output data Dt1 generated based on the information extracted from the integrated space VS. The first display device 110 uses the first output data Dt1 to display an image on the display unit and outputs a sound from the sound output unit.
The first display device 110 may also have a part of the functions of the first information acquisition device 100.

The second information acquisition device 200 acquires the second information Ir2. The second information Ir2 includes second motion information Im2 indicating the position and orientation of the second user P2 in the second space RS2, and second voice information Iv2 indicating voice. The second motion information Im2 does not include information indicating the gesture of the second user P2. The second motion information Im2 is information indicating the movement of the second user P2, and functions as information indicating a position and orientation instruction by the movement of the second user P2 itself.

The position of the second user P2 indicated by the second motion information Im2 may be an absolute position using latitude and longitude, or may be relative to the reference position in the second space RS2 or the initial position of the second user P2. It may be a specific position. Examples of the device for detecting such a position include a GPS receiver and a motion sensor including a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like. The second information acquisition device 200 detects the position of the second user P2. Includes equipment that can be detected in any desired manner.

The orientation of the second user P2 indicated by the second motion information Im2 is, for example, the orientation of the head of the second user P2, in other words, the orientation that defines the direction of the line of sight of the second user P2. The orientation of the second user P2 is calculated with reference to each axis in the three-dimensional orthogonal coordinate system, for example. Examples of the device for detecting the orientation include the above-described motion sensor, devices that configure a head tracking or face tracking system using an optical device or various sensors, and the second information acquisition device 200 includes , A device that can detect the orientation of the second user P2 in a desired manner.

The second voice information Iv2 indicates a voice such as a voice uttered by the second user P2. The second information acquisition device 200 includes a microphone as a device for detecting such a voice.
Furthermore, the second information acquisition device 200 includes a communication unit that performs communication processing with the information management server 10, such as connection processing to a network, and acquisition of the second information Ir2 by each of the above-described devices, and second communication by the communication unit. A control unit that controls transmission of the information Ir2 to the information management server 10 is provided.

The respective devices configuring the second information acquisition device 200, that is, the devices for acquiring the second information Ir2 and the devices configuring the communication unit and the control unit are provided to the second user P2 according to the function of each device. It is worn, carried, or installed in the second space RS2. Devices that need to exchange information with each other are connected wirelessly or by wire.

The second display device 210 is a device that provides AR (Augmented Reality) content to the second user P2, and is embodied in, for example, a transmissive head-mounted display or a tablet terminal including a display unit such as a liquid crystal panel. It

The second display device 210 includes a communication unit that performs a communication process with the information management server 10, such as a connection process to a network, a display unit that displays an image, a voice output unit that outputs a voice, and processes of these units. And a control unit for controlling. The second display device 210 receives, from the information management server 10, the second output data Dt2 generated based on the information extracted from the integrated space VS. The second display device 210 superimposes the image based on the second output data Dt2 on the real environment of the second space RS2, displays the image on the display unit, and outputs the sound from the sound output unit.
The second display device 210 may also have a part of the functions of the second information acquisition device 200.

The information management server 10 includes a communication unit 20, a control unit 30, and a storage unit 40.
The communication unit 20 executes a connection process between the information management server 10 and each of the above-described devices via a network, and transmits and receives data between these devices. The communication unit 20 functions as an acquisition unit that acquires the first information Ir1 from the first information acquisition device 100 and acquires the second information Ir2 from the second information acquisition device 200, and also makes a first output to the first display device 110. It functions as an output unit that outputs the data Dt1 and outputs the second output data Dt2 to the second display device.

The control unit 30 has a configuration including a CPU and a volatile memory such as a RAM. The control unit 30 controls the processing by the communication unit 20, reads and writes information in the storage unit 40, performs various arithmetic processes, and the like included in the information management server 10 based on the programs and data stored in the storage unit 40. Control. The control unit 30 includes a first information processing unit 31, a second information processing unit 32, an integrated space generation unit 33, and an output data generation unit 34 as functional units configuring the space sharing system.

The first information processing section 31 uses the first information Ir1 to generate data for generating the integrated space VS. Specifically, the first information processing section 31 sets the position, orientation, and body movement of the first object Ob1, which is an avatar included in the components of the integrated space VS, based on the first motion information Im1. To do.

The second information processing unit 32 uses the second information Ir2 to generate data for generating the integrated space VS. Specifically, the second information processing unit 32 sets the position and orientation of the second object Ob2 included in the constituent elements of the integrated space VS based on the second motion information Im2.

The integrated space generation unit 33 generates the integrated space VS. Specifically, the integrated space generation unit 33 uses the data stored in the storage unit 40 and the data generated by the first information processing unit 31 and the second information processing unit 32 to create one virtual space. The first object Ob1 and the second object Ob2 are arranged to construct the integrated space VS.

The output data generation unit 34 extracts information from the integrated space VS generated by the integrated space generation unit 33 and generates the first output data Dt1 and the second output data Dt2. The first output data Dt1 includes display data for displaying the integrated space VS viewed from the viewpoint of the first object Ob1 on the first display device 110, and audio data based on the second audio information Iv2. The second output data Dt2 includes display data for displaying the first object Ob1 viewed from the viewpoint of the second object Ob2 on the second display device 210, and audio data based on the first audio information Iv1.

The storage unit 40 has a configuration including a non-volatile memory and stores programs and data necessary for the processing executed by the control unit 30. Specifically, the storage unit 40 stores real space data 41, virtual space data 42, and object data 43.

The real space data 41 includes data used when the first information processing unit 31 sets the first object Ob1 based on the first motion information Im1. The data is data for associating the position and orientation in the first space RS1 with the position and orientation in the integrated space VS. Similarly, the real space data 41 includes data used when the second information processing unit 32 sets the second object Ob2 based on the second motion information Im2. The data is data for associating the position and orientation in the second space RS2 with the position and orientation in the integrated space VS.

The virtual space data 42 is data of the basic space VB, which is a virtual space that constitutes a structure included in the constituent elements of the integrated space VS, in other words, a portion that does not include the first object Ob1 and the second object Ob2. The basic space VB is a space imitating the second space RS2 in the state where the second user P2 does not exist, and includes fixed objects such as exhibits, passages, and walls. The virtual space data 42 is generated in advance using, for example, a 3D scan of the second space RS2. The virtual space data 42 includes information on a three-dimensional orthogonal coordinate system set for the basic space VB, position information of the fixed object described above, and data for drawing the basic space VB.

The object data 43 is data for generating an object that moves in the integrated space VS. The object data 43 includes data for drawing the first object Ob1 and the second object Ob2.

The functions of the first information processing unit 31, the second information processing unit 32, the integrated space generation unit 33, and the output data generation unit 34 in the control unit 30 are various CPUs, memories such as RAMs, and the like. Hardware and software that causes them to be embodied separately, or may be embodied by software that imparts a plurality of functions to one common hardware. Such software is stored in the storage unit 40 as a spatial information management program.

[Processing concept of space sharing system]
With reference to FIG. 2, the process of the space sharing system will be conceptually described with a focus on the process of the information management server 10.

As shown by an arrow A1 in FIG. 2, the first information Ir1 is acquired from the first space RS1 which is a real space, and the position, orientation, and body of the first object Ob1 are acquired based on the first motion information Im1. Is set. The first object Ob1 has an appearance capable of expressing changes in position and orientation and gestures. For example, the first object Ob1 is represented as a humanoid image having a head, a body, and limbs, such as an arbitrary character.

When the first motion information Im1 indicates an instruction based on the movement of the first user P1 itself, the position, the orientation, and the position of the first object Ob1 corresponding to the position, the orientation, and the gesture of the first user P1. Body movement is set. The calculation of the position, orientation, and gesture of the first user P1 may be performed by the first information acquisition device 100 or the information management server 10. That is, the first motion information Im1 may be information indicating the calculation result of the position, orientation, and gesture of the first user P1, or the information management server that indicates the position, orientation, and gesture of the first user P1. The information may be information that can be calculated in 10.

When the first motion information Im1 indicates an instruction by the operation of the first user P1 on the operating device, the position, orientation, and body movement of the first object Ob1 are set according to the operation content.

A plurality of first users P1 may be located in the first space RS1. In this case, the first information Ir1 is acquired for each first user P1. Then, the first object Ob1 for each first user P1 is set, and the position, orientation, and movement of the body are set for each first object Ob1 based on the first motion information Im1 of the corresponding first user P1. To be done. That is, one first object Ob1 is set for one first user P1 and a plurality of first objects Ob1 are arranged in the integrated space VS.

As indicated by the arrow A2, the second information Ir2 is acquired from the second space RS2 that is the actual space, and the position and orientation of the second object Ob2 is set based on the second motion information Im2. That is, the position and orientation of the second object Ob2 are set in correspondence with the position and orientation of the second user P2. The calculation of the position and orientation of the second user P2 may be performed by the second information acquisition device 200 or the information management server 10. That is, the second motion information Im2 may be information indicating the calculation result of the position and orientation of the second user P2, or information indicating that the position and orientation of the second user P2 can be calculated by the information management server 10. May be

Unlike the first object Ob1, the second object Ob2 reflects the position and orientation of the second user P2, but does not reflect the gesture. Therefore, the first object Ob1 is configured to be capable of expressing the gesture in addition to the position and the orientation, whereas the second object Ob2 may be configured to be capable of expressing the change in the position and the orientation. As an example in which the second object Ob2 has an appearance in which only changes in position and orientation can be expressed, there is a case where the second object Ob2 is expressed as an image consisting of only the head and chest, as shown in FIG. To be However, the second object Ob2 has only to have an appearance capable of expressing changes in position and orientation, and may be expressed as a humanoid image including limbs.

In addition, a plurality of second users P2 may be located in the second space RS2. In this case, the second information Ir2 is acquired for each second user P2. Then, the second object Ob2 for each second user P2 is set, and the position and orientation of each second object Ob2 are set based on the second motion information Im2 of the corresponding second user P2. That is, one second object Ob2 is set for one second user P2, and a plurality of second objects Ob2 are arranged in the integrated space VS.

As indicated by an arrow A3, data of the basic space VB having the arrangement of structures that imitate the second space RS2 and data of the first object Ob1 including the set position, orientation, and body movement are set. The integrated space VS is generated by combining these using the data of the second object Ob2 including the position and the orientation. That is, by arranging the first object Ob1 corresponding to the first user P1 and the second object Ob2 corresponding to the second user P2 in one virtual space in which the real environment of the second space RS2 is reflected, , The integrated space VS is constructed. In the integrated space VS, the first object Ob1 and the second object Ob2 may be in the field of view of each other.

As indicated by the arrow A4, the first output data Dt1 transmitted to the first display device 110 is a display for displaying the integrated space VS on the first display device 110 as a VR content from the viewpoint of the first object Ob1. Contains data. That is, the first output data Dt1 is data generated according to the position and orientation of the first object Ob1, and is an image of a predetermined range centered on the front of the first object Ob1 in the integrated space VS. Contains data for display. The image includes all of the components of the integrated space VS located within the above range, that is, the structure and the second object Ob2 that configure the basic space VB.

As indicated by the arrow A5, the second output data Dt2 transmitted to the second display device 210 is for displaying the first object Ob1 on the second display device 210 from the viewpoint of the second object Ob2 as AR content. Contains display data. That is, the second output data Dt2 is data generated according to the position and orientation of the second object Ob2, and is the first data within a predetermined range centered on the front surface of the second object Ob2 in the integrated space VS. When the object Ob1 is included, it includes data for displaying an image including only the first object Ob1 among the constituent elements of the integrated space VS.

On the second display device 210, the image of the first object Ob1 based on the second output data Dt2 is displayed in an overlapping manner with the real environment of the second space RS2 viewed from the viewpoint of the second user P2. For example, when the second display device 210 is a transmissive head mounted display, the real environment is an image of the second space RS2 visually recognized through the display unit, and the second display device 210 is a tablet terminal. In some cases, the front area of the tablet terminal is an image of the second space RS2 displayed on the display unit by being photographed by the camera.

[Setting Object Position and Orientation]
Each mode of setting the position and orientation of the first object Ob1 and setting the position and orientation of the second object Ob2 will be described.

First, regarding the setting of the position and orientation of the second object Ob2, a case where the position of the second user P2 is detected as the position with respect to the reference position in the second space RS2 will be described. In this case, as shown in FIG. 2, the reference position XR is set in the second space RS2 in advance. Further, in the basic space VB and the integrated space VS, the reference position XV is set at a position corresponding to the reference position XR of the second space RS2. The position information of the reference position XV is included in the virtual space data 42.

A marker is provided at the reference position XR of the second space RS2. The marker is configured to be recognizable based on the information acquired by the second information acquisition device 200. For example, when the second information acquisition device 200 has an image capturing function, the marker is configured to be recognizable by image analysis. Specifically, the marker may be a dedicated marker such as a two-dimensional code, or the shape of the structure located at the reference position XR may be used as the marker. When the second information acquisition device 200 has a short-distance wireless communication function, the IC tag installed at the reference position XR functions as a marker, and reading information from the IC tag is regarded as recognition of the marker. May be.

The marker is recognized based on the fact that the second user P2 carrying the second information acquisition device 200 is located in the vicinity of the marker and the second information acquisition device 200 acquires information by photographing the marker or the like. The marker recognition process may be performed by the second information acquisition device 200 and the recognition result may be transmitted to the information management server 10, or the information such as the image acquired by the second information acquisition device 200 may be the information management server 10. The information management server 10 may perform marker recognition processing.

When the marker is recognized, the second information processing unit 32 of the information management server 10 sets the position of the second object Ob2 at a predetermined position with respect to the reference position XV of the virtual space, and sets the orientation of the second object Ob2. Set to the specified orientation.

For example, when the marker is configured to be recognizable when the second information acquisition device 200 is located at a position away from the marker by a predetermined distance in a predetermined direction, the predetermined direction and The position of the second object Ob2 is set at the position corresponding to the predetermined distance. Then, the direction of the second object Ob2 is set from the set position toward the reference position XV. Alternatively, the position and orientation of the second information acquisition device 200 with respect to the reference position XR are calculated from the orientation and size of the marker photographed by the second information acquisition device 200, and the reference position is set so as to correspond to the calculated position and orientation. The position and orientation of the second object Ob2 with respect to the position XV may be set.

Thus, the initial position and orientation of the second object Ob2 are set based on the recognition of the marker. Then, when a change in the position and the orientation of the second user P2 in the second space RS2 is detected based on the second motion information Im2, the second information processing section 32 corresponds to the detected change amount and outputs the second value. The position and orientation of the object Ob2 is changed from the initial state. As a result, the position and orientation of the second object Ob2 changes in response to the change in the position and orientation of the second user P2. As a result, in the second space RS2 and the integrated space VS having the arrangement of the structures that match, the position and orientation in the integrated space VS corresponding to the position and orientation of the second user P2 in the second space RS2 are Two objects Ob2 are arranged.

The position and orientation of the second object Ob2 set based on the recognition of the marker are not limited to the initial position and orientation. For example, a plurality of markers are provided in the second space RS2, and the positions corresponding to the positions of the markers are set in advance in the basic space VB and the integrated space VS. Then, the second information acquisition device 200 that moves together with the second user P2 recognizes each of the markers when the second information acquisition apparatus 200 is located in the vicinity of each of the markers. It may be set based on the position of the virtual space corresponding to the position of the marker.

The mode of setting the position and orientation of the second object Ob2 is not limited to the above. Instead of setting the position and orientation of the second object Ob2 using the reference position, the position and orientation of the second object Ob2 is set using absolute position information detected by using the GPS receiver. May be done. That is, a coordinate system corresponding to the coordinate system of the second space RS2 defined by latitude and longitude is set in the basic space VB and the integrated space VS. Then, the position of the second object Ob2 is set to the position corresponding to the detected position of the second user P2 in the second space RS2. Further, the orientation is also set in the coordinate system, and the orientation of the second object Ob2 is set to the orientation corresponding to the detected orientation of the second user P2 in the second space RS2.

Next, setting of the position and orientation of the first object Ob1 will be described. The initial position and orientation of the first object Ob1 in the integrated space VS are set to a predetermined position and orientation, or an arbitrary position and orientation designated by the first user P1. When the position and the orientation are designated by the first user P1, the instruction of the position and the orientation is given through the operation of the first user P1 to the first information acquisition device 100, that is, from the first information acquisition device 100 to the information management server 10. Is performed based on the transmission of information according to the above operation. The initial position and orientation of the set first object Ob1 are the position and orientation in the virtual space corresponding to the initial position and orientation of the first user P1 in the first space RS1.

Then, based on the motion instruction indicated by the first motion information Im1, the first information processing section 31 changes the position and orientation of the first object Ob1 from the initial state.
Here, when the first motion information Im1 indicates an instruction by the movement of the first user P1 itself and a change in the actual position and orientation of the first user P1 is detected, the first motion information Im1 corresponds to the change, and The position and orientation of the one object Ob1 is changed.

On the other hand, when the first motion information Im1 indicates a movement instruction by the operation of the first user P1 on the operating device, the position and orientation of the first object Ob1 is changed from the initial state according to the operation. Conceptually, such a process can be regarded as the first object Ob1 moving in response to the movement of the first space RS1 in the space including the first space RS1. The details will be described with reference to FIG.

As shown in FIG. 3, when the first user P1 operates the first information acquisition device 100, the position of the first user P1 itself in the first space RS1 does not change. On the other hand, the position of the integrated space VS corresponding to each position of the first space RS1 changes according to the operation, and the position of the integrated space VS corresponding to the position of the first user P1 changes. That is, each position of the inclusion space CS including the first space RS1 is associated with each position of the integrated space VS, and the first space RS1 including the first user P1 is included in the inclusion space CS. By moving relatively, the position of the first object Ob1 moves according to the amount of movement. The same applies to the orientation.

Therefore, the instruction by the operation of the first user P1 is to move the first space RS1 relative to the inclusive space CS so that the integrated space VS corresponding to the position of the first user P1 in the first space RS1. It can also be regarded as an instruction to move the first object Ob1 through changing the position. That is, the first object Ob1 is arranged at the position of the integrated space VS corresponding to the position of the first user P1 in the first space RS1, and the first object Ob1 is moved as the position of the integrated space VS changes.

As described above, if the position and the orientation of the first object Ob1 are changed according to the operation of the first information acquisition device 100, the space required for the movement of the first user P1 to move the first object Ob1. Can be reduced. Therefore, for example, even when the first space RS1 is smaller than the second space RS2, the first object Ob1 can be moved to the entire area of the integrated space VS. That is, the second user P2 can receive the guide of the exhibit from the first object Ob1 through the second display device 210 in the entire area of the second space RS2.

The mode of setting the position and orientation of the first object Ob1 is not limited to the above. Similar to the case of the second object Ob2, the position and orientation of the first object Ob1 may be set using the reference positions set in the first space RS1 and the virtual space, respectively. Further, by using the absolute position information detected by using the GPS receiver, based on the coordinate system set in the virtual space corresponding to the coordinate system of the first space RS1, the position of the first object Ob1 and The orientation may be set.

Regarding the setting of the positions and orientations of the first object Ob1 and the second object Ob2, the coordinate system of each of the first space RS1 and the second space RS2 which are real spaces and the basic space VB and the integrated space VS which are virtual spaces. The data for managing the correspondence between the positions and the positions is included in the real space data 41.

[Operation of space sharing system]
The operation of the space sharing system will be described with reference to FIG. As shown in FIG. 4, the first information acquisition device 100 acquires the first information Ir1 in the first space RS1 (step S10) and transmits the first information Ir1 to the information management server 10 (step S11). In addition, the second information acquisition device 200 acquires the second information Ir2 in the second space RS2 (step S12) and transmits the second information Ir2 to the information management server 10 (step S13).

Upon receiving the first information Ir1, the first information processing section 31 of the information management server 10 sets the position, orientation, and body movement of the first object Ob1 based on the first motion information Im1 (step S14). .. Upon receiving the second information Ir2, the second information processing section 32 sets the position and orientation of the second object Ob2 based on the second motion information Im2 (step S15). Then, the integrated space generation unit 33 uses the respective data of the basic space VB, the first object Ob1, and the second object Ob2 to generate the integrated space VS (step S16).

The output data generation unit 34 uses the data in the integrated space VS to generate the first output data Dt1 and the second output data Dt2 (step S17). The first output data Dt1 also includes audio data based on the second audio information Iv2, and the second output data Dt2 also includes audio data based on the first audio information Iv1.

The first output data Dt1 is transmitted from the information management server 10 to the first display device 110 (step S18), and the first display device 110 displays the first output image Mg1 which is an image based on the first output data Dt1. At the same time, the sound based on the first output data Dt1 is output (step S19).

The second output data Dt2 is transmitted from the information management server 10 to the second display device 210 (step S20), and the second display device 210 displays the second output image Mg2 which is an image based on the second output data Dt2. At the same time, the sound based on the second output data Dt2 is output (step S21).

By sequentially repeating the processing of steps S10 to S21, the video accompanied by the movement of the first object Ob1 and the second object Ob2 is displayed on the first display device 110 and the second display device 210.

Each of the first output image Mg1 displayed on the first display device 110 and the second output image Mg2 displayed on the second display device 210 will be described with reference to FIGS. 5 and 6. 5 and 6, dots are added to the components of the integrated space VS.

As shown in FIG. 5, the first output image Mg1 displayed on the first display device 110 is an image of the integrated space VS viewed from the viewpoint of the first object Ob1. The image viewed by the first user P1 through the first display device 110 is composed of only the components of the integrated space VS. The first output image Mg1 includes a structure Tv such as an exhibit located in the integrated space VS and a second object Ob2. The first user P1 can grasp the positions and orientations of the structure Tv and the second object Ob2 in the integrated space VS by looking at the first output image Mg1.

The structure Tv in the integrated space VS has the same arrangement as that of the structure in the second space RS2, and the position and orientation of the second object Ob2 is the position of the second user P2 in the second space RS2. And the orientation is reflected. Therefore, the first user P1 can grasp the situation of the second space RS2, including the positional relationship between the structures in the second space RS2 and the second user P2, by looking at the first output image Mg1.

The first user P1 moves the first object Ob1 through the first information acquisition device 100 while looking at the first output image Mg1, and guides the exhibit or the like. Since the first output image Mg1 is an image from the viewpoint of the first object Ob1, the first user P1 can provide guidance while feeling as if the user is located in the integrated space VS as the first object Ob1. .. Then, as described above, since the integrated space VS is a virtual space that is an alternative to the second space RS2, the first user P1 can guide the second space RS2 through the guidance in the integrated space VS. ..

As shown in FIG. 6, the second output image Mg2 displayed on the second display device 210 is an image of the first object Ob1 viewed from the viewpoint of the second object Ob2 and viewed from the viewpoint of the second user P2. It is displayed in an overlapping manner with the real environment of the second space RS 2.

As described above, the structure Tv of the integrated space VS has the same arrangement as the structure Ts of the second space RS2, and the position and orientation of the second object Ob2 are the same as those of the second space RS2. The position and orientation of the second user P2 are reflected. Therefore, the arrangement of the structures Tv of the integrated space VS from the viewpoint of the second object Ob2 matches the arrangement of the structures Ts of the second space RS2 from the viewpoint of the second user P2. That is, the image viewed by the second user P2 through the second display device 210 is the image of the integrated space VS from the viewpoint of the second object Ob2, and the structure Tv other than the first object Ob1 is the actual image of the second space RS2. This is an image replacing the structure Ts.

The movement of the first object Ob1 reflects the instruction of the first user P1. As described above, the first user P1 grasps the situation of the second space RS2 through the image of the integrated space VS, and guides the movement. It is carried out. Therefore, the second user P2 is guided by the first user P1 by receiving the guidance of the first object Ob1 while watching the actual structure Ts of the second space RS2 using the second display device 210. You can

As described above, in the space sharing system of the present embodiment, it is possible to communicate with the first user P1 and the second user P2 that are located in mutually separated spaces. Then, the first object Ob1 substituting for the first user P1 and the second object Ob2 substituting for the second user P2 share one virtual space in the integrated space VS, and from each viewpoint in the integrated space VS. Is provided to the first user P1 and the second user P2. Therefore, in the integrated space VS, the positional relationships and movements of the components that reflect the actual space are shared, so that the gestures intended by the first user P1 and the positional relationship between the second user P2 and the structure Ts are shared. Spatial information such as can be easily transmitted.

Further, the first object Ob1 reflects the gesture and the position and orientation intended by the first user P1, while the second object Ob2 reflects only the position and orientation of the second user P2. .. Since the first object Ob1 guides the second space RS2 as an alternative to the first user P1, gestures such as pointing to an exhibit can be expressed to make the guide easier to understand.

On the other hand, in the second object Ob2, it is highly important to show the positional relationship between the structure and the first object Ob1 in the integrated space VS as an alternative to the second user P2. 2 The priority of reproducing up to the gesture of the user P2 is low. Therefore, by reflecting only the position and orientation of the second user P2 on the second object Ob2, it is possible to transmit highly useful spatial information, and at the same time, the load required for acquisition and transmission of the second information Ir2. , And the calculation load required for setting the second object Ob2 can be reduced.

Further, as in the conventional case, the first space RS1 is imaged and the image data is transmitted to the display device in the second space RS2, and the second space RS2 is imaged and the image data is transmitted to the display device in the first space RS1. Since less video data is transmitted than in the form, the amount of communication can be reduced. Further, compared to the form in which the image of the first space RS1 is visually recognized by the second user P2, the first user P1 who instructs the movement of the first object Ob1 during the series of guidance by the first object Ob1 is It is possible to flexibly support smooth guidance such as a change or a different person performing a motion instruction and voice input in the first space RS1.

As described above, according to this embodiment, the following effects can be obtained.
(1) The first object Ob1 substituting for the first user P1 and the second object Ob2 substituting for the second user P2 share one virtual space in the integrated space VS, and from each viewpoint in the integrated space VS. Is provided to the users P1 and P2 through the display devices 110 and 210. Therefore, in the integrated space VS, the positional relationship, movement, etc. of the constituent elements that reflect the actual space are shared, so that spatial information can be easily transmitted.

(2) The integrated space VS is a virtual space having an arrangement of structures imitating the second space RS2, and the second object Ob2 is a virtual space corresponding to the position and orientation of the second user P2 in the second space RS2. Arranged in a position and orientation in space. As a result, the positional relationship of the constituent elements that reflects the actual space in the integrated space VS is accurately shared.

(3) An image from the viewpoint of the first object Ob1 is output as VR content to the first display device 110, and an image from the viewpoint of the second object Ob2 is output to the second display device 210 as AR content. Is output. Therefore, the first user P1 can grasp the situation of the second space RS2 through the VR content provided by the first display device 110 while being in the first space RS1 in which the structure is arranged differently from the second space RS2. Since it is possible, accurate guidance can be provided. Meanwhile, the second user P2 receives the guidance from the first object Ob1 substituting for the first user P1 while viewing the actual image of the second space RS2 through the AR content provided by the second display device 210. You can With such a configuration, the realism of each of the first user P1 and the second user P2 is enhanced.

(4) In the form in which the first motion information Im1 indicates the instruction by the movement of the first user P1 itself, the position, orientation, and gesture of the first user P1 itself are reflected in the first object Ob1. Therefore, the first user P1 feels as if he/she is actually located in the integrated space VS, and the sense of presence is enhanced. On the other hand, when the first motion information Im1 indicates an instruction by the operation of the first user P1 on the operating device, it is possible to reduce the space required for the movement of the first user P1 for moving the first object Ob1. Therefore, even when the first space RS1 is smaller than the second space RS2, the first object Ob1 is moved to the entire area of the integrated space VS, that is, the guidance is performed in the entire area of the second space RS2. You can

(5) The first object Ob1 has an appearance capable of expressing changes in position and orientation and gestures, and the position, orientation, and gestures intended by the first user P1 are reflected. Therefore, it is possible to provide easier-to-understand guidance by the movement of the first object Ob1. On the other hand, the second object Ob2 reflects only the position and orientation of the second user P2. Thereby, it is possible to transmit highly useful spatial information such as the structure and the positional relationship between the first object Ob1 and the second object Ob2, and the load required to acquire or transmit the second information Ir2, and The calculation load required for setting the second object Ob2 can be reduced. If the second object Ob2 has a configuration capable of expressing only the change in the position and the orientation, the second object Ob2 has a configuration suitable for the information in which the second object Ob2 is reflected. It is possible to improve processing efficiency.

(6) The space sharing system is used for guiding the second space RS2 to the second user P2 by the first user P1. The guidance of the second space RS2 is spatial in terms of the positional relationship between the structure and the second user P2 in the second space RS2, the relationship between the structure and the guide content such as which part of the second space RS2 is to be guided, and the like. The transmission of information is highly important. Therefore, by using the space sharing system for guiding the second space RS2, a high benefit can be obtained.

[Modification]
The above embodiment can be modified and implemented as follows.
As shown in FIG. 7, the first output image Mg1 may include a first guidance display Ge1 which is guidance for the first user P1. The first guidance display Ge1 includes, for example, a display for assisting the guidance of the second space RS2 such as a description of an exhibit. With such a configuration, the guidance by the first user P1 can proceed more smoothly. The first guidance display Ge1 is, for example, a component of the integrated space VS and is arranged as an object at a predetermined position in the integrated space VS. As a result, the first output data Dt1 includes data for displaying the first guidance display Ge1 as an image of the integrated space VS, and the first output image Mg1 includes the first guidance display Ge1. Alternatively, the first guidance display Ge1 does not have to be a component of the integrated space VS, and the first guidance display Ge1 is displayed at a specific location on the display unit of the first display device 110 in an overlapping manner with the image of the integrated space VS. As described above, the first output data Dt1 may be configured. The data required for drawing the first guidance display Ge1 is stored in the storage unit 40 in advance.

As shown in FIG. 8, the second output image Mg2 may include a second guidance display Ge2 that is guidance for the second user P2. The second guidance display Ge2 includes, for example, a caution when receiving the guidance from the first user P1, a display showing a route to be moved in the second space RS2 while receiving the guidance, and the like. With such a configuration, the guidance for the second user P2 can be proceeded more smoothly. The second guidance display Ge2 is, for example, a component of the integrated space VS, and is arranged as an object at a predetermined position in the integrated space VS. Then, the second output data Dt2 is configured as data for displaying the first object Ob1 and the second guidance display Ge2 viewed from the viewpoint of the second object Ob2 in an overlapping manner in the real environment of the second space RS2. Alternatively, the second guidance display Ge2 does not have to be a component of the integrated space VS, and the second guidance display Ge2 is superimposed on the real environment of the second space RS2 and is displayed at a specific location on the display unit of the second display device 210. The second output data Dt2 may be configured to be displayed. Data required for drawing the second guidance display Ge2 is stored in the storage unit 40 in advance.

The first guidance display Ge1 may be always displayed on the first display device 110, or may be displayed at a timing when a predetermined condition is satisfied. Similarly, the second guidance display Ge2 may be always displayed on the second display device 210, or may be displayed at a timing when a predetermined condition is satisfied. The timing when the predetermined condition is satisfied is, for example, when the first user P1 gives an instruction through an operation on the operating device, or when the first object Ob1 or the second object Ob2 moves to a specific position in the integrated space VS. Examples include a case where the first object Ob1 makes a specific gesture, a case where a predetermined number or more of the second objects Ob2 gather in a predetermined range, and the like. The display timings of the first guidance display Ge1 and the second guidance display Ge2 are controlled by the integrated space generation unit 33 when these are components of the integrated space VS, and output when they are not components of the integrated space VS. It is controlled by the data generator 34.

In the above embodiment, the position, orientation, and gesture indicated by the first user P1 are reflected in the first object Ob1, but only the position and orientation may be reflected in the first object Ob1. In this case , The first object Ob1 may have an appearance capable of expressing only changes in position and orientation. Further, in the above embodiment, the position and orientation of the second user P2 are reflected in the second object Ob2, but gestures may also be reflected. In this case, the second object Ob2 may have an appearance capable of expressing a gesture in addition to the change in position and orientation. Furthermore, the first motion information Im1 may include information indicating the facial expression and the line of sight of the first user P1, and such facial expression and the line of sight may be reflected in the first object Ob1. Similarly, the second motion information Im2 may include information indicating the facial expression and the line of sight of the second user P2, and such facial expression and the line of sight may be reflected in the second object Ob2.

The first voice information Iv1 may be used for setting the first object Ob1. Specifically, when the voice uttered by the first user P1 is input to the first information acquisition device 100, the first voice information Iv1 indicating the voice is included in the first information Ir1 to acquire the first information. The information is transmitted from the device 100 to the information management server 10. When acquiring the first voice information Iv1, the first information processing section 31 of the information management server 10 changes the first object Ob1 so as to suggest that the first object Ob1 is uttered. Such changes include, for example, changes in the facial expression of the first object Ob1, movements of the body such as movements of the mouth, and changes in colors such as the complexion. In the second display device 210, the integrated space VS including the first object Ob1 that changes according to the acquisition of the first audio information Iv1 is generated, and the second output data Dt2 is output to the second display device 210. , The first object Ob1 is displayed, and the sound based on the first sound information Iv1 is output. As a result, the second user P2 feels as if the first object Ob1 is talking from both the visual and auditory senses, and thus communication is likely to proceed smoothly.

Similarly, the second audio information Iv2 may be used to set the second object Ob2. Specifically, when the voice uttered by the second user P2 is input to the second information acquisition device 200, the second voice information Iv2 indicating the voice is included in the second information Ir2 to obtain the second information. The information is transmitted from the device 200 to the information management server 10. When acquiring the second voice information Iv2, the second information processing unit 32 of the information management server 10 changes the second object Ob2 so as to suggest that the second object Ob2 is speaking. The integrated space VS including the second object Ob2 that changes according to the acquisition of the second audio information Iv2 is generated, and the first output data Dt1 is output to the first display device 110. , The second object Ob2 is displayed, and the sound based on the second sound information Iv2 is output. As a result, the first user P1 feels as if the second object Ob2 is talking from both the visual and the auditory senses, and the communication is likely to proceed smoothly.

The first output data Dt1 may be data for displaying at least a part of the constituent elements of the integrated space VS on the first display device 110, and need not be data for displaying the entire integrated space VS. Good. For example, when the integrated space VS includes the first guidance display Ge1 and the second guidance display Ge2 described above, the second guidance display Ge2 is not included in the first output image Mg1 based on the first output data Dt1. You can

The first output image Mg1 output as VR content on the first display device 110 may be visible together with the real environment of the first space RS1. In particular, when the first user P1 gives an instruction by his/her own movement, the first user P1 can grasp the real environment of the first space RS1, and thus it is possible to easily secure the safety when the first user P1 moves. is there. For example, a small window-shaped region may be provided in the first display device 110, and a captured image that is a peripheral image of the first user P1 captured by the first display device 110 may be displayed in this region. Alternatively, the first output image Mg1 may be displayed in a transmissive state in which the photographed image can be visually recognized so as to be superimposed on the photographed image, or conversely, the photographed image can visually recognize the first output image Mg1. It may be displayed in a transparent state so as to be superimposed on the first output image Mg1.

The first output data Dt1 may be data for displaying the second object Ob2 as an AR content by superimposing it on the real environment of the first space RS1 from the viewpoint of the first object Ob1. For example, when the first space RS1 and the second space RS2 are spaces having the same arrangement of structures, if the second object Ob2 is displayed on the first display device 110 from the viewpoint of the first object Ob1, The first user P1 can grasp the situation of the second space RS2. In this case, the integrated space VS does not have to have the arrangement of the structures that match the second space RS2, for example, only the first object Ob1 and the second object Ob2 are arranged and associated with the actual space. The positional relationship between these objects may be shared based on the determined predetermined position. That is, as the real environment of the second space RS2, at least the position of the second user P2 in the second space RS2 and the position of the second object Ob2 in the integrated space VS can be associated with the integrated space VS. As described above, the information about the position such as the coordinate system may be reflected.

The same image as the image displayed on the second display device 210 may be displayed on the first display device 110 together with the image based on the first output data Dt1. For example, a small window-shaped area is provided in the first display device 110, and the same image as the image displayed on the second display device 210 is displayed in this area. In this case, information for displaying the same image as the image displayed on the second display device 210 on the first display device 110 together with the first output data Dt1, that is, data according to the second output data Dt2 is information. It is transmitted from the management server 10 to the first display device 110.

Similarly, the same image as the image displayed on the first display device 110 may be displayed on the second display device 210 together with the image based on the second output data Dt2.
-In the space sharing system, for the first users P1 located in each of the plurality of first spaces RS1, the first object Ob1 corresponding to each first user P1 may be arranged in the integrated space VS. Further, for the second user P2 located in each of the plurality of second spaces RS2, the second object Ob2 corresponding to each second user P2 may be arranged in the integrated space VS. The plurality of second spaces RS2 have the same arrangement of structures, that is, the integrated space VS is a virtual space in which the common real environment is reflected in the plurality of second spaces RS2. With such a configuration, it becomes possible to communicate between users located in each of the three or more spaces.

In the above embodiment, an example in which the spatial information management device is a server has been described. However, the spatial information management device includes a first information acquisition device 100, a second information acquisition device 200, a first display device 110, and a second information acquisition device. Any computer device may be used as long as it is connected to the display device 210 in a wired or wireless manner to exchange information.

The use of the space sharing system is not limited to the guidance of the second space RS2 by the first user P1 to the second user P2. If the first user P1 located in the first space RS1 needs to communicate with the second user P2 located in the second space RS2 while grasping the situation of the second space RS2, the space sharing system Can be applied.

For example, the space sharing system may be used to instruct the second user P2 by the first user P1 to work in the second space RS2. Examples of the work include construction and product assembly. In this case, the first guidance display Ge1 which is a guidance for the first user P1 and the second guidance display Ge2 which is a guidance for the second user P2 are displays showing the work procedure and the completion state of the intended work. May be. Specifically, when the space sharing system is used to instruct the work of assembling a plurality of members to complete the product, the completion image of the product completed in the second space RS2 is the first guidance display Ge1 or the second guidance display. It may be displayed as Ge2.

[Appendix]
Means for solving the above problems include the following supplementary notes as technical ideas derived from the above embodiments.

(Appendix 1)
When the marker provided at the reference position of the second space is recognized by the information acquisition device carried by the second user, the integrated space generation unit may generate the virtual image corresponding to the reference position of the second space. A spatial information management device that arranges the second object at a position corresponding to a position of the information acquisition device with respect to the reference position in the second space, with the position of the space as a reference.

According to the above configuration, the position of the second object is set based on the recognition of the marker provided at the reference position in the second space, so that the second object is set to the position of the second user in the second space. It is possible to easily and accurately arrange the position in the corresponding virtual space.

(Appendix 2)
The position of the inclusion space including the first space is associated with the position of the virtual space, and the instruction of the first user moves the first space relative to the inclusion space. As a result, it functions as an instruction to move the first object through changing the virtual position that is the position of the virtual space corresponding to the position of the first user in the first space,
The integrated space generation unit arranges the first object at the virtual position and moves the first object according to the change of the virtual position.

According to the above configuration, even when the first space RS1 is smaller than the second space RS2, the first object Ob1 can be moved to the entire area of the integrated space VS imitating the second space RS2.

(Appendix 3)
A first information acquisition device and a first display device arranged in the first space;
A second information acquisition device and a second display device arranged in the second space;
A space sharing system including the first information acquisition device, the second information acquisition device, the first display device, and a spatial information management device connected to the second display device,
The spatial information management device,
The first motion information indicating the instruction of the first user located in the first space is acquired from the first information acquisition device, and the second motion information indicating the motion of the second user located in the second space is acquired as the second motion information. 2. An acquisition unit that acquires from the information acquisition device,
A first object that is moved according to the instruction based on the first motion information and a second object that is moved according to the motion of the second user based on the second motion information are stored in a real environment of the second space. An integrated space generation unit that constructs an integrated space in which the first object is visibly arranged from the second object in one reflected virtual space;
First display data for causing the first display device to display the constituent elements of the integrated space from the viewpoint of the first object, and the first display data for the second display device from the viewpoint of the second object. An output data generation unit that generates second display data for displaying the object in the real environment of the second space in an overlapping manner;
An output unit that outputs the first display data to the first display device and outputs the second display data to the second display device.

According to the above configuration, the first object substituting for the first user and the second object substituting for the second user share one virtual space in the integrated space, and images from respective viewpoints in this integrated space Are provided to each user through the display device. Therefore, the positional relationship, movement, etc. of the constituent elements that reflect the actual space are shared in the integrated space, so that spatial information can be easily transmitted.

Dt1... 1st output data, Dt2... 2nd output data, Ob1... 1st object, Ob2... 2nd object, Im1... 1st movement information, Im2... 2nd movement information, Ir1... 1st information, Ir2... 2nd Information, Mg1... First output image, Mg2... Second output image, P1... First user, P2... Second user, RS1... First space, RS2... Second space, VB... Basic space, VS... Integrated space, 10... Information management server, 20... Communication unit, 30... Control unit, 31... First information processing unit, 32... Second information processing unit, 33... Integrated space generation unit, 34... Output data generation unit, 40... Storage unit , 41... Real space data, 42... Virtual space data, 43... Object data, 100... First information acquisition device, 110... First display device, 200... Second information acquisition device, 210... Second display device.

Claims (6)

An acquisition unit that acquires first motion information indicating an instruction from the first user located in the first space and second motion information indicating the motion of the second user located in the second space;
A first object that is moved according to the instruction based on the first motion information and a second object that is moved according to the motion of the second user based on the second motion information are stored in a real environment of the second space. An integrated space generation unit that constructs an integrated space in which the first object is visibly arranged from the second object in one reflected virtual space;
First display data for displaying the constituent elements of the integrated space from the viewpoint of the first object on the first display device located in the first space, and second display device located in the second space. An output data generation unit that generates second display data for displaying the first object in the real environment of the second space in an overlapping manner from the viewpoint of the second object,
An output unit that outputs the first display data to the first display device and outputs the second display data to the second display device;
And a space information management device used for guiding the second space by the first user to the second user,
The virtual space is a virtual space having an arrangement of a structure imitating the second space, and the structure includes an object imitating an exhibit in the second space,
The first display data is data for causing the first display device to display the components of the integrated space including the second object and the object imitating the exhibit as VR content,
The second display data is data for displaying the first object as an AR content in an overlapping manner in a real environment including the exhibit in the second space,
The first motion information includes a position, a direction, and a gesture instruction,
The second motion information includes the position and orientation of the second user, and gestures are excluded,
The integrated space generation unit arranges the second object at a position and orientation in the virtual space corresponding to the position and orientation of the second user in the second space.
Spatial information management device. The spatial information management device according to claim 1 , wherein the first motion information indicates at least one of an instruction by a movement of the first user itself and an instruction by an operation of the first user on an operating device. The first object has an appearance capable of expressing a change in position and orientation and a gesture,
The second object, spatial information management apparatus according to claim 1 or 2 having a representable only the position and orientation changes appearance. The output data generation unit generates data for displaying a guide for the first user on the first display device, and includes the data in the data output from the output unit to the first display device; and any from the output unit generates data for displaying a guidance for the second user to the second display device according to claim 1 to 3 for performing at least one of including the data to be output to the second display device The spatial information management device as described in 1 above. When the marker provided at the reference position of the second space is recognized by the information acquisition device carried by the second user, the integrated space generation unit causes the virtual space corresponding to the reference position of the second space. The second object is arranged at the position of the virtual space corresponding to the position of the information acquisition device with respect to the reference position in the second space, with the position of the space as a reference.
The spatial information management device according to claim 1. The position of the inclusion space including the first space is associated with the position of the virtual space, and the instruction of the first user moves the first space relative to the inclusion space. Thereby, it functions as an instruction to move the first object through changing the virtual position, which is the position of the virtual space corresponding to the position of the first user in the first space, in the inclusion space,
The integrated space generation unit arranges the first object at the virtual position and moves the first object according to the change of the virtual position.
The spatial information management device according to claim 1.