WO2018216327A1 - Information processing device, information processing method, and program - Google Patents

Abstract

[Problem] To provide an information processing device, an information processing method, and a program. [Solution] This information processing device is provided with a display control unit which, on the basis of relative positional information indicating a relative positional relationship with an imaging device present in real space, controls display such that an object indicating a remote user is visually recognized by a local user at a position in the real space that corresponds to the position of the imaging device.

Description

Information processing apparatus, information processing method, and program

The present disclosure relates to an information processing apparatus, an information processing method, and a program.

・ Communication tools for remote communication are used in various fields such as business, education, and entertainment. In such a communication tool, for example, a remote user (a user existing in a remote place) is photographed and presented to a local user existing in a local environment, thereby realizing natural communication closer to reality.

On the other hand, recently, a technique of presenting a virtual object to a user simultaneously with a real object existing in a real space, such as augmented reality (AR) technology, is spreading. For example, Patent Literature 1 below describes a technique for presenting an interaction between a virtual object and a real object.

International Publication No. 2014/171200

However, in the communication tools as described above, in many cases, the position of the camera that captures the local user is different from the position of the display device that displays the remote user. For this reason, there are cases where the eyes of each other may be felt as if they are not in line with each other, even though they are looking at each other. Even if the AR technology described above is applied to a communication tool and a remote user is presented as a virtual object in real space, depending on the position of the presented remote user, it may be felt that the lines of sight of each other do not match. is there. As a result, smooth communication may be hindered.

Therefore, the present disclosure proposes an information processing apparatus, an information processing method, and a program capable of realizing smoother communication in remote communication.

According to the present disclosure, an object indicating a remote user is located at a position in the real space according to a position of the imaging device based on relative position information indicating a relative positional relationship with the imaging device existing in the real space. An information processing apparatus including a display control unit that controls display so as to be visually recognized by a user is provided.

According to the present disclosure, an object indicating a remote user at a position in the real space according to the position of the imaging device based on relative position information indicating a relative positional relationship with the imaging device existing in the real space. An information processing method is provided that includes a processor controlling the display so that a local user can view the image.

In addition, according to the present disclosure, a remote user is placed at a position in the real space according to the position of the imaging device based on relative position information indicating a relative positional relationship with the imaging device existing in the real space. A program is provided for realizing a function of controlling display so that an object indicating is visually recognized by a local user.

As described above, according to the present disclosure, it is possible to realize smoother communication in remote communication.

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is explanatory drawing for demonstrating the outline | summary of the communication system which concerns on one Embodiment of this indication. It is explanatory drawing for demonstrating the outline | summary of the communication system which concerns on the embodiment. It is a block diagram which shows the structural example of the communication system which concerns on the same embodiment. It is a block diagram showing an example of composition of local terminal 10 concerning the embodiment. It is a mimetic diagram showing field of view V10 of a local user when display control part 135 concerning the embodiment displays a message. It is a block diagram showing an example of composition of remote terminal 20 concerning the embodiment. It is a flowchart figure which shows the operation example of the communication system 1 which concerns on the same embodiment. FIG. 12 is a schematic diagram illustrating a local user's field of view V10 when the display control unit 135 according to Modification 1 displays an avatar in the vicinity of the imaging device 30. FIG. 10 is a block diagram illustrating a configuration example of a communication system 1-2 according to Modification 2. It is explanatory drawing which shows the example of the animation which the display control part 135 which concerns on the modification displays. FIG. 10 is a block diagram illustrating a configuration example of a communication system 1-3 according to Modification 3. It is explanatory drawing which shows the example of the animation which the display control part 135 which concerns on the same modification displays the avatar of a some remote user. It is explanatory drawing which shows the hardware structural example.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numeral. However, when it is not necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given.

The description will be made in the following order.
<< 1. Overview >>
<< 2. Configuration >>
<2-1. Overall configuration>
<2-2. Configuration of local terminal>
<2-3. Configuration of remote terminal>
<< 3. Operation >>
<< 4. Modification >>
<4-1. Modification 1>
<4-2. Modification 2>
<4-3. Modification 3>
<4-4. Modification 4>
<< 5. Hardware configuration example >>
<< 6. Conclusion >>

<< 1. Overview >>
First, an outline of an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2. 1 and 2 are explanatory diagrams for describing an overview of a communication system according to an embodiment of the present disclosure. The communication system according to the present embodiment is an information processing system that realizes remote communication between a local user LU1 existing in the local environment 1000 shown in FIG. 1 and a remote user RU1 existing in the remote environment 2000.

The local environment 1000 and the remote environment 2000 may be any environment in a real space where a real object may exist. Further, the local environment 1000 and the remote environment 2000 may be far away, and for example, it may be a situation where direct face-to-face communication is difficult.

The local terminal 10 shown in FIG. 1 is an information processing apparatus that exists in the local environment 1000 and is used by the local user LU1. In the example shown in FIG. 1, the local terminal 10 has a transmissive (optical see-through) display unit arranged in front of one or both eyes of the local user LU1, and is mounted on the head of the local user LU1. Glasses-type device.

The remote terminal 20 shown in FIG. 1 is an information processing apparatus that exists in the remote environment 2000 and is used by the remote user RU1. In the example shown in FIG. 1, the remote terminal 20 has a display unit arranged in front of one or both of the eyes of the remote user RU1, and is an immersive HMD (Head Mounted Display) attached to the head of the remote user RU1. ).

The imaging device 30 shown in FIG. 1 is a so-called omnidirectional camera that can acquire 360-degree omnidirectional images in all directions in the vertical and horizontal directions by imaging. In this specification, an image is not limited to a still image, but is used as an expression including a moving image.

The local terminal 10, the remote terminal 20, and the imaging device 30 according to the present embodiment are connected to each other via a communication network (not shown). In addition, the communication system according to the present embodiment, for example, in addition to a message intercommunication function and a message output (display or sound output) function between the local terminal 10 and the remote terminal 20, can be remotely controlled by a display function described below. Realize communication. The apparatus used for remote communication is not limited to the example shown in FIG. 1, and examples of other apparatuses will be described later.

As shown in FIG. 1, the remote terminal 20 displays a display image G <b> 20 based on the imaging of the imaging device 30 existing in the local environment 1000. For example, the remote terminal 20 may generate a display image G20 by cutting out an area corresponding to the face direction of the remote user RU1 from the omnidirectional image acquired by imaging of the imaging device 30. With such a configuration, the remote user RU1 can observe the local environment 1000 in all directions from the viewpoint of the imaging device 30 by changing the face orientation. Also, as shown in FIG. 1, the remote user RU1 can obtain a sense of facing the local user LU1 by turning his face so that the local user LU1 is included in the display image G20.

On the other hand, the local terminal 10 displays a virtual object (hereinafter referred to as an avatar) indicating the remote user RU1. As described above, since the local terminal 10 has a transmissive display unit (not shown), the local user LU1 wearing the local terminal 10 visually recognizes the avatar together with the real space through the display unit of the local terminal 10. It is possible. Therefore, the local terminal 10 can obtain a feeling as if the avatar of the remote user RU1 exists in the real space in front of you.

As shown in FIG. 1, the field of view V10 viewed by the local user LU1 through the display unit of the local terminal 10 includes the avatar A1 of the remote user RU1 together with the real space. Further, the local terminal 10 according to the present embodiment has the avatar A1 at the position in the real space according to the position of the imaging device 30 based on the relative position information indicating the relative positional relationship with the imaging device 30 in the real space. Is displayed on the transmissive display unit. For example, in the field of view V10 shown in FIG. 1, the avatar A1 is superimposed on the imaging device 30 and is visually recognized by the local user LU1.

With this configuration, the local user LU1 turns his line of sight toward the imaging device 30 when he / she tries to see the avatar A1. As a result, when the remote user RU1 sees the local user LU1 in the display image G20 displayed on the remote terminal 20, the remote user RU1 feels that the line of sight is aligned with the local user LU1. can get.

Further, as described above, the local user LU1 feels as if the remote user RU1 exists at the position of the imaging device 30, and the remote user RU1 observes the local environment 1000 from the position of the imaging device 30. As described above, since the remote user RU1 sees an image based on the imaging of the imaging device 30, the smooth communication can be realized by the local user LU1 feeling as described above.

Also, the local terminal 10 may change the orientation (face orientation) of the face A11 of the avatar A1 to be displayed according to the face orientation of the remote user RU1. Information regarding the face orientation of the remote user RU1 may be acquired by a sensor included in the remote terminal 20, for example, and provided to the local terminal 10.

FIG. 2 shows a display example of the local terminal 10 and the remote terminal 20 when the remote user RU1 faces the right side. As described above, according to the face orientation of the remote user RU1, the region where the remote terminal 20 cuts out the display image from the omnidirectional image is also changed. Therefore, comparing FIG. 1 with FIG. Yes. Further, comparing FIG. 1 with FIG. 2, the orientation of the face A11 of the avatar A1 displayed by the local terminal 10 also changes according to the face orientation of the remote user RU1. By confirming the orientation of the face A11 of the avatar A1, the local user LU1 confirms the angle of view in the imaging device 30 corresponding to the field of view of the remote user RU1, that is, what range the remote user RU1 is viewing in the local environment 1000. Can be grasped. With this configuration, the local user LU1 can perform communication while grasping the scenery that the remote user RU1 is viewing, and can perform communication more smoothly.

Heretofore, an overview of the communication system according to an embodiment of the present disclosure has been described. Next, a configuration example for realizing the above-described effect by the communication system according to an embodiment of the present disclosure will be described.

<< 2. Configuration >>
FIG. 3 is a block diagram illustrating a configuration example of the communication system according to the present embodiment. As illustrated in FIG. 3, the communication system 1 according to the present embodiment includes an information processing system including a local terminal 10, a remote terminal 20, an imaging device 30, a sensor device 40, a distribution server 50, a sensor device 60, and a communication network 70. It is. In the following, after describing the overall configuration of the communication system 1, more detailed configurations of the local terminal 10 and the remote terminal 20 will be described.

<2-1. Overall configuration>
As shown in FIG. 3, the local terminal 10, the imaging device 30, and the sensor device 40 exist in the local environment 1000 described with reference to FIG. 1, and the remote terminal 20 and the sensor device 60 refer to FIG. Exists in the remote environment 2000 described.

The local terminal 10 according to the present embodiment is a glasses-type device that is worn on the head of a local user existing in the local environment 1000 as described with reference to FIG. In addition, the local terminal 10 has a transmissive display unit that displays a remote user's avatar at a position corresponding to the position of the imaging device 30 in real space. A more detailed configuration of the local terminal 10 will be described later with reference to FIG.

The remote terminal 20 according to the present embodiment is an immersive HMD attached to the head of a remote user existing in the remote environment 2000 as described with reference to FIG. Further, the remote terminal 20 displays an image based on the imaging of the imaging device 30. A more detailed configuration of the remote terminal 20 will be described later with reference to FIG.

As described with reference to FIG. 1, the imaging device 30 according to the present embodiment is an omnidirectional camera that can acquire 360-degree omnidirectional images in all directions in the vertical and horizontal directions by imaging. The imaging device 30 may have a plurality of imaging units, for example, and may acquire an omnidirectional image by performing image processing for combining images obtained by the plurality of imaging units. The imaging device 30 transmits the omnidirectional image to the distribution server 50 via the communication network 70. Note that the imaging apparatus 30 may not have an image processing function. In such a case, for example, another information processing apparatus connected to the imaging apparatus 30 or a distribution server 50 described later is associated with the imaging apparatus 30 instead. It may have an image processing function.

The sensor device 40 acquires sensing data related to the local environment 1000 by sensing. The sensor device 40 may include a plurality of sensors, for example, various sensors such as a camera, an infrared camera, a microphone, a depth sensor, an illuminance sensor, and a human sensor. The sensor device 40 provides (transmits) the acquired sensing data to the local terminal 10.

The distribution server 50 is an information processing apparatus that distributes (transmits) the omnidirectional image received from the imaging apparatus 30 via the communication network 70 to another apparatus (for example, the remote terminal 20). For example, the distribution server 50 may perform streaming distribution while caching the omnidirectional image received from the imaging device 30.

The sensor device 60 acquires sensing data related to the remote environment 2000 by sensing. The sensor device 60 may include a plurality of sensors, for example, various sensors such as a camera, an infrared camera, a microphone, a depth sensor, an illuminance sensor, and a human sensor. The sensor device 60 provides (transmits) the acquired sensing data to the remote terminal 20.

The communication network 70 is a wired or wireless transmission path for information transmitted from a device or system connected to the communication network 70. For example, the communication network 70 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), WAN (Wide Area Network), and the like. . Further, the communication network 70 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

The overall configuration of the communication system 1 according to the present embodiment has been described above. Next, configuration examples of the local terminal 10 and the remote terminal 20 included in the communication system 1 will be sequentially described.

<2-2. Configuration of local terminal>
FIG. 4 is a block diagram illustrating a configuration example of the local terminal 10 according to the present embodiment. As illustrated in FIG. 4, the local terminal 10 according to the present embodiment is an information processing apparatus including a sensor unit 11, a communication unit 12, a control unit 13, a display unit 14, a sound output unit 15, and a storage unit 16.

The sensor unit 11 acquires sensing data regarding the local user wearing the local terminal 10 and the surrounding environment (local environment 1000) by sensing. The sensor unit 11 may include, for example, an acceleration sensor, a gyro sensor, a camera, a microphone, a geomagnetic sensor, a force sensor, and the like. The sensor data acquired by the sensor unit 11 may include information on the position and orientation of the local terminal 10. The sensor unit 11 provides the acquired sensing data to the control unit 13.

The communication unit 12 is a communication interface that mediates communication between the local terminal 10 and other devices. The communication unit 12 supports an arbitrary wireless communication protocol or wired communication protocol, and establishes a communication connection with another device, for example, via the communication network 70 described with reference to FIG. 3 or directly.

The control unit 13 controls the operation of each component of the local terminal 10. The control unit 13 also functions as a communication control unit 131, a relative position acquisition unit 133, a display control unit 135, and a sound output control unit 137, as shown in FIG.

The communication control unit 131 shown in FIG. 4 controls communication by the communication unit 12, and acquires various types of information from other devices, or transmits them to other devices. For example, the communication control unit 131 causes the remote terminal 20 to transmit voice data acquired by a microphone included in the sensor unit 11 as a message. Further, the communication control unit 131 may cause text data input via an input device (not shown) to be transmitted to the remote terminal 20 as a message. Further, the communication control unit 131 may receive user information regarding the remote user wearing the remote terminal 20 from the remote terminal 20. The user information may include, for example, remote user identification information, remote user status information, remote user attitude information, messages (text data, voice data, etc.) transmitted by the remote user, and the like. Good. Further, the communication control unit 131 may receive sensing data from the sensor device 40 described with reference to FIG.

The relative position acquisition unit 133 acquires relative position information indicating a relative positional relationship with the imaging device 30 in real space. In the present embodiment, the relative position information may be information (coordinates) indicating the position of the imaging device 30 expressed in a coordinate system based on the current local terminal 10, for example. The relative position acquisition unit 133 can acquire the relative position information by various methods. Several examples of the relative position information acquisition method will be described below.

The relative position acquisition unit 133 may acquire the relative position information by detecting the imaging device 30 from the image acquired by the camera included in the sensor unit 11. In this case, in order to facilitate detection of the imaging device 30, the imaging device 30 may include a detection marker, or the imaging device 30 may include a light emitting unit, and the light emitting unit may emit light with a predetermined light emission pattern. Also good.

Alternatively, the relative position acquisition unit 133 may acquire the relative position information by detecting the local terminal 10 by image recognition from the omnidirectional image acquired by the imaging device 30. In this case, in order to facilitate detection of the local terminal 10, the local terminal 10 may be provided with a detection marker, or the local terminal 10 may include a light emitting unit, and the light emitting unit may emit light with a predetermined light emission pattern. Also good. Note that the local terminal 10 may acquire an omnidirectional image directly from the imaging device 30 or via the communication network 70 or the distribution server 50.

Alternatively, the relative position acquisition unit 133 acquires relative position information based on the coordinates of the imaging device 30 in the absolute coordinate system (hereinafter referred to as absolute coordinates) and the position and orientation information of the local terminal 10. It is also possible to do. The position and orientation information of the local terminal 10 may be included in the sensing data acquired by the sensor unit 11, or based on the sensing data, using self-position estimation technology such as SLAM (Simultaneous Localization and Mapping). It may be specified.

The absolute coordinates of the imaging device 30 may be stored, for example, in the storage unit 16 described later, or may be acquired from another device (for example, the imaging device 30) via the communication network 70.

Alternatively, the relative position acquisition unit 133 may detect the imaging device 30 from the image acquired by the camera included in the sensor device 40 and specify the absolute coordinates of the imaging device 30. In such a case, a detection marker may be provided in the imaging device 30, or the imaging device 30 may include a light emitting unit to facilitate detection by a light emission pattern. Alternatively, the relative position acquisition unit 133 may specify the absolute coordinates of the imaging device 30 by matching the omnidirectional image acquired by the imaging device 30 with an image obtained by capturing the local environment 1000 in advance. Good. By dynamically specifying the absolute coordinates of the imaging device 30 in this way, even when the imaging device 30 moves, it is possible to specify the absolute coordinates of the imaging device 30 and acquire the relative position information. It is.

As mentioned above, although several methods for acquiring relative position information by the relative position acquisition unit 133 have been described, the relative position information may be acquired by a method other than the above. For example, the relative position information may be acquired by detecting the imaging device 30 from sensing data other than the image acquired by the sensor unit 11. Alternatively, the relative position information may be acquired by the communication unit 12 receiving the relative position information specified by another device.

Based on the relative position information, the display control unit 135 is a transmissive display unit so that the local user can visually recognize the object indicating the remote user at a position in the real space corresponding to the position of the imaging device 30 in the real space. 14 controls display. The object indicating the remote user may be, for example, a virtual avatar, or may be an image of the remote user captured by the camera included in the sensor device 60 described with reference to FIG. Hereinafter, an example in which the display control unit 135 displays a remote user's avatar as an object indicating the remote user will be described.

For example, as described with reference to FIG. 1, the display control unit 135 may display the avatar of the remote user so that the imaging device 30 and the avatar in the real space are superimposed and viewed by the local user. Note that “the imaging device 30 and the avatar are superimposed and visually recognized” means that at least a part of the imaging device 30 overlaps at least a part of the avatar and is visually recognized by the local user.

With this configuration, the local user turns his / her line of sight toward the imaging device 30 when he / she tries to see the avatar of the remote user. Then, when the remote user sees the local user in the image displayed on the remote terminal 20, an effect is obtained in which the remote user feels that his / her line of sight matches the local user.

Further, the local user feels as if the remote user is present at the position of the imaging device 30 and the remote user observes the local environment 1000 with the position of the imaging device 30 as a viewpoint. As described with reference to FIG. 1, since the remote user sees an image based on the imaging of the imaging device 30 (an image displayed on the remote terminal 20), the local user can feel more as described above. Smooth communication can be realized.

When the remote user's avatar includes eyes (portion that looks like eyes), the display control unit 135 is visually recognized by superimposing an imaging unit (not shown) of the imaging device 30 and the eyes included in the avatar. As shown, an avatar may be displayed. With such a configuration, the local user can more easily imagine the viewpoint of the remote user, and smoother communication can be realized.

Further, the display control unit 135 may control the attitude of the avatar based on the attitude information indicating the attitude of the remote user received from the remote terminal 20 via the communication unit 12. For example, when the posture information includes information related to the face orientation of the remote user and the avatar includes a face (a part that appears to be a face), the display control unit 135 includes the information related to the face orientation. The direction of the face to be displayed may be controlled.

With this configuration, for example, as described with reference to FIGS. 1 and 2, the local user confirms the range of the local environment 1000 by the remote user by confirming the face direction of the avatar. It becomes possible. Then, the local user can communicate while grasping the scenery viewed by the remote user, and can communicate more smoothly.

Note that posture control by the display control unit 135 is not limited to face orientation control. For example, the information included in the posture information and the posture of the part corresponding to the human body included in the avatar can be controlled. For example, when the posture information includes a remote user's hand, arm, and body posture, the display control unit 135 may control the avatar's hand, arm, and body posture, respectively. With such a configuration, the local user can feel stronger as if the remote user exists in the local environment 1000.

Further, the display control unit 135 may display an avatar corresponding to the remote user. For example, a plurality of avatars are associated with remote user identification information and stored in the storage unit 16, and the display control unit 135 responds to the remote user identification information received from the remote terminal 20 via the communication unit 12. May be selected to display the selected avatar. With this configuration, the local user can identify the remote user via the avatar.

Further, the display control unit 135 may display a message transmitted by the remote user. Note that the message transmitted by the remote user is received from the remote terminal 20 via the communication unit 12, for example. The message displayed by the display control unit 135 is not limited to text, and may include electronic data such as image data and document data. In such a case, the display control unit 135 may display an icon indicating electronic data. When the electronic data is image data, the icon displayed by the display control unit 135 may be an image.

FIG. 5 is a schematic diagram showing the field of view V10 of the local user when the display control unit 135 displays a message. As shown in FIG. 5, the display control unit 135 displays the message M10 in the vicinity of the avatar A1. In the example shown in FIG. 5, the display control unit 135 displays the message M10 as a balloon from the position of the avatar A1. With this configuration, the local user can easily grasp the message issued by the remote user. When electronic data is included in the message, the display control unit 135 may display an icon M11 indicating electronic data as shown in FIG.

Further, the display control unit 135 may control the display according to the state of the remote user. For example, the display control unit 135 may display an indicator (for example, an icon or text) indicating the state of the remote user in the vicinity of the remote user's avatar. Further, the display control unit 135 may control parameters related to the display of the avatar according to the state of the remote user. Parameters relating to avatar display may include, for example, luminance, color, saturation, transparency, posture, and the like.

Note that information indicating the status of the remote user may be received from the remote terminal 20 via the communication unit 12. The information indicating the status of the remote user includes whether the remote user is online, whether the remote user is wearing the remote terminal 20, whether the remote user has permitted display of the avatar, and the like. Information may be included.

With this configuration, the local user can grasp the status of the remote user.

In addition, the display control unit 135 receives information on whether or not the image of the omnidirectional camera is normally transmitted to the remote terminal 20 from the distribution server 50 or the imaging device 30, and controls display based on the information. May be. In such a case, the display control unit 135 may display an icon or text indicating whether or not the image of the omnidirectional camera is normally transmitted to the remote terminal 20, or may control a parameter related to the display of the avatar. Good.

Further, the display control unit 135 may control the display according to whether or not the voice communication between the local terminal 10 and the remote terminal 20 is normally performed. In such a case, the display control unit 135 may display an icon or text indicating whether or not the voice communication between the local terminal 10 and the remote terminal 20 is normally performed, and controls parameters related to the display of the avatar. May be.

4 controls the sound output from the sound output unit 15. The sound output control unit 137 illustrated in FIG. For example, the sound output control unit 137 may control the sound output unit 15 to sound output a message transmitted by the remote user. In addition, the sound output control unit 137 may control the sound output so that the message can be heard from the position in the real space where the avatar is visually recognized by the local user. With this configuration, the local user can obtain a stronger feeling as if the remote user exists in the local environment 1000.

The display unit 14 is a display that is controlled by the display control unit 135 to display various information including objects such as avatars. As shown in FIG. 1, the display unit 14 may be a transmissive (optical see-through) display. With such a configuration, the local user wearing the local terminal 10 can view the real space and the information displayed on the display unit 14 at the same time. The display unit 14 may be a non-transparent display unit, and such a case will be described later as a modified example.

Further, the display unit 14 may be able to present different images to both eyes of the local user wearing the local terminal 10, for example, and the local user can display the shape of the displayed object, It is possible to recognize the position three-dimensionally.

The sound output unit 15 performs sound output under the control of the sound output control unit 137. The sound output unit 15 may include, for example, a plurality of speakers, and the sound output unit 15 may be capable of outputting sound in a three-dimensional manner (stereo sound).

The storage unit 16 stores a program for the control unit 13 to execute the above processes and various data. For example, the storage unit 16 may store the avatar information described above and a history of information received from the remote terminal 20 via the communication unit 12. The function of the storage unit 16 may exist in an external device, and the local terminal 10 may receive information stored in a storage unit included in the external device via the communication unit 12, for example.

The configuration example of the local terminal 10 has been described above with reference to FIG. 4, but the example illustrated in FIG. 4 is an example, and the present technology is not limited to the example. For example, some functions shown in FIG. 4 may be provided in another information processing apparatus connected to the local terminal 10. For example, some or all of the functions of the control unit 13 are provided in another information processing apparatus connected to the local terminal 10, and the local terminal 10 is controlled by the other information processing apparatus to display or output sound. You may go. In this case, the relative position information may be information indicating a relative position between the local terminal 10 that performs display and the imaging device 30.

<2-3. Configuration of remote terminal>
FIG. 6 is a block diagram illustrating a configuration example of the remote terminal 20 according to the present embodiment. As illustrated in FIG. 6, the remote terminal 20 according to the present embodiment is an information processing apparatus including a sensor unit 21, a communication unit 22, a control unit 23, a display unit 24, a sound output unit 25, and a storage unit 26.

The sensor unit 21 acquires sensing data related to the remote user wearing the remote terminal 20 and the surrounding environment (remote environment 2000) by sensing. The sensor unit 21 may include, for example, an acceleration sensor, a gyro sensor, a camera, a microphone, a geomagnetic sensor, a force sensor, and the like. The sensor data acquired by the sensor unit 21 may include posture information related to the posture (for example, the face orientation) of the remote user. The sensor unit 21 provides the acquired sensing data to the control unit 23.

The communication unit 22 is a communication interface that mediates communication between the remote terminal 20 and other devices. The communication unit 22 supports an arbitrary wireless communication protocol or wired communication protocol, and establishes a communication connection with another device, for example, via the communication network 70 described with reference to FIG.

The control unit 23 controls the operation of each component of the remote terminal 20. Further, as shown in FIG. 6, the control unit 23 also functions as a communication control unit 231, a display control unit 235, and a sound output control unit 237.

The communication control unit 231 illustrated in FIG. 6 controls communication by the communication unit 22 and acquires various types of information from other devices, or transmits them to other devices. For example, the communication control unit 231 transmits user information including information indicating the status of the remote user, attitude information of the remote user, messages (text data, voice data, and the like) transmitted by the remote user to the local terminal 10. Note that the communication control unit 231 may cause the voice data acquired by the microphone included in the sensor unit 21 to be transmitted as a message to the local terminal 10, or text data input via an input device (not shown) as a message. May be transmitted to the local terminal 10. Further, the communication control unit 231 may receive a message (text data, voice data, etc.) transmitted from the local terminal 10 by the local user. In addition, the communication control unit 231 may receive the omnidirectional image from the distribution server 50 described with reference to FIG. The communication control unit 231 may receive sensing data from the sensor device 60 described with reference to FIG.

The display control unit 235 controls display by the display unit 24. For example, the display control unit 235 may generate a display image by cutting out a region corresponding to the face orientation of the remote user acquired by the sensor unit 21 from the omnidirectional image, and display the display image on the display unit 24. .

With this configuration, a remote user wearing the remote terminal 20 can observe the local environment 1000 in all directions from the viewpoint of the imaging device 30 by changing the face orientation. Further, as shown in FIG. 1, the remote user can obtain a sense of facing the local user by facing the face in a direction that is included in the display image.

Note that the display control unit 235 may further perform image processing on the area cut out from the omnidirectional image to generate a display image. For example, the image processing performed by the display control unit 235 includes a processing process that detects a local user from an area cut out from the omnidirectional image and processes the local user area, or converts the local user area into another object (for example, A replacement process may be included. The processing performed by the display control unit 235 may include, for example, processing for removing the local terminal 10 worn by the local user and restoring the facial expression of the local user. In addition, the image processing performed by the display control unit 235 may include processing for synthesizing a message transmitted by the local user.

6 controls the sound output by the sound output unit 25. The sound output control unit 237 illustrated in FIG. For example, the sound output control unit 237 may control the sound output unit 25 to sound output a message transmitted by the local user. In addition, the sound output control unit 237 may control the sound output so that a message can be heard from the position of the local user displayed on the display unit 24. With this configuration, the remote user can obtain a more immersive feeling.

The display unit 24 is a display that displays a display image under the control of the display control unit 235. The display unit 24 may be able to present different images to both eyes of a remote user wearing the remote terminal 20, for example. With such a configuration, the remote user can observe the local environment 1000 in a three-dimensional manner, and can obtain a more immersive feeling.

The sound output unit 25 performs sound output under the control of the sound output control unit 237. The sound output unit 25 may include, for example, a plurality of speakers, and the sound output unit 25 may be capable of outputting sound in a three-dimensional manner (stereo sound).

The storage unit 26 stores a program for the control unit 23 to execute the above processes and various data.

The configuration example of the remote terminal 20 has been described above with reference to FIG. 6, but the example illustrated in FIG. 6 is an example, and the present technology is not limited to the example. For example, some functions shown in FIG. 6 may be provided in another information processing apparatus connected to the remote terminal 20. For example, a part or all of the functions of the control unit 23 shown in FIG. 6 are provided in another information processing apparatus connected to the remote terminal 20, and the remote terminal 20 displays and is controlled by the other information processing apparatus. Or sound output may be performed.

<< 3. Operation >>
The configuration examples of the communication system 1, the local terminal 10, and the remote terminal 20 according to the present embodiment have been described above. Next, an operation example of the communication system 1 according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing an operation example of the communication system 1 according to the present embodiment.

As shown in FIG. 7, first, imaging is performed by the imaging device 30 (S102), and then image processing such as synthesis processing based on the captured image is performed, and an omnidirectional image is acquired (S104). Subsequently, the omnidirectional image is transmitted from the imaging device 30 to the distribution server 50 (S108). Further, the distribution server 50 distributes (transmits) the omnidirectional image received from the imaging device 30 to the remote terminal 20 (S108).

Subsequently, the sensor unit 21 of the remote terminal 20 acquires sensing data including, for example, information on the face direction of the remote user by sensing (S110). Further, the display control unit of the remote terminal 20 cuts out an area corresponding to the remote user's face direction from the omnidirectional image, generates a display image, and causes the display unit 24 to display the display image (S112).

Subsequently, the remote terminal 20 locally stores user information including remote user identification information, information indicating the status of the remote user, attitude information of the remote user, messages (text data, voice data, etc.) transmitted by the remote user, and the like. It transmits to the terminal 10 (S114).

Subsequently, the relative position acquisition unit 133 of the local terminal 10 acquires relative position information indicating the relative positional relationship between the local terminal 10 and the imaging device existing in the real space (S116). Further, the display control unit 135 of the local terminal 10 recognizes the remote user's avatar by the local user at a position in the real space according to the position of the imaging device 30 in the real space based on the user information and the relative position information. As shown, the avatar is displayed (S118).

The operation example of the communication system 1 according to the present embodiment has been described above. However, FIG. 7 shows an example, and the present technology is not limited to the example. For example, the processes in steps S102 to S118 shown in FIG. 7 may be repeated. Moreover, the processing order of step S108 and step S110 may be reversed, and the processing order of step S114 and step S116 may be reversed.

<< 4. Modification >>
The embodiment of the present disclosure has been described above. Below, some modifications of this embodiment are explained. Each modified example described below may be applied to the present embodiment alone, or may be applied to the present embodiment in combination. Each modification may be applied instead of the configuration described in the present embodiment, or may be additionally applied to the configuration described in the present embodiment.

<4-1. Modification 1>
In the above-described embodiment, the example in which the display control unit 135 displays the avatar so that the avatar of the remote user and the imaging device 30 are superimposed and viewed has been described, but the present technology is not limited to the example. For example, the display control unit 135 may display an avatar in the vicinity of the imaging device 30. Such an example will be described as a first modification with reference to FIG.

FIG. 8 is a schematic diagram showing the field of view V10 of the local user when the display control unit 135 displays an avatar in the vicinity of the imaging device 30. In the example illustrated in FIG. 8, the remote user's avatar A <b> 1 is displayed beside the imaging device 30. Note that the position where the avatar is visually recognized is not limited to the side of the imaging device 30 and may be above, below, inside, or in front of the imaging device 30.

<4-2. Modification 2>
In the above-described embodiment, an example in which the communication system 1 includes one imaging device 30 has been described. However, the present technology is not limited to such an example. For example, a plurality of imaging devices 30 may be provided. Such an example will be described as a second modification with reference to FIGS.

FIG. 9 is a block diagram showing a configuration example of the communication system 1-2 according to this modification. The configuration of the communication system 1-2 illustrated in FIG. 9 is the same as the configuration of the communication system 1 described with reference to FIG. 3 except that the two imaging devices 30A and 30B exist in the local environment 1000. Therefore, it will be described while omitting as appropriate.

The imaging device 30A and the imaging device 30B are omnidirectional cameras that can acquire 360-degree omnidirectional images in all directions in the vertical and horizontal directions by imaging, as with the imaging device 30 described above. In addition, the imaging device 30 </ b> A and the imaging device 30 </ b> B transmit an omnidirectional image to the distribution server 50 via the communication network 70.

The remote terminal 20 according to the present modification may display an image based on the imaging of one imaging device selected by the remote user from the imaging device 30A or the imaging device 30B. The remote user may perform an input operation related to the selection via, for example, the sensor device 60, the sensor unit 21 of the remote terminal 20, or an input device (not shown). Note that the distribution server 50 may transmit both the omnidirectional image captured by the imaging device 30A and the omnidirectional image captured by the imaging device 30B to the remote terminal 20, or the imaging selected by the remote user. Only the omnidirectional image captured by the apparatus may be transmitted to the remote terminal 20.

Further, the local terminal 10 according to this modification may have the following functions in addition to the functions described with reference to FIG.

For example, the display control unit 135 of the local terminal 10 according to the present modification has the remote user's avatar at a position in the real space corresponding to the position of one imaging device selected by the remote user among the plurality of imaging devices. The display is controlled so as to be visually recognized by the local user.

In addition, when the position where the avatar is visually recognized is switched according to the selection of the remote user, the display control unit 135 according to this modification displays the avatar so that the avatar fades out and then fades in to a new position. You may let them.

Further, the display control unit 135 according to the present modification is currently selected from the position corresponding to the imaging device selected immediately before when the position where the avatar is visually recognized is switched according to the selection of the remote user. You may display the animation which an avatar moves to the position according to an imaging device. In this animation, a video effect in which the avatar moves three-dimensionally may be added.

FIG. 10 is an explanatory diagram showing an example of an animation displayed by the display control unit 135 according to this modification. FIG. 10 shows a state in which the imaging device 30B is selected by the remote user RU1 after the imaging device 30A has been selected. The remote terminal 20 displays a display image G20 generated by cutting out an area corresponding to the face direction of the remote user RU1 from the omnidirectional image acquired by imaging of the imaging device 30B. Further, in the field of view V10 of the local user LU1, the avatar A1 that has been visually recognized while being superimposed with the imaging device 30A has moved to a position that is visually recognized with being superimposed with the imaging device 30B.

With such a configuration, the remote user can observe the local environment 1000 from various viewpoints, and the local user can easily grasp the imaging device corresponding to the current viewpoint of the remote user. .

<4-3. Modification 3>
In the above-described embodiment, an example in which a local user and a remote user perform one-to-one communication has been described. However, the present technology is not limited to such an example. For example, a plurality of remote users may exist for one local user. Such an example will be described as a third modification with reference to FIGS. 11 and 12.

FIG. 11 is a block diagram showing a configuration example of the communication system 1-3 according to this modification. The configuration of the communication system 1-3 illustrated in FIG. 11 is partially the same as the configuration of the communication system 1 described with reference to FIG.

As shown in FIG. 11, the communication system 1-3 according to this modification includes two remote terminals 20A and 20B, and is shown in FIG. 3 in that it includes two sensor devices 60A and 60B. Different from communication system 1. Further, the remote terminal 20A and the sensor device 60A exist in the remote environment 2000A, and the remote terminal 20B and the sensor device 60B exist in the remote environment 2000B. The remote environment 2000A and the remote environment 2000B may be the same (single) environment. In such a case, the sensor device 60A and the sensor device 60B may be the same (single) device.

Further, the remote terminal 20A may be an information processing device used by a first remote user, and the remote terminal 20B may be an information processing device used by a second remote user different from the first remote user. Note that the configurations of the remote terminal 20A and the remote terminal 20B according to this modification are substantially the same as the configuration of the remote terminal 20 described with reference to FIG.

Also, the distribution server 50 according to this modification distributes (transmits) the omnidirectional image obtained by the imaging of the imaging device 30 to both the remote terminal 20A and the remote terminal 20B.

Further, the local terminal 10 according to this modification may have the following functions in addition to the functions described with reference to FIG. For example, the display control unit 135 of the local terminal 10 according to this modification may display an object (for example, an avatar) indicating a plurality of remote users.

FIG. 12 is an explanatory diagram showing an example of an animation in which the display control unit 135 according to this modification displays a plurality of remote user avatars. In the example shown in FIG. 12, the display control unit 135 displays both the avatar A1 of the first remote user RU1 wearing the remote terminal 20A and the avatar A2 of the second remote user RU2 wearing the remote terminal 20B. ing.

Note that if a plurality of avatars are displayed so as to be viewed in a superimposed manner with the imaging device 30, the plurality of avatars are overlapped. Therefore, as shown in FIG. 12, the display control unit 135 overlaps with other avatars. An avatar may be displayed so as to be visually recognized in the vicinity of the imaging device 30 that should not be. Note that the display control unit 135 may control the display so that a plurality of avatars do not overlap with each other and can be visually recognized while being superimposed on the imaging device 30 by displaying the avatar in a reduced size.

The display control unit 135 of the local terminal 10 has a function of displaying an avatar corresponding to the remote user as described above, and in the example illustrated in FIG. 12, the avatar A1 and the avatar A2 are different avatars. The display control unit 135 may control the avatar posture of each remote user in accordance with the posture information of each remote user. In the example shown in FIG. 12, the face of the avatar A1's face A11 is displayed in a direction corresponding to the face direction of the remote user RU1, and the face of the avatar A2's face A21 is displayed in a direction corresponding to the face direction of the remote user RU2. Yes.

The remote user RU1 existing in the remote environment 2000A is wearing the remote terminal 20A. Further, on the remote terminal 20A, a display image G21 generated by cutting out an area corresponding to the face orientation of the remote user RU1 from the omnidirectional image obtained by imaging by the imaging device 30 is displayed.

The remote user RU2 existing in the remote environment 2000B is wearing the remote terminal 20B. In addition, the remote terminal 20B displays a display image G22 generated by cutting out an area corresponding to the face direction of the remote user RU2 from the omnidirectional image obtained by imaging by the imaging device 30.

Note that, based on the position of each remote user's avatar in the local environment 1000, each remote terminal 20 may display the avatar of another remote user. For example, when the remote user RU1 turns to the left, the remote terminal 20A displays, as a display image, an image obtained by combining the avatar of the remote user RU2 with an image obtained by cutting out an area corresponding to the face direction from the omnidirectional image. May be.

In such a case, the remote terminal 20A displays a message in the vicinity of the avatar of the remote user RU2, or outputs a sound so that a message transmitted by the remote user RU2 can be heard from the position of the avatar of the remote user RU2. Also good.

Note that this modification may be combined with Modification 2 described above. For example, a plurality of remote users select one imaging device from a plurality of imaging devices, and the display control unit 135 of the local terminal 10 displays the remote user's avatar at a position corresponding to the position of the imaging device selected by each remote user. The display may be controlled so as to be visually recognized.

Further, each remote terminal 20 may display the avatar of the other remote user at a position corresponding to the position of the imaging device selected by the remote user other than the remote user wearing the remote terminal 20. In such a case, the remote terminal 20 displays a message in the vicinity of the avatar of the other remote user, or outputs a sound so that a message transmitted by the remote user can be heard from the position of the avatar of the remote user. Also good.

<4-4. Modification 4>
In the above embodiment, an example has been described in which the local terminal 10 is a glasses-type device having a transmissive display unit, the remote terminal 20 is an immersive HMD, and the imaging device 30 is an omnidirectional camera. The embodiment according to the invention is not limited to the example, and can be realized by various apparatus configurations.

For example, the display unit 14 included in the local terminal 10 may not be a transmissive type. In such a case, an avatar (an example of an object indicating a remote user) is combined with a real space image that is included in the sensor unit 11 included in the local terminal 10 and acquired by a camera that captures the field of view (real space) of the local user. May be displayed. With such a configuration, even when the display unit 14 is not a transmissive type, the display control unit 135 of the local terminal 10 allows the local user to visually recognize the avatar at a position in the real space according to the position of the imaging device 30. It is possible to control the display.

Alternatively, the local terminal 10 may be a smartphone or a tablet terminal. Even in such a case, a real space image and an avatar acquired by a camera included in the sensor unit 11 may be combined and displayed.

Alternatively, the display unit 14 of the local terminal 10 may be a projector. In such a case, the local terminal 10 may project an avatar image at a position in the real space according to the position of the imaging device 30. With such a configuration, even when the display unit 14 is a projector, the display control unit 135 of the local terminal 10 controls the projector so that the avatar is positioned at a position in the real space according to the position of the imaging device 30. The display can be controlled to be viewed by a local user.

Further, the display unit 24 of the remote terminal 20 may be a flat display. In such a case, the area corresponding to the field of view of the remote user may be specified according to an operation via an input device such as a remote controller or a gesture operation acquired by the sensor unit 21.

Further, the remote terminal 20 may be a smartphone or a tablet terminal. Even in such a case, the region corresponding to the visual field of the remote user may be specified according to the touch operation or the posture of the remote terminal 20 acquired by the sensor unit 21.

Further, the imaging device 30 may not be a spherical camera, and may be a camera having an imaging angle of view of 180 degrees, for example. Then, the display control unit 135 of the local terminal 10 may limit the face direction of the avatar to be displayed according to the imaging angle of view of the imaging device 30. With such a configuration, it is possible to suppress inconsistency between the avatar's face direction and the field of view actually seen by the remote user.

<< 5. Hardware configuration >>
The embodiment of the present disclosure has been described above. Finally, the hardware configuration of the information processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 13 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus according to the present embodiment. Note that the information processing apparatus 900 illustrated in FIG. 13 can implement the local terminal 10, the remote terminal 20, and the distribution server 50, for example. Information processing by the local terminal 10, the remote terminal 20, and the distribution server 50 according to the present embodiment is realized by cooperation of software and hardware described below.

As shown in FIG. 13, the information processing apparatus 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. The information processing apparatus 900 includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, a communication device 913, and a sensor 915. The information processing apparatus 900 may include a processing circuit such as a DSP or an ASIC in place of or in addition to the CPU 901.

The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation in the information processing apparatus 900 according to various programs. Further, the CPU 901 may be a microprocessor. The ROM 902 stores programs used by the CPU 901, calculation parameters, and the like. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. For example, the CPU 901 can form the control unit 13 and the control unit 23.

The CPU 901, ROM 902, and RAM 903 are connected to each other by a host bus 904a including a CPU bus. The host bus 904 a is connected to an external bus 904 b such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 904. Note that the host bus 904a, the bridge 904, and the external bus 904b do not necessarily have to be configured separately, and these functions may be mounted on one bus.

The input device 906 is realized by a device in which information is input by the user, such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever. The input device 906 may be, for example, a remote control device using infrared rays or other radio waves, or may be an external connection device such as a mobile phone or a PDA that supports the operation of the information processing device 900. . Furthermore, the input device 906 may include, for example, an input control circuit that generates an input signal based on information input by the user using the above-described input means and outputs the input signal to the CPU 901. A user of the information processing apparatus 900 can input various data and instruct a processing operation to the information processing apparatus 900 by operating the input device 906.

The output device 907 is formed of a device that can notify the user of the acquired information visually or audibly. Examples of such devices include CRT display devices, liquid crystal display devices, plasma display devices, EL display devices, display devices such as lamps, audio output devices such as speakers and headphones, printer devices, and the like. For example, the output device 907 outputs results obtained by various processes performed by the information processing device 900. Specifically, the display device visually displays results obtained by various processes performed by the information processing device 900 in various formats such as text, images, tables, and graphs. On the other hand, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, and the like into an analog signal and outputs it aurally. The output device 907 can form, for example, the display unit 14, the sound output unit 15, the display unit 24, and the sound output unit 25.

The storage device 908 is a data storage device formed as an example of a storage unit of the information processing device 900. The storage apparatus 908 is realized by, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like. The storage device 908 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 908 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like. The storage device 908 can form the storage unit 16 and the storage unit 26, for example.

The drive 909 is a storage medium reader / writer, and is built in or externally attached to the information processing apparatus 900. The drive 909 reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and outputs the information to the RAM 903. The drive 909 can also write information to a removable storage medium.

The connection port 911 is an interface connected to an external device, and is a connection port with an external device capable of transmitting data by USB (Universal Serial Bus), for example.

The communication device 913 is a communication interface formed by a communication device or the like for connecting to the network 920, for example. The communication device 913 is, for example, a communication card for wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 913 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communication, or the like. The communication device 913 can transmit and receive signals and the like according to a predetermined protocol such as TCP / IP, for example, with the Internet and other communication devices. The communication device 913 can form the communication unit 12 and the communication unit 22, for example.

The sensor 915 is various sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, a sound sensor, a distance measuring sensor, and a force sensor. The sensor 915 acquires information on the state of the information processing apparatus 900 itself, such as the posture and movement speed of the information processing apparatus 900, and information on the surrounding environment of the information processing apparatus 900, such as brightness and noise around the information processing apparatus 900. . Sensor 915 may also include a GPS sensor that receives GPS signals and measures the latitude, longitude, and altitude of the device. The sensor 915 can form the sensor part 11 and the sensor part 21, for example.

Note that the network 920 is a wired or wireless transmission path for information transmitted from a device connected to the network 920. For example, the network 920 may include a public line network such as the Internet, a telephone line network, and a satellite communication network, various LANs including the Ethernet (registered trademark), a wide area network (WAN), and the like. Further, the network 920 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

Heretofore, an example of the hardware configuration capable of realizing the functions of the information processing apparatus 900 according to the present embodiment has been shown. Each of the above components may be realized using a general-purpose member, or may be realized by hardware specialized for the function of each component. Therefore, it is possible to change the hardware configuration to be used as appropriate according to the technical level at the time of carrying out this embodiment.

It should be noted that a computer program for realizing each function of the information processing apparatus 900 according to the present embodiment as described above can be produced and mounted on a PC or the like. In addition, a computer-readable recording medium storing such a computer program can be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

<< 6. Conclusion >>
As described above, according to the embodiment of the present disclosure, it is possible to realize smoother communication in remote communication.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

In the above embodiment, the example in which the local terminal 10 displays the avatar that is a virtual object as the object indicating the remote user has been described, but the present technology is not limited to the example. For example, the local terminal 10 may display an image obtained by photographing the remote user with a camera included in the sensor device 60 as an object indicating the remote user.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. An information processing apparatus comprising a display control unit that controls display.
(2)
The information processing apparatus according to (1), wherein the display control unit displays the object so that the imaging apparatus and the object are superimposed and viewed.
(3)
The information processing apparatus according to (2), wherein the display control unit displays the object so that an imaging unit of the imaging apparatus and eyes included in the object are superimposed and viewed.
(4)
The information processing apparatus according to (1), wherein the display control unit displays the object so that the object is visually recognized in the vicinity of the imaging apparatus.
(5)
The information processing apparatus according to any one of (1) to (4), wherein the display control unit controls the posture of the object based on posture information of the remote user.
(6)
The information processing apparatus according to (5), wherein the display control unit controls a face orientation included in the object according to information related to the face orientation of the remote user included in the posture information.
(7)
The information processing apparatus according to any one of (1) to (6), wherein the display control unit displays the object according to the remote user.
(8)
The information processing apparatus according to any one of (1) to (7), wherein the display control unit displays a message transmitted by the remote user in the vicinity of the object.
(9)
The information processing apparatus according to any one of (1) to (8), wherein the display control unit controls display according to a state of the remote user.
(10)
The information processing apparatus according to (9), wherein the display control unit displays an indicator indicating the state of the remote user in the vicinity of the object.
(11)
The information processing apparatus according to (10), wherein the display control unit controls a parameter related to display of the object according to a state of the remote user.
(12)
The display control unit controls display so that the object is visually recognized at a position in the real space according to the position of one imaging device selected from a plurality of imaging devices. The information processing apparatus according to any one of (11).
(13)
When the position where the object is visually recognized is switched according to the selection, the display control unit changes from the position corresponding to the imaging device selected immediately before to the position corresponding to the currently selected imaging device. The information processing apparatus according to (12), wherein an animation of moving the object is displayed.
(14)
The information processing apparatus according to any one of (1) to (13), wherein the display control unit displays an object indicating a plurality of remote users.
(15)
An acoustic output control unit that acoustically outputs a message transmitted by the remote user, and the acoustic output control unit controls the acoustic output so that the message can be heard from a position in the real space where the object is visually recognized. The information processing apparatus according to any one of (1) to (14).
(16)
The information processing apparatus according to any one of (1) to (15), wherein the imaging apparatus is an omnidirectional camera.
(17)
The information processing apparatus according to any one of (1) to (16), wherein the display control unit controls display of a transmissive display unit.
(18)
The information processing apparatus according to any one of (1) to (16), wherein the display control unit controls a projector.
(19)
Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. And an information processing method including: a processor controlling display.
(20)
On the computer,
Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. Program for realizing the function to control the display.

DESCRIPTION OF SYMBOLS 1 Communication system 10 Local terminal 11 Sensor part 12 Communication part 13 Control part 14 Display part 15 Sound output part 16 Storage part 20 Remote terminal 21 Sensor part 22 Communication part 23 Control part 24 Display part 25 Sound output part 26 Storage part 30 Imaging device 40, 60 Sensor device 50 Distribution server 60 Sensor device 70 Communication network 131 Communication control unit 133 Relative position acquisition unit 135 Display control unit 137 Sound output control unit 231 Communication control unit 235 Display control unit 237 Sound output control unit 1000 Local environment 2000 Remote environment

Claims (20)

1. Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. An information processing apparatus comprising a display control unit that controls display.
2. The information processing apparatus according to claim 1, wherein the display control unit displays the object so that the imaging apparatus and the object are superimposed and viewed.
3. The information processing apparatus according to claim 2, wherein the display control unit displays the object so that the imaging unit of the imaging apparatus and the eyes included in the object are superimposed and viewed.
4. The information processing apparatus according to claim 1, wherein the display control unit displays the object so that the object is visually recognized in the vicinity of the imaging apparatus.
5. The information processing apparatus according to claim 1, wherein the display control unit controls the posture of the object based on posture information of the remote user.
6. The information processing apparatus according to claim 5, wherein the display control unit controls a face orientation included in the object in accordance with information related to the face orientation of the remote user included in the posture information.
7. The information processing apparatus according to claim 1, wherein the display control unit displays the object corresponding to the remote user.
8. The information processing apparatus according to claim 1, wherein the display control unit displays a message transmitted by the remote user in the vicinity of the object.
9. The information processing apparatus according to claim 1, wherein the display control unit controls display according to a state of the remote user.
10. The information processing apparatus according to claim 9, wherein the display control unit displays an indicator indicating a state of the remote user in the vicinity of the object.
11. The information processing apparatus according to claim 10, wherein the display control unit controls a parameter related to display of the object according to a state of the remote user.
12. 2. The display control unit according to claim 1, wherein the display control unit controls display so that the object is visually recognized at a position in the real space according to a position of one imaging apparatus selected from a plurality of imaging apparatuses. Information processing device.
13. When the position where the object is visually recognized is switched according to the selection, the display control unit changes from the position corresponding to the imaging device selected immediately before to the position corresponding to the currently selected imaging device. The information processing apparatus according to claim 12, wherein an animation of moving the object is displayed.
14. The information processing apparatus according to claim 1, wherein the display control unit displays an object indicating a plurality of remote users.
15. An acoustic output control unit that acoustically outputs a message transmitted by the remote user, and the acoustic output control unit controls the acoustic output so that the message can be heard from a position in the real space where the object is visually recognized. The information processing apparatus according to claim 1.
16. The information processing apparatus according to claim 1, wherein the imaging apparatus is an omnidirectional camera.
17. The information processing apparatus according to claim 1, wherein the display control unit controls display of a transmissive display unit.
18. The information processing apparatus according to claim 1, wherein the display control unit controls a projector.
19. Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. And an information processing method including: a processor controlling display.
20. On the computer,
    Based on relative position information indicating a relative positional relationship with the imaging device existing in the real space, an object indicating the remote user is visually recognized by the local user at a position in the real space corresponding to the position of the imaging device. Program for realizing the function to control the display.

Images