JP2023014867A - Communication system and information processor - Google Patents

Abstract

To provide a communication system and an information processor, capable of implementing realistic communication.SOLUTION: A communication system comprises a remote location device and a target location device. The remote location device is disposed at a remote location apart from a target location and includes a first camera for imaging the remote location and a first display unit. The target location device is a mobile body that is disposed at the remote location and includes a second camera for imaging the target location and a second display unit. The target location device displays first video information imaged by the first camera of the remote location device on the second display unit and, on the basis of information detected from a user existing at the remote location, moves or actuates the second camera. The remote location device displays second video information imaged by the second camera of the target location device on the first display unit.SELECTED DRAWING: Figure 3

Description

The present invention relates to communication systems and information processing apparatuses.

Conventionally, for example, technology such as VR (Virtual Reality) has been proposed to allow remote users to interact with each other in a realistic manner (see, for example, Patent Literature 1).

Japanese Patent Publication No. 2019-537087

However, the conventional technology has room for further improvement in terms of improving the reality of communication.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a communication system and an information processing apparatus capable of realizing communication with reality.

To solve the above problems and achieve the object, a communication system according to the present invention comprises a remote device and a target device. The remote device is arranged at a remote location away from a target location, and has a first camera that captures an image of the remote location, and a first display unit. The target site device is a moving body that is arranged at the target site and has a second camera that captures an image of the target site and a second display unit. The target device displays, on the second display unit, first video information captured by the first camera of the remote device, and based on information detected by the user at the remote location, By moving or operating the second camera, the remote device displays second image information captured by the second camera of the target device on the first display unit.

According to the present invention, communication with reality can be realized.

FIG. 1 is a diagram showing an operation example of a communication system according to an embodiment. FIG. 2 is a diagram showing the appearance of the target device. FIG. 3 is a block diagram illustrating a functional configuration example of the communication system according to the embodiment; FIG. 4 is a sequence diagram illustrating a processing procedure of processing executed by the communication system according to the embodiment;

Hereinafter, embodiments of a communication system and an information processing apparatus disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

First, an operation example of the communication system will be described with reference to FIG. FIG. 1 is a diagram showing an operation example of a communication system according to an embodiment.

As shown in FIG. 1, a communication system S according to the embodiment includes a remote device 1 located at a remote location P1 and a target location device 100 located at a target location P2. Here, the remote location P1 is, for example, a predetermined space (for example, a room at home) where the user U1 exists, and is a location away from the target location P2. The target site P2 is a space (an amusement park in FIG. 1) where the user U2 who is the dialogue target of the user U1 exists.

The communication system S according to the embodiment is a system in which a user U1 existing at a remote location P1 can interact with a user U2 as if he/she exists at a target location P2 and enjoy the scenery of the target location P2.

Specifically, the remote device 1 includes at least a control device 3 , a first camera 21 and a first display section 4 . The target device 100 is a mobile body, and includes at least a second camera 120 (see FIG. 2) and a second display section 140 .

The remote device 1 and the target device 100 realize communication between the user U1 and the user U2 by transmitting and receiving video and audio to each other. Specifically, the remote device 1 detects the orientation of the face of the user U1, detects a movement operation for moving the target device 100, which is a mobile body, via the operation unit 5, and detects the movement of the target device 100. 100.

Then, the target device 100 moves or operates the second camera 120 based on the information detected from the user U1. Specifically, the target device 100 changes (operates) the orientation of the second camera 120 according to the orientation of the face of the user U1, and moves according to the movement operation. Note that the overall orientation of the target device 100 may change according to the movement operation, and the second camera 120 may also change according to the orientation of the face of the user U1. Then, the target device 100 transmits video information (second video information) captured by the second camera 120 to the remote device 1 . As a result, the remote device 1 displays the second video information on the first display section 4 . Note that the control device 3 of the remote device 1 has a function of a projector that projects the second image information on the first display section 4, which is a screen. Therefore, the first display unit 4 may be a facility such as a screen, or may be a wall or the like at the remote location P1.

That is, in the target device 100, the second camera 120 serves as the eyes of the user U1, and the movement function serves as the feet of the user U1. As a result, the user U1 can freely move around the target site P2 and view the video in the desired direction from the remote site P1. That is, the user U1 can feel the reality as if the user U1 exists in the target site P2.

In addition, the target device 100 displays the user U1 on the second display unit 140, and displays the image on the second display unit 140 in the orientation of the user U1 according to the position of the user U2 (the direction in which the second display unit 140 is viewed). do. Specifically, the target device 100 detects the positional relationship between the user U2 and the target device 100 and transmits it to the remote device 1 . Then, the remote device 1 moves the first camera 21 so that the positional relationship between the user U2 and the target device 100 and the positional relationship between the user U1 and the first camera 21 are the same. An image is captured and transmitted to the target device 100 as the first image information. Accordingly, the target device 100 displays the first image information on the second display unit 140 .

That is, the remote device 1 captures an image of the user U1 with the first camera 21 from the direction in which the user U2 is looking at the user U1 (the user U1 displayed on the second display unit 140 of the target device 100), and the first image It is transmitted to the target site device 100 as information. Thereby, the image (first image information) directed to the user U1 seen from the position of the user U2 can be viewed. That is, the user U2 can feel the reality as if the user U1 exists in the target device 100. FIG.

As described above, according to the communication system S according to the embodiment, the user U1 at the remote site P1 and the user U2 at the target site P2 can realize communication with reality.

Furthermore, in the communication system S according to the embodiment, a plurality of target site devices 100 may be arranged at the target site P2, and the target site device 100 operated by the user U1 at the remote site P1 may be switched. Details of the points will be described later.

In the example shown in FIG. 1, the target device 100 has one second display unit 140, but it has a plurality of second display units 140, each of which has a different remote display. The first image information (different user U1) from the local device 1 may be displayed.

Next, a configuration example of the target site device 100 will be described with reference to FIG. FIG. 2 is a diagram showing the appearance of the target device 100. As shown in FIG. As shown in FIG. 2, the target device 100 has a moving unit 200 having a moving function.

In the present disclosure, the moving unit 200 indicates a vehicle-type device that travels on the ground, but may be a flying object such as a drone, for example. In addition, electronic components such as a communication unit 110, a control unit 150, and a storage unit 160, which will be described later with reference to FIG.

A second display unit 140 is installed above the moving unit 200 via a jig 210 . The jig 210 has an adjusting function for adjusting the height of the second display section 140 . Thereby, for example, the height of the second display unit 140 can be adjusted according to the height of the user U1 who is present at the remote location P1. In addition, such an adjustment function may automatically adjust the height according to the control of the control unit 150, which will be described later, or may manually adjust the height by the user U2 present at the target site P2.

In the present disclosure, the second display unit 140 is configured as a transmissive display. As a result, the user U2 present at the target site P2 can view the user U1, which is the first image information, from either the front or back of the second display unit 140. FIG. That is, when the user U2 exists behind the target device 100, by displaying the back view of the user U1 on the second display unit 140, a more realistic image can be displayed.

Also, in the present disclosure, the second display unit 140 is configured as a display having a speaker function. This makes it sound as if the user U1 appearing on the second display unit 140 is emitting a voice, so that the reality can be further enhanced.

Further, as shown in FIG. 2, a camera unit in which the second camera 120 and the second microphone 130 are integrated is provided at the upper end of the second display section 140 . That is, the camera unit is arranged near the face of user U1 displayed on second display unit 140 . As a result, images can be captured by the second camera 120 near the eyes of the user U1, and sound can be collected by the second microphones 130 near the ears of the user U1. can increase Second camera 120 and second microphone 130 rotate according to the orientation of the face of user U1 at remote location P1. Also, the second microphone 130 uses a directional microphone similar to the human ear. As a result, it is possible to obtain images that can be seen and sounds that can be heard according to the orientation of the face of the user U1 at the remote location P1, thereby improving realism.

Next, a functional configuration example of the communication system S will be described with reference to FIG. FIG. 3 is a block diagram showing a functional configuration example of the communication system S according to the embodiment.

As shown in FIG. 3, the communication system S according to the embodiment includes a remote device 1 and a target device 100. As shown in FIG.

The remote device 1 includes a mobile body 2 , a control device 3 , a first display section 4 and an operation section 5 . The moving object 2 is a small moving object such as a radio-controlled car or a drone that can travel autonomously under the control of the control device 3 .

The moving body 2 includes a first camera 21 , a first microphone 22 and a first speaker 23 .

The first camera 21 is an imaging device that captures an image of the remote location P1. The first microphone 22 is a sound collecting device that collects sounds from the remote location P1. The first speaker 23 is an output device that outputs the sound that the control device 3 acquires from the target device 100 .

The first display unit 4 is, for example, a screen that displays image information projected by a projector, as described above. Note that the first display unit 4 may be a liquid crystal display.

The operation unit 5 is a terminal device that receives the movement operation described above, and is, for example, a joystick, a mouse, a controller equipped with a G sensor, a smartphone, or the like.

The control device 3 includes a communication section 31 , a control section 32 and a storage section 33 . The communication unit 31 is a communication interface, and transmits and receives various information to and from the mobile unit 2 and the target device 100 .

Here, the control device 3 includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an input/output port, and various circuits.

The CPU of the computer functions as the control unit 32 by reading and executing programs stored in the ROM, for example.

At least one or all of the functions of the control unit 32 can be configured by hardware such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

Also, the storage unit 33 corresponds to a RAM or a flash memory. The RAM and flash memory can store information of various programs and the like. Note that the control device 3 may acquire the above-described programs and various information via other computers or portable recording media connected via a wired or wireless network.

Next, functions of the control unit 32 will be described in detail.

The control unit 32 performs various control processes. For example, the control unit 32 transmits video information (first video information) captured by the first camera 21 and audio information (first audio information) collected by the first microphone 22 to the target area via the communication unit 31. Send to device 100 .

Further, the control unit 32 displays the video information (second video information) acquired from the target device 100 on the first display unit 4, and communicates the audio information (second audio information) acquired from the target device 100. Output from the first speaker 23 via the unit 31 .

Further, the control unit 32 performs control to move the moving body 2 . Specifically, the control unit 32 acquires information indicating the positional relationship between the target site device 100 and the user U2 present in the target site P2 from the target site device 100, and moves the mobile body 2 based on the positional relationship. Let

More specifically, the control unit 32 makes the positional relationship between the user U1 at the remote location P1 and the moving body 2 (the first camera 21) the same as the positional relationship between the target device 100 and the user U2. The moving body 2 is moved as follows.

Thus, by outputting the second voice information of the user U2 from the first speaker 23, the user U1 feels as if he/she is being spoken to from the position where the user U2 is, so that the reality of the dialogue can be enhanced.

Further, by capturing an image of the user U1 with the first camera 21 from the position of the moving object 2 and displaying it on the second display unit 140 of the target device 100, the image of the user U1 seen from the user U2 is displayed on the second display unit. It can be seen at 140 and can be heard by user U1.

Also, the control unit 32 detects the orientation of the face of the user U1 based on the image of the first camera 21, for example. That is, the control unit 32 detects which position of the first display unit 4 the user U1 is looking at (line-of-sight direction). Then, the control unit 32 transmits information on the orientation of the face of the user U1 to the target device 100 . Although the details will be described later, the target device 100 changes the orientation of the second camera 120 according to the face orientation information. That is, the target site device 100 operates the second camera 120 based on the information detected from the user U1 present at the remote site P1. This allows the second camera 120 to acquire the second image information in the direction in which the user U1 is actually looking (want to see) and display it on the first display unit 4 .

It should be noted that not only the case of detecting the orientation of the face of the user U1 based on the image of the first camera 21, but also a dedicated camera for detecting the orientation of the face is separately arranged at the remote location P1. The orientation of the face may be detected based on the image of .

Further, the control unit 32 may receive a moving operation for moving the moving unit 200 of the target device 100 from the user U1. Specifically, the control unit 32 transmits an operation to the operation unit 5 of the user U1 to the receiving target device 100 as a movement operation. Then, the target device 100 moves the moving unit 200 according to the moving operation. In other words, the target device 100 moves the moving unit 200 based on the information (moving operation) detected from the user U1 present at the remote location P1. This allows the user U1 at the remote location P1 to freely move to the target location P2.

Next, the target device 100 includes a moving unit 200 , a second camera 120 , a second microphone 130 and a second display unit 140 .

The second camera 120 is an image capturing device that captures an image of the target land P2. The second microphone 130 is a sound collecting device that collects the sound of the target site P2. The second display unit 140 is, as described above, a display unit having a sound output function, a so-called display speaker. Note that the second display unit 140 may include a display unit that displays images and a speaker that outputs sound separately.

The moving unit 200 is a small moving object such as a radio-controlled car or a drone that can autonomously travel under the control of the control unit 150 . Moving unit 200 includes communication unit 110 , control unit 150 , and storage unit 160 . For example, the moving unit 200 includes a driving motor and is driven (moved) by electric power supplied from a storage battery (not shown).

The communication unit 110 is a communication interface, and transmits and receives various information to and from the remote device 1 .

Here, the target device 100 includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an input/output port, and various circuits.

The CPU of the computer functions as control unit 150 by reading and executing programs stored in the ROM, for example.

At least one or all of the functions of the control unit 150 can be configured by hardware such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

Also, the storage unit 160 corresponds to a RAM or a flash memory. The RAM and flash memory can store information of various programs and the like. Note that the target device 100 may acquire the above-described programs and various information via another computer or portable recording medium connected via a wired or wireless network.

Next, functions of the control unit 150 will be described in detail.

The control unit 150 performs various control processes. For example, the control unit 150 transmits video information (second video information) captured by the second camera 120 and audio information (second audio information) collected by the second microphone 130 to a remote location via the communication unit 110. Send to device 1.

Further, the control unit 150 displays the video information (first video information) acquired from the remote device 1 on the second display unit 140, and displays the audio information (first audio information) acquired from the remote device 1. Output from the second display unit 140, which is a speaker.

Also, the control unit 150 performs movement control of the moving unit 200 . Specifically, the control unit 150 performs movement control of the moving unit 200 according to the movement operation of the user U1 acquired from the remote device 1 .

In addition to the manual mode in which the user U1 moves manually (moving operation), the control unit 150 follows the user U2 existing in the target area P2 and automatically travels along a predetermined route. It may have an automatic driving mode.

Further, when moving the target site P2, the control unit 150 may detect an obstacle from the image of the second camera 120, for example, and perform control to avoid the obstacle.

In addition, the control unit 150 changes the orientations of the second camera 120, the second microphone 130, and the second display unit 140 according to information on the face orientation of the user U1 acquired from the remote device 1. FIG.

Further, when a plurality of target device 100 are arranged in the target site P2, the control unit 150 of one target device 100, when a predetermined condition is satisfied, establishes a communication connection with the remote device 1 by another device. You may switch to the target place apparatus 100. FIG.

For example, the control unit 150 switches to another target device 100 when receiving a switching operation from the user U1 via the remote device 1 . Further, when the moving unit 200 moves from the pre-allocated first area to the second area in the vast target area P2, the control unit 150 switches to another target area device 100 corresponding to the second area.

In addition, the control unit 150 controls the switching of the target device 100, for example, up to an amount that allows the target device 100 (moving unit 200) to return to a predetermined parking position (for example, charging device). You may switch to another target place apparatus 100, when it falls. In such a case, the switching source target device 100 automatically returns to a predetermined parking position after switching to another target device 100 .

As a result, the first image information displayed on the second display unit 140 of one target device 100 is displayed on the second display unit 140 of another target device 100 . As a result, when the user U1 wants to move to the large target site P2, the user U1 can move by simply switching the target site device 100 that is connected for communication. be able to.

Note that switching among the plurality of target device 100 may be performed between the control units 150 of the respective target device 100, for example, may be performed collectively by a server device (not shown).

Next, a processing procedure of processing executed by the communication system S according to the embodiment will be described with reference to FIG. FIG. 4 is a sequence diagram showing a processing procedure of processing executed by the communication system S according to the embodiment.

As shown in FIG. 4, the remote device 1 and the target device 100 establish mutual communication connection (step S101). For example, when remote device 1 is activated, a map of target location P2 and available target device 100 (target device 100 not connected to other remote device 1 for communication) are displayed on the map. When the user U1 at the remote location P1 selects the desired target device 100 from the map, communication connection is established. Note that the user U1 at the remote location P1 may select a specific location in the target location P2 instead of selecting the target location device 100, and may communicate with the target location device 100 closest to that location. . Subsequently, the remote device 1 and the target device 100 exchange video information and audio information with each other (step S102).

Subsequently, the remote device 1 and the target device 100 display the received video information and output the audio information (step S103).

Subsequently, the remote device 1 receives a movement operation for moving the moving unit 200 of the target device 100 (step S104). Subsequently, the target device 100 moves the moving unit 200 based on the moving operation (step S105).

Subsequently, the remote device 1 detects the orientation of the face of the user U1 at the remote location P1 (step S106). Subsequently, the target device 100 changes the orientation of the second camera 120 according to the orientation of the face of the user U1 (step S107).

It should be noted that each process shown in steps S101 to S107 is performed when a trigger event such as an operation by the user U1 or U2 occurs in each device (remote device 1 and target device 100). Repeatedly.

Further, when the user U1 performs a termination operation via the operation unit 5 on the remote device 1, the communication is disconnected and the display and the like are stopped. Further, the target device 100 automatically returns to a predetermined parking (storage) position when the communication is disconnected. A charging device is arranged at a predetermined parking position, and the vehicle is charged while the vehicle is parked.

As described above, the communication system S according to the embodiment includes the remote device 1 and the target device 100 . The remote device 1 is located at a remote location P1 away from the target location P2 and has a first camera 21 that captures an image of the remote location P1 and a first display section 4 . The target site device 100 is a moving body that is arranged at the target site P2 and has a second camera 120 that captures an image of the target site P2 and a second display section 140 . The target device 100 displays the first image information captured by the first camera 21 of the remote device 1 on the second display unit 140, and based on the information detected from the user U1 who is present at the remote location P1. , moves or operates the second camera 120 , and the remote device 1 displays the second image information captured by the second camera 120 of the target device 100 on the first display section 4 . This makes it possible to realize communication with reality.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

1 Remote device 2 Mobile object 3 Control device 4 First display unit 21 First camera 22 First microphone 23 First speaker 31 Communication unit 32 Control unit 33 Storage unit 100 Target device 110 Communication unit 120 Second camera 130 Second Microphone 140 Second display unit 150 Control unit 160 Storage unit 200 Moving unit 210 Jig P1 Remote site P2 Target site S Communication system

Claims (8)

a remote device disposed at a remote location away from a target location and having a first camera that captures an image of the remote location and a first display;
a target site device, which is a moving object disposed at the target site and having a second camera that captures an image of the target site, and a second display unit;
with
The target device is
The first image information captured by the first camera of the remote device is displayed on the second display unit, and based on the information detected by the user present at the remote location, the second camera moves or moves. run the
The remote device comprises:
A communication system, wherein second image information captured by the second camera of the target device is displayed on the first display unit. The remote device comprises:
2. The communication system according to claim 1, wherein said first image information obtained by imaging said remote user with said first camera is transmitted to said target site device. The remote device comprises:
detecting the orientation of the face of the user at the remote location and transmitting it to the target device;
The target device is
3. The communication system according to claim 1, wherein the direction of said second camera is changed according to the direction of said user's face. The remote device comprises:
Receiving a movement operation for moving the target device from the user at the remote location;
The target device is
4. The communication system according to any one of claims 1 to 3, wherein the mobile device moves according to the moving operation. The target device is
detecting a positional relationship between a user existing at the target site and the target site device and transmitting the positional relationship to the remote site device;
The remote device comprises:
The first camera is moved so that the positional relationship between the user present at the remote location and the first camera is the same as the positional relationship between the user present at the target location and the target device. The communication system according to any one of claims 1 to 4, wherein The target device is
6. The communication according to any one of claims 1 to 5, characterized in that, when a plurality of devices are arranged in the target area and a predetermined condition is satisfied, the target site device connected for communication with the remote device is switched. system. An information processing device placed at a remote location away from a target location,
a first camera that captures an image of the remote location;
a first display unit;
and a control unit that detects information for operating the information processing apparatus located at the target location from the user existing at the remote location. An information processing device, which is a moving object placed at a target location away from a remote location,
a second camera that captures an image of the target area;
a second display unit;
and a control unit that controls movement or operation of the second camera based on information detected from the user existing at the remote location.

Images