JP2006148467A - Face-to-face communication equipment and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To activate communication of a user and his/her opposite party by facilitating the communication. <P>SOLUTION: Displays 103 and 104 are arranged back to back between a user A and a user B. A video image in which the user B is photographed from the diagonal rear part of the interacting position of the user B and a video image in which the user A is photographed from the diagonal front part are overlayed, combined and turned to a virtual objective video image 134, and the virtual objective image 134 is presented to the user B by the display 104. A subjective video image which is the video image from the view point of the user B is acquired by photographing the user A from the front, and the subjective video image and the virtual objective video image are switched and presented to the user B by the display 104. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a communication device using video, and more particularly to a face-to-face communication device and method capable of realizing ease of communication and activation by switching video using a plurality of cameras.

  In video communication, not only monotonous video using only a single camera but also video that has been shot using multiple cameras can be switched and presented to the user without getting bored of the user. Attempts have been made to activate. There is a research example that analyzes a method of switching (switching) a camera in television broadcasting and applying it to a system (Non-patent Document 1).

Nowadays, watching TV has become a daily routine, and there are increasing opportunities for users to interact with information presentation using video, such as large displays on street corners. The video information is told by a “video language” that is not used in daily conversation by a technique unique to video such as cut switching. Specifically, in these video representations, switching between a subjective video, which is a video seen from its own viewpoint such as the main character, and an objective video, which is an entire video including the main character and its partner, is performed. Can increase the degree of importation of people who are living. In contrast, in everyday space, people can always capture images only from their own perspective.
Tomoo Inoue, Kenichi Okada, Atsushi Matsushita, “Use and Effects of Video Expressions in Video Conferencing”, Transactions of Information Processing Society of Japan, Vol. 40, No. 10, 3752-3761, 1999

  There is information that can be transmitted more appropriately in the video expression with strong stimulation, and if the video information as described above can be used effectively in daily communication, there is a possibility that communication of a quality different from the conventional one can be performed. Nevertheless, the current situation is that there is no interactive environment for handling images in daily life, and it cannot be used.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a face-to-face communication apparatus and method that can realize ease of communication, activation, and the like by interactively handling video.

  The face-to-face communication apparatus of the present invention is a face-to-face communication apparatus that promotes communication between a first user and a second user, and is arranged at a position in front of each of the dialogue positions of the first and second users. First and second displays, a first camera and a second camera that respectively photograph the first user from a diagonally backward position of the first user and a diagonally forward direction of the first user, and a second The third camera and the fourth camera that respectively photograph the first user from the diagonally backward position of the user's dialogue position and the diagonally forward direction of the second user, and the first and second users that respectively photograph the first and second users from the front. 5 and 6 cameras, and a switching method that enables input of video from each camera, selection by the user, or selection of video according to the scene and display on each display And, with a.

  The face-to-face communication method of the present invention is a face-to-face communication method that promotes communication between the user and the other party, and includes an image in which the user is photographed from diagonally behind the user's dialogue position, and an image in which the other person is photographed from diagonally forward. Are overlay-synthesized and presented to the user as an objective video on a display arranged in front of the user.

  In the present invention, by using a plurality of cameras, a subjective video and an objective video showing a place for communication between the user and the other party are generated, and the subjective video and the objective video are combined and presented to the user. It is preferable to do so.

  According to the present invention, not only one's own subjectivity but also the relationship with the other party can be viewed objectively, so that it becomes possible to deepen the understanding of each other's relationship. In addition, by mutually adjusting the video switching rhythm, the degree of synchronization with each other is increased, and the ease of communication and activation can be increased.

  Next, a preferred embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a perspective view showing the overall configuration of a face-to-face communication apparatus according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the structure of a portion related to video output in the face-to-face communication apparatus.

  Here, it is assumed that the user A 101 and the user B 102 are communicating with each other, and the users 101 and 102 are located facing each other with the two displays 103 and 104 arranged back to back. Therefore, each user cannot directly see the other user because they are blocked by these displays. The display 103 has a display area facing the user A 101, and the display 104 has a display area facing the user B. The display 103 displays video information presented to the user A 101, and the display 104 displays video information presented to the user B 102. In the following description, the standing position or sitting position of each user when the users 101 and 102 are trying to communicate with each other will be referred to as a dialogue position.

  In order to acquire the image of the user A 101, a front image camera 105 that captures the user A 101 from the front and two shoulder licking image generation cameras 106 and 107 are provided. If an ultra-small camera can be used, the front image camera 105 is embedded in the center of the display area of the display 103, for example. In this case, the front image camera 105 is provided so that the optical axis direction is opposite to that of the display 103. Of the shoulder licking image generation cameras 106 and 107, the first shoulder licking image generation camera 106 is provided so as to photograph the user A from diagonally behind the dialogue position of the user A 101, and the second shoulder licking image generation camera 106, 107 is provided. The image generation camera 107 is provided so as to photograph the user A from diagonally forward of the dialogue position of the user A 101. Similarly, in order to acquire the video of the user B 102, a front image camera 108 that captures the user B 102 from the front, and a first shoulder tanned image that captures the user B from the diagonally rear side of the user B 102's interaction position. A generation camera 109 and a second shoulder-tangled image generation camera 110 that captures the user B from diagonally forward of the user B 102's dialogue position are provided.

  In the present embodiment, a video shot by the front image camera 105 on the user A side is displayed on the display 104 on the user B side, and a video shot by the front image camera 108 on the user B side is displayed on the user A side display. By displaying on 103, it becomes possible for each user to see the state of the other party via the display, and to give the user a feeling as if a glass frame is placed between the users. become. In this case, since the video from the viewpoint of each user is displayed on each display, the subjective video is displayed.

  Further, in this face-to-face communication apparatus, objective images can be displayed on the respective displays 103 and 104 by using the shoulder licking image generation cameras 106, 107, 109, and 110. In this face-to-face communication device, pre-image storage for storing an image (pre-image) captured by the first shoulder lick video generation camera 109 on the user B side when the user B 102 is not in front of the display for the user B 111, a human image cutout unit 112 that cuts out the image portion of user B from the video captured by the first shoulder licking video generation camera 109 using the preimage stored in the preimage storage unit 111, and the user A A person image composition unit 113 that overlays the image from the second shoulder-tangled image generation camera 110 and the image portion of the user B, and a front image that captures the image output from the person image composition unit 113 and the user A An image switching unit 114 that switches the video from the camera 105 and displays it on the display 104 on the user B side is provided. Similarly, a pre-image storage unit 115 that stores a pre-image captured by the first shoulder licking video generation camera 106 on the user A side without the user A, and a pre-image stored in the pre-image storage unit 115 A person image cutout unit 116 that cuts out the image portion of the user A from the video shot by the first shoulder licking video generation camera 106, and a video from the second shoulder licking video generation camera 110 shooting the user B A person image composition unit 117 that overlays the image portion of the user A, a video output from the person image composition unit 117, and a video from the front image camera 108 that captures the user B are switched, and the display 103 on the user A side is switched. And an image switching unit 118 to be displayed.

  In this configuration, the video output by the human image synthesis units 113 and 117 is a video (virtual objective video) as if the user A and the user B were photographed as if they were facing each other from diagonally behind. It is. The image switching units 114 and 118 switch between the subjective video output from the front image cameras 105 and 108 and the virtual objective video output from the human image synthesizing units 113 and 117 according to the situation of the conversation. The data are output to the corresponding displays 104 and 103, respectively. The image switching unit 114 may switch the video in response to a request from the user B. Similarly, the image switching unit 118 may switch the video in response to a request from the user A.

  FIG. 3 is a diagram for explaining generation of a virtual objective video. Here, generation of the virtual objective video displayed on the display 104 on the user B 102 side will be described, but generation of the virtual objective video displayed on the display 102 on the user A 101 side is performed in the same manner.

  3A is a view of the user A 101 and the user B 102 facing each other across the displays 103 and 104 as viewed from above. Here, user A raises his left hand.

  Here, as shown in FIG. 3B, the video shot by the second shoulder-tangled image generation camera 107 on the user A side is a video 131 in which only the user A is shown. An image obtained by photographing the user B by the first shoulder licking image generation camera 109 on the user B side is obtained by photographing the user B from the rear as shown in FIG. The image 132 is shown. In the video 132, the video that is interacting with the user A cannot be obtained. Therefore, the person image cutout unit 112 captures the pre-image 133 (see FIG. 3D) that has been captured and stored in advance by the first shoulder licking image generation camera 109 in a state where the user B is not in the conversation position. By subtracting from the video 132 showing the user B, the video of only the person portion of the user B is extracted. Therefore, the person image composition unit 113 synthesizes the image 131 showing only the user A and the image of only the person portion of the user B, thereby synthesizing the shoulder image 134 shown in FIG. Virtual objective video).

  In the above-described embodiment, the displays 103 and 104 are arranged back to back. However, when the user A and the user B are separated from each other, the display is arranged for each user. There is no need to place them.

It is a figure which shows the structure of the face-to-face communication apparatus of one Embodiment of this invention. It is a block diagram which shows the structure of the part regarding a video output. It is a figure explaining the synthetic | combination procedure of a virtual objective video (composite shoulder lick video).

Explanation of symbols

101, 102 User 103, 104 Display 105, 108 Front image camera 106, 107, 109, 110 Shoulder-tangled image generation camera 111, 115 Pre-image storage unit 112, 116 Human image clipping unit 113, 117 Human image synthesis unit 114, 118 Image switching unit

Claims (6)

A face-to-face communication device that facilitates communication between a first user and a second user,
First and second displays disposed at respective front positions of the first and second user interaction positions;
A first camera and a second camera that respectively photograph the first user from an obliquely backward position of the first user's dialogue position and an obliquely forward direction of the first user;
A third camera and a fourth camera that respectively photograph the first user from an obliquely backward position of the second user's dialogue position and an obliquely forward direction of the second user;
Fifth and sixth cameras for photographing the first and second users respectively from the front;
Switching means for enabling input of video from each camera, selection by the user, or selecting video according to a scene and displaying on each display;
A face-to-face communication device. Means for extracting only the corresponding image portion of the user from the images taken by the first and third cameras;
Means for overlay synthesis of the image portion of the user and the images taken by the second and fourth cameras to output an objective image;
The face-to-face communication apparatus according to claim 1, further comprising:   The face-to-face communication apparatus according to claim 2, wherein the switching unit switches between a subjective video that is a video taken from the viewpoint of each user and captured by the fifth and sixth cameras, and the objective video.   The face-to-face communication apparatus according to claim 1, wherein the first and second displays are arranged back to back. A face-to-face communication method that facilitates communication between the user and the other party,
An image obtained by photographing the user from diagonally behind the user's dialogue position and an image obtained by photographing the opponent from diagonally forward are overlaid and combined as an objective video on the display arranged in front of the user. A face-to-face communication method to present.   6. The subjective video, which is a video from the user's viewpoint, is acquired by photographing the partner from the front, and the subjective video and the objective video are switched and presented to the user by the display. Face-to-face communication method.

Images