JP2004129071A - Two-way communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily grasp a remote participant by watching a monitor in a two-way communication system. <P>SOLUTION: There are provided: a database 3 for storing identification information for identifying a participant scheduled to participate in the two-way communication and attribute information indicative of attribute of the participant, authentication means 3 for authenticating an actual participant participating in the two-way communication at an individual point; detection means 3 for detecting the position of a participant authenticated by the authentication means 3 at the individual point; and adding means 3, 12 for adding data for visually displaying attribute information of a participant stored in the database 3 at a location corresponding to the position of the participant detected by the detection means 3 among video images photographed at the individual positions. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an interactive communication system such as a video conference system.
[0002]
[Prior art]
By transmitting the video shot at each point to the remaining points via a network between multiple points separated from each other, people in remote locations can communicate with each other while seeing the face of the other party Such a system is becoming widespread. A video conference system is an example of such an interactive communication system.
[0003]
FIG. 1 is a block diagram showing devices and circuits for transmitting and receiving video in a conventional video conference system. A video camera 51, an AD converter 52, a video codec 53, a communication interface 54 for network communication, a DA converter 55, and a monitor 56 are provided in the conference room at the point A.
[0004]
A video camera 51, an AD converter 52, a video codec 53, a communication interface 54, a DA converter 55, and a monitor 56 are also provided in a conference room at a point B remote from the point A.
[0005]
At the point A, the video (video of the conference participant) captured by the video camera 51 is digitally converted by the AD converter 52, encoded (compressed) by the video codec 53, and transmitted from the communication interface 54 via the network 101. B is transmitted to the communication interface 54. Then, at the point B, the video received by the communication interface 54 is decoded (expanded) by the video codec 53, converted into an analog signal by the DA converter 55, and displayed on the monitor 56.
[0006]
The image is transmitted from the point B to the point A in exactly the same manner, and the image is displayed on the monitor 56 at the point A.
[0007]
FIGS. 2 and 3 are diagrams exemplifying images displayed on the monitor 56 of FIG. As shown in FIG. 2, images of all conference participants in a remote place (point B for point A and point A for point B) are displayed on the monitor 56, or the zoom amount of the video camera 51 in the remote place is displayed. By controlling the pan / tilt amount (the control system of the video camera 51 is not shown in FIG. 1), the video of one of the conference participants who is currently speaking is displayed on the monitor 56 as shown in FIG. The video is displayed in an up state (in this way, the video of the remote conference participant is displayed on the monitor in the video conference system, for example, see Patent Documents 1 and 2).
[0008]
[Patent Document 1]
JP-A-2002-237991 (paragraph number 0007, FIG. 1)
[Patent Document 2]
JP-A-2002-171499 (paragraph numbers 0002 to 0004)
[0009]
[Problems to be solved by the invention]
However, in such a conventional video conferencing system, if a remote participant includes an unfamiliar person, the person who looks at the monitor will know who the person is (what title and what name Who couldn't figure out).
[0010]
Also, even if all of the prospective participants in the remote area are people who are acquainted, if the number of participants is large, the monitor can be used to understand who is actually participating in the conference in the remote area. (When the images of all the remote participants are displayed on the monitor as shown in Fig. 2, the face of each individual participant is too small due to the compression rate of the video codec. Because it is not clear, it is difficult to know who is actually participating in the conference, and when the image of one person currently speaking is displayed on the monitor as shown in FIG. Participants are not displayed, so it is difficult to know who is actually participating in the meeting).
[0011]
SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a two-way communication system such as a video conference system so that a remote participant can be easily grasped by looking at a monitor.
[0012]
[Means for Solving the Problems]
In order to solve this problem, the present applicant has proposed a two-way communication system in which, between a plurality of points separated from each other, images captured at individual points are respectively transmitted to the remaining points via a network. A database storing identification information for identifying a prospective participant in the bidirectional communication, and attribute information indicating the attribute of the prospective participant; and Authentication means for authenticating the actual participant in the two-way communication, detection means for detecting the position of the participant authenticated by this authentication means at each point, and of the video taken at each point, In the part corresponding to the position of the participant detected by the detection means, the attribute information of the participant stored in this database is stored. The propose that a adding means for adding data to visually display.
[0013]
In this two-way communication system, identification information for identifying a prospective participant in the two-way communication and attribute information indicating the attribute of the prospective participant are stored in a database. When one of the prospective participants actually participates in the interactive communication at any point, the participant is authenticated by the authentication means using the identification information stored in the database. Thereafter, the position of the authenticated participant at that point is detected by the detecting means.
[0014]
Then, in a portion corresponding to the position of the detected participant in the video taken at that point, the attribute information of the participant stored in the database is visually displayed by the adding means. Is added. As a result, from the point, data for visually displaying the attribute information of the participant is added to a part of the video taken at the point corresponding to the position of the detected participant. Things are sent to the rest.
[0015]
As a result, not only the video of this participant (participant in a remote place) is displayed on the monitor at the remaining point, but also the attribute information of the participant is displayed in a portion corresponding to the position of this participant. You.
[0016]
As described above, since the attribute information of the participant in the two-way communication in the remote place is displayed on the monitor together with the video of the participant, the participant in the remote place can be easily grasped just by looking at the monitor. Become like
[0017]
In this two-way communication system, as an example, the database stores face feature data of a prospective participant as identification information, and recognizes a face by using the face feature data from a video taken at each point. It is preferable that authentication be performed by recognition, and that the detecting means be configured to detect the position in the screen of the participant authenticated by the authenticating means from the video taken at each point.
[0018]
This allows the participant to carry a dedicated medium for authentication or position detection (an ID card for authentication, a radio wave or infrared ray generator for position detection, or the like) or to use the medium for authentication or position detection. Authentication or position detection of a participant can be performed without providing a device (an ID card reading device, a radio wave or infrared receiving device, or the like) for acquiring such information.
[0019]
Further, in a case where the two-way communication system is applied to a video conference system, as an example, the title and the name information are stored as attribute information in a database. It is preferable to configure to add.
[0020]
Therefore, even if a remote conference participant includes an unfamiliar person at the time of a video conference, for example, the person who looks at the monitor can determine who the person is. The name of the person with the title) can be easily grasped, and even if all the remote meeting participants are acquainted but there are many participants, just looking at the monitor Can easily understand who is participating in the conference.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an example in which the present invention is applied to a video conference system connecting two points separated from each other will be described with reference to the drawings.
[0022]
FIG. 4 is a diagram showing a conference room at each point (points A and B) in the video conference system to which the present invention is applied, from the viewpoint of conference participants. The conference participants are seated in front of the table 4, and a video conference device 1 and a monitor 2 with a built-in speaker are installed in front of the table 4 and a personal computer (hereinafter simply referred to as a computer). 3) are installed.
[0023]
The video conference device 1 is a device in which a video camera, a microphone, a codec, a communication interface for network communication, and the like are integrated, and the video camera and the microphone are directed toward the front of the device (the direction of the table 4).
[0024]
The video output terminal and the audio output terminal of the video conference device 1 are connected to the video input terminal and the audio input terminal of the monitor 2 by cables, respectively. Further, another video output terminal of the video conference device 1 is connected to a video input terminal of the computer 3 by a cable, and a video input terminal of the video conference device 1 is connected to a video output terminal of the computer 3 by a cable.
[0025]
FIG. 5 is a block diagram showing the configuration of the video conference device 1 together with the overall configuration of the video conference system. The video conference device 1 includes a video camera 11, a mixing circuit 12, an AD converter 13, a video codec 14, a communication interface 15, a DA converter 16, a microphone 17, an AD converter 18, an echo canceller 19, an audio codec 20, a DA converter 21, and a CPU 22. , A camera controller 23.
[0026]
At the point A, the video signal output from the video camera 11 of the video conference device 1 is sent from the video conference device 1 to the computer 3 and to the mixing circuit 12. In the mixing circuit 12, the video signal from the video camera 11 and the video signal sent from the computer 3 to the video conference device 1 are mixed. The video signal output from the mixing circuit 12 is digitally converted by the AD converter 13, encoded (compressed) by the video codec 14, and transmitted from the communication interface 15 to the communication interface 15 of the video conference device 1 at the point B via the network 101. Is done. Then, at the point B, the video data received by the communication interface 15 of the video conference device 1 is decoded (expanded) by the video codec 14, converted into an analog signal by the DA converter 16, and sent from the video conference device 1 to the monitor 2. Displayed on monitor 2.
[0027]
At the point A, the audio signal output from the microphone 17 of the video conference device 1 is digitally converted by the AD converter 18, encoded by the audio codec 20 via the echo canceller 19, and transmitted from the communication interface 15 via the network 101. It is transmitted to the communication interface 15 of the TV conference device 1 of B. Then, at the point B, the audio data received by the communication interface 15 of the video conference device 1 is decoded by the audio codec 14, converted into an analog signal by the DA converter 21 via the echo canceller 19, and transmitted from the video conference device 1 to the monitor 2. It is reproduced by the built-in speaker of the monitor 2.
[0028]
Video and sound are transmitted from the point B to the point A in exactly the same manner. At the point A, the video is displayed on the monitor 2 and the sound is reproduced by the built-in speaker of the monitor 2.
[0029]
The camera controller 23 adjusts the zoom amount and the pan / tilt amount of the video camera 11 under the control of the CPU 22. The CPU 22 is based on a signal generated by operating a zoom or pan / tilt operation key of a remote controller (not shown) at the own location or control data transmitted from the video conference device 1 at the other location via the network 101. A control signal is given to the camera controller 23.
[0030]
The computer 3 stores a database for registering data on each prospective participant in the conference. FIG. 6 shows data registered in this database. For each prospective participant A, B, C..., Face feature data for performing face recognition, and a title such as “XX section XX section manager” and a title / name data indicating a name are included. Each is registered.
[0031]
Further, the computer 3 stores a program for executing a process as shown in FIG. In this process, an actual participant in the conference is first authenticated by performing face recognition from the video signal input from the video conference device 1 to the computer 3 using the facial feature data (FIG. 6) in the database. (Step S1).
[0032]
This face recognition is performed by existing face recognition processing in the following processes (a) to (d).
(A) Face area detection from input video
(B) Extraction of a face area from an input video based on the detection result of the face area, and normalization of variations in the size, luminance, and the like of the extracted face area
(C) Extraction of face features from face area
(D) Matching the extracted facial features with the facial feature data in the database
[0033]
Subsequently, from the video signal input from the video conference device 1, the position of each participant who has been authenticated in step S1 so far is detected on the screen (in the video of one frame) (step S2). This position detection is performed by a method of determining the position of the face area detected in the process (a) of step S1 for each authenticated participant as it is as the position of the participant.
[0034]
Subsequently, it is determined whether or not a new participant who has not been authenticated has been authenticated in the current step S1 (step S3).
[0035]
If yes, the title and name of the participant are placed near the position of each participant detected in step S2 this time and not overlapping with other participants using the title / name data in the database. Is generated (step S4). Then, the character data is sent from the computer 3 to the video conference device 1 (step S5). Then, the process returns to step S1 and repeats the steps from step S1.
[0036]
If NO in step S3, the position of any participant detected in step S2 this time changes from the position detected in previous step S2, or the participant whose position was previously detected is It is determined whether the position has not been detected this time or a participant whose position has not been detected previously has been detected this time (step S6).
[0037]
If yes, proceed to step S4. On the other hand, if no, the process returns to step S1 and repeats the steps from step S1.
[0038]
Next, a state of a conference using this video conference system will be described. For example, it is assumed that the meeting attendees at the point A are the next seven persons at a certain business establishment.
・ Kinoshita
・ Sales Division 1: Manager Sato, Manager Yamada
・ Sales Division 2: Suzuki, Ueda
・ Design Department 1: Manager Yoshida
・ Design Department 2: Section Manager Tanaka
[0039]
In the database (FIG. 6) in the computer 3 at the point A, the facial feature data and the title / name data of the seven potential participants are registered before the conference starts.
[0040]
At the start time of the meeting, it is assumed that, at the point A, of the seven prospective participants, the manager of the second sales department, Suzuki, cannot urgently participate, but the remaining six persons participated in the meeting as scheduled.
[0041]
At the point A, the zoom amount and the pan / tilt amount of the video camera 11 (FIG. 5) are controlled by the camera controller 23 (FIG. 5) at the point A by the video conference device 1 so that all the six persons are included in the angle of view. It is assumed that shooting was performed. Then, the video is transmitted from the video conference apparatus 1 to the video conference apparatus at the point B via the network 101 and from the video conference apparatus 1 to the computer 3.
[0042]
Then, the computer 3 executes steps S1 to S5 of the processing in FIG. 7, and character data for displaying the title and name of the person near the respective positions of the six persons on the screen are transmitted from the computer 3. It is sent to the video conference device 1.
[0043]
Then, the character data is mixed with the video from the video camera 11 by the mixing circuit 12 (FIG. 5) in the video conference device 1. As a result, the video to which the character data is added is transmitted from the video conference device 1 to the video conference device at the point B.
[0044]
As a result, not only the images of the six participants at the point A are displayed on the monitor 2 at the point B, but also the titles of the participants near the faces of the six participants. And the telop with the name are displayed. FIG. 8 is a diagram illustrating an example of an image displayed on the monitor 2 at the point B at this time.
[0045]
It is assumed that the meeting participant at the point B has no meeting with two of the meeting participants at the point A, namely, Kinoshita and the Design Section 1 Yoshida, and has knowledge of the remaining 5 persons. The meeting participant at the point B only saw this telop displayed on the monitor 2 despite having no acquaintance with Mr. Kinoshita and Mr. Yoshida, Design Section 1. In the example of FIG. It can be easily grasped that the person is the Yoshida section manager of the design section 1 and the third person from the left of the screen is the director Kinoshita.
[0046]
In addition, the meeting participants at point B were informed in advance that there were a total of seven potential attendees at point A, including five who were acquainted. Until then, let us know. The conference participant at the point B is displayed on the monitor 2 even when the face of the person is not displayed clearly on the monitor 2 due to the compression ratio of the video codec 14 (FIG. 5) of the video conference device 1. Just by looking at this telop, it is easy to see who actually participates in the meeting at point A (the manager of the Sales Department, Suzuki, does not participate, and the remaining six people participate). I can figure it out.
[0047]
Subsequently, Kinoshita begins to speak, and at point A, the video conferencing apparatus 1 uses the camera controller 23 (FIG. 5) to adjust the zoom amount and the pan / tilt amount of the video camera 11 (FIG. 5) so as to raise the Kinoshita director. It is assumed that the shooting is performed under the control.
[0048]
Then, based on the video, the computer 3 executes steps S1 to S3 of the processing in FIG. Has also been authenticated), so the answer is no in step S3 and the process proceeds to step S6, where the position of Director Kinoshita has changed in the screen (the size of the face area is different from the previous time) and the remaining participants Since the position is not detected, the result in step S6 is YES, and steps S4 and S5 are executed. Therefore, character data for displaying the title and name near the position of Director Kinoshita on the screen are sent from the computer 3 to the video conference apparatus 1 this time.
[0049]
As a result, on the monitor 2 at the point B, the image of Director Kinoshita is displayed in an up state, and the title and telop of the name are displayed near the face of Director Kinoshita. FIG. 9 is a diagram illustrating an example of an image displayed on the monitor 2 at the point B at this time.
[0050]
Also, for example, at the point A, while all the six participants are being photographed with the video conference device 1 in the angle of view, the participants move their seats, or some of the participants leave. Or you did.
[0051]
Then, steps S1 to S3, S6, S4, and S5 of the processing in FIG. 7 are also executed by the computer 3, and the monitor 2 at the point B displays the vicinity of each participant's face after the movement and the abandoned person. The title and the telop of the participant are displayed near the face of the remaining participant except for the participant.
[0052]
Further, for example, at the point A, the sales department manager Suzuki, who was in urgent business, completed the business and participated in the meeting from the middle, and the video conference device 1 also shot the sales department manager Suzuki section manager in the angle of view. Suppose. Then, in the computer 3, steps S1 to S3 of the processing of FIG. 7 are executed, and since a new participant (Sales Division 2, Suzuki section manager) is authenticated, the answer in step S3 is YES, and steps S4 and S5 are executed. . Therefore, character data for displaying the title and name near the position of each participant including the sales department 2 section manager Suzuki on the screen are transmitted from the computer 3 to the video conference apparatus 1 this time.
[0053]
As a result, the title and the telop of the name are displayed on the monitor 2 at the point B also near the face of the manager of the second sales department, Suzuki, who participated in the meeting from the middle.
[0054]
In this way, at the point A, the zoom amount / pan / tilt amount of the video camera 11 of the video conference device 1 changes, a participant moves or leaves, or a participant joins the conference from the middle. However, on the monitor 2 at the point B, the title and the telop of the participant are always displayed near the face of each participant currently displayed.
[0055]
Here, the state of the conference viewed from the conference participant at the point B (display on the monitor 2 at the point B) has been described. A telop of the participant's title and name is always displayed near the participant's face.
[0056]
As described above, according to this video conference system, the title and name of each conference participant at a remote location (point B for point A and point A for point B) are displayed on the monitor together with the video of the participant. Therefore, it is possible to easily grasp the remote participants simply by looking at the monitor.
[0057]
Since the participant is authenticated and the position is detected based on the face recognition processing, the participant is provided with a dedicated medium for authentication and position detection (an ID card for authentication, a radio wave or infrared ray generator for position detection). Participants in the conference room without possessing a device (such as an ID card reader or a radio or infrared receiver) that acquires information for authentication or position detection from the medium. Authentication and position detection.
[0058]
In the above example, the authentication and the position detection of the participant are performed based on the face recognition processing. However, as another example, the participant authentication may be performed by a personal recognition process such as a voice recognition process, a fingerprint recognition process, or a retinal recognition process, or the position of the participant may be detected by a voice recognition process.
[0059]
Alternatively, if necessary, the participant can have a dedicated medium for authentication and position detection (an ID card for authentication, a generator of radio waves or infrared rays for position detection, etc.), and use the medium for authentication and position detection. A device for acquiring information for detection (an ID card reader, a radio or infrared receiver, or the like) may be provided in the conference room.
[0060]
Alternatively, for example, in a case where microphones for the number of people who are going to participate are arranged at predetermined positions on the table 4 in FIG. 4 (audio signals are transmitted from these microphones to the video conference apparatus 1), For example, each microphone incorporates an operating device and a circuit for transmitting the ID information of the participant who uses the microphone to the computer 3 and stores the position of each microphone 3 in the computer 3 based on the ID information. In addition to performing participant authentication, the position of the microphone that sent the ID information may be determined as the position of the authenticated participant.
[0061]
In the above example, one computer 3 is provided for each of the points A and B. However, as another example, only one computer 3 is provided at one of the points A and B (or another one connected to the points A and B via the network 101). 3 may register the face feature data and the title / name data of the prospective participants at both the point A and the point B in the database, and execute the processing of FIG. 7 for both the participants.
[0062]
Further, in the above example, a computer 3 that stores a database and executes the processing in FIG. 7 is provided separately from the video conference device 1. However, as another example, the video conference device 1 itself may be configured to store a database and execute the processing in FIG.
[0063]
In the above example, by registering the title and name data in the database in the computer 3 (FIG. 6), the title and the telop of the participant are displayed near the participant's face displayed on the monitor 2. Is displayed. However, the present invention is not limited to this. By registering attribute information indicating an appropriate attribute of a prospective participant in this database, the attribute information is displayed near the participant's face displayed on the monitor 2. Good. One example is data summarizing, in addition to, or instead of title / name data, the prospective attendees' opinions at past meetings (eg, opinions on or against a project). Is registered in this database so that a summary telop of such a claim is also displayed near the participant's face displayed on the monitor 2.
[0064]
In the above example, the present invention is applied to a video conference system connecting two points, point A and point B. However, the present invention is not limited to this, and the present invention may be applied to a video conference system connecting three or more points or an appropriate two-way communication system other than the video conference system.
[0065]
When the present invention is applied to an interactive two-way communication system for entertainment, for example, by registering image data of a favorite animation of a prospective participant in a database as attribute information, the vicinity of the participant's face displayed on a monitor can be obtained. The animation image may be displayed on the monitor, or the animation image may be displayed on the participant's face displayed on the monitor.
[0066]
When the present invention is applied to a two-way communication system in which there is a person who does not want to display a face on the monitor among the prospective participants, an instruction is given to apply a mosaic as attribute information about the person. By registering the information in the database, a mosaic may be applied to the face of the person displayed on the monitor.
[0067]
Further, the present invention is not limited to the above-described example, and it is needless to say that various other configurations can be adopted without departing from the gist of the present invention.
[0068]
【The invention's effect】
As described above, according to the two-way communication system according to the present invention, the attribute information of the participant in the two-way communication in the remote place is displayed on the monitor together with the image of the participant, so that the user only needs to look at the monitor. The effect is obtained that a participant in a remote place can be easily grasped.
[0069]
In addition, the authentication and position detection of the participant can be performed without having the participant possess a dedicated medium for authentication or position detection, or providing a device that acquires information for authentication or position detection from the medium. The effect that it can be performed is acquired.
[0070]
Also, when conducting a video conference, even if some of the remote meeting participants do not know anyone, just by looking at the monitor, what kind of person is that person? Is easily understood, and even if all the people who are going to the conference in remote areas are all acquaintances but there are many participants, just watching the monitor, who actually participates in the conference It is possible to obtain the effect that the user can easily understand whether the user is performing the job.
[Brief description of the drawings]
FIG. 1 is a diagram showing a device for transmitting and receiving video in a conventional video conference system.
FIG. 2 is a diagram illustrating an example of an image displayed on the monitor of FIG. 1;
FIG. 3 is a diagram illustrating an example of an image displayed on the monitor of FIG. 1;
FIG. 4 is a diagram showing a conference room in a video conference system to which the present invention is applied.
FIG. 5 is a block diagram illustrating a configuration of the video conference device in FIG. 4;
FIG. 6 is a diagram showing a database in the computer of FIG. 4;
FIG. 7 is a flowchart showing a process executed by the computer of FIG. 4;
FIG. 8 is a diagram illustrating an example of an image displayed on the monitor of FIG. 4;
FIG. 9 is a diagram illustrating an example of an image displayed on the monitor of FIG. 4;
[Explanation of symbols]
1 video conference device, 2 monitor, 3 computer, 4 table, 11 video camera, 12 mixing circuit, 13, 18 AD converter, 14 video codec, 15 communication interface, 16, 21 DA converter, 17 microphone, 19 echo canceller, 20 Voice codec, 101 network

Claims (3)

In a two-way communication system in which, between a plurality of points apart from each other, images taken at the individual points are transmitted to the remaining points via a network.
A database storing identification information for identifying a prospective participant in the two-way communication, and attribute information indicating an attribute of the prospective participant;
Using the identification information stored in the database, an authentication unit that authenticates an actual participant in the interactive communication at each of the points,
Detecting means for detecting the position of the participant authenticated by the authentication means at each of the points;
For visually displaying attribute information of the participant stored in the database at a part corresponding to the position of the participant detected by the detection means in the video taken at each of the points. An interactive communication system comprising: an adding unit for adding data. The two-way communication system according to claim 1,
The database stores face feature data of the prospective participant as the identification information,
The recognition unit performs authentication by face recognition using the face feature data from an image captured at each of the points,
The two-way communication system according to claim 1, wherein said detecting means detects a position in a screen of the participant authenticated by said authenticating means, from a video taken at each of said points. The two-way communication system according to claim 1,
In the database, title and name information are stored as the attribute information,
The said adding means adds the character data of the title and the name, The two-way communication system characterized by the above-mentioned.

Images