JP2003244669A - Video conference system having sight line detecting function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To identify a talker and display a sight point of the talker through detection of a visual line so as to provide a feeling of presence even to people at a remote place as if they are present in the same place as the talker, thereby smoothly running a conference. <P>SOLUTION: A sight line detector is attached to a video conference system to detect the sight lines of participants, and a screen at a remote place displays the image. Further, a talker in the conference is identified through detection of the sight line and voice monitor information. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference system which sends out an image signal and an audio signal to a communication line to enable a conference at a plurality of remote places.

[0002]

2. Description of the Related Art A video conference system is a system in which a video camera, a screen, or a monitor TV is installed at each meeting place and an image and a voice are exchanged to advance the meeting. Since different images can be exchanged with the telephone, the presence of visual information is increased by adding visual information, and it is possible to have a meeting even if you are away, which is useful for improving work efficiency and responding quickly in an emergency.

[0003]

Since the conference proceeds while watching the image, the content and quality of the image to be communicated greatly influence the progress.

[0004] For example, in discussions, when the eyes of each other are used to see each other's reactions, the conversation proceeds smoothly, but when the images of distant places are wide-ranging images and the size of the person is small, the face When people who are talking to each other communicate with each other as if they are looking in different directions depending on the position of the camera, there is no visual reciprocity. Sometimes it does not go smoothly like the discussion with the person who is there.

Further, if a person who is listening at a remote place can know what the speaker is looking at and talking, it is easy to grasp the situation of the talk. Also, for the person who is speaking, understanding what kind of image is being viewed at a remote location helps to grasp the situation and enables a better presentation.

[0006]

Therefore, by using the non-contact line-of-sight detector according to the present invention, it is possible to detect what the speaker sees and speaks, and what image the listener is looking at.
By displaying the image on the screen or the monitor TV, a more realistic image can be provided and the progress of the conference can be promoted.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention relates to image communication in a video conference system. Hereinafter, embodiments of the present invention will be described.

(Embodiment 1) A video conference system
It is composed of a plurality of image capturing devices, a voice communication device, and a display device (screen or liquid crystal screen, monitor).

FIG. 1 shows a schematic diagram of a video conference system. The video cameras 1 and 2 as the image capturing device, the line-of-sight detection device 3 and the audio monitor device 4 are installed in the video camera 1, and by connecting the image signal and the audio signal to the communication device 6, the screen 5 and a remote location can be displayed. It transmits signals to the television system and sends images and sounds. The participants 7, 8 and 9 seated on the desk 10 proceed with the conference while displaying images of the remote place and conference materials on the screen 5.

The line-of-sight detecting device 3 uses a near-infrared camera, a reflecting mirror, and an eye camera integrated with each other.
The near-infrared camera detects the position of the eyeball on the face from the retina reflection and traces the line of sight from the eyeball direction. As a result, the line of sight of the speaker can be pursued without contact. By using near-infrared rays, it is possible to detect the eyeball position even in a dark place without significantly affecting humans. Therefore, the line of sight can be effectively tracked even in a conference in a dark room using slides or the like.

The line-of-sight point obtained from the line-of-sight detection device 3 is sent to the video camera 2 via the communication device 6 to capture an image in the range including the line-of-sight point. The image is accumulated for 5 seconds, transmitted to an image display device at a remote place via the communication device 6, and displayed on the screen of the conference room at the remote place. This allows people in remote conference rooms as well as those in the same conference room to know what the speaker is seeing (people, materials, images, etc.) Will help you.

Since the line of sight of a person fluctuates considerably, when a viewpoint for 1 second is displayed as an image at any time, it may change rapidly. Therefore, an image is obtained in which fluctuation is suppressed by accumulating for about 5 seconds and averaging or expanding the field of view.

Further, if the line-of-sight point is shown on the image displayed on the screen 5 in the same manner as the laser pointer, the point of interest of the speaker can be detected, and a real-time conference can be performed.

As for displaying an image, it is easier to understand if the face of the speaker and the place where the speaker is looking can be seen at the same time. Therefore, the image is divided into two and an image surrounded by the speaker and the line-of-sight point is displayed. Is desirable.

Further, since the speaker can freely see the face and line of sight of the person in the same venue, the talk can proceed while seeing the reaction, but the reaction of the person in the remote place cannot be understood. In this way, if you can display the image of the face or line-of-sight of a remote person on the screen, you can see if you know what you are talking about, or if you are listening to it, and proceed while confirming each other. It becomes possible to do.

As described above, by detecting the line of sight of the person who is participating in the conference and displaying the image of the range being viewed on the screen, the presence of the conference participants at the remote location is enhanced and the progress of the conference is smoothed. Can be

(Embodiment 2) It is important to identify the speaker when detecting the line of sight. A method has been proposed in which the speaker level is compared to determine the speaker (Japanese Patent Laid-Open No. 4-150590), and a person who emits a voice that exceeds a certain time is determined to be the speaker (Japanese Patent Laid-Open No. 2-150590). 4095).

However, when the voice is used, the speaker can be easily recognized, but an erroneous determination may occur due to an interfering sound such as noise or cough. Therefore, if the line of sight of the conference participants is tracked and the person whose line of sight is concentrated is determined to be the speaker, the probability of being disturbed by the environment is low. Therefore, by specifying the speaker by combining the voice information and the line-of-sight information, it is possible to more reliably determine the speaker.

The speaker determination algorithm is shown in FIG. First, the voice level of the participant is detected by the voice monitor, and the person with the highest level is selected. At the same time, the line of sight of each participant is detected. Next, the voice levels are compared in each conference room, and the person with the highest level is selected. Furthermore, the speaker is determined by determining whether the line of sight is concentrated on the selected person by detecting the line of sight.

If the speaker can be specified, the person at the remote place can always grasp who is speaking, by emphasizing it by displaying it or marking it larger than other people, and the conference can proceed smoothly. In addition, by cutting out only the voice of the speaker and transmitting the voice largely in the communication device, noise-free voice information can be obtained, and it is possible to approach the level of a person listening in the same conference room.

Thus, if the speaker can be specified, it is very effective in smoothing the progress of the conference.

[0022]

According to the present invention, in a video conference system comprising a plurality of image capturing devices, image display devices, audio monitor devices and communication devices, the image capturing device comprises a line-of-sight detecting device. It is a system, and by detecting the line of sight, the speaker can be identified and the viewpoint of the speaker can be displayed to give the presence of a person in a remote place to be in the same place, which can be useful for smoothly proceeding the conference. .

[Brief description of drawings]

FIG. 1 is a schematic diagram of a video conference system.

FIG. 2 is a diagram showing a speaker identification algorithm.

[Explanation of symbols]

1, 2 video camera 3 Line-of-sight detection device 4 Audio monitor 5 screen 6 Communication device 7,8,9 Participants 10 desks

Claims (6)

[Claims] 1. A video conference system comprising a plurality of image capturing devices, an image display device, a voice monitor device, and a communication device, wherein the image capturing device comprises a line-of-sight detection device. 2. A video conference system comprising a plurality of image capturing devices, an image display device, a voice monitor device, and a communication device, wherein the image capturing device comprises a line-of-sight detection device,
A video conferencing system that provides an image with eye tracking. 3. The video conference system according to claim 2, wherein position information of a place where the line of sight is tracked is accumulated for a certain period of time to provide an image covering the range. 4. The video conference system according to claim 2, wherein the line-of-sight point is displayed on the image of the place where the line-of-sight is tracked. 5. A video conference system comprising a plurality of image capturing devices, an image display device, a voice monitor device and a communication device, wherein the image capturing device comprises a line-of-sight detection device, and the image and audio information tracked by the line-of-sight A video conference system characterized by determining the person. 6. The speaker is determined from the image and the audio information tracked by the line of sight, the speaker is emphasized and displayed on an image display device at a remote location, and an image in the line of sight of the speaker is provided. The video conference system according to claim 5.