JPS62209985A - Video conference equipment - Google Patents

Abstract

PURPOSE:To attain the efficient display and to remarkably reduce transmission information by segmenting a picture with an area having only to include conference participants. CONSTITUTION:Inputted picture information is segmented as a picture having an area as small as possible having only to include the conference participants at conference participant segment sections 3a, 3b and each segmented picture is added with a picture number. Then talker recognition sections 4a, 4b use the segmented picture to recognize the talker in the conference participants. Then the picture information is sent to a remote conference hall by sending sections 5a, 5b together with a picture number and voice information representing the talker and received by reception sections 6b, 6a of an opposite conference hall. The received picture information and the picture number representing the talker are transferred to layout sections 7a, 7b and the voice information is outputted from voice information output sections 9a, 9b.

Description

[Detailed Description of the Invention] [Technical Field of the Invention] The present invention conducts a conference between remote locations by transmitting image information and audio information of each conference hall to the conference hall of the other party, and displaying and emitting sound. The present invention relates to a video conference device.

[Technical background of the invention and its problems] In conventional video conferencing devices, when transmitting and displaying images of a conference hall to a remote location, it is difficult to display images on one screen when there are multiple conference attendees at the conference hall. There was a problem in that the images of each conference attendee were small and difficult to see.

In addition, among the image information to be transmitted, the areas between the conference attendees and the dorsal head portions above the conference attendees, which are indicated by diagonal lines in FIG.
This image is essentially meaningless as conference information, and transmitting and displaying it has the problem of reducing transmission efficiency or display efficiency.

Furthermore, normally people other than the speaker pay attention to the speaker, but in conventional video conferencing devices, the source of the audio information is the same, and as mentioned above, the images of each conference participant are small, so it is difficult to focus on the speaker. cannot be expressed clearly. Published Patent Publication No. 60-116293 attempts to represent the speaker by lighting up a display device that corresponds to each conference attendee, but due to the aforementioned reasons, information such as the speaker's facial expression is insufficient, and There was also the problem of not being able to clearly express who the person was.

[Object of the Invention] The present invention has been made in response to the above-mentioned circumstances, and an object of the present invention is to provide a video conference device that has high transmission and display efficiency and can clearly represent the speaker.

[Summary of the Invention] The present invention provides a video conference device that transmits image information, audio information, and control information via a transmission path, and includes an image input means and a storage means for accumulating images input by the image input means. a means for comparing the accumulated image with the current image inputted by the image inputting means and extracting a region including a changed part; a means for recognizing a speaker and outputting speaker identification information; The extracted image and the extracted area are edited and arranged on one screen using this speaker identification information.

We decided to provide a layout display means for displaying the screen separately.
This improves transmission and display efficiency and allows the speaker to be clearly expressed.

[Embodiments of the invention] Hereinafter, the present invention will be explained in detail with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the present invention.

TV cameras 1a, 1b for transferring conference attendees, and microphones 2a+, 2a2, . . . for collecting sound from each conference attendee.
2b1, 2b2..., conference participant image cropping units 3a and 3b that compare the background image and the current image and crop out an image of only conference attendees, and a speaker recognition unit 4a that recognizes the speaker from this cropped image. , 4b, transmitting sections 5a and 5b that transmit these image information, audio information, and control information, receiving sections 6a and 6b that receive information from the Shinkin Chamber, and a layout that edits and arranges image information behind the received information. Part 7a, 7
b, image information display units 8a and 8b that display the laid out images, and audio information output units 9a and 9b that output audio information.

Next, the operation of this TV conference IA device will be explained.

In this video conference device, first, conference room image information is transmitted by television cameras 1a and 1b, and audio information is transmitted by microphones 2a and 1b.
1.2a2..., 2b1.2b2= is input.

This input image information is stored in the conference participant image cutting section 3a.
, 3b, an image of an area as small as possible encompassing each conference attendee as shown in FIG. 2 (A1
~G+), and an image number is assigned to each cut-out image.

Next, the speaker recognition units 4a and 4b recognize speakers among the conference attendees using the cut out images. As shown in Figure 3, this is done by setting a detection area including the mouth based on the relative position of the conferee image, and detecting changes in the image within that area.
That is, the speaker is recognized by detecting mouth movements.

That is, if the image change (2) is greater than a certain value and this change is detected for a certain period of time or more, the person is recognized as the speaker and the image number is output.

The image information, together with the image number indicating the speaker and the audio information, is sent to a remote conference hall by the transmitters 5a and 5b, and is received by the receivers 6b and 6a at the other party's conference center.

The received image information and the image number indicating the speaker are transferred to the layout sections 7a, 7b, and the audio information is outputted at the audio information output sections 9a, 9b.

Furthermore, in the layout sections 7a and 7b, if there is no image number indicating the speaker, all the images (A2 to C2 in FIG. 4) are laid out in the same size on one screen as shown in FIG. , if there is an image number II indicating the speaker, the image of the conference person with that number (Figure 5 03) is enlarged as shown in Figure 5, and the image of the other person (Figure 5
A3 and B3) are reduced and laid out on one screen, and displayed on the image information display sections 8a and 8b.

FIG. 6 is a block diagram showing the configuration of the conference participant image cutting sections 3a and 3b.

These conference participant image cutting units 3a and 3b include a background image recording unit 10 in which a background image is stored, a change detection unit 11 that compares this background image with an input image, and detects a changed part;
It is comprised of a cutout coordinate calculation unit 12 that calculates the coordinates of a rectangular area that includes the changed portion, and a cutout unit 13 that cuts out the rectangular area from these coordinate values.

In the conference participant image cutting units 3a and 3b, background images without conference attendees are stored in the background image recording unit 10 in advance.

Next, in the change detection section 11, the television cameras 1a, 1b
By taking the difference between the image of the conference hall input from the source and the stored background image, images of only conference attendees are identified as shown in FIG.

Then, the cutout coordinate calculation unit 12 calculates the coordinates of a window that is vertically and horizontally a distance a from the minimum window that includes the image of the conferee, as shown in FIG. This a is set to be as small as possible without making the images of the conference participants difficult to see.

Based on the last cutout coordinates, the cutout unit 13 cuts out an image of the smallest possible area that includes the conference participants.

Note that images of conference participants can be easily detected by looking at connected regions of a certain area or more.

In this television conference device, television cameras 1a, 1
Since an image of an area as small as possible that includes each conference attendee is cut out from the image input in step b, the amount of transmitted image information can be reduced and the efficiency of transmission and display can be improved.

In addition, the speaker is recognized and the conference participant image cutting units 3a, 3
By enlarging and displaying the image of the speaker in the image of the conference participants cut out by b in the layout sections 7a and 7b, and displaying the other people in a reduced size, the speaker can be clearly expressed.

Note that in this embodiment, in order to simplify the explanation, the transmitting side,
Although the receiving side is configured one-to-one, the present invention is not limited to the above embodiment, and for example, transmission and reception can be performed at three or more conference halls. Also, the television camera is not limited to one in the conference room.

In addition, in this embodiment, when cropping out conferee images, clipping is performed based on the difference between the background image and the input image, but in addition to this, changes can be detected by comparing the previous image and the current image at regular intervals. You can also do this.

Roughly speaking, in this embodiment, the speaker is recognized by detecting the movement of the mouth, but there are other methods such as detecting the speaker based on the audio level of each microphone.

Further, although the enlarged display in the layout sections 7a and 7b is automatically performed by recognizing the speaker, manual enlarged display is also possible.

[Effects of the Invention] As explained above, according to the present invention, it is possible to cut out an image of as small an area as possible that includes conference participants, thereby significantly reducing the amount of transmitted information and performing efficient display. be able to.

Since it is possible to roughly recognize the speaker and display the speaker in an enlarged manner while displaying other people in a reduced size, the speaker can be clearly expressed.

[Brief explanation of drawings]

FIG. 1 is a configuration block diagram showing one embodiment of the present invention, and FIG.
The figure is a diagram illustrating the cutting out of a meeting person's image in the same embodiment, FIG. FIG. 5 is a diagram illustrating the layout display of conference participant images in the case where there is a speaker in the same embodiment. FIG. 6 is a configuration block diagram of the conference participant image cutting section. FIG. 8 is a diagram illustrating an algorithm for extracting images of only conference participants, FIG. 8 is a diagram showing a window for cutting out conference participants, and FIG. 9 is a diagram illustrating a screen of a conventional video conference device. 1a, 1b...TV cameras 3a, 3b...Member image cutting units 4a, 4b...
-Speaker recognition units 7a, 7b...Layout unit 10...Background image recording unit 11...
......Change detection section 12...
... Cutting coordinate calculation section 13 ......
... Cutout product applicant Toshiba Corporation Agent Patent attorney Satoshi Suyama - Figure 1 Figure 2 Figure 3 Figure 3L Figure 4 Figure 5 Figure 6 Figure 7

Claims (2)

[Claims] (1) In a television conference device that transmits image information, audio information, and control information via a transmission path, an image input means, a storage means for accumulating images input by the image input means, and a storage means for accumulating images input by the image input means; means for comparing an image with the current image input by the image input means and extracting a region including a changed part; means for recognizing a speaker and outputting speaker identification information; 1. A television conference device comprising: layout display means for editing and arranging the extracted areas based on the speaker identification information and displaying the edited areas on one screen in a divided manner. (2) The video conference device according to claim 1, wherein the layout display means enlarges the speaker and displays other people in a reduced size.