JPH04109784A - Video conference picture display controller and its method - Google Patents

Abstract

PURPOSE:To avoid the necessity of an exclusive operator and the operation by participants by specifying a direction of a speaker based on a voice signal inputted from 2>= microphones, selecting automatically only a picture signal of a specific part, compressing and sending the signal, expanding and displaying automatically only the specific picture part. CONSTITUTION:The above controller is provided with a direction decision circuit 200 being a position specification means specifying the location of a speaker of a voice signal based on the voice collected by a microphone. When a person 10 utters a word, the voice signal inputted to a microphone 21 has a delay of Tsec caused due to a distance difference with respect to the voice signal inputted to a microphone 20. The direction decision circuit 200 specifies the direction of the speaker and generates a control signal representing a coded area according to the specified direction and a decoding or display area as side information and sends a coded area 1 while coding it, After the received picture is decoded at the receiver side, only the part equivalent to the decoded area 1 is magnified and displayed according to the side information.

Description

[Detailed description of the invention] [Industrial application field] INDUSTRIAL APPLICATION This invention is utilized for a video telephone and a conference system.

In particular, the present invention relates to camera and screen control means in a video conference system for multiple people.

〔overview〕

The present invention is a means for controlling the display of an image of a speaker who is speaking in a video conference, which automatically selects the speaker who is speaking, thereby eliminating the need for manual camera operation. .

[Conventional technology]

In some conventional video conference systems, an imaging camera directed toward a speaker among conference participants is controlled on the imaging (transmission) side or the display (reception) side using a joystick or the like. In such video conferencing systems, a dedicated operator is provided on the imaging side or display side, or conference participants can control the camera to point at the speaker and switch the display as appropriate to capture the speaker's image and display the desired image. You can get the effect. However, if there is no operation by a full-time operator or a conference participant, an appropriate image cannot be obtained, and images unrelated to the speaker will be exchanged. Hereinafter, a conventional example in which camera control is performed on the imaging side will be specifically explained using FIG. 6, and a conventional example in which camera control is performed on the display side will be explained using the general block diagram in FIG. 7.

In FIG. 6, 10.11 and 12 are conference participants, 20 and 21 are microphones, 30 is an imaging camera, 31 is a controller for the imaging camera, 50 is an image cog that encodes the image taken by the camera 30, and 51 is A voice cog encodes the voices collected by the microphones 20 and 21; 60 is a multiplexer that multiplexes the encoded voice and image and puts it on a line; 101 is a transmission line; 100 is a receiver for receiving data such as codes. a receiving line; 80 is a demultiplexer that separates received data into audio and image data; 70 is an audio decoder that decodes the separated audio codes into audio signals;
1 is an image decoder that decodes an image code into an image signal; 9
0 is a display TV that displays image signals, and 40 and 41 are speakers that amplify audio signals.

Components in FIG. 7 that are given the same numbers as in FIG. 6 basically have the same functions and operations, so a description thereof will be omitted. What is different in FIG. 7 from FIG. 6 is 61 multiplexers and 81 demultiplexers. The multiplexer 61 multiplexes the image code and the audio code, as well as a control signal from the camera controller 31 that controls the other party's camera, and sends the multiplexed signal. The demultiplexer 81 demultiplexes and separates not only the image code and the audio code but also the signal for controlling the camera sent from the other party.

In conventional TV conference systems, for example, the shooting range is
A human can control the image capturing side of the figure to capture an appropriate screen. On the other hand, a similar effect can be obtained by having a person control the camera on the receiving and displaying side.

[Problem to be solved by the invention]

In such conventional TV conference systems, there is no means to automatically point the camera at the speaker, and human operation is required.
Either an extra person is required to control the camera, or participants are forced to control the camera themselves, causing problems such as extra personnel and difficulty concentrating on the meeting due to camera control.

Furthermore, when controlling the camera on the image receiving side, it is necessary to know in advance the arrangement order of the people on the other side and the characteristics of the voices in order to control the camera on the transmitting side, and use this memory to determine the direction of the speaker. There was a problem in that there was no way to avoid specifying the direction and controlling the camera, and the direction had to be specified by trial and error. Furthermore, when a full-time operator is used, there is the problem that not only does the number of personnel increase, but also the content of the meeting can be heard by a third party.

The present invention aims to eliminate such drawbacks and to provide a video conference image display control device and method that eliminates the need for manual camera operation.

[Means to solve the problem]

The present invention includes a camera that takes an image, a sound collection means that collects sound, an image signal compression means that compresses an image signal given from the camera to generate a compressed image signal, and an audio signal given from the sound collection means. audio signal compression means for compressing and generating a compressed audio signal; multiplexing transmission means for multiplexing the compressed image signal and the compressed audio signal to generate and transmit a multiplexed signal; a signal separating means for separating an incoming multiplexed signal into a compressed image signal and a compressed audio signal; an image signal expanding means for expanding the compressed image signal separated by the signal separating means; and a signal separating means for expanding the compressed image signal separated by the signal separating means; audio signal expansion means for expanding the compressed audio signal; display means for displaying an image signal related to the image signal expanded by the image signal expansion means; and amplification means for amplifying the audio signal expanded by the audio signal expansion means. A video conference type image encoding and display control device comprising a plurality of the above-mentioned sound collecting means, and identifying the position of the speaker of the sound based on the sound collected by the plurality of sound collecting means. It is characterized by being equipped with a position specifying means.

Here, it is desirable to provide an image signal transmission prohibition means for prohibiting image signals other than image signals in the vicinity of the position specified by the position specifying means from being provided to the image signal compression means.

Further, in place of the specific position information transmitting means for transmitting the position information indicating the position specified by the position specifying means and the above-mentioned image signal transmission prohibiting means, the compressed image signal excluding the part of the compressed image signal indicating the incoming position information is provided. It is desirable to include an image signal expansion prohibition means for prohibiting the image signal from being applied to the image signal expansion means.

Further, in place of the image signal decompression inhibiting means, it is preferable to include a display enlarging means for enlarging a portion indicated by the positional information that arrives after the incoming compressed image signal has been decompressed and providing it to the display means.

[Effect]

The direction of the speaker is identified based on the audio input from at least two microphones, and only the image signal of a specific part of the image input from the camera is automatically selected according to the result of the speaker's location identification means. the compressed signal of the television screen is decompressed based on the received positioning result; control, and automatically expands and displays only the signal of a specific image part.

〔Example〕

Hereinafter, one embodiment of the present invention will be explained based on the drawings. 1st
The figure is a first embodiment, and is an overall block configuration diagram showing a case where a specific area of an image taken by - cameras is encoded and transmitted. Components in FIG. 1 labeled with the same numbers as in FIGS. 6 and 7 basically have the same functions, and therefore their explanations will be omitted. In Figure 1, 200 is two microphones 20
and a direction determination circuit that determines the direction of the source of the voice, etc. based on the arrival time difference of the voice input manually from 21 and the installation interval of the microphones 20 and 21, and generates and outputs a signal indicating the area to be encoded. be. 300 is direction determination circuit 2
This is an encoding control circuit that encodes a specific area of a camera-captured image according to a signal output from 00.

FIG. 2 is an overall block configuration diagram showing a second embodiment in which an image captured from a camera is encoded and transmitted as is, but information specifying a decoding area is added to side information. In Figure 2, Figures 1, 6 and 7
Components with the same numbers as those in the drawings have the same functions, so their explanation will be omitted. In the second embodiment, the multiplexer 60 multiplexes the results of the direction determination circuit 200 in addition to the encoded audio and image signals. On the other hand, in contrast to the multiplexer 60, the demultiplexer 80 separates side information specifying the coding area in addition to the coded audio and image signals. The decoding control circuit 400 extracts the encoded image signal of the designated area based on the side information that designates the decoding area and sends it to the image decoder 71.
Send to. The image decoder 71 decodes encoded data of a part of the source image and sends it to the display TV 90.

FIG. 3 shows that in the third embodiment, the image captured from the camera 30 is encoded and transmitted as is, but information specifying the decoding area is added to the side information and transmitted, and the receiving side displays it after decoding. FIG. 2 is an overall block configuration diagram showing a case where an area is specified and displayed in an enlarged manner. In Figure 3, Figure 1, 2
Components with the same numbers as in FIG. 6, FIG. 7, and FIG. 7 have the same functions, so their explanation will be omitted. In the third embodiment, the multiplexer 60 multiplexes the encoded audio and image signals as well as the results of the direction determination circuit 200. On the other hand, in contrast to the multiplexer 60, the demultiplexer 80 separates side information specifying the coding area in addition to the coded audio and image signals. Image decoder 71 decodes the transmitted image data.

The display control circuit 500 takes out the image signal of the designated area based on the transmitted side information designating the display area, enlarges it, and sends it to the display TV 90.

That is, this embodiment includes a camera 30 for capturing an image, microphones 20 and 21 as a sound collecting means for collecting sound, and a camera 30 for capturing an image.
an image coder 50 which is an image signal compression means for compressing an image signal given from 0 to generate a compressed image signal;
Audio signal compression means for compressing the audio signal given from the recording sound collection means to generate a compressed audio signal
1, a multiplexer 60 which is a multiplexing transmission means for multiplexing the compressed image signal and the compressed audio signal to generate and transmit a multiplexed signal; A demultiplexer 80 is a signal separating means for separating a signal and a compressed audio signal; an image decoder 71 is an image signal expanding means for expanding the compressed image signal separated by the signal separating means; an audio decoder 70 which is an audio signal decompressing means for decompressing the compressed audio signal; a display TV which is a display means for displaying an image signal related to the image signal decompressed by the image signal decompressing means; and the audio signal decompressing means. and loudspeakers 40 and 41, which are amplifying means for amplifying the audio signal decompressed by the sound collecting means. A direction determination circuit 200 is provided as a position specifying means for specifying a position, wherein a first invention provides the image signal compression means with an image signal excluding image signals in the vicinity of the position specified by the position specifying means. The 20th invention includes a coding control circuit 300 which is an image signal transmission prohibition means for prohibiting image signal transmission, and the 20th invention includes specific position information transmitting means for transmitting position information indicating the position specified by the position specifying means to the multiplexer 60. , instead of the image signal compression means, a decoding control circuit 400 is provided which is an image signal decompression prohibition means for prohibiting the compressed image signal excluding the portion of the compressed image signal indicated by the incoming position information from being given to the image signal decompression means. Prepare,
A third aspect of the present invention is a display control circuit 500 which is a display enlargement means, instead of the image signal decompression inhibiting means, which enlarges a portion indicated by the positional information that arrives after decompression processing of the incoming compressed image signal and provides it to the display means. Equipped with

Next, the operation will be explained. For ease of explanation, humans 10, 11 and 12 and microphones 20 and 2 in Figure 1 are used.
The positional (distance) relationship of 1 will be explained using the state shown in FIG. 8th
In the figure, microphones 20 and 21 capture the voices of conference participants and convert them into electrical signals for output. When the human 10 is speaking, the voice signal inputted to the microphone 21 is transmitted to the microphone 2.
There is a delay of T seconds caused by the distance difference with respect to the human input audio signal. Therefore, if the speed of sound is S, then Y=
There is a relationship between TXS+X. An example of the output of the microphones 20 and 21 is shown in FIG. In FIG. 9, the upper row shows the signal waveform output from the microphone 20, and the lower row shows the signal waveform output from the microphone 21. In the example shown in FIG. 8, when the signal waveform of the microphone 21 is observed based on the signal waveform of the microphone 20, the following three cases exist.

■ If the delay T of the signal waveform of the microphone 21 is a positive value: the human 10 speaks ■ If the delay T between the signal waveforms of the microphones 20 and 21 is 0: the human 11 speaks ■ The delay T of the signal waveform of the microphone 21 If is a negative value: the human 12 is speaking. In this example, in order to simplify the explanation, there are three people, two microphones, and the distance relationship between them is the same. However, the total number of people, the total number of microphones, etc. are plural, and Even if the distance between the microphones is uneven, the person should be aligned in a straight line at right angles to the camera's optical axis to the extent that it does not interfere with the TV camera's shooting, and the microphone placement should be relative to the person's arrangement. If the microphones are almost parallel or orthogonal, and the distance between the microphones and the difference in arrival time of the signals are clear,
Direction can be determined using a similar basic principle. The direction determining circuit 200 specifies the direction of the speaker, and generates a control signal indicating the encoding area and a decoding or display area as side information according to the specified direction. For example, in the case of (2) above, in the first embodiment, encoding area 1 in FIG. 4 of the camera-captured image is encoded and transmitted. Furthermore, in the second embodiment, side information indicating decoding area 1 in FIG. 5 is generated and sent. Also, in the third embodiment, similarly to the second embodiment, side information is sent, and after decoding the received image on the receiving side, only the portion corresponding to decoding area 1 in FIG. 5 is enlarged according to the side information. and display it.

〔Effect of the invention〕

As explained above, the present invention eliminates the need for a TV camera operator to select a photographed image and avoids imitations by a full-time operator or participants, thereby facilitating the smooth progress of a meeting. This has the effect of avoiding the problem of secret leakage by operators.

[Brief explanation of drawings]

FIG. 1 is an overall block diagram showing a first embodiment of the present invention. FIG. 2 is an overall block diagram showing a second embodiment. FIG. 3 is an overall block diagram showing a third embodiment. FIG. 4 is a diagram showing an example of a coding area. FIG. 5 is a diagram showing an example of a decoding area. FIG. 6 is an overall block diagram showing a conventional example. FIG. 7 is an overall block diagram showing a conventional example. FIG. 8 is a diagram showing an example of the positional relationship between microphones and conference participants. FIG. 9 is a diagram showing the signal waveform of audio input to the microphone. 10.11.12...Human, 20.21...Microphone, 30...Camera, 31...Camera controller,
40.41...Speaker, 50...Image Korg, 5
1... Audio cog, 60. 61... Multiplexer, 70... Audio decoder, 71... Image decoder,
80.81... Demultiplexer, 90... Display T
V. 100... Receiving line, 101... Transmitting line, 200
... Direction determination circuit, 300 ... Encoding control circuit, 4
00...Decoding control circuit, 500...Display control circuit.

Claims (1)

[Scope of Claims] 1. A camera that takes an image, a sound collection means that collects sound, an image signal compression means that compresses an image signal provided from the camera to generate a compressed image signal, and a video signal provided from the sound collection means. audio signal compression means for compressing an audio signal to generate a compressed audio signal; multiplexing transmission means for multiplexing the compressed image signal and the compressed audio signal to generate and transmit a multiplexed signal; a signal separating means for separating the multiplexed signal arriving from the signal separating means into a compressed image signal and a compressed audio signal; an image signal expanding means for expanding the compressed image signal separated by the signal separating means; and the signal separating means. audio signal expansion means for expanding the compressed audio signal separated by the image signal expansion means; display means for displaying an image signal related to the image signal expanded by the image signal expansion means; A video conference image display control device equipped with a loudspeaker means for amplifying the sound, comprising a plurality of the above-mentioned sound collecting means, and a position for identifying the position of the speaker of the sound based on the sound collected by the plurality of sound collecting means. A video conference image display control device comprising specifying means. 2. The video conference image display control according to claim 1, further comprising image signal transmission prohibiting means for prohibiting image signals other than image signals in the vicinity of the position specified by the position specifying means from being provided to the image signal compression means. Device. 3. Specific position information transmitting means for transmitting position information indicating the position specified by the position specifying means, and instead of the image signal compression means, transmitting the compressed image signal excluding the part of the compressed image signal indicating the incoming position information. 2. The video conference image display control device according to claim 1, further comprising image signal decompression prohibition means for prohibiting the image signal from being provided to the image signal decompression means. 4. The television set according to claim 3, further comprising display enlarging means in place of the image signal decompression inhibiting means, which magnifies a portion indicated by positional information that arrives after decompression processing of the incoming compressed image signal and provides it to the display means. Conference image display control device. 5. A first step in which an image signal and an audio signal are generated by a camera that images a scene including multiple speakers and a sound collection means that collects the voices of the speakers; The second step is to compress and encode the received signals, multiplex them and send them out, and separate the incoming multiplexed signals, expand each separated signal to restore the image signal and the audio signal, and then and a third step of displaying an image signal. 1. A video conference image display control method comprising the steps of specifying the location of a speaker based on the location of the speaker, selecting the speaker at the identified location, and generating an image signal related to the selected speaker. 6. A first step of generating an image signal and an audio signal with each of a camera that images a scene including a plurality of speakers and a sound collection means that collects the voices of the speakers; a second step of compressing and encoding the generated signal, multiplexing it and sending it out; separating the incoming multiplexed signal and decompressing each separated signal to restore the image signal and the audio signal; In the video conference image display control method comprising a third step of displaying the image signal, the first step includes placing the sound collecting means at separate positions, the step of identifying the location of the speaker based on the time difference, generating a selection signal indicating the identified location, and generating an image signal corresponding to a scene including the plurality of speakers together with the selection signal; A video conference image display control method characterized in that the third step includes the step of expanding and displaying an image signal related to a speaker selected from incoming image signals based on an incoming selection signal. 7. A first step of generating an image signal and an audio signal with each of a camera that images a scene including a plurality of speakers and a sound collection means that collects the voices of the speakers; a second step of compressing and encoding the generated signal, multiplexing it and sending it out; separating the incoming multiplexed signal and decompressing each separated signal to restore the image signal and the audio signal; In the video conference image display control method comprising a third step of displaying the image signal, the first step includes placing the sound collecting means at separate positions, the step of identifying the location of the speaker based on the time difference, generating a selection signal indicating the identified location, and generating an image signal corresponding to a scene including the plurality of speakers together with the selection signal; The television is characterized in that the third step includes the step of expanding the incoming image signal, and selecting and enlarging and displaying the image signal related to the speaker from among the expanded image signals based on the incoming selection signal. Conference image display control method.