JP2630041B2 - Video conference image display control method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention is used for videophone and conference systems. In particular, the present invention relates to a camera and screen control means in a video conference system for a plurality of people.

〔Overview〕

According to the present invention, in a means for controlling the display of an image of a speaker who is speaking in a video conference, a manual camera operation can be eliminated by automatically selecting a speaker who is speaking.

[Conventional technology]

2. Description of the Related Art Some conventional video conference systems use a joystick or the like to control an imaging camera for a speaking vehicle among conference attendees on an imaging (transmission) side or a display (reception) side. In such a video conference system, a dedicated operator is provided on the imaging side or the display side, or a conference participant captures an image of the speaker by performing camera control or display switching for the speaker as appropriate, and obtains a desired image. You can get the effect. However, if there is no operation by a dedicated operator or a meeting participant, an appropriate image cannot be obtained, and an image irrelevant to the speaker is exchanged. Hereinafter, a conventional example in which camera control is performed on the image pickup side will be specifically described with reference to FIG. 6 and a conventional example in which camera control will be performed on the display side will be described with reference to the entire block diagram of FIG.

In FIG. 6, 10, 11 and 12 are conference participants, 20 and 21
Is a microphone, 30 is an imaging camera, 31 is an imaging camera controller, 50 is an image coder that encodes an image captured by the camera 30, 51 is an audio coder that encodes audio collected by the microphones 20 and 21, and 60 is an audio coder. A multiplexer for multiplexing coded audio and video on a line, 101 for a transmitting circuit, 100 for a receiving line for receiving data such as codes, 80
Is a demultiplexer that separates received data into an audio system and an image system, 70 is an audio decoder that decodes the separated audio code into an audio signal, 71 is an image decoder that decodes the image code into an image signal, and 90 is an image signal Display TV, 40 and
Reference numeral 41 denotes a speaker that loudspeaks an audio signal.

In FIG. 7, the components denoted by the same reference numerals as those in FIG. 6 basically perform the same functions and operations, and thus the description thereof is omitted. FIG. 7 differs from FIG. 6 in that there are 61 multiplexers and 81 demultiplexers. The multiplexer 61 multiplexes and transmits the control signal of the camera controller 31 for controlling the other camera in addition to the image code and the audio code. The demultiplexer 81 demultiplexes and separates a camera control signal sent from the other party in addition to the image code and the audio code.

In a conventional video conference system, for example, the shooting range is set to the sixth
An appropriate screen can be photographed under the control of a human on the imaging side of the image shown in FIG. On the other hand, the same effect can be obtained by controlling the camera on the receiving and displaying side.

[Problems to be solved by the invention]

In such a conventional video conference system, there is no means to automatically point the camera at the speaker, and human operation is required.
This causes problems such as using an extra person for controlling the camera or having the participant control the camera by himself / herself, generating extra personnel and being unable to concentrate on the conference due to the camera control. Furthermore, when controlling the camera on the receiving side of the image, in order to control the camera on the transmitting side, the arrangement sequence and voice characteristics of the other party are grasped in advance, and the direction of the speaker is stored in this memory. There is no means for avoiding controlling the camera, and the direction must be specified by trial and error. In addition, when a dedicated operator is used, there is a problem that the contents of the conference may be heard by a third party as the number of human resources increases.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a video conference image display control apparatus and method which eliminates such disadvantages and eliminates the need for manual camera operation.

[Means for solving the problem]

The present invention provides a first step of generating an image signal and an audio signal with a camera that captures a scene including a plurality of speakers and a sound collecting unit that collects the sounds of the speakers, respectively. A second step of compressing and encoding the signal generated in step 2 and multiplexing and transmitting the separated signal, separating an incoming multiplexed signal, and decompressing each of the separated signals to restore an image signal and an audio signal. And a third step of displaying the image signal. In the video conference image display control method, the first step comprises placing the sound collecting means at a separated position, and arriving at each of the sound collecting means. Identifying the position of the speaker based on the time difference of the speaker, selecting the speaker at the specified position, and generating an image signal related to the selected speaker. I do.

Here, in the first step, the sound collecting means is placed at a separated position, a position of a speaker is specified based on a time difference between sounds arriving at each of the sound collecting means, and a selection indicating the specified position is performed. Generating an image signal corresponding to the scene including the plurality of speakers together with the selection signal, wherein the third step includes selecting an utterance selected from the arriving image signal based on the arriving selection signal. Expanding and displaying the image signal relating to the user.

In the first step, the sound collecting means is placed at a separated position, a position of a speaker is specified based on a time difference between sounds arriving at the sound collecting means, and a selection signal indicating the specified position. Generating an image signal corresponding to a scene including the plurality of speakers together with the selection signal, and the third step decompresses the incoming image signal and generates the image signal based on the incoming selection signal. The method may include a step of selecting an image signal related to the speaker from the expanded image signals and enlarging and displaying the selected image signal.

[Action]

The direction of the speaker is specified based on the voices input from at least two or more microphones, and only the image signal of a specific portion of the image input from the camera is automatically selected according to the result of the position specifying means of the speaker. Compressing and transmitting, and transmitting the result of the speaker's location determining means, receiving the transmitted location determination result, and decompressing the compressed signal of the television screen based on the received location determination result. Control and automatically expand and display only the signal of the specific image part.

〔Example〕

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall block diagram showing a case where a specific area of an image taken by one camera is encoded and transmitted in the first embodiment. In FIG. 1, the components denoted by the same reference numerals as those in FIGS. 6 and 7 have basically the same functions, and therefore description thereof will be omitted. In FIG. 1, reference numeral 200 denotes a difference between the arrival time difference of the sound input from the two microphones 20 and 21 and the microphone 20.
And a direction stabilizing circuit that determines the direction of a transmission source such as a voice based on the installation intervals of and 21 and generates and outputs a signal indicating an encoding target area. Reference numeral 300 denotes an encoding control circuit that encodes a specific area of an image captured by the camera according to a signal output from the direction determination circuit 200.

FIG. 2 is an overall block diagram showing a second embodiment in which an image captured from a camera is directly encoded and transmitted, but information specifying a decoding area is added to side information. In FIG. 2, components denoted by the same reference numerals as those in FIG. 1, FIG. 6, and FIG. 7 have the same functions, and therefore description thereof will be omitted. In the second embodiment, the multiplexer 60 multiplexes the result of the direction determination circuit 200 in addition to the encoded audio and image signals. on the other hand,
The demultiplexer 80 separates coded audio and image signals as well as side information for specifying a coded area, contrary to the multiplexer 60. The decoding control circuit 400 extracts the coded image signal of the designated area based on the side information designating the decoding area, and sends it to the image decoder 71. The image decoder 71 decodes part of the encoded data of the source image and sends it to the display TV 90.

FIG. 3 shows an image taken from the camera 30 in the third embodiment, which is encoded and transmitted as it is, but is transmitted after adding information for designating a decoding area to side information and after decoding on the receiving side. FIG. 3 is an overall block configuration diagram showing a case where an area is specified and enlarged and displayed. In FIG. 3, FIG.
Components having the same reference numerals as those in FIGS. 6, 6 and 7 have the same functions, and therefore description thereof will be omitted. In the third embodiment, the multiplexer 60 multiplexes the result of the direction determination circuit 200 in addition to the encoded audio and image signals. On the other hand, the demultiplexer 80 separates the side information for specifying the coding area in addition to the coded audio and image signals, contrary to the multiplexer 60. The image decoder 71 decodes the transmitted image data. The display control circuit 500 extracts and enlarges the image signal of the designated area based on the transmitted side information designating the display area, and displays the image signal.
Send to VT90.

That is, this embodiment is a camera 30 that captures an image, microphones 20 and 21 that are sound collecting units that collect sound, and an image signal compressing unit that generates a compressed image signal by compressing an image signal given from the camera 30. An image coder 50; an audio coder 51 which is an audio signal compression unit for generating a compressed audio signal by compressing an audio signal provided from the sound collection unit; and multiplexing the compressed image signal and the compressed audio signal by multiplexing. And a demultiplexer 80 serving as signal separating means for separating the multiplexed signal coming from the multiplexing and transmitting means into a compressed image signal and a compressed audio signal. And an image decoder 71 which is an image signal decompression means for decompressing the compressed image signal separated by the signal separation means, and an audio signal for decompressing the compressed audio signal separated by the signal separation means. Audio decoder is Zhang means
70, a display TV which is a display means for displaying an image signal related to the image signal expanded by the image signal expansion means, and a speaker 40 which is a loudspeaker which expands the audio signal expanded by the audio signal expansion means. 41, further comprising, as a feature of the present invention, a direction determining circuit 200 which is a position specifying means for specifying a position of a speaker of the sound based on the sound collected by the sound collecting means. According to a first aspect of the present invention, there is provided an encoding control circuit which is an image signal transmission prohibiting unit that prohibits an image signal excluding an image signal near the position specified by the position specifying unit from being given to the image signal compressing unit.
The second invention includes a specific position information transmitting unit that transmits the position information indicating the position specified by the position specifying unit to the multiplexer 60, and replaces the image signal compression unit with the incoming position information. An encoding control circuit which is an image signal decompression prohibiting means for prohibiting a compressed image signal excluding a part of the compressed image signal shown from being given to the image signal decompression means.
A third aspect of the present invention is a display enlarging unit that includes 400 and replaces the image signal decompression prohibiting unit with a portion that indicates the position information arriving after the decompression processing of the arriving compressed image signal and provides the enlarged portion to the display unit. A display control circuit 500 is provided.

Next, the operation will be described. For the sake of simplicity, the positional (distance) relationship between the humans 10, 11, and 12 and the microphones 20 and 21 in FIG. In FIG. 8, the microphones 20 and 21 capture the utterance of the conference participant, convert it into an electric signal, and output it. When the human 10 is speaking, the audio signal input to the microphone 21 has a delay of T seconds caused by the distance difference from the audio signal input to the microphone 20.
Therefore, if the speed of sound is S, there is a relationship of Y = T × S + X. An example of the output of the microphones 20 and 21 is shown in FIG. In FIG. 9, the upper part shows the signal waveform output from the microphone 20, and the lower part shows the signal waveform output from the microphone 21. In the example shown in FIG. 8, when the signal waveform of the microphone 21 is observed with reference to the signal waveform of the microphone 20, the following three cases exist.

When the delay T of the signal waveform of the microphone 21 is a positive value: human
10 is utterance When delay T of signal waveform of microphones 20 and 21 is 0: human
11 is utterance When the delay T of the signal waveform of the microphone 21 is a negative value: human
12 is utterance. In this embodiment, for the sake of simplicity, three humans, two microphones, and the same mutual distance relationship are used. However, the total number of humans, the total number of microphones, and the like are plural, and Even when the distance relationship between the microphones is not uniform, humans are aligned in a straight line near the optical axis of the camera so that they do not interfere with shooting with a TV camera, and the microphone arrangement is If the distance between the microphones and the arrival time difference of the signal are clear when the directions are almost parallel or nearly orthogonal, the direction can be specified by the same basic principle. The direction determination circuit 200 specifies the direction of the speaker, and generates a control signal indicating a coding area and a decoding or display area as side information according to the specified direction. For example, in the above case, in the first embodiment, the fourth
The encoding area 1 in the figure is encoded and transmitted. In the second embodiment, side information indicating the decoding area 1 in FIG. 5 is generated and transmitted. Also, in the third embodiment, as in the second embodiment, after transmitting the side information and decoding the received image on the receiving side, only the portion corresponding to the decoding area 1 in FIG. 5 is enlarged according to the side information. To display.

〔The invention's effect〕

As described above, the present invention does not require the operator of the TV camera to select a captured image, and can avoid the operation of a dedicated operator or a participant. There is an effect that the problem of secret leakage by the operator can be avoided.

[Brief description of the drawings]

FIG. 1 is an overall block diagram showing a first embodiment of the present invention. FIG. 2 is an overall block diagram showing a second embodiment. FIG. 3 is an overall block diagram showing a third embodiment. FIG. 4 is a diagram showing an example of a coding area. FIG. 5 is a diagram showing an example of a decoding area. FIG. 6 is an overall block diagram showing a conventional example. FIG. 7 is an overall block diagram showing a conventional example. FIG. 8 is a diagram showing an example of a positional relationship between a microphone and conference participants. FIG. 9 is a diagram showing a signal waveform of a sound input to a microphone. 10, 11, 12 ... human, 20, 21 ... microphone, 30 ... camera, 31 ... camera controller, 40, 41 ... speaker,
50 ... Image coder, 51 ... Audio coder, 60, 61 ... Multiplexer, 70 ... Audio decoder, 71 ... Image decoder, 80, 81 ... Demultiplexer, 90 ... Display TV, 100
...... Reception line, 101 ... Transmission line, 200 ... Direction determination circuit, 300 ... Encoding control circuit, 400 ... Decoding control circuit,
500 Display control circuit.

Claims (3)

(57) [Claims] 1. A first step of generating an image signal and an audio signal by a camera that captures a scene including a plurality of speakers and a sound collecting unit that collects the voices of the speakers, respectively. A second step of compressing and encoding the signals generated in the step and then multiplexing and transmitting the signals; separating an incoming multiplexed signal; decompressing each of the separated signals to restore an image signal and an audio signal And a third step of displaying the image signal. In the video conference image display control method, the first step includes arranging the sound collecting means at a separated position and arriving at each of the sound collecting means. Identifying the position of the speaker based on the time difference of the voice, selecting the speaker at the specified position, and generating an image signal related to the selected speaker. Video conference image display control method. 2. A first step of generating an image signal and an audio signal by a camera that captures a scene including a plurality of speakers and a sound collecting unit that collects the sounds of the speakers, respectively. A second step of compressing and encoding the signal generated in one step and then multiplexing and transmitting the separated signal; separating an incoming multiplexed signal; expanding each of the separated signals to form an image signal and an audio signal; Restoring and displaying the image signal in the third step of the video conference image display control method, wherein the first step includes placing the sound collecting means at a separated position, and arriving at each of the sound collecting means. The position of the speaker is specified based on the time difference of the sound to be generated, a selection signal indicating the specified position is generated, and an image signal corresponding to the scene including the plurality of speakers is generated together with the selection signal. A video conference image, wherein the third step includes a step of decompressing and displaying an image signal relating to a selected speaker from the incoming image signal based on the incoming selection signal. Display control method. 3. A first step of generating each of an image signal and an audio signal by each of a camera that captures a scene including a plurality of speakers and sound collecting means that collects the voices of the speakers, and A second step of compressing and encoding the signal generated in one step and then multiplexing and transmitting the separated signal; separating an incoming multiplexed signal; expanding each of the separated signals to form an image signal and an audio signal; Restoring and displaying the image signal in the third step of the video conference image display control method, wherein the first step includes placing the sound collecting means at a separated position, and arriving at each of the sound collecting means. The position of the speaker is specified based on the time difference of the sound to be generated, a selection signal indicating the specified position is generated, and an image signal corresponding to the scene including the plurality of speakers is generated together with the selection signal. Generating the signal, the third step includes expanding the incoming image signal, and selecting and enlarging and displaying, based on the incoming selection signal, an image signal related to the speaker of the expanded image signal. A video conference image display control method comprising the steps of: