JPH06276427A - Voice controller with motion picture - Google Patents

Abstract

PURPOSE:To obtain the voice controller in which conversation is in progress without a stop by varying a delay amount of 5/3 voice data with respect to picture data in response to an output signal of a zoom position detection means. CONSTITUTION:The controller is made up of video cameras 4, 12 picking up an object, microphones 3, 11 collecting voice signals, and zoom position detectors 5, 13 detecting a zoom position of the video cameras 4, 12, and a delay amount of the voice data with respect to picture data is varied with an output signal from the zoom position detectors 5, 13. Moreover, a delay time T for the voice data at a sender side to output the received motion picture data and voice data coincident timewise with each other is obtained by an equation of T=(t1-t2)+(t3-t4) (t1 is coding compression processing time for motion picture data, t2 is coding compression processing time for voice data, t3 is a decoding expansion processing time for motion picture data, t4 is decoding expansion processing time for voice data).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio control apparatus suitable for use in, for example, a video conference system or a video telephone system.

[0002]

2. Description of the Related Art A video conferencing system in which a plurality of conference rooms located at different points are connected with each other by video and audio, and a conference can be held while watching a monitor television in a manner that all attendees are in the same room. Has been put to practical use. This video conference system is a device for reading information of a camera or the like as moving image data or sound data, a transmitter for transmitting moving image data and sound data from this device, and moving image data transmitted from the partner. And monitors that output audio data. More specifically, it includes a television monitor for reproducing image data, a speaker for reproducing received audio data, a video camera for capturing and transmitting a subject, a microphone to which audio data is input, and the like. . In addition, the video camera,
A zoom function for the subject is added.

By the way, in the video conference system, moving image data and audio data are transmitted through digital lines. In this case, the data to be transmitted is digitally encoded and further encoded and compressed to reduce the transmission capacity.

On the transmitting side, the processing time required for encoding and compressing moving image data becomes longer than the processing time required for encoding and compressing audio data. On the other hand, on the receiving side, the data transmitted from the transmitting side is output after being subjected to processing such as decoding / expansion. In this way, the receiving side must also perform a series of processes such as decoding and expansion on the transmitted data. In this case, the processing time required for decoding and expanding the moving image data is longer than the processing time required for decoding and expanding the audio data.

Since such data processing is performed on the moving image data and the sound data on both the transmitting side and the receiving side, the moving image and the sound are delayed and output on the receiving side. In particular, when the subject is a person and the image is captured by zooming in, the generated sound must be output on the receiving side in synchronization with the movement of the lips. However, in reality, the voice is output faster than the movement of the lips.

In order to eliminate such inconsistency between the moving image and the sound, the following method can be considered. That is, the time difference between the encoding / compression processing time of audio data and the encoding / compression processing time of moving image data on the transmitting side, and the decoding / expansion processing time of audio data and the decoding / expansion processing time of moving image data on the receiving side. This is a method of delaying the voice data from the transmitting side by the time difference of. By using this method, the movement of the lips and the voice can be output in synchronization even when the person is zoomed up for imaging.

[0007]

When a person is picked up by zooming in, if the above-described method is used in order to output the moving image and the sound in agreement with each other, the receiving side can give a conversation response. It gets worse. That is, since the audio data is output at the output timing of the image data,
That is, since both the image data and the audio data are delayed and output, the overall flow of the conversation becomes unnatural.

When the subject is imaged in a normal size, the lips of the subject on the transmitting side output to the receiving side are
It becomes small on the screen. Therefore, the receiving side does not feel any unnaturalness even if the lip movement and the voice are not output in synchronization with each other to some extent. That is, when the subject is imaged by other than zooming in, the flow of conversation does not become unnatural even if the moving image data and the audio data are not output in synchronization. However, using the above method,
Image data and audio data are always output with a delay.

Therefore, an object of the present invention is to determine whether to synchronize the output timings of image data and audio data according to the zoom position in a video conference system or the like, and to output the output timings of both data when zooming up. Another object of the present invention is to provide a voice control device capable of performing conversation without delay by synchronizing the above and outputting the output timings of both of them in a non-synchronized state except when zooming in.

[0010]

The present invention comprises a video camera for picking up an image of a subject, a microphone for collecting sound, and a zoom position detecting means for detecting a zoom position of the video camera. A moving image / sound control device characterized in that a delay amount of sound data with respect to image data is changed according to an output signal.

[0011]

When the zoom function is operated, the output voltage of the potentiometer 21 changes, and this voltage value is supplied to the A / D converter 23. The digital signal output from the A / D converter 23 is supplied to the controller 24. When this digital signal is larger than the preset initial value Th, a control signal indicating synchronous output of moving image data and audio data is supplied to the encoder / decoder 25. On the other hand, when the digital signal is smaller than the initial value Th, a control signal indicating that the moving image data and the audio data are output without synchronization is supplied to the encoder / decoder 25.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of a video conference system. In FIG. 1, a first system 1 includes a user 2, a microphone 3 into which the voice of the user 2 is input, a video camera 4 that captures the image of the user 2, and a zoom position detector 5 that detects the position of a zoom lens of the video camera 4. , A speaker 6 for outputting the voice from the other party, a display monitor 7 for displaying the subject of the other party, and an encoding / decoding in which the microphone 3, the video camera 4, the zoom position detector 5, the speaker 6 and the display monitor 7 are connected. It consists of container 8. This encoder / decoder 8 is bidirectionally connected to the second system 9.

The second system 9 includes a user 10, a microphone 11 through which the user 10 inputs a voice, and a user 10.
A video camera 12 that captures images, a zoom position detector 13 that detects the position of the zoom lens of the video camera 12, a speaker 14 that outputs voice from the other party, a display monitor 15 that displays the subject of the other party, and a microphone 1.
1, a video camera 12, a zoom position detector 13, a speaker 14, and an encoder / decoder 16 to which a display monitor 15 is connected.

In such a video conference system,
When data is transmitted and received, for example, when data is transmitted from the first system 1 to the second system 9, the user 2 is imaged by the video camera 4. At the same time, the zoom position detector 5 detects the zoom lens position of the video camera 4. The zoom position detector 5
Is composed of, for example, a potentiometer. An image pickup signal of the subject 2 picked up by the video camera 4 is supplied to the encoder / decoder 8.

The moving image data and audio data supplied to the encoder / decoder 8 are encoded and compressed and transmitted to the second system 9. The encoder / decoder 16 decodes and expands this data. After that, the moving image data is output from the display monitor 15, and the audio data is output from the speaker 14. The same process is performed when data is transmitted from the second system 9 to the first system 1.

By the way, the delay time T of the audio data on the transmitting side for outputting the received moving image data and audio data in a matched manner is T = (t1-t2) + (t3-t4) (t1: moving image Image data encoding / compression processing time t2: Audio data encoding / compression processing time t3: Moving image data decoding / expansion processing time t2: Audio data decoding / expansion processing time). That is, by delaying and outputting the audio data by the audio delay time T, the moving image data and the audio data are output in coincidence.

In FIG. 2, the zoom position detector 5 or 13 is shown.
A block diagram showing a detailed configuration of the above is shown. In FIG. 2, the zoom position detection unit 5 includes a potentiometer 21 and a control unit 22. Further, the control unit 22 includes an A / D converter 23 and a controller 24.

When zoom control is performed in which the zoom lens is moved to the wide-angle side, the voltage value of the potentiometer 21 decreases, for example. Further, when moved to the telephoto side, the voltage value of the potentiometer 21 increases, for example. The output voltage of the potentiometer 21 is supplied to the A / D converter 23 and converted into digital data. A / D converter 23
The digital output data of is supplied to the controller 24. The controller 24 reads the value of this digital data. The output data of the controller 24 is supplied to the encoder / decoder 25 and encoded and compressed.

Depending on the output voltage (zoom position) of the zoom position detector, whether to output the moving image data and the audio data in synchronization is determined. A flow chart for this is shown in FIG. In FIG. 3, an initial value Th is set (step 31). The initial value Th is a threshold level for determining whether to output the received moving image data and audio data in synchronization with each other, and is set to a level (for example, a bust shot) in which the movement of the lips of a person is not a concern. To be done.

In step 32, it is detected whether or not the zoom function of the video camera has been operated. When the zoom function is operated, the output voltage value of the potentiometer 21 changes, and this value is supplied to the A / D converter 23. This value is A / D converted by the A / D converter 23 (step 33). The digital output data of the A / D converter 23 is read by the controller 24 (step 34).

In step 35, it is judged whether or not this data is larger than the initial value Th. If this data is larger than the initial value Th, it indicates that the zoom position is on the telephoto side. At this time, in order to make the flow of conversation natural, it is necessary to output moving image data and audio data in synchronization. Therefore, moving image data (movement of human lips)
And a control signal for synchronously outputting the voice data (human voice, etc.) are supplied to the encoder / decoder 25. On the other hand, the data read by the controller 24 is the initial value T
When it is smaller than h, the zoom position is on the wide angle side. In this case, it is not necessary to output the moving image data and the audio data in synchronization. Therefore, a control signal indicating that it is not necessary to synchronize the output timing of the moving image data and the output timing of the audio data is supplied to the encoder / decoder 25. The encoder / decoder 25 controls the voice delay time T based on this control signal.

In the above embodiment, the present invention is applied to the video conference system, but the present invention is not limited to the video conference system, and the video telephone system and the satellite relay system. The same effect can be obtained for (moving image data and voice data are on different lines).

[0023]

According to the present invention, the zoom position of the video camera is detected, and when the zoom position is on the telephoto side, the image data and the audio data are output in synchronization. On the other hand, when the zoom position is on the wide angle side, the audio data is output earlier than the image data. As described above, when a subject (for example, a person) is zoomed in and imaged, by synchronizing and outputting the sound related to the person, the movement of the person's lips and the sound are output in coincidence. It Therefore, no unnaturalness is felt on the receiving side. Further, when the person is imaged on the wide-angle side, the conversation response can be improved by outputting the movement of the person's lips and the voice thereof without synchronizing them. Therefore, the conversation can be made to flow naturally.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a video conference system.

FIG. 2 is a block diagram showing details of a zoom position detector.

FIG. 3 is a flowchart for explaining the operation of the zoom position detector.

[Explanation of symbols]

 3, 11 Microphone 4, 12 Video camera 5, 13 Zoom position detector 21 Potentiometer 24 Controller

Claims (2)

[Claims] 1. A video camera for picking up an image of a subject, a microphone for collecting sound, and a zoom position detecting means for detecting a zoom position of the video camera. According to an output signal of the zoom position detecting means, A moving image / voice control device, wherein a delay amount of audio data with respect to image data is variable. 2. When the zoom position is set to the telephoto side, the image data is output in synchronization with the audio data, and when the zoom position is set to the wide angle side, the image data is the audio data. 2. The moving picture voice control device according to claim 1, wherein the moving picture voice control device outputs the delayed video sound.