JPH11313272A - Video/audio output device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the presence by detecting the position of a talker in a video image so as to control a variable resistor thereby enabling a listener to hear a voice from the position of the talker even with a monaural voice input. SOLUTION: The position of a talker in video data is detected and the level of an input voice at the position is controlled and the resulting data are outputted to a plurality of speakers. That is, a video data input section 101 receives video data including the image of the photograph of the talker and gives the data to a video output section 105 and an area detection section 103. The area detection section 103 detects the position of the talker in the video image based on the video data and gives information denoting the position of the talker to a volume control section 104. The volume control section 104 controls a volume of the voice fed from a voice data input section 102 based on the position information of the talker in response to respective speakers 106 and provides an output of the result. Thus, by controlling the volume control section 104 based on the detected position and outputting the voice whose volume is controlled to a plurality of the speakers, the position of the voice is changed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio output device for processing monaural voice based on a video of a speaker.

[0002]

2. Description of the Related Art Japanese Patent Laid-Open No. 8-125760 discloses a method of controlling the volume of a plurality of audio outputs in accordance with the display position of an image.
There are techniques shown in

This technique divides one screen into a plurality of display areas, simultaneously displays a plurality of input image data, and controls the volume and localization of audio data attached to video data selected from the plurality. Alternatively, the volume is controlled to be proportional to the size of the display area of the plurality of input image data.

[0004]

In the case of the above-described voice output control technique, voice output control is performed only when there are a plurality of speakers and the divided screen is displayed, and there is only one speaker. When only one speaker is displayed on the screen without time division or screen division, there is a problem in that sound output control is not performed and the localization of the sound is fixed, so that a sense of incongruity is felt.

[0005]

According to the present invention, a position of a speaker in a screen is detected from image data by using an area detecting means.
Controlling the volume control unit based on the detected position,
By outputting the sound whose volume is controlled to a plurality of speakers, the sound is localized.

[0006] With the above configuration, even if the sound is monaurally input, the sound can be heard from a position corresponding to the position of the speaker in the video, so that a realistic sound can be heard.

[0007] Further, by making the sound audible from a position corresponding to the position of the face or lips of the speaker, the position of the sound is localized closer to the mouth that actually emits the sound, and there is a more realistic feeling. It becomes possible to listen to the voice.

Also, the mouth regions of a plurality of persons being photographed are detected, and the speaker that is actually speaking is specified by selecting the one with the largest movement in each region, and the mouth of the speaker is identified. By listening to the sound from the position of, the sound can be heard from the position of the person who is actually speaking among the plurality of persons, and it is possible to listen to the sound with a sense of reality.

[0009]

Embodiments of the present invention will be described below.

FIG. 1 shows an embodiment of an apparatus for detecting the position of a speaker of video data, controlling the loudness of an input voice according to the position, and outputting to a plurality of speakers.

An image data input unit 101 inputs image data of a speaker being photographed, and supplies the image data to an image output unit 105 and an area detection unit 103.

The area detecting section 103 detects the position of the speaker in the image from the supplied image data, and supplies the position information of the speaker to the volume control section 104.

The volume control unit 104 controls the volume of the voice supplied from the voice data input unit 102 according to the respective speakers based on the position information of the speaker, and outputs the voice.

When the display 203 and the speakers 201 and 202 are arranged as shown in FIG. 2, the volume control unit 104 uses the horizontal coordinates of the position information from the area detection unit 103 to obtain the information shown in FIG. As described above, the horizontal ratio a: b of the position of the speaker is obtained, and the volume of the left speaker 201 and the volume of the right speaker 202 are controlled as b: a, so that the sound localization is adjusted to the position of the speaker in the video. To output.

When the display 305 and the speakers 301, 302, 303, and 304 are arranged as shown in FIG. 3, the volume control unit 104
6, the horizontal ratio a: b and the vertical ratio c: d of the speaker position are obtained as shown in FIG. 6, and the volume of the left speaker 301 and the volume of the right speaker 303 are calculated as b. : A,
By controlling the volume of the speaker 304, which is the volume of the upper speaker 302, to be d: c, the sound is output in a sound localization that matches the position of the speaker in the video.

Also, as shown in FIG.
When the speakers 401, 402, 403, and 404 are arranged as described above, the volume control unit 104 uses the position information from the area detection unit 103 to calculate the horizontal ratio a of the speaker's position as shown in FIG. : B, the ratio c: d in the vertical direction is determined,
The sum of the volume of the upper left speaker 401 and the volume of the lower left speaker 403, the volume of the upper right speaker 402, and the lower right speaker 40
4 is b: a, and the ratio of the volume of the upper left speaker 401 to the volume of the lower left speaker 403 and the ratio of the volume of the upper left speaker 401 to the volume of the lower left speaker 403 are d:
By controlling to be c, the sound is localized and output according to the position of the speaker in the video.

Further, by using the face image region extraction technique when detecting the region of the speaker, it is possible to set the sound localization at a position close to the mouth where the speaker actually emits the sound. .

In this case, it is desirable to set a position closer to the mouth in the determined face area as the localization. Therefore, the center 702 in the horizontal direction of the determined face area 701 as shown in FIG.
And the intersection of the center 703 in the vertical direction is used as a reference for voice localization calculation.

Further, it is desirable to determine the localization of the audio by using the mouth region extraction technique in the video because the localization position of the audio can be determined more accurately.

When the above-mentioned mouth region extraction technique can detect a plurality of mouth regions in a video, there are a plurality of speakers and a left speaker mouth region 801 and a right speaker mouth region as shown in FIG. After detecting a plurality of mouth regions as in 802, motion detection within the region is performed, and the mouth region with the largest motion is defined as the mouth region of the speaker who is actually speaking, so that the sound localization comes to that region. To control the volume.

Further, the area of the detected face or mouth area is calculated, and the volume is controlled so that a large sound is output when the image occupies a large portion of the entire screen, and a small sound is output when the ratio is small. I do. At this time, control is performed so that the volume ratio of each speaker does not change so that sound localization does not change.

The above-described area detection and volume control need not be performed by one device, but may be performed by using two or more devices by transmitting and receiving by a communication device.

As shown in FIG. 9, the transmitting device 910 specifies the area of the speaker, and transmits the video, audio, and speaker area data.
It is also possible to receive the signal by the receiver 911, control the sound volume to the plurality of speakers 908 using the speaker area data, and output the sound. In this case, there is no need for the receiving apparatus to have a high calculation cost portion for obtaining the speaker area.

Further, as shown in FIG. 10, the position of the speaker is specified by the transmitting device 1009, a plurality of volume-controlled voice data is created using the position, and the plurality of voice data and video data are transmitted and received. It is also possible for the device 1010 to receive and output. In this case, it is not necessary for the receiving device to have a portion having a high calculation cost for obtaining a speaker region and a volume control device.

FIG. 12 is a flowchart showing an outline of the processing procedure of the present invention. Although the flow of the processing is sequential in this figure, portions that can be processed in parallel may be processed in parallel.

[0026]

According to the present invention, the sound can be heard from the position of the speaker despite the monaural sound input, so that the sense of reality can be improved.

According to the first aspect of the present invention, by detecting the position of the speaker and controlling the sound volume to be output to a plurality of speakers, the sound can be output in the sound localization corresponding to the position of the speaker.

According to the second aspect of the present invention, the position of the speaker is detected, and the volume output to a plurality of speakers is controlled, so that the sound is localized from the position of the speaker despite the monaural voice input. Realism can be improved.

According to the third aspect of the present invention, the face area of the speaker is detected, and the volume output to a plurality of speakers is controlled, so that the voice is output from the vicinity of the face of the speaker despite the monaural voice input. There is localization and the sense of presence can be improved.

According to the fourth aspect of the present invention, the mouth area of the speaker is detected, and the volume output to the plurality of speakers is controlled, so that the voice is output from the position of the mouth of the speaker despite the monaural voice input. Can be heard, so that the sense of reality can be further improved.

According to the fifth aspect of the present invention, by detecting a plurality of mouth regions and selecting a mouth region of a speaker who is actually speaking, a plurality of mouth regions are detected in the image. However, since the voice can be heard from the position of the mouth of the speaker who is actually speaking, the sense of reality can be further improved.

According to the sixth aspect of the present invention, by controlling the loudness of the voice according to the size of the detected area, not only localization in the up, down, left and right directions, but also a loud sound when the speaker is near, When it is reflected in the distance, the sound can be output with a small sound.

According to the seventh aspect of the present invention, when a device for inputting video and audio and a device for outputting video and audio communicate with each other, the transmitting side transmits the position information of the speaker from the transmitting side, so that the receiving side can receive the voice corresponding to the speaker area. You can determine the orientation. Further, the receiving side does not need to have an area detecting unit having a high calculation cost.

According to the eighth aspect of the present invention, when a device for inputting video and audio and a device for outputting video and audio communicate with each other, a plurality of audio signals are transmitted in a localization corresponding to the position of the speaker, so that the volume control unit can be controlled. Even if the receiving device does not have the device, it can output a realistic sound.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram showing an embodiment of the present invention.

FIG. 2 is a diagram showing a positional relationship between a display and a speaker.

FIG. 3 is a diagram showing a positional relationship between a display and a speaker.

FIG. 4 is a diagram showing a positional relationship between a display and a speaker.

FIG. 5 is a diagram showing a position of a speaker in a video.

FIG. 6 is a diagram showing a position of a speaker in a video.

FIG. 7 is a diagram illustrating a method of determining a position of a mouth when a face area is determined.

FIG. 8 is a diagram showing a state where a plurality of speakers exist and a plurality of mouth regions are extracted.

FIG. 9 is a schematic configuration diagram in which the present invention is applied to a communication device.

FIG. 10 is a schematic configuration diagram in which the present invention is applied to a communication device.

FIG. 11 is a schematic configuration diagram of a conventional technique.

FIG. 12 is a flowchart of the present invention.

[Explanation of symbols]

 Reference Signs List 101 video data input unit 102 audio data input unit 103 area detection unit 104 volume control unit 105 video output unit 106 speaker

Claims (8)

[Claims] 1. A device for detecting a position of a speaker in an image, and a volume control unit for controlling a volume of a sound to be provided to a plurality of speakers. And a video / audio output device. 2. The method according to claim 1, wherein, when controlling the volume of the voice, the localization of the voice is shifted left and right, up and down or up and down in accordance with the position of the speaker in the video using the detected position of the speaker. A video / audio output device characterized by being controlled left and right. 3. A video / audio output apparatus, wherein face area detection is used for the speaker area detection means according to claim 1. 4. A video / audio output apparatus, wherein mouth area detection is used as the speaker area detection means according to claim 1. 5. A method according to claim 4, wherein said mouth area detecting means is capable of detecting a plurality of mouth areas in an image, and controls a volume based on the largest motion area in each detected area. A video / audio output device characterized by the following. 6. When the speaker occupies a large area in a video image, by controlling the loudness of all the loudspeakers according to the size of the area detected in claim 3, 4, or 5. A video / audio output device which outputs a loud voice and a small voice when a speaker occupies a small area in the video. 7. A transmitting apparatus comprising: speaker detecting means; and a communication control unit capable of transmitting video data, audio data, and speaker position data; and video data, audio data, and speaker position data. And a receiving device having a volume control means, the receiving device receives the speaker position data detected by the transmitting device,
A video / audio output device for controlling a volume control unit. 8. A transmitting apparatus comprising: a speaker detecting unit; a volume controlling unit; a communication control unit capable of transmitting video data and a plurality of audio data; and receiving the video data and the plurality of audio data. A receiving device having a communication control unit capable of detecting a speaker position, transmitting a plurality of voices whose volume is controlled by a volume means, and transmitting a plurality of voices received by the receiving device to a plurality of speakers And a video / audio output device.