JP2010050755A - Video audio output device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a reproducing technique accompanied with presence in a sound including BGM. <P>SOLUTION: A video audio output device includes a video display unit, a detecting means for detecting a face of a person and motion of a lip within an image displayed on the video display unit, an audio processing unit for controlling characteristics of a sound filter based on an address coordinate indicating a position where the face of the person and the motion of the lip exist within the image detected by the detecting means, a determining means for determining whether an audio signal accompanied with the image displayed on the video display unit is a monaural sound signal, and a plurality of speakers whose inputs are made from outputs of the sound filter to be controlled by the audio processing unit based on results determined by the decision means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video / audio output device, and more particularly, to a reproduction method with a sense of realism of audio.

  As a reproduction method with a sense of reality of monaural sound, recent patent document 1 describes that the frequency characteristics of the output sound signal of each channel are controlled according to the analysis result of the frequency spectrum. The content is that the sound is separated during the unit period or the frequency characteristic is changed according to the tempo. However, there is no audio source model and performance is limited.

  On the other hand, digital cameras and the like have been put to practical use for correcting the brightness of photographs using a detection technique for detecting a person's face and mouth. The video / audio output apparatus is also required to detect and use the position of the speaker using such a technique.

Relatedly, Patent Document 2 describes a video / audio output device that detects the position of a speaker in a video and controls the volume. However, a sound signal without BGM can increase the sense of presence of sound, but there is a problem that when this process is performed on a sound signal including BGM, a sense of incongruity remains.
JP 2006-86558 A JP-A-11-313272 (Claim 1)

  An object of this invention is to provide the reproduction | regeneration technique with the realistic presence of the audio | voice containing BGM.

  In order to solve the above problems, a video / audio output device of the present invention includes a video display unit, a detection unit that detects movement of a person's face and lips in a video displayed on the video display unit, and the detection unit. An audio processing unit that controls the characteristics of an audio filter based on address coordinates indicating the location of movement of a person's face or lips in the video detected by the video, and an audio signal accompanying the video displayed on the video display unit Determining means for determining whether or not the signal is a monaural audio signal, and a plurality of speakers receiving as inputs the output of the audio filter controlled by the audio processing unit based on the determination result by the determining means It is characterized by.

  According to the present invention, it is possible to obtain a reproduction technique with a realistic sensation of sound including BGM.

Embodiments of the present invention will be described below.
(Embodiment 1)
A first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a diagram schematically showing a signal processing system of a television broadcast receiving apparatus described in the embodiment of the present invention. Various circuit blocks constituting the signal processing system are arranged inside the cabinet 12.

  The digital television broadcast signal received by the digital television broadcast receiving antenna 22 is supplied to the tuner unit 24 via the input terminal 23. The tuner unit 24 selects and demodulates a signal of a desired channel from the input digital television broadcast signal. Then, the signal output from the tuner unit 24 is supplied to the decoder unit 25 and, for example, subjected to MPEG (moving picture experts group) 2 decoding processing, and then supplied to the selector 26.

  Further, the analog television broadcast signal received by the analog television broadcast receiving antenna 27 is supplied to the tuner unit 29 via the input terminal 28. The tuner unit 29 selects and demodulates a signal of a desired channel from the input analog television broadcast signal. The signal output from the tuner unit 29 is digitized by an A / D (analog / digital) conversion unit 30 and then output to the selector 26.

  The analog video and audio signals supplied to the analog signal input terminal 31 are supplied to the A / D converter 32 and digitized, and then output to the selector 26. Further, the digital video and audio signals supplied to the digital signal input terminal 33 are supplied to the selector 26 as they are.

  The selector 26 selects one of the four types of input digital video and audio signals and supplies it to the signal processing unit 34. The signal processing unit 34 performs predetermined signal processing on the input digital video signal and provides it for video display on the video display 14. As the video display unit 14, for example, a flat panel display such as a liquid crystal display or a plasma display is adopted. The signal processing unit 34 performs predetermined signal processing on the input digital audio signal, converts the signal into an analog signal, and outputs the analog signal to the speaker 15 for audio reproduction.

  Here, in the television broadcast receiving apparatus 11, various operations including the various receiving operations described above are comprehensively controlled by the control unit 35. The control unit 35 is a microprocessor with a built-in CPU (central processing unit) and the like, and is transmitted from the operation unit 16 and the operation element 21 (not shown in FIG. 2) or from the remote controller 17. By receiving the operated information via the light receiving unit 18, each unit is controlled so that the operation content is reflected.

  Here, the control unit 35 uses the memory unit 36. The memory unit 36 mainly includes a ROM (read only memory) storing a control program executed by the CPU, a RAM (random access memory) for providing a work area to the CPU, various setting information and control. And a nonvolatile memory in which information and the like are stored.

  Here, the control unit 35 is connected to an HDD (Hard Disk Drive) unit 20 housed in the stand 13 as an example. In this case, a line 37 for supplying power and control signals from the control unit 35 to the HDD unit 20 connects the control unit 26 and the HDD unit 20 via the connection unit 38.

  A line 39 for transferring digital video and audio signals between the control unit 35 and the HDD unit 20 connects the control unit 35 and the HDD unit 20 via the connection unit 40. That is, transmission of digital video and audio signals between the control unit 35 and the HDD unit 20 is performed by the power supply and the control signal.

The television broadcast receiver can record the digital video and audio signals selected by the selector 26 with the HDD unit 20 and reproduce the digital video and audio signals recorded on the HDD unit 20. And can be used for viewing.

FIG. 2 is a schematic block diagram of the main part of this embodiment.
The video signal input to the video processing block 101 detects the movement of the lips by a face sensing technique, and outputs address coordinates 102 indicating where the movement of the lips is detected on the screen. Also, the audio signal input to the audio processing block 103 changes the filter characteristics in accordance with the value of the address coordinate 102, and the audio is output from the speaker 104 consisting of four speaker groups, thereby enhancing the sense of presence even in monaural audio. it can. The video processing block 101, the address coordinate 102, and the audio processing block 103 are in the signal processing unit 34, and the speaker 104 corresponds to the speaker 15.

  FIG. 3 is a block diagram showing an example of audio processing according to this embodiment. In this block diagram, if the input audio signal is monaural, audio is output from a speaker close to the position of the face (lips) displayed on the screen. If the input audio signal is stereo, the input audio signal is passed through and returned to the normal viewing state.

The audio signal processing of FIG. 3 will be described below.
The LR audio signal comparison block 202 compares the 2-channel input audio signal 201 with each channel's audio signal, and outputs a monaural or stereo determination result 203. The BPF 204 has a characteristic of passing the band of the human voice, and the frequency of the signal that has passed through the BPF 204 is measured by the frequency comparison 205 and compared with the voice of the human.

The Q value of the notch filter is set according to the value of the coordinate information 206 (102 in FIG. 2) indicating the position of the lips input to the 4ch notch filter 207, and the center frequency of the notch filter is the frequency value measured by the frequency comparison 205 Set (f0).

The selector 208 switches between the audio signal input from the audio signal 201 and the audio signal that has passed through the 4ch notch filter 207 by the monaural determination signal 203, amplified by the AMP 209, and then output to the speaker 210 (104 in FIG. 2).

  4 is a conceptual diagram of a state in which a total of four speakers are installed on the left and right, and the viewer is watching a TV NEWS program. A NEWS program may be broadcast in a bilingual format. When viewing in Japanese, a TV with a stereo (two-channel) function will be heard in monaural audio, so the audio signal when viewing this program in Japanese This path passes through the 4ch notch filter block of FIG. Since the NEWS program video is spoken by the right person (B), the position of the lips is located at the upper right of the entire screen. Since the filter processing is performed so that the Q value of the notch filter increases when the distance between the lip position and each speaker is long, the frequency characteristic output to each speaker becomes the characteristic shown in FIG. The voice of the speaker becomes smaller, so the speaker's voice can be heard from the position of the speaker's mouth.

  FIG. 5 is a characteristic diagram showing an output characteristic curve to each speaker used in the embodiment. Since SP-B is closest to the position of the lips, there is no attenuation, and the attenuation increases as the distance from SP-D, SP-A, and SP-C increases.

  FIG. 6 is a voice waveform diagram for explaining the embodiment. FIG. 6A shows the transition of the amplitude of the speech waveform and the labeling result (sohanasu) when the person utters “speak so”, and FIG. 6B shows the steady part of the vowel a in FIG. It is the waveform which expanded. In FIG. 6B, p is the fundamental pitch of vocal cord vibration, and the reciprocal thereof is the fundamental frequency F0. F0 is distributed between several tens of Hz to several hundreds of Hz, and varies depending on emotions and the like.

  FIG. 7 is an example of a speech waveform as a representation of a speech signal. FIG. 7A shows a part of the temporal change in the amplitude of the audio signal when “sa” is uttered. It consists of a frictional consonant part / s /, followed by a transient part, and a stationary base part / a /. FIG. 7B shows a frequency spectrum when Fourier transformation is performed on the stationary matrix / a / in units of 10 ms. The vertical axis represents intensity, and the horizontal axis represents frequency (unit: kHz).

  This figure shows a male voice sample. In (b), a harmonic line spectrum w indicated by a fine waveform and a spectrum envelope (envelope) e obtained by linear prediction analysis or the like are shown. Three so-called formants (resonance of the vocal tract) are recognized from the spectrum envelope e, and the third formant F3 is located in the vicinity of 2.5 kHz. It should be noted that the position of the third formant, etc. changes for other vowels / i /, / u /, etc., but the intensity around 2.5 kHz is high. The more dominant first formant F1 and second formant F2 are located around 1 kHz.

The output characteristics to each speaker in FIG. 5 may be changed following the formant in consideration of the change in the fundamental frequency F0.
(Embodiment 2)
A second embodiment according to the present invention will be described. Description of the parts common to the first embodiment is omitted.
In the first embodiment, an example in which there is one speaker has been described, but there may be a plurality of speakers. In FIG. 4, it is assumed that the left person (A) is also moving the lips. A relatively high frequency component on the frequency spectrum is likely to include speaker information including a consonant part and a transient part. Speaker recognition is also used for each speaker A and speaker B for each speaker. What is necessary is just to superimpose and output the characteristics of the set notch filter. As a means for speaker recognition, a consonant sounded by the speaker may be identified from the movement of the lips and the result may be used.

  Audio signals such as overseas movies as well as TV NEWS programs may be broadcast in a bilingual format, and when viewed in Japanese, they will be heard in monaural audio, so they are more realistic than programs broadcast in stereo. The feeling is inferior. By performing the video / audio processing according to the contents of the present embodiment, it is possible to enhance the presence of the TV audio signal even in monaural broadcasting, the BGM localization sounds flat even in a program in which BGM is inserted, and the speaker's voice is the speaker's voice. It becomes audible from the position.

  As an overview above, the monaural audio signal is used to control the audio filter characteristics of the multiple speakers arranged according to the coordinates of the detected positions of the face and lips on the screen, and to enhance the audio presence according to the screen. To process. Furthermore, the characteristics of the audio filter control the Q of the notch filter.

  As an effect of this, by performing voice processing according to the contents of the present embodiment, the localization of the BGM sound can be output even with the voice signal including BGM, and the voice of the person can be output according to the position of the speaker on the screen. Therefore, even in the case of a sound signal including BGM, it is possible to enhance the sense of presence of sound without a sense of incongruity.

  In addition, this invention is not limited to the said embodiment, In the range which does not deviate from the summary, it can implement in various modifications. For example, the example in which the speaker groups are arranged on a plane is shown, but a so-called surround arrangement may be used.

  Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements according to different embodiments may be appropriately combined.

The figure which shows schematically the signal processing system of one Embodiment of this invention. The schematic block diagram of the principal part of the embodiment. The block diagram which shows an example of the audio | voice process of the embodiment. The image figure of the state in which the viewer of the embodiment is watching the NEWS program of TV. The characteristic view which shows the output characteristic curve to each speaker used for the embodiment. FIG. 1 is a speech waveform diagram for explaining the embodiment. FIG. 2 is a second waveform diagram for explaining the embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 12 ... Cabinet, 14 ... Image | video display (video display part), 15 ... Speaker, 16 ... Operation part, 18 ... Light-receiving part, 20 ... HDD unit (recording means), 22 ... Antenna, 23 ... Input terminal, 24 ... Tuner , 25 ... Decoder unit, 26 ... Selector, 27 ... Antenna, 28 ... Input terminal, 29 ... Tuner unit, 30 ... A / D conversion unit, 31 ... Input terminal, 32 ... A / D conversion unit, 33 ... Input terminal 34 ... Signal processing unit 35 ... Control unit 35a ... HDD control unit 36 ... Memory unit 37 ... Line 38 ... Connection unit 39 ... Line 40 ... Connection unit 101 ... Video processing block 102 ... Address Coordinates, 103 ... Audio processing block, 104 ... Speaker, 202 ... Audio signal comparison block, 204 ... BPF, 207 ... Notch filter.

Claims (4)

A video display unit;
Detecting means for detecting movement of a person's face and lips in the video displayed on the video display unit;
An audio processing unit that controls characteristics of an audio filter based on address coordinates indicating a location where movement of a person's face or lips in the video detected by the detection unit exists;
An audio / video output apparatus comprising: a determination unit that determines whether an audio signal accompanying an image displayed on the image display unit is a monaural audio signal.   The video / audio output apparatus according to claim 1, wherein the audio processing unit controls a Q value of the notch filter as a characteristic of the audio filter based on the address coordinates. A BPF configured to analyze the passband of a person's voice;
The video / audio output apparatus according to claim 2, wherein the audio processing unit controls a center frequency of the notch filter while comparing frequencies of audio signals that have passed through the BPF.   When the determination result by the determination means is determined to be a monaural audio signal, a plurality of speakers that receive the output of the sound filter controlled by the sound processing unit based on the determination result by the determination means are input to the sound filter. The video / audio output apparatus according to claim 1, wherein the output is an input.

Images