JP4595653B2 - Video / audio processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video/audio processing apparatus capable of considerably suppressing a "deviation" resulting from a delay of a displayed video image from an output sound by employing the effects of sound delay processing and sound processing. <P>SOLUTION: An A/V decoder 17 separates a video signal from a compressed audio signal from a received multiplexed signal comprising the video signal and the compressed audio signal, temporarily stores the compressed audio signal to a memory 18 and thereafter applies expansion processing to the compressed audio signal and provides an output. A high image quality processing circuit 19 applies high image quality processing to the separated video signal. A read timing of the memory 18 is delayed from a write timing by a time corresponding to a time deviation &Delta;t caused in the video signal by the high image quality processing. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a video / audio processing apparatus, and in particular, from a television signal in which a video signal and an audio signal are synchronously multiplexed, the video signal and the audio signal are separately processed and synchronously output. The present invention relates to a processing apparatus.

  In recent years, “large screen” and “higher image quality” have been advanced in video equipment, and technical development corresponding to this has been actively conducted. Infrastructure is also being prepared. However, contrary to these desirable advances, the video signal and the audio signal are separately processed from the television signal in which the video signal and the audio signal are multiplexed synchronously like a television broadcast receiver. If output in this way, a video (image) to be displayed is delayed from the output sound due to the following causes, causing a problem on the viewer side. A similar problem occurs in a television receiver with audio processing in which audio and video are not synchronized.

  As the cause of the above, first, in a television receiver with an audio processing equipped with a high image quality processing device, when a video signal and an audio signal are separated from an input television signal and processed separately. In addition, by performing digital signal processing for high image quality processing on video signals, the phenomenon that “the output signal is delayed with respect to the input signal” occurs for the calculation, whereas for the audio signal, Even when demodulated, signal delay is not a problem.

  For this reason, for example, when the original image of the video signal in the input television signal changes in a predetermined time unit as shown in FIG. 8A, the original image is shown in FIG. As shown in the figure, an image delayed by the calculation time required for the image quality enhancement processing is output and displayed with a delay from the original sound shown in FIG.

  The second cause is that it takes time to display the video signal input to the display device itself (liquid crystal panel display or plasma display) as well as the image quality enhancement processing. That is, among the original video signal (original video) output from the A / V decoder at the timing shown in FIG. 9A and the original audio signal shown in FIG. While the sound is produced on the display device without time delay, the original image is displayed with a time delay of Δt in the display device as shown in FIG.

  The third cause is that in broadcast stations (especially in Europe), there are some television signals themselves that are not synchronized in video and audio. Also in this case, the displayed image and the sound to be sounded are shifted.

  The “deviation” in which the display video (image) is delayed from the output sound caused by the above causes causes “discomfort” and “viewing fatigue” due to the following two items.

  a: Humans do not feel a sense of incongruity even if sound is perceived late with respect to simultaneously occurring images. This is because the speed of sound is very slow compared to that of light, and it is a phenomenon that is always in contact in our lives. Specifically, when the event of 3.4 m ahead is seen, the sound is perceived with a delay of about 10 milliseconds with respect to the video (the speed of sound is 340 m / sec). On the other hand, it is very uncomfortable to perceive the video late with respect to the sound due to the above-mentioned “deviation”.

  b: Magard effect. This is known as a phenomenon in which speech recognition is based not only on sound but also on vision (integrated). One of the Magerk effects is the “time difference between video and audio”. This means that when viewing video and audio with a time difference, the brain tries to integrate and recognize both with a small time difference, but with a certain time difference, the two cannot be integrated and feel uncomfortable. When the time difference is further widened, the brain stops moving to integrate and recognizes video and audio individually. The “certain time difference” is said to feel the most uncomfortable in 50 to 100 milliseconds.

  In view of this, a video / audio processing apparatus that removes “deviation” in which a display video (image) is delayed from the output audio is known (for example, see Patent Document 1). In the conventional video / audio processing apparatus described in Patent Document 1, the decoding time of the MPEG audio decoder for decoding the MPEG audio stream separated from the MPEG (Moving Picture Experts Group) stream, the SCR (System Clock Reference), and the audio time Based on the stamp, the playback timing of the audio output is calculated, the audio output is generated according to the playback timing, and the decoding processing time of the MPEG video decoder for decoding the MPEG video stream separated from the MPEG stream, and the SCR and video time stamp A configuration is disclosed in which the output timing of the video output is calculated based on the above and the video output is generated according to the playback timing to synchronize the outputs.

JP-A-8-212701

  However, in the above conventional video / audio processing apparatus, the output is performed so as to eliminate the difference between the playback timing of the video output and the playback timing of the audio output. However, the monitor device to which those outputs are input receives the video signal ( In the configuration in which the digital signal processing for high image quality processing is performed on the video signal), a “deviation” in which the display video is delayed from the output sound is generated, and “discomfort” and “viewing fatigue” may occur.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and a video / audio processing apparatus capable of significantly suppressing “displacement” in which a display video is delayed with respect to output audio by adding effects of audio delay processing and audio processing. The purpose is to provide.

In order to achieve the above object, the present invention provides a separation means for receiving a multiplexed signal obtained by multiplexing a video signal and a compressed audio signal, and separating the video signal and the compressed audio signal from the multiplexed signal. And a high-quality image processing means for performing a predetermined high-quality image processing on the separated video signal and outputting the image signal, wherein the compressed audio separated from the multiplexed signal Storage means for temporarily storing a signal, expansion processing means for performing an expansion process on the compressed audio signal read from the storage means and outputting the expanded sound field for the expanded audio signal When the speech processing means that outputs the effect or speech speed conversion process and the speech processing means is on, the compressed speech signal stored in the storage means is output without delay, and when the speech processing means is off, For voice processing means Instead of the audio processing, at least the time corresponding to the time delay of the video signal generated by performing the image quality enhancement processing on the video signal, the read timing of the compressed audio signal stored in the storage means is delayed from the time of writing. And reading means for reading out the data.

  In the present invention, when the sound processing setting is on, the sound field expansion effect or the speech speed conversion process is performed, so that the sound is output earlier than the time delay generated in the video signal by performing the image quality enhancement process on the video signal. However, the uncomfortable feeling on viewing can be substantially reduced by the sound field effect or the speech speed conversion process. When the audio processing means is off, the time corresponding to the time delay of the video signal generated by applying the image quality enhancement processing to the video signal and the read timing of the compressed audio signal stored in the storage means are written. Since the readout is delayed, the video signal subjected to the high image quality processing and the audio signal subjected to the decompression processing can be output in synchronization.

  According to the present invention, when the audio processing setting is on, the sound is output earlier than the time delay generated in the video signal by performing the image quality enhancement process by performing the sound field expansion effect or the speech speed conversion process. However, the uncomfortable feeling on viewing can be reduced by the sound field effect or the speech speed conversion process.

  Further, according to the present invention, the read timing of the compressed audio signal stored in the storage means is delayed from the time of writing for a time corresponding to the time delay of the video signal generated by performing the image quality improvement processing on the video signal. Since the video signal that has been subjected to high image quality processing and the audio signal that has been subjected to decompression processing are output in synchronization with each other, the time difference between the video and audio that have undergone high image quality processing and the time difference (time difference) ) Can be substantially eliminated, so that there is no sense of incongruity and viewing without fatigue.

  Next, the best mode for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of an embodiment of a video / audio processing apparatus according to the present invention, and FIG. 2 is a block diagram of an embodiment of a main part in FIG. In FIG. 1, a video / audio processing apparatus 10 includes a microcomputer 15 (hereinafter referred to as a microcomputer) connected to a user input terminal 11, a demodulation circuit 16 connected to a television broadcast signal input terminal 12, and an external for compressed signal. An A / V decoder 17 connected to the input terminal 13 and the output terminal of the demodulation circuit 16; an image quality enhancement processing circuit 19 connected to the video signal external input terminal 14a and the video signal output terminal of the A / V decoder 17; The audio signal processing circuit 20 is connected to the audio signal external input terminal 14 b and the audio signal output terminal of the A / V decoder 17.

  The A / V decoder 17 has a memory 18, and the audio processing circuit 20 also has a memory 21. The microcomputer 15 inputs desired operation control information through the input terminal 11 by a user using a remote controller (remote controller) not shown, a key of the apparatus main body, and the like, and based on that, an A / V decoder 17, an image quality improvement processing circuit 19, and Each operation of the audio processing circuit 20 is controlled independently of each other. The demodulating circuit 16 receives the received television broadcast signal via the television broadcast signal input terminal 12, demodulates the carrier wave and the modulated wave for various digital signal transmissions, and outputs the video signal of the desired broadcast station. Outputs a multiplexed signal of compressed audio signals.

  The A / V decoder 17 receives the multiplexed signal of the video signal and the compressed audio signal output from the demodulation circuit 16 or the compressed signal input via the external input terminal 13 as an input signal, and selects one of them. A switching device for separating the video signal and the compressed audio signal from the selected signal, supplying the video signal to the image quality enhancement processing circuit 19, and the compressed audio signal stored in the memory 18 under the control of the microcomputer 15; The data is temporarily stored in and then read.

  The high image quality processing circuit 19 performs various known processes (such as ghost removal, contour correction, sharpness improvement, and color improvement) on the input video signal by digital signal arithmetic processing, The subsequent video signal is output to the image display circuit.

  The audio processing circuit 20 has a switching device that selects one of the audio signal output from the memory 18 in the A / V decoder 17 and the external audio signal input via the external input terminal 14b. The selected audio signal is subjected to audio signal processing by known digital signal processing using the memory 21 under the control of the microcomputer 15, and the obtained audio signal is output to the audio output circuit.

  Next, the configuration of the A / V decoder 17 and the audio processing circuit 20 will be described in more detail with reference to the block diagram of FIG. In the figure, the same components as those in FIG. In FIG. 2, the A / V decoder 17 separates the video signal and the compressed audio signal from the input signal by digital processing, and outputs the video signal to the image quality enhancement processing circuit 19 shown in FIG. After being supplied to the memory 18 and delayed for output as necessary, it is expanded by the expansion circuit 172 to be converted into a digital audio signal.

  Here, the above-mentioned memory 18 is a memory for compressed audio, which is a compressed audio signal (AAC (Advanced Audio Coding), Dolby DIGITAL, dts, etc.) originally compressed and transmitted from a broadcasting station or the like in order to reduce the amount of data. In this embodiment, as will be described later, the delay time of the video signal by the image quality enhancement processing, and the delay time of the output timing of the video signal by driving a display device such as a liquid crystal display panel are described in this embodiment. This is used for delay including the total delay time Δt. The total delay time Δt can be delayed using the memory 21.

  The D / A converter 173 in the A / V decoder 17 converts the digital audio signal output from the expansion circuit 172 into an analog audio signal and outputs the analog audio signal to the switch circuit (SW) 23. The decompression circuit 172 also outputs the digital audio signal directly to the switch circuit (SW) 24. The SW 23 selects one of the analog audio signal output from the A / V decoder 17 and the external input analog audio signal. The SW 24 selects one of the digital audio signal output from the A / V decoder 17 and the external input digital audio signal.

  The audio processing circuit 20 includes an A / D converter 201 that converts an analog audio signal selected by the SW 23 into a digital audio signal, a digital signal processor (DSP) 202 that performs digital signal processing, and digital signal processing from the DSP 202. It comprises a D / A converter 203 that converts the output digital audio signal into an analog audio signal, and a memory 21 that is used when the DSP 202 performs digital signal processing.

  The DSP 202 and the memory 21 give predetermined processing (for example, sound field effect, speech speed conversion) such as sound field generation and correction described later to the sound signal under the control of the microcomputer 15. The memory 21 can also be used as an audio delay circuit for synchronization purposes. The memories 18 and 21 have a capacity ranging from a delay amount corresponding to at least one field to a delay amount that delays the audio to the maximum corresponding to the video delay.

  Next, the operation of the first embodiment of the present invention having the configuration shown in FIGS. 1 and 2 will be described with reference to the flowchart of FIG. FIG. 3 is a flowchart for explaining the operation of the video / audio processing apparatus according to the first embodiment of the present invention. First, the microcomputer 15 in FIG. 1 determines whether or not the voice processing is turned on by the user (step S1 in FIG. 3). When the audio processing is on, the audio processing is set regardless of whether or not the device 10 performs the high image quality processing on the video signal (step S2 in FIG. 3).

  In the voice processing setting in step S2, sound field effect processing or speech speed conversion processing is performed in accordance with the setting made for the microcomputer 15 in advance. This sound field effect processing is called “hole simulation” or the like performed by controlling the operation of the DSP 202 by the microcomputer 15. Generally, as shown in FIG. If the original video signal is delayed by a delay time Δt consisting of the time and the output timing delay of the video signal by driving the display device, as shown in FIG. 4B, the audio is delayed by the delay time Δt of the video signal as shown in FIG. This is a well-known surround process that delays the output timing and outputs sound, and further adds room reflections and reverberation.

  However, in this embodiment, in the sound field effect processing by the sound processing setting in step S2, as shown in FIG. 4C, the sound expansion by the surround processing that adds the initial reflected sound and the rear reverberation sound directly to the original sound. Perform effect processing. As a result, in this embodiment, the original sound is pronounced by Δt earlier than the display timing of the video signal when the image quality is improved, but the initial reflected sound and the rear reverberation are delayed with respect to the original sound. Since the sound is generated, the sound is drawn backward (in the time delay direction), and it is possible to avoid feeling uncomfortable due to the difference of Δt in terms of hearing. The surround processing itself is well known, and the processing operation itself is not directly related to the present invention, so that detailed description thereof is omitted.

  It is also possible to perform speech speed conversion processing by setting the voice processing in step S2. This speech speed conversion process is a process in which the microcomputer 15 controls the operation of the DSP 202 to reproduce the audio slower than the actual speed. Generally, as shown in FIG. When the audio is delayed from the original video signal by a delay time Δt consisting of a delay time due to the high image quality processing and a delay time of the video signal output timing due to the display device driving, as shown in FIG. Then, the audio output timing is shifted by the delay time Δt of the video signal, and the audio is output, and the speed or pitch (speaking speed) of the audio is reproduced more slowly than the actual speed.

  However, in the present embodiment, in the speech speed conversion process by the voice processing setting in step S2, as shown in FIG. 5C, the speech speed conversion process is performed directly on the original sound. As a result, in this embodiment, the original sound starts to be sounded by Δt earlier than the display timing of the video signal when the image quality is improved, but immediately becomes a sound that is delayed from the video. It is possible to feel less uncomfortable than in the case of FIG. The speech speed conversion process itself is well known, and the processing operation itself is not directly related to the present invention, so that detailed description thereof is omitted.

  By performing the sound field effect process or the speech speed conversion process according to the setting of the audio process in step S2, the fundamental (theoretical) necessary capacity as the capacity of the memory 21 is secured. The capacity of the memory 21 can be reduced as compared with the case of 5 (B). Specifically, it is possible to reduce a memory capacity of about 94 kbits for delay / audio 44.1 kHz sampling of 67 msec for 4 video fields, 16 bits of quantization bits, and stereo.

  On the other hand, if the microcomputer 15 determines in step S1 in FIG. 3 that the audio processing is turned off by the user, the microcomputer 15 determines whether the high image quality processing is turned on (step S3 in FIG. 3). When the high image quality processing is off, there is no time delay of the video signal due to the high image quality processing, so there is almost no gap between the audio and the video, so no processing is performed on the audio signal. If it is on, a delay time Δt is set (step S4 in FIG. 3).

  As described above, the delay time Δt is a total delay time of the delay time of the video signal due to the high image quality processing and the delay time of the image signal output timing due to the driving of a display device such as a liquid crystal display panel.

  The delay time of the video signal due to the above-described image quality improvement processing is such that the image quality improvement processing circuit 19 in FIG. 1 improves the sharpness, color, contrast, etc. with respect to the digitized video signal. This is a time output from the image quality enhancement processing circuit 19 with a delay from the input of the video signal. This is the time required for the calculation for high image quality processing. One field is obtained when the computation time is short, and another field is 6 fields when the computation time is long (1 field≈ 1/60 seconds, NTSC). in the case of).

  In addition, the delay time of the image signal output timing by driving the display device is a delay time of 1 to 2 fields that may be delayed for driving a display device such as a liquid crystal panel. Due to these delay times, even if there is no time lag between the video signal and the audio signal output from the A / V decoder 17, when the signal is emitted from the display device, 1 to 8 fields ( 17-133 milliseconds (in NTSC)) "shifts" occur (in many cases 3, 4 fields (50-67 milliseconds)).

  Therefore, in this embodiment, in step S3 of FIG. 3, the delay time Δt is measured in advance, and the microcomputer 15 reads the read timing of the memory 18 in the A / V decoder 17 with respect to the write timing. A setting process for delaying by the delay time Δt is performed. As a result, the video display timing and the sound generation start timing coincide with each other, and it is possible to prevent a sense of incongruity and viewing fatigue due to “deviation” in which the display video is delayed from the output sound.

  As described above, in this embodiment, a delay circuit for correcting “deviation” in which the display video is delayed from the output sound is not simply added, but is added by adding the DSP 202 or the like having a simple calculation capability. Can increase value.

  Next, a second embodiment of the present invention will be described. 6 and 7 show flowcharts for explaining the operation of the second embodiment of the present invention. The block configuration of the present embodiment is the same as that shown in FIGS. 1 and 2, but this embodiment is characterized in that audio delay setting (storing) is performed in synchronization with the receiving broadcast station (channel).

  In other countries, especially in Europe, there are many broadcast stations that transmit television broadcast signals in which video and audio are not synchronized. Further, the degree of deviation between video and audio (deviation time td) is also a gusset at the broadcasting station. Therefore, in the present embodiment, in order to solve this problem, the user arbitrarily sets the deviation time td by himself, and once set, the audio delay time related to the deviation time td is recorded for each broadcasting station. For example, the means is provided by the microcomputer 15 of FIG.

  To describe the operation of the present embodiment, first, the microcomputer 15 shown in FIG. 1 determines whether or not a delay setting ON instruction is input by the user (step S11 in FIG. 6). If there is no delay setting ON instruction input, the microcomputer 15 terminates the process. If the microcomputer 15 determines that there is a delay setting instruction input, the microcomputer 15 sets a time difference td between the video and audio input from the user ( Step S12 in FIG. The setting of the deviation time td is performed until the broadcast station desired by the user is completed (step S13 in FIG. 6).

  That is, in steps S12 and S13 described above, the user can receive the broadcast signal to the monitor device such as a television receiver for each broadcasting station, while minimizing the time difference between the video and audio being viewed. As described above, the above-described deviation time td is arbitrarily set for all broadcast stations desired by the user. Thereby, the set deviation time td is stored in the memory of the microcomputer 15 (step S14 in FIG. 6).

  Thereafter, when the microcomputer 15 determines that the user has received a television broadcast and has made a channel switching input (step S21 in FIG. 7), the television broadcasting signal of the channel switched and input by the microcomputer 15 is selected and received. At the same time, the microcomputer 15 reads the shift time td of the corresponding channel stored in the memory in step S14 (step S22 in FIG. 7), and then determines whether or not the image quality processing is set to ON by the user (FIG. 7). Step S23).

  When the high image quality processing is off, the time delay of the video signal due to the high image quality processing by the image quality enhancement processing circuit 19 does not occur with respect to the received video signal, so the microcomputer 15 uses the above reception channel as the audio delay time. A deviation time td is set (step S24 in FIG. 7). On the other hand, when the high image quality processing is ON, a value obtained by adding the delay time Δt of the video signal due to the high image quality processing by the image quality improving processing circuit 19 to the above-described shift time td of the reception channel is set as the audio delay time ( Step S25 in FIG. Then, the microcomputer 15 stores the audio delay time obtained in step 24 or S25 in the internal memory (step S26 in FIG. 7). Note that the delay time Δt may include a delay time due to driving of the display device although the delay time is small.

  Thereafter, when the compressed audio signal is read from the memory 18 in the A / V decoder 17 under the control of the microcomputer 15 or when the audio signal is read from the memory 21 in the audio processing circuit 20, the microcomputer 15 in step S 26 described above. The audio delay time stored in the internal memory is delayed from the output timing of the video signal.

  Thereby, according to this embodiment, when a television broadcast signal in which video and audio are not synchronized is received, the received video and received audio can be synchronized and output from the monitor device. Even when high-quality image processing is being performed, the received video and the received audio can be output from the monitor device in synchronization.

  The present invention is not limited to the above embodiment, and a computer program that causes a computer to perform the processing according to the flowcharts of FIGS. 3, 6, and 7 is also included in the present invention. In addition, the delay time Δt in FIG. 3 may not be included because the delay time due to driving of the display device is small.

1 is a block diagram of an embodiment of a video / audio processing apparatus of the present invention. FIG. FIG. 2 is a block diagram of an embodiment of an AV decoder and an audio processing circuit in FIG. 1. It is a flowchart for operation | movement of the 1st Embodiment of this invention. It is a timing chart for description of the sound field effect set in the setting step of the audio processing in FIG. FIG. 4 is a timing chart for explaining speech speed conversion set in a voice processing setting step in FIG. 3. FIG. It is the flowchart for operation | movement of the 2nd Embodiment of this invention (the 1). It is the flowchart (2) for operation | movement of the 2nd Embodiment of this invention. It is a figure explaining an example of the gap of the picture and the sound by high picture quality processing. It is a timing chart explaining an example of gap of an image and sound by high picture quality processing.

Explanation of symbols

10 Video / Audio Processing Equipment 15 Microcomputer
DESCRIPTION OF SYMBOLS 16 Demodulation circuit 17 A / V decoder 18, 21 Memory 19 High quality image processing circuit 20 Audio processing circuit 171 AV separation circuit 172 Decompression circuit 173, 203 D / A converter 201 A / D converter 202 Digital signal processor (DSP)

Claims (1)

A multiplexed signal in which a video signal and a compressed audio signal are multiplexed is input, separation means for separating the video signal and the compressed audio signal from the multiplexed signal, and the separated video signal A video / audio processing device comprising high-quality image processing means for performing and outputting a predetermined high-quality image processing,
Storage means for temporarily storing the compressed audio signal separated from the multiplexed signal;
Decompression processing means for performing decompression processing and outputting the compressed audio signal read from the storage means;
A voice processing means for performing a sound field expansion effect or a speech speed conversion process on the decompressed voice signal;
When the sound processing means is on, the compressed sound signal stored in the storage means is output without delay, and when the sound processing means is off, the sound processing means is replaced with at least sound processing by the sound processing means. Reads out the compressed audio signal read timing stored in the storage unit with a delay corresponding to the time delay corresponding to the time delay of the video signal generated by performing the image quality improvement processing on the video signal. And a video / audio processing apparatus.

Images