TW201828712A - Video/audio processing method and apparatus of providing stereophonic effect based on mono-channel audio data - Google Patents

Abstract

An video/audio processing method includes: de-multiplexing video/audio data to generate video data and audio data; de-multiplexing video data to generate a plurality of images; analyzing the audio data to generate audio data of a first channel and audio data of a second channel; analyzing a position of an object on an image; adjusting at least one of a first volume gain and a second volume gain according to the position to make the first volume gain be different from the second volume gain; generating audio output of the first channel according to the first volume gain; and, generating audio output of the second channel according to the second volume gain.

Description

Audio and video processing method and audio processing device for providing stereo effect according to mono audio data

The present invention relates to a video processing system, and more particularly to a video processing method and a video processing device that provide a stereo effect according to mono audio data.

In general, the existing television broadcasting system usually has a left speaker and a right speaker, which can respectively play the audio data of the left and right channels, so that the user enjoys the stereo effect. However, if the audio data of a video material is mono, the left channel speaker and the right channel speaker will play the same audio signal, and the stereo effect cannot be perceived by the user.

Accordingly, it is an object of the present invention to provide a video processing method and a video processing device that provide a stereo effect based on mono audio data to solve the above problems.

According to an embodiment of the present invention, a video processing method is disclosed. The audio and video processing method includes: multiplexing a video and audio data to generate a video data and an audio data; decoding the video data to generate a plurality of images; and decoding the audio data to generate a first channel audio data and a second channel of audio data; analyzing a position of an object in an image; adjusting at least one of a first volume gain and a second volume gain according to the position, the first volume gain and the first The second volume gain is different; the audio output of the first channel is generated according to the first volume gain value; and the audio output of the second channel is generated according to the second volume gain value.

According to an embodiment of the present invention, a video processing device is further disclosed. The audio processing device comprises a demultiplexer, a video decoder, an audio decoder, a volume gain control circuit, a first audio processing circuit and a second audio processing circuit. The multiplexer is configured to decode a video and audio data to generate a video material and an audio data. The video decoder is configured to decode the video data to generate a plurality of images. The audio decoder is configured to decode the audio data to generate audio data of a first channel and audio data of a second channel. The buffer is configured to temporarily store the plurality of images, the audio data of the first channel, and the audio data of the second channel. The volume gain control circuit is configured to analyze a position of an object in an image and adjust at least one of a first volume gain and a second volume gain according to the position, so that the first volume gain and the second volume gain different. The first audio processing circuit is configured to output the audio data of the first channel according to the first volume gain value. The second audio processing circuit is configured to output the audio data of the second channel according to the second volume gain value.

The embodiment of the present invention aims to dynamically adjust the amplitude of the audio output input to different channels according to the position information of the image objects in the video screen, so that the volume played by the speakers corresponding to different channels is different. Achieve stereo sound based on mono audio data.

FIG. 1 is a block diagram of a video processing apparatus 100 according to an embodiment of the present invention, which provides a stereo effect according to mono audio data. FIG. 2 is a flow chart of the audio-visual processing apparatus 100 illustrated in FIG. 1 according to an embodiment of the present invention. Hereinafter, a detailed embodiment of the present invention will be described with reference to FIGS. 1 and 2.

The video processing device 100 includes a demultiplexer 102, a video decoder 105, an audio decoder 110, a buffer 106, audio processing circuits 120A and 120B, and a volume gain control circuit 125. The video processing device 100 can be a system single chip and disposed in a video playback device, such as a display, a television or an on-board box. In addition, the audio processing circuits 120A and 120B of the audio processing device 100 are coupled to a left speaker 130FL and a right speaker 130FR, respectively, and the left speaker 130FL and the right speaker 130FR may be disposed inside or outside the video playback device.

The multiplexer 102 receives a video material AV and performs demultiplexing operation on the video material AV to generate a video data VD and an audio data AD (step 205), and the video data VD is input to the video decoder 105 for decoding. The audio material AD is input to the audio decoder 110 for decoding. Among them, those having ordinary knowledge in the field to which the present invention pertains should be able to understand how to perform multiplexed operation on video and audio materials, and thus further explanation is omitted here.

The video decoder 105 receives the video material VD and decodes the video material VD to generate a plurality of images I1 to In (step 210). The images I1~In are temporarily stored in the buffer 106 for output to a display panel (not shown in FIG. 1). The video decoder 105 is, for example, an MPEG-4 or H.264 decoder, but is not limited thereto. Among them, those having ordinary knowledge in the field to which the present invention pertains should be able to understand how to decode video data, and thus further description is omitted here.

The audio decoder 110 receives the audio material AD and decodes the audio material AD to generate the left channel audio data ADL and the right channel audio data ADR (step 215). The left channel audio data ADR is temporarily stored in the buffer 106 corresponding to one of the left channels for output to the left speaker 130FL; the right channel audio data ADR is temporarily stored in the buffer 106 corresponding to the right One of the channels for output to the right speaker 130FR. In an embodiment in which the audio data AD is mono, the left channel audio data ADL is the same as the right channel audio data ADR. The audio decoder 110 is, for example, a WAV or MP3 decoder, but is not limited thereto. Among them, those having ordinary knowledge in the field to which the present invention pertains should be able to understand how to decode audio data, and thus further description is omitted here.

The audio processing circuits 120A and 120B respectively receive the audio data ADL of the left channel and the audio data ADR of the right channel, and respectively perform sound processing on the audio data ADL of the left channel and the audio data ADR of the right channel to generate a left sound. The channel audio output AOL and the right channel audio output AOR (step 230), wherein the sound processing includes volume adjustment. More specifically, the audio processing circuit 120A may adjust the gain G ₁ of the left channel audio output volume AOL according to the first volume, the audio processing circuit 120B can output a second AOR adjusting the audio volume gain G ₂ based on the right channel volume. Among them, those having ordinary knowledge in the art to which the present invention pertains should understand how to adjust the volume of an audio output according to a volume gain, and thus further explanation is omitted here.

The volume gain control circuit 125 refers to/detects or analyzes the position of the image object in at least one of the decoded video images I1~In (step 220), and dynamically determines or adjusts the first and second gain values according to the position. At least one of G ₁ ~ G ₂ causes the first volume gain G ₁ to be different from the second volume gain G ₂ (step 225). In practice, the volume gain control circuit 125 loads and executes a specific software code 125P, detects the position of one of the decoded video images I1~In, and dynamically adjusts the audio processing circuits 120A and 120B, respectively. The gain values G ₁ ~G ₂ can be controlled to adjust the amplitude of the audio output of the left and right channels. The specific software code 125P can be stored in an external memory or an internal memory or a temporary memory.

For example, the volume gain control circuit 125 first detects an image object from the video image I1, and identifies the coordinate position of the image object, and then determines whether the image object is located to the left or the right of the image I1 according to the coordinate position. When the image object is located on the left side of the image I1, the volume gain control circuit 125 adjusts at least one of G ₁ and G ₂ such that the volume gain value G ₁ corresponding to the left channel is greater than the volume gain value G ₂ corresponding to the right channel. , for example, increasing the volume gain value corresponding to only the left _{channel. 1} G without adjusting the gain value G corresponding to the volume of the right channel _2, or only reduced corresponding to the volume of the right channel gain value G corresponds to ₂ without adjusting The volume gain value of the left channel is G ₁ . When the image object is located on the right side of the image I1, the volume gain control circuit 125 adjusts at least one of G ₁ and G ₂ such that the volume gain value G ₂ corresponding to the right channel is greater than the volume gain value G corresponding to the right channel. ₁ , for example, may only increase the volume gain value G ₂ corresponding to the right channel without adjusting the volume gain value G ₁ corresponding to the left channel, or only decrease the volume gain value G ₁ corresponding to the left channel without adjusting Corresponds to the volume gain value G _{2 of the} right channel.

Therefore, if the image object is a truck and moves from the right side of the screen to the left side of the screen, the volume of the audio output AOR of the right channel is greater than the volume of the audio output AOL of the left channel, and then the audio of the left channel. The volume of the output AOL is greater than the volume of the audio output AOR of the right channel, so that the speaker corresponding to the left/right channel can produce the stereo effect of the truck; when the image object moves from the left to the right of the screen, and so on. ,No longer.

It should be noted that in other embodiments, the volume gain control circuit 125 of the present invention can continuously adjust the controllable gain values G ₁ , G _{2 of the} control audio processing circuits 120A and 120B. For example, the following table shows that when the image object is moved from the right position of a screen to the left position of the screen, the gain values G ₁ and G _{2 of the} left/right channel audio signals are continuously adjusted, so that left/right The speaker of the channel plays different volume results:

As shown above, for example, the time stamp seconds, 10 seconds when the object began to appear in the screen to the right position on the screen, volume gain control circuit 125 to adjust the audio processing circuit 120A and 120B may control the gain of G _1, G ₂ , the audio volume of the left channel speaker 130FL is 0 and the audio volume of the right channel speaker 130FR is 50. In the 11th to 14th second, the picture object gradually moves to the left position of the screen, and the volume gain control circuit 125 further adjusts the controllable gain values G ₁ , G _{2 of the} audio processing circuits 120A and 120B step by step, and gradually increases the audio volume of the left channel speaker 130FL from 0 to 40 and the audio volume of the right channel speaker 130FR from 50 to 50. Reduced to 10, which produces a stereo sound effect.

It should be noted that the invention of the present invention is not limited to the left and right dual channels, but can also be applied to a multi-channel system, that is, in other embodiments, the buffer 106 can output N audio data to N audio processing respectively. The circuit, corresponding to the N channels, the N audio processing circuits adjust the amplitude of the audio output of the N channels according to the N controllable gain values G ₁ ~G _N to adjust the volume of the different channels; Integer. If the speaker system 130 includes two front speakers 130FL, 130FR (corresponding to two lower channels) and two upper speakers (corresponding to two upper channels), then N is 4. In addition, two different audio outputs can be generated to the speakers of the two upper channels. The number of audio materials and the number of corresponding speakers are determined according to the stereo effect to be produced by the audio processing device 100, and are not limiting of the present invention.

For example, if the audio processing circuit 120A corresponds to an upper channel and the audio processing circuit 120B corresponds to the lower channel, the volume gain control circuit 125 can also determine whether the image object is located on the upper side of the image I1 according to the coordinate position or On the lower side, when the image object is located on the upper side of the image I1, the volume gain control circuit 125 adjusts at least one of G ₁ and G ₂ such that the volume gain value G _{1 is} greater than the volume gain value G ₂ , for example, the volume gain value G ₁ can be increased. Or reduce the volume gain value G ₂ . When the image object is located on the lower side of the image I1, the volume gain control circuit 125 adjusts at least one of G ₁ and G ₂ such that the volume gain value G _{2 is} greater than the volume gain value G ₁ , for example, the volume gain value G ₂ or Reduce the volume gain value G ₁ .

For example, if the image object is an airplane and moves from the position below the screen to the position above the screen, the audio processing device 100 can continuously adjust the volume of the upper channel and the lower channel signal according to the position change of the aircraft object to generate the The stereo effect of the aircraft; others can be deduced by analogy and will not be repeated.

In addition, it should be noted that the present invention does not limit the moving direction of the displacement of the image object to the left or the right or the vertical movement. In other embodiments, for example, when the positional change of the image object is detected, the image is moved from the upper right to the upper side of the screen. The lower left of the screen or the upper left of the screen moves to the lower right of the screen or other irregular moving trajectory, and the above-mentioned volume gain control circuit 125 can dynamically control the corresponding gain of the audio processing circuit 120 according to the position change. Achieve the purpose of enhancing the stereo effect. The above-described screen object embodiments and the corresponding volume adjustment control mechanisms are only for explaining possible implementations, and are not limitations of the present invention. Equivalent changes and modifications made in accordance with the scope of the present invention should be within the scope of the present invention.

100‧‧‧Video processing device
102‧‧‧Demultiplexer
106‧‧‧buffer
105‧‧‧Video Decoder
110‧‧‧Optical decoder
120A, 120B‧‧‧ audio processing circuit
125‧‧‧Volume gain control circuit
125P‧‧‧ Code
130‧‧‧Speaker system
130FL, 130FR‧‧‧ speakers

FIG. 1 is a block diagram of an audio and video processing apparatus according to an embodiment of the present invention. 2 is a flow chart showing the operation of the audio processing device shown in FIG. 1 to dynamically adjust the volume gain of the left and right channels.

Claims (14)

An audio-visual processing method includes: decoding a video and audio data to generate a video data and an audio data; decoding the video data to generate a plurality of images; and decoding the audio data to generate a first channel audio data And a second channel of audio data; analyzing a position of an object in an image; adjusting at least one of a first volume gain and a second volume gain according to the position, the first volume gain and the The second volume gain is different; generating an audio output of the first channel according to the first volume gain value; and generating an audio output of the second channel according to the second volume gain value. The video processing method according to claim 1, wherein the first channel is a left channel, the second channel is a right channel, and at least the first and second volume gains are adjusted according to the position. The step of one of: determining whether the object is located on the left side of the image according to the position; adjusting the at least one of the first and second volume gains when determining that the object is located on the left side of the image, so that the first The volume gain is greater than the second volume gain. The video processing method of claim 2, wherein the step of adjusting at least one of the first and second volume gains such that the first volume gain is greater than the second volume gain comprises: increasing the first volume Gain such that the first volume gain is greater than the second volume gain. The video processing method of claim 2, wherein the step of adjusting at least one of the first and second volume gains such that the first volume gain is greater than the second volume gain comprises: lowering the second volume Gain such that the first volume gain is greater than the second volume gain. The video processing method according to claim 1, wherein the first channel is a left channel, the second channel is a right channel, and at least the first and second volume gains are adjusted according to the position. The step of one of the steps includes: determining, according to the position, whether the object is located to the right of the image; and determining that the object is located to the right of the image, adjusting at least one of the first and second volume gains to enable the second The volume gain is greater than the first volume gain. The video processing method according to claim 1, wherein the first channel is an upper channel, the second channel is a lower channel, and at least the first and second volume gains are adjusted according to the position. The step of one of: determining whether the object is located on the upper side of the image according to the position; adjusting the at least one of the first and second volume gains when determining that the object is located on the upper side of the image, so that the first The volume gain is greater than the second volume gain. The video processing method according to claim 1, wherein the first channel is an upper channel, the second channel is a lower channel, and at least the first and second volume gains are adjusted according to the position. One of the steps includes: determining, according to the position, whether the object is located on a lower side of the image; and determining that the object is located at a lower side of the image, adjusting at least one of the first and second volume gains The second volume gain is greater than the first volume gain. An audio-visual processing device, comprising: a de-multiplexer for decoding a video and audio data to generate a video data and an audio data; a video decoder for decoding the video data to generate a plurality of images; An audio decoder for decoding the audio data to generate a first channel of audio data and a second channel of audio data; a buffer for temporarily storing the plurality of images, the first channel The audio data and the audio data of the second channel; a volume gain control circuit for: analyzing a position of an object in an image; and adjusting a first volume gain and a second volume gain according to the position At least one of the first volume gain is different from the second volume gain; a first audio processing circuit outputs audio information of the first channel according to the first volume gain value; and a second audio processing circuit And outputting the audio data of the second channel according to the second volume gain value. The video processing device of claim 8, wherein the first channel is a left channel, the second channel is a right channel, and the volume gain control circuit: determining whether the object is located according to the position The left side of the image; and when it is determined that the object is located to the left of the image, adjusting at least one of the first and second volume gains such that the first volume gain is greater than the second volume gain. The audio processing device of claim 9, wherein the volume gain control circuit increases the first volume gain such that the first volume gain is greater than the second volume gain. The video processing device of claim 9, wherein the volume gain control circuit reduces the second volume gain such that the first volume gain is greater than the second volume gain. The video processing device of claim 8, wherein the first channel is a left channel, the second channel is a right channel, and the volume gain control circuit: determining whether the object is located according to the position The right side of the image; and when it is determined that the object is located to the right of the image, adjusting at least one of the first and second volume gains such that the second volume gain is greater than the first volume gain. The audio processing device of claim 8, wherein the volume gain control circuit determines whether the object is located on an upper side of the image according to the position; and when determining that the object is located on an upper side of the image, adjusting the At least one of the first and second volume gains causes the first volume gain to be greater than the second volume gain. The video processing device of claim 8, wherein the first channel is an upper channel, the second channel is a lower channel, and the volume gain control circuit: determines whether the object is located according to the position a lower side of the image; and when it is determined that the object is located on a lower side of the image, adjusting at least one of the first and second volume gains such that the second volume gain is greater than the first volume gain.