KR20070008232A - Apparatus and method of reproducing digital multimedia slow or fast - Google Patents

Abstract

An apparatus and a method for adjusting a digital multimedia X-speed rating are provided to set an X-speed rating for reproduction within a range of 0.5 to 2.0 times and offer a function of a digital image device capable of reproducing a natural picture and sound. A DVD(Digital Video Device) decoding member(210) loads and decodes data of a DVD medium. A video processing member processes video data from the DVD data decoded by the DVD decoding member. Video data are stored in a video data member(220). A TSM(Time Scale Modification) member(230) controls a video frame according to audio speed control. A video output member(240) outputs the video frame outputted from the video TSM member to a screen. An audio processing member includes an audio data member(250) for storing audio data.

Description

Digital multimedia speed control device and method {APPARATUS AND METHOD OF REPRODUCING DIGITAL MULTIMEDIA SLOW OR FAST}

1 is a block diagram showing the structure of a conventional DVD player.

2 is a block diagram showing the structure of an audio frame double speed control apparatus according to an embodiment of the present invention.

3 is a flowchart illustrating a method of controlling audio speed according to an exemplary embodiment of the present invention.

4 is a flowchart illustrating an operation of adjusting a video frame rate according to an exemplary embodiment of the present invention.

5 is a view illustrating a change of audio and video data of digital multimedia frame double speed control according to an embodiment of the present invention.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech processing system, and more particularly, to an apparatus and a method for naturally reproducing a screen and sound according to a reproduction speed. In more detail, the present invention is an apparatus and method for adjusting the speed of the screen when the speed is adjusted during playback by an up-sampling / down-sampling method according to the speed of the sound.

The TSM (Time Scale Modification) algorithm is applied to language compression or signal compression as a method of adjusting the speed of digital signals.

Upsampling / Downsampling, the most basic form of TSM algorithm, greatly reduces the sound quality of human voice or music signal, and then adjusts the speed without significantly reducing the sound quality of voice or music by various algorithms such as OLA, SOLA, PSOLA and WSOLA. To do this.

In other words, the variable speed reproduction of a voice signal that controls the speed of speech and audio without a large change in the tone makes the sound sound as if a person speaks quickly or slowly without changing or reproducing the tone even though there is a change in the playback speed. to be.

In case that the speed control of the accurate magnification of voice and audio is applied, it must be synchronized with the video of the time shift function of DTV, and even in the case of MP3 player, multimedia player, mobile phone, etc. It is very important that the rate is correct.

The method of controlling the speed of voice and audio without significant change in timbre was rapidly developed in 1985 when Salim Roucos of the United States proposed the OLA and SOLA algorithm, and the basic concept of OLA is compressed and expanded in FIG. Is indicated.

As shown in FIG. 1, the expansion of the time scale is reproduced slowly during the actual reproduction, and the reduction (compression) is rapidly reproduced.

TSM compresses or expands the input signal on the time axis to change the playback speed of the signal.TSM converts tempo, voice speed for foreign language learning, and data compression and restoration (MPEGx, H.26x, etc.). It is applied to various fields. In particular, it is applied to MP3 players, multimedia players, mobile phones, etc.

The TSM algorithm is a method of transforming the time axis and can be divided into a time domain method and a frequency domain method.

Typical time domain methods include an OLA algorithm for segmenting an input signal in units of windows and compressing or expanding the input signal through a predetermined operation (overlap amp; add) between neighboring windows, and a pitch between neighboring windows ( SOLA algorithm that obtains a more natural output voice by overcoming the shortcomings of OLA clicking (compression) and reverberation (expansion) by performing the above operation using pitch synchronization. And various SOLA modification algorithms.

Typical frequency domain methods include the Griffin and Lim algorithm using STFT.

The SOLA algorithm is a representative method for converting the tempo in the time domain and improves the disadvantages of the existing OLA method by performing an 'overlap-add' operation using pitch information between neighboring windows.

Roucos' SOLA and later proposed PSOLA / WSOLA have evolved into various forms as theories on how to find the optimal OLA location in various ways.

However, in current DVD players, when the playback speed is adjusted, only the screen playback can be controlled and no sound is heard.

Therefore, in case of watching multimedia through DVD, even if the user adjusts the speed at 2x or 0.5x speed, only the screen is adjusted and no sound is heard, causing inconvenience.

An object of the present invention is to provide a digital multimedia speed control device and method for providing a function of a digital video device that can determine the speed of playback between 0.5 and 2.0 times, and naturally reproduces the screen and sound.

It is still another object of the present invention to provide an apparatus and method for adjusting digital multimedia speed that enables natural and accurate adjustment of a sound side signal at a speed ratio of 0.5 to 2.0 times using a TSM algorithm. .

It is still another object of the present invention to provide an apparatus and method for adjusting digital multimedia speed that enables up-sampling / downsampling of a screen speed rate according to a sound speed rate during speed adjustment during playback of a digital video device.

Digital multimedia speed control device according to the present invention to achieve the above object,

An apparatus for adjusting digital multimedia reproduction speed, comprising: an audio TSM unit for converting decoded digital audio data into PCM data at a speed ratio according to a control signal; And a video TSM unit for converting the number of frames of decoded digital video data according to the audio signal converted by the audio TSM unit.

In the digital multimedia speed control apparatus of the present invention, the audio TSM unit converts audio data of a channel selected by a TSM algorithm, and converts audio data of an unselected channel into 0x0.

In the digital multimedia speed control apparatus of the present invention, the video TSM unit converts the number of frames by deleting or inserting a video frame according to the speed ratio of the audio signal converted by the audio TSM unit.

In addition, in the digital multimedia speed control method of the present invention, the playback speed control method of digital multimedia, the method comprising the steps of: loading the decoded audio data and converting according to the TSM algorithm at a speed ratio according to the control signal; Converting the number of frames of the decoded audio data according to the speed ratio of the converted audio data; And synchronizing and outputting the sync of the converted audio data and video data.

In the digital multimedia speed control method of the present invention, when the speed ratio according to the control signal is smaller than 1, TSM expansion is applied slowly, and when the speed ratio is greater than 1, TSM reduction is applied. It is characterized by performing fast playback.

In the digital multimedia speed control method of the present invention, the frame of the video data is characterized in that the selected frame is deleted or copied according to the speed ratio of the audio data.

In addition, in the digital multimedia double speed adjustment method of the present invention, the number of frames of the video data is determined by the following equation, and is determined by rounding when not an integer multiple.

<< V_N '== V_N * (S_N' / S_N) >>

In this case, V_N represents the number of original video frames, V_N 'represents the number of converted video frames, and S_N' / S_N represents a speed ratio of audio data.

In addition, in the digital multimedia speed control method of the present invention, the sink is characterized in that the synchronization of the audio data and video data synchronization based on a predetermined unit.

As mentioned above, the apparatus and method for adjusting digital multimedia speed according to the present invention allows the sound and the screen to be naturally adjusted at a speed ratio of 0.5 to 2.0 times in reproduction of a digital video apparatus, and hereinafter, digital An apparatus and method for controlling multimedia speed will be described in more detail.

However, in the following description of the present invention, a detailed description will be omitted for the known parts.

2 is a block diagram showing the structure of an audio frame double speed control apparatus according to an embodiment of the present invention.

Referring to FIG. 2, a DVD decoding unit 210 for loading and decoding data of DVD media, and a video processing unit for processing video data from the DVD data decoded by the DVD decoding unit 210 may store video data. The video data unit 220 and the video TSM unit 230 for adjusting the video frame according to the audio speed control and the video output unit 24 for outputting the video frame output from the video TSM unit 230 on the screen, It is composed.

The audio processing unit for processing audio data includes an audio data unit 250 storing audio data, an audio TSM unit 260 for quickly or slowly adjusting an audio signal stored in the audio data unit 250 according to a control signal. And a DAC 270 for converting the digital audio signal whose speed is controlled by the TSM unit 240 to analog, and a plurality of speakers 280 for outputting the analog audio signal converted by the DAC 270. .

Assuming that the DAC 270 outputs 5.1 channel audio signals, the speaker 280 has left and right channels 281 and 282, a center channel 283, and surround left and right channels 284 and 285. ) And woofer 286.

First, the audio TSM unit 260 of the audio processing unit compresses or expands an audio signal from 0.5 times to 2.0 times by the TSM algorithm.

At this time, the compression or expansion of the audio signal is performed only the channel selected by the TSM algorithm of the plurality of channels, the audio signal of the remaining channels are adjusted by changing to 0x0.

The compressed or expanded audio signal is received by the DAC 270 and converted into an analog audio signal, and the converted audio signal is output through the speaker 280 for each channel.

In addition, the video TSM unit 230 adjusts the video frame to match the audio signal processed by the audio TSM unit 260.

That is, when the PCM data of the original sound portion is converted into S_N 'through the speed adjustment at S_N, the video TSM unit 230 makes the video frame from V_N to V_N'.

At this time, the V_N 'can be calculated as V_N ㅧ (S_N / S_N'). If V_N 'does not yield an integer value, it is rounded up.

In addition, the synchronization of the screen is adjusted for each decoding unit so that the processing is performed naturally.

The speed control of the above-described audio and video will be described in detail as follows.

3 is a flowchart illustrating a method of adjusting audio speed according to an exemplary embodiment of the present invention, and FIG. 4 is a flowchart of a method of adjusting a video frame speed according to an exemplary embodiment of the present invention.

Referring to FIG. 3, when a speed control command is transmitted by a control signal in order to adjust audio speed, the DVD decoding unit 210 decodes an audio portion of one frame (S301). In this case, one frame represents one unit of decoding.

In addition, to be precise, the DVD decoding unit 210 simultaneously loads and decodes one frame of video and audio from digital multimedia media (DVD, etc.) and stores them in the video data unit 220 and the audio data unit 250, respectively. Done.

The audio TSM unit 260 loads audio PCM data stored in the audio data unit 250 for each channel (S302). At this time, audio data is sampled into S_N pieces.

The audio TSM unit 260 converts only the channels selected by the TSM algorithm among the loaded audio PCM data from S_N PCM data to S_N '(S303).

In this case, the audio TSM unit 260 converts the audio PCM data by the TSM algorithm in various ways such as upsampling / downsampling and OLA / SOLA / WSOLA / "Optimized AMDF & OLA". It is preferable to use the OLA / SOLA / WSOLA / "Optimized AMDF & OLA" method.

Since the above algorithm is a known technique, a detailed description thereof will be omitted.

The audio TSM unit 260 removes sound by changing all PCM data of the unselected channel to 0x0 (S304).

The audio PCM data converted by the steps S303 and S304 are input to the DAC 270 and output through the speaker 280 according to each channel (S305).

If the audio TSM unit 260 converts only the audio PCM data of the surround left and right channels into S_N 'outputs, the user may hear the adjusted audio output only through the surround left and right channel speakers 284 and 285. .

If there is more data to be processed (S306), audio speed control is performed through the process of steps S301 to 305.

The speed control of the video frame according to the above-described audio speed control will be described with reference to FIG. 4.

Referring to FIG. 4, when the DVD decoding unit 210 decodes the digital multimedia data and stores the video data in the video data unit 220 as described above with reference to FIG. 3, the video TSM unit 230 adjusts the frame. do.

First, the video TSM unit 230 decodes and loads a video portion of one frame (S401), and converts V_N frames into V_N 'to match the audio double speed control (S402).

V_N 'may be obtained by Equation 1 below.

V_N '== V_N * (S_N' / S_N)

The video TSM unit 230 converts the number of frames to V_N 'by deleting or copying a frame selected from V_N video frames according to a predetermined rule.

For example, if the audio speed control is 2.0, the even (or odd) frame is deleted. If the audio speed control is 0.5, the frame is copied by copying the previous frame and interposing it with the next frame. Convert the number.

In addition, the conversion of the number of frames is a problem that can be shifted from the audio sync since V_N 'is not divided by an integer multiple, so that the sync is synchronized with the audio double speed adjustment for each decoding unit so that the sync is not shifted. (S403).

The digital multimedia double speed adjustment as described above with reference to FIGS. 3 and 4 will be described with reference to FIG. 5.

5 is a view illustrating a change of audio and video data of digital multimedia frame double speed control according to an embodiment of the present invention.

Referring to FIG. 5, when the audio data and the video data appear in the same shape as the present data (a), the audio data appears in the compressed form in the TSM data (b) at the double speed, and the video data indicates an even numbered frame. You can see that the sink is removed.

Further, in the 0.5 times TSM data (c), the shape in which the audio data is extended twice as long and the shape of the video data copied between each frame and inserted between the next frame can be seen.

Digital multimedia speed control apparatus and method according to an embodiment of the present invention is characterized in that the screen and sound are naturally reproduced by adjusting the speed of the audio data using the TSM algorithm and adjusting the speed of the video data accordingly. However, it will be easy for a person skilled in the art to properly understand the present invention to create another embodiment by selecting the components within the scope of the present invention.

As described above, the apparatus and method for adjusting digital multimedia speed according to the present invention uses the TSM algorithm to adjust the speed of the audio data, and adjust the speed of the video data accordingly to adjust the screen speed to the sound speed. It is effective to make the screen and sound play naturally.

Claims (8)

In the device for adjusting the digital multimedia playback speed, An audio TSM unit converting the decoded digital audio data into PCM data at a speed ratio according to a control signal; And A video TSM unit converting the number of frames of decoded digital video data according to the audio signal converted by the audio TSM unit; Digital multimedia control device including a The method of claim 1, And the audio TSM unit converts audio data of a channel selected by a TSM algorithm and converts audio data of an unselected channel into 0x0. The method of claim 1, And the video TSM unit converts the number of frames by deleting or inserting a video frame according to the speed ratio of the audio signal converted by the audio TSM unit. In the playback speed control method of digital multimedia, Loading the decoded audio data and converting the decoded audio data according to the TSM algorithm at a speed ratio according to the control signal; Converting the number of frames of the decoded audio data according to the speed ratio of the converted audio data; And Synchronizing and outputting the sync of the converted audio data and video data Digital multimedia speed control method comprising a. The method of claim 4, wherein When the speed ratio according to the control signal is less than 1, TSM expansion is applied slowly to perform the playback, and when the speed ratio is greater than 1, the digital multimedia speed adjusting method is characterized in that to perform fast playback by applying TSM reduction. . The method of claim 4, wherein The frame of the video data is digital multimedia speed control method, characterized in that to delete, copy and insert the selected frame according to the rate of the audio data. The method of claim 4, wherein The frame number of the video data is determined by the following equation, and if not an integer multiple, digital multimedia speed adjustment method characterized in that the rounding. << V_N '== V_N * (S_N' / S_N) >> In this case, V_N represents the number of original video frames, V_N 'represents the number of converted video frames, and S_N' / S_N represents a speed ratio of audio data. The method of claim 4, wherein And synchronizing the sync of the audio data and the video data on the basis of a predetermined unit.

Images