KR20090002847A - Method and apparatus to display slow video by motion compensation and image interpolation - Google Patents

Abstract

A method and an apparatus for reproducing slow video by motion compensation and image interpolation are provided to prevent damage of connection of a frame even during a slow video reproduction mode and change a picture smoothly, thereby improving reproduction performance of the slow video. Slow video reproduction speed of inputted video data is determined(602). The number of frames to be reproduced based on the slow video reproduction speed and the number of frames to be generated by using an original frame of video data are determined. A new frame is generated by using the original frame(604). Based on the original reproduction speed of video data and slow video reproduction speed, the generated frame and original frame are arranged and outputted(606).

Description

Method and apparatus for slow video playback through motion compensation and image interpolation {Method and apparatus to display slow video by motion compensation and image interpolation}

Figure 1 shows a block diagram of a conventional video playback apparatus capable of slow video playback.

2 illustrates a conventional slow video playback method.

3 is a block diagram of a slow video playback apparatus according to an embodiment of the present invention.

4A illustrates a slow video playback method at 1 / N double speed according to an embodiment of the present invention.

4B illustrates a 1 / 2x slow video playback method according to an embodiment of the present invention.

4C illustrates a slow video playback method of 1 / 4x according to an embodiment of the present invention.

5 is a block diagram of a video playback apparatus capable of slow playback using an embodiment of the present invention.

6 shows a flowchart of a slow video playback method according to an embodiment of the present invention.

The present invention relates to a slow video playback method and apparatus, and more particularly, to a method and apparatus for implementing slow playback of video data through a motion compensation method or an image interpolation method using motion information or pixel information of the original video data. will be.

As the quality of video data increases and the size of display devices increases, the spread of high-quality display devices increases. As the spread of high quality display devices increases, the demand for large display screens increases. In order to realize a large screen and high definition, user demand for a frame rate, which has not been a problem in the past, is increasing.

For example, TVs supporting a frame rate of 120 Hz have recently emerged, and playing video data at a rate of 120 Hz enables smooth operation even on a large screen and a high-definition display device. In other words, in order for video data to be smoothly reproduced on a large screen and a high quality display device, a high frame rate is required.

In addition to basic playback that plays back video data at a normal playback speed, video playback devices capable of faster double speed playback and slower slow playback modes are widely used. In addition, there is a need for a method of improving the quality of video data when playing the video data in a trick mode for fast forwarding, rewinding, midway execution, and random access execution.

The conventional slow video reproduction method is implemented by slowly flowing video data in a state where PTS (Presentation Time Stamp) / STC (System Time Clock) synchronization is removed and slowly decoding the video data. In addition, if the video decoder supports the slow mode, when the video decoder is set to the slow mode, the video decoder outputs the data slowly and plays in the slow mode.

Figure 1 shows a block diagram of a conventional video playback apparatus capable of slow video playback.

The conventional video playback apparatus 100 capable of slow video playback includes a video buffer 102, a decoder 104, and a video DAC 106.

The video reproducing apparatus 100 decodes the encoded video data stored in the video buffer 102 in the decoder 104 and outputs the decoded video data for display on the screen in the video DAC 104. In this case, the video DAC 104 displays the video data at a predetermined speed according to the frame rate. The video DAC 104 may output video data through rendering.

2 illustrates a conventional slow video playback method.

Sequence 202 represents video data input at 30 Hz. Thus, each frame of sequence 202 is displayed at intervals of 1/30 seconds.

The sequence 204 shows a case where the sequence 202 is played back in the slow video playback mode at 1 / N times speed. Thus, the frames of sequence 204 are displayed at intervals of N × (1/30) seconds.

When the video data is reproduced by the conventional slow video reproduction method, the reproduction time of the video data per frame is increased, so that the time for the viewer to watch the same image is increased. In normal playback mode, the playback time of video data per frame is 1/30 seconds or 1/60 second, but in slow video playback mode, the playback time of video data per frame increases with slow double speed. Thus, the frame rate is reduced and the movements occurring upon reproduction of video data become unnatural.

Particularly in high-quality large display devices, when video data with a frame rate of 30 Hz or 60 Hz is played back in a slow video playback mode, unnatural motions in which motion is interrupted are more clearly observed. In particular, if you look at subtitles that are scrolling, you can see the text moving in a break.

Accordingly, an object of the present invention is to propose a slow video playback method and apparatus having improved slow video playback performance because the screen is smoothly switched without compromising frame connectivity even in the slow video playback mode.

In addition, the technical problem to be achieved by the present invention is to generate a new frame through motion compensation and video image interpolation using the original video data, to maintain the frame rate of the original video data.

Therefore, in order to achieve the above technical problem, the slow video playback method according to an embodiment of the present invention comprises the steps of determining the slow video playback speed of the input video data, the number of frames and video to be played based on the slow video playback speed Determining the number of frames to be generated using the original frame of the data, generating a new frame using the original frame and based on the original playback speed and slow video playback speed of the video data, Arranging and outputting.

In the frame generation step of the present embodiment, a new frame is generated by performing a motion compensation method using at least one frame among neighboring frames through the motion information of the original frame.

In the frame generation step of another embodiment, a new frame is generated by performing an image interpolation method using pixel information of at least one frame among the surrounding frames of the original frame.

In order to achieve the above technical problem, a slow video reproducing apparatus according to an embodiment of the present invention generates a new frame using a playback speed determiner for determining a slow video playback speed of input video data, and an original frame of video data. And a frame reproducing unit for arranging and outputting the generated frames and the original frames based on the frame generating unit and the original video playback speed and the slow video playback speed of the video data.

The frame generation unit of the preferred embodiment of the present invention, the frame generation unit of the motion compensation method using the motion information of the original frame and the pixel information of at least one frame of the peripheral frame of the original frame and at least one frame of the original frame and the peripheral frame A new frame is generated by performing at least one method of an image interpolation method using pixel information.

Hereinafter, a slow video reproduction method and apparatus according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 6.

3 is a block diagram of a slow video playback apparatus according to an embodiment of the present invention.

The slow video playback apparatus 300 includes a playback speed determiner 302, a frame generator 304, and a frame playback unit 306.

The reproduction speed determiner 302 determines the slow video reproduction speed of the input video data and outputs the same to the frame generator 304. An embodiment of the present invention implements a slow playback function of decoded and received video data. In this case, the frame rate for slow video playback may be already determined or may be set by the user.

The frame generator 304 determines the number of frames to be played based on the playback speed input from the playback speed determiner 302, and generates a new frame using the original frame of the video data to generate the new frame. Output to

In order to ensure that the playback time per frame in the slow video playback mode is the same as the playback time in the normal playback mode, the number of frames must increase according to the reduced playback speed for slow playback. In addition, another embodiment of the present invention is not the same as the frame rate of the input video data, but increases the number of frames according to the output frame rate to be reproduced at a similar frame rate.

Accordingly, an embodiment of the present invention newly generates a frame that is insufficient compared to the number of frames in the normal playback mode when playing back a frame of video data in the slow video playback mode. The number of frames to be generated as a new frame is determined based on the input playback speed. A method of generating a new frame will be described later.

The frame reproducing unit 306 arranges and outputs the frames and the original frames generated by the frame generating unit 304 in accordance with the original reproduction speed and the slow video reproduction speed of the video data. Since the frame generator 304 generates a new frame according to the slow video playback speed, the frame playback unit 306 may output the original frame and the generated frame while maintaining the original frame rate.

Hereinafter, a slow video playback method according to an embodiment of the present invention will be described in detail with reference to FIGS. 4A, 4B, and 4C. This is related to the role of the frame generator 304 of the slow video playback device 300.

4A illustrates a slow video playback method at 1 / N double speed according to an embodiment of the present invention.

Sequence 202 represents video data at 30 Hz. Therefore, the playback time of one frame is 1/30 second.

The sequence 402 shows a case in which the sequence 202 is played back at 1 / N speed in the slow video playback mode according to an embodiment of the present invention. One embodiment of the present invention allows a playback time per frame to be 1/30 seconds of the original video data in order to implement a slow video playback method that displays a smooth movement close to the normal playback mode. Thus, in order for the playback time per frame of sequence 402 to be 1/30 second, the number of frames must be increased in accordance with the slow video playback speed.

One embodiment of the present invention uses the original video data to generate a frame that is insufficient compared to the number of frames to be increased. The method for generating a new frame using the original frame uses motion information and pixel information of the surrounding frame of the original frame.

According to an embodiment of the present invention, a new frame is generated through a motion compensation method inferring a new frame from at least one neighboring frame using motion information of the original frame.

The motion compensation method according to an exemplary embodiment of the present invention generates a new frame by employing pixel information of a position designated by motion information of the original frame in a peripheral frame of the original frame. In addition, one peripheral frame may be used according to a characteristic of motion information of the original frame, and pixel information of a plurality of peripheral frames may be used.

Another embodiment of the present invention generates a new frame through an image interpolation method using a pixel degree of at least one peripheral frame of the original frame.

The image interpolation method according to an embodiment of the present invention is based on a temporal interpolation method using frames of different time since new frames are generated using original frames. By a temporal interpolation method, in consideration of the temporal distance from the surrounding frame, a new frame is generated by adopting the weighted sum of all of the at least one pixel information of the surrounding frame by multiplying each temporal weight and using the weighted sum that is added together.

One or more peripheral frames of the original frame for implementing the image interpolation method may be provided.

In addition, the frame generator 304 determines the number of frames to be reproduced according to the reproduction speed determined by the reproduction speed determiner 302 and determines the number of frames to be generated using the original frame.

For example, if one original frame is input and two half times are played, two frames should be displayed. Therefore, one frame must be newly generated, and one new frame is generated using one original frame. Similarly, if two original frames are input and played in the slow video playback mode at 1 / 4x speed, eight frames are required, and six additional frames must be generated using the two original frames.

That is, if one frame is input and is to be reproduced in the slow video playback mode at 1 / N times, ax (N-1) frames should be generated using the one frame.

4B illustrates a 1 / 2x slow video playback method according to an embodiment of the present invention.

Sequence 202 is video data of 30 Hz, and the reproduction time of one frame is 1/30 second.

The sequence 404 shows a case in which the sequence 202 is played back at 1 / 2x speed in a slow video playback mode according to an embodiment of the present invention.

According to an embodiment of the present invention, since one frame must be reproduced at a rate of 1/30 seconds in order to keep the video data at a frame rate of 30 Hz, a frame is generated by using information of the original frame between two original frames. You must play a new frame. Therefore, the original data is reproduced every 2 × (1/30) seconds, but the frame is reproduced every 1/30 second due to the frame generated by using the motion information or pixel information of the original data. Softens

4C illustrates a slow video playback method of 1 / 4x according to an embodiment of the present invention. The sequence 202 is identical to the sequence 202 of FIGS. 2, 4A, and 4B described above.

The sequence 406 shows a case in which the sequence 202 is played back at 1 / 4x in the slow video playback mode according to an embodiment of the present invention. According to an embodiment of the present invention, in order for the video data to maintain a frame rate of 30 Hz, a new frame generated by using information of the original frame must be reproduced between two original frames. Therefore, the original data is reproduced every 4x (1/30) seconds, but the frame generated using the information of the original data is reproduced every 1/30 seconds while maintaining the continuity of the video data.

5 is a block diagram of a video playback apparatus capable of slow playback using an embodiment of the present invention.

The video reproducing apparatus 500 including the slow video reproducing apparatus 300 according to an embodiment of the present invention includes a video buffer 502, a decoder 504, a slow video reproducing apparatus 300, and a video DAC 506. It includes.

The video buffer 502 stores the encoded and transmitted video data and outputs it to the decoder 504.

The decoder 504 decodes the encoded video data and outputs the decoded video data to the slow video reproducing apparatus 300.

Slow video playback apparatus 300 according to an embodiment of the present invention, by using the video data decoded by the decoder 104 generates a new frame at a predetermined playback speed, arranged in order of the output and output do.

The video DAC 506 reproduces the frame output from the slow video reproducing apparatus 300 in the same manner as the reproduction time of the original frame, and maintains the frame rate of the original video data.

Accordingly, the video playback apparatus 500 including the slow video playback apparatus 300 according to an embodiment of the present invention may smoothly implement slow video playback by generating a new frame using the decoded video data.

6 is a flowchart of a slow video playback method according to an embodiment of the present invention.

In step 602, the slow video playback speed of the input video data is determined. The slow video playback speed may be predetermined or set by the user.

In step 604, the number of frames to be reproduced and the number of frames to be generated are determined based on the slow video reproduction speed, and a new frame is generated using the original frame. The new frame is generated through motion compensation or image interpolation using motion information or pixel information of the original frame or pixel information of the surrounding frame.

In step 606, the generated frame and the original frame are arranged and output based on the original playback speed and the slow video playback speed of the video data. The original frame is output according to the slow video playback speed, but as the generated frame is added, the video data can be played at a speed similar to or the same as the playback speed of the original video data.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may include a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), an optical reading medium (eg, CD-ROM, DVD, etc.) and a carrier wave (eg, Internet Storage medium).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

According to the slow video playback method and apparatus of the present invention, there is an effect of smoothing the screen transition due to the switching of the video frame even in the slow video playback mode.

In addition, by generating a new frame through motion compensation and video image interpolation using the original video data, the playback time per frame is approximated to the playback time when the original video frame is normally reproduced by maintaining the frame rate of the original video data. Smooth video playback has the effect possible.

Therefore, the video playback apparatus has an effect of improving the slow video playback performance.

The present invention can be used in industrial fields such as a photonic player that requires trick play of video, media play with storage, and the like.

Claims (5)

Determining a slow video playback speed of the input video data; Determining the number of frames to be reproduced based on the slow video reproduction speed and the number of frames to be generated using the original frame of the video data, and generating a new frame using the original frame; And And arranging and outputting the generated frame and the original frame based on the original playback speed of the video data and the slow video playback speed. The method of claim 1, In the frame generation step, the new video frame is generated by performing a motion compensation method using pixel information of at least one frame of surrounding frames based on the motion information of the original frame. The method of claim 1, The frame generation step may include generating the new frame by performing an image interpolation method using pixel information of at least one frame among the original frame and surrounding frames of the original frame. In the playback device of the video data capable of slow playback, A playback speed determiner configured to determine a slow video playback speed of the input video data; A frame generator configured to determine the number of frames to be reproduced based on the slow video reproduction speed and the number of frames to be generated using the original frames of the video data, and to generate a new frame using the original frames; And And a frame reproducing unit arranged to output the generated frames and the original frames based on the original reproduction speed and the slow video reproduction speed of the video data. The method of claim 4, wherein The frame generation unit using a motion compensation method of the original frame and pixel information of at least one frame among the surrounding frames of the original frame; An image interpolation method using pixel information of at least one frame of the original frame and surrounding frames And generating the new frame by performing at least one method.

Images