KR101042614B1 - Low-power video playback method based on decoding time estimation - Google Patents

Abstract

A low power video playback method through decoding time prediction is provided. A low power video playback method using decoding time prediction may include obtaining size information of a frame included in a video, sampling the decoding time for each frame, and performing log regression analysis using the sampled values. Deriving a relationship with the decoding time and predicting a decoding time of a frame to be reproduced based on the derived relationship, adjusting an operating frequency of a central processing unit based on the predicted decoding time and adjusting the Decoding the video frame based on the specified operating frequency.

Decoding Time, Prediction, Log Regression, Deadline Miss, Operating Frequency

Description

LOW-POWER VIDEO PLAYBACK METHOD BASED ON DECODING TIME ESTIMATION}

The present invention relates to a low power video playback method through decoding time prediction, and more particularly, to predict a decoding time through log regression, and to intentionally generate a deadline miss to control an operating frequency of a central processing unit. It relates to a video playback method.

Recent advances in multimedia and network technologies have made it possible to access video services at any time using devices such as laptops, PMPs and mobile phones. However, video programs require a large amount of CPU calculations and involve a high rate of CPU power consumption. Since the CPU is one of the most power consuming devices in mobile systems, it is important to reduce the CPU power consumption in portable multimedia devices.

An effective way to reduce CPU power consumption is to use dynamic voltage regulation (DVS) technology that changes the operating frequency (speed) of the CPU. Most CPUs used in PMPs, mobile phones and laptops support multiple levels of operating frequency (CPU speed) for low power operation and provide an externally controllable way. Dynamic voltage regulation allows you to vary the voltage applied to the processor based on CPU speed and to dynamically switch CPU speed. The energy consumed by the CPU varies in proportion to the square of the input voltage, so reducing the voltage saves a lot of power. At the same time, however, the execution speed of the program is slowed down, so that the real-time requirements of the video player cannot be satisfied.

Also, because video codecs use compression techniques, each frame must be decoded and scattered on the screen at a fixed refresh rate, which requires a lot of computation. For example, to play 25 frames of video, one frame must be decoded once every 41 ms. However, since it is typically difficult to predict future decoding times, the utility of dynamic voltage regulation techniques will primarily depend on the ability to predict future CPU requirements. If the prediction is larger than the actual decoding time in the prediction of the decoding time, it will select higher CPU speed than necessary, resulting in a waste of power. If the prediction is smaller than the actual decoding time, it will select a lower CPU speed than necessary. It does not meet the real-time requirements of the video player.

Existing prediction techniques predict the decoding time of the next frame based on the decoding time of the previous frame. These methods convert the measured decoding time into the decoding time at the highest speed CPU and then predict the next decoding time, in fact the decoding time of the frame does not have a precise linear relationship with the speed of the CPU. As a result, false predictions can hurt real-time requirements or waste power.

On the other hand, as the first prior art, Korean Patent Publication No. 2006-0071293 discloses an invention entitled "MPEG playback apparatus and method using variation of operating frequency and driving voltage according to decoding time prediction", and decoding The present invention relates to a technique for estimating an operation amount of frame decoding and a time required for decoding using an MPEG frame type and size information to be used, and setting an operating frequency of the processor and a driving voltage according thereto.

However, since the above-described first prior art predicts the decoding time by using an average value of the decoding times per frame, and does not sample the decoding time per frame for each operating frequency, in the video and the rapidly changing video of the frame, There was a problem in that the decoding time prediction could not be accurately made and the power of the video reproducing apparatus was wasted.

Accordingly, there is a strong demand for a video reproducing apparatus capable of minimizing power consumption by accurately analyzing a correlation between a video frame and a decoding time according to the type of frames included in a video and the current system environment.

Some embodiments of the present invention provide a low power video reproduction method of sampling a decoding time for each frame and predicting a decoding time of a frame to be reproduced by log regression analysis.

In addition, an embodiment of the present invention provides a low-power video playback method for sampling the decoding time for each frame sequentially by operating frequency.

In addition, an embodiment of the present invention provides a low-power video playback method for intentionally generating a deadline miss to adjust the operating frequency of the central processing unit.

As a technical means for achieving the above technical problem, the first aspect of the present invention is to obtain the size information of the frame included in the video, sampling the decoding time for each frame, by using the sampled value, Deriving a relationship between the frame and the decoding time through log regression analysis, and predicting a decoding time of a frame to be reproduced, based on the derived relationship, based on the predicted decoding time. The method may provide a low-power video playback method through decoding time prediction, the method including adjusting an operating frequency and decoding the video frame based on the adjusted operating frequency.

In addition, in the first aspect of the present disclosure, adjusting the operating frequency of the central processing unit may generate a deadline miss within a preset range to adjust the operating frequency.

Also, in the first aspect of the present invention, the decoding of the video frame may include: measuring a decoding time of the video frame and providing the measured value as the sampled value to determine a decoding time of the frame to be reproduced. Can be used to predict

Also, in the first aspect of the present invention, the sampling of the decoding time may include sampling the decoding time of the frame for each operating frequency of the CPU.

Also, in the first aspect of the present invention, the sampling of the decoding time may include selecting an operating frequency of the central processing unit by sequentially alternating high and low frequencies, and decoding time of the frame at the selected operating frequency. Can be sampled.

According to the above-described problem solving means of the present invention, by accurately predicting the decoding time of the frame to be reproduced through log regression analysis, it is possible to efficiently reduce the power consumption of the video reproducing apparatus even in a sudden change of the image and frame.

In addition, according to the above-described problem solving means of the present invention, it is possible to cancel the decoding delay time generated in sampling the decoding time for each frame.

In addition, according to the above-described problem solving means of the present invention, it is possible to intentionally generate a deadline miss, effectively reducing the power consumption of the video playback apparatus without harming the quality of service (QoS).

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, which means that it may further include other components, except to exclude other components unless otherwise stated.

Hereinafter, with reference to the accompanying drawings will be described in detail the present invention.

1 is a detailed block diagram of a low power video reproducing apparatus through decoding time prediction according to an embodiment of the present invention.

As shown in FIG. 1, a low power video reproduction apparatus through decoding time prediction according to an embodiment of the present invention may include a frame information acquisition unit 100, a sampling unit 200, a decoding time prediction unit 300, and a deadline. And a setting unit 400 and an operating frequency adjusting unit 500.

The frame information acquisition unit 100 obtains the type and size information of each frame included in the video from the video file.

When the video file is an MPEG file, the frame information acquisition unit 100 may obtain the size information of the video frame from a header or a tail of the MPEG file. For example, the frame information acquisition unit ( 100 may extract frame type information and size information from the idx1 header of the AVI video file compressed by the MPEG method. In the case of a video compressed by the MPEG compression method, there are frames of type I, P, and B, and each type of frame has a different size.

The sampling unit 200 samples the decoding time for each video frame. The sampling unit 200 may sample the decoding time of each video frame for each operating frequency by changing the operating frequencies of the central processing unit after the video playback starts.

In addition, the sampling unit 200 may select an operating frequency of the central processing unit by sequentially alternating a high frequency and a low frequency, and sample a decoding time of a frame at the selected operating frequency. That is, the sampling unit 200 may select the lowest operating frequency before selecting the highest operating frequency, and select the second lowest operating frequency before selecting the second highest operating frequency.

개의 샘플이 얻어질 때 까지

의 순서로 동작 주파수의 단계를 선택할 수 있다. 만약,

일 경우, 동작 주파수의 단계는

의 순으로 선택될 수 있다.For example, if the operating frequency is selectable in NS steps,

Until samples are obtained

The steps of operating frequency can be selected in the order of. if,

If, the step of operating frequency

It may be selected in order.

Accordingly, in sampling the decoding time of the frame, the sampling unit 200 may cancel the decoding delay time generated when sampling at a low operating frequency through sampling at a later high operating frequency.

개의 샘플을 측정하기 위하여

초 동안 샘플링을 해야 하지만, 이는 전체 동영상의 재생 시간을 고려했을 때 매우 짧은 시간에 불과하므로, 동영상을 시청하는 시청자에게는 거의 불편을 주지 않을 수 있다(만약,

이고

라고 가정했을 때, 25

의 동영상의 경우 샘플링에 8초가 필요할 뿐임).Sampling unit 200

To measure two samples

You need to sample for seconds, but this is a very short amount of time given the duration of the entire video, which can be inconvenient for viewers watching the video (

ego

Assuming 25

Videos only require 8 seconds of sampling).

The decoding time predictor 300 predicts a decoding time of a video frame to be reproduced later by using the sampled decoding time for each frame.

The decoding time predictor 300 may derive a relationship between the video frame and the decoding time through log regression analysis, and then predict the decoding time of the frame to be reproduced later.

번째 프레임의 디코딩 시간을 예측할 수 있으며, 디코딩 시간의 계산을 위하여 gettimeofday 함수를 이용할 수 있다. 만약, 동작 주파수 단계가

일 때, 다음 프레임의 디코딩 시간은

라 정의하며 수학식 1과 같이 표현할 수 있으며,

와

는 수학식 2로부터 계산될 수 있다.The decoding time predicting unit 300 is based on the sampled measured value, and the next frame is calculated by using a log linear regression curve.

The decoding time of the first frame can be predicted and the gettimeofday function can be used to calculate the decoding time. If the operating frequency step

When, the decoding time of the next frame is

It can be defined as Equation 1,

Wow

Can be calculated from Equation 2.

&Quot; (1) "

<Equation 2>

,

일 때,

,

when,

는CPU의 동작 주파수의 속도 단계,

는 동영상 화질의 단계,

는

번째 프레임의 예측된 디코딩 시간이다. 또한,

는 화질의 단계인

가 적용되었을 때,

번째 프레임 디코딩 시 선택되는 CPU 속도의 단계이다

.here,

Speed step of the operating frequency of the CPU,

Is the level of video quality,

Is

The predicted decoding time of the first frame. Also,

Is the level of image quality

When is applied,

The speed of the CPU that is selected when decoding the second frame.

.

는

의 화질 단계가 선택되었을 때, 예측에 사용될 프레임의 집합이며

,

는 수학식 3과 같이 나타낼 수 있다.Also,

Is

When the quality level of is selected, the set of frames to be used for prediction

,

Can be expressed as in Equation 3.

&Quot; (3) &quot;

는 CPU의 동작 주파수가

일 때,

번째 프레임의 디코딩 시간이다

.Also,

Is the operating frequency of the CPU

when,

Is the decoding time of the first frame

.

번째 프레임을 디코딩하기 전,

and

를 재계산할 수 있으며, 이를 위해 각 동작 주파수에서 최소한

개의 측정값이 필요하다. 그러나 동영상 재생 초기에, 샘플의 개수가 너무 작으면 정확한 예측이 이루어 질 수 없다.The decoding time predicting unit 300 is a log linear regression curve equation,

Before decoding the first frame,

and

Can be recalculated, for which

Two measurements are required. However, at the beginning of video playback, if the number of samples is too small, accurate prediction cannot be made.

3 shows how accurate the prediction value is according to the change in the number of samples as an average error with the actual decoding time, as shown in FIG. 3, and when the number of samples is 50 or more, accurate prediction can be performed. there was.

The deadline setting unit 400 sets an allowable range of the deadline miss to be intentionally generated.

Deadline misses are those that exceed the decoding timeout (allowed decoding delay time), for example, for a video that needs to play 25 frames per second, even though one frame must be decoded within 40ms, If exceeded.

Also, the deadline setting unit 400 may set an allowable range of the deadline miss within a range that does not harm the quality of service (QoS).

The operating frequency adjuster 500 adjusts the operating frequency of the CPU based on the estimated decoding time.

The operating frequency adjusting unit 500 may adjust the operating frequency based on the allowable range value of the deadline miss and the predicted decoding time, and minimize the energy consumption of the playback device within the allowable range of the deadline miss. Frequency can be selected.

In addition, the operating frequency adjusting unit 500 may adjust the decoding time by reflecting the decoding time of the video frame after predicting the decoding time, and adjust the operating frequency based on the adjusted decoding time.

In addition, the operating frequency adjusting unit 500 may determine a desired level of image quality, determine a set of operating frequencies capable of guaranteeing the determined level of image quality, and adjust the operating frequency to a smaller low operating frequency. .

The operating frequency controller 500 may change the operating frequency of the CPU by using the CPU Frequency Governor provided by the Linux kernel, and may continuously adjust the operating frequency.

Hereinafter, a low power video reproduction method through decoding time prediction according to an embodiment of the present invention will be described with reference to FIG. 2.

2 is a detailed flowchart of a low power video playback method through decoding time prediction according to an embodiment of the present invention.

Step S200 is a step of obtaining size information of a frame included in the video.

In the case of a video compressed by the MPEG compression method, size information of a video frame may be obtained from a header or a tail of an MPEG file.

Step S202 is a step of sampling the decoding time for each frame.

When the video playback starts, the decoding time for each video frame may be sampled by varying the operating frequency of the central processing unit with respect to the video frame being played. As shown in FIG. 3, when the number of samplings is 50 or more, accurate prediction of the decoding time is possible. In addition, the operating frequency of the central processing unit may be selected by sequentially alternating high and low frequencies, and sampling the decoding time of the frame at the selected operating frequency. Decoded time values for each sampled operating frequency and video frame size are used to predict a decoding time of a video frame to be reproduced later.

Step S204 is a step of predicting a decoding time of a frame to be reproduced using the sampled decoding time for each frame.

In step S204, through the logarithmic regression analysis, a relationship between a frame and a decoding time may be derived from a sampled decoding time value of each frame, and the decoding time of a video frame to be reproduced in the future may be predicted. That is, based on the sampled decoding time value, the decoding time of the next frame may be predicted by using a calculation formula of a log linear regression curve.

Step S206 is for setting the allowable range of the deadline miss.

If the range of the deadline miss (decoding timeout), that is, the range of allowable decoding delay time, is set high, the operating frequency can be adjusted to reduce power consumption, but the video quality is deteriorated. On the contrary, when the range of the deadline miss is set narrow, the operating frequency cannot be adjusted low, thereby increasing power consumption.

Therefore, in step S206, the range of the death line miss is varied in consideration of the type of video content (movie, drama, sports, etc.), the resolution of the display device, the level of the viewer, and the like, as long as the viewer does not feel the deterioration of image quality. Can be set.

Step S208 is adjusting the operating frequency of the central processing unit based on the predicted decoding time.

In operation S208, the operating frequency may be adjusted based on the allowable range value of the deadline miss and the predicted decoding time, and the operating frequency may be selected to minimize the energy consumption of the playback apparatus within the allowable range of the deadline miss. have. In addition, it is possible to determine a desired level of image quality, to determine a set of operating frequencies capable of guaranteeing the determined level of image quality, and to adjust the operating frequency to a smaller low operating frequency.

In addition, by using the CPU Frequency Governor provided by the Linux kernel, the operating frequency of the CPU can be changed and the operating frequency can be continuously adjusted.

Step S210 is a step of decoding the video frame based on the adjusted operating frequency. In operation S210, the video frame may be decoded at the adjusted operating frequency and the decoding time may be measured. The measured decoding time value may be provided as a sampling value and used to predict the decoding time of a frame to be reproduced later.

Hereinafter, the accuracy of the prediction value of the decoding time according to the change in the number of samples in the sampling step according to an embodiment of the present invention will be described with reference to FIG. 3.

3 is an example of a graph illustrating an average error between a predicted value of a decoding time and an actual decoding time according to a change in the number of samples in a sampling step according to an embodiment of the present invention.

As shown in FIG. 3, when the number of samplings is 50 or less, an error value between the prediction value of the decoding time and the actual decoding time is large, and when the number of samplings is 50 or more, it can be seen that the accurate decoding time can be predicted. Therefore, it is preferable to perform decoding time prediction after securing 50 or more sampling numbers.

Furthermore, in an embodiment of the present invention, since the decoding time at the operating frequency adjusted based on the predicted decoding time may be added to the sampling value, as the video is played, the number of sampling increases to predict the decoding time more accurately. It becomes possible.

Hereinafter, an example of a graph illustrating decoding time for each frame size at various operating frequencies according to an embodiment of the present invention will be described with reference to FIG. 4.

4 is an example of a graph illustrating decoding time for each frame size at various operating frequencies according to an embodiment of the present invention.

As can be seen in Figure 4, the relationship between the decoding time and the frame size is represented by a log function, it can be seen that the decoding time increases in the form of a log function according to the size of the frame (to confirm this in Figure 2 Linear and logarithmic regression curves are displayed together.). That is, the decoding time does not increase or decrease linearly with the operation speed of the CPU. As the CPU speed increases, the decoding time decreases, but no exact inverse relationship is found. Therefore, when using a log linear regression analysis method as in an embodiment of the present invention, it is possible to predict the decoding time more accurately.

Hereinafter, the accuracy of decoding time prediction using log regression analysis according to an embodiment of the present invention will be described with reference to FIGS. 5 and 6.

FIG. 5 is a chart comparing average error values of the predicted decoding time and the actual decoding time when the decoding time prediction method and the conventional method according to an embodiment of the present invention are used.

6 is a cumulative distribution chart comparing the accuracy of the predicted decoding time and the actual decoding time using the decoding time prediction method and the conventional method according to an embodiment of the present invention.

In addition, in FIG. 5 and FIG. 6, LSE refers to a method of predicting decoding time using linear regression analysis at a maximum CPU speed (operating frequency), and in the LSE method, linearly converts a measurement value of a decoding time at a maximum CPU speed. We derive a linear relationship between frame size and decoding time at the highest CPU speed. In addition, the decoding time of each CPU speed step is estimated by linearly converting the predicted value of the maximum CPU speed.

Also, LBF refers to a method of predicting decoding time using different linear regression curves at various CPU speeds. Unlike the present invention, where the log regression relationship is used for each CPU speed step, the LBF uses a different linear regression curve at each CPU speed.

As shown in FIGS. 5 and 6, the decoding time of 18000 frames is measured by each method. As a result, the decoding time can be most accurately predicted by the method according to an embodiment of the present invention. Able to know.

In addition, according to an embodiment of the present invention, while 82% of the entire frame has an error range of less than 3ms, 49% of the total frame in the LSE, only 72% of the LBF of the LBF is within 3ms of the error range It can be seen that it has.

Furthermore, according to an embodiment of the present invention, the accuracy in the sports video shows a significant difference. That is, according to an embodiment of the present invention, it can be seen that the video and frame can be effectively coped with sudden changes in the video playback.

Hereinafter, the relationship between the allowable range of the deadline miss and the power consumption will be described with reference to FIG. 7.

7 is a graph illustrating a change in power consumption of a video device according to a decoding delay time DB.

As shown in FIG. 7, when the allowable range of the deadline miss is set to 6 ms (decoding delay time DB = 6 ms), when the allowable range of the deadline miss is 0 ms ((decoding delay time DB = 0 ms)) Compared to 8% to 12% power reduction.

Therefore, the power consumption of the video reproducing apparatus can be efficiently reduced when the allowable range of the deadline miss is set within the range in which the user can tolerate the deterioration of the video quality.

One embodiment of the present invention can also be implemented in the form of a recording medium containing instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

1 is a detailed configuration diagram of a low power video playback apparatus through prediction of decoding time according to an embodiment of the present invention.

2 is a detailed flowchart of a low power video reproduction method through decoding time prediction according to an embodiment of the present invention.

3 is an example of a graph illustrating an average error between a predicted value of a decoding time and an actual decoding time according to a change in the number of samples in a sampling step according to an embodiment of the present invention.

4 is an example of a graph illustrating decoding time for each frame size at various operating frequencies according to an embodiment of the present invention.

5 is a table comparing the average error value of the predicted decoding time and the actual decoding time when using the decoding time prediction method and the conventional method according to an embodiment of the present invention.

6 is a cumulative distribution chart comparing the accuracy of the predicted decoding time and the actual decoding time when using the decoding time prediction method and the conventional method according to an embodiment of the present invention.

FIG. 7 is a graph illustrating a change in power consumption of a video device according to a decoding delay time DB. FIG.

Claims (5)

In the low power video playback method by decoding time prediction, Obtaining size information of a frame included in the video, Sampling a decoding time for each frame; Using the sampled value, deriving a relationship between the frame and the decoding time through log regression analysis, and predicting a decoding time of a frame to be reproduced based on the derived relationship; Adjusting an operating frequency of the central processing unit based on the predicted decoding time; and Decoding the video frame based on the adjusted operating frequency Low power video playback method comprising a. The method of claim 1, The adjusting of the operating frequency of the central processing unit includes generating a deadline miss within a preset range to adjust the operating frequency. The method of claim 1, The decoding of the video frame comprises: measuring a decoding time of the video frame and providing the measured value as the sampled value so that it can be used to predict the decoding time of the frame to be played back. How to play the video. The method of claim 1, The sampling of the decoding time may include sampling the decoding time of the frame for each operating frequency of the CPU. The method of claim 4, wherein The sampling of the decoding time may include selecting an operating frequency in order of a lowest operating frequency, the highest operating frequency, the second lowest operating frequency, and the second highest operating frequency among the selectable operating frequencies in the CPU. Sampling the decoding time of the frame at a selected operating frequency.

Images