KR20210084211A - Method and apparatus for video frame transmission based on expected value maximization algorithm in Wi-Fi Direct - Google Patents

Abstract

The present invention discloses a method and an apparatus for transmitting a video frame based on an expected value maximization algorithm in Wi-Fi Direct. According to the present invention, the apparatus for transmitting the video frame based on the expected value maximization algorithm in Wi-Fi Direct includes: a processor; and a memory connected to the processor, wherein the memory stores program instructions executable by the processor to: update distribution of an x-frame size by using previously received x-frame size information (where x is one of I, P, and B) based on the expected value maximization algorithm; determine a length of an awake period for a next x-frame based on the updated distribution of the x-frame size; and transmit the next x-frame to a client based on the determined awake period.

Description

Method and apparatus for video frame transmission based on expected value maximization algorithm in Wi-Fi Direct

The present invention relates to a method and apparatus for transmitting a video frame based on an expected value maximization algorithm in Wi-Fi Direct.

Wi-Fi Direct has been proposed to connect two devices without a wireless access point (AP).

As video streaming becomes more convenient with the recent distribution of Wi-Fi Direct compatible mobile devices, the demand for video quality and improved streaming stability is increasing. Wi-Fi Direct is expected to be mainly used because it has higher data rates and coverage compared to Bluetooth and ZigBee. expected.

Video streaming requires stable information flow and packet delivery up to the deadline, but it is difficult to reliably provide these services due to the limited bandwidth of wireless networks.

In order to provide the best end-to-end service in such a system, the video frame structure and radio resource allocation should be considered together. Various compression techniques have been developed to reduce redundancy in storage, transmission capacity, and space/time in relation to the video frame structure.

According to hierarchical coding techniques such as MPEG (Moving Picture Experts Group)-x and H.264/AVC, three main types of video frames are defined for compressing video streams: I-frames, P-frames and B-frames. .

Given the limited battery capacity of mobile devices, it is important to provide efficient energy consumption for Wi-Fi Direct devices. In the Wi-Fi Direct standard, in order to reduce unnecessary energy consumption, two power saving methods, an opportunistic technique and a notice of absent (NoA) technique, have been proposed.

In Wi-Fi Direct, the group owner (GO) plays the role of an AP in the existing Wi-Fi network, and devices connected around the GO are defined as clients.

The opportunistic technique is a technique characterized in that when the GO starts transmitting data to the client within a predetermined CT Window (Client Traffic Window) time, the GO can enter the sleep section again only after all the transmission is completed, and the NoA technique is This is a technique characterized in that the GO can transmit data to the client only within a predetermined awake interval.

Among them, in the NoA technique, an awake period should be sufficient to transmit a frame in order to prevent loss of a video frame. However, there is a problem in that a considerable amount of energy is wasted because the client is awake even when the frame transmission is completed.

Registered Patent 10-1704655

In order to solve the problems of the prior art, the present invention proposes a method and apparatus for transmitting a video frame based on an expected value maximization algorithm in Wi-Fi Direct that can minimize delay and omission of video frames while reducing power consumption.

In order to achieve the above object, according to an embodiment of the present invention, there is provided an apparatus for transmitting a video frame based on an expected value maximization algorithm in Wi-Fi Direct, comprising: a processor; and a memory coupled to the processor, wherein the memory is configured to distribute the x-frame size using previously received x-frame (x is one of I, P, and B) size information based on an expected value maximization algorithm. to determine the length of the awake interval for the next x-frame based on the updated distribution of the x-frame size, and based on the determined awake interval, the next x-frame is transmitted to the client Preferably, there is provided a video frame transmission apparatus for storing program instructions executable by the processor.

The program instructions may update a parameter of a frame size distribution including a shape parameter, a scale parameter and a weight parameter of all blending components using the expected value maximization algorithm.

The program instructions are configured to: maximize a log-likelihood function for the shape parameter, the scale parameter, and the weight parameter of the blending component using the expected value maximization algorithm, and use the maximized log-likelihood function to determine the shape parameter, the scale parameter and obtain a new estimate of the weight parameter.

The update may be for automatically reconstructing a probability density function of a video frame.

According to another aspect of the present invention, as a video frame transmission method based on an expected value maximization algorithm in Wi-Fi Direct, (a) reconstructing a probability density function of an x-frame (x is one of I, P and B) based on the expected value maximization algorithm step; (b) determining the length of the awake interval for the next x-frame using the reconstructed probability density function; and (c) transmitting the next x-frame to a client based on the determined awake period.

According to another aspect of the present invention, there is provided a computer-readable program stored in a recording medium for performing the above method.

In the present invention, general parameters (shape parameter, scale parameter, and weight of all mixture components) using an expectation maximization algorithm to determine the length of an awake interval required to transmit a video frame It automatically reconstructs a new probability density function for each video frame by updating, which has the advantage of reducing unnecessary energy consumption and transmission delay in Wi-Fi Direct.

1 is a diagram showing the configuration of a video frame transmission apparatus according to an embodiment of the present invention.
2 is a diagram illustrating the sequence of the GoP structure.
Figure 3 shows the process of the expected value maximization algorithm in GMM.
4 is a diagram for explaining a process in which residual I-frames and P-frames are transmitted in the next B-frame period.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

The present invention proposes a method of dynamically adjusting the length of an awake interval in a beacon using an Expectation Maximization (EM) algorithm in order to increase energy efficiency and reduce transmission delay.

In the present embodiment, the number and length of the awake period in the beacon period are adjusted according to the distribution of frame sizes per frame type. The expected maximization algorithm is used to update the parameters of the frame size distribution, including shape parameters, scale parameters and weight parameters of all blending components. In addition, it considers the interdependence between video frames, and increases the transmission success probability of a video frame having a higher priority.

Accordingly, in the present embodiment, the priority of frame transmission per frame type is determined in order to improve transmission efficiency.

The shape parameters, scale parameters and weight parameters of all blending components are used for shape, shift, etc. of the frame size distribution.

The scale parameter is used to shift the peak of the frame size distribution, and increasing values of the scale parameter increase the average and variance of the frame size distribution.

When the shape parameter is fixed, a decreasing value of the scale parameter causes the frame size distribution to become an exponential distribution, whereas an increasing value of the scale parameter causes the frame size distribution to become a normal distribution.

The weight parameter is the coefficient of each frame size distribution. The new distribution consists of the sum of the distribution-weighted results.

1 is a diagram showing the configuration of a video frame transmission apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the apparatus for transmitting a video frame according to the present embodiment may include a processor 100 and a memory 102 .

The device shown in FIG. 1 corresponds to GO in Wi-Fi Direct.

In Wi-Fi Direct, one terminal among terminals forming a network group becomes a GO that starts and ends communication, and the GO functions as a wireless access device and controls the network group.

The GO allocates an IP address to a client connected to it, determines an awake period as described below, and transmits a video frame.

Referring to FIG. 1 , a processor 100 may include a central processing unit (CPU) capable of executing a computer program or other virtual machines.

Memory 102 may include a non-volatile storage device such as a fixed hard drive or a removable storage device. The removable storage device may include a compact flash unit, a USB memory stick, and the like. Memory 102 may also include volatile memory, such as various random access memories.

The memory 102 stores program instructions executable by the processor 100 .

The program instructions according to the present embodiment update the distribution of the x-frame size using previously received x-frame (x is one of I, P, and B) size information based on the expected value maximization algorithm, and the update is performed. The length of the awake period for the next x-frame is determined based on the distribution of the x-frame sizes, and the next x-frame is transmitted to the client based on the determined awake period.

Here, the update of the x-frame size distribution is defined as updating the parameters of the frame size distribution including the shape parameter, the scale parameter and the weight parameter of all blending components using an expected value maximization algorithm.

Further, the program instructions are configured to maximize a log-likelihood function for the shape parameter, the scale parameter, and the weight parameter of the blending component using an expected value maximization algorithm, and use the maximized log-likelihood function to determine the shape parameter, the scale parameter and obtain a new estimate of the weight parameter.

According to the present embodiment, the transmission efficiency of the video frame can be further improved by automatically reconstructing the probability density function of the video frame using an expected value maximization algorithm without fixing the probability density function.

Hereinafter, application of the expectation maximization algorithm to the gamma mixture model for analyzing the video frame size distribution will be described first.

Video codecs that support variable frame sizes generally classify video frames into three types: intra frames (I-frames), predictive frames (P-frames), and bi-directional predictive frames (B-frames).

Video frames may be encoded into sequences called Groups of Pictures (GoPs) to reduce spatial and temporal redundancy.

2 is a diagram illustrating the sequence of the GoP structure.

As shown in Fig. 2, video frames are transmitted in the order of, for example, I, B, B, P, B, B, ....

I-frames are frames that are not encoded without any motion compensation, and P-frames are encoded with reference to previous I-frames and P-frames.

Loss of I-frame results in loss of all frames in GoP, and loss of P-frame results in loss of subsequent P-frames and B-frames, as shown in FIGS. 2B and 2C .

On the other hand, as shown in FIG. 2D, the loss of the B-frame does not affect other frames.

Considering the power saving mode of Wi-Fi Direct, the number of awake sections of the beacon section is determined such that a one-to-one mapping relationship exists between each awake section and each video frame, that is, one awake section is allocated to one video frame do.

The awake period of the video frame is classified according to the frame type, and the awake period assigned to the I-frame, P-frame and B-frame is called as I-frame period, P-frame period, and B-frame period, respectively. .

According to this embodiment, the GO determines the length of the awake interval for the next x-frame based on the distribution of x-frame sizes updated by the previously received x-frames.

According to this embodiment, the distribution update of the x-frame size is done through an expectation maximization algorithm for the GAMMA MIXTURE MODEL (GMM), which is a parameter to search for a probabilistic model best suited to the observed data distribution. It is an iterative approach to find maximum-likelihood estimates of (eg shape parameters, scale parameters, weight parameters).

를 x-프레임 크기의 일련의 샘플이라 하고, 여기서,

는 x-프레임의 전체 개수, x는 I, P 및 B중 하나이다.

Let be a series of samples of x-frame size, where

is the total number of x-frames, and x is one of I, P and B.

The frame size data is considered to be a sample of independent and equally distributed (i,i,d.) random variables, and the GMM of the x-frame size is assumed that these variables are due to the contribution of the N distribution, given by .

는 x-프레임 크기

의 GMM 파라미터 벡터이고,

는 모양 파라미터(

) 및 스케일 파라미터(

)를 나타내며,

(

)는

인 가중치 파라미터이고, N은 혼합 컴포넌트의 수이다. 감마 분포의 확률밀도함수(PDF)는 다음과 같이 정의된다. here,

is the x-frame size

is the GMM parameter vector of

is the shape parameter (

) and scale parameters (

) represents,

(

) is

is a weight parameter, and N is the number of mixing components. The probability density function (PDF) of the gamma distribution is defined as follows.

는here

is

인 오일러 감마 함수이다.

is the Euler gamma function.

Then, the joint distribution of the x-frame size is

The expected value maximization algorithm for GMM consists of the following two steps, E-step and M-step.

가 모델에 도입될 때 기댓값 최대화 방법을 이용하여 파라미터(예를 들어, 가중치 파라미터, 모양 파라미터 및 스케일 파라미터)에 관한 로그-우도(log-likelihood) 함수를 최대화하기 위해 기댓값 최대화 방법을 사용한다. In step E, the hidden discrete

When is introduced into the model, the expected maximization method is used to maximize the log-likelihood function for parameters (eg, weight parameters, shape parameters, and scale parameters) using the expected value maximization method.

는 샘플

가 유형 j의 분포에 속하는 것을 의미한다.

is the sample

means that it belongs to the distribution of type j.

The expected value of log likelihood is defined as

는 로그 우도 함수이고,

는 k번째 반복에서 GMM 파라미터의 벡터이다. here,

is the log-likelihood function,

is the vector of GMM parameters at the kth iteration.

의 기댓값을 최대화함에 의해 파라미터

의 새로운 추정치가 얻어진다. In M step, log-likelihood function

parameter by maximizing the expected value of

A new estimate of is obtained.

A new estimate of the GMM parameter is obtained as follows.

는

로 정의되고,

를 설명하는 분포 유형 j의 책임값(responsibility)은 다음과 같이 정의될 수 있다. Here, the digamma function

is

is defined as

The responsibility of distribution type j that describes

FIG. 4 shows the performance of the expected value maximization algorithm in the GMM, and shows that the optimal parameters of the GMM are obtained when Algorithm 1 stops iteration.

Hereinafter, an algorithm for determining the length of the awake interval for the x-frame will be described in detail.

는 x-프레임에 대한 어웨이크 구간의 길이,

는

에 전송될 수 있는 x-프레임 크기이다.

is the length of the awake interval for the x-frame,

is

x-frame size that can be transmitted in .

를 x-프레임 크기 분포의 평균 및 표준 편차의 함수로 결정한다. This example is as follows:

is determined as a function of the mean and standard deviation of the x-frame size distribution.

는

의 값을 제어하는 스케일 팩터를 나타낸다. here,

is

Represents the scale factor that controls the value of .

The average of the x-frame sizes is

The second moment of the x-frame size is calculated as

이다. here,

to be.

From equations (10) and (11), the variance of the x-frame size distribution is obtained as follows.

Then, the x-frame size can be calculated by

은 비트율이다. here,

is the bit rate.

보다 클 때 x-프레임 전체가

동안 전송될 수 없으며, 어웨이크 구간이 프레임 크기에 비해 작기 때문에 패킷 분할 또는 프레임 드랍이 이루어져야만 한다. x-frame size

When greater than the entire x-frame

Since the awake period is small compared to the frame size, packet segmentation or frame drop must be performed.

동안 완전히 전송될 수 있는 목표 확률(target probability)을 계산한다. Therefore, in this embodiment, the x-frame is

Calculate the target probability that can be completely transmitted during

동안 전체적으로 전송될 수 없음을 의미한다. For example, if the target probability is set to 95%, 95% of x-frames can be transmitted entirely without packet fragmentation or frame drops, and 5%

It means that it cannot be transmitted as a whole during

The target probability is given by

는 불완전 감마 함수이다.here,

is an incomplete gamma function.

Hereinafter, the priority-based frame transmission process will be described in detail.

As mentioned above, the loss of an I-frame results in the loss of all subsequent frames of the GoP, and the loss of a P-frame affects the next P- and B-frames, whereas the loss of a B-frame causes the loss of the next frame. does not affect

Therefore, the priority should be determined in the order of I-frames, P-frames and B-frames.

According to the present embodiment, a method for handling the rest of the frame varies according to priority.

4 is a diagram for explaining a process in which residual I-frames and P-frames are transmitted in the next B-frame period.

For an I-frame with the highest priority, the rest of the I-frame (P-frame) is connected to the immediately following B1-frame (B3-frame) to increase the probability of successful transmission of the I-frame (P-frame) and transmitted together with the B1-frame (B3-frame) during the B1-frame (B3-frame) period.

From the above procedure, the frame to be transmitted during the B1-frame (B3-frame) section is the sum of the size of the remaining portion of the I-frame and the B1-frame (B3-frame).

Therefore, in order to determine the length of the B1-frame section, not only the distribution of the B1-frame size is used, but a new distribution function for the frame size connected thereto is required.

를 Y-프레임의 구간 동안 전송될 수 없는 Y-프레임의 나머지 부분의 크기를 나타내는 변수이고,

는 Y프레임의 크기를 나타내며, Y는 I, P 중 하나이다.

is a variable indicating the size of the remaining portion of the Y-frame that cannot be transmitted during the period of the Y-frame,

represents the size of the Y frame, and Y is one of I and P.

는 다음과 같이 주어진다.

is given as

은 연속적인 랜덤 변수 및 이산적인 랜덤 변수의 조합이다. here,

is a combination of a continuous random variable and a discrete random variable.

이면, 잔여 Y-프레임 사이즈의 분포는 다음과 같이 계산된다. if

, the distribution of the residual Y-frame size is calculated as follows.

이면 다음과 같다. if,

If so:

이면 다음과 같다. if,

If so:

의 확률밀도함수는 다음과 같이 얻어진다. From Equations 16 to 18

The probability density function of is obtained as follows.

는 Dirac-Delta 함수이다.here,

is a Dirac-Delta function.

는 잔여 Y-프레임 및 B-프레임의 크기의 합을 나타내는 랜덤 변수이다.

is a random variable representing the sum of the sizes of the remaining Y-frames and B-frames.

의 확률밀도함수는

와

의 확률밀도함수의 합성곱이 되고, 다음과 같다. here,

The probability density function of

Wow

is the convolution of the probability density function of

이고,

는 퇴화 초기하 함수(degenerate hypergeometric function)이다. here,

ego,

is a degenerate hypergeometric function.

The average of the remaining Y-frames is calculated as follows.

는 Whittaker 함수이다. here,

is a Whitaker function.

The second moment of the remaining Y-frame is calculated as follows.

From Equations (10) and (21), the average of the sum of the remaining Y-frames and B-frames is calculated as follows.

는 다음과 같이 표현된다. Finally

is expressed as

의 닫힌 형태(closed form)는 수학식 12, 21 및 22를 조합함으로서 얻어진다. 따라서, 잔여 Y-프레임 및 B-프레임의 합에 대한 어웨이크 구간의 길이는 수학식 9를 사용하여 계산된다.

The closed form of is obtained by combining equations (12), (21) and (22). Accordingly, the length of the awake interval for the sum of the remaining Y-frames and B-frames is calculated using Equation (9).

Transmission delay and energy consumption are described below.

보다 크면 잔여 Y-프레임은 다음 B-프레임까지 기다려야만 한다. 이러한 대기 시간을 전송 지연으로 정의한다. x-frame size

If greater than, the remaining Y-frames must wait until the next B-frame. This waiting time is defined as the transmission delay.

는 총 Y-프레임의 수이다. T is the interval between frames of the GoP structure,

is the total number of Y-frames.

이 되고, Y-프레임의 구간에서 전체적으로 전송될 수 없는 Y-프레임의 수는

이다. Then, the transmission time of the oversized Y-frame is

, and the number of Y-frames that cannot be transmitted as a whole in the period of the Y-frame is

to be.

At Y=I or P, the average transmission delay of I- and P-frames is defined as follows.

는 I- 및 P-프레임의 총 수이다. here,

is the total number of I- and P-frames.

및

는 수학식 14에 의해 계산될 수 있다. percentage

and

can be calculated by Equation 14.

The average energy consumption is defined as the sum of the two different energies consumed during the awake period and the sleep period.

및

를 각각 어웨이크 구간 및 슬립 구간 동안 소비되는 평균 에너지라 할 때

는 다음에 의해 계산된다.

and

When is the average energy consumed during the awake section and the sleep section, respectively.

is calculated by

는 어웨이크 구간 동안의 전송 전력이고,

는 주어진 시간 구간마다의 x-프레임의 수이다. here,

is the transmit power during the awake period,

is the number of x-frames per given time interval.

은 프레임의 총 개수이다.

is the total number of frames.

는 다음과 같다.

is as follows

은 슬립 구간 동안 소비되는 전력이다. here,

is the power consumed during the sleep period.

From Equations 26 and 27, the average energy consumption per frame can be obtained as follows.

는 슬립 모드에서 어웨이크 모드로 전환되는데에 추가적으로 소비되는 에너지이다. here,

is the energy consumed additionally to switch from the sleep mode to the awake mode.

In the NoA technique, the lengths of the awake interval and the sleep interval are fixed.

는 NoA 기법에서 어웨이크 구간의 길이이고, 그러면 슬립 구간은

가 된다.

is the length of the awake interval in the NoA technique, then the sleep interval is

becomes

동안 전체적으로 전송될 수 있는 확률은 다음과 같이 결정될 수 있다. From Equation 14, the Y-frame is

The probability that can be transmitted as a whole during the period may be determined as follows.

이다. here,

to be.

From Equation 25, the average transmission delay in NoA is as follows.

Finally, energy consumption can be calculated as

The above-described embodiments of the present invention have been disclosed for the purpose of illustration, and various modifications, changes, and additions will be possible within the spirit and scope of the present invention by those skilled in the art having ordinary knowledge of the present invention, and such modifications, changes and additions should be regarded as belonging to the following claims.

Claims (9)

As a video frame transmission device based on the expected value maximization algorithm in Wi-Fi Direct,
processor; and
a memory coupled to the processor;
The memory is
update the distribution of the x-frame size using previously received x-frame (x is one of I, P, and B) size information based on the expected value maximization algorithm;
Determine the length of the awake interval for the next x-frame based on the updated distribution of the size of the x-frame,
Based on the determined awake period, the next x-frame is transmitted to the client,
A video frame transmission device for storing program instructions executable by the processor. According to claim 1,
The program instructions are
A video frame transmission apparatus for updating a parameter of a frame size distribution including a shape parameter, a scale parameter, and a weight parameter of all mixed components by using the expected value maximization algorithm. According to claim 1,
The program instructions are
Maximize a log-likelihood function for the shape parameter, the scale parameter, and the weight parameter of the blending component using the expected value maximization algorithm, and use the maximized log-likelihood function to determine the shape parameter, the scale parameter, and the weight parameter. A video frame transmission device that obtains a new estimate. According to claim 1,
The update is for automatically reconstructing a probability density function of a video frame. A method for transmitting video frames based on an expected value maximization algorithm in Wi-Fi Direct, comprising:
(a) reconstructing a probability density function of an x-frame (x is one of I, P, and B) based on an expected value maximization algorithm;
(b) determining the length of the awake interval for the next x-frame using the reconstructed probability density function; and
(c) transmitting the next x-frame to the client based on the determined awake period. 6. The method of claim 5,
In the step (a), a video frame of updating a parameter related to the distribution of the x-frame size using previously received x-frame (x is one of I, P, and B) size information based on an expected value maximization algorithm transmission method. 7. The method of claim 6,
wherein the parameter is a parameter of a frame size distribution including a shape parameter, a scale parameter and a weight parameter of all blending components. 8. The method of claim 7,
The step (a) is,
maximizing a log-likelihood function for the shape parameter, the scale parameter, and the weight parameter of the blending component using the expected maximization algorithm; and
and obtaining new estimates of the shape parameter, the scale parameter and the weight parameter via the maximized log-likelihood function. A computer readable program stored in a recording medium for performing the method according to claim 5 .

Images