CN106303562A - The multi-view point video adaptive transmitted control algorithm controlled based on PI - Google Patents

Abstract

The present invention relates to a kind of MPEG DASH multi-view point video adaptive algorithm controlled based on PI, including: enter and initially cache the stage: PI controller is set；Calculate and estimate handling capacity；Calculate caching；Calculate the side-play amount of caching；The input of system control variables；Output control variable according to PI controller and step 6 calculated estimation handling capacity, select standard by controlling computing unit output i-th bit stream video segment；Bit stream selects, and selects objective indicator value in the video segment that these are selected, i.e. the multiple views bit stream video segment that SSIM value is bigger is transmitted.The present invention is possible to prevent the video playback that cache overflow causes to stagnate or the problem such as bandwidth availability ratio is low.

Description

The multi-view point video adaptive transmitted control algorithm controlled based on PI

Technical field

The invention belongs to network multi-view point video Adaptive Transmission field, be more particularly applied in 3D environment, a kind of The multi-view point video adaptive transmitted control algorithm controlled based on PI.

Background technology

Along with the Internet and the fast development of each intelligent terminal, people are for the multimedia messages such as video, audio frequency Demand is increasing, in order to solve the problems such as compatibility between each intelligent terminal and suitability, dynamic image expert group (MPEG) proposition dynamic self-adapting streaming media based on HTTP is combined with third generation partner program (3GPP) in 2011 (MPEG-DASH) agreement, this agreement defines media description file (MPD) and the form of media file.

Streaming media based on HTTP dynamic self-adapting (DASH) host-host protocol mechanism, is to be driven by client, is regarding Keep pouring in defeated before, same video resource can be divided into different video segments by server, and carries out different bit rates coding, finally Generating a media description file based on binary system extensible markup language (XML) form, it contains the ratio of video segment The corresponding Video attribute information such as special rate, resolution, time interval.Client is according to current network condition, own hardware The URL (URL) that the relevant parameter such as disposal ability and caching is provided according to media description file is adaptive Select be adapted to current network conditions under video resource carry out applying for and downloading.

However as image and the development of camera work, 3D video is deeply subject to each with user experience quality three-dimensional, real The parent of product operator and researcher looks at.In ITU-T and ISO combines the video encoding standard of future generation (HEVC) of formulation, By 3D-HEVC as the extension of HEVC, double relative to H264/AVC, HEVC compression scheme efficiency.In 3D-HEVC Using multiple views plus depth (MVD) as the compressed format of data, encode a depth information for each viewpoint, for describe be In photographed scene, object is from the distance of video camera.Multiple views plus depth video flowing after coding can be sent to client, client Virtual view can be synthesized according to depth information by deep image rendering (DIBR), and then complete the reconstruction of 3D video.

A lot of research about 2D-DASH adaptation scheme has been had to can be largely classified into two classes at present: based on handling capacity Self adaptation and based on caching self adaptation.In comparison, 3D-DASH adaptation scheme is the most ripe.Current 3D-DASH's The impact of the number of views study the form improvement being concentrated mainly on media description file, the transmitting video quality on finally presenting With, transmission number of views self adaptation switching.But due to multi-vision-point encoding, in 3D environment, client can produce More there is the multiple views bit stream version of different bit rates.Therefore adaptation scheme based on 3D-DASH is wanted relative to 2D Ask stricter.

Summary of the invention

Solved by the invention technical problem is that: under wireless network environment, it is provided that a kind of by client drive based on The MPEG-DASH multi-view point video adaptive algorithm that PI controls, to meet: under existing wireless network environment, it is ensured that application Video quality immediately following the change of the network bandwidth, stagnate with the video playback preventing cache overflow from causing or bandwidth availability ratio be low etc. Problem；Under limited network bandwidth environments, maximize the video quality of user's application, to maximize user experience quality；? When the network bandwidth occurs fluctuation, the video quality selected by maintenance is stable as far as possible, it is to avoid cause video quality to be frequently switched to use Sense tired out is brought at family；Technical scheme is as follows:

A kind of MPEG-DASH multi-view point video adaptive algorithm controlled based on PI, this algorithm uses PI controller: will be slow Deposit the departure degree relative to target cache as error feedback signal, exported multiple views bit stream by feedback signal dynamically Section selection standard, comprises the following steps:

Step 1: resource preparation: before multi-view point video carries out Adaptive Transmission, server end carries out multiple views bit stream Resource preparation, by each position capture to video flowing carry out independent symmetrical coding, and regard at the same time Frequency section cutting；

Step 2: media description file generated: multiple vision point video stream is packed, and by structural similarity index S SIM The objective reference frame switched as the self adaptation arranging multiple vision point video stream, collects the attribute information of multi-view point video Produce media description file；

Step 3: media description file pre-download: before video is downloaded, client is first filed an application to server, and downloads Media description file, understands the code check resource allocation conditions of server end by resolving media description file；

Step 4: enter the stage that initially caches: client is minimum the most to server application viewpoint number and gross bit rate Multi-view bitstream video segment, when the video segment of buffer area reaches predetermined value, performs step 5, starts Adaptive Transmission；

Step 5: the Proportional coefficient K of PI controller is set_pWith integration time constant K_i；

Step 6: calculate and estimate handling capacity: according to a upper video segment speed of download, calculates network throughput, and will meter The network throughput obtained is as network-evaluated handling capacity T during next video segment download_e(i)；

Step 7: calculate caching: according to application the gross bit rate of multiple views bit stream video segment, network throughput with And a upper video segment downloaded the caching surplus being, calculate caching surplus B after i-th video segment has been downloaded (i)；

Step 8: calculate the side-play amount of caching: target cache B is set_opt, by calculated for the step 7 same mesh of caching surplus Mark caching contrast, and calculate both differences as current caching side-play amount B_offet:

B_offset=B (i)-B_opt

Step 9: the input of system variable: using calculated for step 8 caching side-play amount as the input of PI controller, enter Enter the computing unit of PI controller, output control variable u (i):

$u\left(i\right)=K_p\left(B\right(i)-B_{opt})+K_i\underset{\mathrm{\τ}=0}{\overset{i}{\Σ}}\left(B\right(\mathrm{\τ})-B_{opt})$

Step 10: calculate multiple views bit stream section selection standard: according to output control variable and the step of PI controller Rapid 6 calculated estimation handling capacities, by controlling computing unit output i-th bit stream video segment selection standard:

$\tilde{v}\left(i\right)=(1+u(i\left)\right)\×T_e\left(i\right)$

Step 11: bit stream selects: according to the multiple views bit stream section selection standard of step 10 outputAt each Bit stream selects the gross bit rate video segment closest to this standard, and in the video segment that these are selected, selects visitor See desired value, i.e. the multiple views bit stream video segment that SSIM value is bigger is transmitted；

Step 12: video is downloaded；According to the done decision-making of step 11 to next video segment of server end application, wait, directly Download to this video segment；

Step 13: repeat step 6～12, until having downloaded all sections of cost video.

Accompanying drawing explanation

Fig. 1 is server system figure of the present invention

Fig. 2 is FTP client FTP figure of the present invention

Fig. 3 is Adaptive Transmission based on DASH mechanism

Fig. 4 is the adaptive algorithm framework controlled based on PI

Fig. 5 is the logic diagram that the present invention is embodied as

Specific implementation method

For making purpose of the present invention technical scheme and advantage clearer, below in conjunction with the accompanying drawings and specific embodiments, right The present invention is further described in detail.

Step 1: before video Adaptive Transmission, server end carries out the resource preparation of multiple views bit stream, such as Fig. 1 service The system block diagram of device, specific embodiments of the present invention use 3 video cameras with identical parallax range annular array, depending on Frequency catches, and uses the 3D-HEVC standard video flowing to capturing to encode with carrying out independent symmetrical, and each viewpoint is encoded into L Individual bit rate playback version (v₁,v₂,...,v_L).Each viewpoint video is cut into N number of regarding by (8 seconds) at the same time Frequency section.Packing two kinds of bit streams of synthesis: two viewpoint plus depth (2VD) and three viewpoint plus depth (3VD).

Step 2: media description file generated.The multiple views bit stream (2VD, 3VD) obtained according to step 1, compares each Average structure similarity indices (SSIM) value of the virtual view that special stream renders in client, as the switching of multiple views bit stream Objective basis with the actual experience being close to the users.The bit rate of server end bit stream section, number of views, SSIM the most at last The attribute information such as value and URL is aggregated in media description file.

Step 3: media description file pre-download.Adaptive Transmission as based on DASH in Fig. 3 mechanism, client is in advance to clothes Business device sends HTTP GET request downloads of media and describes file medium description.And know server by resolving media description file The attribute informations such as the end bit rate of bit stream, number of views, SSIM value and URL.

Step 4: initially cache the stage.The caching incipient stage, client is according to the service obtained from media description file The attribute information of device end video, to server application the lowest class quality video and the bit stream of viewpoint number minimum (2VD), until Caching reaches preset value, starts client Adaptive Transmission mechanism.

Step 5: design PI controller.Fig. 4 is the adaptive algorithm framework controlled based on PI.The present invention uses P (ratio) Controller and I (integration) controller, arrange the Proportional coefficient K of P controller_pIntegration time constant K of=0.1 and I controller_i= 0.01.Using the departure degree of target cache as error feedback signal.

Step 6: calculate and estimate handling capacity.Adaptive customer end as shown in Figure 2, if from application to i-th video segment Download completes required time t, and the gross bit rate of application video flowing is v, and every section of video segment playing duration is τ, then download i-th During video segment, network throughput is expressed as: T_i(i)=v × τ/t.Use this handling capacity as carrying out next video segment Estimation handling capacity during download, i.e. T_e(i+1)=T_i(i)。

Step 7: calculate caching.If caching surplus when previous video section has been downloaded is B (i-1), according to step 6 T and τ obtained, then current cache calculation is: B (i)=B (i-1)+τ-t

Step 8: calculate side-play amount B of caching_offset.Target cache is set to B_opt, by calculated for step 7 caching Surplus is with target cache B_optContrast, and calculate both differences as caching side-play amount B_offset=B (i)-B_opt。

Step 9: the input of system variable.By calculated for step 8 caching side-play amount B_offsetDefeated as PI controller Enter, enter the computing unit of PI controller.PI controller output control variable u (i):

$u\left(i\right)=K_p\left(B\right(i)-B_{opt})+K_i\underset{\mathrm{\τ}=0}{\overset{i}{\Σ}}\left(B\right(\mathrm{\τ})-B_{opt})$

Step 10: calculate multiple views bit stream section selection standardExported by PI controller and step 6 calculates Estimation handling capacity T arrived_e, control computing unit and calculate multiple views bit stream section selection standard

Step 11: bit stream selects.First according to the bit stream section selection standard of step 10 outputRespectively at 2VD (two viewpoint plus depth) and 3VD (three viewpoint plus depth) both bit streams select gross bit rate closestVideo cut Sheet, then selects the video segment that SSIM value is bigger to be transmitted in two bit streams.

Step 12: video is downloaded.According to the video segment selected by step 11, provided by media description file URL downloads this video segment to server application, waits until that this video segment has been downloaded.

Step 13: repeat step 6～12, until having downloaded all sections of cost video.Fig. 5 is that the present invention is embodied as Logic diagram.

The present invention uses above technical scheme, demonstrates advantages below:

(1) in 3D-DASH applies, fully take into account the caching impact for user, and then ensure that client will not go out The overflow now cached and underflow.

(2) provide a user with the higher multi-view point video of quality, and ensure that video switches more smooth steady, improve user Quality of experience.

(3) control to be applied in 3D environment by PI, can preferably ensure that the buffered video of user is permissible under 3D environment Stable at a safety value, ensure farthest to utilize bandwidth simultaneously.

Claims (1)

1. the MPEG-DASH multi-view point video adaptive algorithm controlled based on PI, this algorithm uses PI controller: will caching Relative to the departure degree of target cache as error feedback signal, exported multiple views bit stream by feedback signal dynamically and cut Sheet selects standard, comprises the following steps:

Step 1: resource preparation: before multi-view point video carries out Adaptive Transmission, server end carries out the money of multiple views bit stream Source is prepared, by each position capture to video flowing carry out independent symmetrical coding, and carry out video at the same time and cut Sheet cuts；

Step 2: media description file generated: multiple vision point video stream is packed, and using structural similarity index S SIM as The objective reference frame of the self adaptation switching of multiple vision point video stream is set, carries out collecting production by the attribute information of multi-view point video Media description file；

Step 3: media description file pre-download: before video is downloaded, client is first filed an application to server, and downloads of media Describe file, understand the code check resource allocation conditions of server end by resolving media description file；

Step 4: enter the stage that initially caches: client is first to the multiple views that server application viewpoint number and gross bit rate are minimum Bit stream video segment, when the video segment of buffer area reaches predetermined value, performs step 5, starts Adaptive Transmission；

Step 5: the Proportional coefficient K of PI controller is set_pWith integration time constant K_i；

Step 6: calculate and estimate handling capacity: according to a upper video segment speed of download, calculate network throughput, and will calculate The network throughput arrived is as network-evaluated handling capacity T during next video segment download_e(i)；

Step 7: calculate caching: according to application the gross bit rate of multiple views bit stream video segment, network throughput and on One video segment has downloaded the caching surplus being, calculates caching surplus B (i) after i-th video segment has been downloaded；

Step 8: calculate the side-play amount of caching: target cache B is set_opt, calculated for step 7 caching surplus is delayed with target Deposit contrast, and calculate both differences as current caching side-play amount B_offet:

B_offset=B (i)-B_opt

Step 9: the input of system variable: using calculated for step 8 caching side-play amount as the input of PI controller, enter PI The computing unit of controller, output control variable u (i):

$u\left(i\right)=K_p(B\left(i\right)-B_{opt})+K_i\underset{\mathrm{\τ}=0}{\overset{i}{\Σ}}(B\left(\mathrm{\τ}\right)-B_{opt})$

Step 10: calculate multiple views bit stream section selection standard: output control variable and step 6 according to PI controller are counted The estimation handling capacity obtained, by controlling computing unit output i-th bit stream video segment selection standard:

$\tilde{v}\left(i\right)=(1+u(i))\×T_e\left(i\right)$

Step 11: bit stream selects: according to the multiple views bit stream section selection standard of step 10 outputAt each bit Stream selects the gross bit rate video segment closest to this standard, and in the video segment that these are selected, selects objective finger Scale value, i.e. the multiple views bit stream video segment that SSIM value is bigger is transmitted；

Step 12: video is downloaded；According to the done decision-making of step 11 to next video segment of server end application, wait until this Video segment has been downloaded；

Step 13: repeat step 6～12, until having downloaded all sections of cost video.