CN109874019A - A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest - Google Patents
A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest Download PDFInfo
- Publication number
- CN109874019A CN109874019A CN201910232071.7A CN201910232071A CN109874019A CN 109874019 A CN109874019 A CN 109874019A CN 201910232071 A CN201910232071 A CN 201910232071A CN 109874019 A CN109874019 A CN 109874019A
- Authority
- CN
- China
- Prior art keywords
- block
- roi
- video
- interest
- panoramic video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention belongs to panoramic video technical fields, disclose a kind of method of unequal error protection auxiliary wireless panoramic video of region-of-interest.The wireless transmission that unequal error protection (ROI-UEP) is used for efficient video code (HEVC) sequence is perceived by area-of-interest；It will be that ROI signal find most suitable not equal forward error corrections (FEC) encoding rate by minimizing expected video distortion；It is used to assess the quality for the panoramic video rebuild by weighting Y-PSNR (WPSNR), wherein considering the weight of pixel to calculate the distortion as caused by related pixel.Simulation result shows equal error protections (ROI-EEP) scheme based on ROI better than EEP, channel SNR 5dB, WPSNR 40dB, and it is 9.4dB when channel SNR is 6dB that ROI-UEP scheme, which further increases its ROI-EEP counterpart by WPSNR,.
Description
Technical field
The invention belongs to panoramic video technical field more particularly to a kind of unequal error protection auxiliary region-of-interest are wireless
The method of panoramic video.
Background technique
Currently, the immediate prior art: panoramic video, having perfect immersion, remotely existing panoramic video is considered
It is recent video format, it is complete based on field programmable gate array (FPGA) because it can project 360 degree of given scenario
Scape video generating system.A kind of mapping scheme indicating panoramic video, to improve visual quality, stability and compression efficiency.Table
Personalized transport scheme is levied, wherein controlling in spheric region according to the specific possibility to be watched during active user interaction
Quality processed.
For navigating between cube domain in video-see mode, this discontinuity eliminated between cube dignity exists
Tiling is applied in real-time interactive panoramic video system, the specific region for gradually paying close attention to increased quality to observer, simultaneously
Bandwidth demand full resolution panorama system is reduced to be reduced to cost relative to the slight experience (QoE) observed.In order to full
The requirement of low latency and large-scale data transmission, proposes one kind in sufficient augmented reality (AR) and virtual reality (VR) application
Software defined network framework for 5G small-sized honeycomb network.Scalable video (SVC) principle is compiled applied to panoramic video
Code analyzes two stages cooperation VR transmission plan to reduce required bandwidth.
Unequal error protection；In order to improve Fault recovery, differently protection has the video bits of different importance
It is intuitive.Obviously, unequal error protection (UEP) is initially proposed by Masnick and Wolf, wherein stronger forward error correction
(FEC) it is assigned to prior bit rather than less important bit.Four class UEP technologies have been examined in, that is, are based on UEP
Transceiver scheme, grouping regression arrangement, bit-level FEC scheme and cross-layer operate assisted solution.Herein, will pay attention to
Power concentrates in the series of bit-level FEC scheme.
The prior art passes through the turbo anisotropically distributed in the successive video frames by H.263 Video Codec coding
The code redundancy that is applied minimizes video distortion.UEP assists low-density checksum (LDPC) encoding scheme to carry out middle
Research.The prior art has studied the H.264/AVC video flowing of the data subregion based on UEP using RSC code, and has evaluated in
UEP assists turbo coded modulation.The prior art considers importance and the view such as or not successive video frames in image group (GOP)
The unequal importance of different macro blocks (MB) in frequency frame.Only with three kinds of protection class in however, which has limited accessible
Performance.There may be the fact that different error probabilities by side information (SI) value in the different location of Wyner-Ziv (WZ) frame
Promotion, in order to reduce required bit rate under the background of distributed video coding (DVC), anisotropically by FEC redundancy point
These SI values of dispensing.UEP is applied to the wireless communication of the following hologram image.
The prior art proposes the so-called bit-level interlayer FEC (IL- of the layering wireless video dependent on soft decoding FEC
FEC), wherein the systematic bits (BL) of base are implanted in the enhancement layer (EL) of transmitter.At receiver, it can be advantageous to benefit
Improve the Fault recovery of EL with the implantation bit of BL.In above-mentioned IL-FEC technology, it is investigated UEP principle, with into one
Step improves system performance.Author develops a kind of algorithm, for " instant " encoding rate for searching optimization, optimizes IL-FEC coding
The performance of system.The three-dimensional video-frequency dependent on depth bitmap-format is had studied, for being communicated in noisy communication channel, wherein to face
Color and depth quantization parameter apply different FEC encoding rates.The so-called layered reuse scheme based on power is had studied, as logical
It crosses and provides the means of UEP in digital terrestrial television (DTT) system using information-theoretic approach.A kind of cross-layer behaviour is contemplated in
Make auxiliary UEP scheme, the handheld device for low complex degree.Song et al. has studied various multi-layer videos and indicates, such as can stretch
Contracting video and simulcast streams, and a kind of UEP scheme is proposed, it is improved for being realized between video storage and rear enhancing cost
Compromise.
In conclusion problem of the existing technology is: existing panoramic video is of poor quality, is easy distortion.In panoramic video
In transmission process, under certain transmission quality, video quality is bad.
Solve the difficulty of above-mentioned technical problem:
Wireless panoramic video transmission is being carried out, how to accomplish the energy consumption that not only can be reduced transmission, but also terminal view can be improved
Frequency quality.
Solve the meaning of above-mentioned technical problem:
Family, which can be used, has better experience to panoramic video, while reducing transmission energy.
Summary of the invention
In view of the problems of the existing technology, the present invention provides a kind of unequal error protection auxiliary region-of-interest is wireless
The method of panoramic video.
The invention is realized in this way a kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest,
It is described inequality error protection auxiliary the wireless panoramic video of region-of-interest method the following steps are included:
Step 1 perceives inequality error protection ROI-UEP by area-of-interest and is used for efficient video code HEVC sequence
The wireless transmission of column；
Step 2 will be that ROI signal find most suitable not equal forward error corrections by minimizing expected video distortion
FEC encoding rate；
Step 3 is used to assess the quality for the panoramic video rebuild by weighting Y-PSNR WPSNR.
Further, the method for the unequal error protection auxiliary wireless panoramic video of region-of-interest obtains sphere and corresponding
Plane coordinates method include:
(1) spherical coordinates is converted into plane coordinates
Point A is the arbitrary point on spherical surface, wherein αo(-π≤αo≤ π) andIt is the longitude of A
With latitude coordinate (xa, ya, za), longitude αoIt is calculated as；
The wherein radius of R expression sphere, and latitude αaIt is expressed as；
Using the longitude and latitude derived, easily the projected position of the point A on plane coordinates can be expressed as
(wa, ha)；
wa=αo·R
ha=αa·R；
The width and height for projecting frame size are W=2 π R, H=π R；
It (2) is spherical coordinate by two-dimensional assemblage, point A is with coordinate (wa, ha) planar rectangular on arbitrary point,
Wherein；
With
αo(-π≤αo≤ π) andIt is the longitude and latitude of A on estimated sphere；Longitude αo
It is calculated as；
Latitude αaIt indicates are as follows:
Using the longitude and latitude derived, the projected position of the point A in spherical coordinate is expressed as (xa, ya, zx), have；
xa=Rcos αa·cos αo
ya=Rcos αa·sin αo
za=sin αa·R。
Further, the method for the unequal error protection auxiliary wireless panoramic video of region-of-interest is encoded based on HEVC
The ROI-UEP framework of wireless channel panoramic video stream includes:
1) receiver module: panoramic frame U is divided into block u1..., un, it is indicated with yuv format and by HEVC encoder compresses,
Generate bit stream b1..., bn.Meanwhile by block b1..., bnInformation input to minimize be distorted block, block is separately optimized
b1..., bnEncoding rate γ1..., γn.Then, the n bit stream b obtained1..., bnIt will encode as follows:
N-bit sequence b1..., bnBy the FEC encoder 1 of Fig. 5 ... n coding, wherein being respectively adopted by " minimizing and losing
Very " the encoding rate γ of block generation1... γn, lead to the bit sequence x being separately encoded1..., xn；
Then by bit sequence x1, xnIt connects into the single bit stream being used for transmission and then applies interleaver Δ prominent to be randomized
Send out error of transmission；
The joint bit sequence of intertexture is sent eventually by antenna；
2) receiver model: received panorama signal wireless receiver processing is as follows:
After the demodulation, sequence x1..., xnSoft Inform ation, i.e. y1..., yn, then by deinterleaver π-1It generates；
Soft Inform ation yiIt is decoded by fec decoder device i, generates the bit sequence of estimationIts expression layer biEstimation version；
Finally, the bit stream of estimationThen it is decoded by HEVC decoder, generates YUV blockIt is most
It is combined eventually to rebuild panoramic video frame
Further, the method for the unequal error protection auxiliary wireless panoramic video of region-of-interest is when sending SNR is S
Γ editor-in-chief's code rate, the encoding rate γ of generation1..., γnMeet the following conditions.
When using γ1..., γnWhen with the encoding rate of SNR=, according to Y-PSNR caused by n erroneous block (PSNR)
Deteriorate D (γ1..., γn) quantitation video distortion, target is export encoding rate γ1..., γn, minimize expected degradation D
(γ1..., γn)；Block bi is used；
d(bi): it indicates by block biDistortion caused by damage；
P (s, | bi|, γi): use encoding rate γiWith the block b of radio transmitted power SiPER；
w(bi): block biWeight；
By block biDestruction caused by distortion be expressed asThe expection of panoramic frame
It is distorted D (γ1..., γn) indicate are as follows:
Objective function OF is expressed as；
Further, the FEC at the method receiver of the unequal error protection auxiliary wireless panoramic video of region-of-interest
Decoding process includes；
Condition limitation；
Eq (10) indicates to send the gross bit rate limit of n encoding block；
Expected video distortion D (γ1..., γn) be expressed as；
C. estimated weight w ().
Another object of the present invention is to provide a kind of application unequal error protection auxiliary region-of-interest is wirelessly complete
The unmanned plane of the method for scape video.
Advantages of the present invention and good effect are as follows: the present invention perceives unequal error protection (ROI- by area-of-interest
UEP) it is used for the wireless transmission of efficient video code (HEVC) sequence；It will be ROI signal by minimizing expected video distortion
Find most suitable not equal forward error corrections (FEC) encoding rate；It is used to assess the complete of reconstruction by weighting Y-PSNR (WPSNR)
The quality of scape video, wherein considering the weight of pixel to calculate the distortion as caused by related pixel.Simulation result shows to be based on
Equal error protections (ROI-EEP) scheme of ROI is better than EEP, channel SNR 5dB, WPSNR 40dB, and ROI-UEP scheme is logical
Crossing WPSNR and further increasing its ROI-EEP counterpart is 9.4dB when channel SNR is 6dB.
The present invention has obtained the part support of National Natural Science Foundation of China (NSFC) (No.61702335, No.61601304),
Part has obtained the scientific and technical innovation committee, Shenzhen (JCYJ20170302154149766, JCYJ20170302142545828)
Part is supported, 2016057 years subsidy .L of Shenzhen University have partially been obtained.Hanzo also to thank EPSRC project EP/P4284/1 and
The financial support of ERC senior researcher Quantcom.
The present invention is transmitted for wireless panoramic video, reduces the energy consumption of transmission, improves terminal video quality.
Detailed description of the invention
Fig. 1 is the method flow that the present invention implements the unequal error protection provided the auxiliary wireless panoramic video of region-of-interest
Figure.
Fig. 2 is that the present invention implements the sphere and corresponding plane coordinates figure that provide.
Fig. 3 is that the present invention implements the considerations of providing panoramic video stream conceptual scheme.
Fig. 4 is the 120 ο ROI figure that the central region v that the present invention implements to provide determines each frame.
Fig. 5 is the framework that the ROI-UEP proposed that the present invention implements to provide assists wireless panorama communication system, wherein according with
It number is to define in tablei." minimizing distortion " block will illustrate in Section IV section.
Fig. 6 is the fec decoder procedure chart at the receiver that present invention implementation provides.
Fig. 7 is that the present invention implements (the d γ of model -1.1 used provided-3+c·γ-2+ b γ+a), λ=1000,
The Mathematical Fitting surface chart of the template surface and p (s, λ, γ) of generation.
Fig. 8 be the present invention implement provide equally distributed point diagram is generated on sphere E.
Fig. 9 is that the present invention implements equally distributed point diagram on the sphere E provided.
Figure 10 is the distribution map that the present invention implements that the well-distributed points on the sphere E provided are mapped on rectangle.
Figure 11 is that the present invention implements view track used in the simulation provided, it includes 60 views of 60 panoramic frames
Centered Graphs.
Figure 12 is that panoramic frame is divided into 64 sub-blocks by the algorithm that the present invention implements to provide, and is discussed in Section IV-C
Algorithm calculation block weight map.
Figure 13 is that the present invention implements the proposition ROI-UEP scheme provided, ROI-EEP scheme, the EEP scheme of RaceVR sequence
WPSNR, BER compared with Eb/N0 for RaceVR sequence chart.
Figure 14 is that the present invention implements the ROI-UEP scheme proposed provided, ROI-EEP scheme, the EEP for the program
The reconstructed frame of scheme and compare RaceVR sequence.The first row and the second row respectively indicate plane and spherical frame, use E b/N 0
=6dB figure.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present invention
It is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to
Limit the present invention.
The present invention sends the scene of panoramic video by wireless channel.Motivation is that panoramic video viewer is past in particular moment
It is past interested in certain parts of panoramic frame.Therefore, intuitively only refresh and protect complete in area-of-interest (ROI)
The specific part of scape video frame.In addition, the different video signal in ROI there may also be different importance, therefore should adopt
With UEP to improve performance.For each panoramic frame, propose that ROI perception UEP (ROI-UEP) is used for efficient video coding (HEVC)
The wireless transmission of stream.Specifically, objective function (OF) is intended to minimize distortion.By depending on Lagrange's multiplier and newton
Down method optimizes the OF, determines one group of specific FEC encoding rate to minimize expected video distortion.
Application principle of the invention is explained in detail with reference to the accompanying drawing.
As shown in Figure 1, the present invention provides a kind of method of unequal error protection auxiliary wireless panoramic video of region-of-interest
The following steps are included:
Step S101 perceives unequal error protection (ROI-UEP) by area-of-interest and is used for efficient video code
(HEVC) wireless transmission of sequence；
Step S102 will be that ROI signal find most suitable not equal forward error corrections by minimizing expected video distortion
(FEC) encoding rate；
Step S103 is used to assess the quality for the panoramic video rebuild, wherein examining by weighting Y-PSNR (WPSNR)
The weight of pixel is considered to calculate the distortion as caused by related pixel.
ROI-UEP framework provided by the invention:
1) receiver module: at transmitter, the panoramic frame U considered is divided into block u1..., un, they are with yuv format
It indicates and by HEVC encoder compresses, generation bit stream b1..., bn.Meanwhile by block b1..., bnInformation input arrive Fig. 5's
" minimizing distortion " block, is separately optimized block b1..., bnEncoding rate γ1..., γn.Then, the n bit stream obtained
b1..., bnIt will encode as follows:
N-bit sequence b1..., bnBy the FEC encoder 1 of Fig. 5 ... n coding, wherein being respectively adopted by " minimizing and losing
Very " the encoding rate γ of block generation1... γn.This leads to the bit sequence x being separately encoded1..., xn。
Then by bit sequence x1, xnIt connects into the single bit stream being used for transmission and then applies interleaver Δ prominent to be randomized
Send out error of transmission.
The joint bit sequence of intertexture is sent eventually by antenna.
2) receiver model: received panorama signal is as follows by the wireless receiver processing of Fig. 5:
After the demodulation, sequence x1..., xnSoft Inform ation, i.e. y1..., yn, then will be by the deinterleaver π of Fig. 5-1It is raw
At.
Then, Soft Inform ation yiIt is decoded by the fec decoder device i of Fig. 5, generates the bit sequence of estimationIts expression layer bi's
Estimate version.
Finally, the bit stream of estimationThen it is decoded by the HEVC decoder of Fig. 5, generates YUV blockIt is finally combined to rebuild panoramic video frame
ROI-UEP encoding rate provided by the invention:
By exporting specific FEC encoding rate γ1..., γnTo minimize block b1..., bnVideo distortion.Based on table
The symbol definition of I characterizes algorithm by using the symbol in Table II.
When using γ1..., γnWhen with the encoding rate of SNR=, according to peak value noise caused by the n erroneous block considered
D (γ is deteriorated than (PSNR)1..., γn) carry out quantitation video distortion.In the proposed solution, target is export encoding rate
γ1..., γn, minimize expected degradation D (γ1..., γn).In order to derive expected frame distortion, to block biUsing
It is assumed hereinafter that
·d(bi): it indicates by block biDistortion caused by damage；
P (s, | bi|, γi): use encoding rate γiWith the block b of radio transmitted power SiPER；
·w(bi): block biWeight.
Then, by block biDestruction caused by distortion can be expressed asEntirely
The expected distortion D (γ of scape frame1..., γn) can indicate are as follows:
Therefore, objective function (OF) can be expressed as
Condition limitation
Eq (10) indicates to send the gross bit rate limit of n encoding block.
In Section IV-A and IV-B section, solve the problems, such as that the component of equation (9) each section, i.e. panoramic video are distorted d
(bi) and packet error rate (PER) p (S, | bi|, γi).Then, the solution i.e. minimum code rate of target equation has been found out.
Application principle of the invention is further described with reference to the accompanying drawing.
1, isometric projection
It can use the sequence that many technologies are multiple conventional two-dimensional cameras records and create panoramic video.Spherical panorama frame is such as
Shown in Fig. 2 a, and its plane panoramic frame is as shown in Figure 2 b.The present briefly rectangular projections (ERP), i.e. Geographical projections such as introduction that allow.
As shown in figure 3, the present invention implements the considerations of providing panoramic video stream scheme, obtains sphere and corresponding plane is sat
Mark.
A. spherical coordinates is converted into plane coordinates
Assuming that the point A of Fig. 2 a is the arbitrary point on spherical surface, wherein αo(-π≤αo≤ π) and
It is the Longitude and latitude coordinates (x of Aa, ya, za), longitude αoIt may be calculated
The wherein radius of R expression sphere, and latitude αaIt can be expressed as
Using the longitude and latitude derived, easily the projected position of the point A on plane coordinates can be expressed as
(wa, ha), there
Correspondingly, it can derive the width of projection frame size and height is W=2 π R, H=π R.
It B. is spherical coordinate by two-dimensional assemblage
Assuming that the point A of Fig. 2 b is with coordinate (wa, ha) planar rectangular on arbitrary point, wherein
With
αo(-π≤αo≤ π) andIt is the longitude and latitude of A on estimated sphere.Longitude αo
It may be calculated,
And latitude αaIt can indicate are as follows:
Using the longitude and latitude derived, easily the projected position of the point A in spherical coordinate can be expressed as
(xa, ya, za), have
2, architecture
In Section III-A section, the panoramic video stream scene considered briefly is introduced first.Then it describes in detail and is based on
The ROI-UEP scheme of the wireless channel panoramic video stream of HEVC coding, as shown in Figure 5.
A. panoramic video stream scene
In the panoramic video stream/broadcasting scenario considered in Fig. 3, the video server of such as Youtube is by original panoramic
Video is sent to multiple panoramic video playback terminals.Playback terminal is only able to display the ROI picture fallen within the scope of 120 ° of observation center
Element, and remaining region is then ignored.Therefore, in order to save Internet resources, such as transimission power and/or bandwidth, people wish naturally
The only ROI signal of transmission compression, rather than complete panoramic video.More specifically, terminal feeds back theirs based on their ROI
Track is observed, then compresses ROI signal using the track by video server.Therefore, the video rate of transmission can be from
10Mbps is reduced to 3Mbps.
Equally, ROI is determined by the viewing center of the panoramic frame considered, because viewer tends to pay attention to his/her
Power concentrates on 120 centered on the focus of the eyes of viewer ° of angular range.It is assumed that in the estimation observation of Fig. 5
Heart v is completely known at transmitter, however the popular deep learning tool that can be used is estimated.ROI and viewing center
The example in Fig. 4.In practice, viewing track must feed back video server, as shown in figure 3, still in the present invention again
Secondary concern wireless streams transmission aspect.
B. the ROI-UEP framework proposed
The architecture of the ROI-UEP proposed is as shown in Figure 5.
Fig. 4: 120 ° of ROI of frame are determined by observing center v.
1) receiver module: at the transmitter of Fig. 5, the panoramic frame U considered is divided into block u1..., un, they with
Yuv format indicates and by HEVC encoder compresses, generates bit stream b1..., bn.Meanwhile by block b1..., bnInformation input
To " minimizing distortion " block of Fig. 5, block b is separately optimized1..., bnEncoding rate γ1..., γn.Then, n obtained
Bit stream b1..., bnIt will encode as follows:
N-bit sequence b1..., bnBy the FEC encoder 1 of Fig. 5 ... n coding, wherein being respectively adopted by " minimizing and losing
Very " the encoding rate γ of block generation1... γn.This leads to the bit sequence x being separately encoded1..., xn。
Then by bit sequence x1, xnIt connects into the single bit stream being used for transmission and then interleaver Δ is applied to be randomized
Burst transfer mistake.
The joint bit sequence of intertexture is sent eventually by antenna.
2) receiver model: received panorama signal is as follows by the wireless receiver processing of Fig. 5:
Table I: the symbol definition of system seen in fig. 5, wherein 1≤i≤n is block index.
After the demodulation, sequence x1..., xnSoft Inform ation, i.e. y1..., yn, then will be by the deinterleaver π of Fig. 5-1It is raw
At.
Then, Soft Inform ation yiIt is decoded by the fec decoder device i of Fig. 5, generates the bit sequence of estimationIts expression layer biEstimate
Count version.
Finally, the bit stream of estimationThen it is decoded by the HEVC decoder of Fig. 5, generates YUV blockIt is finally combined to rebuild panoramic video frame
Fig. 5: the ROI-UEP proposed assists the framework of wireless panorama communication system, and wherein symbol defines in tablei." most
Smallization distortion " block will illustrate in Section IV section.
3, the ROI-UEP encoding rate optimized
Table II: symbol definition, wherein 1≤i≤n indicates block index.
In the portion, " minimizing distortion " block of Fig. 5 is described in detail, determines the difference for encoding panoramic frame U
Active block b1..., bnEncoding rate γ1..., γn." minimize distortion " block is for weight expected from minimizing at receiver
It builds the distortion of video and designs.More specifically, considering editor-in-chief's code rate of the Γ when sending SNR and being S.Therefore, the volume of generation
Code rate γ1..., γnMeet the following conditions.
In the following, by exporting specific FEC encoding rate γ1..., γnTo minimize block b1..., bnVideo distortion.
Symbol definition based on Table I characterizes algorithm by using the symbol in Table II.
When using γ1..., γnWhen with the encoding rate of SNR=, according to peak value noise caused by the n erroneous block considered
D (γ is deteriorated than (PSNR)1..., γn) carry out quantitation video distortion.In the proposed solution, target is export encoding rate
γ1..., γn, minimize expected degradation D (γ1..., γn).In order to derive expected frame distortion, to block biUsing
It is assumed hereinafter that
·d(bi): it indicates by block biDistortion caused by damage；
P (s, | bi|, γi): use encoding rate γiWith the block b of radio transmitted power SiPER；
·w(bi): block biWeight.
Then, by block biDestruction caused by distortion can be expressed asEntirely
The expected distortion D (γ of scape frame1..., γn) can indicate are as follows:
Therefore, objective function (OF) can be expressed as
Fig. 6: the fec decoder process at receiver.
Condition limitation
Eq (10) indicates to send the gross bit rate limit of n encoding block.
In Section IV-A and IV-B section, solve the problems, such as that the component of equation (9) each section, i.e. panoramic video are distorted d
(bi) and packet error rate (PER) p (S, | bi|, γi).Then, the solution i.e. minimum code rate of target equation has been found out.(9) specifically
It is bright in order to determine Section IV-D section in encoding rate.Finally, the transport overhead that 10 schemes proposed are forced discusses in Section IV-E section.
A. estimation block distortion d ()
Distortion d (the b as caused by the damage of block bi is estimated using with solution similar in the pasti), wherein distortion d (bi),
1≤i≤n does not have block biBit stream, can be determined by decoding or solution can use in systems.
B. PERp () is estimated
The fec decoder device of Fig. 5 is as shown in fig. 6, wherein the length of Soft Inform ation isIt is entered fec decoder device,
Export the estimation bit sequence that length is λIn addition, r indicates the encoding rate of FEC codec, and letter is received at SNR s
Number.Based on constant value l, in Fig. 6PER depend on parameter s and γ, p (s, l, γ) can be expressed as.
It has been shown that the generation for the burst error that non-iterative codec encounters still is not influenced by data packet length.
In order to illustrate burst error distribution, consider to generate carrying (n by non-iterative fec decoder device1×n2) bit grouping situation.This
It outside, should (n1×n2) bit section is considered n1A grouping, each grouping and n2A bit is associated or each n2A point
Group all carries n1A bit.Have:
With | bi | instead of the n1 in equation 11, n2, in equation 9 objective function PERp (s, | bi|, γ i) it can indicate
Are as follows:
Wherein l is the bit number in the data packet inputted in Fig. 6 to fec decoder device.Further information about the calculating.
In order to determine the function p (s, l, γ) in equation 12.(12), the decoding process of simulation drawing 6 uses variable s first,
This generates look-up tables shown in " simulation " surface such as Fig. 7 for γ and λ=1000. constant value l=So
Afterwards, as shown, passing through " fitting " surface of Fig. 7,It is that mathematical modeling is
When known to s, a, b, c and d can be easily found.According to the definition of k (s, γ) and p (s, l, γ), Probability p
(s, l, γ) is represented by
Then, in equation PER estimation p (s, | bi|, γi) function k (s, γ) can be used in (12) be expressed as
By Eq. (15) substitute into Eq. (9), it is contemplated that video distortion D (γ1..., γn) can be ultimately expressed as
C. estimated weight w ()
Panoramic frame U is converted from sphere E, and pixel is shown on panoramic indicator, such as Oculus rift.Therefore, sphere E
On pixel importance having the same, and the pixel on U have different importance.Method is that M are generated on sphere E
The point of even distribution, wherein each pixel was expressed as e ().Many points can be evenly distributed on sphere E, following institute
Show:
On the latitude direction of sphere, angle is usedCarry out the circle that parallel divisional angle at equal intervals is δ, such as Fig. 8 a institute
Show.
The angle ∠ COB of Fig. 8 is expressed as α.For each in circle, radius may be calculated r=Rsin α.
On each circle, use angle for all points at equal intervals of δ, this example in Fig. 8 c.Pay attention to | AB | belong to triangle ACB and three
Angular AOB.The length of AB may be calculatedOrTherefore, haveThen, have on the circle consideredPoint, each point can calculate are as follows:
Well-distributed points as shown in Figure 9 are generated according to the above process.
Generation point set on sphere E is expressed as Se, points are expressed as Se.Meet x2+y2+z2=R2Arbitrary point
(x, y, z) ∈ SeIt can be mapped to rectangle according to equal rectangular projections, as discussed in the II-A of part.
The reference point of rectangle is shown in Figure 10 in Fig. 9.
Point on rectangle is expressed as set Sγ, points are expressed as | Sγ|.Since the point on ball E is uniformly distributed,
Positioned at the sub-block u considerediThe quantity of interior point indicates uiImportance, i.e. its weight w ().It is consequently possible to calculate Fig. 5
Block uiWeight w (i):
Definition in equation (18), w (i) indicate to be located at sub-block uiThe percentage of interior point.Correspondingly, have
D.ROI-UEP encoding rate
Objective function and constrained code rate defined in formula (9), there is Lagrange's multiplier equation, as shown in Eq (19),
Wherein λ is Lagrange's multiplier.Corresponding local derviation array
It can easily derive as equation (20).
Equation group (20) is solved using classical Newton-decline method, specifically as shown in algorithm 1.
E. performance cost
All operations discussed above execute all at the transmitter of Fig. 5.It will be briefly described and applied by the optimization process below
The expense added pays attention to not applying expense in receiver side.More specifically, the expense applied includes estimation distortion d (), estimate
It counts PERp (), estimate weight w () and determines encoding rate.
1) the estimation d () being distorted: being discussed, the complexity and the linear ratio of n applied by estimation distortion d (bi)
Example.
2) estimation of PER p (): as being described in detail in the IV-B of part, the complexity of PER estimation is mainly by LUT(s, γ)
Estimation procedure is forced, and the fec decoder device to Fig. 6 is specific.Again, LUT is obtained(s, γ) passes through the decoding of simulation drawing 6
Process, the process execute in off-line design procedure.In addition, LUT is independently of used video sequence, and depend on letter
Road, modulator and used FEC.The size of LUT can be expressed as (ns×nγ), wherein nsAnd nγRespectively indicate variable s and γ
Quantity.
3) estimation of weight w (): as described in Section IV-C portion, the complexity of weight estimation is mainly due to generate M
Uniform, this is executed in off-line design procedure.
4) ROI estimates: note that each of 60 observation tracing points define corresponding estimation viewing center, it will
For determining the ROI of considered panoramic frame, as shown in Figure 4.
This process can only bring appropriate complexity.Herein, this process not be described in detail, because it has exceeded
The scope of the present invention.
5) determination of encoding rate: in the system of proposition, encoding rate is determined using classical Newton-decline method, this
Method has the complexity of appropriateness.Alternatively, adaptive particle swarm optimization (APSO) technology that can use finds encoding rate.
Table III: it is used for transmission the parameter of RaceVR sequence used
In the following, for equal error protection (EEP) and EEP (ROI-EEP) system based on ROI to the ROI-UEP of proposition
System carries out benchmark test.The parameter III of RaceVR sequence used in simulation is detailed in table.Specifically, it is based on 4:2:
The RaceVR video clipping of 0YUV format (4096*2048) pixel resolution is encoded by HEVC reference software, wherein using being based on
The error concealing of low complex degree " frame duplication " replaces the panoramic frame of damage.For the sake of simplicity, IDR/CDR is used only in simulations
Frame.However, algorithm can be easily extended to B frame and P frame.In addition, using standard, H.265 quantization parameter (QPs) passes through HEVC
Scheme carries out video sequence to encode 69.2 megabits (Mbps) per second, 30 frame per second.In addition, there is no the case where error of transmission
Under, the Y-PSNR of 42.5dB can be realized by rebuilding video.
Other than the panoramic video parameter of Table III, details are as follows with FEC parameter for transmission.Specifically, Recursive Systematic Convolutional
(RSC) the volume solution configured by the octal system generator polynomial of [031,027,027,027,027,035,035,035,035,033]
Code device uses FEC for codec, and minimum code rate is 0.1.In addition, sending FEC using binary phase shift keying (BPSK)
The bit stream of coding.In addition, the bit stream of each HEVC coding is RSC coding, BPSK modulation, then by NALU in net
It is sent on network Abstraction Layer units (NALU).In order to generate statistically reasonable performance curve, simulation 100 times is repeated.Remaining parameter
It is listed in Table III.
A.PSNR assessment
In view of the discussion in the second section, it is known that the different pixels on plane framework have different importance.Correspondingly,
Pixel is weighted to assess MAX=2 in m pixel scenesmPosition.
The PSNR for considering frame U uses the symbol definition listed in Table IV.Specifically, by the weight of pixel U (i, j)
It is expressed as I (i, j), i.e. importance.Moreover, it is assumed that the viewpoint on sphere is (cx, cy, cz), visual angle υ.Therefore, it is located at viewing
The longest distance of the pixel on sphere in range can be expressed as
Recall section II, the pixel in same a line of Fig. 2 b is located in the same latitude of Fig. 2 a, and vice versa.Therefore, as
The importance of plain U (i, j) can be calculated as the ratio between related latitudinal perimeter and width W, i.e. I (i, j) in equation 21.
The weighted mean square error (WMSE) of use is defined as
And Y-PSNR (WPSNR) is weighted accordingly and may be calculated
Having has MAX=2 in m pixel scenesm
B. offline LUT is generated
In order to generate LUT, the vector of [- 5:1:25], [0:0.1:9.9] are respectively used toVariable s, γ-1, obtain
To ns=31, nγ=100 in addition, using λ=1000 block length.
C. datum mark
1) framework of ROI-UEP:ROI-UEP scheme is detailed in Fig. 5.
2) ROI-EEP: in ROI-UEP scheme, the signal in ROI is arranged by different protections in ROI-EEP
In, they have identical protection relevant to γ 1==γ n.
As shown in figure 12: the weight of block is calculated by the algorithm discussed in Section IV-C section, and wherein panoramic frame is divided into 64 sons
Block.
3) entire panoramic video signal, including ROI equally EEP: are protected using encoding rate Γ.
D. performance
In the following, assessment WPSNR video quality, the PER and BER of related block.
1) track is checked: in simulations, using view track, i.e., the head motion track of all 100 wheel simulations, to make
Performance improvement is observed with the algorithm of proposition.Used track is as shown in figure 11.Note that each of 60 tracing points all tables
The estimation of deep learning tool can be used in the observation center for showing 60 estimations.
2) weight of block: the block weight of the first panoramic frame is shown in FIG. 12, and wherein panoramic frame is divided into (8*8) height
Block generates n=64 block.
3) WPSNR video quality: WPSNR and Eb/N0As a result it is recorded in Figure 13 a, wherein ROI-EEP scheme is considered base
It is better than EEP scheme in sheet.Specifically, ROI-EEP and EEP scheme realizes 38.7dB's under 7dB and 12dB channel SNR respectively
WPSNR.Alternatively, ROI-EEP scheme realizes the E of 5dB when being directed to the WPSNR of 38.7dBb/N0Gain.This is because ROI-EEP
Scheme passes through the region in the peripheral view for sacrificing viewer to distribute stronger protection for ROI signal.
It can be seen that compared with ROI-EEP scheme from Figure 13 a, ROI-UEP scheme further improves WPSNR performance, especially
It is in lower Eb/N0In range.Specifically, ROI-UEP scheme is better than ROI-EEP scheme when WPSNR is 35dB
0.8dB channel Eb/N0.In addition, in channel Eb/N0When for 6dB, ROI-UEP scheme is better than the WPSNR of ROI-EEP scheme
9.4dB, wherein ROI-UEP scheme and ROI-EEP scheme can be realized the WPSNR and 25.9dB of 35.3dB.This is because ROI-
UEP scheme dynamically finds the suitable FEC encoding rate set of the non-uniform FDTD grids for ROI signal, wherein can sacrifice not
Too important vision signal improves the protection to prior panoramic pixel.
4) BER module: for block u54, u7And u14The encoding rate and E of recordb/N0As a result relationship is respectively displayed on figure
In 13b, Figure 13 c and Figure 13 d, lower BER value is observed.Specifically, for the E from 0dB to 14dBb/N0Value, block u14's
BER is lower than 10-5, and block u54BER drop to 10 in 7dB-5.This is because ROI-UEP scheme is that prior ROI is assigned with
More FEC protective redundancies.Note that in Figure 13 b, in Figure 13 c and Figure 13 d, for ROI-UEP scheme record BER curve with
Eb/N0It is worth and fluctuates.This optimization process that may be attributed to the ROI-UEP seen in algorithm 1 possibly can not find the overall situation most
It is excellent.
5) encoding rate: for block u54, u7And u14The encoding rate and E of recordb/N0As a result relationship is respectively displayed on Figure 13 e,
In Figure 13 f and Figure 13 g.For ROI-EEP and EEP scheme, all Eb/N0All pieces of u in range54, u7And u14Encoding rate
It remains unchanged.It is observed in Figure 13 g, compared with ROI-EEP and EEP scheme, the distribution of ROI-UEP scheme and lower encoding rate phase
Associated stronger protection.This is because to prior piece of u14Stronger protection can moderately reduce panoramic video mistake
Very, as depicted in fig. 13 a.
6) panoramic frame: if Figure 14 is in the systematic comparison record reconstructed frame proposed and using the benchmark of RaceVR video sequence
The channel SNR of label is 6dB.The ROI of viewer is shown in Figure 14 b and Figure 14 g, they are by stronger protection.In addition,
ROI-UEP scheme has rebuild better visual quality in the roi.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Made any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.
Claims (6)
1. a kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest, which is characterized in that the inequality
Error protection assist the wireless panoramic video of region-of-interest method the following steps are included:
Step 1 perceives inequality error protection ROI-UEP for efficient video code HEVC sequence by area-of-interest
Wireless transmission；
Step 2 will be that ROI signal find most suitable not equal forward error corrections FEC volume by minimizing expected video distortion
Code rate；
Step 3 is used to assess the quality for the panoramic video rebuild by weighting Y-PSNR WPSNR.
2. the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest as described in claim 1, feature exist
In the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest obtains sphere and corresponding plane coordinates side
Method includes:
(1) spherical coordinates is converted into plane coordinates
Point A is the arbitrary point on spherical surface, wherein αo(-π≤αo≤ π) andIt is the longitude and latitude of A
Spend coordinate (xa, ya, za), longitude αoIt is calculated as；
The wherein radius of R expression sphere, and latitude αaIt is expressed as；
Using the longitude and latitude derived, the projected position of the point A on plane coordinates easily can be expressed as (wa,
ha)；
wa=αo·R
ha=αa·R；
The width and height for projecting frame size are W=2 π R, H=π R；
It (2) is spherical coordinate by two-dimensional assemblage, point A is with coordinate (wa, ha) planar rectangular on arbitrary point, wherein；
With
αo(-π≤αo≤ π) andIt is the longitude and latitude of A on estimated sphere；Longitude αoIt calculates
For；
Latitude αaIt indicates are as follows:
Using the longitude and latitude derived, the projected position of the point A in spherical coordinate is expressed as (xa, ya, za), have；
xa=Rcos αa·cosαo
ya=Rcos αa·sinαo
za=sin αa·R。
3. the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest as described in claim 1, feature exist
In the wireless channel panorama that the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest is encoded based on HEVC
The ROI-UEP framework of video flowing includes:
1) receiver module: panoramic frame U is divided into block u1..., un, indicated with yuv format and by HEVC encoder compresses, generation
Bit stream b1..., bn；Meanwhile by block b1..., bnInformation input to minimize be distorted block, block b is separately optimized1...,
bnEncoding rate γ1..., γn；Then, the n bit stream b obtained1..., bnIt will encode as follows:
N-bit sequence b1..., bnBy FEC encoder 1 ... n coding is generated wherein being respectively adopted by " minimizing distortion " block
Encoding rate γ1... γn, lead to the bit sequence x being separately encoded1..., xn；
Then by bit sequence x1, xnIt connects into the single bit stream being used for transmission and then applies interleaver Δ to be randomized burst and pass
Input mistake by mistake；
The joint bit sequence of intertexture is sent eventually by antenna；
2) receiver model: received panorama signal wireless receiver processing is as follows:
After the demodulation, sequence x1..., xnSoft Inform ation, i.e. y1..., yn, then by deinterleaver π-1It generates；
Soft Inform ation yiIt is decoded by fec decoder device i, generates the bit sequence of estimationIts expression layer biEstimation version；
Finally, the bit stream of estimationThen it is decoded by HEVC decoder, generates YUV blockIt is finally by group
It closes to rebuild panoramic video frame
4. the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest as described in claim 1, feature exist
In the total coding of Γ of the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest when sending SNR and being S
Rate, the encoding rate γ of generation1..., γnMeet the following conditions；
When using γ1..., γnWhen with the encoding rate of SNR=, deteriorated according to Y-PSNR caused by n erroneous block (PSNR)
D(γ1..., γn) quantitation video distortion, target is export encoding rate γ1..., γn, minimize expected degradation D
(γ1..., γn)；To block biUsing；
d(bi): it indicates by block biDistortion caused by damage；
P (s, | bi|, γi): use encoding rate γiWith the block b of radio transmitted power SiPER；
w(bi): block biWeight；
By block biDestruction caused by distortion be expressed asThe expected distortion of panoramic frame
D(γ1..., γn) indicate are as follows:
Objective function OF is expressed as；
5. the method for the inequality error protection auxiliary wireless panoramic video of region-of-interest as described in claim 1, feature exist
In the fec decoder process at the method receiver of the inequality error protection auxiliary wireless panoramic video of region-of-interest includes；
Condition limitation；
Eq (10) indicates to send the gross bit rate limit of n encoding block；
Expected video distortion D (γ1..., γn) be expressed as；
C. estimated weight w ().
6. inequality error protection described in a kind of application Claims 1 to 5 any one assists the wireless panoramic video of region-of-interest
Method unmanned plane.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910232071.7A CN109874019A (en) | 2019-03-26 | 2019-03-26 | A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910232071.7A CN109874019A (en) | 2019-03-26 | 2019-03-26 | A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109874019A true CN109874019A (en) | 2019-06-11 |
Family
ID=66921195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910232071.7A Pending CN109874019A (en) | 2019-03-26 | 2019-03-26 | A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109874019A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110740334A (en) * | 2019-10-18 | 2020-01-31 | 福州大学 | frame-level application layer dynamic FEC encoding method |
| CN112055263A (en) * | 2020-09-08 | 2020-12-08 | 西安交通大学 | 360-degree video streaming transmission system based on significance detection |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101385357A (en) * | 2006-02-15 | 2009-03-11 | 三星电子株式会社 | Method and system for application of unequal error protection to uncompressed video for transmission over wireless channels |
| CN104463778A (en) * | 2014-11-06 | 2015-03-25 | 北京控制工程研究所 | Panoramagram generation method |
| US20170214491A1 (en) * | 2016-01-25 | 2017-07-27 | Amine Maaref | Apparatus and method for multimedia content transmissions in multi-channel wireless networks |
| CN107040771A (en) * | 2017-03-28 | 2017-08-11 | 北京航空航天大学 | A kind of Encoding Optimization for panoramic video |
| CN107135397A (en) * | 2017-04-28 | 2017-09-05 | 中国科学技术大学 | A kind of panorama video code method and apparatus |
| CN107462228A (en) * | 2017-07-17 | 2017-12-12 | 深圳市华星光电半导体显示技术有限公司 | Positioner and localization method, color film coating machine |
| CN108093263A (en) * | 2017-12-11 | 2018-05-29 | 西安交通大学 | A kind of video transmission method based on minimum distortion optimization in free space optical communication |
-
2019
- 2019-03-26 CN CN201910232071.7A patent/CN109874019A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101385357A (en) * | 2006-02-15 | 2009-03-11 | 三星电子株式会社 | Method and system for application of unequal error protection to uncompressed video for transmission over wireless channels |
| CN104463778A (en) * | 2014-11-06 | 2015-03-25 | 北京控制工程研究所 | Panoramagram generation method |
| US20170214491A1 (en) * | 2016-01-25 | 2017-07-27 | Amine Maaref | Apparatus and method for multimedia content transmissions in multi-channel wireless networks |
| CN107040771A (en) * | 2017-03-28 | 2017-08-11 | 北京航空航天大学 | A kind of Encoding Optimization for panoramic video |
| CN107135397A (en) * | 2017-04-28 | 2017-09-05 | 中国科学技术大学 | A kind of panorama video code method and apparatus |
| CN107462228A (en) * | 2017-07-17 | 2017-12-12 | 深圳市华星光电半导体显示技术有限公司 | Positioner and localization method, color film coating machine |
| CN108093263A (en) * | 2017-12-11 | 2018-05-29 | 西安交通大学 | A kind of video transmission method based on minimum distortion optimization in free space optical communication |
Non-Patent Citations (1)
| Title |
|---|
| YONGKAI HUO等: "Historical Information Aware Unequal Error Protection of Scalable HEVC/H.265 Streaming Over Free Space Optical Channels", 《IEEE EXPLORE》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110740334A (en) * | 2019-10-18 | 2020-01-31 | 福州大学 | frame-level application layer dynamic FEC encoding method |
| CN110740334B (en) * | 2019-10-18 | 2021-08-31 | 福州大学 | Frame-level application layer dynamic FEC encoding method |
| CN112055263A (en) * | 2020-09-08 | 2020-12-08 | 西安交通大学 | 360-degree video streaming transmission system based on significance detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106331711B (en) | A kind of dynamic code rate control method based on network characterization and video features | |
| US20030103571A1 (en) | Combined MPEG-4 FGS and modulation algorithm for wireless video transmission | |
| Alajel et al. | Unequal error protection scheme based hierarchical 16-QAM for 3-D video transmission | |
| JP6151355B2 (en) | Panorama picture processing | |
| US10542265B2 (en) | Self-adaptive prediction method for multi-layer codec | |
| CN109874019A (en) | A kind of method of the inequality error protection auxiliary wireless panoramic video of region-of-interest | |
| Ryu et al. | Home gateway for three-screen TV using H. 264 SVC and raptor FEC | |
| Fujihashi et al. | FreeCast: Graceful free-viewpoint video delivery | |
| Vosoughi et al. | Joint source-channel coding and unequal error protection for video plus depth | |
| US10499112B2 (en) | Robust digital channels | |
| Zhang et al. | Scalable panoramic wireless video streaming relying on optimal-rate FEC-coded adaptive QAM | |
| US20180270471A1 (en) | Hybrid digital-analog coding of stereo video | |
| Huo et al. | Wireless video: An interlayer error-protection-aided multilayer approach | |
| US9936226B2 (en) | Robust digital channels | |
| Ozcinar et al. | Omnidirectional video streaming using visual attention-driven dynamic tiling for VR | |
| Fuiihashi et al. | Graceful quality improvement in wireless 360-degree video delivery | |
| Okamoto et al. | Edge-assisted multi-user 360-degree video delivery | |
| CN113395505B (en) | Panoramic video coding optimization method based on user field of view | |
| Lu et al. | 360Cast: Foveation-based wireless soft delivery for 360-degree video | |
| You et al. | Hierarchical modulation for client-driven selective streaming of multi-view video over AWGN channels | |
| KR101824919B1 (en) | Apparatus for Transmitting and Receiving Scalable Multi-point Video | |
| Zhang | Adaptive unequal error Protection for scalable video streaming | |
| Chandricka et al. | Unequal Power Allocation For Uncompressed Video Transmission Using wireless channel | |
| Kurosaki et al. | 4K digital cinema home theater over high throughput wireless transmission system | |
| Hosany et al. | An unequal error protection scheme employing convolutional codes and asymmetric 8PSK modulation for robust transmission of H. 264/AVC video in wireless channels |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190611 |