CN104539971A - Inter-viewpoint bit allocation method applied to two-viewpoint video coding - Google Patents

Inter-viewpoint bit allocation method applied to two-viewpoint video coding Download PDF

Info

Publication number
CN104539971A
CN104539971A CN201510018246.6A CN201510018246A CN104539971A CN 104539971 A CN104539971 A CN 104539971A CN 201510018246 A CN201510018246 A CN 201510018246A CN 104539971 A CN104539971 A CN 104539971A
Authority
CN
China
Prior art keywords
nbv
viewpoint
centerdot
coding
distortion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510018246.6A
Other languages
Chinese (zh)
Other versions
CN104539971B (en
Inventor
元辉
郭志鑫
刘琚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN201510018246.6A priority Critical patent/CN104539971B/en
Publication of CN104539971A publication Critical patent/CN104539971A/en
Application granted granted Critical
Publication of CN104539971B publication Critical patent/CN104539971B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Provided is an inter-viewpoint bit allocation method applied to two-viewpoint video coding. Firstly, influences of coding distortion of a basic viewpoint on coding distortion of a non-basic viewpoint are determined, and a rate-distortion model of the basic viewpoint and the non-basic viewpoint is determined; then, an inter-viewpoint bit allocation problem is converted into the problem of minimizing overall coding distortion of the basic viewpoint and the non-basic viewpoint under limited total bit cost condition, the optimal solution is solved, and an optimal inter-viewpoint bit allocation scheme is achieved. The method improves coding efficiency of a multi-viewpoint video, so that quality of the multi-viewpoint video reconstructed at a decoding end is higher under the same coding rate condition, and the method has the advantages of being high in performance and universality and the like, and can be used in coding standards of the multi-viewpoint video.

Description

Bit distribution method between a kind of viewpoint being applied to 2 viewpoint video codings
Technical field
The present invention relates to Bit distribution method between a kind of viewpoint for improving multiple view video coding efficiency, belonging to multiple view video coding mark technical field.
Background technology
Three-dimensional (3D) video, as main video applications in future, refers to that user can enjoy real 3 d video content by 3 D video display device.The correlation technique of 3 D video, such as, the technology such as the display of three-dimensional video acquisition, 3 d video encoding, 3 D video are paid close attention to widely.In order to promote the standardization of 3 D video technology, 2002, Motion Picture Experts Group (Motion Picture Experts Group, and Video Coding Experts group (Video Coding ExpertsGroup MPEG), VCEG) formulated based on multiple view video coding (Multi View Video Coding, MVC) standard (H.264/MVC) H.264.Current MPEG and VCEG formulates MVC standard of new generation, i.e. MV-HEVC based on up-to-date high-performance video coding standard (High EfficiencyVideo Coding, HEVC).In multi-view point video, user can watch the 3 D video of this angle from different angles, user is had incorporate the sense of reality in video scene.Multi-view point video can be widely used in the fields such as broadcast communication, amusement, education, medical treatment and video monitoring.When the more viewpoint of needs, multi-view point video-depth representing 3D scene can be adopted, and adopt 3D-HEVC coding standard to encode; The visual point image that user terminal can adopt View synthesis technology (Depth Image Based Rendering, the DIBR) synthesis based on the degree of depth to need.Therefore, in existing MV-HEVC coding standard framework, the coding structure of 2 viewpoints and 3 viewpoints is only defined.
Between existing multi-view point video, Bit distribution method is the Bit distribution method of fixed proportion, and the coding flow process that fixed proportion bit distributes as shown in Figure 1, adopts the proportionality coefficient preset to be different viewpoint allocation bit.For the video of 2 viewpoints, the bits proportion between BV and NBV can be 2:8,4:6,6:4 etc.; Bits proportion for the video BV of 3 viewpoints and left side not substantially viewpoint (NBVL), right side not substantially viewpoint (NBVR) can be 4:3:3,6:2:2 etc.
The Bit distribution method of fixed proportion does not consider the impact of the coding distortion of BV on the coding distortion of NBV, only arranges a fixing allocation proportion by rule of thumb; Therefore do not possess versatility, the total quality of the multi-view point video of reconstruction is not high yet.From theory analysis, BV has considerable influence to NBV, and the coding distortion of BV can directly propagate in NBV, in the bit allocation procedures therefore between multiple views, should consider the impact of BV on NBV.
Summary of the invention
Bit distribution method for fixed proportion does not possess versatility, the shortcoming that reconstruction video total quality is not high, the present invention is according to basic viewpoint (Base View, BV) to not substantially viewpoint (Non-Base View, NBV) impact, proposes Bit distribution method between a kind of viewpoint being applied to 2 viewpoint video codings.Between the viewpoint of the method concern multi-view point video, bit distributes, determine in multi-view point video, the distortion of basic viewpoint is on the impact of not substantially viewpoint, Bit distribution method between the basic viewpoint of devise optimum and not substantially viewpoint, improve the code efficiency of multi-view point video, make under the condition of same-code code check, the multi-view point video quality that decoding end is rebuild is higher.
Bit distribution method between the viewpoint being applied to 2 viewpoint video codings of the present invention, first determines the impact of the coding distortion of basic viewpoint on the coding distortion of not substantially viewpoint, determines the rate-distortion model of basic viewpoint and not substantially viewpoint; Then, the bit assignment problem between viewpoint is converted under limited total overhead bit condition, minimizes the problem of the binary encoding distortion of basic viewpoint and not substantially viewpoint, and obtain optimal solution (between namely optimum viewpoint bit allocation scheme); Specifically comprise the following steps:
(1) Mathematical Modeling defined between the coding distortion of basic viewpoint and number of coded bits is:
D bv=(a bv·R bv+b bv) -1,
Wherein, D bvrepresent the coding distortion of basic viewpoint, R bvrepresent the number of coded bits of basic viewpoint, a bvand b bvfor model parameter, represent R bvto D bvinfluence degree; The coding distortion D of not substantially viewpoint nbvwith the number of coded bits R of not substantially viewpoint nbvwith the number of coded bits R of basic viewpoint bvbetween pass be:
D nbv=η(a bvR bv+b bv) -1+(a nbvR nbv+b nbv) -1,
Wherein, η represents that the coding distortion of the basic viewpoint of present frame is to the influence degree of the coding distortion of NBV, a nbvand b nbvfor model parameter, represent R nbvto D nbvinfluence degree;
η described in step (1) is calculated by following formula:
η = Num PU _ InterViewSKIPMode Num PU ,
Wherein, Num pU_InterViewSKIPModerepresent in the not substantially viewpoint of present frame, adopt the quantity of the predicting unit (Prediction Unit, PU) that dancing mode (Inter View SkipMode) is encoded between viewpoint, Num pUrepresent in the not substantially viewpoint of present frame, the quantity of all predicting unit.
(2) W frame before multi-view video sequences being started most, rate control algorithm is adopted to determine the basic viewpoint of every frame and total bit number of not substantially viewpoint, and be different viewpoint allocation bit by the ratio preset and encode, the coding distortion of the basic viewpoint after coding and number of coded bits are stored in array vector respectively with in, coding distortion and the number of coded bits of not substantially viewpoint are stored in array vector respectively D nbv → = { D nbv , 0 , . . . , D nbv , W - 1 } With R nbv → = { R nbv , 0 , . . . , R nbv , W - 1 } In;
The ratio (initial bit allocation proportion) preset in step (2) is:
(3) according to the distortion of the basic viewpoint in the W frame before present frame with bit number and D in step (1) bvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
D bv , 0 - 1 · · · · · · D bv , W - 1 - 1 = a bv R bv , 0 · · · · · · R bv , W - 1 + b bv ,
Calculate the model parameter a of basic viewpoint bvand b bv;
(4) according to the distortion of the not substantially viewpoint in the W frame before present frame with bit number R nbv → = { R nbv , 0 , . . . , R nbv , W - 1 } The bit number of BV R bv → = { R bv , 0 , . . . , R bv , W - 1 } , And D in step (1) nbvwith R nbvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
( D nbv , 0 - η 0 D bv , 0 ) - 1 · · · · · · ( D nbv , W - 1 - η W - 1 D bv , W - 1 ) - 1 = a nbv R nbv , 0 · · · · · · R nbv , W - 1 + b nbv ,
Wherein η 0η w-1represent that in encoded W frame, in 2 viewpoints of every frame, the distortion of basic viewpoint, to the influence degree of the distortion of not substantially viewpoint, calculates the model parameter a of not substantially viewpoint nbvand b nbv;
(5) a of foundation step (3) and step (4) gained bv, b bv, a nbvand b nbv, and adopt total bit number R of rate control algorithm present frame c, pass through following formula:
R bv = R c + b nbv a nbv - b bv a bv a nbv ( 1 + η ) 1 + a bv a nbv ( 1 + η ) ,
R nbv = a bv a nbv ( 1 + η ) R bv + b bv a nbv a bv ( 1 + η ) - b nbv a nbv ,
For basic viewpoint and the not substantially viewpoint allocation bit number of present frame, namely calculate the R of present frame bvand R nbv;
(6) according to the principle of " first in first out ", by array vector the 0th data delete, and the coding distortion of present frame and number of coded bits are added to the end of array vector, upgrade array vector
(7) return step (3), next frame is encoded.
The present invention mainly improves the code efficiency of multi-view point video, determines the rate-distortion model of multi-view point video, and the basic viewpoint of present frame (comprising multiple visual point image) is on the impact of not substantially viewpoint; Then adopt rate control techniques to calculate the spendable total bit number of present frame, and by optimal method, be basic viewpoint and the not substantially viewpoint allocation bit of every frame, and then improve the code efficiency of multi-view point video.Compared with prior art have the following advantages:
1) the present invention is according to the coding distortion of basic viewpoint on the impact of the coding distortion of not substantially viewpoint, and the Bit distribution method between the multiple views devising distortion performance optimum, improves the code efficiency of multi-view point video;
2) the present invention can adjust bit allocation proportion between viewpoint to any video, has very strong adaptivity.
Accompanying drawing explanation
Fig. 1 is the coding flow chart that fixed proportion bit distributes.
Fig. 2 is the flow chart of Bit distribution method when being applied to 2 viewpoints coding of the present invention.
Fig. 3 be adopt method of the present invention and fixed proportion bit to distribute respectively method to 2 of three-dimensional video sequence Balloons the experimental result that viewpoint is encoded (rate distortion curve) comparison diagrams.
Fig. 4 be adopt method of the present invention and fixed proportion bit to distribute respectively method to 2 of three-dimensional video sequence BookArrival the experimental result that viewpoint is encoded (rate distortion curve) comparison diagrams.
Embodiment
As Fig. 2, Bit distribution method between the viewpoint that the present invention is applied to 2 viewpoint videos coding, detailed process is as follows:
(1) Mathematical Modeling defined between the coding distortion of basic viewpoint (Base View, BV) and number of coded bits is
D bv=(a bv·R bv+b bv) -1,
Wherein, D bvrepresent the coding distortion of BV, R bvrepresent the number of coded bits of BV, a bvand b bvfor model parameter, represent R bvto D bvinfluence degree; Coding distortion (the D of NBV nbv) with the number of coded bits (R of NBV nbv) and the number of coded bits (R of BV bv) between pass be:
D nbv=η(a bvR bv+b bv) -1+(a nbvR nbv+b nbv) -1,
Wherein, η represents the influence degree of the coding distortion of present frame BV to the coding distortion of NBV, a nbvand b nbvfor model parameter, represent R nbvto D nbvinfluence degree;
Described η can be calculated by following formula
η = Num PU _ InterViewSKIPMode Num PU ,
Wherein, Num pU_InterViewSKIPModerepresent in the NBV of present frame, adopt the quantity of the predicting unit (Prediction Unit, PU) that dancing mode (Inter View Skip Mode) is encoded between viewpoint, Num pUrepresent in the NBV of present frame, the quantity of all predicting unit.
(2) W frame before multi-view video sequences being started most, rate control algorithm is adopted to determine total bit number of BV and NBV of every frame, and be different viewpoint allocation bit by the ratio preset and encode, the coding distortion of the BV after coding and number of coded bits are stored in array vector respectively D bv → = { D bv , 0 , . . . , D bv , W - 1 } With R bv → = { R bv , 0 , . . . , R bv , W - 1 } In, coding distortion and the number of coded bits of NBV are stored in array vector respectively D nbv → = { D nbv , 0 , . . . , D nbv , W - 1 } With R nbv → = { R nbv , 0 , . . . , R nbv , W - 1 } In;
The described ratio (initial bit allocation proportion) preset is
(3) according to the distortion of the BV in the W frame before present frame with bit number and D in step (1) bvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
D bv , 0 - 1 · · · · · · D bv , W - 1 - 1 = a bv R bv , 0 · · · · · · R bv , W - 1 + b bv ,
Calculate the model parameter a of BV bvand b bv;
(4) according to the distortion of the NBV in the W frame before present frame with bit number R nbv → = { R nbv , 0 , . . . , R nbv , W - 1 } , The bit number of BV R bv → = { R bv , 0 , . . . , R bv , W - 1 } , And D in step (1) nbvwith R nbvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
( D nbv , 0 - η 0 D bv , 0 ) - 1 · · · · · · ( D nbv , W - 1 - η W - 1 D bv , W - 1 ) - 1 = a nbv R nbv , 0 · · · · · · R nbv , W - 1 + b nbv ,
Wherein η 0η w-1represent that in encoded W frame, in 2 viewpoints of every frame, the distortion of BV, to the influence degree of the distortion of NBV, calculates the model parameter a of NBV nbvand b nbv;
(5) a of foundation step (3) and step (4) gained bv, b bv, a nbv, b nbv, and adopt total bit number R of rate control algorithm present frame c, pass through following formula:
R bv = R c + b nbv a nbv - b bv a bv a nbv ( 1 + η ) 1 + a bv a nbv ( 1 + η ) ,
R nbv = a bv a nbv ( 1 + η ) R bv + b bv a nbv a bv ( 1 + η ) - b nbv a nbv ,
For BV and the NBV allocation bit number of present frame, namely calculate the R of present frame bvand R nbv;
(6) according to the principle of " first in first out ", by array vector the 0th data delete, and the coding distortion of present frame and number of coded bits are added to the end of array vector, upgrade array vector
(7) return step (3), next frame is encoded.
Effect of the present invention can further illustrate by experiment.
Experiment test under different bandwidth, the encoder bit rate after adopting the present invention to encode to multi-view point video and rebuild the overall objective quality (Y-PSNR, PSNR) of multi-view point video.Fig. 3 and Fig. 4 compares the rate distortion curve of the Bit distribution method adopting the present invention and fixed proportion.Wherein Fig. 3 is the experimental result of 2 viewpoint codings to three-dimensional video sequence Balloons, and Fig. 4 is the experimental result of 2 viewpoint codings to three-dimensional video sequence BookArrival.From Fig. 3 and Fig. 4, compared with the Bit distribution method of fixed proportion, after adopting the present invention to encode, under same-code code check, the total quality rebuilding multi-view point video is higher, and the coding efficiency that invention increases multi-view point video is described.

Claims (3)

1. a Bit distribution method between the viewpoint being applied to 2 viewpoint videos coding, is characterized in that, comprise the following steps:
(1) Mathematical Modeling defined between the coding distortion of basic viewpoint and number of coded bits is:
D bv=(a bv(R bv+b bv) -1,
Wherein, D bvrepresent the coding distortion of basic viewpoint, R bvrepresent the number of coded bits of basic viewpoint, a bvand b bvfor model parameter, represent R bvto D bvinfluence degree; The coding distortion D of not substantially viewpoint nbvwith the number of coded bits R of not substantially viewpoint nbvwith the number of coded bits R of basic viewpoint bvbetween pass be:
D nbv=η(a bvR bv+b bv) -1+(a nbvR nbv+b nbv) -1,
Wherein, η represents that the coding distortion of the basic viewpoint of present frame is to the influence degree of the coding distortion of NBV, a nbvand b nbvfor model parameter, represent R nbvto D nbvinfluence degree;
(2) W frame before multi-view video sequences being started most, rate control algorithm is adopted to determine the basic viewpoint of every frame and total bit number of not substantially viewpoint, and be different viewpoint allocation bit by the ratio preset and encode, the coding distortion of the basic viewpoint after coding and number of coded bits are stored in array vector respectively with in, coding distortion and the number of coded bits of not substantially viewpoint are stored in array vector respectively D → nbv = { D nbv , 0 , · · · , D nbv , W - 1 } With R → nbv = { R nbv , 0 , · · · , R nbv , W - 1 } In;
(3) according to the distortion of the basic viewpoint in the W frame before present frame with bit number and D in step (1) bvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
D bv , 0 - 1 . . . . . . D bv , W - 1 - 1 = a bv R bv , 0 . . . . . . R bv , W - 1 + b bv ,
Calculate the model parameter a of basic viewpoint bvand b bv;
(4) according to the distortion of the not substantially viewpoint in the W frame before present frame with bit number R → nbv = { R nbv , 0 , · · · , R nbv , W - 1 } , The bit number of BV R → bv = { R bv , 0 , · · · , R bv , W - 1 } , And D in step (1) nbvwith R nbvand R bvbetween Mathematical Modeling, by linear regression solving equation group:
( D nbv , 0 - η 0 D bv , 0 ) - 1 . . . . . . ( D nbv , W - 1 - η W - 1 D bv , W - 1 ) - 1 = a nbv R nbv , 0 . . . . . . R nbv , W - 1 + b nbv ,
Wherein η 0η w-1represent that in encoded W frame, in 2 viewpoints of every frame, the distortion of basic viewpoint, to the influence degree of the distortion of not substantially viewpoint, calculates the model parameter a of not substantially viewpoint nbvand b nbv;
(5) a of foundation step (3) and step (4) gained bv, b bv, a nbvand b nbv, and adopt total bit number R of rate control algorithm present frame c, pass through following formula:
R bv = R c + b nbv a nbv - b bv a bv a nbv ( 1 + η ) 1 + a bv a nbv ( 1 + η ) ,
R nbv = a bv a nbv ( 1 + η ) R bv + b bv a nbv a bv ( 1 + η ) - b nbv a nbv ,
For basic viewpoint and the not substantially viewpoint allocation bit number of present frame, namely calculate the R of present frame bvand R nbv;
(6) according to the principle of " first in first out ", by array vector the 0th data delete, and the coding distortion of present frame and number of coded bits are added to the end of array vector, upgrade array vector
(7) return step (3), next frame is encoded.
2. Bit distribution method between the viewpoint being applied to 2 viewpoint video codings according to claim 1, it is characterized in that, the η described in described step (1) is calculated by following formula:
η = Num PU _ InterViewSKIPMode Num PU ,
Wherein, Num pU_InterViewSKIPModerepresent in the not substantially viewpoint of present frame, adopt the quantity of the predicting unit of dancing mode coding between viewpoint, Num pUrepresent in the not substantially viewpoint of present frame, the quantity of all predicting unit.
3. Bit distribution method between the viewpoint being applied to 2 viewpoint video codings according to claim 1, it is characterized in that, the ratio preset in described step (2) is:
CN201510018246.6A 2015-01-14 2015-01-14 It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding Expired - Fee Related CN104539971B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510018246.6A CN104539971B (en) 2015-01-14 2015-01-14 It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510018246.6A CN104539971B (en) 2015-01-14 2015-01-14 It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding

Publications (2)

Publication Number Publication Date
CN104539971A true CN104539971A (en) 2015-04-22
CN104539971B CN104539971B (en) 2017-06-30

Family

ID=52855415

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510018246.6A Expired - Fee Related CN104539971B (en) 2015-01-14 2015-01-14 It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding

Country Status (1)

Country Link
CN (1) CN104539971B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242532A (en) * 2007-12-12 2008-08-13 浙江万里学院 A code rate control method oriented to multi-view point video
CN101835056A (en) * 2010-04-29 2010-09-15 西安电子科技大学 Allocation method for optimal code rates of texture video and depth map based on models
US20130321574A1 (en) * 2012-06-04 2013-12-05 City University Of Hong Kong View synthesis distortion model for multiview depth video coding

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242532A (en) * 2007-12-12 2008-08-13 浙江万里学院 A code rate control method oriented to multi-view point video
CN101835056A (en) * 2010-04-29 2010-09-15 西安电子科技大学 Allocation method for optimal code rates of texture video and depth map based on models
US20130321574A1 (en) * 2012-06-04 2013-12-05 City University Of Hong Kong View synthesis distortion model for multiview depth video coding

Also Published As

Publication number Publication date
CN104539971B (en) 2017-06-30

Similar Documents

Publication Publication Date Title
CN101668205B (en) Self-adapting down-sampling stereo video compressed coding method based on residual error macro block
CN106162195B (en) A kind of 3D HEVC deep video information concealing methods based on single depth frame mode
CN101835056B (en) Allocation method for optimal code rates of texture video and depth map based on models
CN102413353B (en) Method for allocating code rates of multi-view video and depth graph in stereo video encoding process
CN106210722B (en) The coding method of depth of seam division video residual error layer data based on HEVC
CN102065296B (en) Three-dimensional video coding method
CN101222627A (en) Multi-viewpoint video coding and decoding system, method and device for estimating vector
CN103024402B (en) A kind of free viewpoint video entire frame loss error concealing method
CN104995916A (en) Video data decoding method and video data decoding apparatus
CN102438167B (en) Three-dimensional video encoding method based on depth image rendering
CN102308583A (en) Apparatus and method for encoding and decoding multi-view image
CN104717515A (en) Texture video and depth map code rate distributing method based on 3D-HEVC
CN103873867B (en) Free viewpoint video depth map distortion prediction method and free viewpoint video depth map coding method
CN105049866A (en) Rendering distortion model-based code rate allocation method of multi-viewpoint plus depth coding
CN102790895B (en) Multi-viewpoint video encoding and viewpoint synthesis predication method based on least square
CN104159095A (en) Code rate control method for multi-view texture video and depth map coding
CN102740081B (en) Method for controlling transmission errors of multiview video based on distributed coding technology
CN103533330A (en) Multi-viewpoint multi-description video encoding method based on data reusing
CN103188500B (en) Encoding method for multi-view video signals
CN103391439A (en) H.264/AVC code rate control method based on active macroblock concealment
CN105430406A (en) Distributed video coding framework
CN106657961A (en) Hybrid digital-analog video coding for stereoscopic video
CN102333230A (en) Method for improving quality of synthetized virtual views in three-dimensional video system
CN104539971A (en) Inter-viewpoint bit allocation method applied to two-viewpoint video coding
CN104519365A (en) Inter-viewpoint bit allocation method applied to three-viewpoint video coding

Legal Events

Date Code Title Description
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170630

CF01 Termination of patent right due to non-payment of annual fee