CN105120290B - A kind of deep video fast encoding method - Google Patents
A kind of deep video fast encoding method Download PDFInfo
- Publication number
- CN105120290B CN105120290B CN201510470699.2A CN201510470699A CN105120290B CN 105120290 B CN105120290 B CN 105120290B CN 201510470699 A CN201510470699 A CN 201510470699A CN 105120290 B CN105120290 B CN 105120290B
- Authority
- CN
- China
- Prior art keywords
- pixel
- present frame
- current
- frame
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of deep video fast encoding method, it first carries out airspace enhancement to deep video sequence and pretreatment is completed in time domain enhancing, every frame in pretreated deep video sequence is encoded again, in an encoding process, directly encoded for the 1st frame and last frame using 3D HEVC original coding platforms；For remaining frame, the area type according to belonging to coding unit, the species for the HEVC coding prediction modes for needing to travel through is reduced, so that in the case where ensureing virtual viewpoint rendering quality and coding distortion performance, significantly reduces the complexity of coding.
Description
Technical field
It is fast more particularly, to a kind of deep video towards 3D-HEVC the present invention relates to a kind of video signal processing method
Fast coding method.
Background technology
With developing rapidly for computer, communication and multimedia technology, user to such as 3D TVs, free view-point TV,
The three-dimensional video system that three-dimensional scenic renders is more and more interested.Multiple views plus depth video (MVD) represents three-dimensional scenic
Main flow representation, MVD include multiple views color video and deep video.In scene display, virtual view by based on
DIBR rendering technique is drawn to obtain virtual view, and MVD data modes meet the demand of three-dimensional video system and can propped up
Hold wide viewing angle 3D display and autostereoscopic display.But MVD includes huge data volume so that the storages of data and network
Transmission faces huge challenge, and therefore, the deep videos of multiple views is as color video, it is necessary to quickly and efficiently be pressed
Contracting.
Color video represents the actual visual information of scene, and corresponding deep video represents the geological information of scene.Mesh
The mode of the preceding Depth Information Acquistion for real scene mainly has three kinds：Kinect sensor, depth camera system and depth
Estimation Software.Due to the limitation of these acquiring technologies, there is inaccurate and discontinuous phenomenon in deep video, particularly flat
Region there is pixel mutation, in the regional depth boundary alignment mistake of corresponding scene boundary, even if this is resulted in using newest
Coded system be also unable to reach optimal encoding efficiency.Many experts are proposed towards deep video processing H.264/AVC
Scheme, but these processing methods are not particularly suited for newest multi-vision-point encoding scheme 3D-HEVC.
The encoder complexity of deep video is also the problem of research needs concern.At present, regarded by ISO and the ITU 3D formed
Frequency coding joint specialist group (JCT-3V) is just being directed to the 3D extended codings framework (3D-HEVC) of efficient video coding (HEVC)
Standardization, 3D-HEVC overall coding framework are as shown in Figure 1.3D-HEVC still uses HEVC Quadtree Partition structure, compiles
Corresponding deep video frame is re-encoded after one frame color video frame of code.Multi-view depth video is divided into a series of code trees
Unit (Coding Tree Units, CTUs), each CTU is divided into the coding of four formed objects in a manner of recursive subdivision
Unit (CodingUnit, CU), it, which splits depth, to be followed successively by 64 × 64,32 according to segmentation depth for 0,1,2,3, CU sizes
× 32,16 × 16,8 × 8, wherein, maximum CU is referred to as maximum coding unit (Largest CodingUnit, LCU).Each CU
Different predicting unit (PredictionUnits, PU) is divided into according to the predictive mode of selection, predictive mode includes SKIP moulds
Formula, merge patterns, interframe inter (AMP, interN_2N, inter2N_N, inter2N_2N) pattern, intra in frame
(intraN_N, intra2N_2N) pattern, during being all employed for inter prediction and interview prediction.Institute is calculated in traversal
There are the partitioning scheme of coding unit and the rate-distortion optimization (Rate-DistortionOptimization, RDO) of predictive mode
Afterwards, a kind of minimum predictive mode of rate distortion costs value is selected as final predictive mode, meanwhile, determine coding unit
Partitioning scheme, the computation complexity of whole process are very high.The formula of calculating rate distortion costs value is in 3D-HEVC：
J=(wsynth×Dsynth+wdepth×Ddepth)+λ × R, wherein, J represents rate distortion costs value, DsynthRepresent to pass through
The distortion value of the virtual view of drafting, or the distortion value for the deep video estimated according to drawing principle, DdepthRepresent that depth regards
The distortion value of frequency, wsynthFor DsynthWeight, wdepthFor DdepthWeight, λ is Lagrange's multiplier, and R represents each prediction mould
The code check encoded under formula.
At present, many experts are studied deep video fast coding, as Shen et al. proposes a kind of profit
With the method for relation between the information such as the predictive mode of color video and deep video, motion vector, encoder complexity is reduced；
And for example More et al. proposes a kind of information using coloud coding, limits the Quadtree Partition and predictive mode of deep video
Fast coding algorithm.But these methods do not account for deep video and inaccurate and non-continuous event, this phenomenon meeting be present
The complexity of increase coding indirectly, therefore, it is necessary to study a kind of this phenomenon increase encoder complexity problem of can solve the problem that
Fast encoding method.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of deep video fast encoding method towards 3D-HEVC,
It can be effectively reduced encoder complexity on the premise of ensureing to draw virtual view quality and coding distortion performance.
Technical scheme is used by the present invention solves above-mentioned technical problem：A kind of deep video fast encoding method, its
It is characterised by comprising the following steps：
1. pending high definition deep video sequence is defined as current depth video sequence；
2. being pre-processed to current depth video sequence, pretreatment includes two parts, and Part I is：To current depth
Remaining in video sequence in addition to the 1st frame depth image and last frame depth image carries out airspace enhancement per frame depth image,
The borderline region of depth image and non-borderline region corresponding to being determined during airspace enhancement；Part II is：Spatial domain is increased
Deep video sequence after the reason of strength carries out time domain enhancing；
3. depth image currently pending in pretreated deep video sequence is defined as present frame；
4. judge whether present frame is that the 1st frame depth image in pretreated deep video sequence or last frame are deep
Image is spent, if it is, directly being encoded using 3D-HEVC original coding platforms to present frame, then performs step 7.；It is no
Then, step is performed 5.；
5. obtain moving region and the non-moving areas of present frame；And in the current frame, with step 2. middle acquisition with work as
Borderline region of the corresponding region of borderline region of depth image corresponding to previous frame as present frame, with step 2. middle acquisition with
Non- borderline region of the corresponding region of non-borderline region of depth image corresponding to present frame as present frame；
6. determine whether each coding unit in present frame had not only belonged to borderline region but also belonged to moving region, for both belonging to
Belong to the coding unit of moving region again in borderline region, travel through all HEVC coding prediction modes, select rate distortion costs value
Optimal prediction modes of the minimum HEVC coding prediction modes as current coded unit；For other coding units, only travel through
SKIP patterns and Merge patterns in all HEVC coding prediction modes, select the minimum pattern of rate distortion costs value and be used as and work as
The optimal prediction modes of preceding coding unit；Then the optimal prediction modes of each coding unit in present frame are utilized to present frame
In each coding unit encoded；
7. using the pending depth image of next frame in pretreated deep video sequence as present frame, it is then back to
4. step continues executing with, until all depth images coding in pretreated deep video sequence finishes.
The detailed process of described step 2. middle Part I is：2. -1a, it currently will wait to locate in current depth video sequence
The depth image of reason is defined as present frame；If 2. -1b, present frame be current depth video sequence in the 1st frame depth image or
Last frame depth image, then present frame is not dealt with；If present frame is not the 1st frame depth in current depth video sequence
Image or last frame depth image, then gaussian filtering is used to all fracture pixels for belonging to non-borderline region in present frame
Device is filtered processing, and all non-fracture pixels for belonging to non-borderline region in present frame are filtered using self-adapting window
Ripple processing, to all fracture pixels for belonging to borderline region in present frame and all non-fracture pixels for belonging to borderline region
Do not deal with；2. -1c, using the pending depth image of next frame in current depth video sequence as present frame, is then returned
Returning step, 2. -1b is continued executing with, until all depth images in current depth video sequence are disposed, completes current depth
The airspace enhancement of remaining every frame depth image in video sequence in addition to the 1st frame depth image and last frame depth image；
The detailed process of described step 2. middle Part II is：2. the depth in -2a, hypothesis current depth video sequence
The width and height of image correspond to W and H, and the totalframes for the depth image for assuming to include in current depth video sequence is T
Frame；2. -2b, carrying out space-time transformation to the deep video sequence after airspace enhancement processing, the deep video after space-time transformation is obtained
Sequence, the picture by coordinate position in the jth frame depth image in the deep video sequence after space-time transformation for the pixel of (i, t)
Plain value is designated as dj(i, t), dj(i, t)=dt(i, j), wherein, 1≤j≤H, 1≤i≤W, 1≤t≤T, dt(i, j) represents spatial domain
Coordinate position is the pixel value of the pixel of (i, j) in the t frame depth images in deep video sequence after enhancing processing；②-
2c, time domain enhancing processing is carried out using time-domain filtering window to every frame depth image in the deep video sequence after space-time transformation,
The deep video sequence after time domain enhancing processing is obtained, the jth frame depth map in deep video sequence after time domain enhancing is handled
Coordinate position is designated as d' for the pixel value of the pixel of (i, t) as inj(i, t),Its
In, t0Represent the size of time-domain filtering window, dj(i, t') represents the jth frame depth in the deep video sequence after space-time transformation
In image coordinate position for (i, t') pixel pixel value, wj(i, t') represents djThe weight of (i, t')；2. -2d, to time domain
Deep video sequence after enhancing processing carries out space-time transformation, obtains pretreated deep video sequence, will be pretreated
Coordinate position is designated as d' for the pixel value of the pixel of (i, j) in t frame depth images in deep video sequencet(i, j), d't
(i, j)=d'j(i,t)。
2. the determination process of the fracture pixel in -1b in present frame and non-fracture pixel is described step：Make cr
(i, j) represents that coordinate position is the fracture mark of the pixel of (i, j) in present frame；Then each pixel in present frame
The coordinate position and pixel value of point, the fracture mark of each pixel in present frame is determined, for coordinate position in present frame
For the pixel of (i, j),Then according to
The fracture mark of pixel in present frame, the scope for the fracture mark that expansion value is 1, detailed process are：For in present frame
Coordinate position is the pixel of (i', j'), if the value of the fracture mark of the pixel is 1, when coordinate position is in present frame
It is the pixel of (i'-1, j') by coordinate position in present frame when the value of the fracture mark of the pixel of (i'-1, j') is 0
The value of fracture mark is changed to 1；The pixel that the value that mark is broken in present frame is 0 is defined as non-fracture pixel again, will
The pixel that the value that mark is broken in present frame is 1 is defined as being broken pixel；
Wherein, cr (i, j) value is 0 or 1, and 1≤i≤W, 1≤j≤H, W and H correspondingly represent current depth video sequence
In depth image width and height, dp (i-1, j) represent present frame in coordinate position for (i-1, j) pixel pixel
Value, dp (i, j) represent that coordinate position is the pixel value of the pixel of (i, j) in present frame, and symbol " | | " is the symbol that takes absolute value,
Th0For the fracture judgment threshold of setting, 2≤i'≤W, 1≤j'≤H.
The fracture judgment threshold Th of described setting0Value is 10.
Described step 2. in -1b the borderline region of present frame and non-borderline region by using Canny border detection sides
Method carries out border detection to present frame and obtained.
High threshold th employed in described Canny boundary detection methodsHValue is 120, and used Low threshold
thLValue is 40.
Described step 2. in -1b to all non-fracture pixels for belonging to non-borderline region in present frame using adaptive
The detailed process that window is filtered processing is：Any one non-fracture pixel for belonging to non-borderline region in present frame,
Using the non-fracture pixel as central pixel point, and using n pixel as step-size in search progress is upper and lower and Left-right Searching, when searching
Rope is to the fracture pixel for belonging to non-borderline region or the fracture pixel for belonging to borderline region or belongs to the non-disconnected of borderline region
Split pixel or stopped search when searching up to the image boundary of present frame, form a crossing window；Then with the cross window
Centered on each pixel on the mouth longitudinal axis, and Left-right Searching is carried out by step-size in search of n pixel, belong to non-when searching
The fracture pixel of borderline region or belong to the fracture pixel of borderline region or belong to borderline region non-fracture pixel or
Stopped search when searching up to the image boundary of present frame, form a self-adapting window；Again by the intraoral institute of the adaptive windows
The average for having the pixel value of pixel is assigned to central pixel point and realizes filtering process.
Described step 2. in -2c time-domain filtering window size t0Value is 5；Described step is 2. in -2cdifj(i, t')=| dj(i,t'+1)-dj(i, t') |, dj(i, t'+1) is represented
Coordinate position is the pixel of the pixel of (i, t'+1) in jth frame depth image in deep video sequence after space-time transformation
Value, symbol " | | " it is the symbol that takes absolute value.
5. the detailed process of the middle moving region for obtaining present frame and non-moving areas is described step：
5. -1, assume current depth video sequence in depth image width and height correspond to W and H, and assume W ×
H can be divided exactly by 4 × 4, then be divided into coloured image corresponding to present frameThe sub-block of individual non-overlapping copies；
5. h-th of sub-block currently pending in coloured image corresponding to present frame -2, is defined as current sub-block, its
In, h initial value is 1,
5. -3, calculate the difference of corresponding region in the previous color image frame of current sub-block coloured image corresponding with present frame
Quadratic sum, it is designated as SSDpre,h,And calculate current sub-block coloured image corresponding with present frame
Latter color image frame in corresponding region squared difference and, be designated as SSDback,h,
Wherein, Ccur(u, v) represents that coordinate position is the pixel value of the pixel of (u, v), C in current sub-blockpre(u, v) represents present frame
Coordinate position is the pixel of (u, v) in region corresponding with current sub-block in the previous color image frame of corresponding coloured image
Pixel value, Cback(u, v) represents corresponding with current sub-block in the latter color image frame of coloured image corresponding to present frame
Coordinate position is the pixel value of the pixel of (u, v) in region, and symbol " | | " is the symbol that takes absolute value；
5. -4, judge min (SSDpre,h,SSDback,h)<Whether Th sets up, if set up, it is determined that current sub-block is non-fortune
Mover block, otherwise, it determines current sub-block is moving sub-block；Wherein, for min () to take minimum value function, Th is motion of setting
Block judgment threshold；
5. h=h+1-5, is made, then using next pending sub-block in coloured image corresponding to present frame as current
Sub-block, return again to step and 5. -3 continue executing with, until all sub-blocks in coloured image corresponding to present frame are disposed, its
In, "=" in h=h+1 is assignment；
5. -6, region corresponding to all moving sub-blocks in coloured image corresponding in present frame is defined as currently
The moving region of frame, region corresponding to all non-athletic sub-blocks in coloured image corresponding in present frame is defined as working as
The non-moving areas of previous frame.
The detailed process of described step 6. is：
6. maximum coding unit currently pending in present frame -1, is defined as current maximum coding unit；
6. coding unit currently pending in current maximum coding unit -2, is defined as current coded unit, and will
Depth where current coded unit is defined as current depth；
If 6. the pixel for the borderline region for belonging to present frame in -3, current coded unit be present, and present encoding list
The pixel for the moving region for belonging to present frame in member be present, then it is assumed that current coded unit had not only belonged to borderline region but also belonged to fortune
Dynamic region, travels through all HEVC coding prediction modes, calculates rate of the current coded unit under every kind of HEVC coding prediction modes
Distortion cost value, then during rate-distortion optimization, select the minimum HEVC coding prediction mode conducts of rate distortion costs value
Optimal prediction modes of current coded unit, and obtain the rate distortion costs value under optimal prediction modes, then perform step 6.-
4；
If exist in current coded unit in the pixel for the borderline region for belonging to present frame and current coded unit not
In the presence of the pixel for the moving region for belonging to present frame, or the borderline region for belonging to present frame is not present in current coded unit
The pixel for the moving region for belonging to present frame in pixel and current coded unit be present, or be not present in current coded unit
Belong to the pixel that the moving region for belonging to present frame is not present in the pixel of the borderline region of present frame and current coded unit
Point, then SKIP patterns and Merge patterns in all HEVC coding prediction modes are only traveled through, calculate current coded unit in SKIP
Respective rate distortion costs value under pattern and Merge patterns, then during rate-distortion optimization, selects rate distortion costs value most
Optimal prediction modes of the small pattern as current coded unit, and the rate distortion costs value under optimal prediction modes is obtained,
Step 6. -4 is performed again；
6. -4, compare the most optimal sorting of coding unit corresponding with current coded unit in coloured image corresponding to present frame
Depth and the size of current depth are cut, if the former is more than the latter, performs step 6. -5；After if the former is less than or equal to
Person, then the optimum segmentation depth using current depth as current coded unit, then perform step 6. -6；
6. -5, according to step 6. -3 process, in an identical manner obtain current coded unit where layer next layer in
The respective optimal prediction modes of four size identical coding units and the rate distortion costs under optimal prediction modes
Value, then compare the rate distortion costs value sums of four coding units in next layer of layer where current coded unit with it is current
The size of rate distortion costs value of the coding unit under optimal prediction modes, if the former is more than the latter, current depth is made
For the optimum segmentation depth of current coded unit, to current coded unit without next layer of segmentation, then perform step 6.-
6；If the former is less than or equal to the latter, next layer of segmentation is carried out to current coded unit, then by current coded unit
Currently pending coding unit is as current coded unit in four coding units in next layer of place layer, and incites somebody to action current
Depth where coding unit returns again to step and 6. -4 continued executing with as current depth；
6. -6, current coded unit is encoded using the optimal prediction modes of current coded unit, in present encoding
After cell encoding terminates, judge whether all coding units in current maximum coding unit are disposed, if it is, really
Settled preceding maximum coding unit end-of-encode, then perform step 6. -7；Otherwise, then judge in the layer of current coded unit place
Whether four size identical coding units are disposed, and when being disposed, judge current coded unit place
Whether the coding unit in the last layer of layer is maximum coding unit, if maximum coding unit, then by current coded unit
Then 6. -7 maximum coding unit in the last layer of place layer performs step as current maximum coding unit, if not
Maximum coding unit, then compiled using next pending coding unit in the last layer of layer where current coded unit as current
Code unit, using the depth where the current coded unit as current depth, is then back to step and 6. -3 continues executing with；
6. -7, judge whether current maximum coding unit is last maximum coding unit in present frame, if it is,
Then perform step 7., otherwise, using next pending maximum coding unit in present frame as current maximum coding unit, so
Afterwards 6. return to step -2 continues executing with.
Compared with prior art, the advantage of the invention is that：
1) the inventive method first carries out airspace enhancement to deep video sequence and pretreatment is completed in time domain enhancing, then to pre- place
Every frame in deep video sequence after reason is encoded, and in an encoding process, is directly used for the 1st frame and last frame
3D-HEVC original coding platforms are encoded；For remaining frame, the area type according to belonging to coding unit, needs time are reduced
The species for the HEVC coding prediction modes gone through, so as to ensure virtual viewpoint rendering quality and encode the situation of distortion performance
Under, significantly reduce the complexity of coding.
2) during deep video sequence airspace enhancement, by using different filters to the pixel in different zones
Wave method processing so that the fringe region that drawing process has a great influence has been obtained in the deep video sequence after airspace enhancement compared with
Good protection, meanwhile, other regions have been carried out it is a certain degree of smooth, so as to ensure that the rendering quality of final virtual view
With coding distortion performance.
3) during deep video sequence time domain strengthens, original depth video sequence time-space domain is changed, to conversion
Every frame afterwards carries out time-domain filtering so that original depth video sequence obtains smoothly in time domain direction, adjacent between frame and frame
The correlation of pixel is strengthened.
Brief description of the drawings
Fig. 1 is the overall coding framework for the multiple views colour plus depth encoder 3D-HEVC that the inventive method uses；
Fig. 2 is that the totality of the inventive method realizes block diagram；
Fig. 3 a are the frame depth image in cycle tests Newspaper1, and the inframe in figure shows original depth image
The partial enlarged drawing of different zones；
Fig. 3 b be the depth image shown in Fig. 3 a it is preprocessed after obtained design sketch, the inframe in figure shows pre- place
The partial enlarged drawing of depth image different zones after reason；
Fig. 4 be cycle tests after pretreatment when being encoded on test model HTM10.0, different coding quantization parameter QP
Under the obtained statistics of code check situation of change；
Fig. 5 be cycle tests after pretreatment when being encoded on test model HTM10.0, different coding quantization parameter QP
Under the change of obtained drafting PSNR compared with original drafting PSNR；
Fig. 6 is the deep video after pretreatment, after being encoded on test model HTM10.0, finally non-athletic
The statistical distribution of the predictive mode of region and non-borderline region.
Embodiment
The present invention is described in further detail below in conjunction with accompanying drawing embodiment.
A kind of deep video fast encoding method towards 3D-HEVC proposed by the present invention, Fig. 1 give the inventive method
The multiple views colour plus depth encoder 3D-HEVC of use overall coding framework, the totality of the inventive method realize block diagram such as
Shown in Fig. 2, the inventive method comprises the following steps：
1. pending high definition deep video sequence is defined as current depth video sequence.
2. being pre-processed to current depth video sequence, pretreatment includes two parts, and Part I is：To current depth
Remaining in video sequence in addition to the 1st frame depth image and last frame depth image carries out airspace enhancement per frame depth image,
The borderline region of depth image and non-borderline region corresponding to being determined during airspace enhancement；Part II is：Spatial domain is increased
Deep video sequence after the reason of strength carries out time domain enhancing.
In this particular embodiment, the detailed process of step 2. middle Part I is：2. -1a, by current depth video sequence
Currently pending depth image is defined as present frame in row.2. if -1b, present frame are the 1st in current depth video sequence
Frame depth image or last frame depth image, then do not deal with to present frame；If present frame is not current depth video sequence
In the 1st frame depth image or last frame depth image, then to all fracture pixels for belonging to non-borderline region in present frame
Point is filtered processing using Gaussian filter, and all non-fracture pixels for belonging to non-borderline region in present frame are used certainly
Adapt to window and be filtered processing, to all fracture pixels for belonging to borderline region in present frame and all belong to borderline region
Non- fracture pixel do not deal with.2. -1c, using the pending depth image of next frame in current depth video sequence as
Present frame, being then back to step, 2. -1b is continued executing with, until all depth images in current depth video sequence have been handled
Finish, complete remaining in current depth video sequence in addition to the 1st frame depth image and last frame depth image per frame depth map
The airspace enhancement of picture.
Here, 2. the determination process of the fracture pixel in -1b in present frame and non-fracture pixel is step：Make cr
(i, j) represents that coordinate position is the fracture mark of the pixel of (i, j) in present frame；Then each pixel in present frame
The coordinate position and pixel value of point, the fracture mark of each pixel in present frame is determined, for coordinate position in present frame
For the pixel of (i, j),Then according to
The fracture mark of pixel in present frame, the scope for the fracture mark that expansion value is 1, detailed process are：For in present frame
Coordinate position is the pixel of (i', j'), if the value of the fracture mark of the pixel is 1, when coordinate position is in present frame
It is the pixel of (i'-1, j') by coordinate position in present frame when the value of the fracture mark of the pixel of (i'-1, j') is 0
Fracture mark value be changed to 1, when in present frame coordinate position be the pixel of (i'-1, j') fracture mark value be 1 when,
Coordinate position in present frame is kept constant for the value that the fracture of the pixel of (i'-1, j') marks；It will be broken again in present frame
The pixel that the value of mark is 0 is defined as non-fracture pixel, and the pixel that the value that mark is broken in present frame is 1 is defined as
It is broken pixel；Wherein, cr (i, j) value is 0 or 1, and 1≤i≤W, 1≤j≤H, W and H correspondingly represent current depth video
The width and height of depth image in sequence, dp (i-1, j) represent that coordinate position is the pixel of (i-1, j) in present frame
Pixel value, dp (i, j) represent the pixel value of coordinate position in present frame for the pixel of (i, j), and symbol " | | " is takes absolute value
Symbol, Th0For the fracture judgment threshold of setting, Th is taken in the present embodiment0=10,2≤i'≤W, 1≤j'≤H.
Here, step 2. in -1b the borderline region of present frame and non-borderline region by using existing Canny border detections
Method carries out border detection to present frame and obtained, wherein, the high threshold th employed in Canny boundary detection methodsHValue is
120, and used Low threshold thLValue is 40.Detailed process is：Gaussian filtering process is carried out to present frame；Then filter is obtained
Coordinate position is the first-order difference of the pixel of (i, j) in the horizontal direction and the vertical direction in depth image after ripple processing, right
G should be designated asx(i, j) and gy(i,j)；Then it is the ladder of the pixel of (i, j) to calculate coordinate position in the depth image after filtering process
Deflection and gradient magnitude are spent, it is corresponding to be designated as θ (i, j) and gt(i, j),
Non-maxima suppression is carried out to the gradient magnitude of each pixel in the depth image after the filtering process tried to achieve afterwards, for
Coordinate position is the pixel of (i, j) in depth image after filtering process, by gt(i, j) is closest with θ (i, j) both ends respectively
The gradient magnitude g1 of pixelt(i,j)、g2t(i, j) compares, if gt(i,j)>g1t(i, j) and gt(i,j)>g2t(i, j), then will
gt(i, j) is set to 1, otherwise, by gt(i, j) is set to 0；Border detection is carried out further according to the dual threshold of setting, obtains the side of present frame
Battery limit (BL) domain and non-borderline region.
Here, step 2. in -1b to all non-fracture pixels for belonging to non-borderline region in present frame using adaptive
The detailed process that window is filtered processing is：Any one non-fracture pixel for belonging to non-borderline region in present frame,
Using the non-fracture pixel as central pixel point, and using n pixel as step-size in search progress is upper and lower and Left-right Searching, when searching
Rope is to the fracture pixel for belonging to non-borderline region or the fracture pixel for belonging to borderline region or belongs to the non-disconnected of borderline region
Split pixel or stopped search when searching up to the image boundary of present frame, form a crossing window；Then with the cross window
Centered on each pixel on the mouth longitudinal axis, and Left-right Searching is carried out by step-size in search of n pixel, belong to non-when searching
The fracture pixel of borderline region or belong to the fracture pixel of borderline region or belong to borderline region non-fracture pixel or
Stopped search when searching up to the image boundary of present frame, form a self-adapting window；Again by the intraoral institute of the adaptive windows
The average for having the pixel value of pixel is assigned to central pixel point and realizes filtering process.N=5 is taken in the present embodiment.
In this particular embodiment, the detailed process of step 2. middle Part II is：2. -2a, assume current depth video
The width and height of depth image in sequence correspond to W and H, and assume the depth image included in current depth video sequence
Totalframes be T frames, take T=60 in the present embodiment.2. -2b, space-time is carried out to the deep video sequence after airspace enhancement processing
Conversion, obtains the deep video sequence after space-time transformation, by the jth frame depth image in the deep video sequence after space-time transformation
Middle coordinate position is designated as d for the pixel value of the pixel of (i, t)j(i, t), dj(i, t)=dt(i, j), wherein, 1≤j≤H, 1≤
I≤W, 1≤t≤T, dt(i, j) represents coordinate bit in the t frame depth images in the deep video sequence after airspace enhancement processing
It is set to the pixel value of the pixel of (i, j).In actual process, the deep video sequence after first airspace enhancement is handled regards
For a three-dimensional system of coordinate, using the three-dimensional system of coordinate as former three-dimensional system of coordinate, the x-axis of former three-dimensional system of coordinate is at airspace enhancement
The width of the depth image in deep video sequence after reason, y-axis are in the deep video sequence after airspace enhancement processing
The short transverse of depth image, the time orientation that t axles are the depth image in the deep video sequence after airspace enhancement processing；
Then space-time transformation is carried out to former three-dimensional system of coordinate, obtains new three-dimensional system of coordinate, the x-axis of new three-dimensional system of coordinate is at airspace enhancement
The width of the depth image in deep video sequence after reason, y-axis are in the deep video sequence after airspace enhancement processing
The time orientation of depth image, the short transverse that t axles are the depth image in the deep video sequence after airspace enhancement processing；Newly
Include H frame depth images, deep video sequence corresponding to new three-dimensional system of coordinate in deep video sequence corresponding to three-dimensional system of coordinate
In depth image width and height correspond to W and T.2. -2c, to every frame depth in the deep video sequence after space-time transformation
Image carries out time domain enhancing processing using time-domain filtering window, obtains the deep video sequence after time domain enhancing processing, time domain is increased
Coordinate position is designated as d' for the pixel value of the pixel of (i, t) in the jth frame depth image in deep video sequence after the reason of strengthj
(i, t),Wherein, t0The size of time-domain filtering window is represented, is taken in the present embodiment
t0=5, dj(i, t') represents that coordinate position is (i, t') in the jth frame depth image in the deep video sequence after space-time transformation
Pixel pixel value, wj(i, t') represents djThe weight of (i, t'),difj
(i, t')=| dj(i,t'+1)-dj(i, t') |, dj(i, t'+1) represents the jth frame in the deep video sequence after space-time transformation
Coordinate position is the pixel value of the pixel of (i, t'+1) in depth image, and symbol " | | " is the symbol that takes absolute value.It is 2. -2d, right
Deep video sequence after time domain enhancing processing carries out space-time transformation, obtains pretreated deep video sequence, will pre-process
Coordinate position is designated as d' for the pixel value of the pixel of (i, j) in t frame depth images in deep video sequence afterwardst(i,
J), d't(i, j)=d'j(i,t)。
3. depth image currently pending in pretreated deep video sequence is defined as present frame.
4. judge whether present frame is that the 1st frame depth image in pretreated deep video sequence or last frame are deep
Image is spent, if it is, directly being encoded using 3D-HEVC original coding platforms to present frame, then performs step 7.；It is no
Then, step is performed 5..
5. obtain moving region and the non-moving areas of present frame；And in the current frame, with step 2. middle acquisition with work as
Borderline region of the corresponding region of borderline region of depth image corresponding to previous frame as present frame, with step 2. middle acquisition with
Non- borderline region of the corresponding region of non-borderline region of depth image corresponding to present frame as present frame.
In this particular embodiment, step 5. it is middle obtain present frame moving region and non-moving areas detailed process
For：
5. -1, assume current depth video sequence in depth image width and height correspond to W and H, and assume W ×
H can be divided exactly by 4 × 4, then be divided into coloured image corresponding to present frameThe sub-block of individual non-overlapping copies.
5. h-th of sub-block currently pending in coloured image corresponding to present frame -2, is defined as current sub-block, its
In, h initial value is 1,
5. -3, calculate the difference of corresponding region in the previous color image frame of current sub-block coloured image corresponding with present frame
Quadratic sum, it is designated as SSDpre,h,And calculate current sub-block coloured image corresponding with present frame
Latter color image frame in corresponding region squared difference and, be designated as SSDback,h,
Wherein, Ccur(u, v) represents that coordinate position is the pixel value of the pixel of (u, v), C in current sub-blockpre(u, v) represents present frame
Coordinate position is the pixel of (u, v) in region corresponding with current sub-block in the previous color image frame of corresponding coloured image
Pixel value, Cback(u, v) represents corresponding with current sub-block in the latter color image frame of coloured image corresponding to present frame
Coordinate position is the pixel value of the pixel of (u, v) in region, and symbol " | | " is the symbol that takes absolute value.
5. -4, judge min (SSDpre,h,SSDback,h)<Whether Th sets up, if set up, it is determined that current sub-block is non-fortune
Mover block, otherwise, it determines current sub-block is moving sub-block；Wherein, for min () to take minimum value function, Th is motion of setting
Block judgment threshold, takes Th=1000 in the present embodiment.
5. h=h+1-5, is made, then using next pending sub-block in coloured image corresponding to present frame as current
Sub-block, return again to step and 5. -3 continue executing with, until all sub-blocks in coloured image corresponding to present frame are disposed, its
In, "=" in h=h+1 is assignment.
5. -6, region corresponding to all moving sub-blocks in coloured image corresponding in present frame is defined as currently
The moving region of frame, region corresponding to all non-athletic sub-blocks in coloured image corresponding in present frame is defined as working as
The non-moving areas of previous frame.
6. determine whether each coding unit in present frame had not only belonged to borderline region but also belonged to moving region, for both belonging to
Belong to the coding unit of moving region again in borderline region, travel through all HEVC coding prediction modes, select rate distortion costs value
Optimal prediction modes of the minimum HEVC coding prediction modes as current coded unit；For other coding units, only travel through
SKIP patterns and Merge patterns in all HEVC coding prediction modes, select the minimum pattern of rate distortion costs value and be used as and work as
The optimal prediction modes of preceding coding unit；Then the optimal prediction modes of each coding unit in present frame are utilized to present frame
In each coding unit encoded.
In this particular embodiment, the detailed process of step 6. is：
6. maximum coding unit currently pending in present frame -1, is defined as current maximum coding unit.
6. coding unit currently pending in current maximum coding unit -2, is defined as current coded unit, and will
Depth where current coded unit is defined as current depth.
If 6. the pixel for the borderline region for belonging to present frame in -3, current coded unit be present, and present encoding list
The pixel for the moving region for belonging to present frame in member be present, then it is assumed that current coded unit had not only belonged to borderline region but also belonged to fortune
Dynamic region, travels through all HEVC coding prediction modes, calculates rate of the current coded unit under every kind of HEVC coding prediction modes
Distortion cost value, then during rate-distortion optimization, select the minimum HEVC coding prediction mode conducts of rate distortion costs value
Optimal prediction modes of current coded unit, and obtain the rate distortion costs value under optimal prediction modes, then perform step 6.-
4；
If exist in current coded unit in the pixel for the borderline region for belonging to present frame and current coded unit not
In the presence of the pixel for the moving region for belonging to present frame, or the borderline region for belonging to present frame is not present in current coded unit
The pixel for the moving region for belonging to present frame in pixel and current coded unit be present, or be not present in current coded unit
Belong to the pixel that the moving region for belonging to present frame is not present in the pixel of the borderline region of present frame and current coded unit
Point, then SKIP patterns and Merge patterns in all HEVC coding prediction modes are only traveled through, calculate current coded unit in SKIP
Respective rate distortion costs value under pattern and Merge patterns, then during rate-distortion optimization, selects rate distortion costs value most
Optimal prediction modes of the small pattern as current coded unit, and the rate distortion costs value under optimal prediction modes is obtained,
Step 6. -4 is performed again.
6. -4, compare the most optimal sorting of coding unit corresponding with current coded unit in coloured image corresponding to present frame
Depth and the size of current depth are cut, if the former is more than the latter, performs step 6. -5；After if the former is less than or equal to
Person, then the optimum segmentation depth using current depth as current coded unit, then perform step 6. -6.
6. -5, according to step 6. -3 process, in an identical manner obtain current coded unit where layer next layer in
The respective optimal prediction modes of four size identical coding units and the rate distortion costs under optimal prediction modes
Value, then compare the rate distortion costs value sums of four coding units in next layer of layer where current coded unit with it is current
The size of rate distortion costs value of the coding unit under optimal prediction modes, if the former is more than the latter, current depth is made
For the optimum segmentation depth of current coded unit, to current coded unit without next layer of segmentation, then perform step 6.-
6；If the former is less than or equal to the latter, next layer of segmentation is carried out to current coded unit, then by current coded unit
Currently pending coding unit is as current coded unit in four coding units in next layer of place layer, and incites somebody to action current
Depth where coding unit returns again to step and 6. -4 continued executing with as current depth.
6. -6, current coded unit is encoded using the optimal prediction modes of current coded unit, in present encoding
After cell encoding terminates, judge whether all coding units in current maximum coding unit are disposed, if it is, really
Settled preceding maximum coding unit end-of-encode, then perform step 6. -7；Otherwise, then judge in the layer of current coded unit place
Whether four size identical coding units are disposed, and when being disposed, judge current coded unit place
Whether the coding unit in the last layer of layer is maximum coding unit, if maximum coding unit, then by current coded unit
Then 6. -7 maximum coding unit in the last layer of place layer performs step as current maximum coding unit, if not
Maximum coding unit, then compiled using next pending coding unit in the last layer of layer where current coded unit as current
Code unit, using the depth where the current coded unit as current depth, is then back to step and 6. -3 continues executing with.
6. -7, judge whether current maximum coding unit is last maximum coding unit in present frame, if it is,
Then perform step 7., otherwise, using next pending maximum coding unit in present frame as current maximum coding unit, so
Afterwards 6. return to step -2 continues executing with.
7. using the pending depth image of next frame in pretreated deep video sequence as present frame, it is then back to
4. step continues executing with, until all depth images coding in pretreated deep video sequence finishes.
Below to test the inventive method, to illustrate the validity of the inventive method and feasibility.
The inventive method mainly carries out performance test on 3D-HEVC test models HTM10.0.The test environment of experiment is set
JCT-3V universal test environment is set to, basic parameter is as shown in table 1, and cycle tests and the viewpoint used are as shown in table 2,
Poznan_Street sequences are provided by Poznan Universities of Science and Technology in table 2, Kendo and Balloons sequences are carried by Nagoya universities
For Newspaper1 is provided by Guangzhou Institute Of Science And Technology.
The test environment of table 1 is set
The cycle tests of table 2
| Cycle tests | Resolution ratio | Viewpoint | Draw viewpoint |
| Balloons | 1024×768 | 3-1 | 2 |
| Kendo | 1024×768 | 5-3 | 4 |
| Newspaper1 | 1024×768 | 4-2 | 3 |
| Poznan_Street | 1920×1088 | 3-5 | 4 |
Total time (color video and the depth that the encoder complexity change of the inventive method is spent using 3D-HEVC codings
The scramble time summation of video) reduction ratio Δ TproRepresent,TproRepresent the inventive method
Coding total time, ToriRepresent the coding total time of original coding platform.Equally, the scramble time of deep video reduces ratio
Using Δ tproRepresent, computational methods and Δ TproIt is identical.Distortion performance is encoded to represent using BDBR, wherein, preprocessing part
BDBR obtained using the Y-PSNR PSNR of the virtual view of drafting and the code check of coding depth video；Fast coding part
BDBR obtained using the Multi-scale model similarity MS-SSIM of virtual view and the total bitrate of encoded color video plus depth video
Go out.The change ratio Δ BR of encoder bit rateproRepresent,BRproRepresent present invention side
Encoder bit rate caused by method coding depth video, BRoriRepresent encoder bit rate caused by primary platform coding depth video.It is original
Encoder bit rate BR caused by plateau coding deep videooriAnd the coding total time T of original coding platformoriAll be by
Fast algorithm in HTM10.0, which is closed, to be tested, the source as initial data.
In order to illustrate the effect that deep video pre-processes in the inventive method and the influence to encoder bit rate, by original depth
The part of the frame of video one and part corresponding to the pretreatment afterwards of this frame are amplified contrast, also, original video is located with pre-
Test obtains encoder bit rate to video after reason on original HTM10.0 platforms respectively.Provide all depth by pretreatment
Δ BR of the Video coding compared with original depth Video codingproStatistical chart, and represent to draw virtual view mass change PSNR
Difference statistical chart.Meanwhile represented using BDBR under identical drafting virtual view quality PSNR, pretreated deep video
Relative to the change of original depth video frequency coding rate, to weigh the distortion performance of pretreated deep video coding.
Fig. 3 a give the frame depth image in Newspaper1 sequences, and Fig. 3 b give the depth image shown in Fig. 3 a
The design sketch obtained after preprocessed.In Fig. 3 a and Fig. 3 b left side frame be a part of non-borderline region enlarged drawing, right side
Frame be a part of borderline region enlarged drawing.Through comparing it is recognised that the picture of the borderline region of pretreated deep video
Element does not almost change, and for other regions, the region being especially broken generates different degrees of filter effect.
Fig. 4 gives cycle tests after pretreatment when being encoded on test model HTM10.0, and different coding quantifies ginseng
The statistics of the code check situation of change obtained under number QP.Fig. 4 can be seen that with coded quantization parameter QP increase, the code check of coding
The amplitude of reduction is reduced, and in the case that coded quantization parameter QP is 40, the code check of the coding of Balloons sequences is almost unchanged, and
The code check of Kendo sequences slightly rises.Generally, after pretreatment, the code check of coding all reaches not all cycle tests
The effect declined with degree, the code check that wherein Poznan_Street is encoded are reduced most.
Fig. 5 gives cycle tests after pretreatment when being encoded on test model HTM10.0, and different coding quantifies ginseng
Changes of the drafting PSNR obtained under number QP compared with original drafting PSNR.Wherein encode the deep video of use viewpoint and
The viewpoint of drafting such as table 2 is listed.Draw using the color video and deep video after coding.From figure 5 it can be seen that by
The PSNR that deep video after pretreatment is drawn has declined, but the amplitude declined is all below 0.1, Balloons sequences
The quality drawn is arranged then slightly to improve.When being encoded using different coding quantization parameter QP, overall rendering quality is almost kept not
Become.
Distortion performance of the preprocess method to coding is weighed to be comprehensive, table 3 gives deep video and passes through present invention side
After the pretreatment of method, BDBR situation of change after being encoded using HTM10.0 Raw encoders, from table 3 it is observed that through
Coding BDBR totally declines after crossing pretreatment, declines most obvious Poznan_Street sequence Bs DBR and has reached 13.38%.
Show deep video by pretreatment in the case where identical draws virtual view quality, it is necessary to less encoder bit rate,
Distortion performance improves.
Influence of the pretreatment to drafting virtual view BDBR in the inventive method of table 3
For the reasonability of predictive mode advance determination method in checking fast coding, Fig. 6 gives deep video and is in non-
Moving region and the predictive mode statistical result of non-borderline region.From fig. 6 it can be seen that the AMP moulds of all cycle tests
Formula selection percentage is both less than 1%, and the ratio of skip patterns is then less than more than 90% for the toatl proportion of other predictive modes choosing
10%, particularly with the deep video sequence for including large area flat site, such as Poznan_Street sequences, skip patterns
Ratio has reached more than 95%.As can be seen that carrying out the limitation of regional prediction pattern traversal to sequence, the effect of coding will be improved
Rate, while larger distortion will not occur.
In order to illustrate the binary encoding effect of the inventive method, table 4 gives the scramble time phase using the inventive method
Time than HTM10.0 Raw encoder coding reduces situation, it can be seen that when the inventive method obtains 45.53% totality
Between reduction, deep video then obtain 74.13% time save.With coded quantization parameter QP increase, scramble time
The ratio of saving is increased slightly.Coding deep video used is the deep video by pretreatment, is added in the scramble time
The time of deep video pretreatment, final overall coding saves the time and deep video coding to save the time be 44.24% He
72%.Show that the complexity of coding is effectively reduced.Table 5 give using the inventive method encode caused by code check phase
Than the situation of change of encoder bit rate caused by HTM10.0 Raw encoder.As can be seen from Table 5, with coded quantization parameter
QP increase, the code check fall of coding are reduced.To all cycle tests, the code check of coding has all declined, mean depth
The code check of Video coding have dropped 24.07%.Table 6 gives the inventive method coding depth video and original HTM10.0 codings
After deep video, different cycle tests and different coding quantization parameter, the PSNR and MS-SSIM of virtual view knot is drawn
Fruit.As can be seen from Table 6, PSNR and MS-SSIM that the inventive method influence coding draws viewpoint degree are smaller, under PSNR
For the amplitude of drop below 0.07, MS-SSIM only has Poznan_Street sequences under conditions of coded quantization parameter is 35 and 40
0.0006 is have dropped, the MS-SSIM of other sequences is held essentially constant.To consider encoder bit rate and drawing virtual view
Quality, so as to weigh the distortion performance of the inventive method.Table 7 give the final BD-MS-SSIM of the inventive method and
BDBR results, as can be seen from Table 7, the average BD-MS-SSIM of the inventive method improve 0.0002, and coding BDBR is averaged
It has dropped 1.65%.To sum up, the inventive method significantly reduces the complexity of coding, meanwhile, it ensure that the rate distortion of coding
Performance and the quality for drawing virtual view.
Influence of the inventive method of table 4 to coding total time and coding depth video time
Influence of the inventive method of table 5 to encoder bit rate
Influence of the inventive method of table 6 to drafting virtual view quality
Influence of the inventive method of table 7 to coding BD-MS-SSIM and BDBR
Claims (8)
1. a kind of deep video fast encoding method, it is characterised in that comprise the following steps：
1. pending high definition deep video sequence is defined as current depth video sequence；
2. being pre-processed to current depth video sequence, pretreatment includes two parts, and Part I is：To current depth video
Remaining in sequence in addition to the 1st frame depth image and last frame depth image carries out airspace enhancement per frame depth image, in sky
The borderline region of depth image and non-borderline region corresponding to being determined during the enhancing of domain；Part II is：At airspace enhancement
Deep video sequence after reason carries out time domain enhancing；
The detailed process of described step 2. middle Part I is：2. -1a, will be currently pending in current depth video sequence
Depth image is defined as present frame；2. if -1b, present frame are the 1st frame depth image or last in current depth video sequence
One frame depth image, then do not deal with to present frame；If present frame is not the 1st frame depth image in current depth video sequence
Or last frame depth image, then all fracture pixels for belonging to non-borderline region in present frame are entered using Gaussian filter
Row filtering process, place is filtered using self-adapting window to all non-fracture pixels for belonging to non-borderline region in present frame
Reason, to all fracture pixels for belonging to borderline region in present frame and all non-fracture pixels for belonging to borderline region not
Deal with；2. -1c, using the pending depth image of next frame in current depth video sequence as present frame, is then back to step
Suddenly 2. -1b is continued executing with, until all depth images in current depth video sequence are disposed, completes current depth video
The airspace enhancement of remaining every frame depth image in sequence in addition to the 1st frame depth image and last frame depth image；
The detailed process of described step 2. middle Part II is：2. the depth image in -2a, hypothesis current depth video sequence
Width and height correspond to W and H, and the totalframes for the depth image for assuming to include in current depth video sequence is T frames；
2. -2b, carrying out space-time transformation to the deep video sequence after airspace enhancement processing, the deep video sequence after space-time transformation is obtained
Row, the pixel by coordinate position in the jth frame depth image in the deep video sequence after space-time transformation for the pixel of (i, t)
Value is designated as dj(i, t), dj(i, t)=dt(i, j), wherein, 1≤j≤H, 1≤i≤W, 1≤t≤T, dt(i, j) is represented at airspace enhancement
Coordinate position is the pixel value of the pixel of (i, j) in t frame depth images in deep video sequence after reason；It is 2. -2c, right
Every frame depth image in deep video sequence after space-time transformation carries out time domain enhancing processing using time-domain filtering window, obtains
Deep video sequence after time domain enhancing processing, the jth frame depth image in deep video sequence after time domain enhancing is handled
Middle coordinate position is designated as d' for the pixel value of the pixel of (i, t)j(i, t),Its
In, t0Represent the size of time-domain filtering window, dj(i, t') represents the jth frame depth in the deep video sequence after space-time transformation
In image coordinate position for (i, t') pixel pixel value, wj(i, t') represents djThe weight of (i, t')；2. -2d, to time domain
Deep video sequence after enhancing processing carries out space-time transformation, obtains pretreated deep video sequence, will be pretreated
Coordinate position is designated as d' for the pixel value of the pixel of (i, j) in t frame depth images in deep video sequencet(i, j), d't
(i, j)=d'j(i,t)；
2. the determination process of the fracture pixel in -1b in present frame and non-fracture pixel is described step：Make cr (i, j)
Represent that coordinate position is the fracture mark of the pixel of (i, j) in present frame；Then each pixel in present frame
Coordinate position and pixel value, determine the fracture mark of each pixel in present frame, for coordinate position in present frame for (i,
J) pixel,Then according to current
The fracture mark of pixel in frame, the scope for the fracture mark that expansion value is 1, detailed process are：For coordinate in present frame
Position is the pixel of (i', j'), if the value of the fracture mark of the pixel is 1, when coordinate position is (i'- in present frame
1, j') when the value of the fracture mark of pixel is 0, the fracture by coordinate position in present frame for the pixel of (i'-1, j')
The value of mark is changed to 1；The pixel that the value that mark is broken in present frame is 0 is defined as non-fracture pixel again, will be current
The pixel that the value that mark is broken in frame is 1 is defined as being broken pixel；Wherein, cr (i, j) value is 0 or 1,1≤i≤W,
1≤j≤H, W and H correspondingly represent the width and height of the depth image in current depth video sequence, and dp (i-1, j) represents to work as
Coordinate position is the pixel value of the pixel of (i-1, j) in previous frame, and dp (i, j) represents that coordinate position is (i, j) in present frame
The pixel value of pixel, symbol " | | " are the symbol that takes absolute value, Th0For the fracture judgment threshold of setting, 2≤i'≤W, 1≤j'
≤H；
3. depth image currently pending in pretreated deep video sequence is defined as present frame；
4. judge whether present frame is the 1st frame depth image or last frame depth map in pretreated deep video sequence
Picture, if it is, directly being encoded using 3D-HEVC original coding platforms to present frame, then perform step 7.；Otherwise,
Perform step 5.；
5. obtain moving region and the non-moving areas of present frame；And in the current frame, with step 2. middle acquisition and present frame
Borderline region of the corresponding region of borderline region of corresponding depth image as present frame, with step 2. middle acquisition with it is current
Non- borderline region of the corresponding region of non-borderline region of depth image corresponding to frame as present frame；
6. determine whether each coding unit in present frame had not only belonged to borderline region but also belonged to moving region, for both belonging to side
Battery limit (BL) domain belongs to the coding unit of moving region again, travels through all HEVC coding prediction modes, selects rate distortion costs value minimum
Optimal prediction modes of the HEVC coding prediction modes as current coded unit；It is all for other coding units, only traversal
SKIP patterns and Merge patterns in HEVC coding prediction modes, select the minimum pattern of rate distortion costs value and compiled as current
The optimal prediction modes of code unit；Then the optimal prediction modes of each coding unit in present frame are utilized in present frame
Each coding unit is encoded；
7. using the pending depth image of next frame in pretreated deep video sequence as present frame, step is then back to
4. continue executing with, until all depth images coding in pretreated deep video sequence finishes.
A kind of 2. deep video fast encoding method according to claim 1, it is characterised in that the fracture of described setting
Judgment threshold Th0Value is 10.
3. a kind of deep video fast encoding method according to claim 1, it is characterised in that described step is 2. in -1b
The borderline region of present frame and non-borderline region carry out border detection to present frame by using Canny boundary detection methods and obtained
Arrive.
A kind of 4. deep video fast encoding method according to claim 3, it is characterised in that described Canny borders inspection
High threshold th employed in survey methodHValue is 120, and used Low threshold thLValue is 40.
5. a kind of deep video fast encoding method according to claim 3, it is characterised in that described step is 2. in -1b
The specific mistake of processing is filtered using self-adapting window to all non-fracture pixels for belonging to non-borderline region in present frame
Cheng Wei：Any one non-fracture pixel for belonging to non-borderline region in present frame, using the non-fracture pixel as center
Pixel, and carried out up and down and Left-right Searching by step-size in search of n pixel, when searching the fracture that belongs to non-borderline region
Pixel belongs to the fracture pixel of borderline region or belongs to the non-fracture pixel of borderline region or search up to present frame
Image boundary when stop search, formed a crossing window；Then using each pixel on the crossing window longitudinal axis in
The heart, and Left-right Searching is carried out by step-size in search of n pixel, when searching the fracture pixel or category that belong to non-borderline region
In borderline region fracture pixel or belong to the non-fracture pixel of borderline region or search image boundary up to present frame
When stop search, formed a self-adapting window；The average of the pixel value of the intraoral all pixels point of the adaptive windows is assigned again
It is worth and realizes filtering process to central pixel point.
6. a kind of deep video fast encoding method according to claim 5, it is characterised in that described step is 2. in -2c
The size t of time-domain filtering window0Value is 5；Described step is 2. in -2c
difj(i, t')=| dj(i,t'+1)-dj(i, t') |, dj(i, t'+1) represents the in the deep video sequence after space-time transformation
Coordinate position is the pixel value of the pixel of (i, t'+1) in j frame depth images, and symbol " | | " is the symbol that takes absolute value.
A kind of 7. deep video fast encoding method according to claim 1, it is characterised in that described step 5. in obtain
The moving region of present frame and the detailed process of non-moving areas is taken to be：
5. the width and height of -1, assuming the depth image in current depth video sequence correspond to W and H, and assume W × H energy
It is enough to be divided exactly by 4 × 4, then coloured image corresponding to present frame is divided intoThe sub-block of individual non-overlapping copies；
5. h-th of sub-block currently pending in coloured image corresponding to present frame -2, is defined as current sub-block, wherein, h's
Initial value is 1,
5. -3, calculate the squared difference of corresponding region in the previous color image frame of current sub-block coloured image corresponding with present frame
Be designated as SSDpre,h,And calculate current sub-block coloured image corresponding with present frame
In latter color image frame the squared difference of corresponding region and, be designated as SSDback,h,
Wherein, Ccur(u, v) represents that coordinate position is the pixel value of the pixel of (u, v), C in current sub-blockpre(u, v) represents present frame
Coordinate position is the pixel of (u, v) in region corresponding with current sub-block in the previous color image frame of corresponding coloured image
Pixel value, Cback(u, v) represents corresponding with current sub-block in the latter color image frame of coloured image corresponding to present frame
Coordinate position is the pixel value of the pixel of (u, v) in region, and symbol " | | " is the symbol that takes absolute value；
5. -4, judge min (SSDpre,h,SSDback,h)<Whether Th sets up, if set up, it is determined that current sub-block is non-athletic son
Block, otherwise, it determines current sub-block is moving sub-block；Wherein, for min () to take minimum value function, Th is that the moving sub-block of setting is sentenced
Disconnected threshold value；
5. h=h+1-5, is made, then using next pending sub-block in coloured image corresponding to present frame as current sub-block,
Step is returned again to 5. -3 to continue executing with, until all sub-blocks in coloured image corresponding to present frame are disposed, wherein, h=h
"=" in+1 is assignment；
5. region corresponding to all moving sub-blocks in coloured image corresponding in present frame -6, is defined as present frame
Moving region, region corresponding to all non-athletic sub-blocks in coloured image corresponding in present frame is defined as present frame
Non-moving areas.
A kind of 8. deep video fast encoding method according to claim 1, it is characterised in that the tool of described step 6.
Body process is：
6. maximum coding unit currently pending in present frame -1, is defined as current maximum coding unit；
6. coding unit currently pending in current maximum coding unit -2, is defined as current coded unit, and will be current
Depth where coding unit is defined as current depth；
If 6. the pixel for the borderline region for belonging to present frame in -3, current coded unit be present, and in current coded unit
In the presence of the pixel for the moving region for belonging to present frame, then it is assumed that current coded unit had not only belonged to borderline region but also belonged to motor area
Domain, all HEVC coding prediction modes are traveled through, calculate rate distortion of the current coded unit under every kind of HEVC coding prediction modes
Cost value, then during rate-distortion optimization, the minimum HEVC coding prediction modes of rate distortion costs value are selected as current
Optimal prediction modes of coding unit, and obtain the rate distortion costs value under optimal prediction modes, then perform step 6. -4；
It is not present if existed in current coded unit in the pixel for the borderline region for belonging to present frame and current coded unit
Belong to the pixel of the moving region of present frame, or the pixel for the borderline region for belonging to present frame is not present in current coded unit
The pixel for the moving region for belonging to present frame in point and current coded unit be present, or be not present and belong in current coded unit
The pixel for the moving region for belonging to present frame is not present in the pixel and current coded unit of the borderline region of present frame, then
The SKIP patterns and Merge patterns in all HEVC coding prediction modes are only traveled through, calculates current coded unit in SKIP patterns
With respective rate distortion costs value under Merge patterns, then during rate-distortion optimization, rate distortion costs value minimum is selected
Optimal prediction modes of the pattern as current coded unit, and the rate distortion costs value under optimal prediction modes is obtained, then hold
Row step 6. -4；
6. the optimum segmentation for -4, comparing coding unit corresponding with current coded unit in coloured image corresponding to present frame is deep
Degree and the size of current depth, if the former is more than the latter, execution step 6. -5；If the former is less than or equal to the latter,
Optimum segmentation depth using current depth as current coded unit, then perform step 6. -6；
6. -5, according to step 6. -3 process, in an identical manner obtain current coded unit where layer next layer in four
The respective optimal prediction modes of individual size identical coding unit and the rate distortion costs value under optimal prediction modes, so
The rate distortion costs value sum and present encoding of four coding units in next layer of layer where comparing current coded unit afterwards
The size of rate distortion costs value of the unit under optimal prediction modes, if the former is more than the latter, using current depth as work as
The optimum segmentation depth of preceding coding unit, to current coded unit without next layer of segmentation, then perform step 6. -6；Such as
The former is less than or equal to the latter to fruit, then next layer of segmentation is carried out to current coded unit, then by where current coded unit
Currently pending coding unit is as current coded unit in four coding units in next layer of layer, and by present encoding
Depth where unit returns again to step and 6. -4 continued executing with as current depth；
6. -6, current coded unit is encoded using the optimal prediction modes of current coded unit, in current coded unit
After end-of-encode, judge whether all coding units in current maximum coding unit are disposed, if it is, determining to work as
Preceding maximum coding unit end-of-encode, then perform step 6. -7；Otherwise, then four in layer where current coded unit are judged
Whether size identical coding unit is disposed, when being disposed, layer where judging current coded unit
Whether the coding unit in last layer is maximum coding unit, if maximum coding unit, then by where current coded unit
Then 6. -7 maximum coding unit in the last layer of layer performs step, if not maximum as current maximum coding unit
Coding unit, then using next pending coding unit in the last layer of layer where current coded unit as present encoding list
Member, using the depth where the current coded unit as current depth, it is then back to step and 6. -3 continues executing with；
6. -7, judge whether current maximum coding unit is last maximum coding unit in present frame, if it is, holding
7. row step, otherwise, next pending maximum coding unit in present frame as current maximum coding unit, is then returned
Step is returned 6. -2 to continue executing with.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510470699.2A CN105120290B (en) | 2015-08-04 | 2015-08-04 | A kind of deep video fast encoding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510470699.2A CN105120290B (en) | 2015-08-04 | 2015-08-04 | A kind of deep video fast encoding method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105120290A CN105120290A (en) | 2015-12-02 |
| CN105120290B true CN105120290B (en) | 2017-12-05 |
Family
ID=54668136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510470699.2A Active CN105120290B (en) | 2015-08-04 | 2015-08-04 | A kind of deep video fast encoding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105120290B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106412611B (en) * | 2016-09-26 | 2018-04-10 | 宁波大学 | A kind of complexity control method of efficient video coding |
| US10742999B2 (en) * | 2017-01-06 | 2020-08-11 | Mediatek Inc. | Methods and apparatus for signaling viewports and regions of interest |
| CN108012150B (en) * | 2017-12-14 | 2020-05-05 | 湖南兴天电子科技有限公司 | Video interframe coding method and device |
| CN108322740B (en) * | 2018-01-10 | 2020-04-17 | 宁波大学 | Encoding method with controllable encoding complexity |
| CN108271025B (en) * | 2018-01-22 | 2019-11-29 | 合肥工业大学 | The coding circuit of depth modelling mode and its coding method in 3D coding and decoding video based on boundary gradient |
| CN108134940B (en) * | 2018-01-22 | 2019-11-08 | 合肥工业大学 | Coding circuit and method applied to depth image intra prediction in Video coding |
| US11115451B2 (en) * | 2018-03-26 | 2021-09-07 | Mediatek Singapore Pte. Ltd. | Methods and apparatus for signaling viewports and regions of interest |
| CN110868613B (en) * | 2018-08-28 | 2021-10-01 | 华为技术有限公司 | Image encoding method, image decoding method and device based on history candidate list |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101374243A (en) * | 2008-07-29 | 2009-02-25 | 宁波大学 | Depth map encoding compression method for 3DTV and FTV system |
| CN101588445A (en) * | 2009-06-09 | 2009-11-25 | 宁波大学 | Video area-of-interest exacting method based on depth |
| CN103716607A (en) * | 2012-09-28 | 2014-04-09 | 中兴通讯股份有限公司 | Encoding method and apparatus applied to high efficiency video coding-based 3-Dimension video coding (HEVC-based 3DVC) |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150049821A1 (en) * | 2013-08-16 | 2015-02-19 | Qualcomm Incorporated | In-loop depth map filtering for 3d video coding |
-
2015
- 2015-08-04 CN CN201510470699.2A patent/CN105120290B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101374243A (en) * | 2008-07-29 | 2009-02-25 | 宁波大学 | Depth map encoding compression method for 3DTV and FTV system |
| CN101588445A (en) * | 2009-06-09 | 2009-11-25 | 宁波大学 | Video area-of-interest exacting method based on depth |
| CN103716607A (en) * | 2012-09-28 | 2014-04-09 | 中兴通讯股份有限公司 | Encoding method and apparatus applied to high efficiency video coding-based 3-Dimension video coding (HEVC-based 3DVC) |
Non-Patent Citations (1)
| Title |
|---|
| 深度视频信号预处理及其在编码中的应用研究;周浩;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150315;摘要、正文第23-24页第3.2节、第44-46页第4.5节、第56-63页第5章第5.1-5.4节，图3.2、5.1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105120290A (en) | 2015-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105120290B (en) | A kind of deep video fast encoding method | |
| CN103873861B (en) | Coding mode selection method for HEVC (high efficiency video coding) | |
| CN104754357B (en) | Intraframe coding optimization method and device based on convolutional neural networks | |
| CN101374243B (en) | Depth map encoding compression method for 3DTV and FTV system | |
| CN111355956B (en) | Deep learning-based rate distortion optimization rapid decision system and method in HEVC intra-frame coding | |
| CN107277509B (en) | A kind of fast intra-frame predicting method based on screen content | |
| CN106507116B (en) | A kind of 3D-HEVC coding method predicted based on 3D conspicuousness information and View Synthesis | |
| CN110087087A (en) | VVC interframe encode unit prediction mode shifts to an earlier date decision and block divides and shifts to an earlier date terminating method | |
| CN101729891B (en) | Method for encoding multi-view depth video | |
| CN105430415A (en) | Fast intraframe coding method of 3D-HEVC depth videos | |
| CN102420988B (en) | Multi-view video coding system utilizing visual characteristics | |
| CN108495135A (en) | A kind of fast encoding method of screen content Video coding | |
| CN108712648B (en) | Rapid intra-frame coding method for depth video | |
| CN105898332B (en) | For the fast deep figure frame mode decision method of 3D-HEVC coding standards | |
| CN107087200A (en) | Coding mode advance decision method is skipped for high efficiency video encoding standard | |
| CN104469336B (en) | Coding method for multi-view depth video signals | |
| CN110446052B (en) | 3D-HEVC intra-frame depth map rapid CU depth selection method | |
| CN107396102A (en) | A kind of inter-frame mode fast selecting method and device based on Merge technological movement vectors | |
| CN103546758A (en) | Rapid depth map sequence interframe mode selection fractal coding method | |
| CN106937116A (en) | Low-complexity video coding method based on random training set adaptive learning | |
| CN107318016A (en) | A kind of HEVC inter-frame forecast mode method for rapidly judging based on zero piece of distribution | |
| CN103634601A (en) | Structural similarity-based efficient video code perceiving code rate control optimizing method | |
| CN106791828A (en) | High performance video code-transferring method and its transcoder based on machine learning | |
| CN106534855B (en) | A kind of Lagrange factor calculation method towards SATD | |
| CN103702131B (en) | Pattern-preprocessing-based intraframe coding optimization method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |