CN104065954A - Method for quickly detecting parallax scope of high-definition stereoscopic video - Google Patents

Abstract

The invention relates to a method for quickly detecting the parallax scope of a high-definition stereoscopic video. The method comprises a step of quick stereo matching, a step of improving the accuracy of quick stereo matching, a step of parallax scope extraction, a step of spatial precision compensation, and a step of time precision compensation. According to the invention, the quick dense stereo matching with the detection method, and the parallax scope of a stereo image can be quickly extracted through a mode of threshold judging. The method employs a distributed idea, and the maximum out-screen scope of the video is calculated through a mode of tracking a picture out-screen maximum object, thereby reducing the calculated amount, and improving the calculating speed. According to the invention, there is no need of calculating the dense parallax image of each frame of the stereoscopic video, and the parallax scope of no-calculation frames is quickly estimated through a mode of cut-out window and parallax scope succession. Compared with the prior art, the method can guarantee the time continuity of parallax information during frame extraction processing, and can accurately calculate the parallax scope of the high-definition stereoscopic video frame by frame in real time.

Description

A kind of disparity range method for quick of high definition three-dimensional video-frequency

Technical field

The present invention relates to Stereo Matching Technology, be specifically related to the Rapid matching technology of big data quantity stereo-picture, and the disparity range method for quick of high definition three-dimensional video-frequency.

Background technology

Along with the continuous growth of people to 3D business interest, watch comfort level and the fail safe of 3D video have been subject to paying close attention to more and more widely.At present, 3D business comfort level and fail safe have become and hinder 3D business and enter one of Main Bottleneck of popular video market.Therefore, for the producer of 3D video, characterize the index of 3D business comfort level, as the information such as disparity range of video, just seem particularly important.Make 3D video improperly and not only can not allow beholder produce joyful viewing experience, also may cause beholder to produce the discomfort on health, even may cause negative effect to part beholder's (as minor) visual capacity.

Parallax information is the important indicator of weighing digital three-dimensional video system level of comfort and fail safe, and the parallax information that obtains picture can be learnt the depth distribution situation of whole scene, thereby without increasing extra transmission cost for the degree of depth key element of transmitting scene.Depth information is the important information of monitoring three-dimensional video-frequency quality, adjustment three-dimensional video-frequency depth bounds.Therefore the depth information that, how to obtain quickly and accurately scene from stereoscopic video content has very high practical value.

The people such as D.Scharsteinand and R.Szeliski is divided into global registration and the large class of Region Matching two by dense Stereo matching.The feature of Region Matching is to adopt window as matching unit, according to the window setting, the result of coupling cost is integrated, and obtains the parallax value of window center point.The advantage of Region Matching is to mate computational process and picture material is irrelevant, can be optimized for a kind of fast matching method; Shortcoming is that the general precision of anaglyph is lower, easily occurs large-area disparity computation mistake.The feature of global registration is in order to solve the optimal solution problem of certain overall situation function by the search problem conversion of match point.The advantage of global registration is to mate computational process can pass through picture material adjustment aim global energy function, thereby obtains high-precision coupling disparity map; Shortcoming is that the amount of calculation of global energy function optimization process is very large, accurate but consuming time longer.Visible, in dense Stereo matching: Region Matching computational speed is very fast, be applicable to application in real time, but its accuracy in computation is low, cannot extract for the disparity range of three-dimensional video-frequency; Global registration accuracy in computation is high, can accurately extract the disparity range of three-dimensional video-frequency, but its computational efficiency is too low, is not suitable for the rapid extraction of disparity range.

Solid matching method based on angle point is the main method of current rapid extraction stereo-picture disparity range.Solid matching method based on angle point does not calculate view picture anaglyph, but calculates by the part point extracting in image.Concrete steps mainly comprise angle point extraction, corners Matching, edge extracting, marginal point coupling.Solid matching method advantage based on angle point is under very little amount of calculation, to provide the parallax value of part point, is suitable for carrying out the disparity range detection of image; Shortcoming is to generate parallax full figure, therefore poor to the estimated capacity of depth distribution, and arithmetic speed is fast not, and distance is processed HD video in real time larger gap.

Summary of the invention

The problem such as the present invention is directed to that the matching speed that exists in prior art is slow, coupling is counted few, a kind of disparity range method for quick of high definition three-dimensional video-frequency is proposed, based on dense Stereo matching, by adding the mode of constraints, estimate fast the disparity range of stereo-picture.

To achieve these goals, the present invention by the following technical solutions.

A disparity range method for quick for high definition three-dimensional video-frequency, comprises step, the step of spatial accuracy compensation and the step of time precision compensation that the step of quick stereo coupling, the step that improves Rapid matching accuracy, disparity range are extracted.

The step of quick stereo coupling, comprises following content:

The present invention has optimized the computational process of stationary window coupling from calculating redundancy and transformation from serial to parallel two aspects.

The calculating redundancy of stationary window coupling mainly comprises the calculating redundancy of consecutive points and the calculating redundancy of adjacent window apertures, as shown in Figure 1.The calculating redundancy of eliminating consecutive points can be by each match window L ²inferior amount of calculation has reduced to 4 times.Eliminate the calculating redundancy of adjacent window apertures enough by every some amount of calculation of 4 times further reduce to every 2 times.

Except calculating redundancy, also can adopt single-instruction multiple-data stream (SIMD) (Single Instruction Multiple Data, SIMD) technology, serial computing is converted into parallel processing, further raise the efficiency.The image-region scope of Fig. 2 for carrying out parallel computation.In figure, the point that calculate for mating cost in stain white background region, white point black matrix region is core calculations region, in this region, the coupling cost calculation times of every is 2 times, is the region that can carry out parallel processing.This method has been selected the MMX instruction set in SIMD technology, and a MMX instruction can be processed the brightness data of 88 bits simultaneously, the cycle-index of calculating can be reduced to original 1/8, has significantly improved the efficiency of computing.

The step that improves Rapid matching accuracy, comprises following content:

For dense Stereo matching, exist fast area matching process inaccurate with global registration method can not fast processing contradiction.The present invention is directed to this application of detection of three-dimensional video-frequency disparity range, following realization approach has been proposed: because the detection of stereoscopic video disparity range does not need disparity map full figure accurately, therefore can adopt the mode of finding the accurate information in fast matching method, fast area matching technique is applied in the middle of the detection of three-dimensional video-frequency disparity range.

Typical Stereo matching based on stationary window and mistake matching area thereof are as shown in Figure 4.In figure, square frame inner region has represented the representational zone errors of three classes, is respectively repetition texture region, low texture region, occlusion area.For this three classes mistake, need to use different constraints to carry out error detection.Left and right consistency detection, from the angle of coupling corresponding relation, is analyzed the mistake coupling existing in Rapid matching.Parallax centrality detects and conspicuousness detects the angle from matching result significance degree, analyzes the mistake coupling existing in Rapid matching.

The step that disparity range is extracted, comprises following content:

By repeatedly to calculate and relatively, in order reaching fast or the requirement of processing in real time, to guarantee that parallax precision does not have too large loss simultaneously, original high definition stereo-picture is carried out to 1/4 down-sampling.After sampling, the calculating of stereo-picture is consuming time declines greatly, can approach real-time processing.In the situation that not considering that other modules of program are consuming time, original series is carried out to the frame of taking out of time upper 1/3 and process and just can reach the effect of real-time processing.

High-definition image, after passing through down-sampling, taking out frame, disparity computation, has just obtained low precision anaglyph.Resulting anaglyph is carried out erroneous matching range check through the method for middle introduction above, can obtain the low precision anaglyph through error checking, as shown in Figure 8.In real stereo video, due near big and far smaller Perspective Principles, the object nearest apart from video camera generally can occupy the region of a screen part, and this region generally can be very not little.According to this hypothesis, can define a threshold value, when thering is the pixel quantity of maximum disparity value pixel and be greater than this threshold value, just think that this parallax value is the maximum disparity of whole scene, otherwise just judge that this parallax value is as mistake coupling.Fig. 9 is the extracting method flow graph journey of maximum disparity, and the extracting method of minimum parallax is similar with it.

The step of spatial accuracy compensation, comprises following content:

Previous step is resulting is rough anaglyph, and the resulting disparity map of this step can provide two important information: the one, and disparity range roughly.By the calculating of rough disparity map, can obtain former stereo-picture disparity range roughly, subsequent calculations can be searched for accurate parallax extreme value near rough parallax extreme value, has greatly reduced hunting zone and amount of calculation.The 2nd, the Position Approximate of maximum disparity object.By the calculating of rough disparity map, can learn former stereo-picture parallax distribution situation roughly, by setting different parallax threshold ranges, can obtain approximate region and position apart from video camera ad-hoc location object.After having obtained the approximate region of maximum disparity object, by the calculating to pixel coordinate in this region, can draw the coordinate valuation of maximum disparity object center.Afterwards, only need to set the size of window, just can be in former high definition stereo-picture " cutting out " go out the high definition texture information of target object.An instantiation of this process as shown in figure 11.

First, the step that original high-definition image extracts through disparity range, can access rough anaglyph after testing and disparity range roughly; Secondly, according to rough anaglyph, calculate and have the approximate range of maximum disparity object, and according to this scope, former high-definition image is intercepted high-definition image after the intercepting that obtains containing maximum disparity object; Afterwards, according to the coupling computer capacity obtaining, to the nearly row matching ratio of part high-definition image after intercepting, wherein, the absolute difference between employing two figure is as matching ratio mode; Finally, the locational matching error of more different parallaxes, selects the position of matching error minimum as matched position, and meticulous matched position and former rough matched position are added, thereby obtains high-precision parallax information.

The step of time precision compensation, comprises following content:

Based on quick high accuracy method for extracting parallax presented above, the present invention proposes the compensation method of a kind of time precision fast.Before mention, even if former high-definition image is carried out to 1/4 space down-sampling, still cannot reach real-time processing, need to further sample in time, through the image of taking out frame parallax value in time, will there will be discontinuous.Because video information has continually varying feature in time, the picture scene between adjacent video frames generally there will not be too large variation, same, and the depth distribution of adjacent video frames also has very strong correlation.In general, within continuous a few frame pictures, the position of picture main body and the degree of depth all can not produce too large variation.

According to this specific character, can adopt the parallax information of the stereoscopic video of method shown in Figure 13 to carry out the compensation on time precision.For non-extraction frame, adopt the method for spatial accuracy compensation to carry out calculating and the extraction of parallax value.To extracting frame, after reading in this two field picture, do not carry out longer disparity computation consuming time, but directly use parallax valuation that former frame calculates and the parallax information of this frame of intercepting range computation, can obtain the high accuracy disparity range of this frame stereoscopic picture plane.

Compared with prior art, the present invention has the following advantages:

(1) the present invention combines quick dense Stereo matching with detection method, the disparity range of the mode rapid extraction stereo-picture of passing threshold judgement.In dense solid matching method, global registration technique computes speed is generally very slow, and the quick parallax that cannot carry out big data quantity stereo-picture calculates; There are a large amount of mistake couplings in region Stereo Matching Technology fast, cannot be applied to disparity range and extract.Compare with the solid matching method based on angle point, effective coupling that the present invention obtains is counted many, is generally greater than 50% of full figure pixel quantity, and the resulting coupling of matching process based on angle point is counted and is generally no more than 5% of full figure pixel quantity.

(2) the present invention adopts distributed thinking, and the maximum that the mode that goes out to shield largest object by following the tracks of picture is calculated video goes out screen scope, has significantly reduced amount of calculation, has improved computational speed.High-definition image for one 1920 * 1080, hunting zone is-128～+ 128 o'clock, the maximum that the present invention can obtain high definition precision with the speed of approximately 0.16 second goes out screen scope, only use quick dense solid matching method to calculate at least to need consuming time more than 2 seconds, and in the situation that do not use stereo-picture disparity range rapid extracting method proposed by the invention, cannot obtain the disparity range of picture.

(3) the present invention does not need to calculate the dense disparity map picture of each frame of three-dimensional video-frequency, and the mode of inheriting by intercepting window and disparity range is estimated the disparity range of non-calculating frame fast.Compare with other the real-time high definition disparity range computational methods that need to take out frame processing, the present invention can guarantee the temporal continuity of parallax information when taking out frame processing, can quasi real time calculate frame by frame the disparity range of HD video (about 15fps), far away from the computational speed based on matching process such as angle points.

Accompanying drawing explanation

Fig. 1 is the calculating redundancy schematic diagram of fixed window algorithm;

The pixel coverage schematic diagram of Fig. 2 for carrying out parallel computation;

Fig. 3 is image preliminary treatment schematic diagram;

Fig. 4 is the Stereo matching result based on stationary window, and left side is original image, and centre is true disparity map, and right side is the matching result based on point;

Fig. 5 is the consistent testing result in left and right;

Fig. 6 is that centrality detects and conspicuousness testing result;

Fig. 7 is comprehensive detection result;

Fig. 8 is the Rapid matching result through error detection;

Fig. 9 is disparity range detection method flow chart;

Figure 10 is disparity range testing result;

Figure 11 is disparity space accuracy compensation realization flow figure;

Figure 12 is disparity space accuracy compensation result;

Figure 13 is parallax time precision compensation implementation procedure schematic diagram;

Figure 14 is parallax time precision compensation result.

Embodiment

A disparity range method for quick for high definition three-dimensional video-frequency, comprises the following steps:

Step 1, initialization and preliminary treatment, method is as follows:

(1) basic parameter of configuration Stereo matching.Comprise match window in image essential information, match search scope, quick dense Stereo matching size, to the requirement of matching precision, requirement to matching speed.Wherein, match search scope need to should be the odd number that is greater than 1 for 8 integral multiple, match window size.

(2) read in target stereo-picture.If target image or video are compressed format, need to, to the processing of decoding of original image or video, obtain the non-compressed image of YCrCb.

(3) extract monochrome information.Monochrome information Y in YCrCb format-pattern is extracted and preserved, for subsequent treatment, give up chrominance information CrCb.

(4) stereoscopic image is offset.As shown in Figure 3.If the hunting zone of Stereo matching is [0, L], in order to make the three-dimensional video-frequency of different styles of shooting can adopt same matching process to process, need to be offset image.If original stereo-picture adopts run-in index stereoscopic shooting, do not need to carry out migration processing; If original stereogram adopts staggered form stereoscopic shooting, need to carry out the skew of L/2 pixel left to reference picture.

(5) stereoscopic image is carried out down-sampling.Reference picture after the target image of original stereo-picture and skew is carried out in level and vertical direction to 1/4 space down-sampling.Actual requirement according to user to matching speed, carries out a certain proportion of time down-sampling (taking out frame) to original series, and the space down-sampling that adoption rate is 1/3 can reach real-time processing.

Step 2, quick stereo coupling realizes, and method is as follows:

(1) adopt quick dense solid matching method stereoscopic image to carry out Stereo matching.Matching process is selected fixed window algorithm, and coupling cost function is selected SAD (absolute difference and), and match window size and match search scope are selected according to stereo-picture actual conditions.

(2) rewrite the Accounting Legend Code of coupling cost SAD.As shown in Figure 2.If reference picture is L, target image is R, and picture traverse is Width, is highly Height, and match window width is W, for the some L (x, y) of diverse location in image:

For the some I in reference picture _r(x, y) (x=W/2, y=W/2), the expression formula of coupling cost function SAD is:

$\mathrm{SAD}(x,y,d)=\underset{x,y\∈S}{\mathrm{\Σ}}|I_R(x,y)-I_T(x+d,y)|$

In formula, d is the parallax value of corresponding points between reference picture and target image, and S is coupling cost polymerization window, I _rfor reference picture, I _tfor target image.

For a L (x, y), (x=W/2, y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x,y+1,d)＝SAD(x,y,d)+U(x,y+1,d)

$\mathrm{SAD}(x,y,d)=\underset{i,j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\Σ}}}|L(x+j,y+i)-R(x+d+j,y+i)|$

$U(x,y+1,d)=\underset{j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\Σ}}}|L(x+j,y+\frac{L-1}{2}+1)-R(x+d+j,y+\frac{L-1}{2}+1)|-\underset{j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\Σ}}}|L(x+j,y-\frac{L-1}{2})-R(x+d+j,y-\frac{L-1}{2})|$

For a L (x, y), (x=[W/2+1, W+W/2], y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x,y+1,d)＝SAD(x,y,d)+U(x,y+1,d)

U(x,y+1,d)＝U(x-1,y+1,d)+|A-A'|-|B-B'|-(|D-D'|-|C-C'|)

Wherein, A, B, C, D are the point in reference picture, as shown in Figure 1, A is the pixel of the outer upper left side of match window adjacent, B is the pixel of the outer upper right side of match window adjacent, and C is bottom-right pixel in match window, and D is the pixel of the outer lower left of match window; A', B', C', D' are the point in target image.

For a L (x, y), (x=[W+W/2+1, Width], y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x+wd,y,d)＝SAD(x+wd,y-1,d)+U(x+wd,y,d)

U(x+wd,y,d)＝U(x+wd-1,y,d)+S(x,y,d)-|B _wd-B _wd'|+|C _wd-C _wd'|

S(x,y,d)＝-|B-B'|+|C-C'|

Wherein, B, C, B _wd, C _wdfor reference picture mid point, B', C', B _wd', C _wd' be the point in target image.Centered by B, put the pixel of the outer upper right side of P (x, y, d) window adjacent, centered by C, put bottom-right pixel in P (x, y, d) window, B _wdcentered by put the pixel of the outer upper right side of P (x+wd, y, d) window adjacent, C _wdcentered by put bottom-right pixel in P (x+wd, y, d) window.

(3) with parallel instruction, rewrite core match code.For a L (x, y), (x=[W+W/2+1, Width], y ∈ [W/2+1, Height]), adopt the MMX instruction set in SIMD to rewrite former code.Particularly, original image be take to 8 points and pack as one group, be saved in and in MMX special register, carry out parallel processing.Meanwhile, the cycle-index of match search is reduced to original 1/8.

Comprehensive two kinds of methods, global optimization effect is as shown in table 1, and image size is 320x240, and hunting zone is [0,16], and window size is 9.

Table 1 calculation times and variation consuming time

Step 3, quick stereo matching optimization, method is as follows:

(1) Rapid matching result anaglyph is carried out to left and right consistency detection.For the fast matching method of describing in step 2, in the process of selecting in coupling cost, increase target image relatively with reference picture mate cost selection course, the parallax selection result of each point of record object image.According to result, the parallax value of target image and reference picture is compared, if the two is identical, think that the former matching result of target image is reliable, retain this matching result; If the two is not identical, think that the former matching result of target image is unreliable, give up this matching result.Actual detection effect as shown in Figure 5.By figure, can be found out, left and right consistency detection can effectively detect the matching error (Fig. 4 region 3) of occlusion area.If coupling result of calculation is compared, can, once completing left and right consistency detection in the middle of coupling, can under the prerequisite that increases not significantly amount of calculation, complete this detection.

(2) Rapid matching result anaglyph is carried out to the detection of parallax conspicuousness.For the matching process of target image, adopt following formula to miss matching detection:

${\mathrm{\δ}}_e=\frac{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}(e_i-e_{\min })}{e_{\min }}$

Wherein, e _minfor coupling Least-cost value, e _ifor N coupling cost sub-minimum, δ _efor threshold value.If result of calculation is less than threshold value δ _e, think that matching result is not remarkable, give up this matching result.

(3) Rapid matching result anaglyph is carried out to the detection of parallax centrality.For the matching process of target image, adopt following formula to miss matching detection:

${\mathrm{\δ}}_d=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}|d_i-d_{\min }|$

Wherein, d _minfor the parallax value of coupling Least-cost value, d _ifor the parallax value of N coupling cost sub-minimum, δ _dfor threshold value.If result of calculation is greater than threshold value δ _d, think that matching result is concentrated, give up this matching result.(2), the actual detection effect of (3) two steps as shown in Figure 6.Visible, this detection can effectively detect low texture region and repeat the matching error (Fig. 4 region 1,2) of texture region.

Comprehensive three kinds of methods, global optimization effect as shown in Figure 7.According to calculating, the rate of calculating accurately that obtains former coupling disparity map is 87%, and the rate of calculating accurately of disparity map is after testing 96%.Visible, adopt above-mentioned detection method can effectively extract the effective information in Rapid matching anaglyph.

Step 4, disparity range is extracted, and method is as follows:

(1) judgement image maximum disparity scope.The anaglyph that step 1,2 is calculated is called Dmap, step 3 is detected to the anaglyph obtaining and be called DmapR.In parallax hunting zone [L, L], the L of take travels through hunting zone [L, L] as initial value.In note DmapR, parallax value is counting as Cntl of l ∈ [L, L], sets the judgment threshold δ of maximum disparity scope _dmaxfor 1% of full figure pixel quantity.If Cntl is less than δ _dmax, reduce l value and continue traversal; If Cntl is more than or equal to δ _dmax, stop traversal, and l value Dmax be now recorded as to the maximum disparity scope of this stereo-picture.

(2) the minimum disparity range of judgement image.In parallax hunting zone [L, L], take-L travels through hunting zone [L, L] as initial value.In note DmapR, parallax value is counting as Cntl of l ∈ [L, L], sets the judgment threshold δ of maximum disparity scope _dminfor 5% of full figure pixel quantity.If Cntl is less than δ _dmin, reduce l value and continue traversal; If Cntl is more than or equal to δ _dmin, stop traversal, and l value Dmin be now recorded as to the minimum disparity range of this stereo-picture.

An example of disparity range leaching process and result as shown in Figure 8, contrasts former figure visible, and it is accurate that disparity range is extracted result.Fig. 9 is the extracting method of maximum disparity, and the extracting method of minimum parallax is similar with it.Adopt said method image in Fig. 8 to be carried out to extraction and the calculating of disparity range, result as shown in figure 10.In figure, the woods that minimum parallax testing result is picture background, maximum disparity testing result is the flowering shrubs near picture, testing result is accurate.

Step 5, spatial accuracy compensation, method is as follows:

(1) obtain maximum and go out to shield object space.For DmapR, take δ dmax ± t as disparity range intercepts former anaglyph.Wherein, t is that maximum goes out to shield object identification scope, and suggestion span is t ∈ [0,2], all gets t=1 in the example of this document.For the anaglyph after intercepting, calculating and recording all has the draw of parallax value point horizontal level AvgX and average upright position AvgY.

(2) intercepting maximum goes out to shield object.If Th is the height of intercepting window, Tw is the width of intercepting window.If picture traverse is Width, be highly Height: if AvgX – is Tw/2<0, AxgX is set to Tw/2; If AvgX+Tw/2>Width, AxgX is set to Width-Tw/2; If AvgY – is Th/2<0, AxgY is set to Th/2; If AvgY+Th/2>Height, AxgY is set to Height-Th/2.Take AxgX ± Tw/2, AxgY ± Th/2 intercepts reference picture as scope.

(3) establish [δ _l, δ _l] for parallax fine setting scope, according to the ratio-dependent δ of down-sampling _lsize.At l ∈ [δ _l, δ _l] scope in, relatively in (2), reference picture and target image center of intercepting are (AxgX+l, AxgY), scope is that SAD calculating is carried out in the region of AxgX+l ± Tw/2, AxgY ± Th/2, records [δ _l, δ _l] have the position l of minimum SAD in scope _sADmin.Note Dmax+l _sADminfor the accurate maximum of this stereo-picture goes out screen scope.

Before and after adopting spatial accuracy compensation, calculate gained parallax information accuracy comparison as shown in figure 12.In figure, middle curve chart is the maximum disparity range detection result of video image.Wherein, Grey curves is the result of calculation before spatial accuracy compensation, and black curve is the result of calculation after spatial accuracy compensation.Four width images of curve top, below are respectively the picture sectional drawing of video the 1st frame, 100 frames, 200 frames, 300 frames.This cycle tests is that a woman who wears kimonos walks before maple woods, and in the scope of 0～150 frame, in picture, woman's the degree of depth remains unchanged substantially; In the scope of 150～300 frames, in picture, woman is gradually near video camera, and the degree of depth reduces; After 300 frames, in picture, woman is gradually away from video camera, and the degree of depth increases.As seen from the figure, the maximum disparity scope that adopts disparity range extracting method presented above to obtain is consistent with actual scenery Depth Motion range trend, can adopt the method for the invention stereoscopic video parallax value to estimate.Parallax curve before and after contrast spatial accuracy compensates, level and smooth and continuous through the parallax curve (black) of spatial accuracy compensation, on spatial accuracy, be significantly higher than former result of calculation, computational accuracy has reached the precision grade of HD video.

For length, be the cycle tests of 450 frames, the computational accuracy of usage space accuracy compensation front and back method and calculate consuming time as shown in the table:

The compensation of table 2 spatial accuracy is consuming time

	Maximum search scope	Minimum search range	Precision	Consuming time
					Original method	+32	-32	±4	60s
Compensation method	+128	-128	±1	74s

From table 2, adopt the method for the invention, can, in the situation that only increasing approximately by 20% amount of calculation, make the computational accuracy of original method improve 4 times.Disparity computation precision after spatial accuracy compensation is original 4 times, and parallax hunting zone has been improved 4 times, so amount of calculation and consuming timely should also can improve 4 times.The present invention adopts based on picture maximum and goes out to shield the method that main body is followed the tracks of, and has significantly simplified the computation complexity of spatial accuracy compensation.

If do not adopt spatial accuracy compensation technique, directly use quick stereo matching process to calculate the disparity range of high-definition image, to the calculating of a frame three-dimensional video-frequency is consuming time will be over 2s, be about consuming time 13 times of spatial accuracy compensation method.

Step 6, time precision compensation, method is as follows:

(1) carry out in time frame of video classification.For taking out the real-time disparity range leaching process of primary video that frame ratio is S, note frame number f _noskipthe picture of=nS is non-extraction frame, note frame number f _skip=nS+k, the picture of k ∈ [1, S-1] is non-extraction frame, n is positive integer.For f _noskipframe.Carry out the calculating of step 5, record Dmax, AvgX, the AvgY in calculating this time.

(2) calculate the accurate parallax value that extracts frame.For f _skipframe, will upper distance f of time _skipthe f that frame is nearest _noskipthe Dmax of frame, AvgX, AvgY be as Dmax, AvgX, the AvgY of this frame, and adopt this numerical value to carry out the calculating of step 5, and resulting result is the accurate parallax value of this extraction frame.

Resulting coupling result of calculation as shown in figure 14.In figure, Grey curves is the result of calculation of time precision compensation method, and black curve is the result of calculation of calculating frame by frame.Can find out, the anaglyph that time precision compensation method is not carried out frame by frame original video sequence is calculated, but its computational accuracy and result of calculation and frame by frame result of calculation are basic identical, have equally higher precision.Because Stereo matching calculating is not carried out in spatial accuracy compensation method, it calculates consuming time much smaller than disparity computation consuming time, it for length, is the cycle tests of 450 frames, original method is consuming time is 73s, compensation method is consuming time is 31s, and the efficiency that can calculate a frame with about 0.07s obtains high accuracy disparity range.Thereby, adopt in this way and can guarantee significantly to reduce the consuming time of calculating when every frame generates high accuracy parallax value.

Described disparity range extraction step is applied to the scene of the rough disparity range of Real-Time Monitoring video, only need be aided with a certain proportion of frame of taking out, and does not need the accuracy compensation (processing speed is greater than 25fps) in the time of carrying out and space; Described spatial accuracy compensation process is applied to obtain high-precision disparity range fast, but does not require the scene of real-time processing, only need carry out spatial accuracy compensation, needn't carry out time precision compensation, and video is carried out to calculating (the about 6fps of processing speed) frame by frame; Described spatial accuracy compensation and time precision compensation can be applied to obtain the scene of high accuracy disparity range quasi real time simultaneously, significantly improve processing speed (the about 15fps of processing speed) in the situation that of a small amount of time precision of loss.

In order to verify the validity of the method for the invention, table 3 provides the present invention and contrast paper < < quick stereo edge matching method > > based on angle point guiding, and (Lee sea is superfine, BJ University of Aeronautics & Astronautics's journal, 05 phase in 2007) experimental result in.Computational speed in table 3 is on average consuming time for every frame after 450 frame videos are calculated.Can find out, the present invention calculates the consuming time of the clear three-dimensional video-frequency disparity range of a vertical frame dimension and only has 0.07 second, much smaller than 1.1 seconds consuming time of method described in contrast paper.

Table 3 the present invention with based on corners Matching method, contrast

	Image size	Hunting zone	Effective match point quantity	Computational speed
					The present invention	1920x1080	Total size > 200	Be greater than 50%	0.07s
Based on corners Matching method	512x512	Total size < 100	Be less than 5%	1.1s

Claims (2)

1. a disparity range method for quick for high definition three-dimensional video-frequency, is characterized in that comprising the following steps:

Step 1, initialization and preliminary treatment, method is as follows:

(1) basic parameter of configuration Stereo matching; Comprise match window in image essential information, match search scope, quick dense Stereo matching size, to the requirement of matching precision, requirement to matching speed; Wherein, match search scope need to should be the odd number that is greater than 1 for 8 integral multiple, match window size;

(2) read in target stereo-picture; If target image or video are compressed format, need to, to the processing of decoding of original image or video, obtain the non-compressed image of YCrCb;

(3) extract monochrome information; Monochrome information Y in YCrCb format-pattern is extracted and preserved, for subsequent treatment, give up chrominance information CrCb;

(4) stereoscopic image is offset; If the hunting zone of Stereo matching is [0, L], in order to make the three-dimensional video-frequency of different styles of shooting can adopt same matching process to process, image is offset: if original stereo-picture adopts run-in index stereoscopic shooting, do not need to carry out migration processing; If original stereogram adopts staggered form stereoscopic shooting, need to carry out the skew of L/2 pixel left to reference picture;

(5) stereoscopic image is carried out down-sampling; Reference picture after the target image of original stereo-picture and skew is carried out in level and vertical direction to 1/4 space down-sampling; Actual requirement according to user to matching speed, carries out a certain proportion of time down-sampling to original series and takes out frame, and the space down-sampling that adoption rate is 1/3 can reach real-time processing;

Step 2, quick stereo coupling realizes, and method is as follows:

(1) adopt quick dense solid matching method stereoscopic image to carry out Stereo matching; Matching process is selected fixed window algorithm, and coupling cost function is selected absolute difference and SAD, and match window size and match search scope are selected according to stereo-picture actual conditions;

(2) rewrite the Accounting Legend Code of coupling cost SAD; If reference picture is L, target image is R, and picture traverse is Width, is highly Height, and match window width is W, for the some L (x, y) of diverse location in image: for the some I in reference picture _r(x, y), x=W/2, y=W/2, the expression formula of coupling cost function SAD is:

$\mathrm{SAD}(x,y,d)=\underset{x,y\&Element;S}{\mathrm{\&Sigma;}}|I_R(x,y)-I_T(x+d,y)|$

In formula, d is the parallax value of corresponding points between reference picture and target image, and S is coupling cost polymerization window, I _rfor reference picture, I _tfor target image;

For a L (x, y), x=W/2, y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x,y+1,d)＝SAD(x,y,d)+U(x,y+1,d)

$\mathrm{SAD}(x,y,d)=\underset{i,j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\&Sigma;}}}|L(x+j,y+i)-R(x+d+j,y+i)|$

$U(x,y+1,d)=\underset{j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\&Sigma;}}}|L(x+j,y+\frac{L-1}{2}+1)-R(x+d+j,y+\frac{L-1}{2}+1)|-\underset{j=\frac{L-1}{2}}{\overset{\frac{L-1}{2}}{\mathrm{\&Sigma;}}}|L(x+j,y-\frac{L-1}{2})-R(x+d+j,y-\frac{L-1}{2})|$

For a L (x, y), (x=[W/2+1, W+W/2], y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x,y+1,d)＝SAD(x,y,d)+U(x,y+1,d)

U(x,y+1,d)＝U(x-1,y+1,d)+|A-A'|-|B-B'|-(|D-D'|-|C-C'|)

Wherein, A, B, C, D are the point in reference picture, and A is the pixel of the outer upper left side of match window adjacent, and B is the pixel of the outer upper right side of match window adjacent, and C is bottom-right pixel in match window, and D is the pixel of the outer lower left of match window; A', B', C', D' are the point in target image;

For a L (x, y), (x=[W+W/2+1, Width], y ∈ [W/2+1, Height]), former SAD algorithm is rewritten as follows:

SAD(x+wd,y,d)＝SAD(x+wd,y-1,d)+U(x+wd,y,d)

U(x+wd,y,d)＝U(x+wd-1,y,d)+S(x,y,d)-|B _wd-B _wd'|+|C _wd-C _wd'|

S(x,y,d)＝-|B-B'|+|C-C'|

Wherein, B, C, B _wd, C _wdfor reference picture mid point, B', C', B _wd', C _wd' be the point in target image; Centered by B, put the pixel of the outer upper right side of P (x, y, d) window adjacent, centered by C, put bottom-right pixel in P (x, y, d) window, B _wdcentered by put the pixel of the outer upper right side of P (x+wd, y, d) window adjacent, C _wdcentered by put bottom-right pixel in P (x+wd, y, d) window;

(3) with parallel instruction, rewrite core match code; For a L (x, y), (x=[W+W/2+1, Width], y ∈ [W/2+1, Height]), adopt the MMX instruction set in SIMD to rewrite former code; Particularly, original image be take to 8 points and pack as one group, be saved in and in MMX special register, carry out parallel processing; Meanwhile, the cycle-index of match search is reduced to original 1/8;

Step 3, quick stereo matching optimization, method is as follows:

(1) Rapid matching result anaglyph is carried out to left and right consistency detection; For the fast matching method of describing in described step 2, in the process of selecting in coupling cost, increase target image relatively with reference picture mate cost selection course, the parallax selection result of each point of record object image; According to result, the parallax value of target image and reference picture is compared, if the two is identical, think that the former matching result of target image is reliable, retain this matching result; If the two is not identical, think that the former matching result of target image is unreliable, give up this matching result;

(2) Rapid matching result anaglyph is carried out to the detection of parallax conspicuousness; For the matching process of target image, adopt following formula to miss matching detection:

${\mathrm{\&delta;}}_e=\frac{\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}(e_i-e_{\min })}{e_{\min }}$

Wherein, e _minfor coupling Least-cost value, e _ifor N coupling cost sub-minimum, δ _efor threshold value; If result of calculation is less than threshold value δ _e, think that matching result is not remarkable, give up this matching result;

(3) Rapid matching result anaglyph is carried out to the detection of parallax centrality; For the matching process of target image, adopt following formula to miss matching detection:

${\mathrm{\&delta;}}_d=\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}|d_i-d_{\min }|$

Wherein, d _minfor the parallax value of coupling Least-cost value, d _ifor the parallax value of N coupling cost sub-minimum, δ _dfor threshold value; If result of calculation is greater than threshold value δ _d, think that matching result is concentrated, give up this matching result;

Step 4, disparity range is extracted, and method is as follows:

(1) judgement image maximum disparity scope; The anaglyph that described step 1,2 is calculated is called Dmap, described step 3 is detected to the anaglyph obtaining and be called DmapR; In parallax hunting zone [L, L], the L of take travels through hunting zone [L, L] as initial value; In note DmapR, parallax value is counting as Cntl of l ∈ [L, L], sets the judgment threshold δ of maximum disparity scope _dmaxfor 1% of full figure pixel quantity; If Cntl is less than δ _dmax, reduce l value and continue traversal; If Cntl is more than or equal to δ _dmax, stop traversal, and l value Dmax be now recorded as to the maximum disparity scope of this stereo-picture;

(2) the minimum disparity range of judgement image; In parallax hunting zone [L, L], take-L travels through hunting zone [L, L] as initial value; In note DmapR, parallax value is counting as Cntl of l ∈ [L, L], sets the judgment threshold δ of maximum disparity scope _dminfor 5% of full figure pixel quantity; If Cntl is less than δ _dmin, reduce l value and continue traversal; If Cntl is more than or equal to δ _dmin, stop traversal, and l value Dmin be now recorded as to the minimum disparity range of this stereo-picture;

Step 5, spatial accuracy compensation, method is as follows:

(1) obtain maximum and go out to shield object space; For DmapR, with δ _dmax± t is that disparity range intercepts former anaglyph; Wherein, t is that maximum goes out to shield object identification scope, and suggestion span is t ∈ [0,2], all gets t=1 in the example of this document; For the anaglyph after intercepting, calculating and recording all has the draw of parallax value point horizontal level AvgX and average upright position AvgY;

(2) intercepting maximum goes out to shield object; If Th is the height of intercepting window, Tw is the width of intercepting window; If picture traverse is Width, be highly Height: if AvgX – is Tw/2<0, AxgX is set to Tw/2; If AvgX+Tw/2>Width, AxgX is set to Width-Tw/2; If AvgY – is Th/2<0, AxgY is set to Th/2; If AvgY+Th/2>Height, AxgY is set to Height-Th/2; Take AxgX ± Tw/2, AxgY ± Th/2 intercepts reference picture as scope;

(3) establish [δ _l, δ _l] for parallax fine setting scope, according to the ratio-dependent δ of down-sampling _lsize; At l ∈ [δ _l, δ _l] scope in, relatively in (2), reference picture and target image center of intercepting are (AxgX+l, AxgY), scope is that SAD calculating is carried out in the region of AxgX+l ± Tw/2, AxgY ± Th/2, records [δ _l, δ _l] have the position l of minimum SAD in scope _sADmin; Note Dmax+l _sADminfor the accurate maximum of this stereo-picture goes out screen scope;

Step 6, time precision compensation, method is as follows:

(1) carry out in time frame of video classification; For taking out the real-time disparity range leaching process of primary video that frame ratio is S, note frame number f _noskipthe picture of=nS is non-extraction frame, note frame number f _skip=nS+k, the picture of k ∈ [1, S-1] is non-extraction frame, n is positive integer; For f _noskipframe; Carry out the calculating of step 5, record Dmax, AvgX, the AvgY in calculating this time;

(2) calculate the accurate parallax value that extracts frame; For f _skipframe, will upper distance f of time _skipthe f that frame is nearest _noskipthe Dmax of frame, AvgX, AvgY be as Dmax, AvgX, the AvgY of this frame, and adopt this numerical value to carry out the calculating of step 5, and resulting result is the accurate parallax value of this extraction frame.

2. the disparity range method for quick of a kind of high definition three-dimensional video-frequency according to claim 1, it is characterized in that, described disparity range extraction step is applied to the scene of the rough disparity range of Real-Time Monitoring video, only need be aided with a certain proportion of frame of taking out, not need the accuracy compensation in the time of carrying out and space; Described spatial accuracy compensation process is applied to obtain high-precision disparity range fast, but does not require the scene of real-time processing, only need carry out spatial accuracy compensation, needn't carry out time precision compensation, and video is carried out to calculating frame by frame; Described spatial accuracy compensation and time precision compensation can be applied to obtain the scene of high accuracy disparity range quasi real time simultaneously, significantly improve processing speed in the situation that of a small amount of time precision of loss.