CN104639933A - Real-time acquisition method and real-time acquisition system for depth maps of three-dimensional views - Google Patents

Abstract

The invention discloses a real-time acquisition method and a real-time acquisition system for depth maps of three-dimensional views, and provides an extraction method for high-density disparity maps required in a generation process of real-time multi-viewpoint views of three-dimensional left-right views. Initial matching is performed by combining a feature matching method with a cutting-based method, and then, optimization is performed by statistics-based method to realize three-dimensional matching of the left-right views; a matching algorithm disclosed by the invention is wider in adaptability; moreover, various three-dimensional views and three-dimensional movie clips with different test features are tested to generate the disparity map comparatively accurate; key problems of repeated texture, blocking and the like are solved very well; the real-time acquisition method and the real-time acquisition system have a very good market promotion and application prospect.

Description

A kind of depth map real time acquiring method of three-dimensional view and system

Technical field

The present invention relates to bore hole 3D and show field, be specifically related to the processing method of 3D view signal, and the multiple views bore hole 3D stereoscopic display device of this type.Relate to the depth extraction method of real-time Stereo Matching Technology and the multi-viewpoint three-dimensional display device of improvement and many picture windows further.

Background technology

The stereo display technique that the bore hole stereo display technique of multiple views is compared to single view can allow the spatially different position of observer obtain different images to the object of observation or scenery, more close to the visual experience in reality.In medical treatment, teaching, the fields such as television broadcasting have broad application prospects.Along with stereo display technique universal of single view, three-dimensional signal source increasingly abundant, generates multiple views view and highlights more and more important researching value from three-dimensional view or video.

Mostly current stereo display technique is application technology condition, allows two to see respective view respectively, two width views people vision system and psychology effect under, the depth information in perception view.Thering is provided depth information to be given to the vision system of people by the horizontal parallax (horizontal disparity) in view, is the main method of 3D stereo display technique main at present.

Wherein, first the acquisition of depth image will obtain disparity map by the algorithm of Stereo matching from left eye and right-eye view, then draws final depth map.

From the right and left eyes view containing parallax, obtain depth information figure accurately, be the part of the basis that rebuilds the stereo-picture of multi views and difficulty the most.Due to generation (DIBR) technology will adopted based on the multi views of depth information after this paper, require that the depth information figure precision obtained is high, dense, preserving edge information, and level and smooth in time domain.

In addition, the distinct issues embodying the inside in the analysis by the theoretical system to the global optimization of MAP-MRF model are the methods not provided the structure of unique energy function by demonstration.Even if the minimized approximate solution of the energy function asked in other words, because energy function itself is not with having between required joint probability probability distribution to set up fully necessary corresponding relation, must there is deviation in the approximate solution solution of so finally trying to achieve.And reveal to come from various research work and document, the difference of the building mode of energy function, will finally form different matching accuracy and disparity map effect.

The energy function employing belief propagation approach that structure is conventional and figure segmentation method solve the disparity map that label obtains, from the result of the standard testing picture of http://vision.middlebury.edu/ website, also prove that the operation method of its iteration itself can not solve to block, the problems such as parallax discontinuity.Above improve the process of energy function structure, essence is the optimization process apriority constraints of view to Stereo matching being incorporated into energy function, is namely the process improving likelihood energy function U (d|f) and priori energy U (f).

Therefore, according to analysis above, although adopt the energy function improved and the derivation algorithm adopting improvement, effectively iteration time can be reduced, restrain faster, but figure cuts and still has higher implementation complexity and time cost with the method for confidence spread.Prior art has yet to be improved and developed.

Summary of the invention

The object of the present invention is to provide a kind of depth map real time acquiring method and system of three-dimensional view, be intended to propose one and multiple method is combined, be applicable to the acquisition methods with the depth map of the real-time transformation applications of video.For the discontinuous problem of parallax, repeat texture error hiding problem, and the problem of occlusion area error hiding adopts corresponding solid matching method, and the result of these methods is optimized and merges, finally carry out error protection and the correction process of disparity map.

Technical scheme of the present invention is as follows:

A depth map real time acquiring method for three-dimensional view, wherein, comprises the following steps:

S1, combine carry out initial matching by feature matching method with based on the method for segmentation, obtain the initial parallax figure of two views in left and right;

S2, initial parallax figure adopted be optimized based on the method for parallax probability statistics, realize the Stereo matching of left and right view;

Wherein, based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described step S1, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

S11, first view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

S12, the histogrammic front M peak value searched in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

S13, compare the gray scale difference value of the peak value searching acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in S14, mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point;

Described step S2 comprises further:

S21, through the condition that presets, the first ground control point is determined to the point in initial parallax figure;

S22, a statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

S23, initial parallax figure to be carried out to gradient level and smooth, and the information of recycling depth information and neighborhood image carries out interpolation.

The depth map real time acquiring method of described three-dimensional view, wherein, wherein, adopts the sobel operator with neighbor smoothing effect to carry out feature point detection by feature matching method in described step S1;

All first left and right view is divided into 16 × 16, the block of pixels of 32 × 32 or 64 × 64.

The depth map real time acquiring method of described three-dimensional view, wherein, in described S2, the first ground control point is divided into reliable point and unreliable point;

Wherein, the formation of described reliable point is divided into two classes: a class is the ground control point that the strong edge in view is formed, and two classes are ground control points that texture pixel is formed;

The formation of described unreliable point is divided three classes: occluded pixels point, flat pixels point and the improper point of coupling.

The depth map real time acquiring method of described three-dimensional view, wherein, carries out gradient to initial parallax figure in described S23 and smoothly comprises: three-dimensional filtering.

A depth map real-time acquisition system for three-dimensional view, wherein, comprising:

First matching unit, for combining carry out initial matching by feature matching method with based on the method for segmentation, obtains the initial parallax figure of two views in left and right;

Second matching unit, for adopting the method based on parallax probability statistics to be optimized to initial parallax figure, realizes the Stereo matching of left and right view;

Wherein, based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described first matching unit, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

First view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

Search the histogrammic front M peak value in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

Relatively search the gray scale difference value of the peak value of acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point;

Described second matching unit comprises further:

Through the condition preset, the first ground control point is determined to the point in initial parallax figure;

One statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

Carry out gradient smoothly to initial parallax figure, the information of recycling depth information and neighborhood image carries out interpolation.

The depth map real-time acquisition system of described three-dimensional view, wherein, adopts the sobel operator with neighbor smoothing effect to carry out feature point detection by feature matching method in described first matching unit;

All first left and right view is divided into 16 × 16, the block of pixels of 32 × 32 or 64 × 64.

The depth map real-time acquisition system of described three-dimensional view, wherein, in described second matching unit, the first ground control point is divided into reliable point and unreliable point;

Wherein, the formation of described reliable point is divided into two classes: a class is the ground control point that the strong edge in view is formed, and two classes are ground control points that texture pixel is formed;

The formation of described unreliable point is divided three classes: occluded pixels point, flat pixels point and the improper point of coupling.

The depth map real-time acquisition system of described three-dimensional view, wherein, carries out gradient to initial parallax figure in described second matching unit and smoothly comprises: three-dimensional filtering.

The depth map real time acquiring method of three-dimensional view provided by the present invention and system, propose the extracting method of the high density disparity map needed for a kind of real-time multiple views view generation process from three-dimensional left and right view.Carry out initial matching by adopting feature matching method and combining based on the method for segmentation, then adopt Statistics-Based Method to be optimized, realize the Stereo matching of left and right view.Matching algorithm in this paper has wider adaptability.And adopt the three-dimensional view of multiple different test feature and three-dimensional film fragment to test, generate disparity map more accurately, to repetition texture with the critical problem such as to block and achieve good result, have good marketing application prospect.

Accompanying drawing explanation

Fig. 1 is the flow chart of the depth map real time acquiring method of three-dimensional view of the present invention.

Fig. 2 is the schematic diagram of Region Segmentation in the depth map real time acquiring method of three-dimensional view of the present invention.

Fig. 3 a, Fig. 3 b and Fig. 3 c are respectively the schematic diagram of the similarity mode of three kinds of parallaxes in the embodiment of the depth map real time acquiring method of three-dimensional view of the present invention.

Fig. 4 is the schematic diagram that in the depth map real time acquiring method of three-dimensional view of the present invention, neighborhood is chosen.

Fig. 5 is the structured flowchart of the depth map real-time acquisition system of three-dimensional view of the present invention.

Fig. 6 is the schematic diagram in extension block 8 directions in the depth map real time acquiring method of three-dimensional view of the present invention.

Embodiment

The invention provides a kind of depth map real time acquiring method and system of three-dimensional view, for making object of the present invention, technical scheme and effect clearly, clearly, the present invention is described in more detail below.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.

Refer to Fig. 1, it is the flow chart of the depth map real time acquiring method of three-dimensional view of the present invention.As shown in the figure, the depth map real time acquiring method of described three-dimensional view comprises the following steps:

S1, combine carry out initial matching by feature matching method with based on the method for segmentation, obtain the initial parallax figure of two views in left and right;

S2, initial parallax figure adopted be optimized based on the method for parallax probability statistics, realize the Stereo matching of left and right view.

Specifically, in order to make the present invention have reasonable subjective effect in the three-dimensional video-frequency DIBR process of reality, require have than good treatment effect for the most typical difficulties in Stereo matching.Therefore method design of the present invention is respectively for the discontinuous problem of parallax, repeat texture error hiding problem, and the problem of occlusion area error hiding adopts different solid matching methods, and the result of these methods is merged and optimized.Whole method is the thick coupling of feature based coupling, based on the initial matching split in block and the global optimization three phases based on parallax probabilistic method, finally carries out the reprocessing of disparity map.

Namely the method split in the fixed block for the discontinuous employing of parallax, for repeat texture adopt the method based on Texture Points feature point detection, for occlusion area and based on global disparity probability statistics estimate method.

Based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described step S1, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

S11, first view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

S12, the histogrammic front M peak value searched in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

S13, compare the gray scale difference value of the peak value searching acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in S14, mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point.

Specifically, the present invention improves relative to the histogrammic dividing method of prior art, has been applied in the middle of Stereo matching.And in the application the histogrammic feature of some steps wherein according to block image is optimized.When tonal gradation is 256, the pixel that image comprises is more, clusters and more easily occurs.Because what in this programme, we adopted is carry out histogram divion in small pixel block, pixel quantity is limited, such as, are 256 pixels when the window of employing 16 × 16.Flatness and the continuity of histogram bunch all can be poor, easily there is more local peaking.So in order to obtain the result that same object segmentation meaning matches, need addition of constraints condition to choosing of peak value, remove distracter.The constraint of the consideration in this programme has:

(1) numerical value of peak value itself must be greater than the value of certain setting.

(2) adjacent too near peak value illustrates gray scale closely, merges into a peak value.

And for when asking for variance maximum in standard OSTU method, the mode traveling through all gray values is unusual consumption calculations time.Adopt after having demarcated effective peak herein, the way of the valley between mark effective peak.Cut-point travels through in these valleies.In experiment in a lot of situation, all can produce the situation that valley is 0, just select this time this point to be cut-point.Adopt the mid point of two peak values also obvious as the segmentation result physical significance of cut-point to fritter in an experiment.

The method implementation procedure repeating to split of the multi thresholds that this programme adopts is as follows: the window first view being divided (n × n), each window carries out multi-threshold segmentation based on the histogram distribution of gray scale.Setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are being carried out Gaussian smoothing.The number of peaks to vicinity in histogram can be reduced like this, avoid gray scale point closely to do the segmentation that there is no need simultaneously.And the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes.Stereo-picture is divided into multistage window, adopts the method based on segmentation in block and polylith merging improved to carry out disparity map coupling.In block segmentation adopt simultaneously based on improve the method for grey level histogram and the dividing method based on colored self-propagation, do all respectively and mate.Because human eye is low for the susceptibility at the edge of colour, so intensity slicing guarantees the accuracy of marginal information, Color Segmentation strengthens the actual physics meaning of segmentation.

Search the histogrammic front M peak value in block again; According to the threshold value Th (n) of the size of window setting effective peak, what be greater than that Th (n) is just thinks effective peak.Relatively search the gray scale difference value of the peak value of acquisition; Gray scale difference value is less than threshold value Th (g) and merges into one and have effective peak.Valley in mark histogram between effective peak, respectively as threshold calculations variance.Choose make inter-class variance maximum using valley as cut-point.Specifically, three-dimensional view is divided into less m*n rectangular block and larger M*N rectangular block, the grey level histogram Threshold segmentation that the carrying out of pixel is improved in less piecemeal: search the histogrammic front M peak value in block, the maximum number cutting block that setting allows is M; According to the threshold value Th (n) of the size of window setting effective peak, what be greater than that Th (n) is just thinks effective peak.Relatively search the gray scale difference value of the peak value of acquisition; Gray scale difference value is less than threshold value Th (g) and merges into one and have effective peak.Valley in mark histogram between effective peak, respectively as threshold calculations variance.Choose make inter-class variance maximum valley as cut-point, or valley is that the point of 0 is as cut-point.

In addition, the Color Segmentation of self adaptation growth can also be carried out in the window of secondary simultaneously.The example of a basic block of colored growth is 2*2 (being not limited only to 2*2), the direction eliminating eight fields is carried out growth and merges.The definition in extension block 8 directions of growth as shown in Figure 6.The criterion that central point W0 carries out merging growth on the direction of eight neighborhood is the colored similarity criterion of the polymerization of four points in block, the sad value of Y U V tri-chrominance components.

δ (p, q)=| Y _p-Y _q|+| U _p-U _q|+| V _p-V _q| the cut-off condition of additional merging growth is the strong edge having gray scale in the direction of growth.In order to convergence speedup, the four neighborhood blocks of W0 not as the growth point that next time grows continuously, and only have the next block in W8, W7, W6, W5, W4 direction, are both ending the block of two the block coordinate width of block in the direction of growth as the central point increased continuously

Finally, completing after to reference map segmentation, associating similarity criterion is adopted.Based on Y U V tri-chrominance components sad value δ (p, q)=| Y _p-Y _q|+| U _p-U _q|+| V _p-V _q| (formula 3-1) and Census conversion is carried out to gray component, compare the SAD of the difference of Hamming distance to replace the sad value of general only gray scale.Wherein colored associating sad value priority is high.

In experiment in a lot of situation, all can produce the situation that valley is 0, just select this time this point to be cut-point.Adopt the mid point of two peak values also obvious as the segmentation result physical significance of cut-point to fritter in an experiment.

The window value n=16 chosen in this article, n=24 ,=32.The complexity that the variance limiting each segmentation calculates.Simultaneously because secure the number M of segmentation and the threshold value Th (n) of effective peak is constant, the sub-fraction of object is merged into the phenomenon in large region by the edge appearing in rectangular window.In order to improve this situation to the impact of searching for match block in alignment image, we adopt the process repeating segmentation with region, as shown in Figure 2.Region ABDE is the window of 24 × 24, adopts above-mentioned process to split, and mates in aligning figure, then adopts same process to split and mate to region BCEF.In vertical direction, respectively D E G H and E F H I is split and mated.The result of segmentation remains edge in view and details preferably.Preserving edge information is by most important for the coupling accuracy of the discontinuity zone to the parallax solved in coupling.

Complete after to reference map segmentation, adopt the SDA value of Y U V associating to replace the SDA value of only gray scale as matching similarity function.Do the accuracy improving initial matching further like this.In aligning figure, adopt colored SDA value to be used as the cost of mating.When this programme design said method mates, compare the form of three kinds of matching similarity measurement functions, be respectively the sad value based on gray scale, based on the sad value of Y U V tri-chrominance components, and use morphing to carry out Census conversion to gray component.The mode of the sad value of pixel intensity in block is wherein only adopted to mate accuracy minimum; And the method adopting Census to convert, the Hamming distance of corresponding points new in computing block and mode, to the parallax of the gradation zone in some view etc. more accurately with having flatness.Being example at the resolution chart of conventional cones and teddy bear, adopting the sad value of chrominance component than only adopting the matching precision of the sad value of gray scale to exceed general about 6%.In this process, a special problem is: in the histogrammic repeated matching method adopted herein, the region of selected block size is repeated quickly and easily as many times as required segmentation.Simultaneously each segmentation all with the primitive of the arrived subregion gathering that is Matching power flow, and is all mated in aligning figure, makes it likely have multiple different parallax value in the process of repeatedly mating.Get the parallax value as uniquely tagged of wherein minimum parallax value this time.The part of the hand of the lower right corner teddy bear such as, in figure below in the E of region is the sub regions be wherein partitioned into, aim in view along the horizontal direction shown in dotted line, mate with the region that same shape is, the difference of the SAD of the brightness Y of zoning and chrominance component U and chrominance component V.In the disparity range of setting, the SAD of Y U V tri-components of the block moved horizontally in the block of reference map and aligning figure with time minimum, then think and find coupling parallax.

With by repeat to split the maximum region E of number of times for example, it is as follows that it lacks the process of determining parallax value:

dif_vdl(s)＝abs(y_l(j,i)-y_r(j,i+vdl_sel(s)))+

abs(u_l(j,i)-u_r(j,i+vdl_sel(s)))+

abs(v_l(j,i)-v_r(j,i+vdl_sel(s)))；

[A,index]＝sort(dif_vdl(1),dif_vdl(2),dif_vdl(3),dif_vdl(4))；

blk_vdl(j,i)＝vdl_sel(index(1))；

Dif_vdl (s) represents that left view is reference map, calculates the absolute value of the pixel in this segmentation subregion with the difference of the Y U V component of four repeated matching points in right figure as matching similarity, s=1,2,3,4.

Then by obtain 4 colored SAD's with sort, and get the parallax of minimum value as this segmentation block.

Then, bulk region stereo-picture is mated:

View is divided into three grades of window sizes by the thick matching algorithm of this programme design, in every grade of window, all carry out the detection of soble operator.The performing step of stereo matching algorithm is as follows:

1) reference view and aligning view are carried out to the setting of adaptive threshold in the block of Matching unit setting:

Set two 3 support windows, its size is respectively W1*2W1, W2*2W2, W3*2W3 (W1 < W2 < W3); To maximum max (y) and the minimum value min (y) of the gray scale in each window, and set the threshold value of sobel rim detection:

Th1＝×(max(y)-min(y))

Th2＝×(max(y)-min(y))

Th3＝×(max(y)-min(y))

2) feature point detection and description

Algorithm carries out the strong rim detection of large threshold value and the skin texture detection of little threshold value in maximized window simultaneously, exports the information needed for all couplings.First Sobel gradient operator is adopted to calculate its Grad respectively to pixel in window.When its Grad is greater than respective threshold value, then obtain Edge Feature Points P k (k=1,2 ...), not and obtain quantity of these points and gradient magnitude and direction.

The information detecting logic output comprises:

A) obtain the gradient magnitude of all characteristic points and the value in direction and its size be reduced to the value that can represent with two bit;

S＝1；(<<2)

S＝2；(<<4)

S＝3；(<)

The quantity of b) total in block edge pixel point;

The pixel coordinate (x, y) of the Edge Feature Points c) obtained under large threshold value setting;

D) texture obtained under little threshold value setting and the pixel coordinate (x, y) of Edge Feature Points;

When the quantity of the Edge Feature Points of minimum window is less than certain threshold value Th (texture), then secondary windows is proceeded to detect as Matching unit.Namely second level window can regard an automatic growth of minimum window as, and this two-stage window will only carry out a match search.

When Edge Feature Points quantity in maximized window is less than certain threshold value Th (edge), maximized window is increased in the horizontal direction and is twice, as the new maximized window applied in later step.

3) for the block of pixels of different size generates Matching power flow polymerization

E) in maximized window, large threshold skirt detects relative coordinate (x, the y) list of characteristic point;

F) sad value of the gradient magnitude of one-level window and secondary windows characteristic point;

G) sad value of the direction value of one-level window and secondary windows characteristic point;

H) there is in block the apex coordinate corn1 () of maximum x and y and minimum x and y; Corn2 (); Corn3 (); Corn4 ();

4) matching similarity is calculated in disparity range in the horizontal direction

A) for maximized window, the difference of trying to achieve the quantity of the edge pixel point in list in alignment image and benchmark image is less than threshold value Th=× N, then think large Block-matching.N is that benchmark image block is in step 2) in the quantity of characteristic point in whole piece of record.

B) after the list that the block of reference map and aligning figure obtain, by reference map from step 3) the list that obtains compare the comparison of sad value respectively with the list of the acquisition of each piece in the disparity range of aligning figure.In order to reduce required operation time (total clock number) scope in, can first carry out Grad and compare, the sad value of the point that the sad value of Grad is less than the threshold value of setting calculated direction again.

5) in the search window of horizontal parallax scope, obtain mating between two blocks according to above-mentioned two kinds of differentiations, the difference d=X1-X2 of horizontal coordinate label is composed the parallax value for block.

In above-mentioned implementation procedure, two kinds of different matching similarities realize cost slightly difference, also will obtain not convertible parallax.Adopt mode implementation complexity Least-cost a), but contain all pixels in bulk simultaneously all there is the hypothesis of same disparity value.Therefore be applicable to maximum block window and larger block window, experimental result shows to make moderate progress in conjunction with the disparity continuity of this mode for large-area flat site on the matching process basis based on segmentation.When adopting the mode that b) describes to mate, coupling can obtain to characteristic point when adopting large Block-matching, and the point inside and outside apex coordinate scope or edge line segment is more likely the point beyond object boundary, and may have parallax that is different and characteristic point.Therefore in this module, the meaning of output vertex coordinate is that the parallax fusion process for detecting based on segmentation and feature based provides reference.

Above-mentioned based on segmentation and correspond based on the value that two kinds of matching process of texture and rim detection parallax make in reference map pixel have three parallaxes.In the search procedure in pixel level direction, according to the method introduced below, these three kinds of parallaxes are merged, obtain the parallax value of just coupling, as shown in Fig. 3 a, Fig. 3 b and Fig. 3 c.

In figure, abscissa represents the disparity range of the search of horizontal direction, ordinate represent based on segmentation block in a pixel carry out in matching process, the matching similarity cost obtained under abscissa.Be in this article the sad value of Y U V tri-chrominance components and.When there is the situation in Fig. 3 a, thinking that this pixel textural characteristics is clear and definite, not belonging to repetition texture region.Namely the value that its abscissa is corresponding is taken as the parallax value of just coupling.When there is the situation shown in Fig. 3 b, often this pixel is in repetition texture region, cannot to judge in aligning figure which is as the match point of its correspondence.So parallax value is herein by the parallax result of value in the bulk based on texture and rim detection.When appearance is from the situation shown in Fig. 3 c, although there is the pixel of clearer and more definite smallest match cost in possible disparity range, but smallest match cost itself is larger than the threshold value of setting, can not think that it is reliable matching, thus the parallax value that the parallax value of this pixel will be got bulk and slightly mates.

Described step S2 comprises further:

S21, through the condition that presets, the first ground control point is determined to the point in initial parallax figure;

S22, a statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

S23, initial parallax figure to be carried out to gradient level and smooth, and the information of recycling depth information and neighborhood image carries out interpolation.

Specifically, in stereoscopic vision research, the concept of ground control point (GCP) is to represent in disparity map with the actual reliable parallax point be consistent.Stereo matching obtains GCP can not ensure its accuracy completely, but still can ensure its " high reliability ".In this programme, obtain the basis that the disparity map for the overall situation is optimized by GCP point.Above-mentioned all carries out reference view and aligning view based on the method repeating to split with based on the method for texture rim detection simultaneously, in the principle above-mentioned according to this section after the fusion of two kinds of couplings, obtain two the view two width initial parallax figure in left and right respectively, the point in this initial parallax figure thinks initial GCP point (i.e. the first ground control point) through the judgement of following condition:

Condition is ((abs (vdr (j, i)+vdl (j, i+vdr (j, i))) <=1 a);

Condition b) vdr (j, i)+vdl (j, i+vdr (j, i)+1)==0;

Condition c) vdr (j, i)+vdl (j, i+vdr (j, i)-1)==0;

Condition d) abs (h_filter (dif_h, y_r)) >hf_h_th;

Wherein vdr (j, i) and vdl (j, i) represents that in right view and left view, coordinate is the initial parallax value of the point of (j, i) respectively; The coordinate of point represented by the point (j, i) of v right view in left view should be (i+vdr (j, i)), so required by left view disparity map in, the parallax of this point is vdl (j, i+vdr (j, i)).If the parallax value asked is accurate, should to obtain absolute value identical with scheming coupling left from right figure so to scheme coupling to the right from left figure, a pair parallax value that symbol is contrary.In order to compatibility error in a small amount, condition a) in represent that the absolute value of the GCP parallax value of two matching direction gained can allow the change of 1.At condition b) and condition c) in, the position of GCP pixel is left respectively to the tolerance of 1 pixel at left and right directions.Condition d) what state is the differential filtering that pixel in view does simple horizontal direction, think that when difference is greater than threshold value th this pixel is the point with texture features or local edge, ensure the reliability of its parallax value.Condition d) though introducing be because the flat site of the few texture in view meets above condition, also not representing that the coupling of both direction obtains is same coupling corresponding points.And the uniqueness with the coupling of the pixel of textural characteristics is can be guaranteed.

After obtaining initial parallax figure and GCP of left and right view, this programme describes and carries out the implementation procedure of probability probability optimization according to above-mentioned thought and step for initial parallax figure on this basis.First the calculating carried out in this process is to the statistical information setting up a little (Y, U, V, vd, N) in GCP figure.Wherein vd is the parallax value of pixel, and N is that colour is described as (Y U V), and parallax is the number of the pixel of vd.Y U V and vd is respectively 8bit numerical value, and in the hardware implementing of reality, certainly needing them, it compresses.The way of high 4 or high 6 of the data of direct intercepting 8bit is the most simple mode.But find that this way is by making the differentiation of colored similitude accurate not, causes the different objects being in prospect and background respectively may be assigned identical parallax value in an experiment.In order to avoid this problem, save hardware resource, this experiment have employed carries out histogram equalization by the grey scale signal in view simultaneously.After entering histogram equalization, the difference distribution of gray scale Y is more even.The scope of [0 ~ ~ 255] again Y U V tri-components represented from 8bit by this basis is linear [0 ~ ~ 63] span being mapped to 6bit and representing respectively.

To in the process of specific implementation, in order to set up the convergence strategy of rational parallax value, the reliable point in GCP figure and unreliable point are classified all respectively.In GCP figure, the formation of reliable point is divided into two classes:

One class is the GCP that the strong edge in view is formed, and the criterion of classification is: the point that some gcp (x, y) is accurate GCP figure; When pixel p (x, y) for carrying out endpoint detections in thick coupling simultaneously, the test point that sobel operator sets larger threshold value and obtains.

An other class is the GCP that texture pixel is formed, and removes the point outside GCP that strong edge forms in accurate GCP figure.The subset of the texture characteristic points that these gcp (x, y) obtain when being rim detection is the point of the textural characteristics determining unique parallax value.

Due in large Block-matching before this paper of the information of texture pixel dot information and strong edge pixel point as calculated and store, above-mentioned classification does not need new calculating and hardware resource.

Unreliable some formation in GCP figure is divided three classes: occluded pixels point, and flat pixels point, mates improper point.The reaction of occluded pixels point is being blocked in GCP figure, but the occluded pixels of blocking in GCP figure is subject to the impact of the blocking effect of bulk skin texture detection matching result, and its edge is also not accurate enough; If GCP mid point o is (x, y) be non-reliable point in accurate GCP figure, pixel p (x, y) in the view that this position is corresponding simultaneously has again texture information feature, and so it may for having the occluded pixels point of texture information or mating improper point.Be thought of as partial occlusion herein to occluded pixels, be namely in a part for the object of background by foreground occlusion, another part of object still can by accurate match.According to the colored similitude of same object, the parallax value of this occlusion area should obtain in the disparity estimation of appropriate area.Mate improper point to be pixel there is texture information, but feature interpretation and the coupling of pixel can not be realized accurately due to reasons such as the similarity measurement function of coupling and the polymerization methodses of Matching power flow.So the non-reliable point in accurate GCP figure is divided into two classes in optimized algorithm: a class is non-reliable in accurate GCP figure is also simultaneously the point with texture and edge feature, includes the occluded pixels point and the improper point of coupling with textural characteristics.Another kind of is flat site point, and it is the non-reliable point in accurate GCP figure, does not also have texture information in the view simultaneously.

For the non-reliable point of the first kind, in this programme, select the non-reliable point blocked in GCP.The weight of the parallax peak value of the GCP that the texture pixel with high colored similitude in pocket is formed should be chosen when carrying out disparity estimation.Reducing strong edge GCP can effectively avoid object boundary to be in error propagation in disparity map to the parallax effects of this kind of point.For the non-reliable point of Equations of The Second Kind, then increase the parallax transmission of the GCP of neighborhood.Comprise in neighborhood the transmission of the parallax at the strong edge with very strong colored similitude and the whole secondary view overall situation have the GCP of high colored similitude the weight of parallax value.

After reference view being carried out to gray-level histogram equalization and yuv data compression, as follows to the non-reliable point of the first kind-the block process of a little carrying out assignment:

1) set up a little vector lists (Y, U, V, vd) in the accurate GCP figure of view picture view, vd is corresponding parallax value;

2) add up in view picture GCP figure (Y, U, V, vd, N), N is in view picture view, the number of component to be (Y, U, V) parallax the be pixel of vd.

3) view picture picture is divided into the subregion of 3 × 3, statistics (Y, U, V, vd, Ni), wherein Ni is the number of component to be (Y, U, V) parallax the be pixel of vd in i-th piece of subregion.Set up vector lists (Y, U, V, vd, N, N1,,, N9).

4) first first kind occluded pixels is carried out tabling look-up assignment in every sub regions.Each pixel is searched in vector table (Y, U, V, vd, Ni) has same color component Y U V, and vd in the maximum vector of the value of Ni is as the temporary transient parallax value of oneself.

5) secondary statistics is carried out: do the concentricity regional window of multiwindow the 3*3 region of view picture diagram root each and do secondary together and repeat statistics, histogrammic statistical value after statistics with histogram value at this window and the merging at this window and center window compares, this parallax of conduct that statistical value is the highest.

6) carry out secondary to block point diagram and obtain: the disparity map that right view obtains as reference map carry out step 5) in obtain into block coupling point diagram after probability optimization.At this moment block coupling point diagram in do not mate place, be mainly occlusion area time not good enough.

The non-reliable point of Equations of The Second Kind and flat pixels point are by removing occluded pixels point as the non-reliable point in GCP point obtain accurate.

The process of carrying out assignment is as follows:

1) set up a little vector lists (Y, U, V, vd) in the accurate GCP figure of view picture view, vd is corresponding parallax value;

2) add up in view picture GCP figure (Y, U, V, vd, N), N is in view picture view, the number of component to be (Y, U, V) parallax the be pixel of vd.

3) view picture picture is divided into the subregion of 3 × 3, statistics (Y, U, V, vd, Ni), wherein Ni is the number of component to be (Y, U, V) parallax the be pixel of vd in i-th piece of subregion.Set up vector lists (Y, U, V, vd, N, N1,,, N9).

4) in accurate GCP figure, the transmission of neighborhood parallax value is carried out for Equations of The Second Kind flat pixels.Its neighborhood is chosen, and as shown in Figure 4, comprising:

A) accurate GCP point;

B) textural characteristics pixel;

C) non-strong edge pixel point;

D) sad value of the chrominance component of the Y U V of concentricity pixel P0 is less than threshold value th;

When the quantity of the point of the above-mentioned condition that satisfies condition while in P1, P2, P3, P4, P5, P6, P7, P8} is greater than 3, then get the parallax that this intermediate value satisfied condition is used as current P0 point:

Vd(x,y)＝Mediam(vd_Pi)；

Through said process, the Mismatching point in flat pixels point can be reduced, keep the flatness of reliable area.

4) first remaining Equations of The Second Kind flat pixels is carried out tabling look-up assignment in every sub regions.Each pixel is searched in vector table (Y, U, V, vd, Ni) has same color component Y U V, and the maximum vector of value in vd as oneself temporary transient parallax value.

5) in vector table (Y, U, V, vd, the Ni) search of the overall situation, there is same color component Y U V, and the vector that the value of getting N is maximum.If the value of N and step 4) in the comparing of value of N, when the vd difference of corresponding vector is large.Get parallax in global vector table as this pixel parallax value.

After have passed through process, will can not carry out assignment again by pixel in initial parallax figure herein.Find still to need to be further improved result in experimentation.Such as in final result, still may there is obvious blocking effect, when namely flat pixels point strides across the border of piecemeal, assignment in one block.Or in the process to flat pixels point again assignment, although when part foreground image and part background image space length distant, when but the similitude of chrominance component is very high, the parallax value of assignment mistake may be there is, occur having obscured prospect and background to some pixels.Therefore error detection is carried out, to find manifest error assignment to the result of the disparity map produced.The protection increased in algorithm is herein the error detection that the assignment detecting flat site point in fixing statistical window upgrades.Method uses simple filter { 1,0 ,-1} compute gradient in the marginal position of block respectively to the gray scale Y value of pixel and the parallax value obtained.If change does not appear in the gray value of the pixel of multiple quantity, and the gradient of parallax value is comparatively large, then assignment mistake in little statistical window is described.After this process can be put into the assignment procedure (4) to the non-reliable point of two classes, if there is assignment mistake to occur then selecting the employing neighborhood in step (5) or the parallax value in Global Vector to carry out assignment renewal.

In order to improve the accuracy of disparity map further, under the condition realizing resource permission, secondary GCP can be carried out and scheme statistics.The acquisition of the secondary GCP figure on normal meanings needs aligning figure and reference map intermodulation, generates the probability optimization disparity map in another sub-picture of three-dimensional view view centering with above-mentioned optimizing process, and produces secondary GCP that blocks and scheme.Will more close to the meaning that " blocking GCP figure " states in new GCP figure, namely most non-GCP points is all occluded pixels, certainly also comprises the improper point of coupling.The non-reliable point upgraded in second time assignment procedure with blocking in GCP upgrades, and can improve the matching precision in flat site in disparity map and error hiding region on the basis that have passed through optimization assignment once further.The storage resources needing almost to need to double and the time delay that increases by a whole frame again realize by such way in optimizing process.

After obtaining disparity map through above-mentioned steps, the reason of two aspects is had to need the disparity map to generating to carry out reprocessing further.One is owing to still there is manifest error information in the reason disparity maps such as error hiding, can be repaired.In statistic optimization herein, because the parallax of the non-reliable point based on individual element upgrades, in the disparity map obtained, have the match point of the mistake of some isolated several pixel compositions.Such as the parallax value of single pixel or adjacent several pixels has obvious difference than its neighborhood, does not have correlation with adjacent view value.If this mistake appears at texture region, so carry out next step based on when the playing up of depth information, the positional fault of picture element interpolation can be caused, affect final viewing quality.This mistake is referred to the noise in depth image by us, attempts to adopt the noise reduction algorithm in Digital image post-processing to remove.

On the other hand, studied herein application provides depth information to play up i.e. DIBR to the multiple views view figure based on depth information.In DIBR Rendering, core methed in its process realized of 3D conversion (3Dwarping) and basic steps, namely according to the depth information of three dimensions point corresponding to each pixel in source images, pixel is transformed to the process in the image plane under new viewpoint.Need in DIBR technology solve a key problem be due to exist in the view eclipse phenomena and picture expansion phenomenon, in the new view produced, often create the inc phenomenon in view, often be called cavitation, must solve this problem is the key of DIBR technology.So can by improve cavitation to the preliminary treatment of disparity map and the method that utilizes the neighborhood around pixel to carry out interpolation in the virtual view generated.Generally, the change of depth map large gradient in the horizontal direction can cause undesired cavity to produce in render process.Carry out to disparity map the area that gradient smoothly can reduce the cavity produced to a great extent, then, inevitably little cavity, again by the method for image procossing, utilizes the information of depth information and neighborhood image to carry out interpolation.Therefore, herein stereoscopic video sequence carry out actual application evaluation and test and after analyzing, add the process of the disparity map produced being carried out to reprocessing.Gradient in disparity map is also carried out level and smooth in this process simultaneously, thus advantageously in improving the view generation effect based on depth information Rendering.

Further, the application of this programme also comprises dynamic video image, find in the process of the depth extraction and multi-view image observation of carrying out stereoscopic video, relative to the more satisfactory display effect of static stereoscopic to view, often occur in stationary part in the view in continuous print stereoscopic video sequence, still there are some texture structure and occlusion area, in continuous print two frame depth image, still likely have a small amount of pixel or the block of the different depth value that difference is larger.Even if therefore the peripheral region of these pixels is static, different pixel values can be given by virtual pixel herein in the render process of virtual view below.And for large-area image without texture flat site, even if depth map has a small amount of isolated mistake, when neighborhood territory pixel around render process produces new pixel, neighborhood territory pixel is all same value, does not thus reflect the mistake of depth information in virtual view.These and neighborhood have the depth value of significant difference to have opposite segments to be because the robustness of algorithm is not high, because the change of view picture view content, can calculate the parallax value of mistake to not vicissitudinous pixel region.

Therefore, three-dimensional filtering method and the noise-reduction method of interframe also should in the middle of the scopes considered.The many estimation based on interframe of noise reduction filtering method of interframe, adopt interframe level and smooth to the pixel of stationary part, can adopt two-dimensional filtering or adopt the filtering of frame difference signal to the image of motion parts.The hardware complexity of this process need is high, but in current most video post process chips, all done effect realizes quite accurately, such as, at 3-D view decrease of noise functions, and based on the frame rate conversion merit function etc. of Motion estimation and compensation.The motion estimation module of video post process chip can provide the movable information of each pixel in view, the realization of the noise reducing of disparity map can utilize these movable informations, depth map is used as common frame of video be inserted in the flow process of the noise reducing of Video post-processing and carry out, does not need new modular design.From the angle of system cost, because the cost that the unit capacity of the dynamic memory RAM such as DDR3 outside sheet stores declines fast, therefore reducing logical resource in sheet in possible from now on hardware implementing and make full use of the outer larger dynamic memory resource of sheet, is the scheme of the consideration of value with the time domain continuity that the comparative approach between simple-frame improves disparity map.Step is as follows:

1) disparity map of present frame does difference with the disparity map of former frame, and mark difference is greater than the pixel Vd_edge (x, y) of threshold value th.

2) pixel of Vd_edge (x, y) position is done difference at the gray value of active view with the gray value of former frame, when the difference of gray scale is less than threshold value th_l, mark vd_error (x, y).

3) in disparity map, calculate the average vd_avrage_p (x, y) of the spatial neighborhood of vd_error (x, y); The average vd_avrage_c (x, y) of the parallax of vd_error (x, y) neighborhood is asked in current disparity map.

4) average of vd_avrage_c (x, y) and vd_avrage_p (x, y) is upgraded the parallax value of current frame.

Finally, in the effect to proposed algorithm, undertaken by Matlab, in the process verified, first adopting the test pattern of conventional static state.All have the image spatial feature of generally acknowledged difficulty in treatment in every sub-picture, such as, repeat texture region, without texture flat site, prospect background has the region etc. of same color.By these test pictures, discovery algorithm that can be clearer and more definite and the treatment effect of realization to these aspects, be conducive to doing algorithm and realization improving targetedly.Can see from the test pattern of standard and the image measurement of three-dimensional film, method herein has most outstanding feature: have good identical border for the edge in the edge of object in natural scene scene and depth map.Because human eye is very responsive for the object edge in image, in all disparity maps, the standard at the edge of object lacks the comfort level expressed and can improve stereopsis and view and admire.There is the effect of good suppression erroneous matching for repeating texture region simultaneously.According to the evaluation method of online definition, devise matlab comparison program and carried out the comparison of error rate, result is as follows:

The confidence spread carrying out Fast Convergent blocked a little is carried out in the time difference map obtained.

First, for belief propagation algorithm, under identical MRF parameter prerequisite, shown in the general type following formula of minimized energy function:

E(f)＝E _data(f)+E _smooth(f)＝Σ _p∈PD _p(f(p))+Σ _(p，q)∈MV _p，q(f(p)，f(q))

In above formula, data item E_data is exactly the otherness being characterized reference picture and target image by certain Matching power flow, and this value is less illustrates that corresponding pixel intensity is more close, and difference is less; Level and smooth item Esmooth is then the penalty value that spatial neighborhood pixels has same parallax, and the cause defining this is that pixel adjacent in image often has relevance, and the change of parallax is generally level and smooth.In the process that globalize solves, the structure of different energy functions is the principal element of the difference of the result causing global optimization.

Hypothesis space point has closely similar color characteristic on the width image of left and right two, data item Edata (f) be used for weighing image between the similarity degree of pixel, general type is defined as:

D _p(f _p)＝(I ₁(p)-I ₁₂(p+f _p)) ²

Wherein, p represent pixel (x, y), the parallax value that f (p) is pixel p; In the present invention, the I in formula represents the colour information primary color values of p point and match point (p+f (p)) respectively, and unconventional gray value.

In coupling, utilize half-tone information, make matching process very responsive to object structures, geometry and illumination.And when body surface lacks enough texture informations, colour information has more image detail information, the impact of the factor such as illumination, low texture can be reduced to a certain extent.Therefore build new energy function data item, comprise the color primaries value of the R G B of pixel.

(2) Section 2 of the energy equation set up in the research of classical energy function Method for minimization is level and smooth item, statement be penalty term in piece image between adjacent two pixels. general simple smoothness constraint is as follows:

$V(f_p,f_q)=\left\{\begin{array}{cc}0& f_p=f_q\\ d& f_p\&NotEqual;f_q\end{array}\right.$

In formula, d is fixing level and smooth punishment amount, when p and q is neighbor, thinks that parallax is level and smooth, increases punishment amount; When p and q is not neighbor, punishment amount is 0.

In the present invention, by initial matching above, having determined algorithm all needs to travel through all label space disparity range that also both pixel is possible, therefore the process that solves of label large portion node obtained by initial assignment.Because obtain block coupling point diagram be very limited in whole little baseline binocular vision, generally can not exceed 20% of whole view.Therefore, in the process of carrying out confidence spread, accurate match point is by very fast convergence.

Above-mentioned steps obtains the feature of disparity map, redesigns the segmentation item of the strong marginal information of object: E _segment(f).Segmentation item is in order to block the information transmission across object edge.In fact in a first step, the strong marginal information of pixel has been obtained.

Integral energy equation is as follows.

E (f)=E _data(f)+E _smooth(f)+E _segment(f) (formula 2-28)

(2) when enforcement confidence spread, the selection of the specific direction of propagation is carried out.

In the disparity map obtained for reference view at left view, the direction of propagation is always process from left to right and from the top down.In the time difference map obtained for reference view with right view, relay direction always process from the top down from right to left.This is conducive to the result obtaining convergence into excessively less iterations.

Finally, the method for the correction of error matching points possible in disparity map is carried out with the movable information between frame of pixels

By the scheme that the time domain continuity that the comparative approach between simple-frame improves disparity map is the consideration of value.Step is as follows:

The disparity map of present frame does difference with the disparity map of former frame, and mark difference is greater than the pixel of threshold value th

Vd_edge(x，y)。

The pixel of Vd_edge (x, y) position is done difference at the gray value of active view with the gray value of former frame, when the difference of gray scale is less than threshold value th_l, mark vd_error (x, y).

The average vd_avrage_p (x, y) of the spatial neighborhood of vd_error (x, y) is calculated in disparity map; The average vd_avrage_c (x, y) of the parallax of vd_error (x, y) neighborhood is asked in current disparity map.

The average of vd_avrage_c (x, y) and vd_avrage_p (x, y) is upgraded the parallax value of current frame.

The constructible multi-viewpoint three-dimensional system of application said method:

The multi-viewpoint three-dimensional of current commercialization is propagated and the form of display system many employings nine grids is connected by interfaces such as VGA, HDMI, DVI between signal handling equipment and display device.In the form of nine grids, a two field picture has nine subgraphs, and the most content of each image repeats, but all has fine distinction because of being different viewpoints.This makes the resolution of each image low, is 1/9 of whole frame.And carrying out the picture-in-picture of multiple views, and when the lamination of multi-layer image shows, needing to make similar viewpoint content respectively at nine pictures.Carry out the display of the picture-in-picture of two windows, need the image of making 18 views.This all will bring complexity to program making and distribution, increase cost.Meanwhile, carrying out the scaling of the second picture, when the typical apply such as displacement, needing to operate nine local window pictures simultaneously simultaneously.This needs the support of complicated hardware and software, is almost irrealizable.

The system adopting the present invention to form, can carry out real-time operation by the one-view image of two pictures, obtains depth map, carries out the generation of the multiple views based on DIBR in the display of rear end.When carrying out being similar to the application such as picture-in-picture commercial breaks, can in the content only increasing by the second picture.Also the second image content and depth map can be inserted again after depth map extracts link acquisition, the same effect can be played equally.Under such system configuration, stacked cascade and the combination of the system that realizes can be easy to, and carry out the application such as the second picture carries out moving, scaling.

Present invention also offers a kind of depth map real-time acquisition system of three-dimensional view, as shown in Figure 5, comprising:

First matching unit 100, for combining carry out initial matching by feature matching method with based on the method for segmentation, obtains the initial parallax figure of two views in left and right;

Second matching unit 200, for adopting the method based on parallax probability statistics to be optimized to initial parallax figure, realizes the Stereo matching of left and right view;

Wherein, based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described first matching unit 100, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

First view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

Search the histogrammic front M peak value in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

Relatively search the gray scale difference value of the peak value of acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point;

Described second matching unit 200 comprises further:

Through the condition preset, the first ground control point is determined to the point in initial parallax figure;

One statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

Carry out gradient smoothly to initial parallax figure, the information of recycling depth information and neighborhood image carries out interpolation.

In the depth map real-time acquisition system of described three-dimensional view, in described first matching unit, the sobel operator with neighbor smoothing effect is adopted to carry out feature point detection by feature matching method;

All first left and right view is divided into 16 × 16, the block of pixels of 32 × 32 or 64 × 64.

In the depth map real-time acquisition system of described three-dimensional view, in described second matching unit, the first ground control point is divided into reliable point and unreliable point; Wherein, the formation of described reliable point is divided into two classes: a class is the ground control point that the strong edge in view is formed, and two classes are ground control points that texture pixel is formed; The formation of described unreliable point is divided three classes: occluded pixels point, flat pixels point and the improper point of coupling.

In the depth map real-time acquisition system of described three-dimensional view, in described second matching unit, gradient is carried out to initial parallax figure and smoothly comprise: three-dimensional filtering.

In sum, the depth map real time acquiring method of three-dimensional view provided by the invention and system, propose the extracting method of the high density disparity map needed for a kind of real-time multiple views view generation process from three-dimensional left and right view.Carry out initial matching by adopting feature matching method and combining based on the method for segmentation, then adopt Statistics-Based Method to be optimized, realize the Stereo matching of left and right view.Matching algorithm in this paper has wider adaptability.And adopt the three-dimensional view of multiple different test feature and three-dimensional film fragment to test, generate disparity map more accurately, to repetition texture with the critical problem such as to block and achieve good result, have good marketing application prospect.

In method of the present invention, finally can simultaneously or the depth map based on left view reference map and right view as depth map during reference view.In the system configuration of rear end DIBR, be formed in the generation of the virtual view on the left of central viewpoint region all based on left view and left depth map.In the generation of the virtual view of rear side all based on right view and right depth map.The system of such DIBR can obtain more level and smooth and virtual view more accurately.

Should be understood that, for those of ordinary skills, can be improved according to the above description or convert, and all these improve and convert the protection range that all should belong to claims of the present invention.

Claims (8)

1. a depth map real time acquiring method for three-dimensional view, is characterized in that, comprise the following steps:

S1, combine carry out initial matching by feature matching method with based on the method for segmentation, obtain the initial parallax figure of two views in left and right;

S2, initial parallax figure adopted be optimized based on the method for parallax probability statistics, realize the Stereo matching of left and right view;

Wherein, based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described step S1, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

S11, first view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

S12, the histogrammic front M peak value searched in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

S13, compare the gray scale difference value of the peak value searching acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in S14, mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point;

Described step S2 comprises further:

S21, through the condition that presets, the first ground control point is determined to the point in initial parallax figure;

S22, a statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

S23, initial parallax figure to be carried out to gradient level and smooth, and the information of recycling depth information and neighborhood image carries out interpolation.

2. the depth map real time acquiring method of three-dimensional view according to claim 1, is characterized in that, wherein, adopts the sobel operator with neighbor smoothing effect to carry out feature point detection in described step S1 by feature matching method;

All first left and right view is divided into 16 × 16, the block of pixels of 32 × 32 or 64 × 64.

3. the depth map real time acquiring method of three-dimensional view according to claim 1, is characterized in that, in described S2, the first ground control point is divided into reliable point and unreliable point;

Wherein, the formation of described reliable point is divided into two classes: a class is the ground control point that the strong edge in view is formed, and two classes are ground control points that texture pixel is formed;

The formation of described unreliable point is divided three classes: occluded pixels point, flat pixels point and the improper point of coupling.

4. the depth map real time acquiring method of three-dimensional view according to claim 1, is characterized in that, carries out gradient and smoothly comprises: three-dimensional filtering in described S23 to initial parallax figure.

5. a depth map real-time acquisition system for three-dimensional view, is characterized in that, comprising:

First matching unit, for combining carry out initial matching by feature matching method with based on the method for segmentation, obtains the initial parallax figure of two views in left and right;

Second matching unit, for adopting the method based on parallax probability statistics to be optimized to initial parallax figure, realizes the Stereo matching of left and right view;

Wherein, based on the fritter in the method for segmentation, all first left and right view being divided into n × n in described first matching unit, split in fixing rectangular block; Adopt the repetition dividing method based on histogram multi thresholds, it comprises further:

First view is divided the window of n × n, each window carries out multi-threshold segmentation based on the histogram distribution of gray scale, setting up in histogrammic process, histogrammic each point and two consecutive points in left and right are carried out Gaussian smoothing, and the maximum number cutting block that setting allows is M, to reduce the complexity that hardware algorithm realizes;

Search the histogrammic front M peak value in block; According to the threshold value of the size setting effective peak of window, what be greater than the threshold value of effective peak just thinks effective peak;

Relatively search the gray scale difference value of the peak value of acquisition; Gray scale difference value is less than gray threshold and merges into one and have effective peak;

Valley in mark histogram between effective peak, respectively as threshold calculations variance, choose make inter-class variance maximum using valley as cut-point;

Described second matching unit comprises further:

Through the condition preset, the first ground control point is determined to the point in initial parallax figure;

One statistical information is set up to the first ground control point; Wherein, described statistical information comprises: the parallax value of pixel and the number of pixel;

Carry out gradient smoothly to initial parallax figure, the information of recycling depth information and neighborhood image carries out interpolation.

6. the depth map real-time acquisition system of three-dimensional view according to claim 5, is characterized in that, adopts the sobel operator with neighbor smoothing effect to carry out feature point detection in described first matching unit by feature matching method;

All first left and right view is divided into 16 × 16, the block of pixels of 32 × 32 or 64 × 64.

7. the depth map real-time acquisition system of three-dimensional view according to claim 5, is characterized in that, in described second matching unit, the first ground control point is divided into reliable point and unreliable point;

Wherein, the formation of described reliable point is divided into two classes: a class is the ground control point that the strong edge in view is formed, and two classes are ground control points that texture pixel is formed;

The formation of described unreliable point is divided three classes: occluded pixels point, flat pixels point and the improper point of coupling.

8. the depth map real-time acquisition system of three-dimensional view according to claim 5, is characterized in that, carries out gradient and smoothly comprises: three-dimensional filtering in described second matching unit to initial parallax figure.