CN104517317A - Three-dimensional reconstruction method of vehicle-borne infrared images - Google Patents

Abstract

The invention discloses a three-dimensional reconstruction technology of vehicle-borne infrared images. The technology mainly adopts an infrared three-dimensional reconstruction technology combining a panel parameter Markov random field PP-MRF model and a superpixel division technology. As an infrared image division technology, the superpixel division technology adopts Law's mask to extract superpixel features in a multiscale condition. A multi-condition training method is introduced to adjust internal parameters of the panel parameter Markov random field PP-MRF model. A trained model is used for estimating panel parameters of each panel of a test image, namely the relative position relation between the panel and a camera. The relative positions between a road and the sky are determined through a method of searching a horizontal line, and finally the three-dimensional reconstruction operation of the vehicle-borne infrared images is performed through deep and structure information. The technology can perform three-dimensional reconstruction on the vehicle-borne infrared images, and ensures that drivers can more visually understand unicolor infrared image and enhance the sensory ability to the environment.

Description

A kind of vehicle mounted infrared 3-dimensional reconstruction method

Technical field

The present invention relates to a kind of three-dimensional reconstruction of vehicle mounted infrared image.

Background technology

Along with universal at civil area of vehicle mounted infrared auxiliary drive system, the research work of vehicle mounted infrared aspect gradually pay close attention to by people.The estimation of Depth of infrared image has become study hotspot in recent years.The research of vehicle mounted infrared three-dimensional reconstruction, on the estimation of Depth of infrared image, a uncharted field as vehicle mounted infrared research is significant.

In recent years, the three-dimensional reconstruction work of binocular visible ray tends to ripe gradually, and wherein based on BeliefPropagation algorithm, carry out three-dimensional reconstruction, the application in current stereoscopic vision field widely.But the three-dimensional reconstruction job development of monocular image is more late comparatively speaking, wherein most is representational is shape-from-shading and shape-from-texture algorithm, but these algorithms superficial makings and color change not obvious time, effect is poor.The 3-dimensional reconstruction of the monocular based on PP-MRF model proposed by Ashutosh Saxena, Min Sun and AndrewY.Ng is recently compared other algorithms and is all significantly improved in accuracy, effect.

Infrared image, compared to visible ray, lacks abundant texture information and color, but which contain comparatively significantly edge feature comparatively speaking.Based on these features, the method for super-pixel segmentation is highly suitable for infrared image field, can obtain the good segmentation effect of infrared image.P.Felzenszwalb proposes the superpixel segmentation method based on graph theory, and Liu etc. propose the superpixel segmentation method based on entropy rate.

In the estimation of Depth of infrared image, infrared depth estimation algorithm based on KPCA and BP neural network and the infrared image depth estimation algorithm based on SVM, just carry out feature extraction between pixel and the pixel of surrounding certain limit, do not consider the constituent relation between actual panel, so correct three-dimensional reconstruction cannot be carried out to infrared image.

Summary of the invention

The object of this invention is to provide and a kind ofly under the condition that vehicle mounted infrared road conditions is changeable, can correctly carry out the method for the three-dimensional reconstruction of vehicle mounted infrared image.

In order to achieve the above object, technical scheme of the present invention there is provided a kind of vehicle mounted infrared 3-dimensional reconstruction method, it is characterized in that, comprises the following steps:

The first step, obtain several training images and every depth map corresponding to width training image;

Second step, super-pixel segmentation is carried out to training image, each super-pixel is an image block be made up of the neighbor with similar features, simultaneously, the panel parameter of the corresponding Markov random field model of each super-pixel is calculated, i.e. the panel parameter of PP-MRF model from the depth map of correspondence;

3rd step, use Louth mask carry out multiple dimensioned feature extraction to each super-pixel in the training image after segmentation, obtain the proper vector of each super-pixel;

4th step, build training set by the panel parameter obtained by second step of all proper vectors of being obtained by the 3rd step and correspondence;

5th step, build PP-MRF model after, utilize training set to adjust the parameter θ of PP-MRF model;

6th step, acquisition vehicle mounted infrared image;

7th step, super-pixel segmentation is carried out to vehicle mounted infrared image, the each super-pixel of PP-MRF model to vehicle mounted infrared image utilizing the 5th step to obtain is done panel parameter and is estimated and structure analysis, horizontal line is found on the basis of structure analysis, find out road and super-pixel corresponding to sky, determine the relative position of road plane and sky;

8th step, calculated the depth value of the pixel on each panel by panel parameter, then carry out 3D reconstruction in conjunction with the relative position of road plane and sky.

Preferably, the step that described training image or described vehicle mounted infrared image carry out super-pixel segmentation is comprised:

Regard each pixel in described training image or described vehicle mounted infrared image as a summit, the set on all summits is V, V is divided into a lot of zonules, calculate the outside difference between the internal diversity of each zonule and any two zonules, wherein, the internal diversity of each zonule is the maximum weights on the minimum spanning tree of this zonule, outside difference between two zonules is the minimum weights limit connecting these two zonules, if the outside difference between any two zonules is greater than the internal diversity of any one zonule in these two zonules, then these two zonules do not belong to same super-pixel, otherwise, these two zonules belong to same super-pixel.

Preferably, each super-pixel of PP-MRF model to vehicle mounted infrared image utilizing the 5th step to obtain does panel parameter estimation and structure analysis comprises the steps:

Described PP-MRF model is defined as follows shown in formula:

$P\left(\mathrm{\α}\right|X,v,y,R;\mathrm{\θ})=\frac{1}{Z}\underset{i}{\mathrm{\Π}}{f}_1\left({\mathrm{\α}}_i\right|X_i,v_i,R_i;\mathrm{\θ})\underset{i,j}{\mathrm{\Π}}{f}_2({\mathrm{\α}}_i,{\mathrm{\α}}_j|y_{\mathrm{ij}},R_i,R_j),$ In formula:

α _ibe the panel parameter of i-th super-pixel, in i-th super-pixel, contain S _iindividual pixel, the s in i-th super-pixel _ithe feature of individual pixel is then in i-th super-pixel, the feature of all pixels is vectorial $X_i=\{x_{i,s_i}:s_i=1,...,S_i\}$ Represent; $R_i=\{R_{i,s_i}:s_i=1,...,S_i\}$ For the set of the unit direction vector of each pixel on from camera photocentre to i-th super-pixel; Vector v _idescribe from local feature to describe the confidence level of panel parameter, wherein, Section 1 f ₁the local feature of () counter plate parameter alpha and pixel between relation carry out modeling, θ needs the parameter of adjusting, and its value is relevant to the line number at panel place, Section 2 f ₂there is closed curve boundary mainly between i-th super-pixel and a jth super-pixel in (), now needs to consider that the relation between counter plate carries out modeling, supposes pixel s _iand s _jrespectively from i-th super-pixel and a jth super-pixel, f ₂() is defined as: by selecting different function h () and the pixel between different super-pixel to { s _i, s _jcome to be described these important structural relations of adjacency, coplanarity and collinearity and to catch respectively, wherein:

Adjacency structure: selected pixels point s respectively on the border of the connection of i-th super-pixel and a jth super-pixel _iand s _j, then adjacency probability model is:

$h_{s_i,s_j}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_i,R_j)=\exp (-y_{\mathrm{ij}}|(R_{i,s_i}^T{\mathrm{\α}}_i-R_{j,s_J}^T{\mathrm{\α}}_j)\left|\hat{d}\right);$

Coplanarity: selected pixels pair in adjacent super-pixel with if really there is coplanar relation between adjacent super-pixel, panel parameter α so in theory _iwith α _jvalue equal, the relation function of coplanarity is shown below:

$h_{s_j^{\′\′}}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_{j,s_j^{\′\′}})=\exp (-y_{\mathrm{ij}}|R_{j,s_j^{\′\′}}^T{R}_{j,s_j^{\′\′}}^T{\mathrm{\α}}_j\left|{\hat{d}}_{s_j^{\′\′}}\right),$ If two super-pixel are coplanar, so exist condition under theoretical value be 1;

Collinearity: the funtcional relationship of two super-pixel collinearities is shown below:

$h_{s_j}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_{j,s_j})=\exp (-y_{\mathrm{ij}}|(R_{j,s_j}^T{\mathrm{\α}}_i-R_{j,s_J}^T{\mathrm{\α}}_j)\left|\hat{d}\right),$ If two panel conllinear, so theoretical value be 1, utilize the funtcional relationship of two super-pixel collinearities to find out all long straight lines existed between two super-pixel.

Preferably, horizontal line is found on the basis of structure analysis, find out road and super-pixel corresponding to sky, determine that the relative position of road plane and sky comprises the steps:

Item in the funtcional relationship of step 7.1, two super-pixel collinearities identify the collinearity structure results of all panels, thus obtain many long straight lines;

Step 7.2, calculate the probability of the long straight line of every bar:

On the long straight line l of the edge conllinear of two super-pixel, choose arbitrarily 2 pixel S _i, S _j, calculate connection 2 pixel S on the image plane _i, S _jthe slope k of straight line, then have the probability of long straight line l to be P (l), P (l)=ue ^{| k|}, in formula, u is scale-up factor, and its value is relevant to the position residing for long straight line l central point;

Step 7.3, the long straight line corresponding to maximum probability step 7.2 calculated are set as horizontal line, if having horizontal super-pixel panel to be sky up, otherwise are road.

Preferably, after super-pixel segmentation is carried out to described vehicle mounted infrared image, and before carrying out 3D reconstruction, also comprise the block of pixels little especially ignored after super-pixel segmentation, replace with its nearest neighbouring super pixels, comprise the following steps:

If the S that i-th super-pixel comprises _iindividual pixel, S _jbe less than the number of pixels that around i-th super-pixel, any one adjacent super-pixel comprises, then record this super-pixel, above-mentioned retrieval is carried out for each super-pixel, record the super-pixel meeting above-mentioned condition, and sort according to the number size of the pixel comprised, from that super-pixel including minimum pixel, the panel parameter of current super-pixel is replaced by the panel parameter including the super-pixel of maximum pixel be adjacent, the super-pixel number that the super-pixel number replaced is total after being no more than segmentation preset number percent.

The present invention is owing to taking above technical scheme, and it has the following advantages:

1, panel parameter Markov model just considers general picture structure relation when modeling, and there is not the priori hypothesis in some conditions, and this makes three-dimensional reconstruction algorithm have stronger robustness, is adapted to changeable road scene.

2, the process of road and sky is carried out owing to employing road and sky mask, remarkably productive at the three-dimensional reconstruction of road and sky, enhance the complementary relation between panel, be convenient to the 3-D effect better holding the overall situation on three-dimensional reconstruction.

3, owing to have ignored the impact of super-pixel little especially on overall three-dimensional reconstruction, the structure of small panel depends on larger panel, so three-dimensional structure has good structure connectivity and robustness after rebuilding.。

Accompanying drawing explanation

Fig. 1 is algorithm flow chart;

Fig. 2 is vehicle mounted infrared image;

Fig. 3 is the super-pixel segmentation result based on graph theory;

Fig. 4 is Louth mask and direction detector;

Fig. 5 a is the collinearity structure results of all panels;

Fig. 5 b is the searching of long straight line;

Fig. 6 is vehicle mounted infrared 3-dimensional reconstruction result figure.

Embodiment

For making the present invention become apparent, hereby with preferred embodiment, and accompanying drawing is coordinated to be described in detail below.

As shown in Figure 1, the present invention comprises: the depth map and the camera parameter that 1, obtain training image and correspondence; 2, super-pixel segmentation is carried out to training image; 3, from the depth map of correspondence, corresponding panel parameter is calculated; 4, super-pixel is carried out feature extraction, extract the proper vector of one 464 dimension, use characteristic of correspondence vector sum panel parameter to train as input parameter, the model parameter θ of the PP-MRF that adjusts; 5, for test pattern, carry out super-pixel segmentation equally, utilize PP-MRF model to do panel parameter to super-pixel and estimate and structure analysis.Horizontal line is found on the basis of structure analysis, finds out road and super-pixel corresponding to sky, determine the relative position of road plane and sky; 6, correction is made for the panel parameter of less super-pixel simultaneously.The last integrated structure information again of depth value being calculated the pixel on each panel by panel parameter carries out 3D reconstruction.

First super-pixel segmentation is carried out for image, so-called super-pixel, refer to the image block that the neighbor with features such as similar grain, color, brightness is formed.Infrared image, compared to visible images, lacks abundant texture information and color, but which contain comparatively significantly edge feature comparatively speaking.Based on these features, the method for super-pixel segmentation is highly suitable for infrared image, can be partitioned into a lot of little panel classes, and keep image boundary preferably, speed on infrared image, is applicable to very much the estimation of follow-up panel parameter.

The super-pixel that present invention employs based on graph theory is split, and adopt the thought of minimum spanning tree, object makes the element in the same area similar as much as possible, and the element of zones of different is dissimilar as much as possible.Each pixel in image is regarded as a summit, a limit e is there is between summit and summit, containing corresponding weight w (e) on every bar limit, the set on all summits is V, V is divided into a lot of zonules, like this using any pixel in the C of region as summit, each region can regard a tree structure as.

For subset internal diversity, be exactly this region minimum spanning tree MST (C, E) on maximum weights.Internal diversity computing formula is as shown in formula (1):

$\mathrm{int}\left(C\right)=\underset{e\∈\mathrm{MST}(C,E)}{\max }w\left(e\right)---\left(1\right)$

Two parts subset between difference for connect this two-part minimum weights limit.Outside difference is as shown in formula (2):

$\mathrm{dif}(C_1,C_2)=\underset{v_i\∈C_1,v_j\∈C_2,(v_i,v_j)\∈E}{\min }w\left((v_i,v_j)\right)---\left(2\right)$

If C ₁, C ₂outside difference between the region of two parts is greater than C ₁and C ₂the internal diversity of any one then two parts be different parts, otherwise just regard as same part, as shown in formula (3):

$D(C_1,C_2)=\left\{\begin{array}{cc}\mathrm{true}& \mathrm{if\; dif}(C_1,C_2)>\mathrm{MInt}(C_1,C_2)\\ \mathrm{false}& \mathrm{otherwise}\end{array}\right.---\left(3\right)$

As shown in Figure 2, super-pixel segmentation result as shown in Figure 3 for original image.

It is secondary carries out feature extraction, and use Louth mask, i.e. Law's mask, carries out multiple dimensioned feature extraction to image.The feature of Law's as shown in Figure 4.Wherein first 9 is the feature mask of Law's, and latter 6 is the edge detector of different directions, is separated by 30 degree between any two.In other respects, we also need shape and the position feature of adding up super-pixel, form the proper vector of whole super-pixel together.

In model is set up, in order to the local feature making model not only can comprise image, take into account the relations such as adjacent, coplanar, the conllinear between panel simultaneously.The definition of panel parameter Markov model is as shown in formula (4):

$P\left(\mathrm{\α}\right|X,v,y,R;\mathrm{\θ})=\frac{1}{Z}\underset{i}{\mathrm{\Π}}{f}_1\left({\mathrm{\α}}_i\right|X_i,v_i,R_i;\mathrm{\θ})\underset{i,j}{\mathrm{\Π}}{f}_2({\mathrm{\α}}_i,{\mathrm{\α}}_j|y_{\mathrm{ij}},R_i,R_j)---\left(4\right)$

Wherein α _ibe the panel parameter of super-pixel i, suppose to contain S in super-pixel i _iindividual pixel, represent the S in super-pixel i _ithe feature of individual pixel.A little feature vector table is not. for the set of the unit direction vector of each pixel on from camera photocentre to super-pixel i.Vector v _idescribe from local feature to describe the confidence level of panel parameter.Wherein Section 1 f ₁the local feature of (g) counter plate parameter alpha and pixel between relation carry out modeling, θ needs the parameter of adjusting, and its value is relevant to the line number at panel place.Section 2 f ₂g (), mainly for there is closed curve boundary between super-pixel i, j, now needs to consider that the relation between counter plate carries out modeling.Suppose pixel s _iand s _jrespectively from super-pixel i, j, f ₂() definition is as shown in formula (5):

$f_2(\·)={\mathrm{\Π}}_{\{s_i,s_j\}\∈N}{h}_{s_i,s_j}(\·)---\left(5\right)$

By selecting different function h () and the pixel between different super-pixel to { s _i, s _jcome to be described these important structural relations of adjacency, coplanarity and collinearity and to catch respectively.

1, adjacency structure: we choose s respectively on the border of the connection of super-pixel i, j _iand s _j, so h () such as formula (6) can well provide probability model to adjacency.

$h_{s_i,s_j}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_i,R_j)=\exp (-y_{\mathrm{ij}}|(R_{i,s_i}^T{\mathrm{\α}}_i-R_{j,s_J}^T{\mathrm{\α}}_j)\left|\hat{d}\right)---\left(6\right)$

2, coplanarity: selected pixels pair in adjacent super-pixel with if really there is coplanar relation between adjacent super-pixel, panel parameter α so in theory _iwith α _jthe value of parameter is equal.The relation function of coplanarity is as shown in formula (7):

$h_{s_j^{\′\′}}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_{j,s_j^{\′\′}})=\exp (-y_{\mathrm{ij}}|R_{j,s_j^{\′\′}}^T{R}_{j,s_j^{\′\′}}^T{\mathrm{\α}}_j\left|{\hat{d}}_{s_j^{\′\′}}\right)---\left(7\right)$

If two super-pixel are coplanar, so exist condition under theoretical value be 1.

3, collinearity: the collinearity of super-pixel is also the very important problem needing to consider.If 2 super-pixel conllinear on the image plane, so in the 3D model of reality, the probability of their conllinear is just very high.Funtcional relationship is as shown in Equation 8:

$h_{s_j}({\mathrm{\α}}_i,{\mathrm{\α}}_j,y_{\mathrm{ij}},R_{j,s_j})=\exp (-y_{\mathrm{ij}}|(R_{j,s_j}^T{\mathrm{\α}}_i-R_{j,s_J}^T{\mathrm{\α}}_j)\left|\hat{d}\right)---\left(8\right)$

If two panel conllinear, so theoretical value be 1.This can be utilized to find out all long straight lines existed between two super-pixel in the plane of delineation.

Be directed to the fundamental characteristics of vehicle mounted infrared image, following improvement has been carried out for PP-MRF model.

(1) for actual conditions vehicle mounted infrared image having road of great width and obvious sky, the mask devising sky and road identifies them, better to understand and to hold the 3D structure of overall vehicle mounted infrared image.Its step is as follows: first need to carry out horizontal search.Item in utilizing above-mentioned formula (8) described identify the collinearity structure results of all panels as shown in Figure 5 a.On the basis of above-mentioned panel conllinear, the long straight line possible to every bar is needed to judge.On the straight line l of the edge conllinear of super-pixel, choose arbitrarily 2 pixel S _i, S _j, calculate slope calculations on the image plane as shown in formula (9):

k＝(y _j-y _i)/(x _j-x _i) (9)

Horizontal probability is as shown in formula (10):

p(l)＝ue ^|k|(10)

Wherein k is slope, u for scale-up factor relevant to the position residing for long straight central point.According to priori, the probability that horizontal line appears at 1/3 ~ 1/2 place of the whole height of image is larger.If the total long straight line of N bar, being then defined as shown in formula (11) of terminal level line l.

p(l)＝max(p(l _i))i∈1，...N (11)

Namely that straight line corresponding to maximum probability is horizontal line.If have horizontal super-pixel panel be sky up otherwise be road.Again in conjunction with corresponding depth information when carrying out three-dimensional reconstruction, just first can determine the three-dimensional structure of road.Road is structurally similar to the expansion of level, and the degree of depth of sky is infinite distance.Horizontal searching as shown in Figure 5 b.

(2) ignore the block of pixels little especially after super-pixel segmentation, replace with its nearest neighbouring super pixels.Reduce the special role of some noise spots and increase the continuity of whole three-dimensional reconstruction effect.Be super-pixel and maximum four the adjacent super-pixel around it due to what select when feature extraction, the feature of less super-pixel is often difficult to extract.So when carrying out three-dimensional reconstruction, Panel Estimator being carried out to less super-pixel and has little significance.Substitute concrete implementation step as follows: if the number of pixels S that super-pixel i comprises after carrying out super-pixel segmentation to image _i, be less than the number of pixels that any one adjacent super-pixel comprises around and then record this pixel.I.e. S _i< S _{j ∈ δ (i)}, wherein δ (i) represents the neighborhood of super-pixel i.Above-mentioned retrieval is carried out for each super-pixel, records the super-pixel point meeting above-mentioned condition, and according to the pixel number size sequence comprised, to wherein from minimum super-pixel, the panel parameter α of less super-pixel _idetermined as shown in formula (12) by the panel parameter of adjacent maximum super-pixel.5% of the super-pixel number that the super-pixel number replaced is total after being no more than segmentation.

S _j＝m a x S _i∈δ(i)

α _i＝α _j(12)

Due to scene broad on vehicle mounted infrared image mainly outdoor road, by above-mentioned improvement, enhance the complementary relation between panel, be convenient to the 3-D effect better holding the overall situation on three-dimensional reconstruction, make three-dimensional reconstruction algorithm have stronger robustness, be adapted to changeable road scene.

Just can be adjusted by the training of panel parameter model the inner parameter of PP-MRF, due to PP-MRF model as shown in formula (5), parameter to be learned is wherein θ, owing to considering that the implication of θ on the different row of image can be different, such as when line number is lower, the possibility being road is larger, is that the likelihood ratio of sky at a distance and trees is comparatively large when line number is higher, so be divided into by θ 10 kinds of different situations to consider.θ _r∈ i ⁴⁶⁴(r=1,2...10) each parameter represents the parameter of 1/10 row of part corresponding in image.

Use many condition study in study part, the problem concerning study of whole complexity, split into a series of conditional probability problem, simplify the complicacy of study.Finally the estimation of parameter θ is transferred to the problem of linear minimization.The training image used and the official website of corresponding depth image from School of Computer Science of Cornell University.The resolution of the 400 width training images used is 2272*1704, and corresponding depth map is (55*305*4).Wherein the first dimension is the X-coordinate of image, and the second dimension is Y-coordinate, and third dimension is the degree of depth of perspective, and fourth dimension coordinate is real range coordinate.Coordinate unit all represents with rice.

Last for test pattern, utilize the panel parameter model trained to estimate the panel parameter in test pattern and the degree of depth, and then utilize road and sky information, the optimization of panel parameter carries out the three-dimensional reconstruction of vehicle mounted infrared image.Reconstructed results as shown in Figure 6.

Claims (5)

1. a vehicle mounted infrared 3-dimensional reconstruction method, is characterized in that, comprises the following steps:

The first step, obtain several training images and every depth map corresponding to width training image;

Second step, super-pixel segmentation is carried out to training image, each super-pixel is an image block be made up of the neighbor with similar features, simultaneously, the panel parameter of the corresponding Markov random field model of each super-pixel is calculated, i.e. the panel parameter of PP-MRF model from the depth map of correspondence;

3rd step, use Louth mask carry out multiple dimensioned feature extraction to each super-pixel in the training image after segmentation, obtain the proper vector of each super-pixel;

4th step, build training set by the panel parameter obtained by second step of all proper vectors of being obtained by the 3rd step and correspondence;

5th step, build PP-MRF model after, utilize training set to adjust the parameter θ of PP-MRF model;

6th step, acquisition vehicle mounted infrared image;

7th step, super-pixel segmentation is carried out to vehicle mounted infrared image, the each super-pixel of PP-MRF model to vehicle mounted infrared image utilizing the 5th step to obtain is done panel parameter and is estimated and structure analysis, horizontal line is found on the basis of structure analysis, find out road and super-pixel corresponding to sky, determine the relative position of road plane and sky;

8th step, calculated the depth value of the pixel on each panel by panel parameter, then carry out 3D reconstruction in conjunction with the relative position of road plane and sky.

2. a kind of vehicle mounted infrared 3-dimensional reconstruction method as claimed in claim 1, is characterized in that, comprises the step that described training image or described vehicle mounted infrared image carry out super-pixel segmentation:

Regard each pixel in described training image or described vehicle mounted infrared image as a summit, the set on all summits is V, V is divided into a lot of zonules, calculate the outside difference between the internal diversity of each zonule and any two zonules, wherein, the internal diversity of each zonule is the maximum weights on the minimum spanning tree of this zonule, outside difference between two zonules is the minimum weights limit connecting these two zonules, if the outside difference between any two zonules is greater than the internal diversity of any one zonule in these two zonules, then these two zonules do not belong to same super-pixel, otherwise, these two zonules belong to same super-pixel.

3. a kind of vehicle mounted infrared 3-dimensional reconstruction method as claimed in claim 1, it is characterized in that, in described 7th step, each super-pixel of PP-MRF model to vehicle mounted infrared image utilizing the 5th step to obtain does panel parameter estimation and structure analysis comprises the steps:

Described PP-MRF model is defined as follows shown in formula:

$P\left(\mathrm{\&alpha;}\right|X,v,y,R;\mathrm{\&theta;})=\frac{1}{Z}\underset{i}{\mathrm{\&Pi;}}{f}_1\left({\mathrm{\&alpha;}}_i\right|X_i,v_i,R_i;\mathrm{\&theta;})\underset{i,j}{\mathrm{\&Pi;}}{f}_2({\mathrm{\&alpha;}}_i,{\mathrm{\&alpha;}}_j|y_{\mathrm{ij}},R_i,R_j),$ In formula:

α _ibe the panel parameter of i-th super-pixel, in i-th super-pixel, contain S _iindividual pixel, the s in i-th super-pixel _ithe feature of individual pixel is then in i-th super-pixel, the feature of all pixels is vectorial $X_i=\{x_{i,s_i}:s_i=1,...,S_i\}$ Represent; $R_i=\{R_{i,s_i}:s_i=1,...,S_i\}$ For the set of the unit direction vector of each pixel on from camera photocentre to i-th super-pixel; Vector v _idescribe from local feature to describe the confidence level of panel parameter, wherein, Section 1 f ₁the local feature of () counter plate parameter alpha and pixel between relation carry out modeling, θ needs the parameter of adjusting, and its value is relevant to the line number at panel place, Section 2 f ₂there is closed curve boundary mainly between i-th super-pixel and a jth super-pixel in (), now needs to consider that the relation between counter plate carries out modeling, supposes pixel s _iand s _jrespectively from i-th super-pixel and a jth super-pixel, f ₂() is defined as: by selecting different function h () and the pixel between different super-pixel to { s _i, s _jcome to be described these important structural relations of adjacency, coplanarity and collinearity and to catch respectively, wherein:

Adjacency structure: selected pixels point s respectively on the border of the connection of i-th super-pixel and a jth super-pixel _iand s _j, then adjacency probability model is:

$h_{s_i,s_j}({\mathrm{\&alpha;}}_i,{\mathrm{\&alpha;}}_j,y_{\mathrm{ij}},R_i,R_j)=\exp (-y_{\mathrm{ij}}|(R_{i,s_i}^T{\mathrm{\&alpha;}}_i-R_{j,s_j}^T{\mathrm{\&alpha;}}_j)\left|\hat{d}\right);$

Coplanarity: selected pixels pair in adjacent super-pixel with if really there is coplanar relation between adjacent super-pixel, panel parameter α so in theory _iwith α _jvalue equal, the relation function of coplanarity is shown below:

$h_{s_j^{*}}({\mathrm{\&alpha;}}_i,{\mathrm{\&alpha;}}_j,y_{\mathrm{ij}},R_{j,s_j^{*}})=\exp (-y_{\mathrm{ij}}|R_{j,s_j^{*}}^T{\mathrm{\&alpha;}}_i-R_{j,s_j^{*}}^T{\mathrm{\&alpha;}}_j\left|{\hat{d}}_{s_j^{*}}\right),$ If two super-pixel are coplanar, so exist $h_{s_i^{*},s_j^{*}}(\&CenterDot;)=h_{s_i^{*}}(\&CenterDot;)h_{s_f^{*}}(\&CenterDot;)$ Condition under theoretical value be 1;

Collinearity: the funtcional relationship of two super-pixel collinearities is shown below:

$h_{s_j}({\mathrm{\&alpha;}}_i,{\mathrm{\&alpha;}}_j,y_{\mathrm{ij}},R_{j,s_j})=\exp (-y_{\mathrm{ij}}|(R_{i,s_j}^T{\mathrm{\&alpha;}}_i-R_{j,s_J}^T{\mathrm{\&alpha;}}_j)\left|\hat{d}\right),$ If two panel conllinear, so theoretical value be 1, utilize the funtcional relationship of two super-pixel collinearities to find out all long straight lines existed between two super-pixel.

4. a kind of vehicle mounted infrared 3-dimensional reconstruction method as claimed in claim 3, is characterized in that, horizontal line is found on the basis of structure analysis, finds out road and super-pixel corresponding to sky, determines that the relative position of road plane and sky comprises the steps:

Item in the funtcional relationship of step 7.1, two super-pixel collinearities identify the collinearity structure results of all panels, thus obtain many long straight lines;

Step 7.2, calculate the probability of the long straight line of every bar:

On the long straight line l of the edge conllinear of two super-pixel, choose arbitrarily 2 pixel S _i, S _j, calculate connection 2 pixel S on the image plane _i, S _jthe slope k of straight line, then have the probability of long straight line l to be P (l), P (l)=ue ^{| k|}, in formula, u is scale-up factor, and its value is relevant to the position residing for long straight line l central point;

Step 7.3, the long straight line corresponding to maximum probability step 7.2 calculated are set as horizontal line, if having horizontal super-pixel panel to be sky up, otherwise are road.

5. a kind of vehicle mounted infrared 3-dimensional reconstruction method as claimed in claim 1, it is characterized in that, after super-pixel segmentation is carried out to described vehicle mounted infrared image, and before carrying out 3D reconstruction, also comprise the block of pixels little especially ignored after super-pixel segmentation, replace with its nearest neighbouring super pixels, comprise the following steps:

If the S that i-th super-pixel comprises _iindividual pixel, S _ibe less than the number of pixels that around i-th super-pixel, any one adjacent super-pixel comprises, then record this super-pixel, above-mentioned retrieval is carried out for each super-pixel, record the super-pixel meeting above-mentioned condition, and sort according to the number size of the pixel comprised, from that super-pixel including minimum pixel, the panel parameter of current super-pixel is replaced by the panel parameter including the super-pixel of maximum pixel be adjacent, the super-pixel number that the super-pixel number replaced is total after being no more than segmentation preset number percent.