CN103824303A - Image perspective distortion adjusting method and device based on position and direction of photographed object - Google Patents

Abstract

The invention provides an image perspective distortion adjusting method and device based on a position and a direction of a photographed object. The method comprises the following steps: step A, namely photographing the photographed object by using an imaging module through focal length adjustment from the infinity point to the nearest point to capture a plurality of original images; step B, namely carrying out operation on imaging information of the original images to obtain depth information of the photographed object; and step C, namely carrying out perspective distortion processing on the first original image in the original images by the depth information. With the adoption of the method and the device, a plurality of pictures with different photographing angles in a same scene can be obtained by using the imaging module and imaging information is used for carrying out the operation on the original images; the scene depth is obtained by calculation; the distortion processing is carried out on the photographed images based on the obtained scene depth so as to overcome the perspective distortion.

Description

Position based on object, the method and apparatus that direction is adjusted image perspective distortion

Technical field

The present invention relates to image processing techniques, relate in particular to a kind of position based on object, direction and adjust the method and apparatus of image perspective distortion.

Background technology

Because camera site restriction, or composition needs, usually can run into the situation that cannot keep camera sensor parallel with subject.As shown in Figure 1, to take buildings as example, due to image aspects relation, whole building building all can not be photographed.For this reason, just need to adjust camera shooting angle.If but adjusted the shooting angle of camera in the mode shown in Fig. 2, because buildings bottom is compared with near and top is far away, the degree of depth of the relative camera lens of its each several part will change so.Correspondingly, the imaging size on sensor also can change, thereby produces perspective distortion.

Along with popularizing of smart mobile phone, the various application such as video calling and auto heterodyne are widely used.But existing technique for taking is not considered above-mentioned due to shooting distance, the image aspects problem causing, makes to take picture and physical presence certain error out, for example, the general face of taking out will seem loose than actual, has affected shooting effect and user and has experienced.

Summary of the invention

The problem that the embodiment of the present invention solves is how to solve the perspective distortion producing in image processing.

For addressing the above problem, the embodiment of the present invention provides a kind of position based on object, direction to adjust the method for image perspective distortion, comprise step as described below: steps A: by from infinite distance to nearest focus adjustment, by imaging modules, object is taken, captured several original images; Step B: by the image-forming information of described original image is carried out to computing, obtain the depth information of object; Step C: the first original image in described original image is carried out to perspective distortion processing by described depth information.

Optionally, in the time that focal plane and described object are not exclusively a plane, in described step B, also comprise: the coupling of the coupling in the coupling of the feature based on described image-forming information, the region based on described image-forming information or the phase place based on image-forming information, obtaining after corresponding intermediate parameters information, further calculating described depth information.

Optionally, the coupling of the described feature based on described image-forming information comprises: step B1: adopt horizontal dual camera, catch the original image obtaining and be respectively I _land I _r; Step B2: collection apparatus step: the detected value C that obtains single pixel in template: $C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I({\mathrm{<figure-callout\; id="0"\; label="X"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">X</figure-callout>}}_0,y_0)|\&le;\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">t</figure-callout>}\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Detect the each pixel in template is carried out, I (x ₀, y ₀) be template center's point gray-scale value, I (x, y) is other gray-scale values in template, t is the threshold value of determining similarity degree, x, and y is the coordinate in the coordinate system of initial point take the source images I lower left corner, detected value C to the point that belongs to template A sues for peace, and obtains the distance of swimming and the S of output:

the eigenwert R of the respective point (x0, y0) of source images I is: $R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<\mathrm{<figure-callout\; id="0"\; label="h"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">h</figure-callout>}\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.,$ Wherein h is geometry threshold value and h=3S _max/ 4, wherein S _maxthe maximal value that the distance of swimming and S can get, to described two width original image I _land I _rdeal with, obtain characteristic pattern and be respectively H _land H _r; Step B3: parallax matrix computations step: with H _lin point (x to be matched ₀, y ₀) centered by point create the rectangular window Q that a size is wide m, high n; At H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) adjacent onesize be another rectangular window Q ' of m × n; By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare; , H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window of middle correspondingly-sized horizontal offset dx is: ${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j)-H_L{[x}_0+i,{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j{\left)\right]}^2,$ I, j is the coordinate take rectangular window Q as coordinate system; How much threshold value k are set in preliminary election, if Γ _dx(x ₀, y ₀)≤k, is that the match is successful; Γ _dx(x ₀, y ₀) value of dx while obtaining minimum value, come the successfully off-set value dx of point of record matching, D (x by parallax matrix D here ₀, y ₀)=dx; At traversal characteristic pattern H _lafter, to parallax matrix D interpolation, to the unique point that the match is successful and the coordinate valuation of unsuccessful extract minutiae; The offset information that parallax matrix D is comprised is for compute depth; Step B4:I _lupper 1 P _lT(x ₁, y ₁, z ₁), by matching I _rupper 1 P _rT(x ₂, y ₂, z ₂), calculate spatial point P _w(x ₀, y ₀, z ₀) depth z ₀; For source images I _lon any point (x, y), in image-forming module optical axis center base length b and lens focus f situation, the degree of depth of its corresponding spatial point is:

d is the described parallax matrix that comprises offset information.

Optionally, when focal plane and described object are a plane, and when parallel with the camera lens of described image-forming module, pass through: formula: 1/L '-1/L=1/F ' calculates the depth information of described object, wherein: L is object distance, L ' is image distance, and F is lens focus, and described depth information is object distance.

Optionally, in described step C, for described P _lT=(x ₁, y ₁, z ₁), its volume coordinate is P in the coordinate system that is initial point through sensor _w=(x ₀, y ₀, z ₀), have: if rotation θ angle, subpoint P _w'=(x ₀', y ₀', _z0'), have: ${\mathrm{\&alpha;}=\arctan y\left(\frac{y_0}{z_0}\right);}_0\&prime;=\sqrt{{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}^2+{{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});$

x ₀'=x ₀; Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), have:

z ₁'=f; Calculate successively every bit, obtain image E; To image, E makes interpolation, cuts out, and obtains described the first original image figure G after treatment.

Optionally, described anglec of rotation θ obtains by following any one mode: acceleration transducer, user specify, image modeling.

The embodiment of the present invention also provides a kind of position based on object, direction to adjust the device of image perspective distortion, comprising: taking module, object is carried out to the shooting of multiple angles, and capture several original images; Depth calculation module, by the image-forming information of more described original image is carried out to computing, obtains the depth information of object; Image processing module, carries out perspective distortion processing by described depth information to the first original image in described original image.

Optionally, in the time that focal plane and described object are not exclusively a plane, described depth calculation module can be based on described image-forming information the coupling in coupling, region based on described image-forming information or the coupling of phase place based on image-forming information of feature obtaining after corresponding intermediate parameters information, further calculate described depth information.

Optionally, the coupling of the feature of described depth calculation module based on described image-forming information comprises: adopt horizontal dual camera, catch the original image obtaining and be respectively I _land I _r; Obtain the detected value C of single pixel in template: $C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I({\mathrm{<figure-callout\; id="0"\; label="X"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">X</figure-callout>}}_0,y_0)|\&le;\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">t</figure-callout>}\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Detect the each pixel in template is carried out, I( _x0, y ₀) be template center's point gray-scale value, I(x, y) be other gray-scale values in template, t is the threshold value of determining similarity degree, x, and y is the coordinate in the coordinate system of initial point take the source images I lower left corner, detected value C to the point that belongs to template A sues for peace, and obtains the distance of swimming and the S of output: respective point (the x of source images I ₀, y ₀) eigenwert R be: $R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<\mathrm{<figure-callout\; id="0"\; label="h"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">h</figure-callout>}\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.;$ Wherein h is geometry threshold value and h=3S _max/ 4, wherein S _maxit is the maximal value that the distance of swimming and S can get.To described two width original image I _land I _rdeal with, obtain characteristic pattern and be respectively H _land H _r; Parallax matrix computations step: H _lin point (x to be matched ₀, y ₀) centered by point create the rectangular window Q that a size is wide m, high n; H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) adjacent onesize be another rectangular window Q ' of m × n; By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare; , H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window of middle correspondingly-sized horizontal offset dx is: ${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j)-H_L{[x}_0+i,{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j{\left)\right]}^2,$ I, j is the coordinate take rectangular window as coordinate system; How much threshold value k are set in preliminary election, if Γ _dx(x ₀, y ₀)≤k, is that the match is successful.Come the successfully off-set value dx of point of record matching, D (x by parallax matrix D ₀, y ₀)=dx; At traversal characteristic pattern H _lafter, to parallax matrix D interpolation, to the unique point that the match is successful and the coordinate valuation of unsuccessful extract minutiae; The offset information that parallax matrix D is comprised is for compute depth.I _lupper 1 P _lT(x ₁, y ₁, z ₁), by matching I _rupper 1 P _rT(x ₂, y ₂, z ₂), calculate spatial point P _w(x ₀, y ₀, z ₀) depth z ₀; For source images I _lon any point (x, _y), in image-forming module optical axis center base length b and lens focus f situation, the degree of depth of its corresponding spatial point is:

d is the described parallax matrix that comprises offset information.

Optionally, when focal plane and described object are a plane, and when parallel with the camera lens of described image-forming module, pass through: formula: 1/L '-1/L=1/F ' calculates the depth information of described object; Wherein: L is object distance, L ' is image distance, and F is lens focus, and described depth information is object distance.

Optionally, in described step C, for described P _lT=(x ₁, y ₁, z ₁), its volume coordinate is P in the coordinate system that is initial point through sensor _w=(x ₀, y ₀, z ₀), have:

if rotation θ angle, subpoint P _w'=(x ₀', y ₀', z ₀'), have: $\mathrm{\&alpha;}=\arctan \left(\frac{y_0}{z_0}\right);{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\&prime;=\sqrt{{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}^2+{{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});$ x ₀'=x ₀; Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), have:

z ₁'=f; Calculate successively every bit, obtain image E; To image, E makes interpolation, cuts out, and obtains described the first original image figure G after treatment.

Optionally, described anglec of rotation θ obtains by following any one mode: acceleration transducer, user specify, image modeling.

Compared with prior art, the technical scheme of the embodiment of the present invention has the following advantages:

Use imaging modules to obtain the multiple pictures of the different camera angle of Same Scene, and by the image-forming information of original image is carried out to computing, calculate scene depth, and scene depth based on obtaining does distortion to photographic images and process, thereby overcome perspective distortion.

Accompanying drawing explanation

Fig. 1 is the shooting schematic diagram of taking buildings in prior art by imaging device;

Fig. 2 is the shooting schematic diagram changing in prior art after shooting angle;

Fig. 3 is the embodiment of the present invention a kind of position based on object, the schematic flow sheet that direction is adjusted the method for image perspective distortion;

Fig. 4 uses SUSAN to detect the schematic diagram of template in the embodiment of the present invention;

Fig. 5 is the schematic diagram that carries out characteristic matching of the embodiment of the present invention;

Fig. 6 be the embodiment of the present invention take horizontal dual camera scheme as example, the schematic diagram that depth information is calculated;

Fig. 7 is the surface level perspective view of the horizontal dual camera imaging of the embodiment of the present invention;

Fig. 8 is the schematic diagram of the surface level projection of longitudinal dual camera imaging of the embodiment of the present invention;

Fig. 9 is the schematic diagram of the calculating focusing mapping of the embodiment of the present invention;

Figure 10 is the embodiment of the present invention a kind of position based on object, the structural representation that direction is adjusted the device of image perspective distortion;

Figure 11 represents a kind of application form of the present invention in camera installation, is wherein used for range finding at sensor of the each placement in the left and right of camera installation.

Figure 12 represents the another kind of application form of the present invention in camera installation, and wherein a side of camera installation is periscopic master reference, and opposite side is that microsensor is for range finding;

Figure 13 represents the another kind of application form of the present invention in camera installation, and wherein camera installation comprises symmetrical periscopic sensor;

Figure 14 represents a kind of application form of the present invention in picture pick-up device, is wherein master reference in a side of picture pick-up device, and opposite side is that microsensor is for range finding.

Embodiment

In existing technical scheme, due in the process of shooting, need to adjust the shooting angle of imaging device, cause the part of subject compared with near and partly far away, thereby the degree of depth of the relative camera lens of its each several part is changed, cause perspective distortion.

The embodiment of the present invention, is used imaging modules to obtain the multiple pictures of the different camera angle of Same Scene, and by the image-forming information of original image is carried out to computing, calculates scene depth, thereby the transparent effect of image is changed, and prevents perspective distortion.

For above-mentioned purpose of the present invention, feature and advantage can more be become apparent, below in conjunction with accompanying drawing, specific embodiments of the invention are described in detail.

The embodiment of the present invention provides a kind of position based on object, direction to adjust the method for image perspective distortion.With reference to Fig. 3, below be elaborated by concrete steps.

Steps A, by from infinite distance to nearest focus adjustment, takes object by imaging modules, captures several original images.

In concrete enforcement, above-mentioned imaging modules can be single camera, can be also multiple cameras.Based on the repeatedly imaging of single camera, level or the vertical displacement of photographing device position can arrange repeatedly imaging with operator time, also can arrange repeatedly imaging with operator time, photographing device position moves forward and backward distance.Based on dual camera, can predetermined level or vertical displacement be set by particular device, apply horizontal dual camera scheme and calculate parallax matrix, or by particular device, the predetermined distance that moves forward and backward is set, apply longitudinal dual camera and calculate parallax matrix.Based on the more repeatedly imaging of multi-cam, similar with dual camera, can be that the multiple cameras of application calculate parallax matrix.

Step B, by the image-forming information of more described original image is carried out to computing, obtains the depth information of object.

In a kind of specific embodiment, when described object is a plane, when described object is a plane, and when parallel with the camera lens of described image-forming module, pass through formula: 1/L '-1/L=1/F ', calculate the depth information of described object, described depth information is focal length.Wherein: L is object distance, L ' is image distance, and F is lens focus.

In another kind of specific embodiment, in the time that described object is not exclusively a plane, when not parallel between camera lens and sensor and object, can pass through Region Matching Algorithm, obtaining after corresponding intermediate parameters information, further calculating described depth information.The method of the Region Matching that specifically, range finding is used can comprise three classes: the coupling of the feature based on described image-forming information, the coupling of the coupling in the region based on described image-forming information and the phase place based on described image-forming information.

Wherein, the coupling primitive that the coupling based on feature is used has comprised the dirigibility on abundant statistical property and arithmetic programming, is easy to realize by hardware.Coupling based on region is comparatively applicable to indoor grade and has the environment of notable feature, has larger limitation, need to have other artificial intelligence approaches to assist.Based on the coupling of phase place, because can causing disparity map, the reason such as existence and occlusion effect of periodicity pattern, smooth domain produces error, and also need to have other method to carry out error-detecting and correction, comparatively complicated.

In above-mentioned specific embodiment, set forth a kind of concrete methods of realizing of the present invention with the matching process based on feature.Wherein disparity computation comprises feature extraction and characteristic matching.But it should be understood that the present invention is not limited to the coupling based on feature.

Step B1, first needs to obtain source images I.Take horizontal dual camera as example, the source images obtaining by two of left and right sensor is respectively I _land I _r.To pre-service such as source images process figure image intensifying, filtering, convergent-divergents, enter feature extraction.Feature extraction can realize by following steps:

Step B2, obtains the detected value C of single pixel in template by feature point detection.Described unique point is the extreme point of image, substantially possesses translation, rotation, convergent-divergent, affine unchangeability, such as grey scale pixel value, angle point, edge, flex point etc.Conventional feature point detecting method comprises: SUSAN(Smallest Univalue Segment Assimilating Nucleus, small nut Tong Zhi district) angle point extracts, Harris angle point extracts, SIFT(Scale-invariant feature transform, yardstick invariant features) extract etc.As a kind of example, be extracted as example with SUSAN angle point here and describe:

In SUSAN angle point extracts, He Tongzhi district is the core with respect to template, has certain region and it to have identical gray scale in template.As shown in Figure 4, use 37 pixels in described original graph as detecting template, pass through formula $C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I({\mathrm{<figure-callout\; id="0"\; label="X"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">X</figure-callout>}}_0,y_0)|\&le;\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">t</figure-callout>}\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Can obtain detecting the detected value C of each single pixel in template.Wherein, I (x ₀, y ₀) be template center's point gray-scale value, I (x, y) detects other gray-scale values in template, and t is the threshold value of determining similarity degree, x, y is the coordinate in the coordinate system of initial point take the source images I lower left corner.

Calculate after the detected value that detects each point in template, to detected value C summation, the distance of swimming and the S that obtain output are: further, according to formula: $R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<\mathrm{<figure-callout\; id="0"\; label="h"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">h</figure-callout>}\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.,$ Calculate the respective point (x of original image I ₀, y ₀) eigenwert R (x ₀, y ₀).Wherein, h is geometry threshold value and h=3S _max/ 4, S _maxit is the maximal value that the distance of swimming and S can get.

By above-mentioned steps, respectively to two secondary original image I _land I _rcarry out above-mentioned computing, obtain characteristic of correspondence figure H _land H _r.

Step B3, on this basis, need to do further characteristic matching to the characteristic pattern obtaining, to obtain parallax matrix.Characteristic matching can realize as follows:

As shown in Figure 5, with First Characteristic figure H _lin point (x to be matched ₀, y ₀) centered by point, create one wide be m, the high rectangular window Q for n.At Second Characteristic figure H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) (reference point) adjacent, onesize is another rectangular window Q ' of m × n.By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare.If can match the corresponding point with maximum comparability in two rectangular windows, just can be judged to be optimum matching.

Can mate two rectangular windows corresponding to characteristic pattern by several different methods.To adopt the quadratic sum algorithm of gray scale difference as example, H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window Q ' of middle correspondingly-sized horizontal offset dx is: ${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j)-H_L{[x}_0+i,{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j{\left)\right]}^2,$ Wherein, i, j is any the coordinate take rectangular window as coordinate system.

The initial point of described coordinate system is the corresponding position of rectangular window Q and rectangular window Q '.For example, can be the coordinate in the coordinate system of initial point take rectangular window Q and the rectangular window Q ' lower left corner, can be also coordinate in the coordinate system of initial point etc. take rectangular window Q and the rectangular window Q ' lower right corner.

By Γ _dx(x ₀, y ₀) and preset how much threshold value k comparison.Work as Γ _dx(x ₀, y ₀when)≤k, can be judged to be that the match is successful.If Γ _dx(x ₀, y ₀) can obtain minimum value, be template and mate completely.Come the successfully off-set value dx of point of record matching, D (x by parallax matrix D ₀, y ₀)=dx.

Repeat above-mentioned method, travel through whole characteristic pattern H _lafterwards with characteristic pattern H _rcarry out matching ratio, and by parallax matrix D interpolation, to the unique point that the match is successful and the coordinate valuation of unsuccessful extract minutiae, to form complete parallax matrix D.

Be understandable that, the above-mentioned method of obtaining parallax matrix can be also at H _rin choose rectangle frame, with H _lthe rectangle frame of middle correspondence is done to mate, and by further traveling through H _r, calculate complete parallax matrix D.For convenience's sake, below based in this step to characteristic pattern H _ltraversal describe.

Step B4, based on the offset information comprising in parallax matrix D, compute depth further, the i.e. depth of field.In the optional specific embodiment of one, to adopt horizontal dual camera scheme as example, can pass through triangle formula compute depth.

Shown in Fig. 6-8, for the sensor of two parallel placements, the source images I that left side sensor obtains _lupper 1 P _lT(x ₁, _y1, z ₁), by matching spatial point point P _w(x ₀, _y0, z ₀) 1 P in right sensor _rT(x ₂, y ₂, z ₂), can calculate spatial point point P _w(x ₀, y ₀, z ₀) depth z ₀.For source images I _lon any point (x, _y), the in the situation that of known two sensors optical axis center base length b and lens focus f, the degree of depth of its corresponding spatial point is:

$Z(x,y)=f\&times;\frac{b}{D(x,y)},$

Wherein, D is the parallax matrix that comprises offset information calculating in characteristic matching.Through ergodic source image I _l, get final product to obtain the matrix of depths Z of the depth of field.

Step C, carries out perspective distortion processing by described depth information to the first original image in more described original image, improves the effect of the first original image of processing.

Be arranged on imaging device left side as example, for I take distance measuring sensor _lupper 1 P _lT=(x ₁, y ₁, z ₁), its volume coordinate, through sensor, is P in the coordinate system of initial point _w=(x ₀, y ₀, z ₀).Wherein,

${\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=\frac{y_1\&times;z_0}{f};$

, as axle center three dimensions every bit is remapped take a straight line in three dimensions, thereby form final target image.Can be based on the imaging device left side horizontal axis of sensor or the rotation of vertical axis.The embodiment of the present invention, using the rotation based on horizontal axis as example, supposes that the anglec of rotation is θ, and subpoint is P so _w'=(x ₀', y ₀', z ₀').As seen from Figure 9, $\mathrm{\&alpha;}=\arctan {\left(\frac{y_0}{z_0}\right);\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\&prime;=\sqrt{{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}^2+{{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});$

x ₀'=x ₀；。

Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), obtain, $x_1\&prime;=\frac{x_0\&prime;\&times;\mathrm{<figure-callout\; id="0"\; label="f\; z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">f</figure-callout>}}{z_{}},y_1\&prime;=\frac{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\&prime;\&times;\mathrm{<figure-callout\; id="0"\; label="f\; z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">f</figure-callout>}}{z_{}},z_1\&prime;=f.$

Successively every bit in the image after taking pictures is calculated to new images E, and new images E is made to interpolation, cut out, thereby obtain final target figure G.

Above-mentioned anglec of rotation θ is as described in the background art, and user is in order to photograph object completely, and the shooting angle of adjustment imaging device.As can be seen here, although adjusted shooting angle, by as the space relocation process of the embodiment of the present invention, can overcome because shooting angle changes the perspective distortion problem causing.

In the above-described embodiments, can obtain anglec of rotation θ by several different methods.For example, can utilize acceleration transducer (G-Sensor) to obtain.Acceleration transducer is a kind of electronic equipment that can measure accelerating force, comprises, if the gyroscope of detection angular velocity varies etc.By described acceleration transducer, rotational angle that can automatic acquisition imaging device, and then according to embodiments of the invention, calculate corresponding theoretical image point position.

The above-mentioned embodiment that obtains the anglec of rotation by acceleration transducer is only a kind of example.Other methods that can obtain the anglec of rotation all belong to technological thought of the present invention.For example, specify anglec of rotation size by user, or can calculate by image modeling.

In the above embodiment of the present invention, sensor setting is only a kind of example in imaging device left side, and the setting position of sensor also can be arranged on right side or other positions of imaging device, or can be provided with multiple sensors on imaging device.Be understandable that, other embodiment all can be according to the technological thought of the embodiment of the present invention, the i.e. depth parameter based on obtaining in range finding step, by remapping of space, on corresponding sensor, coupling obtains theoretical picture point, thereby complete the adjustment to image perspective distortion, therefore repeat no longer one by one herein.

To sum up, the embodiment of the present invention is found range by the depth of field, and depth of field parameter based on obtaining completes image correction, can compensate because of the not parallel perspective distortion causing between camera lens and sensor and object.Wherein, depth of field range finding can be the depth data (being the depth of field) that obtains each unique point in scene according to several steps such as shooting, coupling, depth calculation.

The embodiment of the present invention also provides corresponding a kind of position based on object, the device that direction is adjusted image perspective distortion, as shown in figure 10, comprising: taking module 101, object is carried out to the shooting of multiple angles, and capture several original images.Depth calculation module 102, by the image-forming information of more described original image is carried out to computing, obtains the depth information of object.Image processing module 103, carries out perspective distortion processing by described depth information to the first original image in described original image, causes the first original image of processing consistent with described object shape, improves the effect of the first original image.

In instantiation, in the time that focal plane and described object are not exclusively a plane, described depth calculation module 102 can be based on described image-forming information the coupling, the coupling of phase place based on image-forming information in coupling, region based on described image-forming information of feature obtaining after corresponding intermediate parameters information, further calculate described depth information.

In instantiation, the coupling of the feature of described depth calculation module based on described image-forming information comprises: adopt horizontal dual camera, catch the original image obtaining and be respectively I _land I _r; Obtain the detected value C of single pixel in template: $C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I({\mathrm{<figure-callout\; id="0"\; label="X"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">X</figure-callout>}}_0,y_0)|\&le;\mathrm{<figure-callout\; id="0"\; label="t"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">t</figure-callout>}\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Detect the each pixel in template is carried out, I(x ₀, y ₀) be template center's point gray-scale value, I(x, y) be other gray-scale values in template, t is the threshold value of determining similarity degree, x, y is the coordinate in the coordinate system of initial point take the source images I lower left corner.

Detected value C to the point that belongs to template A sues for peace, and obtains the distance of swimming and the S of output: $S(x_0,y_0)=\underset{(x,y)\&Element;A}{\mathrm{\&Sigma;}}C(x,y).$

Respective point (the x of source images I ₀, y ₀) eigenwert R be: $R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<\mathrm{<figure-callout\; id="0"\; label="h"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">h</figure-callout>}\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.,$ Wherein h is geometry threshold value and h=3S _max/ 4, wherein S _maxthe maximal value that the distance of swimming and S can get, to described two width original image I _land I _rdeal with, obtain characteristic pattern and be respectively H _land H _r.

Parallax matrix computations step: H _lin point (x to be matched ₀, y ₀) centered by point create the rectangular window Q that a size is wide m, high n; H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) adjacent onesize be another rectangular window Q ' of m × n; By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare.

H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window of middle correspondingly-sized horizontal offset dx is: ${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j)-H_L{[x}_0+i,{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j{\left)\right]}^2,$ I, j is the coordinate take rectangular window as coordinate system.How much threshold value k are set in preliminary election, if Γ _dx(x ₀, y ₀)≤k, is that the match is successful.Come the successfully off-set value dx of point of record matching, D (x by parallax matrix D ₀, y ₀)=dx.

At traversal characteristic pattern H _lafter, to parallax matrix D interpolation, to the unique point that the match is successful and the coordinate valuation of unsuccessful extract minutiae; The offset information that parallax matrix D is comprised is for compute depth.

I _lupper 1 P _lT(x ₁, y ₁, z ₁), by matching I _rupper 1 P _rT(x ₂, y ₂, z ₂), calculate spatial point P _w(x ₀, y ₀, z ₀) depth z ₀; For source images I _lon any point (x, y), in image-forming module optical axis center base length b and lens focus f situation, the degree of depth of its corresponding spatial point is:

wherein D is the described parallax matrix that comprises offset information.

In instantiation, when focal plane and described object are a plane, and when parallel with the camera lens of described image-forming module, pass through: formula: 1/L '-1/L=1/F ' calculates the depth information of described object, described depth information is object distance.Wherein: L is object distance, L ' is image distance, and F is lens focus.

In instantiation, in described step C, for I _lupper 1 P _lT=(x ₁, y ₁, z ₁),, its volume coordinate is P in the coordinate system that is initial point through left sensor _w=(x ₀, y ₀, z ₀), have: $x_0=\frac{x_1\&times;z_0}{f},{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=\frac{y_1\&times;z_0}{f}.$

If rotation θ angle, subpoint P _w'=(x ₀', y ₀', z ₀'), have,

${\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\&prime;=\sqrt{{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}^2+{{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0\&prime;=\sqrt{{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}^2+{{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">z</figure-callout>}}_0}^2}\&times;\cos (\mathrm{\&theta;}+\mathrm{\&alpha;});$

Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), have, $x_1\&prime;=\frac{x_0\&prime;\&times;\mathrm{<figure-callout\; id="0"\; label="f\; z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">f</figure-callout>}}{z_{}},y_1\&prime;=\frac{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\&prime;\&times;\mathrm{<figure-callout\; id="0"\; label="f\; z"\; filenames="HDA0000477490470000012.png,HDA0000477490470000032.png"\; state="\{\{state\}\}">f</figure-callout>}}{z_{}},z_1\&prime;=f.$

According to above-mentioned method, successively every bit on the image after taking pictures is calculated, obtain image E; To image, E makes interpolation, cuts out, and obtains described the first original image figure G after treatment.

In instantiation, described anglec of rotation θ passes through: acceleration transducer, user specify, image modeling calculates and obtains.

Be understandable that, the above embodiment of the present invention all can apply to various imaging devices, for example smart mobile phone, camera or video camera etc.Because the embodiment of the present invention is without using large scale sensor, long-focus large aperture camera lens or moving the devices such as lens shaft, therefore can in miniaturization imaging device, uses, thereby significantly reduce hardware cost and the space requirement of imaging device.

Specifically, concrete application of the present invention can include but not limited to following form:

1, camera installation (for example: card type camera, mobile phone etc.)

Figure 11 represents a kind of application form of the present invention in camera installation, is wherein used for range finding at sensor of the each placement in the left and right of camera installation.

Figure 12 represents the another kind of application form of the present invention in camera installation, and wherein a side of camera installation is periscopic master reference, and opposite side is that microsensor is for range finding.

Figure 13 represents the another kind of application form of the present invention in camera installation, and wherein camera installation comprises symmetrical periscopic sensor.

2, picture pick-up device (for example: video camera etc.)

Figure 14 represents a kind of application form of the present invention in picture pick-up device, is wherein master reference in a side of picture pick-up device, and opposite side is that microsensor is for range finding.

One of ordinary skill in the art will appreciate that all or part of step in the whole bag of tricks of above-described embodiment is can carry out the hardware that instruction is relevant by program to complete, this program can be stored in a computer-readable recording medium, and storage medium can comprise: ROM, RAM, disk or CD etc.

Although the present invention discloses as above, the present invention is not defined in this.Any those skilled in the art, without departing from the spirit and scope of the present invention, all can make various changes or modifications, and therefore protection scope of the present invention should be as the criterion with claim limited range.

Claims (12)

1. the position based on object, direction are adjusted a method for image perspective distortion, it is characterized in that, comprise step as described below:

Steps A: by from infinite distance to nearest focus adjustment, by imaging modules, object is taken, captured several original images;

Step B: by the image-forming information of described original image is carried out to computing, obtain the depth information of object;

Step C: the first original image in described original image is carried out to perspective distortion processing by described depth information.

2. position based on object according to claim 1, direction are adjusted the method for image perspective distortion, it is characterized in that, in the time that focal plane and described object are not exclusively a plane, in described step B, also comprise: the coupling of the coupling in the coupling of the feature based on described image-forming information, the region based on described image-forming information or the phase place based on image-forming information, obtaining after corresponding intermediate parameters information, further calculating described depth information.

3. position based on object according to claim 2, direction are adjusted the method for image perspective distortion, it is characterized in that, the coupling of the described feature based on described image-forming information comprises:

Step B1: adopt horizontal dual camera, catch the original image obtaining and be respectively I _land I _r;

Step B2: collection apparatus step: the detected value C that obtains single pixel in template:

$C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I(X_0,y_0)|\&le;t\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Detect the each pixel in template is carried out, I (x ₀, y ₀) be template center's point gray-scale value, I (x, y) is other gray-scale values in template, t is the threshold value of determining similarity degree, x, and y is the coordinate in the coordinate system of initial point take the source images I lower left corner, detected value C to the point that belongs to template A sues for peace, and obtains the distance of swimming and the S of output:

$S(x_0,y_0)=\underset{(x,y)\&Element;A}{\mathrm{\&Sigma;}}C(x,y);$

Respective point (the x of source images I ₀, y ₀) eigenwert R be:

$R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<h\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.,$

Wherein h is geometry threshold value and h=3S _max/ 4, wherein S _maxthe maximal value that the distance of swimming and S can get, to described two width original image I _land I _rdeal with, obtain characteristic pattern and be respectively H _land H _r;

Step B3: parallax matrix computations step: with H _lin point (x to be matched ₀, y ₀) centered by point create the rectangular window Q that a size is wide m, high n; At H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) adjacent onesize be another rectangular window Q ' of m × n; By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare;

, H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window of middle correspondingly-sized horizontal offset dx is:

${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},y_0+j)-H_L{[x}_0+i,y_0+j{\left)\right]}^2,$

I, j is the coordinate take rectangular window as coordinate system;

How much threshold value k are set in preliminary election, if Γ _dx(x ₀, y ₀)≤k, is that the match is successful;

Γ _dx(x ₀, y ₀) value of dx while obtaining minimum value, carry out the successful point of record matching by parallax matrix D here

Off-set value dx,

D(x ₀,y ₀)=dx；

At traversal characteristic pattern H _lafter, to parallax matrix D interpolation, to the unique point that the match is successful and do not become

The coordinate valuation of merit extract minutiae;

The offset information that parallax matrix D is comprised is for compute depth;

Step B4:I _lupper 1 P _lT(x ₁, y ₁, z ₁), by matching I _rupper 1 P _rT(x ₂, y ₂, z ₂), calculate spatial point P _w(x ₀, y ₀, z ₀) depth z ₀; For source images I _lon any point (x, _y), in image-forming module optical axis center base length b and lens focus f situation, the degree of depth of its corresponding spatial point is:

$Z(x,y)=f\&times;\frac{b}{D(x,y)},$

D is the described parallax matrix that comprises offset information.

4. position based on object according to claim 1, direction are adjusted the method for image perspective distortion, it is characterized in that, when focal plane and described object are a plane, and when parallel with the camera lens of described image-forming module, pass through: formula: 1/L '-1/L=1/F ' calculates the depth information of described object, wherein: L is object distance, L ' is image distance, F is lens focus, and described depth information is object distance.

5. position based on object according to claim 3, direction are adjusted the method for image perspective distortion, it is characterized in that, in described step C, for described P _lT=(x ₁, y ₁, z ₁), its volume coordinate is P in the coordinate system that is initial point through sensor _w=(x ₀, y ₀, z ₀), have:

$x_0=\frac{x_1\&times;z_0}{f};$

$y_0=\frac{y_1\&times;z_0}{f};$

If rotation θ angle, subpoint P _w'=(x ₀', y ₀', z ₀'), have:

$\mathrm{\&alpha;}=\arctan \left(\frac{y_0}{z_0}\right);$

$y_0\&prime;=\sqrt{{y_0}^2+{z_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});$

$z_0\&prime;=\sqrt{{y_0}^2+{z_0}^2}\&times;\cos (\mathrm{\&theta;}+\mathrm{\&alpha;});$ x ₀′=x ₀；

Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), have:

$x_1\&prime;=\frac{x_0\&prime;\&times;f}{z_0\&prime;};$

$y_1\&prime;=\frac{y_0\&prime;\&times;f}{z_0\&prime;};$

z ₁'=f；

Calculate successively every bit, obtain image E;

To image, E makes interpolation, cuts out, and obtains described the first original image figure G after treatment.

6. position based on object according to claim 5, direction are adjusted the method for image perspective distortion, it is characterized in that, described anglec of rotation θ obtains by following any one mode: acceleration transducer, user specify, image modeling.

7. the position based on object, direction are adjusted a device for image perspective distortion, it is characterized in that, comprising:

Taking module, carries out the shooting of multiple angles to object, capture several original images;

Depth calculation module, by the image-forming information of more described original image is carried out to computing, obtains the depth information of object; Image processing module, carries out perspective distortion processing by described depth information to the first original image in described original image.

8. position based on object according to claim 7, direction are adjusted the device of image perspective distortion, it is characterized in that, in the time that focal plane and described object are not exclusively a plane, described depth calculation module can be based on described image-forming information the coupling in coupling, region based on described image-forming information or the coupling of phase place based on image-forming information of feature obtaining after corresponding intermediate parameters information, further calculate described depth information.

9. position based on object according to claim 8, direction are adjusted the device of image perspective distortion, it is characterized in that, the coupling of the feature of described depth calculation module based on described image-forming information comprises:

Adopt horizontal dual camera, catch the original image obtaining and be respectively I _land I _r;

Obtain the detected value C of single pixel in template:

$C(x,y)=\left\{\begin{array}{c}1,|I(x,y)-I(X_0,y_0)|\&le;t\\ 0,|I(x,y)-I(x_0,y_0)|>t\end{array}\right.,$ Detect the each pixel in template is carried out, I(x ₀, y ₀) be template center's point gray-scale value, I(x, y) be other gray-scale values in template, t is the threshold value of determining similarity degree, x, and y is the coordinate in the coordinate system of initial point take the source images I lower left corner, detected value C to the point that belongs to template A sues for peace, and obtains the distance of swimming and the S of output:

$S(x_0,y_0)=\underset{(x,y)\&Element;A}{\mathrm{\&Sigma;}}C(x,y);$

Respective point (the x of source images I ₀, y ₀) eigenwert R be:

$R(x_0,y_0)=\left\{\begin{array}{cc}h-S(x_0,y_0),& S(x_0,y_0)<h\\ 0,& S(x_0,y_0)\&GreaterEqual;h\end{array}\right.$

Wherein h is geometry threshold value and h=3S _max/ 4, wherein S _maxthe maximal value that the distance of swimming and S can get,

To described two width original image I _land I _rdeal with, obtain characteristic pattern and be respectively H _land H _r;

Parallax matrix computations step: H _lin point (x to be matched ₀, y ₀) centered by point create the rectangular window Q that a size is wide m, high n; H _rin, along continuous straight runs side-play amount dx takes out and point (x to be matched in disparity range ₀, y ₀) adjacent onesize be another rectangular window Q ' of m × n; By First Characteristic figure H _lrectangular window Q and Second Characteristic figure H _rrectangular window Q ' compare;

, H _lin with point (x to be matched ₀, y ₀) centered by point m × n rectangular window and H _rthe matching factor of the rectangular window of middle correspondingly-sized horizontal offset dx is:

${\mathrm{\&Gamma;}}_{\mathrm{dx}}(x_0,y_0)=\underset{(i,j)\&Element;Q}{\mathrm{\&Sigma;}}{[H}_R(x_0+i+\mathrm{dx},y_0+j)-H_L{[x}_0+i,y_0+j{\left)\right]}^2,$

I, j is the coordinate take rectangular window as coordinate system;

How much threshold value k are set in preliminary election, if Γ _dx(x ₀, y ₀)≤k, is that the match is successful;

Carry out the successfully off-set value dx of point of record matching by parallax matrix D,

D(x ₀,y ₀)=dx；

At traversal characteristic pattern H _lafter, to parallax matrix D interpolation, to the unique point that the match is successful and the coordinate valuation of unsuccessful extract minutiae;

The offset information that parallax matrix D is comprised is for compute depth;

I _lupper 1 P _lT(x ₁, y ₁, z ₁), by matching I _rupper 1 P _rT(x ₂, y ₂, z ₂), calculate spatial point P _w(x ₀, y ₀, z ₀) depth z ₀; For source images I _lon any point (x, y), in image-forming module optical axis center base length b and lens focus f situation, the degree of depth of its corresponding spatial point is:

$Z(x,y)=f\&times;\frac{b}{D(x,y)};$

D is the described parallax matrix that comprises offset information.

10. position based on object according to claim 7, direction are adjusted the device of image perspective distortion, it is characterized in that, when focal plane and described object are a plane, and when parallel with the camera lens of described image-forming module, pass through: formula: 1/L '-1/L=1/F ' calculates the depth information of described object; Wherein: L is object distance, L ' is image distance, and F is lens focus, and described depth information is object distance.

11. positions based on object according to claim 9, direction are adjusted the device of image perspective distortion, it is characterized in that, in described step C, for described P _lT=(x ₁, y ₁, z ₁), its volume coordinate is P in the coordinate system that is initial point through sensor _w=(x ₀, y ₀, z ₀), have:

$x_0=\frac{x_1\&times;z_0}{f};$

$y_0=\frac{y_1\&times;z_0}{f};$

If rotation θ angle, subpoint P _w'=(x ₀', y ₀', z ₀'), have:

$\mathrm{\&alpha;}=\arctan \left(\frac{y_0}{z_0}\right);$

$y_0\&prime;=\sqrt{{y_0}^2+{z_0}^2}\&times;\sin (\mathrm{\&theta;}+\mathrm{\&alpha;});$

$z_0\&prime;=\sqrt{{y_0}^2+{z_0}^2}\&times;\cos (\mathrm{\&theta;}+\mathrm{\&alpha;});$

x ₀'=x ₀；

Recalculate P _w'=(x ₀', y ₀', z ₀') theoretical picture point P on the sensor of left side _lT'=(x ₁', y ₁', z ₁'), have:

$x_1\&prime;=\frac{x_0\&prime;\&times;f}{z_0\&prime;};$

$y_1\&prime;=\frac{y_0\&prime;\&times;f}{z_0\&prime;};$

z ₁′=f；

Calculate successively every bit, obtain image E;

To image, E makes interpolation, cuts out, and obtains described the first original image figure G after treatment.

12. positions based on object according to claim 11, direction are adjusted the device of image perspective distortion, it is characterized in that, described anglec of rotation θ obtains by following any one mode: acceleration transducer, user specify, image modeling.

Images