CN101943566B - Method and device for measuring tiny two-dimensional displacement by computer camera - Google Patents

Abstract

The invention discloses a method and a device for measuring tiny two-dimensional displacement by a computer camera, providing a method and a device for measuring displacement in a non-contact mode by taking a computer camera as a displacement sensor. The device is formed by connecting the computer camera with a common computer. The computer is also provided with a camera shooting and digital image processing program, so that the computer can obtain the tiny two-dimensional displacement vector and velocity vector of a measured object on a plane which is vertical to the optical axis of the camera under the situation that the lighting condition is good and relatively constant; and the measurement precision is less than one pixel unit. The method for measuring displacement of the invention comprises: by means of autocorrelation calculation carried out on a reference frame, selecting an optimal matching comparison window; by means of cross correlation matching calculation, obtaining the instant relative displacement vector and the total displacement vector of the object; according to the measuring result, determining whether to update the reference frame and the optimal matching comparison window thereof; and automatically regulating to search the magnitude of the matched cross correlation operator array.

Description

Use the computing machine camera to measure method and the device of small two-dimension displacement

Technical field

The invention belongs to the digital picture field of measuring technique, particularly adopt the computing machine camera to measure method and the device thereof of the two-dimentional micro-displacement of object.

Background technology

The development of microelectric technique has promoted the progress of videographic measurment technology.With " shooting " and " measurement " word that is the theme, can retrieve 25 of domestic patents of invention, 5 of utility model patents.For example, the measuring process that patent of invention " a kind of spatial three-dimensional position attitude measurement method for video camera " (publication number: CN 1804541A, the applying date: 2006.7.19, Beijing Institute of Aeronautics is large) proposes is: 1) calibration phase, set cooperative target; In camera coverage, freely, non-parallel mobile cooperative target at least 5 positions, piece image is taken in every movement position, is called the camera calibration image; The perspective projection image of 2 circles on cooperative target is 2 ellipses, be that the ratio of area and perimeter square is as Expressive Features take closeness, auto Segmentation goes out 2 ellipses in uncalibrated image, extract 8 all public point of contacts of 2 ellipses in the camera calibration image of each position, obtain the line intersection point at different public point of contacts, public point of contact and line intersection point be totally 21 points, is called the target signature point; Utilize target signature dot image coordinate and the corresponding world coordinates calibrating camera inner parameter thereof of all positions, comprise effective focal length and principal point coordinate.2) measuring phases: fixing cooperative target, mobile camera, perhaps mobile cooperative target, fixed cameras is taken the image of cooperative target, and the image coordinate of extracting 21 target signature points on cooperative target is also corresponding with its world coordinates; According to the camera motion model of setting up and perspective projection model, utilize image coordinate and the corresponding world coordinates of intrinsic parameters of the camera, target signature point, calculate the three-dimensional space position attitude of video camera; The continuous coverage video camera calculates linear velocity and the angular velocity of mobile object at the three-dimensional position attitude of 2 diverse locations.This shows, this characteristic feature of an invention is: use video camera to measure, need cooperative target, need to carry out a series of staking-out work before measurement.

for another example, patent of invention " digital camera measurement of three-dimensional geologic structural surface information and acquisition method " (publication number: CN101029826A, the applying date: 2007.9.5, the Chengdu science and engineering is large) claim: with the information carrier of digital video frequency flow as the mapping object, abandon, reduce traditional photography and measured resolving of required interior outer orientation unit, in the computer stereo vision environment of building again on computers, by to continuously, the control of dynamic video flowing parameter-loading velocity and frame position realizes the three-dimensional location compute of object point, thereby complete the videographic measurment of structural plane on various ground slope, decipher and edit and record work.Obviously, video camera is adopted in this invention, for needed decipher precision, need to consider the resolution of video flowing itself, the camera optical factors such as the visual angle of shooting and magnification.

And for example, " a kind of photographing measurement method of long straight rail geometric parameter " (patent of invention publication number: CN 101021417A; The applying date: 2007.3.21, science and techniques of defence are large) be: the wireline inspection car headstock on long straight rail is the cooperation sign fixedly, at circuit inspection vehicle the place ahead installation video camera, take with the cooperation sign of track fluctuating deflection campaign at least two in the same time images not with video camera, process the geometric parameter that obtains described track by image, comprise gauge, superelevation, indulge partially and at least one item in versed sine.This invention also proposes, and installs adaptive optics equipment additional, or does the atmospheric agitation correction with the form of software algorithm, but further instruction is not arranged.Also having similarly: " detecting the photographing measurement method of highroad pavement planeness " (patent of invention publication number: CN 101126638A, the applying date: 2008.2.20, science and techniques of defence are large), " orbit geometry parameter image measuring device and the method for alternative device for carrying a tripot collimation method " (patent of invention publication number: CN 101264766A, the applying date: 2008.5.15, science and techniques of defence are large).(digital signal pattern) video camera is all used in these three kinds of inventions, needs to install the cooperation sign of shooting use; Specifically, although all adopted digital image processing techniques, all do not further illustrate the sub-pixel positioning technology that adopts.

in recent years, development along with CCD and CMOS photoelectronic imaging chip, the computing machine camera is popularized soon, its core is the photoelectric sensing cell array, common resolution has: QSIF (160*120), QCIF (176*144), SIF (320*240), CIF (352*288) and VGA (640*480) etc. are several, the actual speed of taking is respectively: 30fps (frame per second), 30fps, 20-26fps, 20-26fps and 10fps, its vision signal belongs to digital signaling, export with USB interface, possessed the physical basis that is used for videographic measurment.With respect to video camera, the computing machine camera is cheap, and is easy to use, generally is used for the Internet video chat or monitors shooting work; Yet, not yet see the report that it is used to the videographic measurment aspect.

Scanner and optical mouse also have the photosensor arrays chip of one dimension or two dimension, and its principle of work is particularly obtained some United States Patent (USP)s of relative motion vectors about them, provide useful inspiration for using camera to measure displacement.

For example, " Navigation technique for detecting movement of navigation sensors relative to anobject " (US 5,644,139, Jul.1,1997) have discussed the characteristics of four kinds of scanners, that is:

1) drum-type.

2) (drawing of 8.5in * 11in), some reaches the drawing of A1, adopts principal computer to make control, storage and managing image to A4 greatly for dull and stereotyped (linear array transducer) formula, suitable scanning.

3) 2 dimension sensor array formulas, the machinery-free coding, the exposure period original graph is motionless, and 2D array photosensitive unit conversion imaging pixel array, size are that the figure of the 300dpi of A4 needs 2500 * 3300=8.25 mega pixel.

4) hand-held mobile adopts less linear array electro-optical sensor unit, directly is connected with personal computer, and takes this storage, processes and use view data.The data rate of imageing sensor restriction sweep speed for this reason, to the user, to keep suitable sweep velocity, reaches required image resolution ratio by the indication feedback of red, green light emitting diode.Some hand-held scanner uses electromagnetism to stop, and sweep velocity increases, and mechanical resistance also increases, and has prevented from dragging too fast.

Hand-held scanner determines that the method for positional information of pel array is similar to the principle of work of mechanical mouse.When linear array transducer moved, wheel, ball or roller pearl were perceived with the rolling that original drawing contacts, and its positional information is determined according to the mechanical detail that rotates.Usually, having high friction factor with the surface of the contacted mechanical organ of original drawing, is for example rubber, slides and brake to stop.Rotatingcylindrical drum or be used for strengthening the scanning process motion with two wheels that stiff shaft connects and keep single translational degree of freedom, the direction of scanning of the fixing relatively original drawing of straight line commonly used or other fixture, and the translation of needing in scanning process further strengthening retrains.When lacking the fixture of these maintenance moving directions, when perhaps user's elbow dropped on certain plane, certain deflection can occur in the motion track of one dimension photosensor arrays, caused scanning the charge pattern that obtains and was distorted.Certainly, position encoded method usually is easy to produce slides or jumps, and can cause the pixel data array of one dimension and the bit errors of original image.

This patent has provided the method for compute associations coefficient, and takes this to obtain the positional information of each pixel.For reference, continue summary as follows:

First obtain reference frame, it has certain immanent structure characteristics, the for example imaging of T shape, typical surperficial immanent structure feature is between 10 μ m-40 μ m, the size of reference frame depends on the sweep velocity of the maximum of scanister, the main spatial frequency of surface structure characteristics and the factors such as resolution of sensor, and the actual size of the reference frame of the navigation sensor of N * M=32 * 64 is taken as the pel array of N * M=24 * 56.After dt after a while, obtain the sampling frame, it has showed same immanent structure characteristics.Last dt before this patent is emphasized to take a sample and will satisfy " micro-stepping " condition: under the point-to-point speed of scanner, relative displacement is less than the spacing of a pixel of navigation sensor; An acceptable sample interval is: 50 delicate (sweep velocity 0.45 metre per second (m/s) is during 600dpi (pixel per inch)).

The position at the so-called related same characteristics place that is used for finding successive frame, to determine the displacement from frame-frame characteristics, sue for peace or accumulate these displacements, proofread and correct the calibration factor of introducing by relevant optical design, being specified to the displacement of image-position sensor, is the program of a scanning process.Frame-frame association refers to select frame per second sufficiently high corresponding to " micro-stepping ", guarantees that displacement is no more than the size of a single pixel.Excessive sampling can provide " displacement accuracy of sub-pix ".

Have some correlating methods to adopt, the method that this patent proposes is the related of " poor square with ":

$C_k=\underset{i}{\mathrm{\Σ}}\underset{j}{\mathrm{\Σ}}{(S_{\mathrm{ij}}-R_{\left(\mathrm{ij}\right)+k})}^2$

Wherein, S _ijThe expression navigation sensor is at the measured value of the position (i, j) of sampling frame, R _ijThe value of expression reference frame after direction k moves, 8 possible moving directions that k representation feature element is the most contiguous, k=1-8 if k=0 represents that this element is not mobile, has active 8+1=9 correlation coefficient from skew altogether.The corresponding k of the value of correlation coefficient minimum has provided direction and the amplitude of frame-frame feature relative motion.

Follow each related calculating, make a decision whether adopting " micro-stepping ".If necessary " micro-stepping ", reference frame shifts, and namely in this step, the sampling frame has become reference frame.Then, obtain a new sampling frame, repeat related calculating.

When process provides correlation degree the highest coupling, follow each sampling frame to be updated to the continuous moving of reference frame, any error that can occur will accumulate, the growth rate of this in order to limit " walk random " error, a sampling frame is stored in an independent memory buffer, the sampling frame of this Separate Storage has become a new reference frame, is used for a series of related calculating subsequently.This person's association refers to " grand step ".Use " grand step ", can determine more accurately to cross the scanner displacement of the distance (being m micro-stepping) of m picture frame." grand step " inner error is a single related result of calculating, and the error of the equivalence of m " micro-stepping " is the error in single " grand step "

Doubly.Although m increases, the mean value of the error of m " micro-stepping " is close to 0, the average standard deviation of error with

Increase.Like this, use large as far as possible " the grand step " of m, be conducive to the standard deviation of the error that reduces to accumulate, prerequisite is, two frames that define " grand step " are not to separate too far each other, and both marking areas have common picture material.

This patent also proposes Proper Sampling Period dt and needs not to be constant, and the function that can be used as previous measurement is determined.Use the method for variable dt to be, between successive frame, relative displacement within certain constraint, improves the accuracy that displacement is calculated by keeping.For example, perhaps the upper limit is the displacement of a pixel, and lower limit is determined by the numerical roundoffconsideration in the navigation data processing procedure.Just here not clear language is herein.

Due to one dimension sensor array less, greater than A6 the time, use the film size face (stiching algorithms join together multiple swaths of a larger documents) that algorithm is pasted Multiple-Scan of sewing up when the breadth of single pass.For this reason, this patent is provided with a photoelectric sensor separately at pixel sensor array two ends, to determine starting point and the terminal point coordinate of this row pixel, and the data of pressing form storage every delegation charge pattern of starting point coordinate, pixel value and terminal point coordinate, nature can calculate the coordinate of each pixel, and the calculation expression of " sub-pix adds (sub pixel position interpolation) " has been proposed, the picture of twice sweep is stitched into a large view picture.This calculating is quite puzzling.

In fact, (US 7 about the patent " Method for calculating movement value of optical mouse andoptical mouse using the same " of optical mouse, 167,161B2, jan.23,2007) help to be expressly understood the technological concept of above-mentioned " micro-stepping " or " grand step ".This patent graphical analysis road: relative displacement Δ x＜0.5 is occuring, under the situation of Δ y 〉=1 pixel, if change reference frame, can cause the displacement abandoned of Δ x＜0.5 pixel, and can not get accumulative total; If move comparison window reference frame in reverse direction toward the relative displacement vector that calculate according to the shift value that calculates this moment, in the cycle, the result of calculation of " rounding up " can occur second sampling coupling, improved the order of accuarcy of measuring.This patent has also proposed to upgrade the scheme of reference frame: in previous some sample periods, or the mean value of the result of interior calculating relative displacement of a certain predetermined time, determine an estimated value; If the shift value that this sample period calculates is not less than this estimated value, so, each sample period is upgraded a reference frame with the sampling frame; If the shift value that this sample period calculates is less than this estimated value, so, every 4 sample periods are upgraded a reference frame with the sampling frame; If the shift value that this sample period calculates is much smaller than this estimated value, so, every 16 sample periods are upgraded a reference frame with the sampling frame; Namely the relative displacement vector is divided into large, medium and small three class, respectively corresponding three frequencies of upgrading reference frame.And in other sample period, only the reverse direction of the comparison window in reference frame (interior perhaps position) toward the relative displacement vector that calculates moved according to the shift value that calculates, thereby improved the accuracy of measuring the displacement of mouse low-angle.

In addition, (US 6,664,948B2 for patent " Tracking pointing device motion using a single buffer for cross and auto correlation determination ", Dec.16,2003) mechanical mouse and optical mouse have been introduced more particularly; Illustrate the computing method of cross correlation and self-correlation; It calculates movement velocity according to measured motion vector, takes this to judge the overlapping degree of reference frame and sampling frame, considers whether to upgrade reference frame; Also for the situation of the feature Fuzzy of related reflecting surface, propose, if when the threshold value that the value of the minimum of 9 correlation coefficients presets less than certain, computing array should be transformed into 5 * 5 by 3 * 3, namely enlarge the scope of coupling, to comprise more feature as a means of comparing; Specifically, this patent is said, relative position is by obtaining corresponding to the additional distance extrapolation of the speed of current location, and the calculation expression of " sub-pixel interpolation " coefficient (a sub-pixel interpolation component of the relativemovement) of relative motion proposed, with self-correlation and the relative displacement of the cross correlation coefficient calculations Δ x less than a pixel size, Δ y; But, its calculating formula indigestion.This patent is claimed, the maximum movement speed of the mouse that can accurately follow the tracks of is 18in/second, image capture speed is 6000frames/sec-9000frames/sec, the size of picture frame is 20 * 20, the scope of pixel value is 0-255, control light source and imageing sensor, make the mean value ≈ 128=255/2 of pixel value.

From the above, the photoelectric sensing cell array of computing machine camera is greater than more than 5 times of optical mouse, is conducive to surveying work, and its optical lens can regulate, and can be used for object far away is carried out noncontacting measurement.Its shortcoming is, image to catch frame per second very low, can only adapt to motion conditions more at a slow speed.

Summary of the invention

Function for the photosensor arrays of giving full play to the computing machine camera, the invention provides a kind of method and device that uses the computing machine camera to measure small two-dimension displacement, it is take the computing machine camera as photoelectric conversion sensor, can be in the good and geostationary situation of lighting condition, utilize the computing machine camera immediately to take continuously measured object, again by Digital Image Processing, obtain this object with the optical axis of camera perpendicular plane on small two-dimension displacement vector velocity.

The technical solution adopted for the present invention to solve the technical problems is: measured object is positioned in the effective focal imaging scope in computing machine camera front, described camera is connected to the logical computing machine of a Daepori by USB interface, and this allocation of computer USB interface, internal memory, CPU, hard disk, display card and display, keyboard and mouse, operating system, webcam driver program and the camera of writing are specially taken and the Digital Image Processing program.

The step of using the computing machine camera to measure the small two-dimension displacement of object comprises:

One, move on computers the webcam driver program, connect camera to computing machine.

Two, in assurance surveying work environment, the illumination of light well and does not substantially change.

Three, camera is aimed at measured object, and the good camera of focus adjustment makes imaging clearly.

Four, the operation camera is taken and the Digital Image Processing program.

The method that the present invention uses the computing machine camera to measure the small two-dimension displacement of object is:

1) operation computing machine camera is taken the image of a frame testee, preserves with the form of bitmap, as the reference frame.Select the big or small M * N of bitmap frame according to the specific targets of the camera of concrete use, M, N ∈ positive integer is selected shooting speed frame per second faster as far as possible.

Being that pel array corresponding to described bitmap select a rectangular coordinate system, take the position of first pixel in its upper left corner as initial point, is laterally the x direction of principal axis to right, and direction vertically downward is the y direction of principal axis.

Open up a comparison window at the middle section of described pel array, size is taken as m * n, m, and n ∈ integer is chosen suitable initial value: m ₀* n ₀The horizontal direction of described comparison window and described pel array and the edge pixel of vertical direction be distance h and v pixel respectively, namely has: m+2h=M, n+2v=N.

2) calculate the self-correlation of the pel array of comparison window in described reference frame:

$\mathrm{auto}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{reference}(x+a,y+b)|$

In formula, x and y are respectively the coordinates of pixel in comparison window, the value of each pixel of comparison window in reference (x, y) expression reference frame, and a=-1,0,1, b=-1,0,1, common property is given birth to 9 self-correlation auto_correlation (a, b).

If these self-correlations have approximately equal more than half and are tending towards 0, the architectural feature on surface that subject is described is meticulous not, value between neighborhood pixels be can not distinguish, be unfavorable for adopting from now on frame-frame coupling comparative approach to measure displacement, need to enlarge the zone that is used for matching ratio pixel, make m=m ₀+ step, n=n ₀+ step, step ∈ positive integer namely enlarges the capable and step row of each step of scope of comparison window, recomputate the self-correlation of comparison window, so repeat, until find the coupling comparison window that is fit to this body surface texture characteristic, at this moment, 2h=M-m, 2v=N-n.If exceed certain scope of frame, also do not find suitable comparison window, think that this part reflecting surface of this object is unsuitable for the surveying work of this device, and provide the prompting warning.

If these self-correlation approximately equals and be tending towards 0 less than half, the architectural feature on surface that subject is described is enough meticulous, value between neighborhood pixels can be distinguished, and can further attempt choosing less pel array, to reduce amount of calculation.Make m=m ₀-step, n=n ₀-step, step ∈ positive integer, namely dwindle the capable and step row of each step of scope of comparison window, recomputate the self-correlation of comparison window, so repeat, until at this moment the present self-correlation approximately equal in zone and be tending towards 0 the half that is not less than has found under present body surface situation to be used for carrying out matching ratio best pixel array: m=m ₀-step, n=n ₀-step, 2h=M-m, 2v=N-n.

3) through taking the second framing bit figure after Δ t after a while, as sampling frame and preservation.

The speed of taking the sampling frame is preferably faster than the speed of a pixel unit of object relative displacement.

4) value of the pel array in comparison window in described reference frame is carried out cross-matched relatively according to 9 * 9 associative arrays in described sampling frame scope, specific algorithm is:

$\mathrm{cross}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{comparison}(x+a,y+b)|$

In formula, the value of each pixel of comparison window in the described reference frame of reference (x, y) expression, the value of the pixel of the position corresponding with the contingent situation of movement of described reference frame comparison window in the described sampling frame of comparison (x+a, y+b) expression, described reference frame has 81 kinds of contingent situation of movement, corresponding a=-4 ,-3 ,-2,-1,0 ,+1, + 2 ,+3 ,+4 and b=-4,-3,-2 ,-1,0, + 1, + 2 ,+3 ,+4 combinations, common property is given birth to 81 cross correlation coefficient cross_correlation (a, b).

5) value is minimum and be tending towards 0 cross correlation coefficient cross_correlation (a, b) represent that its frame-frame correlation degree is the highest, therefore obtain direction and mobile amplitude that described sampling frame moves relative to described reference frame: Δ x=a, Δ y=b, this is the relative displacement vector that in this sample period, object occurs, in measuring process, the total relative displacement vector of object is: Δ X=Δ X+ Δ x, Δ Y=Δ Y+ Δ y, in formula, (Δ X, the Δ Y) on the right is the object relative displacement vector of accumulation in the past.

6) according to above-mentioned displacement vector and take Δ t interval time between above-mentioned two continuous frames, obtain the velocity Δ v of ohject displacement _x=Δ x/ Δ t, Δ v _y=Δ y/ Δ t;

7) if | Δ X| 〉=m/2, or | Δ Y| 〉=n/2, m, n ∈ integer, the relative displacement that namely comparison window occurs in described reference frame have exceeded half amplitude of this comparison window, at this moment, replace described reference frame with up-to-date sampling frame, its comparison window is repositioned at the central part of new reference frame.If | Δ X|＜m/2﹠amp; | Δ Y|＜n/2, do not upgrade described reference frame, but the comparison window generation relative displacement Δ x=-a in described reference frame, Δ y=-b.

8) if upgraded reference frame, imitative program 2) self-correlation of calculating book reference frame, watch surface texture featur, again determines the big or small m * n of best comparison window.

9) the above-mentioned cross correlation operator array that comparison window is searched for coupling in described sampling frame scope in described reference frame is adjusted automatically according to this measured value: if | a|＜5﹠amp; | b|＜5, changing to take is 7 * 7, if | a|＜3﹠amp; | b|＜3, changing to take is 5 * 5, otherwise still is taken as 9 * 9 associate operator arrays, to reduce amount of calculation.

10) take a new frame sample frame and preservation; Take the frame of sampling frame-frame period time preferably faster than or time of spending near pixel unit of object relative displacement.

11) jump to program 4), continue to calculate the cross correlation coefficient, obtain relative displacement vector and velocity.

In the actual measurement process, can also further implement to measure calibration, take this to obtain direct measurement result.

The invention has the beneficial effects as follows, it is that a kind of computing machine camera that adopts is realized new method and the device of noncontacting measurement two dimension micro-displacement as displacement transducer, has given full play to the effect of the photosensor arrays of computing machine camera; The method adopts the texture characteristic of self-correlation ANALYSIS OF CALCULATING measured object reflecting surface, and the size of automatically regulating matching area is to realize the minimum of computation workload; The method is in time upgraded reference frame automatically, does not need the preset threshold value of intervention program, and measuring accuracy is less than a pixel unit; The method realizes searching for best matching area according to taking the sample interval and calculating the size that the velocity of displacement vector that obtains is adjusted the cross correlation operator automatically; This measuring technique is advanced, and operation is convenient, and is with low cost.

Description of drawings

Fig. 1 is measurement mechanism block scheme of the present invention.

Fig. 2 is the schematic diagram that the photoelectric sensor chip in the computing machine camera carries out the pel array that produces after opto-electronic conversion.

Fig. 3 is the schematic diagram with respect to the auto correlation situation of a point.

In Fig. 1,1. computing machine camera, 2. optical lens, 3. photoelectric sensor chip, 4.USB interface, 5. computer system, 6.USB interface, 7.CPU, 8. internal memory, 9. display card and display, 10. hard disk, 11. keyboards and mouse, 12. operating system, 13. the webcam driver program, 14. cameras are taken and the Digital Image Processing program, 15. light fixture.

In Fig. 2,20. 1 frame pel arrays, 21. comparison window, 22. make matching ratio sampling frame inner region (illustrated embodiment), the extreme position that in 23. reference frames, comparison window is subjected to displacement with the interior comparison window of reference frame.

Embodiment

Fig. 1 is the block scheme of measurement mechanism of the present invention.Its Computer camera (1) is aimed at measured object, and both distances are in the effective optical lens focal imaging scope of camera (1).described camera (1) is by optical lens (2), photoelectric sensor chip (3) and USB interface (4) form, be connected to the USB interface (6) of the logical computer system (5) of a Daepori by the USB cable of its configuration, this computer system (5) also disposes CPU (7), internal memory (8), display card and display (9), hard disk (10), keyboard and mouse (11), operating system (12), webcam driver program (13) and the camera of writing are specially taken and Digital Image Processing program (14).

The step of using the computing machine camera to measure the small two-dimension displacement of object comprises:

One, move on computers the webcam driver program (13) of rationing with camera (1), connect camera (1) to computing machine.

Two, keep the illumination of light in the surveying work environment well and substantially not change.

Three, camera is aimed at measured object, and the good camera of focus adjustment makes imaging clearly.

Four, the operation camera is taken and Digital Image Processing program (14), implements to measure in real time.

The method that described camera is taken and Digital Image Processing program (14) adopts progressively is described below:

1) operation camera (1) is taken the image of a frame testee, is saved in hard disk (10) with the form of bitmap.Select the big or small M * N of bitmap frame according to the concrete model of camera, M, N ∈ positive integer is selected shooting speed frame per second faster as far as possible.

The common resolution of digital camera head has QSIF (160*120), QCIF (176*144), SIF (320*240), CIF (352*288) and VGA (640*480), its actual shooting speed is corresponding respectively: 30fps (frame per second), 30fps, 20-26fps, 20-26fps and 10fps.

Fig. 2 represents that photoelectric sensor chip (3) carries out the corresponding pel array (20) that produces after opto-electronic conversion.The size of this pel array (20) is M * N, M, N ∈ integer, corresponding bitmap format, wherein each lattice (being pixel) has the numerical value (0-255) of the 8bit of the power that represents three kinds of primary colours of red, green and blue, 24bit, get the typical value (0-255) that its comprehensive brightness number is made this pixel now altogether.We are that this pel array (20) is selected a rectangular coordinate system, take the position of first pixel in its upper left corner as initial point, are laterally the x direction of principal axis to right, and direction vertically downward is the y direction of principal axis.So each lattice is that pixel is orientated a pair of coordinate points (x, y) as, 0≤x≤M, 0≤y≤N, x, y ∈ integer.

Middle section in Fig. 2 has been opened up a comparison window (21), size is taken as m * n, m, n ∈ integer, comparison window (21) and the horizontal direction of picture element matrix (20) and edge pixel difference distance h and v pixel of vertical direction namely have: m+2h=M, n+2v=N, h, v ∈ positive integer.Therefore, the coordinate of first pixel grid of the upper left corner of comparison window is: x=h+1, y=v+1.

Choose m, n is relevant with the fine degree of the architectural feature of object reflecting surface, is related to (coupling) measuring accuracy, is determining amount of calculation, affects the speed of response of measurement mechanism.For above-mentioned resolution and frame per second index, for example can choose initial value: m ₀=80, n ₀=80.

2) self-correlation of the pel array of comparison window in the computing reference frame:

$\mathrm{auto}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{reference}(x+a,y+b)|$

In formula, x and y are respectively the coordinates of certain pixel in comparison window, the value of each pixel of comparison window in reference (x, y) expression reference frame, and a=-1,0,1, b=-1,0,1, common property is given birth to 9 self-correlation auto_correlation (a, b).

In order to understand above-mentioned calculating formula, the auto correlation situation of first read fortune to a some P (a, b): m=1, n=1.As shown in Figure 3, for a P (0,0), it has the pixel of 8 arest neighbors, in other words, and in its arest neighbors direction, it has 8 possible moving directions, a kind of situation that the P that puts a spot (0,0) is not moved, amount to 9 possible situation of movement, respectively corresponding: a=-1,0,1 and b=-1,0,1 combined result.If the reflecting surface of measured object has careful differentiable texture and structural characteristic, for example, the surfaces such as character edge, speckle pattern or fiber papery, these object surface structure features belong to microscopic property, and the order of magnitude can be surveyed by photosensor arrays at 10 μ m-40 μ m, so, the value of each the most contiguous pixel is generally unequal, and the result of calculation auto_correlation (a, b) of above-mentioned 9 self-correlations can not be 0.If the texture and structural characteristic of the reflecting surface of measured object can not distinguished so subtly, or smoother is well-balanced, so, the value of each the most contiguous pixel generally approaches or equates, the result of calculation auto_correlation (a, b) of above-mentioned 9 self-correlations can or be close to 0 for 0.Therefore, self-correlation auto_correlation (a according to frame, b) result of calculation, whether the quality architectural feature that can judge the testee reflecting surface is fit to the computing machine camera in the present invention, whether needs to enlarge the match search zone of two frames.

Self-correlation with respect to a point is calculated expand to a little picture element matrix or comparison window, can check a zone and another regional match condition, this is expressed as above-mentioned auto correlation calculating formula on mathematics.Comparison window has the onesize zone of 8 arest neighbors, and in other words, comparison window adds comparison window oneself in 8 kinds of situations that its arest neighbors might be moved, and one has 9 zones, all belongs to same frame; Comparison window is made matching ratio with them respectively, so be referred to as self-correlation.If one of comparison window and 9 kinds of possible situations are coincide, the result that auto correlation calculates should be 0; If one of this comparison window and 9 kinds of possible situations are more approaching, the end value that auto correlation calculates is less, shows that correlation degree is higher.

If these self-correlations have approximately equal more than half and are tending towards 0, the architectural feature on surface that subject is described is meticulous not, value between neighborhood pixels be can not distinguish, be unfavorable for adopting from now on frame-frame coupling comparative approach to measure displacement, need to enlarge the zone that is used for matching ratio pixel, make m=m ₀+ step, n=n ₀+ step namely, enlarges the capable and step row of each step of scope of comparison window, for example gets step=5, recomputate the self-correlation of comparison window, so repeat, until find the comparison window that is fit to this body surface texture characteristic, at this moment, 2h=M-m, 2v=N-n.If exceed certain scope of frame, 3/5 (m of frame for example _max≤ 3/5M, n _max≤ 3/5N, m _max, n _maxThe ∈ positive integer), also do not find suitable comparison window, think that this part reflecting surface of this object is too smooth even, be unsuitable for the surveying work of this device, and provide the prompting warning.

If these self-correlation approximately equals and be tending towards 0 less than half, the architectural feature on surface that subject is described is enough meticulous, value between neighborhood pixels can be distinguished, and can further attempt choosing less pel array, to reduce amount of calculation.Make m=m ₀-step, n=n ₀-step, namely dwindle the capable and step row of each step of scope of comparison window, for example get step=5, recomputate the self-correlation of comparison window, so repeat, until at this moment the present self-correlation approximately equal in zone and be tending towards 0 the half that is not less than has found under present body surface situation to be used for carrying out matching ratio best pixel array: m=m ₀-step, n=n ₀-step, 2h=M-m, 2v=N-n.

3) take the second framing bit figure after Δ t after a while, as the sampling frame and be saved in hard disk (10).

The speed of taking the sampling frame is preferably faster than the speed of a pixel unit of object relative displacement.

4) value of the pel array in comparison window in reference frame is carried out cross-matched relatively according to 9 * 9 associate operator arrays in sampling frame scope, specific algorithm is:

$\mathrm{cross}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{comparison}(x+a,y+b)|$

In formula, the value of each pixel of comparison window in reference (x, y) expression reference frame, the value of the pixel of the position corresponding with the contingent situation of movement of reference frame comparison window in comparison (x+a, y+b) expression sampling frame, reference frame has 81 kinds of contingent situation of movement, corresponding a=-4 ,-3 ,-2,-1,0 ,+1 ,+2, + 3 ,+4 and b=-4 ,-3 ,-2,-1,0 ,+1 ,+2, + 3 ,+4 combination, common property is given birth to 81 cross correlation coefficient cross_correlation (a, b).

The calculating of above-mentioned correlation coefficient is made comparisons respectively based on the comparison window in reference frame and this comparison window one of possible 81 kinds of situation of movement---the mobile result that comprises a kind of reality---in the sampling frame, therefore, is referred to as the cross correlation coefficient.

The meaning of above-mentioned cross correlation coefficient calculations expression is that the comparison window in reference frame is searched for matching ratio in the sampling frame.Also can open up comparison window in the sampling frame, it be done the search matching ratio in reference frame, cross correlation coefficient calculations expression at this moment should be write as:

$\mathrm{cross}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{comparision}(x,y)-\mathrm{reference}(x+a,y+b)|$

The sampling frame has reflected the later mobile result of reference frame elapsed time dt.Suppose that object is rigidity, position reference (the x of first pixel in the upper left corner of comparison window in can reference frame, y) represent whole movement, in reference frame, comparison window correspondingly can move to the position comparison (x+a of sampling frame, y+b), for example in Fig. 2 with reference frame in comparison window do shown in matching ratio sampling frame zone (22).At this moment, after comparison window moves with it in reference frame in the sampling frame coupling of residing zone generation top, or make correlation degree the highest.In two zones like this value of pixel in twos correspondence subtract each other, result should each be 0 or level off to 0.In reference frame, 81 mobile results might occur at its arest neighbors in comparison window, all there is a zone onesize with comparison window corresponding to wherein each situation of movement in the sampling frame, the value of pixel that should the zone in the sampling frame respectively the value of the pixel corresponding with comparison window zone in reference frame subtract each other in twos, result generally can not be 0, can there be a difference, add up again this relevant absolute value that differs from of all pixels in the comparison window zone, so obtain 81 cross correlation coefficient cross_correlation (a, b), it can be one away from 0 difference.Briefly, see exactly comparison window in reference frame has moved to or approached which zone in 81 possible zones in the sampling frame, if true with may match, the difference of both pixel values will be identical or approaching correspondingly, otherwise, there are differences.The size of difference will obtain optimum matching accordingly, obtains the relative displacement vector.

5) value is minimum and be tending towards 0 cross correlation coefficient cross_correlation (a, b) represent that its frame-frame correlation degree is the highest, therefore obtain direction and mobile amplitude that sampling frame relative reference frame moves: Δ x=a, Δ y=b, this is the relative displacement vector that in this sample period, object occurs, in measuring process, the total relative displacement vector of object is: Δ x ₀=Δ x ₀+ Δ x, Δ y ₀=Δ y ₀+ Δ y, in formula, (Δ X, the Δ Y) on the right is the object relative displacement vector of accumulation in the past.

6) according to institute's interlude between above-mentioned displacement vector and shooting two continuous frames, can obtain the velocity of ohject displacement.

7) if | Δ X| 〉=m/2, or | Δ Y| 〉=n/2, m, n ∈ integer, the relative displacement that namely comparison window occurs in reference frame have exceeded half amplitude of comparison window, at this moment, replace reference frame with up-to-date sampling frame, its comparison window is repositioned at the central part of new reference frame.If | Δ X|＜m/2﹠amp; | Δ Y|＜n/2, do not upgrade reference frame, but the comparison window generation relative displacement Δ x=-a in reference frame, Δ y=-b.

The cardinal rule of upgrading reference frame is, the match search zone in comparison window wherein and the sampling frame of taking later, and in other words, sampling frame and reference frame have considerable overlapping region, otherwise have lost the meaning of calculating cross correlation coefficient.

The method of above-mentioned renewal reference frame can reflect the measuring accuracy less than a pixel unit, is better than each related method of all upgrading later reference frame of calculating, and hereby is described as follows.

Suppose that the displacement that object occured in each sample period is :+directions X moves 0.3 pixel unit, and+Y-direction moves 3 pixel units.First take a frame, preserved as the reference frame, and mark comparison window at the middle section of this reference frame, the size of comparison window is m * n=60 * 60 for example.Then, take a frame in first sample period, also preserved as the sampling frame.Mate in the scope of whole sampling frame, can find with reference frame in comparison window have that the highest related zone---both not necessarily fit like a glove, perhaps some local pixel content of comparison window scope of having overflowed frame in frame in sampling, this zone comparison window in the relative reference frame in the scope of whole frame has moved 0.3 pixel unit at+directions X, has moved 3 pixel units in+Y-direction.The result of calculating the cross correlation coefficient and getting its minimum value is, in first sample period movement of objects (0,3) (pixel), therefore, lost+0.3 pixel unit that directions X moves.Next step upgrades reference frame and is the sampling frame in above-mentioned first sample period, and the middle section of new reference frame as new comparison window.Then, take a frame in second sample period, preserved as new sampling frame.Then, find with the comparison window of reference frame to have the highest related zone in whole sampling frame scope, this zone comparison window in the relative reference frame in the scope of whole frame has moved 0.3 pixel unit at+directions X, has moved 3 pixel units in+Y-direction.But the result of calculating the cross correlation coefficient is, in second sample period movement of objects (0,3) (pixel), therefore, again lost+0.3 pixel unit that directions X moves.This shows, the actual value of ohject displacement be (0.6,6) (pixel), the result of calculating be (0,6) (pixel), the method for above-mentioned calculating cross correlation coefficient can not reflect the shift value of 0.6 actual pixel unit.The reason that the method can not be calculated the low-angle displacement is: it has upgraded reference frame in each sample period, and its comparison window is fixed in the middle section of reference window.Calculate the cross correlation coefficient and get the result of its minimum value, " rounding up "---optimum matching occurs in possible direction corresponding to cross correlation coefficient minimum value to the generation of relative displacement value, is calculated as the displacement of a pixel unit greater than the displacement of certain predetermined value.

Adopt now another kind of method to calculate the relative displacement vector of the problems referred to above.Known: in first sample period, the result of calculating the cross correlation coefficient is, movement of objects (a=0, b=3) (pixel).At this moment, upgrade reference frame without the sampling frame in first sample period, but the reverse direction of the past relative displacement vector that calculates of the comparison window in reference frame is moved (a=-0 ,-b=-3) (pixel).Then, take a frame in second sample period, also preserved as new sampling frame.Find with the new comparison window of reference frame in whole sampling frame scope and have the highest related zone, this zone in the scope of whole (sampling) frame in the relative reference frame new comparison window moved 0.6 pixel unit at+directions X, moved 3 pixel units in+Y-direction.The result of calculating the cross correlation coefficient and getting its minimum value is, in second sample period movement of objects (1,3) (pixel).That is to say, in second sample period, comprised in first sample period shift value at+0.3 pixel unit that directions X moves at the shift value of+directions X, reality has moved 0.6 pixel unit, based on the calculation expression of cross correlation coefficient, it is calculated as 1pixel.Therefore, the last shift value in the first two sample period is actually (0.6,3) (pixel), and the shift value that calculates be (1,3) (pixel), actual shift value 0.6pixel has been reflected in the shift value that calculates.

The range limit that comparison window in reference frame is moved is subject to the restriction of frame width size, has for example illustrated the extreme position (23) that comparison window is subjected to displacement in Fig. 2.

8) imitative program 2) calculate the self-correlation of the reference frame after upgrading, again watch surface texture featur, again determine the big or small m * n of best comparison window.

9) the above-mentioned cross correlation operator array that comparison window is searched for coupling in described sampling frame scope in described reference frame is adjusted automatically according to this measured value: if | a|＜5﹠amp; | b|＜5, changing to take is 7 * 7, if | a|＜3﹠amp; | b|＜3, changing to take is 5 * 5, otherwise still is taken as 9 * 9 associate operator arrays, to reduce amount of calculation.

10) take a new frame as the sampling frame and preserve; Take the frame of sampling frame-frame period time preferably faster than or time of spending near pixel unit of object relative displacement.

11) jump to program 4), continue to calculate the cross correlation coefficient, obtain relative displacement vector and speed of related movement vector.

In the actual measurement process, can also further carry out and measure calibration, to obtain direct measurement result.

The cross correlation operator array that above-mentioned search coupling adopts not necessarily must be since 9 * 9, its basic thought is the amplitude size according to ohject displacement, the speed of displacement in other words, select from big to small, but want the scope of coverture displacement body, guarantee coupling may with correctly, also the sampling frame per second to camera is relevant for this, affects the response performance of measurement.

Claims (1)

1. method of using the computing machine camera to measure small two-dimension displacement, adopt the logical computing machine of a Daepori to coordinate camera to measure the small two-dimension displacement that object occurs, this allocation of computer has USB interface, internal memory, CPU, hard disk, display card and display, keyboard and mouse, operating system, webcam driver program and camera are taken and the Digital Image Processing program, described camera is connected to described computing machine by its USB interface, it is characterized in that, the method take and pass through by camera Digital Image Processing obtain object occur in the optical axis of camera perpendicular plane on small two-dimension displacement vector velocity, comprise the steps:

1) operation computing machine camera is taken the image of a frame testee, preserves with the form of bitmap, as the reference frame; Select the big or small M * N of bitmap frame according to the parameter index of concrete camera, M, N ∈ positive integer, and select as far as possible shooting speed frame per second faster; For the pel array that described bitmap is corresponding is selected a rectangular coordinate system; Open up a comparison window at the middle section of described pel array, size is taken as m * n, and m, n ∈ integer are chosen suitable initial value: m ₀* n ₀, the horizontal direction of described comparison window and described pel array and the edge pixel of vertical direction be distance h and v pixel respectively, namely has: m+2h=M, n+2v=N, h, v ∈ positive integer;

2) calculate the self-correlation of the pel array of comparison window in described reference frame:

$\mathrm{auto}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{reference}(x+a,y+b)|$

In formula, x and y are respectively the coordinates of pixel in comparison window, the value of each pixel of comparison window in reference (x, y) expression reference frame, and a=-1,0,1, b=-1,0,1, common property is given birth to 9 self-correlation auto_correlation (a, b);

If these self-correlations have approximately equal more than half and are tending towards 0, the architectural feature on surface that subject is described is meticulous not, value between neighborhood pixels be can not distinguish, be unfavorable for adopting from now on frame-frame coupling comparative approach to measure displacement, need to enlarge the zone that is used for matching ratio pixel, make m=m ₀+ step, n=n ₀+ step, step ∈ positive integer, namely enlarge the capable and step row of each step of scope of described comparison window, recomputate the self-correlation of described comparison window, so repeat, increase gradually step, until find the coupling comparison window that is fit to this body surface texture characteristic, at this moment, 2h=M-m, 2v=N-n; If exceed certain scope of frame, also do not find suitable comparison window, think that this part reflecting surface of this object is unsuitable for surveying work, and provide the prompting warning;

If these self-correlation approximately equals and be tending towards 0 the number less than half, the architectural feature on surface that subject is described is enough meticulous, value between neighborhood pixels can be distinguished, and further attempts choosing less pel array, to reduce amount of calculation: make m=m ₀-step, n=n ₀-step, step ∈ positive integer, namely dwindle the capable and step row of each step of scope of described comparison window, recomputate the self-correlation of comparison window, so repeat, increase gradually step, until at this moment the present self-correlation approximately equal in zone and be tending towards 0 its half the number that is not less than has found under present body surface situation to be used for carrying out matching ratio best pixel array: m=m ₀-step, n=n ₀-step, 2h=M-m, 2v=N-n;

3) through taking the second framing bit figure after Δ t after a while, as sampling frame and preservation;

4) value of the pel array in comparison window in described reference frame is carried out cross-matched relatively according to 9 * 9 associate operator arrays in described sampling frame scope, specific algorithm is:

$\mathrm{cross}\_\mathrm{correlation}(a,b)=\underset{y=v+1}{\overset{v+1+n}{\mathrm{\Σ}}}\underset{x=h+1}{\overset{h+1+m}{\mathrm{\Σ}}}|\mathrm{reference}(x,y)-\mathrm{comparison}(x+a,y+b)|$

In formula, the value of each pixel of comparison window in the described reference frame of reference (x, y) expression, the value of the pixel of the position corresponding with the contingent situation of movement of described reference frame comparison window in the described sampling frame of comparison (x+a, y+b) expression, described reference frame has 81 kinds of contingent situation of movement, corresponding a=-4 ,-3 ,-2,-1,0 ,+1, + 2 ,+3 ,+4 and b=-4,-3,-2 ,-1,0, + 1, + 2 ,+3 ,+4 combination, common property is given birth to 81 cross correlation coefficient cross_correlation (a, b);

5) value is minimum and be tending towards 0 cross correlation coefficient cross_correlation (a, b) represent that its frame-frame correlation degree is the highest, therefore obtain direction and mobile amplitude that described sampling frame moves relative to described reference frame: Δ x=a, Δ y=b, this is the relative displacement vector that in this sample period, object occurs; In measuring process, the total relative displacement vector of object is the vector of the relative displacement of the relative displacement vector of the former accumulation of object described reference frame generation relative to described sampling frame;

6) according to above-mentioned displacement vector and take Δ t interval time between above-mentioned two continuous frames, obtain the velocity of ohject displacement

Δv _x＝Δx/Δt，Δv _y＝Δy/Δt；

7) if in described reference frame relatively windowsill X-direction or be equal to or greater than half of amplitude of this comparison window along the relative displacement that Y direction occurs, replace described reference frame with up-to-date sampling frame, its comparison window is repositioned at the central part of new reference frame; No, do not upgrade described reference frame, but the comparison window generation relative displacement Δ x=-a in described reference frame, Δ y=-b;

8) if upgraded described reference frame, according to step 2) calculate the self-correlation of the reference frame after described renewal, again watch object reflecting surface architectural feature, again determine the best big or small m * n of its comparison window;

9) the above-mentioned cross correlation operator array that comparison window is searched for coupling in described sampling frame scope in described reference frame is adjusted automatically according to this measured value: if | a|＜3﹠amp; | b|＜3, change to take 5 * 5 associate operator arrays, if 3≤| a|＜5﹠amp; 3≤| b|＜5, change to take 7 * 7 associate operator arrays, if 7≤| a|﹠amp; 7≤| b| still is taken as 9 * 9 associate operator arrays, to reduce amount of calculation;

10) take a new frame as the sampling frame and preserve; Take the frame of sampling frame-frame period time faster than or time of spending near pixel unit of object relative displacement;

11) jump to step 4), continue to calculate the cross correlation coefficient, obtain relative displacement vector and velocity.

Images