CN102022981B - Peak-valley motion detection method and device for measuring sub-pixel displacement - Google Patents

Abstract

The peak-to-valley motion detection method and device for measuring sub-pixel displacement are composed of an ordinary computer and its camera. The method for measuring sub-pixel displacement is to first shoot a frame, and select the most The observation row and observation column with the reflection characteristics of the measured object; then take a frame and determine the edge direction data and peaks and valleys of the three primary colors of the two frames. By comparison, if the mutual displacement cannot be determined in the observation row and observation column If the number of peaks or valleys of the relationship exceeds a certain predetermined percentage of the sum of the number of peaks and valleys of the primary color on its axis, it is considered "lost tracking", and another frame needs to be shot, otherwise, according to the The direction and number of peaks and valleys move, respectively calculate the sub-pixel displacement of each primary color in each axis, and take the average value as the total displacement; effectively overcome the influence of environmental light changes, suitable for both displacement and light change speed When the frame rate is much lower than the shooting frame rate.

Description

Peak and valley motion detection method and device for measuring sub-pixel displacement

technical field

The invention belongs to the technical field of digital image measurement, in particular to a method and a device for non-contact two-dimensional micro-displacement measurement of an object by using a computer camera.

Background technique

Computer cameras have gained popularity rapidly in recent years, and are generally used for network video chatting or surveillance shooting. The core is CCD and CMOS photoelectric imaging chips, that is, photoelectric sensor arrays. The core of the optical orientation device invented before it is also a photodetection pixel array, which is used to detect its two-dimensional movement relative to the surface on which it is located. Therefore, it is beneficial to investigate optical orientation devices for wide-open camera applications.

U.S. Pat. No. 5,288,993 discloses a cursor orienting device utilizing an array of photodetectors in which an illuminated target sphere must have randomly distributed markings. U.S. Pat. No. 5,703,356 further discloses an optical pointing device in the form of a mouse which does not require a target ball and which utilizes light reflected directly from the surface on which the pointing device is moved. The above-mentioned US patent extracts motion-related information based on the so-called "edge motion detection technique". Tracking and measurement whereby the total displacement representing the relative motion between the light detection array and the illuminated portion of the surface upon which it is determined is determined.

According to US Pat. No. 5,288,993, "edges" are defined between pairs of pixels arranged along a first axis and a second axis, respectively, of a photosensor array, which may or may not be orthogonal. Edges Ex and E _y in this patent are defined as being located in the middle of two nearest neighbor pixels. According to USPat.No.5,288,993 and USPat.No.5,703,356, on the one hand, calculate the difference between the number of sides Ex moving along the positive direction of the first axis and the number of sides Ex moving along the opposite direction of the first axis On the other hand, computing the normalized difference between the number of edges E _y moving in the positive direction of the second axis and the number of edges E _y moving in the opposite direction of the second axis The relative motion of the edges can be determined by comparing the positions of the edges in the photosensor array at two consecutive time points.

A typical optical pointing device includes a light source, such as an infrared LED, that intermittently illuminates a portion of the surface according to a determined sequence. The pixel signals of the photodetection array are sampled according to the determined sequence, providing two consecutive Frame the image of the edge to determine the motion of the array relative to the surface.

According to one approach of the above two US patents, the use of differential techniques is advantageous for determining the condition of an edge between two pixels. Its definition of an edge is: if the ratio of the intensities of two photosensitive units is greater than a certain value, there is an edge between these two pixels. The literature also gives a definition expression about the edge along the horizontal or vertical direction, and points out that the above conditions E _x and E _y do not depend on the direction or meaning of the edge, but only indicate whether there is a side.

The definition of "edge" and the motion detection algorithm described above have been successful in optical trackballs. Optical trackballs detect the light intensity pattern of an illuminated ball that is manipulated by the user. The above definitions and algorithms require that the detected light intensity patterns show clear and fine light intensity differences. Therefore, the surface of the ball is covered with a color random logo pattern to show the contrast with the background. Furthermore, these marks are mostly in the form of spots, which require a predetermined size. The spot size in USPat.No.5,288,993 is 0.5mm ² -0.7mm ² , and the density is about 1 spot/mm ² ; the size of these marks is relatively independent of the diameter of the ball, but depends mainly on the photodetector array. resolution and size. US Pat. No. 5,703,356 proposes that the image of a single spot on the sensor should at least cover the center-to-center distance (pixel pitch) between two adjacent pixels. In practice, a typical spot size is preferably chosen such that the surface of the image of the spot covers about 5 pixels. Under this condition, the number of edges remains sufficiently constant, sufficiently smaller than the number of pixels, to be determined and used to detect motion.

The above scheme for detecting motion applied to an optical orientation device presents a serious problem if the illuminated surface does not appear in a predetermined pattern; for example, a "ballless" mouse used directly from a random surface such as paper Or the surface of a table, reflecting incoming light. U.S.Pat.No. 5,703,356 describes an optical mouse that does not require any balls, but the solution requires that the illuminated surface exhibit a suitable pattern with a sufficient number of dark and bright areas of the size full.

In any case, for a featureless surface, when applying the above technique, the edges defined above can be detected between virtually every pixel. Therefore, without being able to clearly detect and track specific and well-defined edge patterns, it is impossible to derive any measurements about the relative motion between the optical detection device and the illuminated surface. Therefore, the above-mentioned motion detection technique cannot be practically applied.

In fact, U.S. Pat. No. 5,578,813 and U.S. Pat. No. 5,644,139 have disclosed some schemes for "generating a purely random pattern of light intensity on an illuminated surface". These two inventions propose the principle of motion detection based on the correlation of continuous image frames output by the photodetector array. However, their comparison of light intensities between adjacent pixels does not provide any information about spatial intensity differences (ie edges) between adjacent pixels, and the association of consecutive image frames proposed in the invention implies some limitations. In particular, in order to derive a sufficiently accurate measurement of relative motion, it is actually necessary to illuminate the reference surface at a very low angle relative to the surface, typically within 16° of the surface, U.S. Pat. No. 5,288,720 being more precise for this explained. This results in constraining the structure of the optical orientation device. Furthermore, the above image frame correlation techniques exhibit certain limitations when analyzing grayscale image frames, and the complexity of the correlation process increases exponentially as the bit length of the image intensity increases. In practice, the application of the aforementioned correlation techniques to motion detection in optical devices is limited to analyzing binary black-and-white images.

Referring to the above-mentioned US patent, the recently submitted invention patent "method and device for measuring small two-dimensional displacement using computer camera" (application number: 200910104277.8, application date: 2009.7.7) analyzed domestic patents related to "camera" and "measurement" The document proposes a method of using a computer camera as a photoelectric conversion sensor to measure tiny two-dimensional displacement vectors and velocity vectors through frame-frame correlation matching and comparison. However, it still matches light intensity patterns, so it also Uniform and stable lighting is required.

"Method and sensing device for motion detection in an optical pointing device, such as an optical mouse" (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.) provides two solutions that are more suitable for optical pointing devices. One is: preprocessing image frames to extract binary data describing the difference in light intensity between adjacent pixels (i.e., edges), directly tracking the direction of motion of positive and negative edges, and computing a displacement measure of relative motion, ie "Local Edge DirectionMotion Detection Algorithm (local edge direction motion detection algorithm)". The second method is: further extracting edge reflection conditions from the extracted edge direction data, which describes the continuity of positive and negative edges along an axis direction of the photodetector array, and then comparing the edge reflection conditions of two consecutive frames position, calculate the displacement measurement of the relative motion, namely "Peak/Null MotionDetection Algorithm (peak/valley motion detection algorithm)". The characteristics of the invention are: 1) The surface to be illuminated has no applied features and produces purely random light intensity patterns; it is easy to extract characteristic features and patterns that can be tracked and directly used for motion detection. 2) Shows great sensitivity to slight variations in intensity between pixels, allowing a wide range of illumination angles to be applied, rather than requiring a low illumination angle, to enhance U.S. Pat. No. 5,5788,813, U.S. Contrast of surface patterns in related techniques used in Pat. No. 5,644,139 and U.S. Pat. No. 5,686,720. 3) Grayscale images can be analyzed, and the required processing appears particularly dynamic and simple without increasing the complexity of the processing.

One of the algorithms for calculating the total displacement measurement value of the patent's so-called "Pcak/Null Motion Detection Algorithm" is as follows:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; RIGHT</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; LEFT</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; XPEAK</span>}}}$

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; RIGHT</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; LEFT</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; XNULL</span>}}}$

${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; UP</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DOWN</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; YPEAK</span>}}}$

${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; UP</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DOWN</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; YNULL</span>}}}$

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="DISPLACEMENT"\; filenames="H2009101909241E00012.png"\; state="\{\{state\}\}">DISPLACEMENT</figure-callout></span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; PIXEL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; PIXEL</span>}}<span\; class="notranslate">\; ;</span>$

${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="DISPLACEMENT"\; filenames="H2009101909241E00012.png"\; state="\{\{state\}\}">DISPLACEMENT</figure-callout></span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; PIXEL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; PLXEL</span>}}$

The second peak/valley motion detection algorithm in which the patent calculates the total displacement measurement is:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; RIGHT</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; RIGHT</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; LEFT</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; LEFT</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; :</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; XPEAK</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; XNULL</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; PIXEL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; PIXEL</span>}}$

${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; DISPLACEMENT</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; UP</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; UP</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; PEAK</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DOWN</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; NULL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; DOWN</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; :</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; YPEAK</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; YNULL</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; PIXEL</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; PIXEL</span>}}$

In the formula, X, Y and N represent the two coordinate axes and the number (Number) respectively, and the meanings of each item are shown in words or symbols, for example, N _XPEAK and N _PEAK-RIGHT respectively represent the number of peaks in the X-axis direction and their The number of peaks that have shifted to the right, L _PIXEL-PIXEL represents the pixel pitch, and its unit can be μm, mm, or inch.

The above two peak/valley motion detection algorithms are not equivalent, however, the total number of peaks and valleys is the same for both, so the total displacement measurement is a fraction of the pixel spacing L _PIXEL-PIXEL , and the calculation results of the two Close, all belong to sub-pixel level displacement. Also, the second method of computing displacement requires only one division per axis.

The above-mentioned peak-valley motion detection algorithm is based on extracting the features of the image frame, and overcomes the difficulty brought by the random change of the light intensity pattern to the frame-frame matching. Optically reflective texture requirements for moving surfaces.

In addition, the above-described peak-to-valley motion detection algorithm has other advantages. First, the number of edge reflections that the detector "sees" depends on the focus state of the sensor. Second, edge reflections are assumed to move across the pixel array, and these edge reflections do not move by more than one unit of pixel pitch during two consecutive exposures. For a given motion and its maximum motion velocity, this condition is easy to satisfy. If an edge reflection moves beyond one unit of pixel pitch, the motion detection scheme described above cannot determine where the edge reflection came from. This occurs if the detector moves too quickly relative to the illuminated surface between two consecutive exposures, and represents a "loss-off-track" situation. This "lost-of-tracking" situation can be detected by looking at the number of edge reflections that don't know where they came from, where the reflections from the previous state (frame) were at or one pixel apart from the current position There are no similar edge reflection conditions in range, these edge reflection conditions are defined as "ghost edge reflection conditions (ghost edge reflection conditions) (ghost edge reflection conditions)", they can potentially occur along the direction of the two coordinate axes, which along the direction of each coordinate axis The number can be detected and tracked by two accumulators respectively, and compared with a predetermined threshold value, thereby determining the "lost track" situation, and reported to the external operator as a warning signal. Edge reflections on the sides of the photocell do not count as "ghost side reflection conditions"; while they appear to be coming from nowhere, they may be coming from outside the photocell.

The above-mentioned method of detecting and judging "lost tracking" is practical and can be realized by algorithm. Of course, other methods of detecting and judging "lost tracking" can also be found. However, after careful analysis, it will be found that the above-mentioned "peak/valley motion detection algorithm" and "local edge direction motion detection algorithm" in this patent lack calculation basis and are difficult to understand.

In summary, the US patent technique for detecting relative motion in an optical orientation device inspired the idea of using computer cameras to measure small displacements occurring on surfaces of random conditions. The distance between the photoelectric sensor array in the optical orientation device is extremely small (on the order of millimeters), and it uses a lighting source such as a monochromatic LED. The surface area is relatively small, the illumination is uniform, and the reflected light beam will not be affected by ambient light radiation, which better reflects the optical reflection properties of the surface. However, when using a computer camera for displacement measurement, the photoelectric sensor array is far away from the measured object, and the optical reflection properties of the surface of the measured object will have a greater impact. For example, due to the short frequency of visible light, the reflected light of an object that is photoelectrically converted by a photoelectric sensor will be an average value of several reflections; with the different illumination angles, the diffuse reflection of a rough surface will be very strong, and the change will also vary. will be larger; the reflected light produced by strong reflective surfaces such as metals and flat mirrors will change strongly with the change of the incident light angle, and even the images of other surrounding objects will be reflected, which will be blurred , confuse the texture characteristics of the reflective surface itself; the incident sunlight will be rich in polarized light after being reflected by the object; the influence of radiation from other light sources in the measurement environment or floating dust or smoke will also cause image frames on the surface of the measured object Overall or local light intensity changes, or random fluctuations in the light intensity data of the grayscale image. These raise new questions for the application of "peak/valley motion detection sub-pixel displacement algorithm" to measure small displacements of random surfaces.

Contents of the invention

In order to give full play to the function of the photoelectric sensor array of the computer camera, the present invention provides a method and device for detecting peak-to-valley motions that can adapt to relatively small changes in the lighting environment and measure sub-pixel displacement. It uses a computer camera as a photoelectric conversion sensor. The object to be measured is photographed accurately, and then processed by the peak/valley sub-pixel displacement detection algorithm to obtain the tiny two-dimensional displacement vector and velocity vector of the object on a plane perpendicular to the optical axis of the camera.

The technical solution adopted by the present invention to solve its technical problems is: the object to be measured is located within the effective focus imaging range in front of the optical lens of the computer camera, the lighting environment of the object to be measured is relatively uniform and stable, and the camera is connected to a computer via a USB interface. An ordinary computer, the computer is equipped with USB interface, memory, CPU, hard disk, display card and monitor, keyboard and mouse, operating system, camera driver and camera shooting and peak/valley sub-pixel displacement detection program, the program includes the following step:

Step 1. Take a frame (M rows×N columns) of the measured object, regardless of the two rows and two columns on the side of the image frame, and obtain a reference frame [(M-2) rows×(N-2) columns] , determine the same coordinate axis for the reference frame and all subsequent image frames, wherein, M, N∈ positive integer;

Step 2. For the pixel array of the above reference frame, according to the data of the red, green and blue components, export the red, green and blue edge direction data along the X-axis and Y-axis direction row by row and column by column respectively ;

Step 3. Calculate the sum of the numbers of positive and negative sides of red, green and blue respectively row by row and column by column, and take the row and column where the sum of the number of positive and negative sides is the largest as the observation rows and columns of observations;

Step 4. According to the edge direction data of the three primary colors of the pixels in the selected observation row and observation column, respectively derive the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the accumulator to count their corresponding Number of peaks and valleys: N _{RX axis peak} (j), N _{RX axis valley} (j), N _{RY axis peak} (j) and N _{RY axis valley} (j), N _{GX axis peak} (j), N _{GY axis valley} (j), N _{GY axis peak} (j) and N _{GY axis valley} (j), N _{BX axis peak} (j), N _{BX axis valley} (j), N _{BY axis peak} (j) and N _{BY axis valley} (j ), wherein j represents the sequential count value of the frame obtained by shooting, j∈positive integer, N (umber) represents the number, and R, G, B represent red, green and blue respectively;

Step 5, after photographing the reference frame, after a certain intermittent time dt, photograph the image (M×N) of the second frame of the measured object, remove two rows and two columns at the edge of the pixel frame to obtain a comparison frame [(M- 2) × (N-2)]; along the observation row or observation column selected in step 3, respectively derive the edge direction data and edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the cumulative The counter counts the number of peaks and valleys along the X and Y axes: N _{RX axis peak} (j+1), N _{RX axis valley} (j+1), N _{RY axis peak} (j+1), and N _{RY axis Valley} (j+1), N _{GX axis peak} (j+1), N _{GX axis valley} (j+1), N _{GY axis peak} (j+1) and N _{GY axis valley} (j+1), N _{BX axis Peak} (j+1), N _{BX axis valley} (j+1), N _{BY axis peak} (j+1) and N _{BY axis valley} (j+1);

Step 6. Comparing the positions of the edge reflection conditions of the three primary colors in the selected observation row and observation column in the reference frame and the comparison frame, and checking those "ghost edge reflection conditions" from where they do not know, That is: the peaks or troughs found in the selected observation row or observation column of the reference frame are missing within a pixel pitch range near the corresponding position of the selected observation row or observation column of the comparison frame, Vice versa, at this time, use six accumulators to count and track their numbers respectively: N _{RX ghost edge} (j+1), N _{RY ghost edge} (j+1), N _{GX ghost edge} (j+ 1), N _{GY ghost edge} (j+1), N _{BX ghost edge} (j+1), N _{BY ghost edge} (j+1);

If the "ghost edge reflection condition" of two or more of the three primary colors satisfies:

or

Then it is considered that the measurement is "lost tracking" and a warning signal is issued. In the formula, the error tolerance value error1 represents the ghost of one of the three primary colors (Red, Green, Blue) along the selected observation row or observation column in the comparison frame The percentage of the number of shadow edge reflections taking place in the total number of edge reflections of the base color can be preset according to the optical properties of the object surface and the measurement environment, for example, it is preset as: error1=5%; on the edge of the comparison frame The edge reflections of , do not consider the "ghost edge reflections", which may come from the outside of the photopixel array;

Step 7. If the measurement "lost tracking", go back to step 1 and restart the measurement work;

Step 8. If there is no "lost tracking" in the measurement, continue the measurement work: along the selected observation row and the selected observation column, according to the three primary color component data of the camera, use the peak/valley sub-pixel displacement detection algorithm to calculate respectively Subpixel displacement of three primary color components:

In this measurement, the total sub-pixel displacement calculation formula is:

Step 9. In this measurement, the velocity vector calculation formula of the object is:

Step 10. Take the comparison frame in step 5 as a new reference frame, take a certain interval time dt from the shooting of this frame, and re-shoot a frame as a new comparison frame, that is, skip to step 5 and continue the measurement.

Wherein, the definition of the red, green or blue edge direction data in the second step of the sub-pixel displacement program of the camera shooting and peak/valley motion detection is:

According to the component data of one of the three primary colors of red, green or blue in the pixel array, along the X-axis or along the Y-axis direction, if a pixel has a value of a certain three-primary color component that is greater than that of the second pixel behind it The value of the three primary color components is smaller by an error tolerance value error, that is, if

I(X, Y) _red < I _red (X+2, Y)-error or I(X, Y) _red < I(X, Y+2) _red -error

I(X,Y) _Green < _IGreen (X+2,Y)-error or I(X,Y) _Green <I(X,Y+2) _Green- error

I(X, Y) _blue < I _blue (X+2, Y)-error or I(X, Y) _blue < I(X, Y+2) _blue- error

It is defined that there is a red, green or blue positive edge between these two pixels; if the value of a certain three-primary color component of a pixel is one larger than the corresponding three-primary color component value of the second pixel behind it The error tolerance value error, that is, if

I(X,Y) _red >I _red (X+2,Y)+error or I(X,Y) _red >I(X,Y+2) _red +error

I(X, Y) _green > I _green (X+2, Y) + error or I (X, Y) _green > I (X, Y + 2) _green + error

I (X, Y) _blue > I _blue (X+2, Y) + error or I (X, Y) _blue > I (X, Y + 2) _blue + error

Then it is defined that there is a red, green or blue negative edge between these two pixels; the edge thus obtained is located at the position of the first pixel after this pixel, that is, at the middle position of the two pixels participating in the comparison on that pixel; if the value of a certain three-primary color component of a pixel is close to the corresponding three-primary color component value of the second pixel behind it, the difference between its RGB component values does not exceed an error tolerance value error, that is, if

I(X+2, Y) _red -error<I(X, Y) _red <I(X+2, Y) _red +error or I(X, Y+2) _red -error<I(X, Y) _Red <I(X, Y+2) _red +error;

I(X+2,Y) _green -error<I(X,Y) _green <I(X+2,Y) _green +error or I(X,Y+2) _green -error<I(X,Y) _Green <I(X, Y+2) _Green +error;

I(X+2, Y) _blue -error<I(X,Y) _blue <I(X+2,Y) _blue +error or I(X,Y+2) _blue- error<I(X,Y) _Blue <I(X, Y+2) _blue +error;

It is considered that there is no "edge" corresponding to the color wavelength between the two pixels, or the edge of the third type of color; along a certain coordinate axis, all red positive edges and red negative edges And the third type of red edge constitutes the red edge direction data of this direction, all the green positive edges and green negative edges and the third type of green edge form the green edge direction data of this direction, all the blue positive edges The blue negative edge and the third type of blue edge form the blue edge direction data in this direction; the error tolerance value in the above formula can be preset to a small value according to the specific lighting situation, for example: error =10; the pixel positions on the four sides and corners in the pixel array do not have side direction data.

The edge reflection conditions of the three primary colors along the X-axis and Y-axis directions in the step 4 and step 5 of the camera shooting and peak/valley motion detection sub-pixel displacement program are defined as:

According to the red, green or blue side direction data of the pixels in the selected observation row and observation column, along the X axis or along the Y axis direction, if there are two or more consecutive positive sides of a certain primary color , or two or more consecutive third-type edges of the base color followed by a negative edge of the base color, which is called the first-type edge reflection condition of the base color, that is, it is considered that there is a base color at this position peak; if two or more consecutive negative sides of a certain basic color, or two or more consecutive third-type sides of this basic color are followed by a positive side of this basic color, it is called It is the second type of edge reflection condition of the base color, that is, it is considered that there is a valley of the base color at this position.

The peak/valley sub-pixel displacement detection algorithm described in the step eight of the camera shooting and peak/valley motion detection sub-pixel displacement procedure includes:

For a certain trichromatic wavelength, compare the positions of peaks and valleys corresponding to the selected observation row and observation column in the reference frame and the comparison frame, thereby judging the corresponding peaks in the observation row and observation column , the direction of movement of the valley, and use the accumulator count to track the movement of the peak and valley along the coordinate axis, specifically,

In the observed row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then tracking the number of the peaks of the primary color shifted to the right The N _{peak of the accumulator is shifted to the right by} +1. If the position of a certain three primary color peaks in the comparison frame is displaced to the left by one pixel pitch unit relative to its position in the reference frame, the number of the primary color peaks shifted to the left is tracked. The N _{peak of the accumulator is shifted to the left by} +1; if the position of a valley of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted to the right The accumulator N of the number of _{valleys is moved to the right} + 1, if the position of a valley of a certain three primary colors in the comparison frame is shifted to the left by one pixel pitch unit relative to its position in the reference frame, then the valley of the primary color is tracked The accumulator N _{valley of the number shifted to the left is shifted to the left} + 1;

In the observation row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is displaced upward by one pixel pitch unit relative to its position in the reference frame, the accumulation of the number of upward displacements of the peaks of the primary color is tracked The N _{peak of the detector moves up} +1, if the position of a peak of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the number of the peaks of the primary color shifted downward is tracked The N _{peak of the accumulator moves down by} +1; if the position of a valley of a certain three primary colors in the comparison frame is displaced upwards by one pixel pitch unit relative to its position in the reference frame, then tracking the valleys of the primary color is shifted upward by 1 The accumulator N _{valley of the number moves up} +1, if the position of a valley of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted downward The number of the accumulator N _{valley moves down} +1;

If N _{peak shifts to the right} (j+1)>N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=+0.5,

If N _{peak shifts to the right} (j+1)<N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=-0.5;

If the N _{valley moves to the right} (j+1)>N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=+0.5,

If N _{valley moves to the right} (j+1)<N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=-0.5;

Similarly,

If the N _{peak moves up} (j+1)>N _{peak moves down} (j+1), it is calculated as ΔY _{peak displacement} (j+1)=+0.5,

If the N _{peak moves up} (j+1)<N _{peak moves down} (j+1), it is calculated as ΔY _{peak displacement} (j+1)=-0.5,

If N _{valley moves up} (j+1)>N _{valley moves down} (j+1), it is counted as ΔY _{valley displacement} (j+1)=+0.5,

If N _{valley moves up} (j+1)<N _{valley moves down} (j+1), it is counted as ΔY _{valley displacement} (j+1)=-0.5,

In the above formula, L _pixel-pixel represents the pixel pitch, and its unit can be μm, mm or inch, etc. The meanings of other symbols are as marked by characters, for example, N is the number (Number), and the above calculation formulas should be applied to the camera respectively A specific three primary color component data of the pixel array.

The advantage of the present invention is, this method uses the photoelectric sensor array in the ordinary computer camera head as displacement sensor, according to the RGB data of image frame pixel, respectively calculates three primary colors and describes the difference of light intensity between adjacent pixels. Direction information, which uses "peaks" and "valleys", that is, the distribution characteristics of the physical light and dark contrast of the three primary colors in the reflected light pattern on the surface of the measured object, as the basis for measuring sub-pixel displacement, because the wavelengths of the three primary colors are simultaneously Measurement and calculation are carried out separately, supplemented by an average algorithm, which overcomes the influence of other wavelengths by environmental light changes, and reduces the measurement error to a considerable extent. The displacement measurement error corresponding to the obtained calculation results is ≤±0.25L _{pixels -pixel} to ±0.5L _pixel-pixel ; at the same time, the invention only takes the row and column with the largest contrast change of the image frame on the reflective surface as the representative of the measurement, which not only utilizes the photoelectric sensor array, but also simplifies the calculation workload and ensures two-dimensional Displacement measurement and its measurement sensitivity. The invention utilizes the photoelectric conversion fast response characteristic of the photoelectric sensor device, and is suitable for the situation that both the moving speed of the object and the light changing speed are far lower than the shooting frame rate of the camera.

Description of drawings

Further illustrate the present invention below in conjunction with accompanying drawing.

Fig. 1 is a block diagram of the measuring device of the present invention.

Figure 2 shows a 3 × 3 pixel array and its derived edge orientation data by definition. In the figure, a box represents a pixel, and the larger the number of points in the box, the darker the brightness of the pixel. The determined positive and negative sides are represented by thick line segments, and the four peripheral sides of the pixel array The positive and negative directions of are also indicated by arrows, respectively.

Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain axis and a sequence of side direction data (positive side and negative side) and side reflection conditions (peak and valley) derived therefrom.

FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis.

Figure 5 is a diagram of a peak/valley sub-pixel displacement detection algorithm. Among them, the upward arrow indicates the positive edge, and the downward arrow indicates the negative edge. Both the dotted line and the thin solid line indicate the uniform bisector on the coordinate plane, and the equal division interval of the abscissa indicates the separated range of each pixel.

In Fig. 1, 1. computer camera, 2. optical lens, 3. photoelectric imaging chip, 4. USB interface of camera, 5. computer system, 6. USB interface of computer, 7. CPU, 8. memory, 9. display Cards and monitors, 10. Hard drives, 11. Keyboards and mice, 12. Operating systems, 13. Camera drivers, 14. Sub-pixel displacement programs for camera capture and peak/valley motion detection, 15. Lighting.

Among Fig. 3, 21. one row of image light-electric signal that photoelectric sensor array detects; 22. one row light that photoelectric sensor array detects-electrical signal 21 is through the digitized output signal after integrator peak value circuit processing; 23. and The edge direction data corresponding to a line of output signal 22 of the photoelectric sensor array, the upward arrow indicates a positive edge, and the downward arrow indicates a negative edge; 24. The edge reflection condition corresponding to a line of edge direction data 23 includes two peaks and a valley.

In Fig. 4, 31. a row of edge direction data in the reference frame of the previous shooting; 32. the edge reflection condition corresponding to a row of edge direction data 31 in the reference frame of the previous shooting, including two peaks and a valley; 41. after One line of edge direction data in the comparison frame of one shot; 42. The edge reflection condition corresponding to one line of edge direction data 41 in the comparison frame of the next shot, including two peaks and one valley; the upward arrow indicates the positive edge, and the downward arrow indicates negative side.

The above Figure 2, Figure 3 and Figure 4 are all borrowed from reference materials (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.).

In Fig. 5, 51. a line of image light-electric signal detected by the photoelectric sensor array includes three situations of one line of image light-electric signal 510, one line of image light-electric signal 511 and one line of image light-electric signal 512 ; 52. One line of image light--electrical signal 510 is processed by the integrator peak circuit and the digitized output signal and its corresponding positive side, negative side, peak and valley; 53.～57. One line of image light-electricity Signal 510 gradually undergoes a sub-pixel shift to the right, and the digital output signal after processing by the integrator peak circuit and its corresponding positive side, negative side, peak and valley are shown.

Detailed ways

Fig. 1 is a block diagram of the measuring device of the present invention. Described camera head (1) is made up of optical lens (2), photoelectric imaging chip (3) and the USB interface (4) of camera head, is connected to the USB interface (5) of a common computer system (5) by its configured USB cable 6), the computer system (5) is also equipped with CPU (7), internal memory (8), display card and monitor (9), hard disk (10), keyboard and mouse (11), operating system (12), camera driver Program (13) and sub-pixel displacement program (14) for camera capture and peak/valley motion detection.

First, run the camera driver (13) distributed with the camera (1) on the computer (5), and install the camera (1) to the computer (5). The focal length of the optical lens (2) of the camera (1) is adjusted so that the image of the measured object is clear.

Then, the measurement environment is selected, or the related lighting equipment (15) is adjusted, so that the illumination on the measured object is uniform and does not change greatly as far as possible. For example, the measurement can be carried out in an indoor environment, or the occasion where the lighting source is stronger than the ambient light can also be selected, and the lighting source can also be cast on a ring-shaped wall curtain in front of the object, and then the diffusely reflected light on the wall curtain can shine on the measured object. objects, trichroic polarizers can also be applied.

Then, run the camera shooting and peak/valley motion detection sub-pixel displacement program (14), and use the peak/valley sub-pixel displacement detection algorithm to implement real-time measurement. This is explained item by item as follows.

(1) Define the edge direction data of the photoelectric pixel array

The pixel position relationship for light intensity comparison is: 1) between two adjacent pixels, 2) between adjacent pixel sub-arrays (U.S.patent application Ser.No.10/001,963, Dec.5, 2001; U.S.patent application Ser. No.10/001,959, Dec.5, 2001), 3) between non-adjacent adjacent pixels (there is a third pixel between the two pixels participating in the comparison), 4) between non-adjacent adjacent pixels in other positions.

When the light intensities of the two pixels involved in the comparison are equal, the so-called "hysteresis function" (U.S. patent application Ser. No. 10/001,963) can be used to avoid the uncertainty of the edge direction data, or simply defined as the positive edge or negative side (US 7,122,781 B2). The pixel positions on the four sides and the four corners in the pixel array do not have side direction data.

U.S. Patent No. 7,122,781 B2 uses the method of comparing the light intensity between non-adjacent adjacent pixels to determine the edge direction data. The edge obtained in this way is located on the pixel in the middle of the two pixels participating in the comparison. It is believed that there are two effects: 1) It is equivalent to implementing a spatial medium filter on the light intensity map of a pixel to reduce noise; 2) comparing non-adjacent adjacent pixels is equivalent to comparing three adjacent pixels: P _i -P _i+2 =(P _i +P _{i +1} )-(P _i+1 +P _i+2 ), without setting up a summation circuit. An example is shown in Figure 2, which represents a 3 × 3 pixel array with edge orientation data derived from the above definition. For a given photodetector array, there is a predetermined number of edge direction data.

The present invention adopts the above-mentioned definition of comparing the light intensity of non-adjacent adjacent pixels, and thinks that the reflection feature of the surface of the measured object is the so-called surface physical light-dark contrast, which belongs to the optical reflection property of the surface material of the object. There will be no significant change in degree.

Considering that the computer camera is quite far away from the measured object, the light intensity pattern reflected by the surface of the measured object detected by the photoelectric sensor array will be affected by the ambient light radiation, which will bring some random fluctuations in the pixel light intensity value, so this paper The invention adds an error tolerance value error in the definition of the side, and thinks that when the strength of the two is similar, for example, I(X, Y+2)-error<I(X, Y)<I(X , Y+2)+error, it is defined that there is no edge between these two pixels, or in other words, there is a third type of edge, which does not affect the analysis of peaks and valleys described later, but further reflects the side Reflective features of the surface of an object. The gray value depth of the camera is generally 8bit, 256 levels. Therefore, the error tolerance value in the formula can be preset as a small value, for example, error=10, according to the specific situation that the light is disturbed.

In order to further effectively eliminate the influence of ambient light, the present invention uses the photoelectric conversion data components of the three primary colors (wavelengths) of the camera itself, rather than its comprehensive light intensity value, to derive the three primary colors of red, green and blue by comparison. edge direction data. For a specific definition of edge direction data, see "Summary of the Invention".

The invention firstly detects the optical reflection characteristics of the surface of the object to be measured, and selects a row and a column with the largest number and the largest number of positive and negative sides of the three primary colors along the direction of the coordinate axis as the object of observation.

(2) Define edge reflection conditions

According to the edge direction data of a selected row or column of pixels, along the X-axis and Y-axis directions, the edge reflection conditions are respectively derived: two or more consecutive positive edges or a third type of edge followed by a negative edge, It is called the first type of edge reflection condition, that is, it is considered that there is a peak of light intensity at this position; two or more consecutive negative sides or the third type of side followed by a positive side are called the second In the case of edge reflection, it is considered that there is a valley of light intensity at this position; otherwise, it is called the third type of edge reflection. Therefore, the edge reflection conditions of the three primary colors (wavelengths) of red, green and blue are respectively obtained.

(3) Tracking the movement of peaks and valleys under stable light conditions

The so-called stable light means that the physical contrast between light and dark on the surface of the measured object will not change with the intensity of light or the change of angle.

Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain coordinate axis and a sequence of edge direction data (positive edge and negative edge) and edge reflection conditions (peaks and valleys) derived therefrom.

FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis. It can be seen from the figure that the valley obtained in the first shooting is located between the fourth (negative) side and the fifth (positive) side, while the valley obtained in the second shooting is located between the fifth (negative) side and Between the sixth (positive) side, during the two sequential shots, the valley moved to the right by a unit pixel pitch. The literature calls this situation a typical sub-pixel motion (sub-pixel motion: displacement of less than the pixelpitch between two successive flashes), which means that the displacement is less than one unit of pixel pitch. That is to say, within the time interval dt from shooting the reference frame to shooting the comparison frame, the measured displacement of the imaging pixel array of the moving object should be less than one pixel pitch unit. This can be achieved by adjusting the recording time interval dt according to the maximum velocity of the object being measured.

By tracking only edge reflection conditions regardless of their type, a loss of information and measurement accuracy occurs during the tracking process.

Therefore, 2×2 accumulators are set corresponding to each coordinate axis, and the number of peaks and valleys moving towards the positive or negative coordinate axis direction is tracked respectively along the direction of the coordinate axis. The method is: compare the reference frame The positions of peaks and valleys corresponding to the selected pixel rows and columns in the frame are compared with the corresponding peaks and valleys, so as to determine the moving direction of the corresponding peaks and valleys in the selected pixel rows and columns. Two accumulators are respectively set along each coordinate axis to count the respective total numbers of peaks and valleys on each coordinate axis. See "Content of the Invention" for details.

The displacement follows the principle of vector composition. As a matrix of pixels, it is also possible to detect movement of edge reflections along the diagonal directions of the X-axis and Y-axis. Thus, a corresponding edge reflection condition can originate from: 1) the two nearest neighbors left and right on the coaxial line, and 2) one of the four nearest neighbors on the diagonal.

(4) Peak/valley sub-pixel displacement detection algorithm

In FIG. 3 , the light intensity pattern 21 detected by the photoelectric sensor array along a certain coordinate axis belongs to the surface reflection property of the measured object, and the light intensity pattern will not change under the condition of constant illumination. When the surface of the measured object is displaced relative to the camera, the light intensity pattern will move to the left or right accordingly. Assuming that the measured object only shifts in one direction once during the shooting time interval, and the displacement is less than 1 pixel pitch, the situation shown in Figure 4 will appear, and the peaks and valleys only move in one direction.

FIG. 5 further examines the situation that a line of image light-electric signal 510 detected by the photoelectric sensor array gradually shifts to the right. This line of image light--electrical signal 510 is processed by the digitized output signal and its corresponding positive side, negative side, peak and valley after the integrator peak circuit 510 shows the initial situation. Signal 510 gradually undergoes sub-pixel displacement to the right, and the digitized output signal after processing by the integrator peak circuit and its corresponding positive side, negative side, and peak and valley diagrams 53-57 represent the subsequent sub-pixel displacement respectively. . As can be seen from the figure, when a displacement of half a pixel size takes place, the negative edge on the right side of its peak (label 54-D--E in Figure 5) becomes the third type of edge (label 54-- in Figure 5). -E), but the position of the peak does not change; when a displacement greater than half a pixel size occurs, the negative edge of the right neighbor of the peak passes through the third type of edge (label 54--E in Figure 5 The change process of the peak) changes into a positive side (mark 55--E in Figure 5), and the position of the peak also moves to the right by one pixel unit. The same conclusion can be reached when examining the asymmetrical image light-electric signal 511 or 512 of a row when it gradually shifts to the right. The sub-pixel displacement that occurs at this time can be counted as 0.5 pixcel, and the measurement error is between ±0.25 pixcel and ±0.5 pixcel.

Since the measured object is a rigid solid, when the photopixel array is displaced relative to the measured surface, all peaks and all valleys should move in the same direction. In fact, although the displacement of the measured object during the shooting interval dt is less than 1 pixel pitch, it may have multiple displacements both to the left and to the right. Therefore, the peaks and valleys move to the left, and There is a move to the right. However, taken together, the result should still be that all peaks and all valleys move in the same direction. If the peaks (or valleys) have both leftward and rightward cumulative numbers, the two should be very different, and the smaller number reflects the impact of illumination changes on the reflection contrast characteristics of the measured surface. therefore,

If N _{peak shifts to the right} (j+1)>N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=+0.5,

If N _{peak shifts to the right} (j+1)<N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=-0.5;

If the N _{valley moves to the right} (j+1)>N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=+0.5,

If N _{valley moves to the right} (j+1)<N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=-0.5;

A similar conclusion can be drawn for the Y-axis direction.

(5) Tracking the movement of peaks and valleys when the light changes

In Figure 3, if the illumination changes, the light intensity pattern 21 detected by the photoelectric sensor array along a certain coordinate axis no longer only belongs to the surface reflection properties of the measured object, but also reflects the impact of the change in ambient light radiation .

One situation is that although the light intensity pattern detected by the photoelectric sensor array has changed, the reflective properties of the surface of the measured object still dominate, and the sensor takes two shots of the edge in the direction of the coordinate axis that it "sees". The orientation data and the number of edge reflection conditions did not change significantly.

Another situation is that the overall illumination of the measured object changes, or its local surface is suddenly strongly changed by ambient light, resulting in a noteworthy change in the light intensity pattern 21 detected by the photoelectric sensor array in Figure 3, the sensor The edge direction data of the axis direction and the number of edge reflection conditions "viewed" by two consecutive shots will change obviously. At this time, restart the measurement, including reselecting the observation row and observation column. Since the shooting speed of the camera is much faster than the displacement speed of the measured object, it is also much faster than the irradiance change speed of the ambient light, and the peak/valley sub-pixel displacement detection algorithm only targets one row and one column of pixels in the photosensor array , the calculation speed is relatively fast, as long as the optical reflection properties of the surface of the measured object still dominate the imaging frame, the above-mentioned peak/valley sub-pixel displacement detection algorithm expression can still be used to calculate the real-time displacement value. The measurement results will not bring much error.

The above two variations can be checked together, either to detect the number of edge reflection conditions, or to detect the number of "ghost edge reflection conditions" described in the literature. It is more convenient and flexible to use the "preset threshold" of "ghost edge reflection" described in the literature to check than to use its percentage relative to the number of peaks and valleys in this axis.

The optical reflection properties of the surface of the measured object are required to take a dominant position in the imaging frame, which means that the peak/valley sub-pixel displacement detection algorithm can identify the characteristics of the measured object, which is the key of the present invention; Optical properties (material texture, texture, etc.), lighting source and measurement environment (interfering light source, dust, smoke) and other factors are related. Better measurement results can be obtained if a material whose optical reflection property of the surface texture is easier to distinguish is selected as the target, or if the light of the measured object is controlled to avoid excessive interference from the environment.

The output of the camera includes the component values of the three primary colors. Therefore, measuring the edge reflections of the three primary colors and averaging them can overcome the influence of ambient stray light changes of other wavelengths on the measurement.

In fact, it was noticed that the optical mouse with laser diode illumination was able to adapt to the smooth tile surface compared to the optical mouse with infrared diode illumination. If a light source with a wavelength that is not easily affected by environmental stray light sources can be used, and a photoelectric imaging sensor array (camera) sensitive to this wavelength can be selected, the present invention can fully exert its ability to adapt to more complex lighting environments, and achieve non-contact measurement. Excellent effect of small displacement of objects.

Claims (4)

1. a kind of peak-valley motion detection method of measuring sub-pixel displacement, it measures displacement by a common computer, a computer camera, and described camera is connected to described computer by USB interface, and this computer is equipped with internal memory, CPU, hard disk , display card and monitor, keyboard and mouse, operating system and camera driver, it is characterized in that, the method is taken by camera and adopts peak/valley sub-pixel displacement detection algorithm to measure sub-pixel displacement, comprising the following measurement steps: Step 1. Take a frame of the measured object: M rows×N columns, regardless of the two rows and two columns on the side of the image frame, and obtain a reference frame: (M-2) rows×(N-2) columns, which is The reference frame and all image frames captured thereafter determine the same coordinate axis, where M, N∈positive integers; Step 2. For the pixel array of the above reference frame, according to the data of the red, green and blue components, export the red, green and blue edge direction data along the X-axis and Y-axis direction row by row and column by column respectively ; Step 3. Calculate the sum of the numbers of positive and negative sides of red, green and blue respectively row by row and column by column, and take the row and column where the sum of the number of positive and negative sides is the largest as the observation rows and columns of observations; Step 4. According to the edge direction data of the three primary colors of the pixels in the selected observation row and observation column, respectively derive the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the accumulator to count their corresponding Number of peaks and valleys: N _{RX axis peak} (j), N _{RX axis valley} (j), N _{RY axis peak} (j) and N _{RY axis valley} (j), N _{GX axis peak} (j), N _{GX axis valley} (j), N _{GY axis peak} (j) and N _{GY axis valley} (j), N _{BX axis peak} (j), N _{BX axis valley} (j), N _{BY axis peak} (j) and N _{BY axis valley} (j ), wherein, j represents the sequence count value of the frames obtained by shooting, j ∈ positive integer, N represents the number, and R, G, B represent red, green and blue respectively; Step 5, after photographing the reference frame, after a certain intermittent time dt, photograph the image of the second frame of the measured object: M rows×N columns, remove two rows and two columns at the edge of the pixel frame to obtain a comparison frame: (M -2) row × (N-2) column; along the observation row or observation column selected in step 3, respectively derive the edge direction data and edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and respectively Use accumulators to count the number of peaks and valleys along the X and Y axes: N _{RX axis peak} (j+1), N _{RX axis valley} (j+1), N _{RY axis peak} (j+1), and N _{RY axis valley} (j+1), N _{GX axis peak} (j+1), N _{GX axis valley} (j+1), N _{GY axis peak} (j+1) and N _{GY axis valley} (j+1), N _{BX axis peak} (j+1), N _{BX axis valley} (j+1), N _{BY axis peak} (j+1) and N _{BY axis valley} (j+1); Step 6. Compare the positions of the edge reflection conditions of the three primary colors in the selected observation row and observation column in the reference frame and the comparison frame, and check those "ghost edge reflection conditions" that do not know where they come from- -- these peaks or valleys found in the selected observation row or observation column of the said comparison frame are missing within a range of one pixel pitch near the corresponding position of the selected observation row or observation column of the said reference frame, Use six accumulators to count and track their numbers: N _{RX ghost edges} (j+1), N _{RY ghost edges} (j+1), N _{GX ghost edges} (j+1), N _{GY ghost edges Edge} (j+1), N _{BX Ghost Edge} (j+1), N _{BY Ghost Edge} (j+1); If the "ghost edge reflection condition" of two or more of the three primary colors satisfies:

or

Then the measurement is considered "lost tracking" and a warning signal is issued, where the error tolerance value error1 represents the ghost edge of one of the three primary colors red, green and blue along the selected observation row or observation column in the comparison frame The percentage of the number of reflective conditions to the total number of edge reflective conditions of the primary color is preset according to the optical properties of the object surface and the measurement environment; the edge reflective conditions on the edges of the comparison frame do not consider the "ghost edge reflective condition" , they may come from the outside of the photopixel array; Step 7. If the measurement "lost tracking", go back to step 1 and restart the measurement work; Step 8. If there is no "lost tracking" in the measurement, continue the measurement work: along the selected observation row and the selected observation column, according to the three primary color component data of the camera, use the peak/valley sub-pixel displacement detection algorithm to calculate respectively Subpixel displacement of three primary color components:

In this measurement, the total sub-pixel displacement calculation formula is:

Step 9. In this measurement, the velocity vector calculation formula of the object is:

Step 10. Take the comparison frame in step 5 as a new reference frame, take a certain interval time dt from the shooting of this frame, and re-shoot a frame as a new comparison frame, that is, skip to step 5 and continue the measurement. 2. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, wherein the definition of the red, green or blue edge direction data in the measurement step 2 is: According to the component data of one of the three primary colors of red, green or blue in the pixel array, along the X-axis or along the Y-axis direction, if a pixel has a value of a certain three-primary color component that is greater than that of the second pixel behind it The value of the three primary color components is smaller by an error tolerance value error, that is, if I(X, Y) _red < I _red (X+2, Y)-error or I(X, Y) _red < I(X, Y+2) _red -error I(X,Y) _Green < _IGreen (X+2,Y)-error or I(X,Y) _Green <I(X,Y+2) _Green- error I(X, Y) _blue < I _blue (X+2, Y)-error or I(X, Y) _blue < I(X, Y+2) _blue- error It is defined that there is a red, green or blue positive edge between these two pixels; if the value of a certain three-primary color component of a pixel is one larger than the corresponding three-primary color component value of the second pixel behind it Error tolerance value error, that is, if I (X, Y) _red > I _red (X+2, Y) + error or I (X, Y) _red > I (X, Y + 2) _red + error I(X, Y) _green > I _green (X+2, Y) + error or I (X, Y) _green > I (X, Y + 2) _green + error I (X, Y) _blue > I _blue (X+2, Y) + error or I (X, Y) _blue > I (X, Y + 2) _blue + error Then it is defined that there is a red, green or blue negative edge between these two pixels; the edge thus obtained is located at the position of the first pixel after this pixel, that is, at the middle position of the two pixels participating in the comparison on that pixel; if the value of a certain three-primary color component of a pixel is close to the corresponding three-primary color component value of the second pixel behind it, the difference between its RGB component values does not exceed an error tolerance value error, that is, if I(X+2, Y) _red -error<I(X, Y) _red <I(X+2, Y) _red +error Or I(X, Y+2) _red -error<I(X, Y) _red <I(X, Y+2) _red +error; I(X+2,Y) _green -error<I(X,Y) _green <I(X+2,Y) _green +error or I(X, Y+2) _green -error<I(X, Y) _green <I(X, Y+2) _green +error; I(X+2, Y) _blue -error<I(X, Y) _blue <I(X+2, Y) _blue +error Or I(X, Y+2) _blue -error<I(X, Y) _blue <I(X, Y+2) _blue +error; It is considered that there is no "edge" corresponding to the color wavelength between the two pixels, or the edge of the third type of color; along a certain coordinate axis, all red positive edges and red negative edges And the third type of red edge constitutes the red edge direction data of this direction, all the green positive edges and green negative edges and the third type of green edge form the green edge direction data of this direction, all the blue positive edges and the blue negative edge and the third type of blue edge form the blue edge direction data in this direction; according to the specific lighting situation, the error tolerance value error in the above formula is preset to a small value; in the pixel array There is no edge direction data for the pixel positions on the four sides and corners of . 3. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, characterized in that, the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions described in the measurement step 4 and step 5 , which is defined as: According to the red, green or blue side direction data of the pixels in the selected observation row and observation column, along the X axis or along the Y axis direction, if there are two or more consecutive positive sides of a certain primary color , or two or more consecutive third-type edges of the base color followed by a negative edge of the base color, which is called the first-type edge reflection condition of the base color, that is, it is considered that there is a base color at this position peak; if two or more consecutive negative sides of a certain basic color, or two or more consecutive third-type sides of this basic color are followed by a positive side of this basic color, it is called It is the second type of edge reflection condition of the base color, that is, it is considered that there is a valley of the base color at this position; 4. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, characterized in that, the peak/valley sub-pixel displacement detection algorithm described in the measuring step eight comprises: For a certain three primary colors, compare the positions of peaks and valleys corresponding to the selected observation row and observation column in the reference frame and the comparison frame, and judge the corresponding peaks and valleys in the observation row and observation column accordingly. The direction of movement of the valley, and use the accumulator count to track the movement of the peak and valley along the coordinate axis, specifically, In the observed row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then tracking the number of the peaks of the primary color shifted to the right The N _{peak of the accumulator is shifted to the right by} +1. If the position of a certain three primary color peaks in the comparison frame is displaced to the left by one pixel pitch unit relative to its position in the reference frame, the number of the primary color peaks shifted to the left is tracked. The N _{peak of the accumulator is shifted to the left by} +1; if the position of a valley of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted to the right The accumulator N of the number of _{valleys is moved to the right} + 1, if the position of a valley of a certain three primary colors in the comparison frame is shifted to the left by one pixel pitch unit relative to its position in the reference frame, then the valley of the primary color is tracked The accumulator N _{valley of the number shifted to the left is shifted to the left} + 1; In the observation row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is displaced upward by one pixel pitch unit relative to its position in the reference frame, the accumulation of the number of upward displacements of the peaks of the primary color is tracked The N _{peak of the detector moves up} +1, if the position of a peak of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the number of the peaks of the primary color shifted downward is tracked The N _{peak of the accumulator moves down by} +1; if the position of a valley of a certain three primary colors in the comparison frame is displaced upwards by one pixel pitch unit relative to its position in the reference frame, then tracking the valleys of the primary color is shifted upward by 1 The accumulator N _{valley of the number moves up} +1, if the position of a valley of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted downward The number of the accumulator N _{valley moves down} +1; If N _{peak shifts to the right} (j+1)>N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=+0.5, If N _{peak shifts to the right} (j+1)<N _{peak shifts to the left} (j+1), it is calculated as ΔX _{peak displacement} (j+1)=-0.5; If the N _{valley moves to the right} (j+1)>N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=+0.5, If N _{valley moves to the right} (j+1)<N _{valley moves to the left} (j+1), it is counted as ΔX _{valley displacement} (j+1)=-0.5;

Similarly, If the N _{peak moves up} (j+1)>N _{peak moves down} (j+1), it is calculated as ΔY _{peak displacement} (j+1)=+0.5, If the N _{peak moves up} (j+1)<N _{peak moves down} (j+1), it is calculated as ΔY _{peak displacement} (j+1)=-0.5, If N _{valley moves up} (j+1)>N _{valley moves down} (j+1), it is counted as ΔY _{valley displacement} (j+1)=+0.5, If N _{valley moves up} (j+1)<N _{valley moves down} (j+1), it is counted as ΔY _{valley displacement} (j+1)=-0.5,

In the above formula, L _pixel-pixel represents the pixel pitch, and its unit is one of μm, mm or inch, and N is the number. The meanings of other symbols are as marked by the characters. The above calculation formulas should be applied to a certain pixel array of the camera respectively. Specific three primary color component data.

Images