CN102022981B - Peak-valley motion detection method and device for measuring sub-pixel displacement - Google Patents

Peak-valley motion detection method and device for measuring sub-pixel displacement Download PDF

Info

Publication number
CN102022981B
CN102022981B CN 200910190924 CN200910190924A CN102022981B CN 102022981 B CN102022981 B CN 102022981B CN 200910190924 CN200910190924 CN 200910190924 CN 200910190924 A CN200910190924 A CN 200910190924A CN 102022981 B CN102022981 B CN 102022981B
Authority
CN
China
Prior art keywords
valley
peak
axis
pixel
edge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 200910190924
Other languages
Chinese (zh)
Other versions
CN102022981A (en
Inventor
曾艺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Technology and Business University
Original Assignee
Chongqing Technology and Business University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Technology and Business University filed Critical Chongqing Technology and Business University
Priority to CN 200910190924 priority Critical patent/CN102022981B/en
Publication of CN102022981A publication Critical patent/CN102022981A/en
Application granted granted Critical
Publication of CN102022981B publication Critical patent/CN102022981B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

测量亚像素位移的峰谷运动探测方法及装置,由一台普通的计算机及其摄像头组成,其测量亚像素位移的方法是,先拍摄一帧,根据沿坐标轴方向的边方向数据,选取最具被测物反射特征的观测行与观测列;接着拍摄一帧,确定这两帧三基色各自的边方向数据及其峰和谷,通过对比,如果观测行和观测列中不能确定其相互位移关系的峰或谷的数目和超出其轴向该基色的峰、谷数目和的某个预定百分比,就认为“失去跟踪”,需要再拍摄一帧,否则,根据先后连续两帧中各基色的峰、谷移动的方向及其数目,分别计算各个轴向各个基色的亚像素位移,取其平均值为总的位移;有效地克服了环境光照变化的影响,适于位移与光照变化的速度都远低于拍摄帧速率的场合。

Figure 200910190924

The peak-to-valley motion detection method and device for measuring sub-pixel displacement are composed of an ordinary computer and its camera. The method for measuring sub-pixel displacement is to first shoot a frame, and select the most The observation row and observation column with the reflection characteristics of the measured object; then take a frame and determine the edge direction data and peaks and valleys of the three primary colors of the two frames. By comparison, if the mutual displacement cannot be determined in the observation row and observation column If the number of peaks or valleys of the relationship exceeds a certain predetermined percentage of the sum of the number of peaks and valleys of the primary color on its axis, it is considered "lost tracking", and another frame needs to be shot, otherwise, according to the The direction and number of peaks and valleys move, respectively calculate the sub-pixel displacement of each primary color in each axis, and take the average value as the total displacement; effectively overcome the influence of environmental light changes, suitable for both displacement and light change speed When the frame rate is much lower than the shooting frame rate.

Figure 200910190924

Description

测量亚像素位移的峰谷运动探测方法及装置Peak and valley motion detection method and device for measuring sub-pixel displacement

技术领域 technical field

本发明属于数字图像测量技术领域,特别是涉及采用计算机摄像头无接触测量物体的二维微小位移的方法及其装置。The invention belongs to the technical field of digital image measurement, in particular to a method and a device for non-contact two-dimensional micro-displacement measurement of an object by using a computer camera.

背景技术 Background technique

计算机摄像头在近几年迅速地获得了普及,一般用于网络视频聊天或监视拍摄工作,其核心是CCD和CMOS光电成像芯片,即光电传感器阵列。而在其前面发明的光学定向装置的核心也是光电探测像素阵列,用于探测它相对所在表面的二维移动。因此,考察光学定向装置有益于开阔摄像头的应用。Computer cameras have gained popularity rapidly in recent years, and are generally used for network video chatting or surveillance shooting. The core is CCD and CMOS photoelectric imaging chips, that is, photoelectric sensor arrays. The core of the optical orientation device invented before it is also a photodetection pixel array, which is used to detect its two-dimensional movement relative to the surface on which it is located. Therefore, it is beneficial to investigate optical orientation devices for wide-open camera applications.

U.S.Pat.No.5,288,993揭示了一个利用光探测器阵列的光标定向装置,其结构中有一个被照明的靶球,该靶球必须具有随机分布的斑纹。U.S.Pat.No.5,703,356进一步揭示了一个鼠标形式的光学定向装置,它不需要靶球,它所利用的光是直接地从定向装置所移动的表面上反射来的。上述美国专利基于所谓的“边运动探测技术”来提取与运动相关的信息,所述边被定义为光探测阵列里两个像素之间的空间强度的差,这些边的每一个的相对运动被跟踪和测量,籍此确定代表光探测阵列与所在表面的被照明部分之间相对运动的总的位移。U.S. Pat. No. 5,288,993 discloses a cursor orienting device utilizing an array of photodetectors in which an illuminated target sphere must have randomly distributed markings. U.S. Pat. No. 5,703,356 further discloses an optical pointing device in the form of a mouse which does not require a target ball and which utilizes light reflected directly from the surface on which the pointing device is moved. The above-mentioned US patent extracts motion-related information based on the so-called "edge motion detection technique". Tracking and measurement whereby the total displacement representing the relative motion between the light detection array and the illuminated portion of the surface upon which it is determined is determined.

根据U.S.Pat.No.5,288,993,分别沿着光电传感器阵列的第一个轴和第二个轴排列的像素对之间确定“边”,光电传感器阵列可以是正交的也可以不是正交的。该专利中的边Ex和Ey定义为位于两个最邻近的像素的中间。根据U.S.Pat.No.5,288,993和U.S.Pat.No.5,703,356,一方面,计算沿着第一个轴的正方向运动的边Ex的数目与沿着第一个轴的相反的方向运动的边Ex的数目之间的归一化差,另一方面,计算沿着第二个轴的正方向运动的边Ey的数目与沿着第二个轴的相反的方向运动的边Ey的数目之间的归一化差,比较光电传感器阵列里这些边在相继的两个时间点的位置可以确定边的相对运动。According to US Pat. No. 5,288,993, "edges" are defined between pairs of pixels arranged along a first axis and a second axis, respectively, of a photosensor array, which may or may not be orthogonal. Edges Ex and E y in this patent are defined as being located in the middle of two nearest neighbor pixels. According to USPat.No.5,288,993 and USPat.No.5,703,356, on the one hand, calculate the difference between the number of sides Ex moving along the positive direction of the first axis and the number of sides Ex moving along the opposite direction of the first axis On the other hand, computing the normalized difference between the number of edges E y moving in the positive direction of the second axis and the number of edges E y moving in the opposite direction of the second axis The relative motion of the edges can be determined by comparing the positions of the edges in the photosensor array at two consecutive time points.

典型的光学定向装置包括一个光源,例如红外线LED,该光源根据一个确定的顺序间歇地照明表面的一部分,光电探测阵列的像素信号根据所确定的顺序被取样,提供了用于比较的相继的两帧边的图像,从而确定阵列相对表面的运动。A typical optical pointing device includes a light source, such as an infrared LED, that intermittently illuminates a portion of the surface according to a determined sequence. The pixel signals of the photodetection array are sampled according to the determined sequence, providing two consecutive Frame the image of the edge to determine the motion of the array relative to the surface.

根据上述两个美国专利的一个方案,采用差分技术有利于确定两个像素之间的一个边的条件。其关于边的定义是:如果两个光敏单元的强度的比率大于某个确定了的值,这两个像素之间存在一个边。文献中还给出了关于边沿着水平的或垂直的方向的定义表示式,并指出,上述条件Ex和Ey并不依赖于边的方向或意义,只是表明两个像素之间是否存在一个边。According to one approach of the above two US patents, the use of differential techniques is advantageous for determining the condition of an edge between two pixels. Its definition of an edge is: if the ratio of the intensities of two photosensitive units is greater than a certain value, there is an edge between these two pixels. The literature also gives a definition expression about the edge along the horizontal or vertical direction, and points out that the above conditions E x and E y do not depend on the direction or meaning of the edge, but only indicate whether there is a side.

上述“边”的定义和运动探测算法在光学跟踪球里取得了成功。光学跟踪球是探测一个受用户操作的被照明了的球的光强图案。上述定义与算法需要被探测的光强图案显示出清晰的精细的光强差异。因此,球的表面被覆盖了一种色彩的随机性标志图案,以显示出有别于背景的对比度。此外,这些标记大多呈斑点的形式,需要一个预定的大小。U.S.Pat.No.5,288,993中的斑点大小在0.5mm2-0.7mm2,密度约为1个斑点/mm2;这些标记的大小相对地独立于球的直径,但是,主要取决于光电探测器阵列的分辨率和大小。U.S.Pat.No.5,703,356提出,一个单独的斑点在传感器上的像最小地应当覆盖两个邻近像素之间中心到中心的距离(pixel pitch)。实际上,优先选择的典型的斑点的大小应当使得该点的像的表面覆盖大约5个像素。在此条件下,边的数目保持充分地不变,充分地小于像素的数目,从而可以被确定并用于探测运动。The definition of "edge" and the motion detection algorithm described above have been successful in optical trackballs. Optical trackballs detect the light intensity pattern of an illuminated ball that is manipulated by the user. The above definitions and algorithms require that the detected light intensity patterns show clear and fine light intensity differences. Therefore, the surface of the ball is covered with a color random logo pattern to show the contrast with the background. Furthermore, these marks are mostly in the form of spots, which require a predetermined size. The spot size in USPat.No.5,288,993 is 0.5mm 2 -0.7mm 2 , and the density is about 1 spot/mm 2 ; the size of these marks is relatively independent of the diameter of the ball, but depends mainly on the photodetector array. resolution and size. US Pat. No. 5,703,356 proposes that the image of a single spot on the sensor should at least cover the center-to-center distance (pixel pitch) between two adjacent pixels. In practice, a typical spot size is preferably chosen such that the surface of the image of the spot covers about 5 pixels. Under this condition, the number of edges remains sufficiently constant, sufficiently smaller than the number of pixels, to be determined and used to detect motion.

上述探测运动的方案应用到光学定向装置,如果被照明的表面不出现一个预定的图案,就会出现一个严重的问题;例如,“无球”鼠标直接地使用从一个随机的表面,例如纸的或桌子的表面,反射来的光。U.S.Pat.No.5,703,356描述了一个不需要任何球的的光学鼠标,但是,该方案需要被照明的表面显示出一个合适的图案,该图案具有充分数目的暗的和亮的区域,这些区域大小充分。The above scheme for detecting motion applied to an optical orientation device presents a serious problem if the illuminated surface does not appear in a predetermined pattern; for example, a "ballless" mouse used directly from a random surface such as paper Or the surface of a table, reflecting incoming light. U.S.Pat.No. 5,703,356 describes an optical mouse that does not require any balls, but the solution requires that the illuminated surface exhibit a suitable pattern with a sufficient number of dark and bright areas of the size full.

任何情形下,对于一个没有特征的表面,应用上述技术的时候,上述定义的边实际上在每一个像素之间都能够被探测到。因此,如果不能够清晰地探测到并跟踪具体的和很好定义的边的图案,就不可能导出关于光学探测器件和被照明表面之间的相对运动的任何测量。因此,上述运动探测技术不可能获得实际的应用。In any case, for a featureless surface, when applying the above technique, the edges defined above can be detected between virtually every pixel. Therefore, without being able to clearly detect and track specific and well-defined edge patterns, it is impossible to derive any measurements about the relative motion between the optical detection device and the illuminated surface. Therefore, the above-mentioned motion detection technique cannot be practically applied.

其实,U.S.Pat.No.5,578,813和U.S.Pat.No.5,644,139已经揭示过一些关于“被照明的表面产生纯粹随机的光强图案”的方案。这两个发明根据光电探测器阵列输出的连续的像帧的关联,提出了运动探测的原理。然而,它们比较邻近像素之间的光强没有提供关于邻近像素之间空间的强度差(即边)的任何信息,发明中提出的连续像帧的关联隐含了一些限制。特别地,为了导出相对运动的一个足够精确的测量,实际上需要以一个相对表面很低的角度照明参考表面,这个角度典型地处于表面的16°以内,U.S.Pat.No.5,288,720对此更精确地给予了说明。这导致约束光学定向装置的结构。此外,分析灰度像帧的时候,上述像帧关联技术表现了某些限制,随着像的强度的位数长度的增加,关联处理的复杂性成指数函数增加。实际中,上述关联技术应用到光学装置的运动探测局限于分析二进制的黑-白像。In fact, U.S. Pat. No. 5,578,813 and U.S. Pat. No. 5,644,139 have disclosed some schemes for "generating a purely random pattern of light intensity on an illuminated surface". These two inventions propose the principle of motion detection based on the correlation of continuous image frames output by the photodetector array. However, their comparison of light intensities between adjacent pixels does not provide any information about spatial intensity differences (ie edges) between adjacent pixels, and the association of consecutive image frames proposed in the invention implies some limitations. In particular, in order to derive a sufficiently accurate measurement of relative motion, it is actually necessary to illuminate the reference surface at a very low angle relative to the surface, typically within 16° of the surface, U.S. Pat. No. 5,288,720 being more precise for this explained. This results in constraining the structure of the optical orientation device. Furthermore, the above image frame correlation techniques exhibit certain limitations when analyzing grayscale image frames, and the complexity of the correlation process increases exponentially as the bit length of the image intensity increases. In practice, the application of the aforementioned correlation techniques to motion detection in optical devices is limited to analyzing binary black-and-white images.

参考上述美国专利,最近提交审查的发明专利“使用计算机摄像头测量微小二维位移的方法及装置”(申请号:200910104277.8,申请日:2009.7.7)分析了国内有关“摄像”“测量”的专利文件,提出了一种以计算机摄像头为光电转换传感器,通过帧-帧关联匹配比较,测量微小的二维位移矢量和速度矢量的的方法,但是,它仍然是针对光强图案进行匹配,因而也需要照明均匀且稳定。Referring to the above-mentioned US patent, the recently submitted invention patent "method and device for measuring small two-dimensional displacement using computer camera" (application number: 200910104277.8, application date: 2009.7.7) analyzed domestic patents related to "camera" and "measurement" The document proposes a method of using a computer camera as a photoelectric conversion sensor to measure tiny two-dimensional displacement vectors and velocity vectors through frame-frame correlation matching and comparison. However, it still matches light intensity patterns, so it also Uniform and stable lighting is required.

”Method and sensing device for motion detection in an optical pointing device,such as anoptical mouse”(US 7,122,781 B2,Oct.17,2006,Rotzoll et al.)提供更适合光学定向装置的二个解决方案,其方法之一是:预处理图像帧以提取描述邻近像素之间的光强差(即边)的二进制数据,直接跟踪正边和负边的运动方向,计算相对运动的位移测量,即“Local Edge DirectionMotion Detection Algorithm(局域边方向运动探测算法)”。其方法之二是:从所提取的边方向数据进一步提取边反射状况,它描述沿着光探测器阵列的一个轴方向的正边和负边的连续性,然后,比较先后两帧边反射状况的位置,计算出相对运动的位移测量,即“Peak/Null MotionDetection Algorithm(峰/谷运动探测算法)”。该发明的特点是:1)被照明的表面没有施加特征,产生纯粹随机的光强图案;易于提取可以被跟踪的有特征的特点和图案,并直接用于运动探测。2)显示了对像素之间强度的轻微的变化的巨大敏感性,允许适用一个宽范围的照明角度,而不是需要一个低的照明角度,以增强U.S.Pat.No.5,5788,813、U.S.Pat.No.5,644,139和U.S.Pat.No.5,686,720所使用的关联技术中表面图案的对比度。3)可以分析灰度图像,所需要的处理表现得特别地有活力和简单,而不会增加处理的复杂程度。"Method and sensing device for motion detection in an optical pointing device, such as an optical mouse" (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.) provides two solutions that are more suitable for optical pointing devices. One is: preprocessing image frames to extract binary data describing the difference in light intensity between adjacent pixels (i.e., edges), directly tracking the direction of motion of positive and negative edges, and computing a displacement measure of relative motion, ie "Local Edge DirectionMotion Detection Algorithm (local edge direction motion detection algorithm)". The second method is: further extracting edge reflection conditions from the extracted edge direction data, which describes the continuity of positive and negative edges along an axis direction of the photodetector array, and then comparing the edge reflection conditions of two consecutive frames position, calculate the displacement measurement of the relative motion, namely "Peak/Null MotionDetection Algorithm (peak/valley motion detection algorithm)". The characteristics of the invention are: 1) The surface to be illuminated has no applied features and produces purely random light intensity patterns; it is easy to extract characteristic features and patterns that can be tracked and directly used for motion detection. 2) Shows great sensitivity to slight variations in intensity between pixels, allowing a wide range of illumination angles to be applied, rather than requiring a low illumination angle, to enhance U.S. Pat. No. 5,5788,813, U.S. Contrast of surface patterns in related techniques used in Pat. No. 5,644,139 and U.S. Pat. No. 5,686,720. 3) Grayscale images can be analyzed, and the required processing appears particularly dynamic and simple without increasing the complexity of the processing.

该专利所谓“Pcak/Null Motion Detection Algorithm(峰/谷运动探测算法)”计算总的位移测量值的算法之一如下:One of the algorithms for calculating the total displacement measurement value of the patent's so-called "Pcak/Null Motion Detection Algorithm" is as follows:

Xx PEAKPEAK -- DISPLACEMENTDISPLACEMENT == NN PEAKPEAK -- RIGHTRIGHT -- NN PEAKPEAK -- LEFTLEFT NN XPEAKXPEAK

Xx NULLNULL -- DISPLACEMENTDISPLACEMENT == NN NULLNULL -- RIGHTRIGHT -- NN NULLNULL -- LEFTLEFT NN XNULLXNULL

YY PEAKPEAK -- DISPLACEMENTDISPLACEMENT == NN PEAKPEAK -- UPUP -- NN PEAKPEAK -- DOWNDOWN NN YPEAKYPEAK

YY NULLNULL -- DISPLACEMENTDISPLACEMENT == NN NULLNULL -- UPUP -- NN NULLNULL -- DOWNDOWN NN YNULLYNULL

Xx DISPLACEMENTDISPLACEMENT == Xx PEAKPEAK -- DISPLACEMENTDISPLACEMENT ++ Xx NULLNULL -- DISPLACEMENTDISPLACEMENT 22 ×× LL PIXELPIXEL -- PIXELPIXEL ;;

YY DISPLACEMENTDISPLACEMENT == YY PEAKPEAK -- DISPLACEMENTDISPLACEMENT ++ YY NULLNULL -- DISPLACEMENTDISPLACEMENT 22 ×× LL PIXELPIXEL -- PLXELPLXEL

该专利计算总的位移测量值的峰/谷运动探测算法之二是:The second peak/valley motion detection algorithm in which the patent calculates the total displacement measurement is:

Xx DISPLACEMENTDISPLACEMENT == (( NN PEAKPEAK -- RIGHTRIGHT ++ NN NULLNULL -- RIGHTRIGHT )) -- (( NN PEAKPEAK -- LEFTLEFT ++ NN NULLNULL -- LEFTLEFT )) :: (( NN XPEAKXPEAK ++ NN XNULLXNULL )) ×× LL PIXELPIXEL -- PIXELPIXEL

YY DISPLACEMENTDISPLACEMENT == (( NN PEAKPEAK -- UPUP ++ NN NULLNULL -- UPUP )) -- (( NN PEAKPEAK -- DOWNDOWN ++ NN NULLNULL -- DOWNDOWN )) :: (( NN YPEAKYPEAK ++ NN YNULLYNULL )) ×× LL PIXELPIXEL -- PIXELPIXEL

式中,X、Y和N分别表示两个坐标轴和数目(Number),各项的含义如文字或符号所示,例如,NXPEAK和NPEAK-RIGHT分别表示X轴方向峰的数目及其发生向右移动了的峰的数目,LPIXEL-PIXEL表示像素间距,其单位可以是μm、mm或inch等。In the formula, X, Y and N represent the two coordinate axes and the number (Number) respectively, and the meanings of each item are shown in words or symbols, for example, N XPEAK and N PEAK-RIGHT respectively represent the number of peaks in the X-axis direction and their The number of peaks that have shifted to the right, L PIXEL-PIXEL represents the pixel pitch, and its unit can be μm, mm, or inch.

上述两种峰/谷运动探测算法并不等同,但是,两者的峰和谷的总的数目相同,因而总的位移测量值都是一个像素间距LPIXEL-PIXEL的分数,两者的计算结果接近,都属于亚像素级位移。而且,第二种计算位移的方法对于每一个坐标轴只需要进行一次除法。The above two peak/valley motion detection algorithms are not equivalent, however, the total number of peaks and valleys is the same for both, so the total displacement measurement is a fraction of the pixel spacing L PIXEL-PIXEL , and the calculation results of the two Close, all belong to sub-pixel level displacement. Also, the second method of computing displacement requires only one division per axis.

上述峰谷运动探测算法以提取图像帧的特征为基础,克服了光强图案的随机变化给帧-帧匹配所带来的困难,因而,它优越于前述种种专利技术,大大地降低了对其运动的表面的光学反射质地的要求。The above-mentioned peak-valley motion detection algorithm is based on extracting the features of the image frame, and overcomes the difficulty brought by the random change of the light intensity pattern to the frame-frame matching. Optically reflective texture requirements for moving surfaces.

此外,上述峰谷运动探测算法还有其它的优点。首先,探测器“看”到的边反射数目取决于传感器的聚焦状态。第二,边反射被假设在像素阵列上移动,并且,在连续的两次曝光期间这些边反射的移动幅度不会超出一个单位的像素间距。对于一个给定的运动及其最大的运动速度,该条件易于满足。如果一个边反射的移动超出一个单位的像素间距,上述运动探测方案不能够决定该边反射来自何处。如果在连续的两次曝光期间,探测器相对被照明的表面移动过快,就会发生此类情况,它代表了一种“失去跟踪(loss-off-track)”的情形。通过查看那些不知道来自何处的边反射状况的数目能够探测到这种“失去跟踪”的情形,此时,前一个状态(帧)的反射状况在当前的位置或当前位置附近的一个像素间距范围内没有相似的边反射状况,这些边反射状况被定义为“鬼影边反射状况(ghost edge inflection conditions)”,它们沿着两个坐标轴的方向潜在地会出现,其沿各坐标轴方向的数目可以用两个累加器分别加以检测和跟踪,并与预先确定的某个门槛值相比较,籍此确定“失去跟踪”的情况,并作为一个警告信号报告给外部的操作者。光电阵列边上的边反射不计为“鬼影边反射状况”;当它们看起来不知来自何处的时候,有可能来自光电阵列的外部。In addition, the above-described peak-to-valley motion detection algorithm has other advantages. First, the number of edge reflections that the detector "sees" depends on the focus state of the sensor. Second, edge reflections are assumed to move across the pixel array, and these edge reflections do not move by more than one unit of pixel pitch during two consecutive exposures. For a given motion and its maximum motion velocity, this condition is easy to satisfy. If an edge reflection moves beyond one unit of pixel pitch, the motion detection scheme described above cannot determine where the edge reflection came from. This occurs if the detector moves too quickly relative to the illuminated surface between two consecutive exposures, and represents a "loss-off-track" situation. This "lost-of-tracking" situation can be detected by looking at the number of edge reflections that don't know where they came from, where the reflections from the previous state (frame) were at or one pixel apart from the current position There are no similar edge reflection conditions in range, these edge reflection conditions are defined as "ghost edge reflection conditions (ghost edge reflection conditions) (ghost edge reflection conditions)", they can potentially occur along the direction of the two coordinate axes, which along the direction of each coordinate axis The number can be detected and tracked by two accumulators respectively, and compared with a predetermined threshold value, thereby determining the "lost track" situation, and reported to the external operator as a warning signal. Edge reflections on the sides of the photocell do not count as "ghost side reflection conditions"; while they appear to be coming from nowhere, they may be coming from outside the photocell.

上述探测判断“失去跟踪”的方法符合实际,可以算法实现。当然,还可以找到其它的探测、判断“失去跟踪”的方法。但是,经过仔细分析,会发现,该专利上述“峰/谷运动探测算法”,以及“局域边方向运动探测算法”缺乏计算的根据,难于理解。The above-mentioned method of detecting and judging "lost tracking" is practical and can be realized by algorithm. Of course, other methods of detecting and judging "lost tracking" can also be found. However, after careful analysis, it will be found that the above-mentioned "peak/valley motion detection algorithm" and "local edge direction motion detection algorithm" in this patent lack calculation basis and are difficult to understand.

总之,关于光学定向装置中探测相对运动的美国专利技术,激励了使用计算机摄像头测量随机状况的表面上所发生的微小位移的思想。光学定向装置中的光电传感器阵列相对所在的表面之间的距离极小(毫米数量级),它采用单色LED之类照明光源,其发光强度受电子电路控制,实施间歇式照明,因此,被照表面面积相当地小,光照均匀,其反射光束不会受到环境光辐射的影响,较好地反映了该表面的光学反射性质。而应用计算机摄像头进行位移测量,光电传感器阵列距离被测物体较远,被测物表面的光学反射性质会起到更大的影响。例如,由于可见光的频率较短,被光电传感器进行光电转换的物体的反射光会是若干次反射的一个平均值;随着光照角度的不同,质地粗糙的表面其漫反射会很强烈,变化也会较大;金属与平面镜之类强反射面所产生的反射光更会随着入射光角度的变化而强烈地发生变化,甚至于连周围其它物体的映像也将会被反射出来,这些都会模糊、混淆反射面本身的质地特征;入射的阳光被物体反射以后会出现丰富的偏振光;测量环境中其它的光源的辐射或飘游灰尘或烟雾等的影响,也会导致被测物体表面的图像帧发生整体的、或者局部区域的光强变化,或者表现为灰度图像的光强数据出现随机的波动。这些为应用“峰/谷运动探测亚像素位移算法”测量随机表面的微小位移提出了新的问题。In summary, the US patent technique for detecting relative motion in an optical orientation device inspired the idea of using computer cameras to measure small displacements occurring on surfaces of random conditions. The distance between the photoelectric sensor array in the optical orientation device is extremely small (on the order of millimeters), and it uses a lighting source such as a monochromatic LED. The surface area is relatively small, the illumination is uniform, and the reflected light beam will not be affected by ambient light radiation, which better reflects the optical reflection properties of the surface. However, when using a computer camera for displacement measurement, the photoelectric sensor array is far away from the measured object, and the optical reflection properties of the surface of the measured object will have a greater impact. For example, due to the short frequency of visible light, the reflected light of an object that is photoelectrically converted by a photoelectric sensor will be an average value of several reflections; with the different illumination angles, the diffuse reflection of a rough surface will be very strong, and the change will also vary. will be larger; the reflected light produced by strong reflective surfaces such as metals and flat mirrors will change strongly with the change of the incident light angle, and even the images of other surrounding objects will be reflected, which will be blurred , confuse the texture characteristics of the reflective surface itself; the incident sunlight will be rich in polarized light after being reflected by the object; the influence of radiation from other light sources in the measurement environment or floating dust or smoke will also cause image frames on the surface of the measured object Overall or local light intensity changes, or random fluctuations in the light intensity data of the grayscale image. These raise new questions for the application of "peak/valley motion detection sub-pixel displacement algorithm" to measure small displacements of random surfaces.

发明内容 Contents of the invention

为了充分发挥计算机摄像头的光电传感器阵列的功能,本发明提供一种可以适应变化相对不大的照明环境的测量亚像素位移的峰谷运动探测方法及装置,它以计算机摄像头为光电转换传感器,即时地拍摄被测量的物体,再经过峰/谷亚像素位移探测算法处理,获得该物体在与摄像头的光轴相垂直的平面上的微小的二维位移矢量和速度矢量。In order to give full play to the function of the photoelectric sensor array of the computer camera, the present invention provides a method and device for detecting peak-to-valley motions that can adapt to relatively small changes in the lighting environment and measure sub-pixel displacement. It uses a computer camera as a photoelectric conversion sensor. The object to be measured is photographed accurately, and then processed by the peak/valley sub-pixel displacement detection algorithm to obtain the tiny two-dimensional displacement vector and velocity vector of the object on a plane perpendicular to the optical axis of the camera.

本发明解决其技术问题所采用的技术方案是:测量的物体位于计算机摄像头的光学镜头前面有效的聚焦成像范围以内,被测量物体的照明环境相对均匀、稳定,所述摄像头通过USB接口连接到一台普通的计算机,该计算机配置USB接口、内存、CPU、硬盘、显示卡与显示器、键盘和鼠标、操作系统、摄像头驱动程序以及摄像头拍摄及峰/谷亚像素位移探测程序,该程序包括下述步骤:The technical solution adopted by the present invention to solve its technical problems is: the object to be measured is located within the effective focus imaging range in front of the optical lens of the computer camera, the lighting environment of the object to be measured is relatively uniform and stable, and the camera is connected to a computer via a USB interface. An ordinary computer, the computer is equipped with USB interface, memory, CPU, hard disk, display card and monitor, keyboard and mouse, operating system, camera driver and camera shooting and peak/valley sub-pixel displacement detection program, the program includes the following step:

步骤一、拍摄一帧(M行×N列)被测物体,不考虑该图像帧的边上的两行与两列,得到参考帧[(M-2)行×(N-2)列],为该参考帧以及其后所有拍摄的图像帧确定相同的坐标轴,其中,M,N∈正整数;Step 1. Take a frame (M rows×N columns) of the measured object, regardless of the two rows and two columns on the side of the image frame, and obtain a reference frame [(M-2) rows×(N-2) columns] , determine the same coordinate axis for the reference frame and all subsequent image frames, wherein, M, N∈ positive integer;

步骤二、对于上述参考帧之像素阵列,根据其红色、绿色和蓝色分量的数据,分别逐行、逐列导出沿X轴和Y轴方向的红色的、绿色的和蓝色的边方向数据;Step 2. For the pixel array of the above reference frame, according to the data of the red, green and blue components, export the red, green and blue edge direction data along the X-axis and Y-axis direction row by row and column by column respectively ;

步骤三、分别逐行、逐列计算红色的、绿色的和蓝色的正边和负边的数目之和,取其中正边和负边的数目之和最多者所在的行与列分别作为观测行和观测列;Step 3. Calculate the sum of the numbers of positive and negative sides of red, green and blue respectively row by row and column by column, and take the row and column where the sum of the number of positive and negative sides is the largest as the observation rows and columns of observations;

步骤四、根据所选观测行与观测列的像素的三种基色的边方向数据,分别导出沿X轴和Y轴方向的三种基色的边反射状况,并分别使用累加器计数它们所对应的峰、谷的数目:NRX轴峰(j)、NRX轴谷(j)、NRY轴峰(j)和NRY轴谷(j),NGX轴峰(j)、NGY轴谷(j)、NGY轴峰(j)和NGY轴谷(j),NBX轴峰(j)、NBX轴谷(j)、NBY轴峰(j)和NBY轴谷(j),其中,j表示拍摄获得的帧的顺序计数值,j∈正整数,N(umber)表示数目,R、G、B分别表示红、绿和蓝色;Step 4. According to the edge direction data of the three primary colors of the pixels in the selected observation row and observation column, respectively derive the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the accumulator to count their corresponding Number of peaks and valleys: N RX axis peak (j), N RX axis valley (j), N RY axis peak (j) and N RY axis valley (j), N GX axis peak (j), N GY axis valley (j), N GY axis peak (j) and N GY axis valley (j), N BX axis peak (j), N BX axis valley (j), N BY axis peak (j) and N BY axis valley (j ), wherein j represents the sequential count value of the frame obtained by shooting, j∈positive integer, N (umber) represents the number, and R, G, B represent red, green and blue respectively;

步骤五、拍摄了所述参考帧之后,经过一定的间歇时间dt,拍摄第二帧被测物体的像(M×N),去掉像素帧边缘的两行和两列得到比较帧[(M-2)×(N-2)];沿着步骤三中选定的观测行或观测列,分别导出沿X轴和Y轴方向的三种基色的边方向数据和边反射状况,并分别使用累加器计数沿着X轴和Y轴方向的峰、谷的数目:NRX轴峰(j+1)、NRX轴谷(j+1)、NRY轴峰(j+1)和NRY轴谷(j+1),NGX轴峰(j+1)、NGX轴谷(j+1)、NGY轴峰(j+1)和NGY轴谷(j+1),NBX轴峰(j+1)、NBX轴谷(j+1)、NBY轴峰(j+1)和NBY轴谷(j+1);Step 5, after photographing the reference frame, after a certain intermittent time dt, photograph the image (M×N) of the second frame of the measured object, remove two rows and two columns at the edge of the pixel frame to obtain a comparison frame [(M- 2) × (N-2)]; along the observation row or observation column selected in step 3, respectively derive the edge direction data and edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the cumulative The counter counts the number of peaks and valleys along the X and Y axes: N RX axis peak (j+1), N RX axis valley (j+1), N RY axis peak (j+1), and N RY axis Valley (j+1), N GX axis peak (j+1), N GX axis valley (j+1), N GY axis peak (j+1) and N GY axis valley (j+1), N BX axis Peak (j+1), N BX axis valley (j+1), N BY axis peak (j+1) and N BY axis valley (j+1);

步骤六、对比所述参考帧与所述比较帧中所选观测行与观测列的三种基色的边反射状况的位置,检查其中的那些不知道来自何处的“鬼影边反射状况”,即:在所述参考帧的所选观测行或观测列发现的这些峰或谷,在所述比较帧的所选观测行或观测列的相应位置及其附近的一个像素间距范围内发生缺失,反之亦然,这时,用六个累加器分别统计和跟踪它们的数目:NRX鬼影边(j+1)、NRY鬼影边(j+1)、NGX鬼影边(j+1)、NGY鬼影边(j+1)、NBX鬼影边(j+1)、NBY鬼影边(j+1);Step 6. Comparing the positions of the edge reflection conditions of the three primary colors in the selected observation row and observation column in the reference frame and the comparison frame, and checking those "ghost edge reflection conditions" from where they do not know, That is: the peaks or troughs found in the selected observation row or observation column of the reference frame are missing within a pixel pitch range near the corresponding position of the selected observation row or observation column of the comparison frame, Vice versa, at this time, use six accumulators to count and track their numbers respectively: N RX ghost edge (j+1), N RY ghost edge (j+1), N GX ghost edge (j+ 1), N GY ghost edge (j+1), N BX ghost edge (j+1), N BY ghost edge (j+1);

如果三种基色中有两种或两种以上的基色的“鬼影边反射状况”满足:If the "ghost edge reflection condition" of two or more of the three primary colors satisfies:

Figure G2009101909241D00061
Figure G2009101909241D00061

or

则认为测量“失去跟踪”,并发出一个警告信号,式中,误差容限值error1表示比较帧中沿着所选观测行或观测列,三种基色(Red、Green、Blue)之一的鬼影边反射状况发生的数目占该基色的全部边反射状况数目的百分比,可以根据物体表面的光学性质和测量环境来预置,例如预置为:error1=5%;所述比较帧的边上的边反射状况不考虑“鬼影边反射状况”,它们有可能来自光电像素阵列的外部;Then it is considered that the measurement is "lost tracking" and a warning signal is issued. In the formula, the error tolerance value error1 represents the ghost of one of the three primary colors (Red, Green, Blue) along the selected observation row or observation column in the comparison frame The percentage of the number of shadow edge reflections taking place in the total number of edge reflections of the base color can be preset according to the optical properties of the object surface and the measurement environment, for example, it is preset as: error1=5%; on the edge of the comparison frame The edge reflections of , do not consider the "ghost edge reflections", which may come from the outside of the photopixel array;

步骤七、如果测量“失去跟踪”,回到步骤一,重新开始测量工作;Step 7. If the measurement "lost tracking", go back to step 1 and restart the measurement work;

步骤八、如果测量没有“失去跟踪”,则继续本次测量工作:沿着所选观测行与所选观测列,根据摄像头的三基色分量数据,运用峰/谷亚像素位移探测算法,分别计算三基色分量的亚像素位移:Step 8. If there is no "lost tracking" in the measurement, continue the measurement work: along the selected observation row and the selected observation column, according to the three primary color component data of the camera, use the peak/valley sub-pixel displacement detection algorithm to calculate respectively Subpixel displacement of three primary color components:

Figure G2009101909241D00071
Figure G2009101909241D00071

Figure G2009101909241D00072
Figure G2009101909241D00072

Figure G2009101909241D00073
Figure G2009101909241D00073

Figure G2009101909241D00074
Figure G2009101909241D00074

Figure G2009101909241D00075
Figure G2009101909241D00075

Figure G2009101909241D00076
Figure G2009101909241D00076

本次测量中,总的亚像素位移计算式是:In this measurement, the total sub-pixel displacement calculation formula is:

步骤九、本次测量中,物体的速度矢量计算式为:Step 9. In this measurement, the velocity vector calculation formula of the object is:

Figure G2009101909241D00079
Figure G2009101909241D000710
Figure G2009101909241D00079
Figure G2009101909241D000710

步骤十、以步骤五中的比较帧作为新的参考帧,距拍摄此帧一定的间歇时间dt,重新拍摄一帧作为新的比较帧,即跳到步骤五,继续测量。Step 10. Take the comparison frame in step 5 as a new reference frame, take a certain interval time dt from the shooting of this frame, and re-shoot a frame as a new comparison frame, that is, skip to step 5 and continue the measurement.

其中,所述摄像头拍摄及峰/谷运动探测亚像素位移程序之步骤二所述红色、绿色或蓝色的边方向数据的定义是:Wherein, the definition of the red, green or blue edge direction data in the second step of the sub-pixel displacement program of the camera shooting and peak/valley motion detection is:

根据像素阵列中红色、绿色或蓝色这三种基色之一的分量数据,沿着X轴或者沿着Y轴方向,如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要小一个误差容限值error,即如果According to the component data of one of the three primary colors of red, green or blue in the pixel array, along the X-axis or along the Y-axis direction, if a pixel has a value of a certain three-primary color component that is greater than that of the second pixel behind it The value of the three primary color components is smaller by an error tolerance value error, that is, if

I(X,Y)<I(X+2,Y)-error或I(X,Y)<I(X,Y+2)-errorI(X, Y) red < I red (X+2, Y)-error or I(X, Y) red < I(X, Y+2) red -error

I(X,Y)绿<I绿(X+2,Y)-error或I(X,Y)绿<I(X,Y+2)绿-errorI(X,Y) Green < IGreen (X+2,Y)-error or I(X,Y) Green <I(X,Y+2) Green- error

I(X,Y)<I(X+2,Y)-error或I(X,Y)<I(X,Y+2)-errorI(X, Y) blue < I blue (X+2, Y)-error or I(X, Y) blue < I(X, Y+2) blue- error

则定义这两个像素之间存在一个红色的、绿色的或蓝色的正边;如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要大一个误差容限值error,即如果It is defined that there is a red, green or blue positive edge between these two pixels; if the value of a certain three-primary color component of a pixel is one larger than the corresponding three-primary color component value of the second pixel behind it The error tolerance value error, that is, if

I(X,Y)>I(X+2,Y)+error或I(X,Y)>I(X,Y+2)+errorI(X,Y) red >I red (X+2,Y)+error or I(X,Y) red >I(X,Y+2) red +error

I(X,Y)绿>I绿(X+2,Y)+error或I(X,Y)绿>I(X,Y+2)绿+errorI(X, Y) green > I green (X+2, Y) + error or I (X, Y) green > I (X, Y + 2) green + error

I(X,Y)>I(X+2,Y)+error或I(X,Y)>I(X,Y+2)+errorI (X, Y) blue > I blue (X+2, Y) + error or I (X, Y) blue > I (X, Y + 2) blue + error

则定义这两个像素之间存在一个红色的、绿色的或蓝色的负边;如此获得的边位于该像素之后的第一个像素的位置,也即位于参与比较的两个像素的中间位置的那个像素上;如果一个像素的某种三基色分量值与其后面的第二个像素相应的三基色分量值接近,其RGB分量值相差不超过一个误差容限值error,即如果Then it is defined that there is a red, green or blue negative edge between these two pixels; the edge thus obtained is located at the position of the first pixel after this pixel, that is, at the middle position of the two pixels participating in the comparison on that pixel; if the value of a certain three-primary color component of a pixel is close to the corresponding three-primary color component value of the second pixel behind it, the difference between its RGB component values does not exceed an error tolerance value error, that is, if

I(X+2,Y)-error<I(X,Y)<I(X+2,Y)+error或I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error;I(X+2, Y) red -error<I(X, Y) red <I(X+2, Y) red +error or I(X, Y+2) red -error<I(X, Y) Red <I(X, Y+2) red +error;

I(X+2,Y)绿-error<I(X,Y)绿<I(X+2,Y)绿+error或I(X,Y+2)绿-error<I(X,Y)绿<I(X,Y+2)绿+error;I(X+2,Y) green -error<I(X,Y) green <I(X+2,Y) green +error or I(X,Y+2) green -error<I(X,Y) Green <I(X, Y+2) Green +error;

I(X+2,Y)-error<I(X,Y)<I(X+2,Y)+error或I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error;I(X+2, Y) blue -error<I(X,Y) blue <I(X+2,Y) blue +error or I(X,Y+2) blue- error<I(X,Y) Blue <I(X, Y+2) blue +error;

则认为这两个像素之间不存在该颜色波长对应的“边”,或称之为第三类该颜色的边;沿着某一个坐标轴方向,所有的红色的正边和红色的负边以及第三类红色的边组成该方向红色的边方向数据,所有的绿色的正边和绿色的负边以及第三类绿色的边组成该方向绿色的边方向数据,所有的蓝色的正边和蓝色的负边以及第三类蓝色的边组成该方向蓝色的边方向数据;上式中的误差容限值可以根据具体的光照情况,预置为一个小的数值,例如:error=10;像素阵列中的四个边与角上的像素位置不存在边方向数据。It is considered that there is no "edge" corresponding to the color wavelength between the two pixels, or the edge of the third type of color; along a certain coordinate axis, all red positive edges and red negative edges And the third type of red edge constitutes the red edge direction data of this direction, all the green positive edges and green negative edges and the third type of green edge form the green edge direction data of this direction, all the blue positive edges The blue negative edge and the third type of blue edge form the blue edge direction data in this direction; the error tolerance value in the above formula can be preset to a small value according to the specific lighting situation, for example: error =10; the pixel positions on the four sides and corners in the pixel array do not have side direction data.

所述摄像头拍摄及峰/谷运动探测亚像素位移程序之所述步骤四和步骤五中所述沿X轴和Y轴方向的三种基色的边反射状况,其定义为:The edge reflection conditions of the three primary colors along the X-axis and Y-axis directions in the step 4 and step 5 of the camera shooting and peak/valley motion detection sub-pixel displacement program are defined as:

根据所选观测行与观测列的像素的红色、绿色或蓝色的边方向数据,沿着X轴或沿着Y轴方向,如果连续的两个或多于两个的某种基色的正边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的负边,称之为该基色的第一类边反射状况,即认为在此位置存在一个该基色的峰;如果连续的两个或多于两个的某种基色的负边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的正边,称之为该基色的第二类边反射状况,即认为在此位置存在一个该基色的谷。According to the red, green or blue side direction data of the pixels in the selected observation row and observation column, along the X axis or along the Y axis direction, if there are two or more consecutive positive sides of a certain primary color , or two or more consecutive third-type edges of the base color followed by a negative edge of the base color, which is called the first-type edge reflection condition of the base color, that is, it is considered that there is a base color at this position peak; if two or more consecutive negative sides of a certain basic color, or two or more consecutive third-type sides of this basic color are followed by a positive side of this basic color, it is called It is the second type of edge reflection condition of the base color, that is, it is considered that there is a valley of the base color at this position.

所述摄像头拍摄及峰/谷运动探测亚像素位移程序之所述步骤八中所述的峰/谷亚像素位移探测算法包括:The peak/valley sub-pixel displacement detection algorithm described in the step eight of the camera shooting and peak/valley motion detection sub-pixel displacement procedure includes:

对于某种三基色波长,比较所述参考帧与所述比较帧里所选定的观测行与观测列之对应的峰、谷的位置,籍此判断所述观测行与观测列里对应的峰、谷的移动方向,并分别使用累加器计数跟踪所述峰、谷沿所述坐标轴的移动情况,具体地,For a certain trichromatic wavelength, compare the positions of peaks and valleys corresponding to the selected observation row and observation column in the reference frame and the comparison frame, thereby judging the corresponding peaks in the observation row and observation column , the direction of movement of the valley, and use the accumulator count to track the movement of the peak and valley along the coordinate axis, specifically,

在所述观测行像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色峰向右位移的个数的累加器N峰向右移+1,如果某一个三基色峰在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色峰向左位移的个数的累加器N峰向左移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色的谷向右位移的个数的累加器N谷向右移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色的谷向左位移的个数的累加器N谷向左移+1;In the observed row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then tracking the number of the peaks of the primary color shifted to the right The N peak of the accumulator is shifted to the right by +1. If the position of a certain three primary color peaks in the comparison frame is displaced to the left by one pixel pitch unit relative to its position in the reference frame, the number of the primary color peaks shifted to the left is tracked. The N peak of the accumulator is shifted to the left by +1; if the position of a valley of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted to the right The accumulator N of the number of valleys is moved to the right + 1, if the position of a valley of a certain three primary colors in the comparison frame is shifted to the left by one pixel pitch unit relative to its position in the reference frame, then the valley of the primary color is tracked The accumulator N valley of the number shifted to the left is shifted to the left + 1;

在所述观测列像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的峰向上位移的个数的累加器N峰向上移+1,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的峰向下位移的个数的累加器N峰向下移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的谷向上位移的个数的累加器N谷向上移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的谷向下位移的个数的累加器N谷向下移+1;In the observation row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is displaced upward by one pixel pitch unit relative to its position in the reference frame, the accumulation of the number of upward displacements of the peaks of the primary color is tracked The N peak of the detector moves up +1, if the position of a peak of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the number of the peaks of the primary color shifted downward is tracked The N peak of the accumulator moves down by +1; if the position of a valley of a certain three primary colors in the comparison frame is displaced upwards by one pixel pitch unit relative to its position in the reference frame, then tracking the valleys of the primary color is shifted upward by 1 The accumulator N valley of the number moves up +1, if the position of a valley of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted downward The number of the accumulator N valley moves down +1;

如果N峰向右移(j+1)>N峰向左移(j+1),则计为ΔX峰位移(j+1)=+0.5,If N peak shifts to the right (j+1)>N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=+0.5,

如果N峰向右移(j+1)<N峰向左移(j+1),则计为ΔX峰位移(j+1)=-0.5;If N peak shifts to the right (j+1)<N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=-0.5;

如果N谷向右移(j+1)>N谷向左移(j+1),则计为ΔX谷位移(j+1)=+0.5,If the N valley moves to the right (j+1)>N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=+0.5,

如果N谷向右移(j+1)<N谷向左移(j+1),则计为ΔX谷位移(j+1)=-0.5;If N valley moves to the right (j+1)<N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=-0.5;

Figure G2009101909241D00101
Figure G2009101909241D00101

类似地,Similarly,

如果N峰向上移(j+1)>N峰向下移(j+1),则计为ΔY峰位移(j+1)=+0.5,If the N peak moves up (j+1)>N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=+0.5,

如果N峰向上移(j+1)<N峰向下移(j+1),则计为ΔY峰位移(j+1)=-0.5,If the N peak moves up (j+1)<N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=-0.5,

如果N谷向上移(j+1)>N谷向下移(j+1),则计为ΔY谷位移(j+1)=+0.5,If N valley moves up (j+1)>N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=+0.5,

如果N谷向上移(j+1)<N谷向下移(j+1),则计为ΔY谷位移(j+1)=-0.5,If N valley moves up (j+1)<N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=-0.5,

Figure G2009101909241D00102
Figure G2009101909241D00102

上述式中,L像素-像素表示像素间距,其单位可以是μm、mm或inch等,其它符号的含义如字符所标记,例如,N即数目(Number),上述计算式均当分别应用于摄像头像素阵列的某个具体的三基色分量数据。In the above formula, L pixel-pixel represents the pixel pitch, and its unit can be μm, mm or inch, etc. The meanings of other symbols are as marked by characters, for example, N is the number (Number), and the above calculation formulas should be applied to the camera respectively A specific three primary color component data of the pixel array.

本发明的优点是,该方法以普通的计算机摄像头中的光电传感器阵列为位移传感器,根据图像帧像素的RGB数据,分别计算出三种基色的描述邻近像素之间的光强的差的边的方向信息,它以“峰”和“谷”,也就是被测物体表面的反射光图案中三种基色的物理明暗对比度分布特征,作为测量亚像素位移的依据,由于对三种基色的波长同时分别进行测量计算,又辅之以平均算法,克服了其它波长受环境光照变化的影响,在相当的程度上减小了测量的误差,所得计算结果所对应的位移测量误差为≤±0.25L像素-像素~±0.5L像素-像素;同时,该发明只取反射面图像帧对比度变化最大的一行和一列作为测量的代表,既利用了光电传感器阵列,又简化了计算工作量,保证了二维位移测量及其测量灵敏度。本发明利用了光电传感器件的光电转换快速响应特性,适合于物体运动速度与光照变化速度都远低于摄像头的拍摄帧速率的情况。The advantage of the present invention is, this method uses the photoelectric sensor array in the ordinary computer camera head as displacement sensor, according to the RGB data of image frame pixel, respectively calculates three primary colors and describes the difference of light intensity between adjacent pixels. Direction information, which uses "peaks" and "valleys", that is, the distribution characteristics of the physical light and dark contrast of the three primary colors in the reflected light pattern on the surface of the measured object, as the basis for measuring sub-pixel displacement, because the wavelengths of the three primary colors are simultaneously Measurement and calculation are carried out separately, supplemented by an average algorithm, which overcomes the influence of other wavelengths by environmental light changes, and reduces the measurement error to a considerable extent. The displacement measurement error corresponding to the obtained calculation results is ≤±0.25L pixels -pixel to ±0.5L pixel-pixel ; at the same time, the invention only takes the row and column with the largest contrast change of the image frame on the reflective surface as the representative of the measurement, which not only utilizes the photoelectric sensor array, but also simplifies the calculation workload and ensures two-dimensional Displacement measurement and its measurement sensitivity. The invention utilizes the photoelectric conversion fast response characteristic of the photoelectric sensor device, and is suitable for the situation that both the moving speed of the object and the light changing speed are far lower than the shooting frame rate of the camera.

附图说明 Description of drawings

下面结合附图进一步说明本发明。Further illustrate the present invention below in conjunction with accompanying drawing.

图1是本发明的测量装置方框图。Fig. 1 is a block diagram of the measuring device of the present invention.

图2表示一个3×3像素阵列及其根据定义导出的边方向数据。图中,一个方框表示一个像素,方框内的点的数目越多表示该像素的亮度越暗,所确定的正边和负边以粗线段表示,其中该像素阵列的外围的四个边的正、负方向还分别以箭头表示。Figure 2 shows a 3 × 3 pixel array and its derived edge orientation data by definition. In the figure, a box represents a pixel, and the larger the number of points in the box, the darker the brightness of the pixel. The determined positive and negative sides are represented by thick line segments, and the four peripheral sides of the pixel array The positive and negative directions of are also indicated by arrows, respectively.

图3表示光电探测器阵列沿着某个轴向获得的图像光电信号以及据此导出的一个序列的边方向数据(正边和负边)和边反射状况(峰和谷)。Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain axis and a sequence of side direction data (positive side and negative side) and side reflection conditions (peak and valley) derived therefrom.

图4是沿着某个轴跟踪边反射状况进行亚像素位移测量的示意图。FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis.

图5是峰/谷亚像素位移探测算法的图解。其中,箭头向上表示正边,箭头向下表示负边,虚线以及细实线均表示坐标平面图上均匀的等分线,横坐标的等分区间表示分离的各个像素范围。Figure 5 is a diagram of a peak/valley sub-pixel displacement detection algorithm. Among them, the upward arrow indicates the positive edge, and the downward arrow indicates the negative edge. Both the dotted line and the thin solid line indicate the uniform bisector on the coordinate plane, and the equal division interval of the abscissa indicates the separated range of each pixel.

图1中,1.计算机摄像头,2.光学透镜,3.光电成像芯片,4.摄像头的USB接口,5.计算机系统,6.计算机的USB接口,7.CPU,8.内存,9.显示卡与显示器,10.硬盘,11.键盘和鼠标,12.操作系统,13.摄像头驱动程序,14.摄像头拍摄及峰/谷运动探测亚像素位移程序,15.照明设备。In Fig. 1, 1. computer camera, 2. optical lens, 3. photoelectric imaging chip, 4. USB interface of camera, 5. computer system, 6. USB interface of computer, 7. CPU, 8. memory, 9. display Cards and monitors, 10. Hard drives, 11. Keyboards and mice, 12. Operating systems, 13. Camera drivers, 14. Sub-pixel displacement programs for camera capture and peak/valley motion detection, 15. Lighting.

图3中,21.光电传感器阵列探测到的一行图像光--电信号;22.光电传感器阵列探测到的一行光--电信号21经过积分器峰值电路处理以后的数字化输出信号;23.与光电传感器阵列的一行输出信号22对应的边方向数据,箭头向上表示正边,箭头向下表示负边;24.与一行边方向数据23对应的边反射状况,包括二个峰和一个谷。Among Fig. 3, 21. one row of image light-electric signal that photoelectric sensor array detects; 22. one row light that photoelectric sensor array detects-electrical signal 21 is through the digitized output signal after integrator peak value circuit processing; 23. and The edge direction data corresponding to a line of output signal 22 of the photoelectric sensor array, the upward arrow indicates a positive edge, and the downward arrow indicates a negative edge; 24. The edge reflection condition corresponding to a line of edge direction data 23 includes two peaks and a valley.

图4中,31.前一次拍摄的参考帧里一行边方向数据;32.与前一次拍摄的参考帧里一行边方向数据31对应的边反射状况,包括两个峰和一个谷;41.后一次拍摄的比较帧里一行边方向数据;42.与后一次拍摄的比较帧里一行边方向数据41对应的边反射状况,包括两个峰和一个谷;箭头向上表示正边,箭头向下表示负边。In Fig. 4, 31. a row of edge direction data in the reference frame of the previous shooting; 32. the edge reflection condition corresponding to a row of edge direction data 31 in the reference frame of the previous shooting, including two peaks and a valley; 41. after One line of edge direction data in the comparison frame of one shot; 42. The edge reflection condition corresponding to one line of edge direction data 41 in the comparison frame of the next shot, including two peaks and one valley; the upward arrow indicates the positive edge, and the downward arrow indicates negative side.

上述图2、图3和图4均借鉴自参考资料(US 7,122,781 B2,Oct.17,2006,Rotzoll et al.)。The above Figure 2, Figure 3 and Figure 4 are all borrowed from reference materials (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.).

图5中,51.光电传感器阵列探测到的一行图像光--电信号,包括一行图像光--电信号510、一行图像光--电信号511以及一行图像光--电信号512三种情况;52.一行图像光--电信号510经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示;53.~57.一行图像光--电信号510向右逐渐发生亚像素位移,经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示。In Fig. 5, 51. a line of image light-electric signal detected by the photoelectric sensor array includes three situations of one line of image light-electric signal 510, one line of image light-electric signal 511 and one line of image light-electric signal 512 ; 52. One line of image light--electrical signal 510 is processed by the integrator peak circuit and the digitized output signal and its corresponding positive side, negative side, peak and valley; 53.~57. One line of image light-electricity Signal 510 gradually undergoes a sub-pixel shift to the right, and the digital output signal after processing by the integrator peak circuit and its corresponding positive side, negative side, peak and valley are shown.

具体实施方式 Detailed ways

图1是本发明的测量装置的方框图。所述摄像头(1)由光学透镜(2)、光电成像芯片(3)和摄像头的USB接口(4)组成,通过其配置的USB电缆连接到一台普通的计算机系统(5)的USB接口(6),该计算机系统(5)还配置有CPU(7)、内存(8)、显示卡与显示器(9)、硬盘(10)、键盘和鼠标(11)、操作系统(12)、摄像头驱动程序(13)以及摄像头拍摄及峰/谷运动探测亚像素位移程序(14)。Fig. 1 is a block diagram of the measuring device of the present invention. Described camera head (1) is made up of optical lens (2), photoelectric imaging chip (3) and the USB interface (4) of camera head, is connected to the USB interface (5) of a common computer system (5) by its configured USB cable 6), the computer system (5) is also equipped with CPU (7), internal memory (8), display card and monitor (9), hard disk (10), keyboard and mouse (11), operating system (12), camera driver Program (13) and sub-pixel displacement program (14) for camera capture and peak/valley motion detection.

首先,在计算机(5)上运行随摄像头(1)配售的摄像头驱动程序(13),安装好摄像头(1)到计算机(5)。调节摄像头(1)的光学镜头(2)的焦距,使得被测物体成像清晰。First, run the camera driver (13) distributed with the camera (1) on the computer (5), and install the camera (1) to the computer (5). The focal length of the optical lens (2) of the camera (1) is adjusted so that the image of the measured object is clear.

然后,选择测量环境,或调节相关照明设备(15),尽量做到被测物体上面光照均匀且不发生大的变化。例如,可以在室内环境下进行测量,也可以选择照明光源强于环境光照的场合,还可以把照明光源投向物体前面的一个环形墙幕,再让墙幕上漫反射回来的光线照向被测物体,还可以施加三基色偏振片。Then, the measurement environment is selected, or the related lighting equipment (15) is adjusted, so that the illumination on the measured object is uniform and does not change greatly as far as possible. For example, the measurement can be carried out in an indoor environment, or the occasion where the lighting source is stronger than the ambient light can also be selected, and the lighting source can also be cast on a ring-shaped wall curtain in front of the object, and then the diffusely reflected light on the wall curtain can shine on the measured object. objects, trichroic polarizers can also be applied.

接着,运行摄像头拍摄及峰/谷运动探测亚像素位移程序(14),运用峰/谷亚像素位移探测算法实施实时测量。对此,逐项说明如下。Then, run the camera shooting and peak/valley motion detection sub-pixel displacement program (14), and use the peak/valley sub-pixel displacement detection algorithm to implement real-time measurement. This is explained item by item as follows.

(一)定义光电像素阵列的边方向数据(1) Define the edge direction data of the photoelectric pixel array

进行光强比较的像素位置关系有:1)两个毗连像素之间,2)毗连像素子阵列之间(U.S.patent application Ser.No.10/001,963,Dec.5,2001;U.S.patent application Ser.No.10/001,959,Dec.5,2001),3)非毗连邻近像素之间(参与比较的两个像素之间还有第三个像素),4)其它位置的非毗连邻近像素之间。The pixel position relationship for light intensity comparison is: 1) between two adjacent pixels, 2) between adjacent pixel sub-arrays (U.S.patent application Ser.No.10/001,963, Dec.5, 2001; U.S.patent application Ser. No.10/001,959, Dec.5, 2001), 3) between non-adjacent adjacent pixels (there is a third pixel between the two pixels participating in the comparison), 4) between non-adjacent adjacent pixels in other positions.

当两个参与比较的像素的光强相等的时候,可以采用所谓“滞回函数”(U.S.patentapplication Ser.No.10/001,963)免除边方向数据的不确定性,或者,简单地规定为正边或负边(US 7,122,781 B2)。像素阵列中的四边与四个角上的像素位置不存在边方向数据。When the light intensities of the two pixels involved in the comparison are equal, the so-called "hysteresis function" (U.S. patent application Ser. No. 10/001,963) can be used to avoid the uncertainty of the edge direction data, or simply defined as the positive edge or negative side (US 7,122,781 B2). The pixel positions on the four sides and the four corners in the pixel array do not have side direction data.

美国专利US 7,122,781 B2采用比较非毗连邻近像素之间的光强的方法来确定边方向数据,如此获得的边位于参与比较的两个像素的中间的那个像素上,并认为效果有二:1)等效于对像素的光强图实施了空间媒介过滤器,降低了噪声;2)比较非毗连邻近像素等效于比较三个毗连像素:Pi-Pi+2=(Pi+Pi+1)-(Pi+1+Pi+2),而不需要设置求和电路。一个例子如图2所示,该图表示一个3×3像素阵列以及根据上述定义导出的边方向数据。对于一个给定的光电探测器阵列,存在预定数目的边方向数据。U.S. Patent No. 7,122,781 B2 uses the method of comparing the light intensity between non-adjacent adjacent pixels to determine the edge direction data. The edge obtained in this way is located on the pixel in the middle of the two pixels participating in the comparison. It is believed that there are two effects: 1) It is equivalent to implementing a spatial medium filter on the light intensity map of a pixel to reduce noise; 2) comparing non-adjacent adjacent pixels is equivalent to comparing three adjacent pixels: P i -P i+2 =(P i +P i +1 )-(P i+1 +P i+2 ), without setting up a summation circuit. An example is shown in Figure 2, which represents a 3 × 3 pixel array with edge orientation data derived from the above definition. For a given photodetector array, there is a predetermined number of edge direction data.

本发明采用了上述比较非毗连邻近像素光强大小的边的定义,认为被测物体表面的反射特征就是所谓的表面物理明暗对比度,属于该物体表面材料的光学反射属性,在光照变化相当大的程度上不会发生明显的变化。The present invention adopts the above-mentioned definition of comparing the light intensity of non-adjacent adjacent pixels, and thinks that the reflection feature of the surface of the measured object is the so-called surface physical light-dark contrast, which belongs to the optical reflection property of the surface material of the object. There will be no significant change in degree.

考虑到计算机摄像头距离被测量物体相当远,光电传感器阵列探测到的被测物体表面反射的光强图案会受到周围环境光辐射的影响,带来像素光强值的某种随机的波动,因而本发明在关于边的定义中添加了一个误差容限值error,并认为当两者的强弱差不多的时候,例如I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error,则定义这两个像素之间不存在一个边,或者说,存在一个第三类边,这对于后述峰、谷的分析不产生影响,但进一步反映了被侧物体表面的反射特征。摄像头的灰度值深度一般为8bit,256个等级值。因此,式中的误差容限值可以根据具体的光照被干扰情况,预置为一个小的数值,例如:error=10。Considering that the computer camera is quite far away from the measured object, the light intensity pattern reflected by the surface of the measured object detected by the photoelectric sensor array will be affected by the ambient light radiation, which will bring some random fluctuations in the pixel light intensity value, so this paper The invention adds an error tolerance value error in the definition of the side, and thinks that when the strength of the two is similar, for example, I(X, Y+2)-error<I(X, Y)<I(X , Y+2)+error, it is defined that there is no edge between these two pixels, or in other words, there is a third type of edge, which does not affect the analysis of peaks and valleys described later, but further reflects the side Reflective features of the surface of an object. The gray value depth of the camera is generally 8bit, 256 levels. Therefore, the error tolerance value in the formula can be preset as a small value, for example, error=10, according to the specific situation that the light is disturbed.

为了进一步有效消除环境光的影响,本发明利用摄像头本身的三种基色(波长)的光电转换数据分量,而非其综合性的光强值,通过比较,导出红、绿和蓝三种基色各自的边方向数据。具体的边方向数据的定义见“发明内容”。In order to further effectively eliminate the influence of ambient light, the present invention uses the photoelectric conversion data components of the three primary colors (wavelengths) of the camera itself, rather than its comprehensive light intensity value, to derive the three primary colors of red, green and blue by comparison. edge direction data. For a specific definition of edge direction data, see "Summary of the Invention".

本发明首先探测被测物体表面的光学反射特征,选取沿坐标轴方向三基色的正边和负边的数目和最多者的一行与一列作为观测的对象。The invention firstly detects the optical reflection characteristics of the surface of the object to be measured, and selects a row and a column with the largest number and the largest number of positive and negative sides of the three primary colors along the direction of the coordinate axis as the object of observation.

(二)定义边反射状况(2) Define edge reflection conditions

按照选定的一行或一列像素的边方向数据,沿X轴和Y轴方向,分别导出边反射状况:连续的两个或两个以上的正边或第三类边的后面跟着一个负边,称之为第一类边反射状况,即认为在此位置存在一个光强的峰;连续的两个或两个以上的负边或第三类边的后面跟着一个正边,称之为第二类边反射状况,即认为在此位置存在一个光强的谷;除此以外,称之为第三类边反射状况。因此,分别获得红、绿和蓝三种基色(波长)的边反射状况。According to the edge direction data of a selected row or column of pixels, along the X-axis and Y-axis directions, the edge reflection conditions are respectively derived: two or more consecutive positive edges or a third type of edge followed by a negative edge, It is called the first type of edge reflection condition, that is, it is considered that there is a peak of light intensity at this position; two or more consecutive negative sides or the third type of side followed by a positive side are called the second In the case of edge reflection, it is considered that there is a valley of light intensity at this position; otherwise, it is called the third type of edge reflection. Therefore, the edge reflection conditions of the three primary colors (wavelengths) of red, green and blue are respectively obtained.

(三)光照稳定的情况下跟踪峰、谷的运动(3) Tracking the movement of peaks and valleys under stable light conditions

所谓光照稳定,意味着被测物体表面的物理明暗对比度不会随着光照的强弱或角度的变化而变化。The so-called stable light means that the physical contrast between light and dark on the surface of the measured object will not change with the intensity of light or the change of angle.

图3表示光电探测器阵列沿着某个坐标轴向获得的图像光电信号以及据此导出的一个序列的边方向数据(正边和负边)和边反射状况(峰和谷)。Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain coordinate axis and a sequence of edge direction data (positive edge and negative edge) and edge reflection conditions (peaks and valleys) derived therefrom.

图4是沿着某个轴跟踪边反射状况进行亚像素位移测量的示意图。由图可以看到,第一次拍摄获得的谷位于第四个(负)边与第五个(正)边之间,而在第二次拍摄获得的谷位于第五个(负)边与第六个(正)边之间,两次顺序拍摄期间,该谷向右运动了一个单位像素间距,文献称此情形为典型的亚像素运动(sub-pixel motion:displacement of less than the pixelpitch between two successive flashes),意思是其位移小于一个单位的像素间距。即是说,在拍摄参考帧到拍摄比较帧的时间间隔dt内,所测量的运动物体其成像像素阵列发生的位移应当小于一个像素间距单位。这可以根据被测量的物体运动的最大的速度,通过调节拍摄时间间隔dt来实现。FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis. It can be seen from the figure that the valley obtained in the first shooting is located between the fourth (negative) side and the fifth (positive) side, while the valley obtained in the second shooting is located between the fifth (negative) side and Between the sixth (positive) side, during the two sequential shots, the valley moved to the right by a unit pixel pitch. The literature calls this situation a typical sub-pixel motion (sub-pixel motion: displacement of less than the pixelpitch between two successive flashes), which means that the displacement is less than one unit of pixel pitch. That is to say, within the time interval dt from shooting the reference frame to shooting the comparison frame, the measured displacement of the imaging pixel array of the moving object should be less than one pixel pitch unit. This can be achieved by adjusting the recording time interval dt according to the maximum velocity of the object being measured.

只跟踪边反射状况而不论其类型,在跟踪过程中会发生信息与测量精度的损失。By tracking only edge reflection conditions regardless of their type, a loss of information and measurement accuracy occurs during the tracking process.

因此,对应每一个坐标轴各设置2×2个累加器,沿着坐标轴的方向分别跟踪峰、谷各自向正的或向负的坐标轴方向发生运动的个数,方法是:比较参考帧与比较帧里所选定的像素行与列之对应的峰、谷的位置,籍此判断所选定的像素行与列里对应的峰、谷的移动方向。沿每个坐标轴再分别设置2个累加器,计数每个坐标轴上峰、谷各自的总的数目。具体说明见“发明内容”。Therefore, 2×2 accumulators are set corresponding to each coordinate axis, and the number of peaks and valleys moving towards the positive or negative coordinate axis direction is tracked respectively along the direction of the coordinate axis. The method is: compare the reference frame The positions of peaks and valleys corresponding to the selected pixel rows and columns in the frame are compared with the corresponding peaks and valleys, so as to determine the moving direction of the corresponding peaks and valleys in the selected pixel rows and columns. Two accumulators are respectively set along each coordinate axis to count the respective total numbers of peaks and valleys on each coordinate axis. See "Content of the Invention" for details.

位移遵守矢量合成的原则。作为像素矩阵,还可以沿着X轴、Y轴的对角线方向探测边反射状况的运动。因此,一个对应的边反射状况可以源自:1)同轴上左右两个最邻近位置,2)对角线上四个最邻近位置之一。The displacement follows the principle of vector composition. As a matrix of pixels, it is also possible to detect movement of edge reflections along the diagonal directions of the X-axis and Y-axis. Thus, a corresponding edge reflection condition can originate from: 1) the two nearest neighbors left and right on the coaxial line, and 2) one of the four nearest neighbors on the diagonal.

(四)峰/谷亚像素位移探测算法(4) Peak/valley sub-pixel displacement detection algorithm

图3中,光电传感器阵列沿着某个坐标轴方向探测得到的光强图案21属于被测量物体的表面反射性质,光照不变的情况下,该光强图案不会发生变化。被测物体表面相对摄像头发生位移,该光强图案会相应地向左或向右移动。假设拍摄的时间间隔内该被测物体只发生了一次一个方向的位移,并且,发生的位移小于1个像素间距,便会出现图4所示的情形,峰、谷只朝一个方向移动。In FIG. 3 , the light intensity pattern 21 detected by the photoelectric sensor array along a certain coordinate axis belongs to the surface reflection property of the measured object, and the light intensity pattern will not change under the condition of constant illumination. When the surface of the measured object is displaced relative to the camera, the light intensity pattern will move to the left or right accordingly. Assuming that the measured object only shifts in one direction once during the shooting time interval, and the displacement is less than 1 pixel pitch, the situation shown in Figure 4 will appear, and the peaks and valleys only move in one direction.

图5进一步考察了光电传感器阵列探测到的一行图像光--电信号510逐步发生向右位移的情形。该行图像光--电信号510经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示52表示了最初的情形,该行图像光--电信号510向右逐渐发生亚像素位移,经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示53~57分别表示了随后发生的亚像素位移情形。由图可见,在发生半个像素大小的位移的时候,其峰(图5中标号54-D--E处)的右邻的负边变成了第三类边(图5中标号54--E处),但该峰的位置没有发生变化;当发生了大于半个像素大小的位移的时候,该峰的右邻的负边经过了第三类边(图5中标号54--E处)的变化过程变化成正边(图5中标号55--E处),该峰的位置也向右移动了一个像素单位。考察非对称的一行图像光--电信号511或512,当其逐渐向右位移的时候,也有相同的结论。此时发生的亚像素位移可以计为0.5pixcel,测量误差在±0.25pixcel~±0.5pixcel之间。FIG. 5 further examines the situation that a line of image light-electric signal 510 detected by the photoelectric sensor array gradually shifts to the right. This line of image light--electrical signal 510 is processed by the digitized output signal and its corresponding positive side, negative side, peak and valley after the integrator peak circuit 510 shows the initial situation. Signal 510 gradually undergoes sub-pixel displacement to the right, and the digitized output signal after processing by the integrator peak circuit and its corresponding positive side, negative side, and peak and valley diagrams 53-57 represent the subsequent sub-pixel displacement respectively. . As can be seen from the figure, when a displacement of half a pixel size takes place, the negative edge on the right side of its peak (label 54-D--E in Figure 5) becomes the third type of edge (label 54-- in Figure 5). -E), but the position of the peak does not change; when a displacement greater than half a pixel size occurs, the negative edge of the right neighbor of the peak passes through the third type of edge (label 54--E in Figure 5 The change process of the peak) changes into a positive side (mark 55--E in Figure 5), and the position of the peak also moves to the right by one pixel unit. The same conclusion can be reached when examining the asymmetrical image light-electric signal 511 or 512 of a row when it gradually shifts to the right. The sub-pixel displacement that occurs at this time can be counted as 0.5 pixcel, and the measurement error is between ±0.25 pixcel and ±0.5 pixcel.

由于被测物体属于刚性固体,当光电像素阵列与被测表面发生相对位移的时候,所有的峰和所有的谷都应当向着同一个方向移动。实际上,在拍摄间歇期dt内被测物体发生的位移虽然小于1个像素间距,但是,它可能既朝左又朝右发生了多次位移,因此,峰、谷有朝左的移动,又有朝右的移动。但是,综合起来,结果应当还是所有的峰和所有的谷都向着同一个方向移动。如果峰(或谷)同时有向左和向右的累计数目,两者应当相差悬殊,数目少的反映了光照变化对被测表面的反射对比度特征的影响。因此,Since the measured object is a rigid solid, when the photopixel array is displaced relative to the measured surface, all peaks and all valleys should move in the same direction. In fact, although the displacement of the measured object during the shooting interval dt is less than 1 pixel pitch, it may have multiple displacements both to the left and to the right. Therefore, the peaks and valleys move to the left, and There is a move to the right. However, taken together, the result should still be that all peaks and all valleys move in the same direction. If the peaks (or valleys) have both leftward and rightward cumulative numbers, the two should be very different, and the smaller number reflects the impact of illumination changes on the reflection contrast characteristics of the measured surface. therefore,

如果N峰向右移(j+1)>N峰向左移(j+1),则计为ΔX峰位移(j+1)=+0.5,If N peak shifts to the right (j+1)>N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=+0.5,

如果N峰向右移(j+1)<N峰向左移(j+1),则计为ΔX峰位移(j+1)=-0.5;If N peak shifts to the right (j+1)<N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=-0.5;

如果N谷向右移(j+1)>N谷向左移(j+1),则计为ΔX谷位移(j+1)=+0.5,If the N valley moves to the right (j+1)>N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=+0.5,

如果N谷向右移(j+1)<N谷向左移(j+1),则计为ΔX谷位移(j+1)=-0.5;If N valley moves to the right (j+1)<N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=-0.5;

Figure G2009101909241D00151
Figure G2009101909241D00151

对于Y轴方向也会有类似的结论。A similar conclusion can be drawn for the Y-axis direction.

(五)光照发生变化的情况下跟踪峰、谷的运动(5) Tracking the movement of peaks and valleys when the light changes

图3中,如果光照发生变化,光电传感器阵列沿着某个坐标轴方向探测得到的光强图案21不再只是属于被测量物体的表面反射性质,还要反映出环境光辐射发生改变的影响效果。In Figure 3, if the illumination changes, the light intensity pattern 21 detected by the photoelectric sensor array along a certain coordinate axis no longer only belongs to the surface reflection properties of the measured object, but also reflects the impact of the change in ambient light radiation .

一种情况是,该光电传感器阵列探测到的光强图案虽然发生了变化,但是,被测物体表面的反射属性仍然占主要地位,传感器先后两次拍摄“看”到的该坐标轴方向的边方向数据以及边反射状况的数目并没有发生明显的变化。One situation is that although the light intensity pattern detected by the photoelectric sensor array has changed, the reflective properties of the surface of the measured object still dominate, and the sensor takes two shots of the edge in the direction of the coordinate axis that it "sees". The orientation data and the number of edge reflection conditions did not change significantly.

另一种情况是,被测物体整体上的光照发生变化,或其局部表面突然被环境光强烈地改变,导致图3中光电传感器阵列探测到的光强图案21发生了值得关注的变化,传感器先后两次拍摄“看”到的该轴方向的边方向数据以及边反射状况的数目会发生明显的变化。这时,重新开始测量,包括重新选取观测行与观测列。由于摄像头拍摄的速率远快于被测物体的位移速度,也远快于环境光的辐照变化速度,加上所述峰/谷亚像素位移探测算法只针对光电传感器阵列里的一行和一列像素,计算速度相对地快,只要被测物体表面的光学反射属性依然在成像帧里占主要地位,仍然可以使用上述峰/谷亚像素位移探测算法表示式来计算即时的位移值,这项措施对于测量结果不会带来多少误差。Another situation is that the overall illumination of the measured object changes, or its local surface is suddenly strongly changed by ambient light, resulting in a noteworthy change in the light intensity pattern 21 detected by the photoelectric sensor array in Figure 3, the sensor The edge direction data of the axis direction and the number of edge reflection conditions "viewed" by two consecutive shots will change obviously. At this time, restart the measurement, including reselecting the observation row and observation column. Since the shooting speed of the camera is much faster than the displacement speed of the measured object, it is also much faster than the irradiance change speed of the ambient light, and the peak/valley sub-pixel displacement detection algorithm only targets one row and one column of pixels in the photosensor array , the calculation speed is relatively fast, as long as the optical reflection properties of the surface of the measured object still dominate the imaging frame, the above-mentioned peak/valley sub-pixel displacement detection algorithm expression can still be used to calculate the real-time displacement value. The measurement results will not bring much error.

上述两种变化情况可以统一起来进行检查,或者探测边反射状况的数目,或者探测文献所述的“鬼影边反射状况”的数目。采用文献所述的“鬼影边反射状况”的“预置阈值”来检验,不如采用其相对该轴向所有的峰、谷数目的百分比来得方便和灵活。The above two variations can be checked together, either to detect the number of edge reflection conditions, or to detect the number of "ghost edge reflection conditions" described in the literature. It is more convenient and flexible to use the "preset threshold" of "ghost edge reflection" described in the literature to check than to use its percentage relative to the number of peaks and valleys in this axis.

需要被测物体表面的光学反射属性在成像帧里占主要地位,意味着峰/谷亚像素位移探测算法能够识别被测物体的特征,这是本发明的关键所在;这与被测物体表面的光学性质(材料的质地、纹路等)、照明光源以及测量环境(干扰光源、灰尘、烟雾)等因素有关。如果选用表面质地的光学反射属性较易辨析的材料作靶标,或者控制被测物体的光照免遭环境的过分干扰,可以取得较好的测量效果。The optical reflection properties of the surface of the measured object are required to take a dominant position in the imaging frame, which means that the peak/valley sub-pixel displacement detection algorithm can identify the characteristics of the measured object, which is the key of the present invention; Optical properties (material texture, texture, etc.), lighting source and measurement environment (interfering light source, dust, smoke) and other factors are related. Better measurement results can be obtained if a material whose optical reflection property of the surface texture is easier to distinguish is selected as the target, or if the light of the measured object is controlled to avoid excessive interference from the environment.

摄像头的输出包括三种基色的分量值,因此,分别测量这三种基色的边反射状况,并加以平均,可以克服其它波长的环境杂散光变化对测量的影响。The output of the camera includes the component values of the three primary colors. Therefore, measuring the edge reflections of the three primary colors and averaging them can overcome the influence of ambient stray light changes of other wavelengths on the measurement.

实际上,注意到,较之红外线二极管照明的光电鼠标,采用激光二极管照明的光电鼠标能够适应光滑的瓷砖表面。如果能够应用某种不易受环境杂散光源影响的波长的光源,配合选用敏感该波长的光电成像传感器阵列(摄像头),本发明能够充分地发挥适应较复杂的光照环境的能力,取得无接触测量物体微小位移的优良效果。In fact, it was noticed that the optical mouse with laser diode illumination was able to adapt to the smooth tile surface compared to the optical mouse with infrared diode illumination. If a light source with a wavelength that is not easily affected by environmental stray light sources can be used, and a photoelectric imaging sensor array (camera) sensitive to this wavelength can be selected, the present invention can fully exert its ability to adapt to more complex lighting environments, and achieve non-contact measurement. Excellent effect of small displacement of objects.

Claims (4)

1.一种测量亚像素位移的峰谷运动探测方法,它通过一台普通的计算机、一个计算机摄像头测量位移,所述摄像头通过USB接口连接到所述计算机,该计算机配置有内存、CPU、硬盘、显示卡与显示器、键盘和鼠标、操作系统以及摄像头驱动程序,其特征在于,该方法通过摄像头拍摄并采用峰/谷亚像素位移探测算法测量亚像素位移,包括下述测量步骤:1. a kind of peak-valley motion detection method of measuring sub-pixel displacement, it measures displacement by a common computer, a computer camera, and described camera is connected to described computer by USB interface, and this computer is equipped with internal memory, CPU, hard disk , display card and monitor, keyboard and mouse, operating system and camera driver, it is characterized in that, the method is taken by camera and adopts peak/valley sub-pixel displacement detection algorithm to measure sub-pixel displacement, comprising the following measurement steps: 步骤一、拍摄一帧被测物体:M行×N列,不考虑该图像帧的边上的两行与两列,得到参考帧:(M-2)行×(N-2)列,为该参考帧以及其后所有拍摄的图像帧确定相同的坐标轴,其中,M、N∈正整数;Step 1. Take a frame of the measured object: M rows×N columns, regardless of the two rows and two columns on the side of the image frame, and obtain a reference frame: (M-2) rows×(N-2) columns, which is The reference frame and all image frames captured thereafter determine the same coordinate axis, where M, N∈positive integers; 步骤二、对于上述参考帧之像素阵列,根据其红色、绿色和蓝色分量的数据,分别逐行、逐列导出沿X轴和Y轴方向的红色的、绿色的和蓝色的边方向数据;Step 2. For the pixel array of the above reference frame, according to the data of the red, green and blue components, export the red, green and blue edge direction data along the X-axis and Y-axis direction row by row and column by column respectively ; 步骤三、分别逐行、逐列计算红色的、绿色的和蓝色的正边和负边的数目之和,取其中正边和负边的数目之和最多者所在的行与列分别作为观测行和观测列;Step 3. Calculate the sum of the numbers of positive and negative sides of red, green and blue respectively row by row and column by column, and take the row and column where the sum of the number of positive and negative sides is the largest as the observation rows and columns of observations; 步骤四、根据所选观测行与观测列的像素的三种基色的边方向数据,分别导出沿X轴和Y轴方向的三种基色的边反射状况,并分别使用累加器计数它们所对应的峰、谷的数目:NRX轴峰(j)、NRX轴谷(j)、NRY轴峰(j)和NRY轴谷(j),NGX轴峰(j)、NGX轴谷(j)、NGY轴峰(j)和NGY轴谷(j),NBX轴峰(j)、NBX轴谷(j)、NBY轴峰(j)和NBY轴谷(j),其中,j表示拍摄获得的帧的顺序计数值,j∈正整数,N表示数目,R、G、B分别表示红、绿和蓝色;Step 4. According to the edge direction data of the three primary colors of the pixels in the selected observation row and observation column, respectively derive the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the accumulator to count their corresponding Number of peaks and valleys: N RX axis peak (j), N RX axis valley (j), N RY axis peak (j) and N RY axis valley (j), N GX axis peak (j), N GX axis valley (j), N GY axis peak (j) and N GY axis valley (j), N BX axis peak (j), N BX axis valley (j), N BY axis peak (j) and N BY axis valley (j ), wherein, j represents the sequence count value of the frames obtained by shooting, j ∈ positive integer, N represents the number, and R, G, B represent red, green and blue respectively; 步骤五、拍摄了所述参考帧之后,经过一定的间歇时间dt,拍摄第二帧被测物体的像:M行×N列,去掉像素帧边缘的两行和两列得到比较帧:(M-2)行×(N-2)列;沿着步骤三中选定的观测行或观测列,分别导出沿X轴和Y轴方向的三种基色的边方向数据和边反射状况,并分别使用累加器计数沿着X轴和Y轴方向的峰、谷的数目:NRX轴峰(j+1)、NRX轴谷(j+1)、NRY轴峰(j+1)和NRY轴谷(j+1),NGX轴峰(j+1)、NGX轴谷(j+1)、NGY轴峰(j+1)和NGY轴谷(j+1),NBX轴峰(j+1)、NBX轴谷(j+1)、NBY轴峰(j+1)和NBY轴谷(j+1);Step 5, after photographing the reference frame, after a certain intermittent time dt, photograph the image of the second frame of the measured object: M rows×N columns, remove two rows and two columns at the edge of the pixel frame to obtain a comparison frame: (M -2) row × (N-2) column; along the observation row or observation column selected in step 3, respectively derive the edge direction data and edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and respectively Use accumulators to count the number of peaks and valleys along the X and Y axes: N RX axis peak (j+1), N RX axis valley (j+1), N RY axis peak (j+1), and N RY axis valley (j+1), N GX axis peak (j+1), N GX axis valley (j+1), N GY axis peak (j+1) and N GY axis valley (j+1), N BX axis peak (j+1), N BX axis valley (j+1), N BY axis peak (j+1) and N BY axis valley (j+1); 步骤六、对比所述参考帧与所述比较帧中所选观测行与观测列的三种基色的边反射状况的位置,检查其中的那些不知道来自何处的“鬼影边反射状况”---在所述比较帧的所选观测行或观测列发现的这些峰或谷,在所述参考帧的所选观测行或观测列的相应位置及其附近的一个像素间距范围内发生缺失,用六个累加器分别统计和跟踪它们的数目:NRX鬼影边(j+1)、NRY鬼影边(j+1)、NGX鬼影边(j+1)、NGY鬼影边(j+1)、NBX鬼影边(j+1)、NBY鬼影边(j+1);Step 6. Compare the positions of the edge reflection conditions of the three primary colors in the selected observation row and observation column in the reference frame and the comparison frame, and check those "ghost edge reflection conditions" that do not know where they come from- -- these peaks or valleys found in the selected observation row or observation column of the said comparison frame are missing within a range of one pixel pitch near the corresponding position of the selected observation row or observation column of the said reference frame, Use six accumulators to count and track their numbers: N RX ghost edges (j+1), N RY ghost edges (j+1), N GX ghost edges (j+1), N GY ghost edges Edge (j+1), N BX Ghost Edge (j+1), N BY Ghost Edge (j+1); 如果三种基色中有两种或两种以上的基色的“鬼影边反射状况”满足:If the "ghost edge reflection condition" of two or more of the three primary colors satisfies:
Figure FSB00000934281600021
Figure FSB00000934281600021
Figure FSB00000934281600022
or
Figure FSB00000934281600022
则认为测量“失去跟踪”,并发出一个警告信号,式中,误差容限值error1表示比较帧中沿着所选观测行或观测列,红、绿和蓝三种基色之一的鬼影边反射状况发生的数目占该基色的全部边反射状况数目的百分比,根据物体表面的光学性质和测量环境来预置;所述比较帧的边上的边反射状况不考虑“鬼影边反射状况”,它们有可能来自光电像素阵列的外部;Then the measurement is considered "lost tracking" and a warning signal is issued, where the error tolerance value error1 represents the ghost edge of one of the three primary colors red, green and blue along the selected observation row or observation column in the comparison frame The percentage of the number of reflective conditions to the total number of edge reflective conditions of the primary color is preset according to the optical properties of the object surface and the measurement environment; the edge reflective conditions on the edges of the comparison frame do not consider the "ghost edge reflective condition" , they may come from the outside of the photopixel array; 步骤七、如果测量“失去跟踪”,回到步骤一,重新开始测量工作;Step 7. If the measurement "lost tracking", go back to step 1 and restart the measurement work; 步骤八、如果测量没有“失去跟踪”,则继续本次测量工作:沿着所选观测行与所选观测列,根据摄像头的三基色分量数据,运用峰/谷亚像素位移探测算法,分别计算三基色分量的亚像素位移:Step 8. If there is no "lost tracking" in the measurement, continue the measurement work: along the selected observation row and the selected observation column, according to the three primary color component data of the camera, use the peak/valley sub-pixel displacement detection algorithm to calculate respectively Subpixel displacement of three primary color components:
Figure FSB00000934281600023
Figure FSB00000934281600023
Figure FSB00000934281600024
Figure FSB00000934281600024
Figure FSB00000934281600025
Figure FSB00000934281600025
Figure FSB00000934281600026
Figure FSB00000934281600026
Figure FSB00000934281600027
Figure FSB00000934281600027
Figure FSB00000934281600028
Figure FSB00000934281600028
本次测量中,总的亚像素位移计算式是:In this measurement, the total sub-pixel displacement calculation formula is:
Figure FSB00000934281600029
Figure FSB00000934281600029
步骤九、本次测量中,物体的速度矢量计算式为:Step 9. In this measurement, the velocity vector calculation formula of the object is:
Figure FSB00000934281600032
Figure FSB00000934281600032
步骤十、以步骤五中的比较帧作为新的参考帧,距拍摄此帧一定的间歇时间dt,重新拍摄一帧作为新的比较帧,即跳到步骤五,继续测量。Step 10. Take the comparison frame in step 5 as a new reference frame, take a certain interval time dt from the shooting of this frame, and re-shoot a frame as a new comparison frame, that is, skip to step 5 and continue the measurement.
2.根据权利要求1所述的测量亚像素位移的峰谷运动探测方法,其特征在于,所述测量步骤二所述红色、绿色或蓝色的边方向数据的定义是:2. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, wherein the definition of the red, green or blue edge direction data in the measurement step 2 is: 根据像素阵列中红色、绿色或蓝色这三种基色之一的分量数据,沿着X轴或者沿着Y轴方向,如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要小一个误差容限值error,即如果According to the component data of one of the three primary colors of red, green or blue in the pixel array, along the X-axis or along the Y-axis direction, if a pixel has a value of a certain three-primary color component that is greater than that of the second pixel behind it The value of the three primary color components is smaller by an error tolerance value error, that is, if I(X,Y)<I(X+2,Y)-error或I(X,Y)<I(X,Y+2)-errorI(X, Y) red < I red (X+2, Y)-error or I(X, Y) red < I(X, Y+2) red -error I(X,Y)绿<I绿(X+2,Y)-error或I(X,Y)绿<I(X,Y+2)绿-errorI(X,Y) Green < IGreen (X+2,Y)-error or I(X,Y) Green <I(X,Y+2) Green- error I(X,Y)<I(X+2,Y)-error或I(X,Y)<I(X,Y+2)-errorI(X, Y) blue < I blue (X+2, Y)-error or I(X, Y) blue < I(X, Y+2) blue- error 则定义这两个像素之间存在一个红色的、绿色的或蓝色的正边;如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要大一个误差容限值error,即如果  I(X,Y)>I(X+2,Y)+error或I(X,Y)>I(X,Y+2)+errorIt is defined that there is a red, green or blue positive edge between these two pixels; if the value of a certain three-primary color component of a pixel is one larger than the corresponding three-primary color component value of the second pixel behind it Error tolerance value error, that is, if I (X, Y) red > I red (X+2, Y) + error or I (X, Y) red > I (X, Y + 2) red + error I(X,Y)绿>I绿(X+2,Y)+error或I(X,Y)绿>I(X,Y+2)绿+errorI(X, Y) green > I green (X+2, Y) + error or I (X, Y) green > I (X, Y + 2) green + error I(X,Y)>I(X+2,Y)+error或I(X,Y)>I(X,Y+2)+errorI (X, Y) blue > I blue (X+2, Y) + error or I (X, Y) blue > I (X, Y + 2) blue + error 则定义这两个像素之间存在一个红色的、绿色的或蓝色的负边;如此获得的边位于该像素之后的第一个像素的位置,也即位于参与比较的两个像素的中间位置的那个像素上;如果一个像素的某种三基色分量值与其后面的第二个像素相应的三基色分量值接近,其RGB分量值相差不超过一个误差容限值error,即如果Then it is defined that there is a red, green or blue negative edge between these two pixels; the edge thus obtained is located at the position of the first pixel after this pixel, that is, at the middle position of the two pixels participating in the comparison on that pixel; if the value of a certain three-primary color component of a pixel is close to the corresponding three-primary color component value of the second pixel behind it, the difference between its RGB component values does not exceed an error tolerance value error, that is, if I(X+2,Y)-error<I(X,Y)<I(X+2,Y)+errorI(X+2, Y) red -error<I(X, Y) red <I(X+2, Y) red +error 或I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error;Or I(X, Y+2) red -error<I(X, Y) red <I(X, Y+2) red +error; I(X+2,Y)绿-error<I(X,Y)绿<I(X+2,Y)绿+errorI(X+2,Y) green -error<I(X,Y) green <I(X+2,Y) green +error or I(X,Y+2)绿-error<I(X,Y)绿<I(X,Y+2)绿+error;I(X, Y+2) green -error<I(X, Y) green <I(X, Y+2) green +error; I(X+2,Y)-error<I(X,Y)<I(X+2,Y)+errorI(X+2, Y) blue -error<I(X, Y) blue <I(X+2, Y) blue +error 或I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error;Or I(X, Y+2) blue -error<I(X, Y) blue <I(X, Y+2) blue +error; 则认为这两个像素之间不存在该颜色波长对应的“边”,或称之为第三类该颜色的边;沿着某一个坐标轴方向,所有的红色的正边和红色的负边以及第三类红色的边组成该方向红色的边方向数据,所有的绿色的正边和绿色的负边以及第三类绿色的边组成该方向绿色的边方向数据,所有的蓝色的正边和蓝色的负边以及第三类蓝色的边组成该方向蓝色的边方向数据;根据具体的光照情况,预置上式中的误差容限值error为一个小的数值;像素阵列中的四个边与角上的像素位置不存在边方向数据。It is considered that there is no "edge" corresponding to the color wavelength between the two pixels, or the edge of the third type of color; along a certain coordinate axis, all red positive edges and red negative edges And the third type of red edge constitutes the red edge direction data of this direction, all the green positive edges and green negative edges and the third type of green edge form the green edge direction data of this direction, all the blue positive edges and the blue negative edge and the third type of blue edge form the blue edge direction data in this direction; according to the specific lighting situation, the error tolerance value error in the above formula is preset to a small value; in the pixel array There is no edge direction data for the pixel positions on the four sides and corners of . 3.根据权利要求1所述的测量亚像素位移的峰谷运动探测方法,其特征在于,所述测量步骤四和步骤五中所述沿X轴和Y轴方向的三种基色的边反射状况,其定义为:3. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, characterized in that, the edge reflection conditions of the three primary colors along the X-axis and Y-axis directions described in the measurement step 4 and step 5 , which is defined as: 根据所选观测行与观测列的像素的红色、绿色或蓝色的边方向数据,沿着X轴或沿着Y轴方向,如果连续的两个或多于两个的某种基色的正边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的负边,称之为该基色的第一类边反射状况,即认为在此位置存在一个该基色的峰;如果连续的两个或多于两个的某种基色的负边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的正边,称之为该基色的第二类边反射状况,即认为在此位置存在一个该基色的谷;According to the red, green or blue side direction data of the pixels in the selected observation row and observation column, along the X axis or along the Y axis direction, if there are two or more consecutive positive sides of a certain primary color , or two or more consecutive third-type edges of the base color followed by a negative edge of the base color, which is called the first-type edge reflection condition of the base color, that is, it is considered that there is a base color at this position peak; if two or more consecutive negative sides of a certain basic color, or two or more consecutive third-type sides of this basic color are followed by a positive side of this basic color, it is called It is the second type of edge reflection condition of the base color, that is, it is considered that there is a valley of the base color at this position; 4.根据权利要求1所述的测量亚像素位移的峰谷运动探测方法,其特征在于,所述测量步骤八中所述的峰/谷亚像素位移探测算法包括:4. The peak-to-valley motion detection method for measuring sub-pixel displacement according to claim 1, characterized in that, the peak/valley sub-pixel displacement detection algorithm described in the measuring step eight comprises: 对于某种三基色,比较所述参考帧与所述比较帧里所选定的观测行与观测列之对应的峰、谷的位置,据此判断所述观测行与观测列里对应的峰、谷的移动方向,并分别使用累加器计数跟踪所述峰、谷沿所述坐标轴的移动情况,具体地,For a certain three primary colors, compare the positions of peaks and valleys corresponding to the selected observation row and observation column in the reference frame and the comparison frame, and judge the corresponding peaks and valleys in the observation row and observation column accordingly. The direction of movement of the valley, and use the accumulator count to track the movement of the peak and valley along the coordinate axis, specifically, 在所述观测行像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色峰向右位移的个数的累加器N峰向右移+1,如果某一个三基色峰在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色峰向左位移的个数的累加器N峰向左移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色的谷向右位移的个数的累加器N谷向右移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色的谷向左位移的个数的累加器N谷向左移+1;In the observed row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then tracking the number of the peaks of the primary color shifted to the right The N peak of the accumulator is shifted to the right by +1. If the position of a certain three primary color peaks in the comparison frame is displaced to the left by one pixel pitch unit relative to its position in the reference frame, the number of the primary color peaks shifted to the left is tracked. The N peak of the accumulator is shifted to the left by +1; if the position of a valley of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted to the right The accumulator N of the number of valleys is moved to the right + 1, if the position of a valley of a certain three primary colors in the comparison frame is shifted to the left by one pixel pitch unit relative to its position in the reference frame, then the valley of the primary color is tracked The accumulator N valley of the number shifted to the left is shifted to the left + 1; 在所述观测列像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的峰向上位移的个数的累加器N峰向上移+1,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的峰向下位移的个数的累加器N峰向下移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的谷向上位移的个数的累加器N谷向上移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的谷向下位移的个数的累加器N谷向下移+1;In the observation row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is displaced upward by one pixel pitch unit relative to its position in the reference frame, the accumulation of the number of upward displacements of the peaks of the primary color is tracked The N peak of the detector moves up +1, if the position of a peak of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the number of the peaks of the primary color shifted downward is tracked The N peak of the accumulator moves down by +1; if the position of a valley of a certain three primary colors in the comparison frame is displaced upwards by one pixel pitch unit relative to its position in the reference frame, then tracking the valleys of the primary color is shifted upward by 1 The accumulator N valley of the number moves up +1, if the position of a valley of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted downward The number of the accumulator N valley moves down +1; 如果N峰向右移(j+1)>N峰向左移(j+1),则计为ΔX峰位移(j+1)=+0.5,If N peak shifts to the right (j+1)>N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=+0.5, 如果N峰向右移(j+1)<N峰向左移(j+1),则计为ΔX峰位移(j+1)=-0.5;If N peak shifts to the right (j+1)<N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=-0.5; 如果N谷向右移(j+1)>N谷向左移(j+1),则计为ΔX谷位移(j+1)=+0.5,If the N valley moves to the right (j+1)>N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=+0.5, 如果N谷向右移(j+1)<N谷向左移(j+1),则计为ΔX谷位移(j+1)=-0.5;If N valley moves to the right (j+1)<N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=-0.5;
Figure FSB00000934281600051
Figure FSB00000934281600051
类似地,Similarly, 如果N峰向上移(j+1)>N峰向下移(j+1),则计为ΔY峰位移(j+1)=+0.5,If the N peak moves up (j+1)>N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=+0.5, 如果N峰向上移(j+1)<N峰向下移(j+1),则计为ΔY峰位移(j+1)=-0.5,If the N peak moves up (j+1)<N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=-0.5, 如果N谷向上移(j+1)>N谷向下移(j+1),则计为ΔY谷位移(j+1)=+0.5,If N valley moves up (j+1)>N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=+0.5, 如果N谷向上移(j+1)<N谷向下移(j+1),则计为ΔY谷位移(j+1)=-0.5,If N valley moves up (j+1)<N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=-0.5,
Figure FSB00000934281600052
Figure FSB00000934281600052
上述式中,L像素-像素表示像素间距,其单位是μm、mm或inch之一,N即数目,其它符号的含义如字符所标记,上述计算式均当分别应用于摄像头像素阵列的某个具体的三基色分量数据。In the above formula, L pixel-pixel represents the pixel pitch, and its unit is one of μm, mm or inch, and N is the number. The meanings of other symbols are as marked by the characters. The above calculation formulas should be applied to a certain pixel array of the camera respectively. Specific three primary color component data.
CN 200910190924 2009-09-22 2009-09-22 Peak-valley motion detection method and device for measuring sub-pixel displacement Expired - Fee Related CN102022981B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200910190924 CN102022981B (en) 2009-09-22 2009-09-22 Peak-valley motion detection method and device for measuring sub-pixel displacement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200910190924 CN102022981B (en) 2009-09-22 2009-09-22 Peak-valley motion detection method and device for measuring sub-pixel displacement

Publications (2)

Publication Number Publication Date
CN102022981A CN102022981A (en) 2011-04-20
CN102022981B true CN102022981B (en) 2013-04-03

Family

ID=43864565

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200910190924 Expired - Fee Related CN102022981B (en) 2009-09-22 2009-09-22 Peak-valley motion detection method and device for measuring sub-pixel displacement

Country Status (1)

Country Link
CN (1) CN102022981B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102052900B (en) * 2009-11-02 2013-09-25 重庆工商大学 Peak valley motion detection method and device for quickly measuring sub-pixel displacement

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1135062A (en) * 1995-03-02 1996-11-06 惠普公司 Freehand image scanning device and method
US6249591B1 (en) * 1997-08-25 2001-06-19 Hewlett-Packard Company Method and apparatus for control of robotic grip or for activating contrast-based navigation
CN1350633A (en) * 1999-05-14 2002-05-22 北京三维世界科技有限公司 3D-imaging system
CN1363827A (en) * 2000-12-08 2002-08-14 格莱特格-曼克贝斯公开股份有限公司 Individual element photoelectric measuring device for plane object

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1135062A (en) * 1995-03-02 1996-11-06 惠普公司 Freehand image scanning device and method
US6249591B1 (en) * 1997-08-25 2001-06-19 Hewlett-Packard Company Method and apparatus for control of robotic grip or for activating contrast-based navigation
CN1350633A (en) * 1999-05-14 2002-05-22 北京三维世界科技有限公司 3D-imaging system
CN1363827A (en) * 2000-12-08 2002-08-14 格莱特格-曼克贝斯公开股份有限公司 Individual element photoelectric measuring device for plane object

Also Published As

Publication number Publication date
CN102022981A (en) 2011-04-20

Similar Documents

Publication Publication Date Title
US7557935B2 (en) Optical coordinate input device comprising few elements
US5793357A (en) Device and method for determining movement of a surface
CN101971072B (en) Image sensor and focus detection apparatus
Callenberg et al. Low-cost SPAD sensing for non-line-of-sight tracking, material classification and depth imaging
CN104395694B (en) Motion sensor device with multiple light sources
TWI482054B (en) High resolution and high sensitivity cursor maneuvering device using multiple color light sources
JP6509027B2 (en) Object tracking device, optical apparatus, imaging device, control method of object tracking device, program
JP6299983B2 (en) Motion sensor device having a plurality of light sources
CN112924028B (en) A light field polarization imaging detection system for oil spills on the sea surface
US6907672B2 (en) System and method for measuring three-dimensional objects using displacements of elongate measuring members
Tabata et al. High-speed 3D sensing with three-view geometry using a segmented pattern
CN111707413A (en) A Centroid Detection Method Based on Single Pixel Detector
GB2272763A (en) Device and method for determining movement
US20100110209A1 (en) Fast motion measurement device for gaming
TWI421737B (en) Methods for measuring relative motion between a surface and an optical sensing device
CN101452349B (en) Cursor control device, method and image system on image display device
CN108269270A (en) Sun detection, tracks of device, method and application
CN101216745A (en) Optical point plane position detection system based on linear array CCD
CN102022981B (en) Peak-valley motion detection method and device for measuring sub-pixel displacement
CN115598744A (en) High-dimensional light field event camera based on micro-lens array and extraction method
GB2353856A (en) High resolution detector array
WO2022016412A1 (en) Depth information image acquisition apparatus and electronic device
CN111583191A (en) Refraction feature detection method based on light field EPI Fourier transform
US20190346598A1 (en) Imaging systems and methods with periodic gratings with homologous pixels
CN102052900B (en) Peak valley motion detection method and device for quickly measuring sub-pixel displacement

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130403

Termination date: 20130922