CN102022981B - Peak-valley motion detection method and device for measuring sub-pixel displacement - Google Patents
Peak-valley motion detection method and device for measuring sub-pixel displacement Download PDFInfo
- Publication number
- CN102022981B CN102022981B CN 200910190924 CN200910190924A CN102022981B CN 102022981 B CN102022981 B CN 102022981B CN 200910190924 CN200910190924 CN 200910190924 CN 200910190924 A CN200910190924 A CN 200910190924A CN 102022981 B CN102022981 B CN 102022981B
- Authority
- CN
- China
- Prior art keywords
- valley
- peak
- axis
- pixel
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000006073 displacement reaction Methods 0.000 title claims abstract description 104
- 230000033001 locomotion Effects 0.000 title claims abstract description 55
- 238000001514 detection method Methods 0.000 title claims abstract description 45
- 239000003086 colorant Substances 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000005259 measurement Methods 0.000 claims description 52
- 230000003287 optical effect Effects 0.000 claims description 33
- 238000004364 calculation method Methods 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 7
- 238000009825 accumulation Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 18
- 230000007613 environmental effect Effects 0.000 abstract description 3
- 238000005286 illumination Methods 0.000 description 11
- 241000699666 Mus <mouse, genus> Species 0.000 description 8
- 238000003384 imaging method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000005855 radiation Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 239000000779 smoke Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241000699670 Mus sp. Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N ferric oxide Chemical compound O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 150000002739 metals Chemical class 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000012925 reference material Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
测量亚像素位移的峰谷运动探测方法及装置,由一台普通的计算机及其摄像头组成,其测量亚像素位移的方法是,先拍摄一帧,根据沿坐标轴方向的边方向数据,选取最具被测物反射特征的观测行与观测列;接着拍摄一帧,确定这两帧三基色各自的边方向数据及其峰和谷,通过对比,如果观测行和观测列中不能确定其相互位移关系的峰或谷的数目和超出其轴向该基色的峰、谷数目和的某个预定百分比,就认为“失去跟踪”,需要再拍摄一帧,否则,根据先后连续两帧中各基色的峰、谷移动的方向及其数目,分别计算各个轴向各个基色的亚像素位移,取其平均值为总的位移;有效地克服了环境光照变化的影响,适于位移与光照变化的速度都远低于拍摄帧速率的场合。
The peak-to-valley motion detection method and device for measuring sub-pixel displacement are composed of an ordinary computer and its camera. The method for measuring sub-pixel displacement is to first shoot a frame, and select the most The observation row and observation column with the reflection characteristics of the measured object; then take a frame and determine the edge direction data and peaks and valleys of the three primary colors of the two frames. By comparison, if the mutual displacement cannot be determined in the observation row and observation column If the number of peaks or valleys of the relationship exceeds a certain predetermined percentage of the sum of the number of peaks and valleys of the primary color on its axis, it is considered "lost tracking", and another frame needs to be shot, otherwise, according to the The direction and number of peaks and valleys move, respectively calculate the sub-pixel displacement of each primary color in each axis, and take the average value as the total displacement; effectively overcome the influence of environmental light changes, suitable for both displacement and light change speed When the frame rate is much lower than the shooting frame rate.
Description
技术领域 technical field
本发明属于数字图像测量技术领域,特别是涉及采用计算机摄像头无接触测量物体的二维微小位移的方法及其装置。The invention belongs to the technical field of digital image measurement, in particular to a method and a device for non-contact two-dimensional micro-displacement measurement of an object by using a computer camera.
背景技术 Background technique
计算机摄像头在近几年迅速地获得了普及,一般用于网络视频聊天或监视拍摄工作,其核心是CCD和CMOS光电成像芯片,即光电传感器阵列。而在其前面发明的光学定向装置的核心也是光电探测像素阵列,用于探测它相对所在表面的二维移动。因此,考察光学定向装置有益于开阔摄像头的应用。Computer cameras have gained popularity rapidly in recent years, and are generally used for network video chatting or surveillance shooting. The core is CCD and CMOS photoelectric imaging chips, that is, photoelectric sensor arrays. The core of the optical orientation device invented before it is also a photodetection pixel array, which is used to detect its two-dimensional movement relative to the surface on which it is located. Therefore, it is beneficial to investigate optical orientation devices for wide-open camera applications.
U.S.Pat.No.5,288,993揭示了一个利用光探测器阵列的光标定向装置,其结构中有一个被照明的靶球,该靶球必须具有随机分布的斑纹。U.S.Pat.No.5,703,356进一步揭示了一个鼠标形式的光学定向装置,它不需要靶球,它所利用的光是直接地从定向装置所移动的表面上反射来的。上述美国专利基于所谓的“边运动探测技术”来提取与运动相关的信息,所述边被定义为光探测阵列里两个像素之间的空间强度的差,这些边的每一个的相对运动被跟踪和测量,籍此确定代表光探测阵列与所在表面的被照明部分之间相对运动的总的位移。U.S. Pat. No. 5,288,993 discloses a cursor orienting device utilizing an array of photodetectors in which an illuminated target sphere must have randomly distributed markings. U.S. Pat. No. 5,703,356 further discloses an optical pointing device in the form of a mouse which does not require a target ball and which utilizes light reflected directly from the surface on which the pointing device is moved. The above-mentioned US patent extracts motion-related information based on the so-called "edge motion detection technique". Tracking and measurement whereby the total displacement representing the relative motion between the light detection array and the illuminated portion of the surface upon which it is determined is determined.
根据U.S.Pat.No.5,288,993,分别沿着光电传感器阵列的第一个轴和第二个轴排列的像素对之间确定“边”,光电传感器阵列可以是正交的也可以不是正交的。该专利中的边Ex和Ey定义为位于两个最邻近的像素的中间。根据U.S.Pat.No.5,288,993和U.S.Pat.No.5,703,356,一方面,计算沿着第一个轴的正方向运动的边Ex的数目与沿着第一个轴的相反的方向运动的边Ex的数目之间的归一化差,另一方面,计算沿着第二个轴的正方向运动的边Ey的数目与沿着第二个轴的相反的方向运动的边Ey的数目之间的归一化差,比较光电传感器阵列里这些边在相继的两个时间点的位置可以确定边的相对运动。According to US Pat. No. 5,288,993, "edges" are defined between pairs of pixels arranged along a first axis and a second axis, respectively, of a photosensor array, which may or may not be orthogonal. Edges Ex and E y in this patent are defined as being located in the middle of two nearest neighbor pixels. According to USPat.No.5,288,993 and USPat.No.5,703,356, on the one hand, calculate the difference between the number of sides Ex moving along the positive direction of the first axis and the number of sides Ex moving along the opposite direction of the first axis On the other hand, computing the normalized difference between the number of edges E y moving in the positive direction of the second axis and the number of edges E y moving in the opposite direction of the second axis The relative motion of the edges can be determined by comparing the positions of the edges in the photosensor array at two consecutive time points.
典型的光学定向装置包括一个光源,例如红外线LED,该光源根据一个确定的顺序间歇地照明表面的一部分,光电探测阵列的像素信号根据所确定的顺序被取样,提供了用于比较的相继的两帧边的图像,从而确定阵列相对表面的运动。A typical optical pointing device includes a light source, such as an infrared LED, that intermittently illuminates a portion of the surface according to a determined sequence. The pixel signals of the photodetection array are sampled according to the determined sequence, providing two consecutive Frame the image of the edge to determine the motion of the array relative to the surface.
根据上述两个美国专利的一个方案,采用差分技术有利于确定两个像素之间的一个边的条件。其关于边的定义是:如果两个光敏单元的强度的比率大于某个确定了的值,这两个像素之间存在一个边。文献中还给出了关于边沿着水平的或垂直的方向的定义表示式,并指出,上述条件Ex和Ey并不依赖于边的方向或意义,只是表明两个像素之间是否存在一个边。According to one approach of the above two US patents, the use of differential techniques is advantageous for determining the condition of an edge between two pixels. Its definition of an edge is: if the ratio of the intensities of two photosensitive units is greater than a certain value, there is an edge between these two pixels. The literature also gives a definition expression about the edge along the horizontal or vertical direction, and points out that the above conditions E x and E y do not depend on the direction or meaning of the edge, but only indicate whether there is a side.
上述“边”的定义和运动探测算法在光学跟踪球里取得了成功。光学跟踪球是探测一个受用户操作的被照明了的球的光强图案。上述定义与算法需要被探测的光强图案显示出清晰的精细的光强差异。因此,球的表面被覆盖了一种色彩的随机性标志图案,以显示出有别于背景的对比度。此外,这些标记大多呈斑点的形式,需要一个预定的大小。U.S.Pat.No.5,288,993中的斑点大小在0.5mm2-0.7mm2,密度约为1个斑点/mm2;这些标记的大小相对地独立于球的直径,但是,主要取决于光电探测器阵列的分辨率和大小。U.S.Pat.No.5,703,356提出,一个单独的斑点在传感器上的像最小地应当覆盖两个邻近像素之间中心到中心的距离(pixel pitch)。实际上,优先选择的典型的斑点的大小应当使得该点的像的表面覆盖大约5个像素。在此条件下,边的数目保持充分地不变,充分地小于像素的数目,从而可以被确定并用于探测运动。The definition of "edge" and the motion detection algorithm described above have been successful in optical trackballs. Optical trackballs detect the light intensity pattern of an illuminated ball that is manipulated by the user. The above definitions and algorithms require that the detected light intensity patterns show clear and fine light intensity differences. Therefore, the surface of the ball is covered with a color random logo pattern to show the contrast with the background. Furthermore, these marks are mostly in the form of spots, which require a predetermined size. The spot size in USPat.No.5,288,993 is 0.5mm 2 -0.7mm 2 , and the density is about 1 spot/mm 2 ; the size of these marks is relatively independent of the diameter of the ball, but depends mainly on the photodetector array. resolution and size. US Pat. No. 5,703,356 proposes that the image of a single spot on the sensor should at least cover the center-to-center distance (pixel pitch) between two adjacent pixels. In practice, a typical spot size is preferably chosen such that the surface of the image of the spot covers about 5 pixels. Under this condition, the number of edges remains sufficiently constant, sufficiently smaller than the number of pixels, to be determined and used to detect motion.
上述探测运动的方案应用到光学定向装置,如果被照明的表面不出现一个预定的图案,就会出现一个严重的问题;例如,“无球”鼠标直接地使用从一个随机的表面,例如纸的或桌子的表面,反射来的光。U.S.Pat.No.5,703,356描述了一个不需要任何球的的光学鼠标,但是,该方案需要被照明的表面显示出一个合适的图案,该图案具有充分数目的暗的和亮的区域,这些区域大小充分。The above scheme for detecting motion applied to an optical orientation device presents a serious problem if the illuminated surface does not appear in a predetermined pattern; for example, a "ballless" mouse used directly from a random surface such as paper Or the surface of a table, reflecting incoming light. U.S.Pat.No. 5,703,356 describes an optical mouse that does not require any balls, but the solution requires that the illuminated surface exhibit a suitable pattern with a sufficient number of dark and bright areas of the size full.
任何情形下,对于一个没有特征的表面,应用上述技术的时候,上述定义的边实际上在每一个像素之间都能够被探测到。因此,如果不能够清晰地探测到并跟踪具体的和很好定义的边的图案,就不可能导出关于光学探测器件和被照明表面之间的相对运动的任何测量。因此,上述运动探测技术不可能获得实际的应用。In any case, for a featureless surface, when applying the above technique, the edges defined above can be detected between virtually every pixel. Therefore, without being able to clearly detect and track specific and well-defined edge patterns, it is impossible to derive any measurements about the relative motion between the optical detection device and the illuminated surface. Therefore, the above-mentioned motion detection technique cannot be practically applied.
其实,U.S.Pat.No.5,578,813和U.S.Pat.No.5,644,139已经揭示过一些关于“被照明的表面产生纯粹随机的光强图案”的方案。这两个发明根据光电探测器阵列输出的连续的像帧的关联,提出了运动探测的原理。然而,它们比较邻近像素之间的光强没有提供关于邻近像素之间空间的强度差(即边)的任何信息,发明中提出的连续像帧的关联隐含了一些限制。特别地,为了导出相对运动的一个足够精确的测量,实际上需要以一个相对表面很低的角度照明参考表面,这个角度典型地处于表面的16°以内,U.S.Pat.No.5,288,720对此更精确地给予了说明。这导致约束光学定向装置的结构。此外,分析灰度像帧的时候,上述像帧关联技术表现了某些限制,随着像的强度的位数长度的增加,关联处理的复杂性成指数函数增加。实际中,上述关联技术应用到光学装置的运动探测局限于分析二进制的黑-白像。In fact, U.S. Pat. No. 5,578,813 and U.S. Pat. No. 5,644,139 have disclosed some schemes for "generating a purely random pattern of light intensity on an illuminated surface". These two inventions propose the principle of motion detection based on the correlation of continuous image frames output by the photodetector array. However, their comparison of light intensities between adjacent pixels does not provide any information about spatial intensity differences (ie edges) between adjacent pixels, and the association of consecutive image frames proposed in the invention implies some limitations. In particular, in order to derive a sufficiently accurate measurement of relative motion, it is actually necessary to illuminate the reference surface at a very low angle relative to the surface, typically within 16° of the surface, U.S. Pat. No. 5,288,720 being more precise for this explained. This results in constraining the structure of the optical orientation device. Furthermore, the above image frame correlation techniques exhibit certain limitations when analyzing grayscale image frames, and the complexity of the correlation process increases exponentially as the bit length of the image intensity increases. In practice, the application of the aforementioned correlation techniques to motion detection in optical devices is limited to analyzing binary black-and-white images.
参考上述美国专利,最近提交审查的发明专利“使用计算机摄像头测量微小二维位移的方法及装置”(申请号:200910104277.8,申请日:2009.7.7)分析了国内有关“摄像”“测量”的专利文件,提出了一种以计算机摄像头为光电转换传感器,通过帧-帧关联匹配比较,测量微小的二维位移矢量和速度矢量的的方法,但是,它仍然是针对光强图案进行匹配,因而也需要照明均匀且稳定。Referring to the above-mentioned US patent, the recently submitted invention patent "method and device for measuring small two-dimensional displacement using computer camera" (application number: 200910104277.8, application date: 2009.7.7) analyzed domestic patents related to "camera" and "measurement" The document proposes a method of using a computer camera as a photoelectric conversion sensor to measure tiny two-dimensional displacement vectors and velocity vectors through frame-frame correlation matching and comparison. However, it still matches light intensity patterns, so it also Uniform and stable lighting is required.
”Method and sensing device for motion detection in an optical pointing device,such as anoptical mouse”(US 7,122,781 B2,Oct.17,2006,Rotzoll et al.)提供更适合光学定向装置的二个解决方案,其方法之一是:预处理图像帧以提取描述邻近像素之间的光强差(即边)的二进制数据,直接跟踪正边和负边的运动方向,计算相对运动的位移测量,即“Local Edge DirectionMotion Detection Algorithm(局域边方向运动探测算法)”。其方法之二是:从所提取的边方向数据进一步提取边反射状况,它描述沿着光探测器阵列的一个轴方向的正边和负边的连续性,然后,比较先后两帧边反射状况的位置,计算出相对运动的位移测量,即“Peak/Null MotionDetection Algorithm(峰/谷运动探测算法)”。该发明的特点是:1)被照明的表面没有施加特征,产生纯粹随机的光强图案;易于提取可以被跟踪的有特征的特点和图案,并直接用于运动探测。2)显示了对像素之间强度的轻微的变化的巨大敏感性,允许适用一个宽范围的照明角度,而不是需要一个低的照明角度,以增强U.S.Pat.No.5,5788,813、U.S.Pat.No.5,644,139和U.S.Pat.No.5,686,720所使用的关联技术中表面图案的对比度。3)可以分析灰度图像,所需要的处理表现得特别地有活力和简单,而不会增加处理的复杂程度。"Method and sensing device for motion detection in an optical pointing device, such as an optical mouse" (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.) provides two solutions that are more suitable for optical pointing devices. One is: preprocessing image frames to extract binary data describing the difference in light intensity between adjacent pixels (i.e., edges), directly tracking the direction of motion of positive and negative edges, and computing a displacement measure of relative motion, ie "Local Edge DirectionMotion Detection Algorithm (local edge direction motion detection algorithm)". The second method is: further extracting edge reflection conditions from the extracted edge direction data, which describes the continuity of positive and negative edges along an axis direction of the photodetector array, and then comparing the edge reflection conditions of two consecutive frames position, calculate the displacement measurement of the relative motion, namely "Peak/Null MotionDetection Algorithm (peak/valley motion detection algorithm)". The characteristics of the invention are: 1) The surface to be illuminated has no applied features and produces purely random light intensity patterns; it is easy to extract characteristic features and patterns that can be tracked and directly used for motion detection. 2) Shows great sensitivity to slight variations in intensity between pixels, allowing a wide range of illumination angles to be applied, rather than requiring a low illumination angle, to enhance U.S. Pat. No. 5,5788,813, U.S. Contrast of surface patterns in related techniques used in Pat. No. 5,644,139 and U.S. Pat. No. 5,686,720. 3) Grayscale images can be analyzed, and the required processing appears particularly dynamic and simple without increasing the complexity of the processing.
该专利所谓“Pcak/Null Motion Detection Algorithm(峰/谷运动探测算法)”计算总的位移测量值的算法之一如下:One of the algorithms for calculating the total displacement measurement value of the patent's so-called "Pcak/Null Motion Detection Algorithm" is as follows:
该专利计算总的位移测量值的峰/谷运动探测算法之二是:The second peak/valley motion detection algorithm in which the patent calculates the total displacement measurement is:
式中,X、Y和N分别表示两个坐标轴和数目(Number),各项的含义如文字或符号所示,例如,NXPEAK和NPEAK-RIGHT分别表示X轴方向峰的数目及其发生向右移动了的峰的数目,LPIXEL-PIXEL表示像素间距,其单位可以是μm、mm或inch等。In the formula, X, Y and N represent the two coordinate axes and the number (Number) respectively, and the meanings of each item are shown in words or symbols, for example, N XPEAK and N PEAK-RIGHT respectively represent the number of peaks in the X-axis direction and their The number of peaks that have shifted to the right, L PIXEL-PIXEL represents the pixel pitch, and its unit can be μm, mm, or inch.
上述两种峰/谷运动探测算法并不等同,但是,两者的峰和谷的总的数目相同,因而总的位移测量值都是一个像素间距LPIXEL-PIXEL的分数,两者的计算结果接近,都属于亚像素级位移。而且,第二种计算位移的方法对于每一个坐标轴只需要进行一次除法。The above two peak/valley motion detection algorithms are not equivalent, however, the total number of peaks and valleys is the same for both, so the total displacement measurement is a fraction of the pixel spacing L PIXEL-PIXEL , and the calculation results of the two Close, all belong to sub-pixel level displacement. Also, the second method of computing displacement requires only one division per axis.
上述峰谷运动探测算法以提取图像帧的特征为基础,克服了光强图案的随机变化给帧-帧匹配所带来的困难,因而,它优越于前述种种专利技术,大大地降低了对其运动的表面的光学反射质地的要求。The above-mentioned peak-valley motion detection algorithm is based on extracting the features of the image frame, and overcomes the difficulty brought by the random change of the light intensity pattern to the frame-frame matching. Optically reflective texture requirements for moving surfaces.
此外,上述峰谷运动探测算法还有其它的优点。首先,探测器“看”到的边反射数目取决于传感器的聚焦状态。第二,边反射被假设在像素阵列上移动,并且,在连续的两次曝光期间这些边反射的移动幅度不会超出一个单位的像素间距。对于一个给定的运动及其最大的运动速度,该条件易于满足。如果一个边反射的移动超出一个单位的像素间距,上述运动探测方案不能够决定该边反射来自何处。如果在连续的两次曝光期间,探测器相对被照明的表面移动过快,就会发生此类情况,它代表了一种“失去跟踪(loss-off-track)”的情形。通过查看那些不知道来自何处的边反射状况的数目能够探测到这种“失去跟踪”的情形,此时,前一个状态(帧)的反射状况在当前的位置或当前位置附近的一个像素间距范围内没有相似的边反射状况,这些边反射状况被定义为“鬼影边反射状况(ghost edge inflection conditions)”,它们沿着两个坐标轴的方向潜在地会出现,其沿各坐标轴方向的数目可以用两个累加器分别加以检测和跟踪,并与预先确定的某个门槛值相比较,籍此确定“失去跟踪”的情况,并作为一个警告信号报告给外部的操作者。光电阵列边上的边反射不计为“鬼影边反射状况”;当它们看起来不知来自何处的时候,有可能来自光电阵列的外部。In addition, the above-described peak-to-valley motion detection algorithm has other advantages. First, the number of edge reflections that the detector "sees" depends on the focus state of the sensor. Second, edge reflections are assumed to move across the pixel array, and these edge reflections do not move by more than one unit of pixel pitch during two consecutive exposures. For a given motion and its maximum motion velocity, this condition is easy to satisfy. If an edge reflection moves beyond one unit of pixel pitch, the motion detection scheme described above cannot determine where the edge reflection came from. This occurs if the detector moves too quickly relative to the illuminated surface between two consecutive exposures, and represents a "loss-off-track" situation. This "lost-of-tracking" situation can be detected by looking at the number of edge reflections that don't know where they came from, where the reflections from the previous state (frame) were at or one pixel apart from the current position There are no similar edge reflection conditions in range, these edge reflection conditions are defined as "ghost edge reflection conditions (ghost edge reflection conditions) (ghost edge reflection conditions)", they can potentially occur along the direction of the two coordinate axes, which along the direction of each coordinate axis The number can be detected and tracked by two accumulators respectively, and compared with a predetermined threshold value, thereby determining the "lost track" situation, and reported to the external operator as a warning signal. Edge reflections on the sides of the photocell do not count as "ghost side reflection conditions"; while they appear to be coming from nowhere, they may be coming from outside the photocell.
上述探测判断“失去跟踪”的方法符合实际,可以算法实现。当然,还可以找到其它的探测、判断“失去跟踪”的方法。但是,经过仔细分析,会发现,该专利上述“峰/谷运动探测算法”,以及“局域边方向运动探测算法”缺乏计算的根据,难于理解。The above-mentioned method of detecting and judging "lost tracking" is practical and can be realized by algorithm. Of course, other methods of detecting and judging "lost tracking" can also be found. However, after careful analysis, it will be found that the above-mentioned "peak/valley motion detection algorithm" and "local edge direction motion detection algorithm" in this patent lack calculation basis and are difficult to understand.
总之,关于光学定向装置中探测相对运动的美国专利技术,激励了使用计算机摄像头测量随机状况的表面上所发生的微小位移的思想。光学定向装置中的光电传感器阵列相对所在的表面之间的距离极小(毫米数量级),它采用单色LED之类照明光源,其发光强度受电子电路控制,实施间歇式照明,因此,被照表面面积相当地小,光照均匀,其反射光束不会受到环境光辐射的影响,较好地反映了该表面的光学反射性质。而应用计算机摄像头进行位移测量,光电传感器阵列距离被测物体较远,被测物表面的光学反射性质会起到更大的影响。例如,由于可见光的频率较短,被光电传感器进行光电转换的物体的反射光会是若干次反射的一个平均值;随着光照角度的不同,质地粗糙的表面其漫反射会很强烈,变化也会较大;金属与平面镜之类强反射面所产生的反射光更会随着入射光角度的变化而强烈地发生变化,甚至于连周围其它物体的映像也将会被反射出来,这些都会模糊、混淆反射面本身的质地特征;入射的阳光被物体反射以后会出现丰富的偏振光;测量环境中其它的光源的辐射或飘游灰尘或烟雾等的影响,也会导致被测物体表面的图像帧发生整体的、或者局部区域的光强变化,或者表现为灰度图像的光强数据出现随机的波动。这些为应用“峰/谷运动探测亚像素位移算法”测量随机表面的微小位移提出了新的问题。In summary, the US patent technique for detecting relative motion in an optical orientation device inspired the idea of using computer cameras to measure small displacements occurring on surfaces of random conditions. The distance between the photoelectric sensor array in the optical orientation device is extremely small (on the order of millimeters), and it uses a lighting source such as a monochromatic LED. The surface area is relatively small, the illumination is uniform, and the reflected light beam will not be affected by ambient light radiation, which better reflects the optical reflection properties of the surface. However, when using a computer camera for displacement measurement, the photoelectric sensor array is far away from the measured object, and the optical reflection properties of the surface of the measured object will have a greater impact. For example, due to the short frequency of visible light, the reflected light of an object that is photoelectrically converted by a photoelectric sensor will be an average value of several reflections; with the different illumination angles, the diffuse reflection of a rough surface will be very strong, and the change will also vary. will be larger; the reflected light produced by strong reflective surfaces such as metals and flat mirrors will change strongly with the change of the incident light angle, and even the images of other surrounding objects will be reflected, which will be blurred , confuse the texture characteristics of the reflective surface itself; the incident sunlight will be rich in polarized light after being reflected by the object; the influence of radiation from other light sources in the measurement environment or floating dust or smoke will also cause image frames on the surface of the measured object Overall or local light intensity changes, or random fluctuations in the light intensity data of the grayscale image. These raise new questions for the application of "peak/valley motion detection sub-pixel displacement algorithm" to measure small displacements of random surfaces.
发明内容 Contents of the invention
为了充分发挥计算机摄像头的光电传感器阵列的功能,本发明提供一种可以适应变化相对不大的照明环境的测量亚像素位移的峰谷运动探测方法及装置,它以计算机摄像头为光电转换传感器,即时地拍摄被测量的物体,再经过峰/谷亚像素位移探测算法处理,获得该物体在与摄像头的光轴相垂直的平面上的微小的二维位移矢量和速度矢量。In order to give full play to the function of the photoelectric sensor array of the computer camera, the present invention provides a method and device for detecting peak-to-valley motions that can adapt to relatively small changes in the lighting environment and measure sub-pixel displacement. It uses a computer camera as a photoelectric conversion sensor. The object to be measured is photographed accurately, and then processed by the peak/valley sub-pixel displacement detection algorithm to obtain the tiny two-dimensional displacement vector and velocity vector of the object on a plane perpendicular to the optical axis of the camera.
本发明解决其技术问题所采用的技术方案是:测量的物体位于计算机摄像头的光学镜头前面有效的聚焦成像范围以内,被测量物体的照明环境相对均匀、稳定,所述摄像头通过USB接口连接到一台普通的计算机,该计算机配置USB接口、内存、CPU、硬盘、显示卡与显示器、键盘和鼠标、操作系统、摄像头驱动程序以及摄像头拍摄及峰/谷亚像素位移探测程序,该程序包括下述步骤:The technical solution adopted by the present invention to solve its technical problems is: the object to be measured is located within the effective focus imaging range in front of the optical lens of the computer camera, the lighting environment of the object to be measured is relatively uniform and stable, and the camera is connected to a computer via a USB interface. An ordinary computer, the computer is equipped with USB interface, memory, CPU, hard disk, display card and monitor, keyboard and mouse, operating system, camera driver and camera shooting and peak/valley sub-pixel displacement detection program, the program includes the following step:
步骤一、拍摄一帧(M行×N列)被测物体,不考虑该图像帧的边上的两行与两列,得到参考帧[(M-2)行×(N-2)列],为该参考帧以及其后所有拍摄的图像帧确定相同的坐标轴,其中,M,N∈正整数;
步骤二、对于上述参考帧之像素阵列,根据其红色、绿色和蓝色分量的数据,分别逐行、逐列导出沿X轴和Y轴方向的红色的、绿色的和蓝色的边方向数据;
步骤三、分别逐行、逐列计算红色的、绿色的和蓝色的正边和负边的数目之和,取其中正边和负边的数目之和最多者所在的行与列分别作为观测行和观测列;Step 3. Calculate the sum of the numbers of positive and negative sides of red, green and blue respectively row by row and column by column, and take the row and column where the sum of the number of positive and negative sides is the largest as the observation rows and columns of observations;
步骤四、根据所选观测行与观测列的像素的三种基色的边方向数据,分别导出沿X轴和Y轴方向的三种基色的边反射状况,并分别使用累加器计数它们所对应的峰、谷的数目:NRX轴峰(j)、NRX轴谷(j)、NRY轴峰(j)和NRY轴谷(j),NGX轴峰(j)、NGY轴谷(j)、NGY轴峰(j)和NGY轴谷(j),NBX轴峰(j)、NBX轴谷(j)、NBY轴峰(j)和NBY轴谷(j),其中,j表示拍摄获得的帧的顺序计数值,j∈正整数,N(umber)表示数目,R、G、B分别表示红、绿和蓝色;
步骤五、拍摄了所述参考帧之后,经过一定的间歇时间dt,拍摄第二帧被测物体的像(M×N),去掉像素帧边缘的两行和两列得到比较帧[(M-2)×(N-2)];沿着步骤三中选定的观测行或观测列,分别导出沿X轴和Y轴方向的三种基色的边方向数据和边反射状况,并分别使用累加器计数沿着X轴和Y轴方向的峰、谷的数目:NRX轴峰(j+1)、NRX轴谷(j+1)、NRY轴峰(j+1)和NRY轴谷(j+1),NGX轴峰(j+1)、NGX轴谷(j+1)、NGY轴峰(j+1)和NGY轴谷(j+1),NBX轴峰(j+1)、NBX轴谷(j+1)、NBY轴峰(j+1)和NBY轴谷(j+1);Step 5, after photographing the reference frame, after a certain intermittent time dt, photograph the image (M×N) of the second frame of the measured object, remove two rows and two columns at the edge of the pixel frame to obtain a comparison frame [(M- 2) × (N-2)]; along the observation row or observation column selected in step 3, respectively derive the edge direction data and edge reflection conditions of the three primary colors along the X-axis and Y-axis directions, and use the cumulative The counter counts the number of peaks and valleys along the X and Y axes: N RX axis peak (j+1), N RX axis valley (j+1), N RY axis peak (j+1), and N RY axis Valley (j+1), N GX axis peak (j+1), N GX axis valley (j+1), N GY axis peak (j+1) and N GY axis valley (j+1), N BX axis Peak (j+1), N BX axis valley (j+1), N BY axis peak (j+1) and N BY axis valley (j+1);
步骤六、对比所述参考帧与所述比较帧中所选观测行与观测列的三种基色的边反射状况的位置,检查其中的那些不知道来自何处的“鬼影边反射状况”,即:在所述参考帧的所选观测行或观测列发现的这些峰或谷,在所述比较帧的所选观测行或观测列的相应位置及其附近的一个像素间距范围内发生缺失,反之亦然,这时,用六个累加器分别统计和跟踪它们的数目:NRX鬼影边(j+1)、NRY鬼影边(j+1)、NGX鬼影边(j+1)、NGY鬼影边(j+1)、NBX鬼影边(j+1)、NBY鬼影边(j+1);
如果三种基色中有两种或两种以上的基色的“鬼影边反射状况”满足:If the "ghost edge reflection condition" of two or more of the three primary colors satisfies:
或 or
则认为测量“失去跟踪”,并发出一个警告信号,式中,误差容限值error1表示比较帧中沿着所选观测行或观测列,三种基色(Red、Green、Blue)之一的鬼影边反射状况发生的数目占该基色的全部边反射状况数目的百分比,可以根据物体表面的光学性质和测量环境来预置,例如预置为:error1=5%;所述比较帧的边上的边反射状况不考虑“鬼影边反射状况”,它们有可能来自光电像素阵列的外部;Then it is considered that the measurement is "lost tracking" and a warning signal is issued. In the formula, the error tolerance value error1 represents the ghost of one of the three primary colors (Red, Green, Blue) along the selected observation row or observation column in the comparison frame The percentage of the number of shadow edge reflections taking place in the total number of edge reflections of the base color can be preset according to the optical properties of the object surface and the measurement environment, for example, it is preset as: error1=5%; on the edge of the comparison frame The edge reflections of , do not consider the "ghost edge reflections", which may come from the outside of the photopixel array;
步骤七、如果测量“失去跟踪”,回到步骤一,重新开始测量工作;Step 7. If the measurement "lost tracking", go back to
步骤八、如果测量没有“失去跟踪”,则继续本次测量工作:沿着所选观测行与所选观测列,根据摄像头的三基色分量数据,运用峰/谷亚像素位移探测算法,分别计算三基色分量的亚像素位移:Step 8. If there is no "lost tracking" in the measurement, continue the measurement work: along the selected observation row and the selected observation column, according to the three primary color component data of the camera, use the peak/valley sub-pixel displacement detection algorithm to calculate respectively Subpixel displacement of three primary color components:
本次测量中,总的亚像素位移计算式是:In this measurement, the total sub-pixel displacement calculation formula is:
步骤九、本次测量中,物体的速度矢量计算式为:
步骤十、以步骤五中的比较帧作为新的参考帧,距拍摄此帧一定的间歇时间dt,重新拍摄一帧作为新的比较帧,即跳到步骤五,继续测量。
其中,所述摄像头拍摄及峰/谷运动探测亚像素位移程序之步骤二所述红色、绿色或蓝色的边方向数据的定义是:Wherein, the definition of the red, green or blue edge direction data in the second step of the sub-pixel displacement program of the camera shooting and peak/valley motion detection is:
根据像素阵列中红色、绿色或蓝色这三种基色之一的分量数据,沿着X轴或者沿着Y轴方向,如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要小一个误差容限值error,即如果According to the component data of one of the three primary colors of red, green or blue in the pixel array, along the X-axis or along the Y-axis direction, if a pixel has a value of a certain three-primary color component that is greater than that of the second pixel behind it The value of the three primary color components is smaller by an error tolerance value error, that is, if
I(X,Y)红<I红(X+2,Y)-error或I(X,Y)红<I(X,Y+2)红-errorI(X, Y) red < I red (X+2, Y)-error or I(X, Y) red < I(X, Y+2) red -error
I(X,Y)绿<I绿(X+2,Y)-error或I(X,Y)绿<I(X,Y+2)绿-errorI(X,Y) Green < IGreen (X+2,Y)-error or I(X,Y) Green <I(X,Y+2) Green- error
I(X,Y)蓝<I蓝(X+2,Y)-error或I(X,Y)蓝<I(X,Y+2)蓝-errorI(X, Y) blue < I blue (X+2, Y)-error or I(X, Y) blue < I(X, Y+2) blue- error
则定义这两个像素之间存在一个红色的、绿色的或蓝色的正边;如果一个像素的某种三基色分量值比其后面的第二个像素相应的三基色分量值还要大一个误差容限值error,即如果It is defined that there is a red, green or blue positive edge between these two pixels; if the value of a certain three-primary color component of a pixel is one larger than the corresponding three-primary color component value of the second pixel behind it The error tolerance value error, that is, if
I(X,Y)红>I红(X+2,Y)+error或I(X,Y)红>I(X,Y+2)红+errorI(X,Y) red >I red (X+2,Y)+error or I(X,Y) red >I(X,Y+2) red +error
I(X,Y)绿>I绿(X+2,Y)+error或I(X,Y)绿>I(X,Y+2)绿+errorI(X, Y) green > I green (X+2, Y) + error or I (X, Y) green > I (X, Y + 2) green + error
I(X,Y)蓝>I蓝(X+2,Y)+error或I(X,Y)蓝>I(X,Y+2)蓝+errorI (X, Y) blue > I blue (X+2, Y) + error or I (X, Y) blue > I (X, Y + 2) blue + error
则定义这两个像素之间存在一个红色的、绿色的或蓝色的负边;如此获得的边位于该像素之后的第一个像素的位置,也即位于参与比较的两个像素的中间位置的那个像素上;如果一个像素的某种三基色分量值与其后面的第二个像素相应的三基色分量值接近,其RGB分量值相差不超过一个误差容限值error,即如果Then it is defined that there is a red, green or blue negative edge between these two pixels; the edge thus obtained is located at the position of the first pixel after this pixel, that is, at the middle position of the two pixels participating in the comparison on that pixel; if the value of a certain three-primary color component of a pixel is close to the corresponding three-primary color component value of the second pixel behind it, the difference between its RGB component values does not exceed an error tolerance value error, that is, if
I(X+2,Y)红-error<I(X,Y)红<I(X+2,Y)红+error或I(X,Y+2)红-error<I(X,Y)红<I(X,Y+2)红+error;I(X+2, Y) red -error<I(X, Y) red <I(X+2, Y) red +error or I(X, Y+2) red -error<I(X, Y) Red <I(X, Y+2) red +error;
I(X+2,Y)绿-error<I(X,Y)绿<I(X+2,Y)绿+error或I(X,Y+2)绿-error<I(X,Y)绿<I(X,Y+2)绿+error;I(X+2,Y) green -error<I(X,Y) green <I(X+2,Y) green +error or I(X,Y+2) green -error<I(X,Y) Green <I(X, Y+2) Green +error;
I(X+2,Y)蓝-error<I(X,Y)蓝<I(X+2,Y)蓝+error或I(X,Y+2)蓝-error<I(X,Y)蓝<I(X,Y+2)蓝+error;I(X+2, Y) blue -error<I(X,Y) blue <I(X+2,Y) blue +error or I(X,Y+2) blue- error<I(X,Y) Blue <I(X, Y+2) blue +error;
则认为这两个像素之间不存在该颜色波长对应的“边”,或称之为第三类该颜色的边;沿着某一个坐标轴方向,所有的红色的正边和红色的负边以及第三类红色的边组成该方向红色的边方向数据,所有的绿色的正边和绿色的负边以及第三类绿色的边组成该方向绿色的边方向数据,所有的蓝色的正边和蓝色的负边以及第三类蓝色的边组成该方向蓝色的边方向数据;上式中的误差容限值可以根据具体的光照情况,预置为一个小的数值,例如:error=10;像素阵列中的四个边与角上的像素位置不存在边方向数据。It is considered that there is no "edge" corresponding to the color wavelength between the two pixels, or the edge of the third type of color; along a certain coordinate axis, all red positive edges and red negative edges And the third type of red edge constitutes the red edge direction data of this direction, all the green positive edges and green negative edges and the third type of green edge form the green edge direction data of this direction, all the blue positive edges The blue negative edge and the third type of blue edge form the blue edge direction data in this direction; the error tolerance value in the above formula can be preset to a small value according to the specific lighting situation, for example: error =10; the pixel positions on the four sides and corners in the pixel array do not have side direction data.
所述摄像头拍摄及峰/谷运动探测亚像素位移程序之所述步骤四和步骤五中所述沿X轴和Y轴方向的三种基色的边反射状况,其定义为:The edge reflection conditions of the three primary colors along the X-axis and Y-axis directions in the
根据所选观测行与观测列的像素的红色、绿色或蓝色的边方向数据,沿着X轴或沿着Y轴方向,如果连续的两个或多于两个的某种基色的正边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的负边,称之为该基色的第一类边反射状况,即认为在此位置存在一个该基色的峰;如果连续的两个或多于两个的某种基色的负边,或连续的两个或多于两个的该基色的第三类边之后跟着一个该基色的正边,称之为该基色的第二类边反射状况,即认为在此位置存在一个该基色的谷。According to the red, green or blue side direction data of the pixels in the selected observation row and observation column, along the X axis or along the Y axis direction, if there are two or more consecutive positive sides of a certain primary color , or two or more consecutive third-type edges of the base color followed by a negative edge of the base color, which is called the first-type edge reflection condition of the base color, that is, it is considered that there is a base color at this position peak; if two or more consecutive negative sides of a certain basic color, or two or more consecutive third-type sides of this basic color are followed by a positive side of this basic color, it is called It is the second type of edge reflection condition of the base color, that is, it is considered that there is a valley of the base color at this position.
所述摄像头拍摄及峰/谷运动探测亚像素位移程序之所述步骤八中所述的峰/谷亚像素位移探测算法包括:The peak/valley sub-pixel displacement detection algorithm described in the step eight of the camera shooting and peak/valley motion detection sub-pixel displacement procedure includes:
对于某种三基色波长,比较所述参考帧与所述比较帧里所选定的观测行与观测列之对应的峰、谷的位置,籍此判断所述观测行与观测列里对应的峰、谷的移动方向,并分别使用累加器计数跟踪所述峰、谷沿所述坐标轴的移动情况,具体地,For a certain trichromatic wavelength, compare the positions of peaks and valleys corresponding to the selected observation row and observation column in the reference frame and the comparison frame, thereby judging the corresponding peaks in the observation row and observation column , the direction of movement of the valley, and use the accumulator count to track the movement of the peak and valley along the coordinate axis, specifically,
在所述观测行像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色峰向右位移的个数的累加器N峰向右移+1,如果某一个三基色峰在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色峰向左位移的个数的累加器N峰向左移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向右位移了一个像素间距单位,则跟踪该基色的谷向右位移的个数的累加器N谷向右移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向左位移了一个像素间距单位,则跟踪该基色的谷向左位移的个数的累加器N谷向左移+1;In the observed row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then tracking the number of the peaks of the primary color shifted to the right The N peak of the accumulator is shifted to the right by +1. If the position of a certain three primary color peaks in the comparison frame is displaced to the left by one pixel pitch unit relative to its position in the reference frame, the number of the primary color peaks shifted to the left is tracked. The N peak of the accumulator is shifted to the left by +1; if the position of a valley of a certain three primary colors in the comparison frame is shifted to the right by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted to the right The accumulator N of the number of valleys is moved to the right + 1, if the position of a valley of a certain three primary colors in the comparison frame is shifted to the left by one pixel pitch unit relative to its position in the reference frame, then the valley of the primary color is tracked The accumulator N valley of the number shifted to the left is shifted to the left + 1;
在所述观测列像素中,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的峰向上位移的个数的累加器N峰向上移+1,如果某一个三基色的峰在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的峰向下位移的个数的累加器N峰向下移+1;如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向上位移了一个像素间距单位,则跟踪该基色的谷向上位移的个数的累加器N谷向上移+1,如果某一个三基色的谷在比较帧里的位置相对它在参考帧里的位置向下位移了一个像素间距单位,则跟踪该基色的谷向下位移的个数的累加器N谷向下移+1;In the observation row of pixels, if the position of a peak of a certain three primary colors in the comparison frame is displaced upward by one pixel pitch unit relative to its position in the reference frame, the accumulation of the number of upward displacements of the peaks of the primary color is tracked The N peak of the detector moves up +1, if the position of a peak of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the number of the peaks of the primary color shifted downward is tracked The N peak of the accumulator moves down by +1; if the position of a valley of a certain three primary colors in the comparison frame is displaced upwards by one pixel pitch unit relative to its position in the reference frame, then tracking the valleys of the primary color is shifted upward by 1 The accumulator N valley of the number moves up +1, if the position of a valley of a certain three primary colors in the comparison frame is shifted down by one pixel pitch unit relative to its position in the reference frame, then the valley of the tracking color is shifted downward The number of the accumulator N valley moves down +1;
如果N峰向右移(j+1)>N峰向左移(j+1),则计为ΔX峰位移(j+1)=+0.5,If N peak shifts to the right (j+1)>N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=+0.5,
如果N峰向右移(j+1)<N峰向左移(j+1),则计为ΔX峰位移(j+1)=-0.5;If N peak shifts to the right (j+1)<N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=-0.5;
如果N谷向右移(j+1)>N谷向左移(j+1),则计为ΔX谷位移(j+1)=+0.5,If the N valley moves to the right (j+1)>N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=+0.5,
如果N谷向右移(j+1)<N谷向左移(j+1),则计为ΔX谷位移(j+1)=-0.5;If N valley moves to the right (j+1)<N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=-0.5;
类似地,Similarly,
如果N峰向上移(j+1)>N峰向下移(j+1),则计为ΔY峰位移(j+1)=+0.5,If the N peak moves up (j+1)>N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=+0.5,
如果N峰向上移(j+1)<N峰向下移(j+1),则计为ΔY峰位移(j+1)=-0.5,If the N peak moves up (j+1)<N peak moves down (j+1), it is calculated as ΔY peak displacement (j+1)=-0.5,
如果N谷向上移(j+1)>N谷向下移(j+1),则计为ΔY谷位移(j+1)=+0.5,If N valley moves up (j+1)>N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=+0.5,
如果N谷向上移(j+1)<N谷向下移(j+1),则计为ΔY谷位移(j+1)=-0.5,If N valley moves up (j+1)<N valley moves down (j+1), it is counted as ΔY valley displacement (j+1)=-0.5,
上述式中,L像素-像素表示像素间距,其单位可以是μm、mm或inch等,其它符号的含义如字符所标记,例如,N即数目(Number),上述计算式均当分别应用于摄像头像素阵列的某个具体的三基色分量数据。In the above formula, L pixel-pixel represents the pixel pitch, and its unit can be μm, mm or inch, etc. The meanings of other symbols are as marked by characters, for example, N is the number (Number), and the above calculation formulas should be applied to the camera respectively A specific three primary color component data of the pixel array.
本发明的优点是,该方法以普通的计算机摄像头中的光电传感器阵列为位移传感器,根据图像帧像素的RGB数据,分别计算出三种基色的描述邻近像素之间的光强的差的边的方向信息,它以“峰”和“谷”,也就是被测物体表面的反射光图案中三种基色的物理明暗对比度分布特征,作为测量亚像素位移的依据,由于对三种基色的波长同时分别进行测量计算,又辅之以平均算法,克服了其它波长受环境光照变化的影响,在相当的程度上减小了测量的误差,所得计算结果所对应的位移测量误差为≤±0.25L像素-像素~±0.5L像素-像素;同时,该发明只取反射面图像帧对比度变化最大的一行和一列作为测量的代表,既利用了光电传感器阵列,又简化了计算工作量,保证了二维位移测量及其测量灵敏度。本发明利用了光电传感器件的光电转换快速响应特性,适合于物体运动速度与光照变化速度都远低于摄像头的拍摄帧速率的情况。The advantage of the present invention is, this method uses the photoelectric sensor array in the ordinary computer camera head as displacement sensor, according to the RGB data of image frame pixel, respectively calculates three primary colors and describes the difference of light intensity between adjacent pixels. Direction information, which uses "peaks" and "valleys", that is, the distribution characteristics of the physical light and dark contrast of the three primary colors in the reflected light pattern on the surface of the measured object, as the basis for measuring sub-pixel displacement, because the wavelengths of the three primary colors are simultaneously Measurement and calculation are carried out separately, supplemented by an average algorithm, which overcomes the influence of other wavelengths by environmental light changes, and reduces the measurement error to a considerable extent. The displacement measurement error corresponding to the obtained calculation results is ≤±0.25L pixels -pixel to ±0.5L pixel-pixel ; at the same time, the invention only takes the row and column with the largest contrast change of the image frame on the reflective surface as the representative of the measurement, which not only utilizes the photoelectric sensor array, but also simplifies the calculation workload and ensures two-dimensional Displacement measurement and its measurement sensitivity. The invention utilizes the photoelectric conversion fast response characteristic of the photoelectric sensor device, and is suitable for the situation that both the moving speed of the object and the light changing speed are far lower than the shooting frame rate of the camera.
附图说明 Description of drawings
下面结合附图进一步说明本发明。Further illustrate the present invention below in conjunction with accompanying drawing.
图1是本发明的测量装置方框图。Fig. 1 is a block diagram of the measuring device of the present invention.
图2表示一个3×3像素阵列及其根据定义导出的边方向数据。图中,一个方框表示一个像素,方框内的点的数目越多表示该像素的亮度越暗,所确定的正边和负边以粗线段表示,其中该像素阵列的外围的四个边的正、负方向还分别以箭头表示。Figure 2 shows a 3 × 3 pixel array and its derived edge orientation data by definition. In the figure, a box represents a pixel, and the larger the number of points in the box, the darker the brightness of the pixel. The determined positive and negative sides are represented by thick line segments, and the four peripheral sides of the pixel array The positive and negative directions of are also indicated by arrows, respectively.
图3表示光电探测器阵列沿着某个轴向获得的图像光电信号以及据此导出的一个序列的边方向数据(正边和负边)和边反射状况(峰和谷)。Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain axis and a sequence of side direction data (positive side and negative side) and side reflection conditions (peak and valley) derived therefrom.
图4是沿着某个轴跟踪边反射状况进行亚像素位移测量的示意图。FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis.
图5是峰/谷亚像素位移探测算法的图解。其中,箭头向上表示正边,箭头向下表示负边,虚线以及细实线均表示坐标平面图上均匀的等分线,横坐标的等分区间表示分离的各个像素范围。Figure 5 is a diagram of a peak/valley sub-pixel displacement detection algorithm. Among them, the upward arrow indicates the positive edge, and the downward arrow indicates the negative edge. Both the dotted line and the thin solid line indicate the uniform bisector on the coordinate plane, and the equal division interval of the abscissa indicates the separated range of each pixel.
图1中,1.计算机摄像头,2.光学透镜,3.光电成像芯片,4.摄像头的USB接口,5.计算机系统,6.计算机的USB接口,7.CPU,8.内存,9.显示卡与显示器,10.硬盘,11.键盘和鼠标,12.操作系统,13.摄像头驱动程序,14.摄像头拍摄及峰/谷运动探测亚像素位移程序,15.照明设备。In Fig. 1, 1. computer camera, 2. optical lens, 3. photoelectric imaging chip, 4. USB interface of camera, 5. computer system, 6. USB interface of computer, 7. CPU, 8. memory, 9. display Cards and monitors, 10. Hard drives, 11. Keyboards and mice, 12. Operating systems, 13. Camera drivers, 14. Sub-pixel displacement programs for camera capture and peak/valley motion detection, 15. Lighting.
图3中,21.光电传感器阵列探测到的一行图像光--电信号;22.光电传感器阵列探测到的一行光--电信号21经过积分器峰值电路处理以后的数字化输出信号;23.与光电传感器阵列的一行输出信号22对应的边方向数据,箭头向上表示正边,箭头向下表示负边;24.与一行边方向数据23对应的边反射状况,包括二个峰和一个谷。Among Fig. 3, 21. one row of image light-electric signal that photoelectric sensor array detects; 22. one row light that photoelectric sensor array detects-
图4中,31.前一次拍摄的参考帧里一行边方向数据;32.与前一次拍摄的参考帧里一行边方向数据31对应的边反射状况,包括两个峰和一个谷;41.后一次拍摄的比较帧里一行边方向数据;42.与后一次拍摄的比较帧里一行边方向数据41对应的边反射状况,包括两个峰和一个谷;箭头向上表示正边,箭头向下表示负边。In Fig. 4, 31. a row of edge direction data in the reference frame of the previous shooting; 32. the edge reflection condition corresponding to a row of
上述图2、图3和图4均借鉴自参考资料(US 7,122,781 B2,Oct.17,2006,Rotzoll et al.)。The above Figure 2, Figure 3 and Figure 4 are all borrowed from reference materials (US 7,122,781 B2, Oct.17, 2006, Rotzoll et al.).
图5中,51.光电传感器阵列探测到的一行图像光--电信号,包括一行图像光--电信号510、一行图像光--电信号511以及一行图像光--电信号512三种情况;52.一行图像光--电信号510经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示;53.~57.一行图像光--电信号510向右逐渐发生亚像素位移,经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示。In Fig. 5, 51. a line of image light-electric signal detected by the photoelectric sensor array includes three situations of one line of image light-
具体实施方式 Detailed ways
图1是本发明的测量装置的方框图。所述摄像头(1)由光学透镜(2)、光电成像芯片(3)和摄像头的USB接口(4)组成,通过其配置的USB电缆连接到一台普通的计算机系统(5)的USB接口(6),该计算机系统(5)还配置有CPU(7)、内存(8)、显示卡与显示器(9)、硬盘(10)、键盘和鼠标(11)、操作系统(12)、摄像头驱动程序(13)以及摄像头拍摄及峰/谷运动探测亚像素位移程序(14)。Fig. 1 is a block diagram of the measuring device of the present invention. Described camera head (1) is made up of optical lens (2), photoelectric imaging chip (3) and the USB interface (4) of camera head, is connected to the USB interface (5) of a common computer system (5) by its configured USB cable 6), the computer system (5) is also equipped with CPU (7), internal memory (8), display card and monitor (9), hard disk (10), keyboard and mouse (11), operating system (12), camera driver Program (13) and sub-pixel displacement program (14) for camera capture and peak/valley motion detection.
首先,在计算机(5)上运行随摄像头(1)配售的摄像头驱动程序(13),安装好摄像头(1)到计算机(5)。调节摄像头(1)的光学镜头(2)的焦距,使得被测物体成像清晰。First, run the camera driver (13) distributed with the camera (1) on the computer (5), and install the camera (1) to the computer (5). The focal length of the optical lens (2) of the camera (1) is adjusted so that the image of the measured object is clear.
然后,选择测量环境,或调节相关照明设备(15),尽量做到被测物体上面光照均匀且不发生大的变化。例如,可以在室内环境下进行测量,也可以选择照明光源强于环境光照的场合,还可以把照明光源投向物体前面的一个环形墙幕,再让墙幕上漫反射回来的光线照向被测物体,还可以施加三基色偏振片。Then, the measurement environment is selected, or the related lighting equipment (15) is adjusted, so that the illumination on the measured object is uniform and does not change greatly as far as possible. For example, the measurement can be carried out in an indoor environment, or the occasion where the lighting source is stronger than the ambient light can also be selected, and the lighting source can also be cast on a ring-shaped wall curtain in front of the object, and then the diffusely reflected light on the wall curtain can shine on the measured object. objects, trichroic polarizers can also be applied.
接着,运行摄像头拍摄及峰/谷运动探测亚像素位移程序(14),运用峰/谷亚像素位移探测算法实施实时测量。对此,逐项说明如下。Then, run the camera shooting and peak/valley motion detection sub-pixel displacement program (14), and use the peak/valley sub-pixel displacement detection algorithm to implement real-time measurement. This is explained item by item as follows.
(一)定义光电像素阵列的边方向数据(1) Define the edge direction data of the photoelectric pixel array
进行光强比较的像素位置关系有:1)两个毗连像素之间,2)毗连像素子阵列之间(U.S.patent application Ser.No.10/001,963,Dec.5,2001;U.S.patent application Ser.No.10/001,959,Dec.5,2001),3)非毗连邻近像素之间(参与比较的两个像素之间还有第三个像素),4)其它位置的非毗连邻近像素之间。The pixel position relationship for light intensity comparison is: 1) between two adjacent pixels, 2) between adjacent pixel sub-arrays (U.S.patent application Ser.No.10/001,963, Dec.5, 2001; U.S.patent application Ser. No.10/001,959, Dec.5, 2001), 3) between non-adjacent adjacent pixels (there is a third pixel between the two pixels participating in the comparison), 4) between non-adjacent adjacent pixels in other positions.
当两个参与比较的像素的光强相等的时候,可以采用所谓“滞回函数”(U.S.patentapplication Ser.No.10/001,963)免除边方向数据的不确定性,或者,简单地规定为正边或负边(US 7,122,781 B2)。像素阵列中的四边与四个角上的像素位置不存在边方向数据。When the light intensities of the two pixels involved in the comparison are equal, the so-called "hysteresis function" (U.S. patent application Ser. No. 10/001,963) can be used to avoid the uncertainty of the edge direction data, or simply defined as the positive edge or negative side (US 7,122,781 B2). The pixel positions on the four sides and the four corners in the pixel array do not have side direction data.
美国专利US 7,122,781 B2采用比较非毗连邻近像素之间的光强的方法来确定边方向数据,如此获得的边位于参与比较的两个像素的中间的那个像素上,并认为效果有二:1)等效于对像素的光强图实施了空间媒介过滤器,降低了噪声;2)比较非毗连邻近像素等效于比较三个毗连像素:Pi-Pi+2=(Pi+Pi+1)-(Pi+1+Pi+2),而不需要设置求和电路。一个例子如图2所示,该图表示一个3×3像素阵列以及根据上述定义导出的边方向数据。对于一个给定的光电探测器阵列,存在预定数目的边方向数据。U.S. Patent No. 7,122,781 B2 uses the method of comparing the light intensity between non-adjacent adjacent pixels to determine the edge direction data. The edge obtained in this way is located on the pixel in the middle of the two pixels participating in the comparison. It is believed that there are two effects: 1) It is equivalent to implementing a spatial medium filter on the light intensity map of a pixel to reduce noise; 2) comparing non-adjacent adjacent pixels is equivalent to comparing three adjacent pixels: P i -P i+2 =(P i +P i +1 )-(P i+1 +P i+2 ), without setting up a summation circuit. An example is shown in Figure 2, which represents a 3 × 3 pixel array with edge orientation data derived from the above definition. For a given photodetector array, there is a predetermined number of edge direction data.
本发明采用了上述比较非毗连邻近像素光强大小的边的定义,认为被测物体表面的反射特征就是所谓的表面物理明暗对比度,属于该物体表面材料的光学反射属性,在光照变化相当大的程度上不会发生明显的变化。The present invention adopts the above-mentioned definition of comparing the light intensity of non-adjacent adjacent pixels, and thinks that the reflection feature of the surface of the measured object is the so-called surface physical light-dark contrast, which belongs to the optical reflection property of the surface material of the object. There will be no significant change in degree.
考虑到计算机摄像头距离被测量物体相当远,光电传感器阵列探测到的被测物体表面反射的光强图案会受到周围环境光辐射的影响,带来像素光强值的某种随机的波动,因而本发明在关于边的定义中添加了一个误差容限值error,并认为当两者的强弱差不多的时候,例如I(X,Y+2)-error<I(X,Y)<I(X,Y+2)+error,则定义这两个像素之间不存在一个边,或者说,存在一个第三类边,这对于后述峰、谷的分析不产生影响,但进一步反映了被侧物体表面的反射特征。摄像头的灰度值深度一般为8bit,256个等级值。因此,式中的误差容限值可以根据具体的光照被干扰情况,预置为一个小的数值,例如:error=10。Considering that the computer camera is quite far away from the measured object, the light intensity pattern reflected by the surface of the measured object detected by the photoelectric sensor array will be affected by the ambient light radiation, which will bring some random fluctuations in the pixel light intensity value, so this paper The invention adds an error tolerance value error in the definition of the side, and thinks that when the strength of the two is similar, for example, I(X, Y+2)-error<I(X, Y)<I(X , Y+2)+error, it is defined that there is no edge between these two pixels, or in other words, there is a third type of edge, which does not affect the analysis of peaks and valleys described later, but further reflects the side Reflective features of the surface of an object. The gray value depth of the camera is generally 8bit, 256 levels. Therefore, the error tolerance value in the formula can be preset as a small value, for example, error=10, according to the specific situation that the light is disturbed.
为了进一步有效消除环境光的影响,本发明利用摄像头本身的三种基色(波长)的光电转换数据分量,而非其综合性的光强值,通过比较,导出红、绿和蓝三种基色各自的边方向数据。具体的边方向数据的定义见“发明内容”。In order to further effectively eliminate the influence of ambient light, the present invention uses the photoelectric conversion data components of the three primary colors (wavelengths) of the camera itself, rather than its comprehensive light intensity value, to derive the three primary colors of red, green and blue by comparison. edge direction data. For a specific definition of edge direction data, see "Summary of the Invention".
本发明首先探测被测物体表面的光学反射特征,选取沿坐标轴方向三基色的正边和负边的数目和最多者的一行与一列作为观测的对象。The invention firstly detects the optical reflection characteristics of the surface of the object to be measured, and selects a row and a column with the largest number and the largest number of positive and negative sides of the three primary colors along the direction of the coordinate axis as the object of observation.
(二)定义边反射状况(2) Define edge reflection conditions
按照选定的一行或一列像素的边方向数据,沿X轴和Y轴方向,分别导出边反射状况:连续的两个或两个以上的正边或第三类边的后面跟着一个负边,称之为第一类边反射状况,即认为在此位置存在一个光强的峰;连续的两个或两个以上的负边或第三类边的后面跟着一个正边,称之为第二类边反射状况,即认为在此位置存在一个光强的谷;除此以外,称之为第三类边反射状况。因此,分别获得红、绿和蓝三种基色(波长)的边反射状况。According to the edge direction data of a selected row or column of pixels, along the X-axis and Y-axis directions, the edge reflection conditions are respectively derived: two or more consecutive positive edges or a third type of edge followed by a negative edge, It is called the first type of edge reflection condition, that is, it is considered that there is a peak of light intensity at this position; two or more consecutive negative sides or the third type of side followed by a positive side are called the second In the case of edge reflection, it is considered that there is a valley of light intensity at this position; otherwise, it is called the third type of edge reflection. Therefore, the edge reflection conditions of the three primary colors (wavelengths) of red, green and blue are respectively obtained.
(三)光照稳定的情况下跟踪峰、谷的运动(3) Tracking the movement of peaks and valleys under stable light conditions
所谓光照稳定,意味着被测物体表面的物理明暗对比度不会随着光照的强弱或角度的变化而变化。The so-called stable light means that the physical contrast between light and dark on the surface of the measured object will not change with the intensity of light or the change of angle.
图3表示光电探测器阵列沿着某个坐标轴向获得的图像光电信号以及据此导出的一个序列的边方向数据(正边和负边)和边反射状况(峰和谷)。Fig. 3 shows the image photoelectric signals obtained by the photodetector array along a certain coordinate axis and a sequence of edge direction data (positive edge and negative edge) and edge reflection conditions (peaks and valleys) derived therefrom.
图4是沿着某个轴跟踪边反射状况进行亚像素位移测量的示意图。由图可以看到,第一次拍摄获得的谷位于第四个(负)边与第五个(正)边之间,而在第二次拍摄获得的谷位于第五个(负)边与第六个(正)边之间,两次顺序拍摄期间,该谷向右运动了一个单位像素间距,文献称此情形为典型的亚像素运动(sub-pixel motion:displacement of less than the pixelpitch between two successive flashes),意思是其位移小于一个单位的像素间距。即是说,在拍摄参考帧到拍摄比较帧的时间间隔dt内,所测量的运动物体其成像像素阵列发生的位移应当小于一个像素间距单位。这可以根据被测量的物体运动的最大的速度,通过调节拍摄时间间隔dt来实现。FIG. 4 is a schematic diagram of sub-pixel displacement measurement by tracking edge reflection along a certain axis. It can be seen from the figure that the valley obtained in the first shooting is located between the fourth (negative) side and the fifth (positive) side, while the valley obtained in the second shooting is located between the fifth (negative) side and Between the sixth (positive) side, during the two sequential shots, the valley moved to the right by a unit pixel pitch. The literature calls this situation a typical sub-pixel motion (sub-pixel motion: displacement of less than the pixelpitch between two successive flashes), which means that the displacement is less than one unit of pixel pitch. That is to say, within the time interval dt from shooting the reference frame to shooting the comparison frame, the measured displacement of the imaging pixel array of the moving object should be less than one pixel pitch unit. This can be achieved by adjusting the recording time interval dt according to the maximum velocity of the object being measured.
只跟踪边反射状况而不论其类型,在跟踪过程中会发生信息与测量精度的损失。By tracking only edge reflection conditions regardless of their type, a loss of information and measurement accuracy occurs during the tracking process.
因此,对应每一个坐标轴各设置2×2个累加器,沿着坐标轴的方向分别跟踪峰、谷各自向正的或向负的坐标轴方向发生运动的个数,方法是:比较参考帧与比较帧里所选定的像素行与列之对应的峰、谷的位置,籍此判断所选定的像素行与列里对应的峰、谷的移动方向。沿每个坐标轴再分别设置2个累加器,计数每个坐标轴上峰、谷各自的总的数目。具体说明见“发明内容”。Therefore, 2×2 accumulators are set corresponding to each coordinate axis, and the number of peaks and valleys moving towards the positive or negative coordinate axis direction is tracked respectively along the direction of the coordinate axis. The method is: compare the reference frame The positions of peaks and valleys corresponding to the selected pixel rows and columns in the frame are compared with the corresponding peaks and valleys, so as to determine the moving direction of the corresponding peaks and valleys in the selected pixel rows and columns. Two accumulators are respectively set along each coordinate axis to count the respective total numbers of peaks and valleys on each coordinate axis. See "Content of the Invention" for details.
位移遵守矢量合成的原则。作为像素矩阵,还可以沿着X轴、Y轴的对角线方向探测边反射状况的运动。因此,一个对应的边反射状况可以源自:1)同轴上左右两个最邻近位置,2)对角线上四个最邻近位置之一。The displacement follows the principle of vector composition. As a matrix of pixels, it is also possible to detect movement of edge reflections along the diagonal directions of the X-axis and Y-axis. Thus, a corresponding edge reflection condition can originate from: 1) the two nearest neighbors left and right on the coaxial line, and 2) one of the four nearest neighbors on the diagonal.
(四)峰/谷亚像素位移探测算法(4) Peak/valley sub-pixel displacement detection algorithm
图3中,光电传感器阵列沿着某个坐标轴方向探测得到的光强图案21属于被测量物体的表面反射性质,光照不变的情况下,该光强图案不会发生变化。被测物体表面相对摄像头发生位移,该光强图案会相应地向左或向右移动。假设拍摄的时间间隔内该被测物体只发生了一次一个方向的位移,并且,发生的位移小于1个像素间距,便会出现图4所示的情形,峰、谷只朝一个方向移动。In FIG. 3 , the
图5进一步考察了光电传感器阵列探测到的一行图像光--电信号510逐步发生向右位移的情形。该行图像光--电信号510经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示52表示了最初的情形,该行图像光--电信号510向右逐渐发生亚像素位移,经过积分器峰值电路处理以后的数字化输出信号及其相应的正边、负边和峰、谷的图示53~57分别表示了随后发生的亚像素位移情形。由图可见,在发生半个像素大小的位移的时候,其峰(图5中标号54-D--E处)的右邻的负边变成了第三类边(图5中标号54--E处),但该峰的位置没有发生变化;当发生了大于半个像素大小的位移的时候,该峰的右邻的负边经过了第三类边(图5中标号54--E处)的变化过程变化成正边(图5中标号55--E处),该峰的位置也向右移动了一个像素单位。考察非对称的一行图像光--电信号511或512,当其逐渐向右位移的时候,也有相同的结论。此时发生的亚像素位移可以计为0.5pixcel,测量误差在±0.25pixcel~±0.5pixcel之间。FIG. 5 further examines the situation that a line of image light-
由于被测物体属于刚性固体,当光电像素阵列与被测表面发生相对位移的时候,所有的峰和所有的谷都应当向着同一个方向移动。实际上,在拍摄间歇期dt内被测物体发生的位移虽然小于1个像素间距,但是,它可能既朝左又朝右发生了多次位移,因此,峰、谷有朝左的移动,又有朝右的移动。但是,综合起来,结果应当还是所有的峰和所有的谷都向着同一个方向移动。如果峰(或谷)同时有向左和向右的累计数目,两者应当相差悬殊,数目少的反映了光照变化对被测表面的反射对比度特征的影响。因此,Since the measured object is a rigid solid, when the photopixel array is displaced relative to the measured surface, all peaks and all valleys should move in the same direction. In fact, although the displacement of the measured object during the shooting interval dt is less than 1 pixel pitch, it may have multiple displacements both to the left and to the right. Therefore, the peaks and valleys move to the left, and There is a move to the right. However, taken together, the result should still be that all peaks and all valleys move in the same direction. If the peaks (or valleys) have both leftward and rightward cumulative numbers, the two should be very different, and the smaller number reflects the impact of illumination changes on the reflection contrast characteristics of the measured surface. therefore,
如果N峰向右移(j+1)>N峰向左移(j+1),则计为ΔX峰位移(j+1)=+0.5,If N peak shifts to the right (j+1)>N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=+0.5,
如果N峰向右移(j+1)<N峰向左移(j+1),则计为ΔX峰位移(j+1)=-0.5;If N peak shifts to the right (j+1)<N peak shifts to the left (j+1), it is calculated as ΔX peak displacement (j+1)=-0.5;
如果N谷向右移(j+1)>N谷向左移(j+1),则计为ΔX谷位移(j+1)=+0.5,If the N valley moves to the right (j+1)>N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=+0.5,
如果N谷向右移(j+1)<N谷向左移(j+1),则计为ΔX谷位移(j+1)=-0.5;If N valley moves to the right (j+1)<N valley moves to the left (j+1), it is counted as ΔX valley displacement (j+1)=-0.5;
对于Y轴方向也会有类似的结论。A similar conclusion can be drawn for the Y-axis direction.
(五)光照发生变化的情况下跟踪峰、谷的运动(5) Tracking the movement of peaks and valleys when the light changes
图3中,如果光照发生变化,光电传感器阵列沿着某个坐标轴方向探测得到的光强图案21不再只是属于被测量物体的表面反射性质,还要反映出环境光辐射发生改变的影响效果。In Figure 3, if the illumination changes, the
一种情况是,该光电传感器阵列探测到的光强图案虽然发生了变化,但是,被测物体表面的反射属性仍然占主要地位,传感器先后两次拍摄“看”到的该坐标轴方向的边方向数据以及边反射状况的数目并没有发生明显的变化。One situation is that although the light intensity pattern detected by the photoelectric sensor array has changed, the reflective properties of the surface of the measured object still dominate, and the sensor takes two shots of the edge in the direction of the coordinate axis that it "sees". The orientation data and the number of edge reflection conditions did not change significantly.
另一种情况是,被测物体整体上的光照发生变化,或其局部表面突然被环境光强烈地改变,导致图3中光电传感器阵列探测到的光强图案21发生了值得关注的变化,传感器先后两次拍摄“看”到的该轴方向的边方向数据以及边反射状况的数目会发生明显的变化。这时,重新开始测量,包括重新选取观测行与观测列。由于摄像头拍摄的速率远快于被测物体的位移速度,也远快于环境光的辐照变化速度,加上所述峰/谷亚像素位移探测算法只针对光电传感器阵列里的一行和一列像素,计算速度相对地快,只要被测物体表面的光学反射属性依然在成像帧里占主要地位,仍然可以使用上述峰/谷亚像素位移探测算法表示式来计算即时的位移值,这项措施对于测量结果不会带来多少误差。Another situation is that the overall illumination of the measured object changes, or its local surface is suddenly strongly changed by ambient light, resulting in a noteworthy change in the
上述两种变化情况可以统一起来进行检查,或者探测边反射状况的数目,或者探测文献所述的“鬼影边反射状况”的数目。采用文献所述的“鬼影边反射状况”的“预置阈值”来检验,不如采用其相对该轴向所有的峰、谷数目的百分比来得方便和灵活。The above two variations can be checked together, either to detect the number of edge reflection conditions, or to detect the number of "ghost edge reflection conditions" described in the literature. It is more convenient and flexible to use the "preset threshold" of "ghost edge reflection" described in the literature to check than to use its percentage relative to the number of peaks and valleys in this axis.
需要被测物体表面的光学反射属性在成像帧里占主要地位,意味着峰/谷亚像素位移探测算法能够识别被测物体的特征,这是本发明的关键所在;这与被测物体表面的光学性质(材料的质地、纹路等)、照明光源以及测量环境(干扰光源、灰尘、烟雾)等因素有关。如果选用表面质地的光学反射属性较易辨析的材料作靶标,或者控制被测物体的光照免遭环境的过分干扰,可以取得较好的测量效果。The optical reflection properties of the surface of the measured object are required to take a dominant position in the imaging frame, which means that the peak/valley sub-pixel displacement detection algorithm can identify the characteristics of the measured object, which is the key of the present invention; Optical properties (material texture, texture, etc.), lighting source and measurement environment (interfering light source, dust, smoke) and other factors are related. Better measurement results can be obtained if a material whose optical reflection property of the surface texture is easier to distinguish is selected as the target, or if the light of the measured object is controlled to avoid excessive interference from the environment.
摄像头的输出包括三种基色的分量值,因此,分别测量这三种基色的边反射状况,并加以平均,可以克服其它波长的环境杂散光变化对测量的影响。The output of the camera includes the component values of the three primary colors. Therefore, measuring the edge reflections of the three primary colors and averaging them can overcome the influence of ambient stray light changes of other wavelengths on the measurement.
实际上,注意到,较之红外线二极管照明的光电鼠标,采用激光二极管照明的光电鼠标能够适应光滑的瓷砖表面。如果能够应用某种不易受环境杂散光源影响的波长的光源,配合选用敏感该波长的光电成像传感器阵列(摄像头),本发明能够充分地发挥适应较复杂的光照环境的能力,取得无接触测量物体微小位移的优良效果。In fact, it was noticed that the optical mouse with laser diode illumination was able to adapt to the smooth tile surface compared to the optical mouse with infrared diode illumination. If a light source with a wavelength that is not easily affected by environmental stray light sources can be used, and a photoelectric imaging sensor array (camera) sensitive to this wavelength can be selected, the present invention can fully exert its ability to adapt to more complex lighting environments, and achieve non-contact measurement. Excellent effect of small displacement of objects.
Claims (4)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200910190924 CN102022981B (en) | 2009-09-22 | 2009-09-22 | Peak-valley motion detection method and device for measuring sub-pixel displacement |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200910190924 CN102022981B (en) | 2009-09-22 | 2009-09-22 | Peak-valley motion detection method and device for measuring sub-pixel displacement |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102022981A CN102022981A (en) | 2011-04-20 |
CN102022981B true CN102022981B (en) | 2013-04-03 |
Family
ID=43864565
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 200910190924 Expired - Fee Related CN102022981B (en) | 2009-09-22 | 2009-09-22 | Peak-valley motion detection method and device for measuring sub-pixel displacement |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102022981B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102052900B (en) * | 2009-11-02 | 2013-09-25 | 重庆工商大学 | Peak valley motion detection method and device for quickly measuring sub-pixel displacement |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1135062A (en) * | 1995-03-02 | 1996-11-06 | 惠普公司 | Freehand image scanning device and method |
US6249591B1 (en) * | 1997-08-25 | 2001-06-19 | Hewlett-Packard Company | Method and apparatus for control of robotic grip or for activating contrast-based navigation |
CN1350633A (en) * | 1999-05-14 | 2002-05-22 | 北京三维世界科技有限公司 | 3D-imaging system |
CN1363827A (en) * | 2000-12-08 | 2002-08-14 | 格莱特格-曼克贝斯公开股份有限公司 | Individual element photoelectric measuring device for plane object |
-
2009
- 2009-09-22 CN CN 200910190924 patent/CN102022981B/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1135062A (en) * | 1995-03-02 | 1996-11-06 | 惠普公司 | Freehand image scanning device and method |
US6249591B1 (en) * | 1997-08-25 | 2001-06-19 | Hewlett-Packard Company | Method and apparatus for control of robotic grip or for activating contrast-based navigation |
CN1350633A (en) * | 1999-05-14 | 2002-05-22 | 北京三维世界科技有限公司 | 3D-imaging system |
CN1363827A (en) * | 2000-12-08 | 2002-08-14 | 格莱特格-曼克贝斯公开股份有限公司 | Individual element photoelectric measuring device for plane object |
Also Published As
Publication number | Publication date |
---|---|
CN102022981A (en) | 2011-04-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7557935B2 (en) | Optical coordinate input device comprising few elements | |
US5793357A (en) | Device and method for determining movement of a surface | |
CN101971072B (en) | Image sensor and focus detection apparatus | |
Callenberg et al. | Low-cost SPAD sensing for non-line-of-sight tracking, material classification and depth imaging | |
CN104395694B (en) | Motion sensor device with multiple light sources | |
TWI482054B (en) | High resolution and high sensitivity cursor maneuvering device using multiple color light sources | |
JP6509027B2 (en) | Object tracking device, optical apparatus, imaging device, control method of object tracking device, program | |
JP6299983B2 (en) | Motion sensor device having a plurality of light sources | |
CN112924028B (en) | A light field polarization imaging detection system for oil spills on the sea surface | |
US6907672B2 (en) | System and method for measuring three-dimensional objects using displacements of elongate measuring members | |
Tabata et al. | High-speed 3D sensing with three-view geometry using a segmented pattern | |
CN111707413A (en) | A Centroid Detection Method Based on Single Pixel Detector | |
GB2272763A (en) | Device and method for determining movement | |
US20100110209A1 (en) | Fast motion measurement device for gaming | |
TWI421737B (en) | Methods for measuring relative motion between a surface and an optical sensing device | |
CN101452349B (en) | Cursor control device, method and image system on image display device | |
CN108269270A (en) | Sun detection, tracks of device, method and application | |
CN101216745A (en) | Optical point plane position detection system based on linear array CCD | |
CN102022981B (en) | Peak-valley motion detection method and device for measuring sub-pixel displacement | |
CN115598744A (en) | High-dimensional light field event camera based on micro-lens array and extraction method | |
GB2353856A (en) | High resolution detector array | |
WO2022016412A1 (en) | Depth information image acquisition apparatus and electronic device | |
CN111583191A (en) | Refraction feature detection method based on light field EPI Fourier transform | |
US20190346598A1 (en) | Imaging systems and methods with periodic gratings with homologous pixels | |
CN102052900B (en) | Peak valley motion detection method and device for quickly measuring sub-pixel displacement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
C17 | Cessation of patent right | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130403 Termination date: 20130922 |