CN102243537B - Method and device for detecting displacement of motion image as well as optical mouse - Google Patents

Method and device for detecting displacement of motion image as well as optical mouse Download PDF

Info

Publication number
CN102243537B
CN102243537B CN 201010176100 CN201010176100A CN102243537B CN 102243537 B CN102243537 B CN 102243537B CN 201010176100 CN201010176100 CN 201010176100 CN 201010176100 A CN201010176100 A CN 201010176100A CN 102243537 B CN102243537 B CN 102243537B
Authority
CN
China
Prior art keywords
image
frame
difference
data
current
Prior art date
Application number
CN 201010176100
Other languages
Chinese (zh)
Other versions
CN102243537A (en
Inventor
李冰
杨智明
王卓华
Original Assignee
深圳市汇春科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市汇春科技有限公司 filed Critical 深圳市汇春科技有限公司
Priority to CN 201010176100 priority Critical patent/CN102243537B/en
Publication of CN102243537A publication Critical patent/CN102243537A/en
Application granted granted Critical
Publication of CN102243537B publication Critical patent/CN102243537B/en

Links

Abstract

一种运动图像位移检测方法及装置,用于分析具有时间关系的图像序列以确定采集该图像序列过程中光感应器件相对于物体的运动轨迹,可用于光电鼠标。 A moving image displacement detecting method and apparatus for analyzing image sequences having a temporal relationship to determine a sequence of images acquired through Cheng Zhongguang sensing device relative to the trajectory of the object, can be used for optical mouse. 令图像信号处理单元按时间顺序接收所述图像序列;基于所述图像序列,依次对各当前图像进行数据处理,依据与该当前图像相对应的差分图像上的各像素进行二维光流计算处理,采用全局矩阵运算方法来计算并输出与当前图像对应的位移矢量。 So that the image signal processing unit receives the sequence of images in chronological order; based on the sequence of images, sequentially for each current image data processing, based on a two-dimensional optical flow calculation process on each pixel of the image corresponding to the current difference image using global matrix operation method calculates and outputs the displacement vector corresponding to the current image. 采用本发明的光电鼠标,导航定位的精确度和检测的稳定性均得到有效提高。 According to the present invention, optical mouse, the detection accuracy and stability navigation positioning are improved effectively.

Description

运动图像位移检测方法及装置、光电鼠标 Moving image displacement detecting method and apparatus, optical mouse

技术领域 FIELD

[0001 ] 本发明涉及以运动图像为分析对象的电数字数据处理技术,尤其涉及用于光电鼠标领域的运动位移检测的数据处理方法及装置。 [0001] The present invention relates to electric digital data processing techniques for the analysis of a moving image object, in particular, it relates to a data processing method and apparatus for detecting motion displacement field of optical mouse.

背景技术 Background technique

[0002] 作为当前鼠标界的主流,光电鼠标所采用的发光二极管定位方法也成了目前的主流定位方法。 [0002] The method of positioning a light emitting diode as a current mainstream industry mouse, the optical mouse has become used in current mainstream positioning method.

[0003] 图1示意了光电鼠标的内部结构,包括发光二极管1,来自该发光二极管I的光线被一组光学反射镜2引导并照射往与光电鼠标底部相接触的表面;被该表面反射回的光线一部分经一组光学透镜2'传输到光感应器件3 (例如但不限于可输出数字图像信号的微成像器)进行成像。 [0003] FIG 1 illustrates a optical mouse internal structure, including a light emitting diode 1, the light from the light emitting diode I is a set of optical reflector 2 is guided and irradiated surface to the photoelectric bottom of the mouse contact; is reflected back to the surface a portion of the light of a set of optical lenses 2 'to a light sensing device 3 (e.g., but not limited to micro-imager may output a digital image signal) imaging. 这样,当光电鼠标移动时,其移动轨迹便会被高速“连拍”所记录下来的一组图像所揭示。 Thus, when the optical mouse movement, which would be a set of trajectories speed image "CS" is recorded disclosed. 利用光电鼠标所包括的图像分析器4(可以是基于数字信号处理器DSP的专用芯片)来分析处理该组图像,即可得出所述移动轨迹。 Using the optical mouse comprising the image analyzer 4 are (which may be based on a digital signal processor DSP ASIC) to analyze the set of image processing, the trajectories can be derived. 在实际应用中,所述光感应器件3和图像分析器4往往被集成在一颗芯片内。 In practical applications, the optical sensing device 3 and the image analyzer 4 are often integrated in one chip.

[0004] 图2示意了光电鼠标的电工作原理,虚线框内示意的是光电鼠标的电结构,具有一连接按键的接口及一连接电脑主机的接口。 [0004] FIG 2 illustrates a principle of an electrical optical mouse, within the dashed box is a schematic electrical configuration of the optical mouse, having a connection interface, and an interface connected to a host computer keys. 该电结构包括图像传感器(sensor),用来采集并将光图像信号转换为电图像信号;该电图像信号被送往模数转换器以转换成数字图像信号,再被送往图像信号处理单元来计算生成鼠标位移矢量;微控制单元通过电连接来控制包括图像传感器、模数转换器或图像信号处理单元在内的各电路模块,及接收来自各电路模块的状态信号;来自图像信号处理单元的位移矢量数据和来自鼠标按键的按键信息通过该微控制单元一起打包发送给电脑主机。 The electrical configuration comprises an image sensor (Sensor), and used to collect light image signal is converted into an electrical image signal; electrical image signal is sent to the analog to digital converter into a digital image signal, and then is sent to the image signal processing unit generating a displacement vector is calculated mouse; micro control unit controls each circuit module comprises an image sensor, an analog image signal processing unit or by an electrical connector including, and receive status signals from each circuit block; from the image signal processing unit displacement vector data and the key information from the mouse button with the package sent to the host computer through the micro control unit. 所述微成像器至少包括了图像传感器和模数转换器。 The micro-imager comprises at least the image sensor and ADC. 图像信号处理单元可以采用专用的一颗芯片,或者与微控制单元一起集成在一颗芯片内。 The image signal processing unit may use a dedicated chip, or integrated in a chip with the micro control unit.

[0005] 图3示意了所述鼠标位移矢量。 [0005] FIG. 3 illustrates a displacement vector of the mouse. 假设a、b为对应先后时间的两帧图像,三角形代表图像上的同一特征点,则图像信号处理单元通过提取所述特征点和分析其位置变化来判断鼠标的移动方向和移动距离。 Suppose a, b is the image corresponding to two times successively, the same triangles represent the feature points on the image, the image signal processing unit extracts the feature points and analyzed to determine the position change of moving direction and a moving distance of the mouse. 图c示意了特征点位置变化所计算得出的ΛΧ,ΛΥ,唯一地对应于鼠标位移矢量,从而完成光标定位。 FIG c illustrates a variation of the calculated feature point position obtained ΛΧ, ΛΥ, uniquely displacement vector corresponding to the mouse, thereby completing the cursor. 现有定位方法大多采用基于相关匹配的插值算法来获取鼠标的运动矢量,如图4所示,可以通过在下一帧(如图b)中寻找与当前帧(如图a)特征点最佳匹配的位置来获得当前的位移矢量(如图c),其中图d示意了一个3*3像素的模板,可以以该模板为单元在图像区域中寻找最佳匹配。 Most conventional positioning method using a mouse to obtain a motion vector interpolation algorithm based on correlation matching, shown in Figure 4, by looking for the next frame (as shown in b) best matches the current frame (as shown in a) of feature points to obtain the current position of the displacement vector (FIG. C), where d schematically in FIG. 3 * 3 pixels of a template, can find the best match to the template image region units. 该方法具有性能稳定的优点,但由于插值算法的局限性,无法有效获得亚像素大小的精度。 This method has the advantages of stable performance, but because of the limitations of the interpolation algorithm, the sub-pixel accuracy can not be efficiently obtained size.

[0006] 为了获得较高的运动精度,现有技术进一步将计算区域光流的方法用于鼠标定位,以获得高达I / 4像素以上的精度。 [0006] To achieve higher precision movement, the prior art method further calculates the optical flow region for mouse positioning accuracy of up to obtain I / 4 or more pixels. 该方法先对图像上的像素建立超定二元光流方程,通过该方程来求解获得单个像素的运动矢量,并用最小二乘法来拟合出多个像素的运动矢量均值,即鼠标的位移向量。 The method of establishing the first pixels on the image over a given binary optical flow equations to solve for a single pixel motion vector obtained by the equation, and using the least squares method to fit a plurality of pixels of the mean motion vectors, i.e., displacement vectors mouse . 为了简化对该方法的描述,图5把来自光感应器件的图像信号简化成一维信号为例进行示意。 To simplify the description of the method, FIG. 5 the image signal from the light sensing device is a one-dimensional signal as an example to simplify the schematic. 设曲线A为t时刻信号,曲线B为t+At时刻信号;假定在t时刻,X位置的信号幅度值为f (X,t) ;it+A t时亥ϋ,χ+Δχ位置的信号幅度值为f (χ+Δχ,t+At)。 Curve A is a set signal at time t, t + At is the curve B time signal; assumed that at time t, X position signal magnitude value f (X, t); it + A when t Hai ϋ, χ + Δχ position signal amplitude value of f (χ + Δχ, t + At). 如果信号在At时间内移动了At距离,那么在不考虑其它干扰的情况下,可以知道: If the signal distance moved At At the time, the other is not considered in the case of interference, it is known:

[0007] f (χ+ Δ x, t+ Δ t) =f (χ, t) (I) [0007] f (χ + Δ x, t + Δ t) = f (χ, t) (I)

[0008] 对上式左边作泰勒展开得: [0008] on the left-hand side of the Taylor expansion was made:

[0009] [0009]

Figure CN102243537BD00061

[0010] 联立⑴式,并忽略ο(Λχ,At)得: [0010] simultaneous ⑴ formula, and ignore ο (Λχ, At) to give:

[0011 ] [0011]

Figure CN102243537BD00062

[0012]即: [0012] That is:

[0013] Ixux+It=0 (3) [0013] Ixux + It = 0 (3)

[0014] 式中, [0014] In the formula,

Figure CN102243537BD00063

为信号在χ位置的斜率, Signal slope χ position,

Figure CN102243537BD00064

为信号在X位置随时间的变化率。 Time rate of change of the signal in the X position.

Figure CN102243537BD00065

为信号在X方向上的运动速度。 A motion speed signal in the X direction. 式(3)即为一维运动信号的光流方程。 (3) is the optical flow equation of one-dimensional motion signal.

[0015] 同理,二维信号的光流方程为: [0015] Similarly, the optical signal is a two-dimensional flow equation:

[0016] Ixux+Iyuy+It=0 ⑷ [0016] Ixux + Iyuy + It = 0 ⑷

[0017] 此时一个方程不足以求解两个未知数ux,uy。 [0017] At this time, insufficient to solve one equation with two unknowns ux, uy. 因此要寻找多个关于ux,Uy的方程来 So to look for multiple equations on ux, Uy to

求解: Solving:

[0018] [0018]

Figure CN102243537BD00066

[0021] 以公式6为指导依据,在所述图像信号处理单元内,现有光流法计算位移的信号处理电路具有如图6所示的处理流程。 [0021] In Equation 6 based on the guidance, in the image signal processing unit calculates the displacement of the conventional optical flow signal processing circuit having a processing flow shown in FIG. 6. 数字图像信号被送往该信号处理电路后,先经过一个除噪、求导处理模块来生成空间导数Ix,Iy(像素梯度值的两个分量)和时间导数It,及由 After the digital image signal is sent to the signal processing circuit, via a first coring, the processing module generates derivation spatial derivative Ix, Iy (two component gradient value of the pixel) and the time derivative of It, and the

此进一步计算获得导数的积7K,人A,7』,7』.特征区域查找模块及其后续的参数计算模 This product is further obtained by calculation of the derivative 7K, people A, 7 ', 7. "Lookup module feature region and the subsequent parameter calculation module

块根据所述数字图像信号来选取图像上的特定区域,以该区域的m*m=n个像素为模板来查找特征区域,并基于该特征区域利用来自除噪、求导处理模块的对应数据来计算方程6中的参数a、b、C、e和f,即 The block of the digital image signal to select a specific region on the image to m * m = n pixels in the region as templates to find the feature region, and with the corresponding data from the coring, the processing module based on the derivation of the feature region in equation 6 to calculate parameters a, b, C, e and f, i.e.,

Figure CN102243537BD00071

各参数的累加和;位 And accumulating each parameter; bit

移矢量计算模块再根据这些参数来计算并输出位移矢量(ux,Uy)。 Displacement vector calculation module further calculates these parameters and outputs the displacement vector (ux, Uy). 上述处理中,一般选取梯度值较大的区域作为特征区域。 In the above-described process, generally a larger area gradient value selected as a feature region.

[0022] 上述现有技术的缺点在于:计算所述位移矢量时容易被本底干扰所影响,从而影响鼠标的定位精度。 Drawback [0022] The prior art is that: is easily affected by background interference when calculating the displacement vector, which affects the positioning accuracy of the mouse. 由于光学透镜(例如带杂质或反射率不均匀)及光传感器后期封装等方面的影响,来自图1所示结构中光感应器件3的图像将带有不可避免的本底噪声,此外鼠标电路本身也存在固定的噪声干扰。 Since the optical lens (e.g., impure or non-uniform reflectance) and the light sensor affect other aspects of post-packaging, the structure of the light image from the induction device 3 shown in FIG. 1 with the inevitable background noise, in addition the mouse circuit itself there is a fixed noise. 如图7所示,图7a为干扰信号可忽略不计时以两条曲线I和2分别代表前后两帧图像信号;7b表示本底干扰不能忽略,即相对于本底干扰(以弯曲的粗线3来表示)有效信号较微弱的情况,此时采用光流法进行参数计算将带来很大计算误差。 7, FIG. 7a is a signal interference is negligible to two image signals before and after the two curves I and 2 represent; 7B represents the background interference can not be ignored, i.e., with respect to background interference (curved thick line 3 represented) valid signal weaker case, at this time the parameter is calculated using an optical flow calculation method great error.

发明内容 SUMMARY

[0023] 本发明要解决的技术问题是针对上述现有技术的不足之处,而提出一种用于运动图像位移检测的方法及装置,来有效降低运动采集图像中的本底干扰的影响。 [0023] The present invention is to solve the technical problem for the shortcomings of the above-described prior art, the proposed method and device for detecting a displacement of the moving image to reduce the influence of background interference in the captured image motion.

[0024] 为解决上述技术问题,本发明的基本构思为:考虑到本底噪声具有变化缓慢的特点,对于高速连拍所获得的前后帧图像而言,可以假设其本底噪声近似一致,因此若用帧间差分图像信号来代替图像信号进行二维光流计算处理,且在运算过程中采用全局矩阵运算方法,无疑将大为排除本底噪声干扰的影响,并简化运算过程。 [0024] To solve the above technical problem, the basic idea of ​​the invention is: Taking into account the noise floor has a slow change in the characteristics for high-speed continuous image before and after the frame obtained, it can be assumed that the noise floor is approximately uniform, so when the interframe difference image signal instead of the image signal is a two-dimensional optical flow calculation process, and a method using a global matrix operation in the arithmetic process, will undoubtedly greatly eliminate the influence of background noise, and to simplify calculations.

[0025] 作为实现本发明构思的技术方案是,提供一种运动图像位移检测方法,用于分析具有时间关系的图像序列以确定采集该图像序列时光感应器件相对于物体的运动轨迹,包括步骤: [0025] As the concept of the present invention is to realize the technical solution is to provide a moving image displacement detection method for analyzing an image sequence having a temporal relationship to determine a sequence of images acquired of the sensor devices with respect to time trajectory of the object, comprising the steps of:

[0026] A.图像信号处理单元按时间顺序接收所述图像序列; [0026] A. The image signal processing unit receives the sequence of images in chronological order;

[0027] 尤其是,所述图像序列包括三帧或三帧以上的图像;还包括步骤 [0027] In particular, the sequence of images comprising three or more than three image frames; further comprising the step of

[0028] B.基于所述图像序列,依次对各当前图像进行数据处理,依据与该当前图像相对应的差分图像上的各像素进行计算,按下列公式计算并输出与当前图像对应的位移矢量 [0028] B. Based on the sequence of images, sequentially processing each of the current image data, each pixel is calculated based on the image corresponding to the current difference image is calculated by the following equation and outputs the current image and the corresponding displacement vector

(ux, Uy) (Ux, Uy)

[0029] [0029]

Figure CN102243537BD00072

.其中Dx,Dy为差分像素的空间 Among Dx, Dy differential pixel space

.导数,代表着差分像素梯度值的两个分量;Dt为差分图像的时间导数;n为该差分图像内的 . Derivative, two components representing the gradient pixel difference values; Dt is the time derivative of the difference image; differential image for the n-

像素数。 The number of pixels.

[0030] 具体地说,上述方案中,所述步骤B具体为包括步骤: [0030] Specifically, the above-described embodiment, the step B comprises the steps of:

[0031] ①根据所述图像序列来产生差分图像序列,即把该图像序列中当前帧图像数据的一后帧图像数据减去当前帧图像数据后作为当前帧的差分图像数据,再设定该后帧图像数 [0031] ① generates a differential image sequence according to the sequence of images, i.e. the image data of a subsequent frame of the sequence of image data of the current frame image by subtracting the difference image data of the current frame image as the current frame data, and then sets the after the frame image number

据为当前帧图像数据,......依此类推直到获得两帧或两帧以上的差分图像来构成所述差 According to the current frame image data, ...... and so on until two or more than two of the difference image constituted of the difference

分图像序列;[0032] ②利用当前差分图像及其一后帧差分图像来计算当前差分图像中每一像素的所述空间导数Dx,Dy和时间导数Dt ; Partial image sequence; [0032] ② calculating a current difference image to the spatial derivative of each pixel Dx, Dy, and derivative time Dt after using the current difference image and a difference image;

[0033]③通过两两相乘来运算得到所述公式中的各中间系数巧,巧2,ZW仏舄,AA; Each intermediate coefficients [0033] ③ by calculating pairwise to give the formula Qiao, Qiao 2, ZW Fo shoe, AA;

[0034] ④在当前差分图像范围内计算所述公式中的各中间系数的累加和;并进而计算得到所述位移矢量(ux, Uy)。 [0034] ④ the intermediate coefficients calculated for each formula and accumulated within the current range of the differential image; thus the calculated displacement vector (ux, Uy).

[0035] 作为实现本发明构思的技术方案还是,提供一种运动图像位移检测装置,用于光电鼠标;包括一个用来接收数字图像信号的数据接口以及一个数据输出口;尤其是,还包括串连连接的差分处理模块及求导处理模块,来自所述数据接口的数字图像信号经差分处理模块以帧为单位进行帧间数据的差分运算及求导处理模块的求导运算后,输出与当前帧图像相对应的差分图像的每一像素的空间导数Dx,Dy和时间导数Dt送往全局运算 [0035] As the concept of the present invention achieve technical solution is to provide a moving image displacement detecting means for optical mouse; comprises a data interface for output of data, and a received digital image signal; especially, further comprising a string even differential processing modules connected to the processing module and the derivation, the difference digital image signal processing module from the data interface in frame units interframe difference calculation and data processing module derivation derivation operation, the output current each pixel spatial derivative Dx corresponding to the frame image of the difference image, Dy, and the time derivative Dt sent global operator

模块,分别计算该差分图像预定范围内所有像素的各个中间系数,Zg,久Z)y,AA Modules, each intermediate coefficients were calculated for all the pixels in the difference image within the predetermined range, the Zg, long-Z) y, AA

的累加和;该全局运算模块输出这些累加和送往位移矢量计算单元,以按公式 And accumulation; global operator module outputs the accumulated and sent to the displacement vector calculating unit, according to the formula

Figure CN102243537BD00081

来计算位移矢量(Ux,uy),其中η为所述 Calculating a displacement vector (Ux, uy), wherein said η

差分图像预定范围内的像素的数量;该位移矢量计算单元输出位移矢量(ux,Uy)往所述数据输出口。 The number of pixels within a predetermined range of the differential image; vector calculating unit outputs a displacement of the displacement vector (ux, Uy) to said data output port.

[0036] 作为实现本发明构思的技术方案还是,提供一种光电鼠标,包括提供照射光的发光二极管,用来感应被物面反射的照射光并将光信号转换成模拟图像信号的图像传感器,将所述模拟图像信号转换成数字图像信号的模数转换器,以及一个接收所述数字图像信号的图像信号处理单元;还包括一用来电连接电脑主机和鼠标按键的微控制单元;所述图像信号处理单元的数据输出口连接所述微控制单元;尤其是,该图像信号处理单元为所述一种运动图像位移检测装置。 [0036] As the concept of the present invention achieve technical solution is to provide an optical mouse, including light emitting diodes to provide illumination light, an image sensor for sensing light signal and converts the irradiation light reflected by the object plane into an analog image signal, the analog to digital converter converts the analog image signal into a digital image signal, and an image signal processing unit for receiving the digital image signal; further comprising a micro control unit for electrically connecting a host computer and a mouse button; and the image the data output port is connected to the signal processing unit micro control unit; in particular, the image signal processing unit is a moving image of the one displacement detecting means.

[0037] 采用上述各技术方案的光电鼠标,受本底干扰的影响大大降低,从而有效提高了鼠标导航定位的精确度的同时,增加了检测的稳定性;有利于提高产品成品率。 [0037] each of the above aspect of the optical mouse, affected by background interference is greatly reduced, thereby effectively improving the accuracy of the mouse while the navigation, increasing the stability of detection; help to improve the product yield.

附图说明 BRIEF DESCRIPTION

[0038] 图1是现有鼠标工作的光原理及内部结构示意图; [0038] FIG. 1 is a schematic view of the internal structure of an optical principle and the work of the conventional mouse;

[0039] 图2是现有鼠标工作的电原理及电路结构示意图; [0039] FIG. 2 is an electrical schematic diagram of the circuit structure and principle of the conventional mouse work;

[0040] 图3示意了现有鼠标位移矢量的计算原理; [0040] FIG 3 illustrates a conventional mouse calculation principle of the displacement vector;

[0041] 图4是现有基于图3的相关匹配定位特征区域的算法示意图; [0041] FIG. 4 is a schematic diagram of the conventional matching algorithms based on a correlation of FIG. 3 locating feature region;

[0042] 图5是以一维信号为例基于图3的光流方程的生成示意图; [0042] FIG. 5 is a schematic diagram of one-dimensional signal generated based on an example in FIG. 3 of the optical flow equation;

[0043] 图6示意了现有图像信号处理单元内的信号处理流程框图; [0043] FIG. 6 illustrates a block diagram of signal processing in the conventional image signal processing unit;

[0044] 图7示意了本底干扰对信号处理的影响,其中7a示意了本底干扰忽略不计时光传感器所拍摄的图像信号,7b示意了存在本底干扰时的所述图像信号,7c示意了本发明方法消除本底干扰后差分图像信号;[0045] 图8示意了本发明方法流程; [0044] FIG 7 illustrates the effect of background interference for signal processing, wherein 7a illustrates an image signal background interference is negligible time sensor taken, 7b illustrates the image signal in the presence of background interference, 7c illustrates the method of the present invention is to eliminate the differential image signal background interference; [0045] FIG. 8 illustrates a flow process of the present invention;

[0046] 图9为采用图8方法的本发明图像信号处理单元内部信号处理电路框图; [0046] FIG. 8 using the method of FIG. 9 is a circuit block diagram of an image signal processing according to the present invention, the internal signal processing unit;

[0047] 图10示意了图8中的图像采集输入与差分求导过程; [0047] FIG. 10 illustrates a differential input image acquisition in FIG. 8 derivation process;

[0048] 图11为图10中图像差分与求导处理的实例过程图; [0048] FIG. 11 is an example of a difference with the image process of FIG derivation process in FIG 10;

[0049] 图12为图10或图11中用来选择图像像素以求导的模板实例图; [0049] FIG. 12 or FIG. 10 to FIG. 11 in order to select an image pixel is a guide template instance of FIG;

[0050] 图13为实现图8方法的实施例之一; Example One [0050] FIG. 13 is implemented embodiment of the method of Figure 8;

[0051] 图14为实现图8方法的实施例之二。 [0051] FIG. 14 is a realization of the method of FIG. 8 embodiment the two cases.

具体实施方式 Detailed ways

[0052] 下面,结合附图所示之最佳实施例进一步阐述本发明。 [0052] Hereinafter, the present invention is further illustrated in conjunction with the accompanying drawings of the preferred embodiments illustrated.

[0053] 本发明运动图像位移检测方法,用于分析具有时间关系的图像序列以确定采集该图像序列过程中光感应器件相对于物体的运动轨迹。 [0053] The moving image displacement detecting method of the present invention, for analyzing image sequences having a temporal relationship to determine a sequence of images acquired through Cheng Zhongguang sensing device relative to the trajectory of the object. 获得所述图像序列的途径包括但不限于:以图像传感器为例,将处于相对运动中的该图像传感器所拍摄到的反映输入光强度变化信息的模拟电信号经过模数转换器,生成数字信号送入DSP (数字信号处理器)。 Access to the image sequence include but are not limited to: an image sensor as an example, in which the sensor relative motion image captured analog electrical signal reflecting the input light intensity variation information through the analog to digital converter, generates a digital signal into DSP (digital signal processor). 在该DSP中,所述数字信号先被转换成一帧帧的图像数据,即所述的图像序列。 In the DSP, the digital signal is first converted into image data of one frame, i.e., the sequence of images. 本发明方法包括步骤: The method of the present invention comprises the steps of:

[0054] A.图像信号处理单元按时间顺序接收所述图像序列;所述图像序列包括三帧或三帧以上的图像; [0054] A. The image signal processing unit receives the sequence of images chronologically; the sequence of images includes more than three or three images;

[0055] B.基于所述图像序列,依次对各当前图像进行数据处理,依据与该当前图像相对应的差分图像上的各像素进行计算。 [0055] B. Based on the sequence of images, sequentially processing each of the current image data, each pixel is calculated based on the image corresponding to the current difference image. 例如对连续相邻的三帧图像或连续相隔k(k为自然数)帧的三帧图像的数据进行帧间两两差分,即三帧图像的灰度值按顺序两两相减来得到差分后的两帧图像数据,再基于该两帧差分图像数据用光流法建立模型来计算位移矢量,这样可以消除原始图像信号中所存在的本底噪声干扰。 After continuous data, for example, three adjacent image or three consecutive images separated by k (k is a natural number) frame twenty-two inter-frame difference, i.e., three gray value image in order to obtain twenty-two subtracting the difference the two image data, based on the two and then the difference image data to calculate the displacement vector model optical flow method, which can remove background noise in the original image signal is present.

[0056] 因为光流法必须以两帧图像为基础,只有采用三帧原始图像才可以得到一个位移矢量;对于包括k' (k' >3)帧以上有效图像的图像序列,我们依次设定每一帧有效图像为当前图像,则可以最多获得k' -2个位移矢量,因此本发明方法还可以包括步骤: [0056] Since the optical flow method to be based on two images, only the use of only three of the original image a displacement vector may be obtained; for '(3 k'>) of the effective image frame of the sequence of images includes more than k, we sequentially setting each frame of the current image is a valid image, can be obtained up to k '-2 a displacement vector, thus the method of the present invention may further comprise the step of:

[0057] C.根据步骤B中的各位移矢量及其对应图像间的时间间隔来拟合出所述运动轨迹。 [0057] C. Step B vector according to the respective displacement between images and the corresponding time interval with the trajectory fitting.

[0058] 上述描述中,所述帧间两两差分指的是图像各像素均进行对应的帧间数据差分(当所述图像数据以视频流信号的方式进行传输时,指的是相邻或相隔k帧所对应的时差的视频流信号间的数据差分,其中k为自然数;为叙述简便,本处不再对视频流信号方式作区别描述):即D(x, y, t)=P (x, y, t)_P(x, y, t_k),其中P(x, y, t)代表t时刻巾贞图像数据中X,y位置处的像素的灰度值,D(x, y, t)代表t时刻帧差分图像数据中X,y位置处的像素的灰度值。 [0058] In the above description, the inter-twenty-two interframe difference data refers to a difference image corresponding to each pixel are carried out (when the image data is transmitted in streaming mode video signal, means adjacent or spaced corresponding k-th frame difference data is a difference between the video stream signal, where k is a natural number; to recite simple, present at no video stream signaling for the differences described below): i.e., D (x, y, t) = P (x, y, t) _P (x, y, t_k), where P (x, y, t) representative gray value of a pixel at image data towel Chen X, y location at time t, D (x, y , t) representative of frame difference image data X, the gradation value of the pixel at the position y at time t. 其中k值取决于成像的帧频以及成像过程中相对运动的运动速度。 Wherein the k value and the frame rate depends on the speed of movement of the imaging process in the imaging relative movement. 帧频大而运动速度慢的情况下,k可以适当取大;反之,帧频小而运动速度快的情况下,k越小越好。 Frame frequency large movement speed is low, k may be suitably take large; conversely, the frame rate is small and fast moving speed of the case where, k as small as possible.

[0059] 在上述步骤B中使用光流法的详细过程如下: [0059] Using the detailed procedure of the optical flow method in Step B above as follows:

[0060] 仍以一维信号为例。 [0060] is still a one-dimensional signal as an example. 设t时刻的图像信号如图7(a)中曲线I所示,t+At时刻图像信号如曲线2所示;如图7(b)所示,受本底噪声影响,图像传感器获取的信号沿曲线3的轨迹波动。 Curve provided an image signal at time t in FIG. 7 (a) in FIG I, t + At the time the image signal as shown in graph 2; FIG. 7 (b), the influence by the noise floor, the image sensor acquires signals 3 along a curved trajectory fluctuations. 若只考虑本底噪声干扰而不考虑其它噪声干扰,实际检测到的图像信号一般可以表示为: If only consider the background noise without considering other noise, the actually detected image signal can be generally expressed as:

[0061] P (x, t) =I (x, t) A (x) +B (χ) (7) [0061] P (x, t) = I (x, t) A (x) + B (χ) (7)

[0062] 其中,I (χ, t)代表真实信号,Α(χ)代表本底的乘性干扰系数,B(X)代表本底的加性干扰系数,设AU)和B(X)两者不随时间t变化。 [0062] wherein, I (χ, t) represent the real signal, multiplicative interference coefficient Α (χ) on behalf of the background, B (X) plus interference coefficients represent the bottom of this, provided AU) and B (X) two It does not change with time t.

[0063] 将公式(3)中的Ix换成Px且两边对时间t求导,并且设在信号匀速移动条件下(反映在二维图像信号即是鼠标匀速移动),公式(3)中的Ux不随时间变化或近似不变,可得差分光流方程: [0063] Ix is the formula (3) is replaced by Px and derivation of both sides of the time t, and let the signal at a constant speed movement conditions (i.e., reflected in the two-dimensional image signal is a mouse uniform speed), the equation (3) Ux does not change with time or approximately constant, differential optical flow equation can be obtained:

[0064] Pxtux+Ptt=0 (8) [0064] Pxtux + Ptt = 0 (8)

[0065]将(8)式作归一化处理,且 Ixt=P (χ, t)xt, Itt=P (χ, t)tt,联立式(7)得: [0065] A (8) for the normalization process, and Ixt = P (χ, t) xt, Itt = P (χ, t) tt, vertical joint (7) to give:

[0066] {[Ii+1 (X) —Ii (χ) ] ΧΑ (χ) + [Ii+1 (χ) -1i (χ) ] A (χ) J Ux+ [Ii+1 (X)-1i (χ) ] tA (χ) =0 [0066] {[Ii + 1 (X) -Ii (χ)] ΧΑ (χ) + [Ii + 1 (χ) -1i (χ)] A (χ) J Ux + [Ii + 1 (X) -1i (χ)] tA (χ) = 0

[0067] 通常,本底噪声在空间的变化率很小,[Ii+1 (χ)-1i(X)LA(X) □ [Iw(X)-1i(X)]A (X)x,故上式可化简为: [0067] In general, the rate of change of the background noise in the space is small, [Ii + 1 (χ) -1i (X) LA (X) □ [Iw (X) -1i (X)] A (X) x, therefore, the equation can be simplified as:

[0068] [Ii+1 (χ) -1i (χ) ] xux+ [Ii+1 (χ) -1i (χ) ] t=0 [0068] [Ii + 1 (χ) -1i (χ)] xux + [Ii + 1 (χ) -1i (χ)] t = 0

[0069]又令 Di (χ) =Ii+1 (χ) -1i (χ),代入上式得: [0069] lie Di (χ) = Ii + 1 (χ) -1i (χ), into the above equation to give:

[0070] Dxiix+Dt=0 (9) [0070] Dxiix + Dt = 0 (9)

[0071] 则式(9)为归一化后的以差分信号Di(X)为信号源的差分光流方程。 [0071] Expression (9) is the normalized differential optical flow equation with a differential signal Di (X) as a signal source of the. 该式表明,D就是信号I前后两帧的差信号。 This formula shows that the difference signal of the two signals before and after that D I. 如图7(c)所示,经过差分后的信号,基本上消除了本底干扰。 FIG. 7 (c), the difference signal after substantially eliminating background interference.

[0072] 对于二维差分光流方程: [0072] The two-dimensional differential optical flow equation:

[0073] Dxux+Dyuy+Dt=0 (10) [0073] Dxux + Dyuy + Dt = 0 (10)

[0074] 因为要求取两个变量ux,uy,故对多个像素建立超定差分光流方程组: [0074] Since two variables taken claim ux, uy, so establishing overdetermined differential optical flow equations for a plurality of pixels:

[0075] [0075]

Figure CN102243537BD00101

[0076] 这里假设每个像素的位移矢量(ux,uy)近似相等(皆等于鼠标移动位移矢量),整理后得差分光流方程如下: [0076] It is assumed that each pixel displacement vector (ux, uy) is approximately equal to (are equal to mouse movement displacement vector), finishing after differential optical flow equation as follows:

[0077] [0077]

Figure CN102243537BD00102

[0078] 在该方程中用Dx,Dy, Dt取代了传统光流方程中的Ix,Iy, It,即在对图像求导(求斜率)时,使用差分图像来取代原始图像。 [0078] In this equation by Dx, Dy, Dt replace the traditional optical flow equation Ix, Iy, It, i.e. when the derivative of the image (required slope), the differential image instead of using the original image. 因此,本发明步骤B中可以按公式(12)来计算并输出与当前图像对应的位移矢量(^,!^,其中匕七为差分像素的空间导数'代表着差分像素梯度值的两个分量;Dt为时间导数;n为该图像内的像素数。 Thus, step B of the present invention may be calculated according to formula (12) and outputs the current image corresponding to the displacement vector (^,! ^, Wherein dagger seven spatial derivative of the difference pixels' represent the two components of the differential pixel gradient values ; Dt of the time derivative; n is the number of pixels in the image.

[0079] 图8示意了使用本发明方法的详细流程:[0080] 先是获取图像序列:具体可以如图10所示,图像传感器以行扫描方式来采集反映了真实图像的光强度信号进而转换成一维的模拟电压信号,该模拟电压信号经过模数转换器转换成数字图像信号后送入DSP ;假设单帧图像像素为mXm个,例如取自然数m为16,则在DSP中所述数字图像信号被按(T=)每256个时钟周期分割开来,转换成一帧帧的图像数 [0079] FIG. 8 illustrates a detailed flow of the method of the present invention: [0080] First, a sequence of images acquired: specifically shown in FIG. 10, in a scanning type image sensor to collect the reflected light intensity signal is then converted into a real image a dimensional analog voltage signal, converts the analog voltage signal through the analog to digital converter into a digital image signal into the DSP; mXm assumed a single image pixel, such as the natural number m is 16, the DSP in the digital image signal is pressed (T =) every 256 clock cycles separated, converted into a number of image frames

据。 according to. 如图示的原始图像F2,F3,F4......中每帧图像包含了256个像素,每个像素具有一 The original image illustrated F2, F3, F4 ...... in each image frame includes 256 pixels, each pixel having a

灰度值。 grayscale value. 我们可以用? We can use? ^」(1=1,2......m、j=l,2......m)来表示第k帧图像在坐标(i,j) ^ "(1 = 1,2 ...... m, j = l, 2 ...... m) to represent the k-th frame image at coordinates (i, j)

位置的像素所对应的灰度值(其物理含义为该像素对应的采样信号强度,下同)。 Pixel position corresponding to the gradation value (meaning that the physical sampling signal corresponding to a pixel intensity, the same below).

[0081] 如图8所示,接着进行差分处理,根据所述图像序列来产生差分图像序列,即把该图像序列中的一帧后帧图像数据减去当前帧图像数据后作为当前帧的差分图像数据,再设 After [0081] 8, followed by differential processing sequence to produce a differential image from the image sequence, i.e. the sequence of images after the frame image data of a frame image by subtracting the current data of the current frame difference The image data is then provided

定该后帧图像数据为当前帧图像数据,......依此类推直到获得两帧或两帧以上的差分图 After the frame image data set for the current frame image data, ...... and so on until two or more two differential FIG.

像来构成所述差分图像序列。 Like the difference image sequence is constituted. 所述后帧图像或者为所述图像序列中当前图像的下一帧图像,或者为所述图像序列中当前图像的下若干0 2)帧图像。 The rear frame image or a sequence of images of the current image of the next frame, or a plurality of the current picture 02) the frame image in the sequence of images. 也就是说,设以Fk表示第k帧的图像,该帧图像对应的差分图像是Fk+1-Fk的结果,该帧差分图像的下一帧差分图像则是Fk+i+m-Fk+i的结果,其中k、1、m为自然数。 That is, set to Fk represents the k-th frame image, difference image corresponding to the frame image is the result Fk + 1-Fk, the difference image of the next frame is a frame difference image Fk + i + m-Fk + results i, wherein k, 1, m is a natural number. 当所述i=m=l,则代表先后两帧差分图像是由图像序列中依次相邻的三帧图像进行帧间两两差分获得的;当i=m# 1,则代表先后两帧差分图像是由图像序列中依次相隔等帧距的三帧图像进行帧间两两差分获得的;当i #m,则代表先后两帧差分图像是由图像序列中依次以不等帧距相邻或相隔的三帧图像进行帧间两两差分获得的。 When the three images i = m = l, representative of the difference image has two adjacent images by a sequence of inter-twenty-two sequentially obtained difference; when i = m # 1, has two differential represents the image is separated by a sequence of images sequentially from the other three image frames are inter-frame difference twenty-two obtained; when i #m, has two represents the difference image sequentially adjacent to the image ranging from a frame or sequence three spaced inter-image difference twenty-two obtained.

[0082] 图10以三帧图像采用相邻帧进行帧间两两差分为例,即原始图像F3-原始图像F2=差分图像DF5、原始图像F4-原始图像F3=差分图像DF6。 [0082] In FIG 10 using three image frames adjacent interframe difference twenty-two an example, i.e., the original image of the original image F3- DF5, F2 = the difference image, the original image of the original image F3 = F4- differential image DF6. 图11示意了图10中的一个 Figure 11 illustrates a 10 in FIG.

差分实施例:设Ph,i,」,Pu,」,PkUa=I, 2......16 ;j=l,2......16)分别表示第k_l,k, Differential Example: Let Ph, i, ', Pu, ", PkUa = I, 2 ...... 16; j = l, 2 ...... 16) represent the first k_l, k,

k+1帧图像在(i,j)位置的像素的灰度值,则差分图像第k帧数据在(1、j)位置处的像素的灰度值为Duj=Puj-Pk^j ;差分图像第k+Ι帧数据在1、j位置处的像素的灰度值为Dk+1, k + 1 frame image gradation value of a pixel (i, j) position of the k-th frame difference image data at the gradation of the pixel (1, j) is the position Duj = Puj-Pk ^ j; Difference k + Ι first image data in the frame 1, pixels at positions j gradation value Dk + 1,

依此类推可求得该实施例中其它各像素的灰度值Dk,i+1,j,Dk, η,」,Dk+1,i+1,j,Dk+i,1-1' j,Dk i+1 ' j,Dk+1,i+1 ' j, Dk j,Dk+1' j,Dk+1,i' j, Dk+1'i; j。 And so on can be obtained in other embodiments of the gradation value of each pixel of the embodiment Dk, i + 1, j, Dk, η, ", Dk + 1, i + 1, j, Dk + i, 1-1 'j , Dk i + 1 'j, Dk + 1, i + 1' j, Dk j, Dk + 1 'j, Dk + 1, i' j, Dk + 1'i; j. 这样用三帧图像产生了两帧差分图像Dk,u,Dk+1,u。 Such image generation by three two-frame difference image Dk, u, Dk + 1, u. 实际应用中,还可以在多于三帧的图像序列中设定图像调用条件,将满足预定条件的依次相邻或相隔的图像选定为欲进行差分运算的三帧图像,并依此类推直到该图像序列被访问遍。 Image of the actual application, the image can also be set in more than three call condition image sequence, satisfying the predetermined condition are sequentially selected adjacent or spaced three image difference operation is to be performed, and so on until the sequence of images is accessed times. 因此,所述图像序列中可能会有一些图像因未能满足该预定条件而被排除在所述有效图像之外。 Thus, the sequence of images may be some images because of the failure to satisfy the predetermined condition is excluded from the active image.

[0083] 接着图8中对差分图像序列进行求导处理,利用当前差分图像及其一后帧差分图像来计算当前差分图像中每一像素的所述空间导数Dx,Dy和时间导数Dt。 [0083] 8. Next the difference image sequence derivation process, calculates the current difference image of the spatial derivative of each pixel Dx, Dy, and derivative time Dt after using the current difference image and a difference image of FIG. 图10以差分图像DF5,DF6为例。 FIG 10 is a differential image DF5, DF6 Example. 在此基础上,通过两两相乘可以运算得到公式(12)中的各中间系数 Each intermediate coefficients based on this, the operation can be obtained pairwise formula (12)

次,0,化巧,仏^0^以进行图8中的全局运算,即在当前差分图像范围内计算所述公式 Times, 0, of clever, Fo ^ 0 ^ for global operator in FIG. 8, i.e., the difference image calculated in the current range of the formula

中的各中间系数的累加和 The coefficients of each intermediate accumulation and

Figure CN102243537BD00111

所述两两相乘可以用乘法器来实现,例如Z^2 = Z^xZ)i;其余不再在此赘述。 The pairwise multiplier can be implemented, for example, Z ^ 2 = Z ^ xZ) i; to rest not repeated here.

[0084] 以上,可以仅对各帧图像的中心区域进行处理,相应地,所述η为该中心区域内的像素数,即设一帧图像的像素为Μ*Μ个,其中M为自然数,则η小于Μ*Μ。 [0084] or more, only the central region of each frame image is processed, accordingly, the number of pixels in the η for the central region, i.e., a pixel image is provided for a Μ * Μ, where M is a natural number, η is less than Μ * Μ. 如上述实施例中M=16,此处计算累加和时η可以但不限于取为10 X 10=100。 As the above-described embodiment, M = 16, where the cumulative sum is calculated when η can be taken, but not limited to 10 X 10 = 100. 一般地,处理区域最大为图像区域的可求导数的最大相关区域,例如对同样Μ=16的实施例,该最大相关区域包括的像素数η 为14X14 个。 Generally, treatment zone up to the maximum correlation region can be evaluated derivative image region, for example of the same embodiment Μ = 16, the number of pixels in the region including the maximum correlation η is a 14X14.

[0085] 最后如图8基于公式(12)计算及输出位移矢量(ux,uy)。 [0085] The last 8 based on equation (12) and an output of the displacement vector is calculated (ux, uy) in FIG.

[0086] 上述描述中关于求导的具体实施过程可以采用现有技术,只是求导的对象换作为差分图像数据。 [0086] The foregoing description of specific embodiments derivation procedure may be employed on the prior art, but as objects to wrap Derivative difference image data. 下面以一实施例来说明:因为所要获取的Dx,Dy, Dt分别为D^j在X,y方向上的偏导数,归一化到像素上的计算即为相邻像素灰度值的差;时间t的导数归一化到像素上的计算即同一个像素在相邻两帧的灰度值的差,具体计算如下: In the following an embodiment will be described: Dx be acquired because, Dy, Dt D ^ j are partial derivatives in the X, y directions, normalized to calculate the difference between the pixel values ​​is the neighboring pixel grayscale ; derivative of time t normalized to the same calculation on the pixel that is a pixel adjacent to a difference in gray scale value of two, specifically calculated as follows:

[0087] Dx- [Dk, i, j+1+Dk+1,j'j+1]_ [Dk,j-1+Dk+1; i; j—J [0087] Dx- [Dk, i, j + 1 + Dk + 1, j'j + 1] _ [Dk, j-1 + Dk + 1; i; j-J

[0088] Dy= [Dk,i+1, j+Dk+1, i+1, j] - [Dk,h,j+Dk+1; h,」] [0088] Dy = [Dk, i + 1, j + Dk + 1, i + 1, j] - [Dk, h, j + Dk + 1; h, "]

[0089] Dt=4X [Dk^ijj-Dkjijj] [0089] Dt = 4X [Dk ^ ijj-Dkjijj]

[0090] 其中系数可以用图12所示的模板来表示:其中I表示系数为I,-1表示系数为-1,两个方块代表相邻两帧。 [0090] wherein the coefficient templates may be represented as shown in FIG 12: where I represents the I coefficients, -1 -1 represents a coefficient, two adjacent squares represent two. 依据该模板随Dk, i, j移动来选择两帧图像上相应的各像素的数据以求得该Dk^对应的空间导数。 The template with Dk, i, j moves the selection data of each pixel corresponding to the image based on the two in order to achieve the Dk ^ corresponding spatial derivative.

[0091] 计算Dx,Dy, Dt除了采用上述先差分处理后求导处理的步骤(如图13)以外,还可以采用第二种方式,包括步骤: [0091] than the calculated Dx, Dy, Dt except that the above-described first step of differentiation after treatment derivation process (FIG. 13), the second embodiment may also be employed, comprising the steps of:

[0092] ①根据所述图像序列利用当前图像及其一后帧图像来计算获得当前图像中每一像素的空间导数Ix,Iy和时间导数It; [0092] ① is calculated according to the sequence of images using the current image and an image obtained in the current image of each pixel spatial derivative of Ix, Iy, and It the time derivative;

[0093] ②把一后帧图像中各像素的空间导数Ix,Iy和时间导数It减去当前帧图像中相应像素的空间导数和时间导数,来产生与当前图像相对应的差分图像的每一像素的所述空间导数Dx,Dy和时间导数Dt。 [0093] ② to a rear frame image each pixel of the spatial derivative Ix, Iy and the time derivative It subtracts the corresponding pixel spatial derivative number and the time derivative of the current image to produce a differential image and the current image corresponding to each the spatial derivative pixels Dx, Dy, and derivative time Dt.

[0094] 即先求导处理后差分处理,具体以如图14所示的取三帧图像为例:先在原始图像上做求导计算得到两对导数值Ixl,Iyl, Itl和Ix2,Iy2, It2,再对这两对导数值做差分得到Dx, [0094] i.e. the first differential derivative processing process, particularly taking three images shown in FIG. 14 as an example: the first derivative calculation made on the original image to obtain two pairs of guide values ​​Ixl, Iyl, Itl and Ix2, Iy2 , It2, and then the two are differentiated to obtain the derivative value Dx,

Dy,DtO Dy, DtO

[0095] 现有光流技术不得不在图像中寻找及确定梯度较大的特征区域,主要考虑到检测视频区域中存在多个运动物体,且特征区域的大小将受限于所估计的运动目标的大小。 [0095] The prior art optical flow have to find and determine a gradient larger feature region, mainly in consideration of the presence of a plurality of image motion video object detection region, and the size of the feature region is limited to the estimated motion of the target size. 本发明方法则基于鼠标运动时,因图像传感器拍摄到的图像上每个像素的位移矢量基本一致,即运动目标只有一个,且图像信噪比不高,若采用特征区域较小则易由于噪声而被误判为最佳计算区域,进而给出完全错误的结果。 When the process of the invention are based on the mouse movement on the image captured by the image sensor of the displacement vector of each pixel substantially uniform, i.e. there is only one moving object, and the image signal to noise ratio is not high, the use of a feature region is small due to the noise is easy It misjudged as being the best calculation region, and further gives completely wrong result. 故本发明方法采用全局运算,放弃进行特征区域的寻找过程而将运算区域扩张为全局。 Therefore, the present method employs global operator, to forgo the feature region and the process of finding the global calculation area expansion. 一般对16X16像素的整个图像区域,本发明方法最大可选取的计算区域为14X14。 Usually 16X16 pixels for the entire image, the method of the present invention can be selected maximum calculated area 14X14. 因可保证对图像上大多数点进行光流计算,用小于14X14的区域(例如但不限于图像中心区域IOX 10像素)来计算参数A、B、C、E和F也是可行的。 Because they can ensure the majority of image points on optical flow computation, the region with less than 14X14 (e.g., but not limited to a central region of the image pixel IOX 10) to calculate the parameters A, B, C, E, and F are possible. 经matlab仿真实验验证,本发明方法既有利于提高算法稳定性,也有利于提高鼠标定位的准确度,尤其精简掉了寻找特征区域的步骤将使得实现本方法的电路更简单。 By matlab simulation experiment, the method of the present invention helps improve the stability of the algorithm, also help to improve the accuracy of the positioning of the mouse, in particular a streamlined out to find the step feature region so that the circuit implementation of the method simpler.

[0096] 根据本发明方法,本发明还提出一种实施本发明方法的运动图像位移检测装置,用于光电鼠标;包括一个用来接收数字图像信号的数据接口以及一个数据输出口;尤其是,如图9所示,该装置包括串连连接的差分处理模块及求导处理模块,来自所述数据接口的数字图像信号经差分处理模块以帧为单位进行帧间数据的差分运算,及求导处理模块的求导运算后,输出与当前帧图像相对应的差分图像的每一像素的空间导数Dx,Dy和时间导数Dt送往全局运算模块,分别计算该图像预定范围内所有像素的各个中间系数次,2)丨,Z)八,的累加和;该全局运算模块输出这些累加和往位移矢量计算单元,以按公式 [0096] The method of the present invention, the present invention also provides a moving image displacement detecting apparatus of the present invention, one embodiment of a method for optical mouse; comprises a data interface for receiving data and output a digital image signal; in particular, 9, the apparatus comprising a processing module connected in series and a differential derivation processing module, a digital image signal by the differential processing module from the data interface in frame units interframe difference calculation data, and derivation after derivation arithmetic processing module, each pixel of the spatial derivative Dx outputs a difference image of the current frame image corresponding to, Dy, and the time derivative Dt sent to the global calculation module, calculates the respective center of all pixels within the image a predetermined range coefficient times 2) Shu, Z) eight, and accumulation; global operator module outputs the accumulated sum to a displacement vector calculation means according to the formula

Figure CN102243537BD00131

来计算位移矢量 Calculating a displacement vector

ΣΌΛ (ux, uy),其中η为所述图像预定范围内的像素的数量;该位移矢量计算单元输出位移矢量(ux, uy)往所述数据输出口。 ΣΌΛ (ux, uy), wherein an amount of the image within the predetermined range of pixels [eta]; unit outputs a displacement vector (ux, uy) calculating the displacement vectors to said data output port. 所述公式进一步展开,实际为 The equation is further expanded, as a practical

Figure CN102243537BD00132

[0099] 考虑到本发明装置的最佳使用领域包括光电鼠标,本发明装置可以被封装为一颗芯片,在该芯片内还可以集成入现有微控制单元,该微控制单元具有用来接收按键信号的按键接口;该微控制单元接收来自所述按键接口的数据和来自所述位移矢量计算单元的位移矢量数据,并将这些数据打包输出往所述数据输出口。 [0099] Considering the art the best use of the present invention includes a photoelectric mouse device, the device of the present invention may be packaged as a chip in the chip can also be integrated into the existing micro control unit, the micro control unit having means for receiving key interface key signal; micro control unit calculates the receiving data from the key interface and the displacement vector from the displacement vector data unit, and outputs the data package to the data output port. 所述数据输出口可用来电连接电脑主机。 The data output port used to electrically connect the host computer.

[0100] 当所述数字图像信号是以通过串口的视频数据流形式来提供时,本发明装置还必须包括至少两个相串连的帧延迟器,连接所述数据接口,将所述数字图像信号以帧为单位进行分离;令远离所述数据接口的第二帧延迟器输出当前帧图像数据,则第一帧延迟器输出的是该当前帧的一后帧图像的数据,所述数据接口同时输出的是该后帧图像的一后帧图像的数据,这三帧图像数据同时输出往差分处理模块和求导处理模块的串连支路。 [0100] When the digital image signal is provided by the form of the serial video data stream, the apparatus of the invention further comprises at least two phases must be serially connected frame delay device, connected to the data interface, the digital image signal separating in frame units; so remote from the second data interface frame delay output image data of the current frame, the first frame delay output after data of one frame image is the current frame, the data interface is simultaneously output image data of one frame after the frame image, the frame image data while the three branches to the differential output series and derivation processing module processing module.

[0101] 具体连接参考图13和图14。 [0101] DETAILED connections 13 and 14 in FIG. 图13示意了实施例之一:所述差分处理模块串连在所述数据接口和求导处理模块之间,所述第二帧延迟器和第一帧延迟器分别提供一路输出送往所述差分处理模块进行帧间差分,从而该差分处理模块同时产生两帧差分图像数据送往所述求导处理模块。 FIG 13 illustrates one of the embodiments: the difference processing module is connected in series between the data interface module and the derivation process, the second frame and the first frame delay are provided all the way to delay the output sent difference processing module interframe difference, so that the difference processing module generates two difference image data to the derivative of the processing module. 具体来说,差分处理模块将来自数据接口的图像数据减去来自第一帧延迟器的图像数据得到差分图像Dl的图像数据,将来自第一帧延迟器的图像数据减去来自第二帧延迟器的图像数据得到差分图像D2的图像数据;求导处理模块对来自差分处理模块的差分图像D1、D2分别求空间导数(Dxl, Dyl)和(Dx2, Dy2),对Dxl, Dx2求和得到 Specifically, the image data, the difference data from the processing module interface is subtracted from the image data of a first frame delay device Dl differential image to obtain image data, the image data from the first frame delay is subtracted from the second frame delay image data is the image data obtained difference image D2; derivation processing module processes the difference image D1 from the differential processing module, D2 are seeking spatial (Dxl, Dyl), and (Dx2, Dy2) turned on Dxl, Dx2 summation

Dyl,Dy2求和得到Dy,以及对差分图像D1、D2做差得到时间导数Dt。 Dyl, Dy2 summation Dy, and the difference image D1, D2 calculating the difference between the time derivative obtained Dt.

[0102] 图14示意了实施例之二:所述求导处理模块串连在所述数据接口和差分处理模块之间,所述第二帧延迟器和第一帧延迟器分别提供一路输出送往所述求导处理模块,从而该求导处理模块同时产生两帧空间导数Ix,Iy和时间导数It送往所述差分处理模块进行帧间差分以获得差分后的空间导数Dx,Dy和时间导数Dt。 [0102] FIG. 14 illustrates embodiments of two cases: the derivative of the processing module connected in series between the data interface and the difference processing module, the second frame and the first frame delay device delays transmitting a single output are provided the derivation to the processing module, whereby the derivation processing module generates two spatial derivative Ix, Iy, and It sent to the time derivative of the difference processing module interframe difference to obtain the spatial derivative of the difference Dx, Dy, and time derivative Dt. 具体来说,求导处理模块对来自数据接口的图像数据、来自第一帧延迟器的图像数据、来自第二帧延迟器的图像数据两两相减求两个时间导数Itl,It2,及对来自数据接口的图像数据、来自第一帧延迟器的图像数据、来自第二帧延迟器的图像数据分别求空间导数(1x1,Iyl),(1x2, I12), (1x3, Iy3);差分处理模块对来自求导处理模块的时间导数Itl,It2相减得到Dt,对空间导数(1x1,Iyl),(1x2,Iy2), (1x3,Iy3)两两相减得到(Dxl,Dyl),(Dx2,Dy2),再对Dxl,Dx2 求和得到Dx,对Dyl, Dy2求和得到Dy。 In particular, the derivative of the image data processing module from the data interface, the image data from the first frame delay device, the second image data from the frame delay request pairwise subtraction two-phase time derivative Itl, It2, and for the image data from the data interface, the image data from the first frame delay device, the image data from the second frame of the delays required spatial (1x1, Iyl) guide, (1x2, I12), (1x3, Iy3); differential processing module time derivative Itl from derivation processing module, It2 obtained by subtracting Dt, for spatial (1x1, Iyl) derivative, (1x2, Iy2), (1x3, Iy3) twenty-two obtained by subtracting (Dxl, Dyl), ( Dx2, Dy2), and then the Dxl, Dx2 summation Dx, for Dyl, Dy2 summation Dy.

[0103] 图13和图14中的两个帧延迟器还可以用帧缓存存储器来等同替代。 [0103] FIG 13 and FIG 14 can further delay of two frames with a frame buffer memory for equivalents.

[0104] 本发明装置的求导处理模块和差分处理模块、全局运算模块、位移矢量计算模块均可以以大量的加法器或乘法器为基础来实现各种运算处理,如何使用及连接这些加法器或乘法器,在其限定的运算处理下均为现有技术。 [0104] differential derivative processing module and a processing module, the overall operation module, a displacement vector calculation module of the present invention the device may have a large number of adders or multipliers basis to implement various arithmetic processing, and how to connect these adders or a multiplier, both the prior art in which the arithmetic processing is defined. 因此本发明装置的帧延迟器、差分处理模块、求导处理模块、全局运算模块或位移矢量计算模块均可以基于现有技术,用FPGA(FieldProgrammable Gate-Array 现场可编程门阵列)或ASIC (Application Specific IntegrateCircuit专用集成电路)或DSP (Digital Signal Processor数字信号处理器)根据其所包括的处理算法来实现,并根据需要使用片内存储器或外挂存储器。 Thus the present invention, the frame delay means, the difference processing module, the processing module derivative, or a global displacement vector computation module computing module can be based on the prior art, with FPGA (FieldProgrammable Gate-Array field programmable gate array) or an ASIC (Application Specific IntegrateCircuit ASIC) or DSP (digital signal processor digital signal processor) to implement the algorithm according to which it comprises a processing and use of plug-in memory or on-chip memory as needed. 因这些均为现有技术,亦非本发明的重点,故不在此赘述。 They are due to the prior art, the present invention nor the key, so this is not repeated here. 但凡借助这些现有技术的功能模块来构成本发明装置时,无疑将落入本发明的保护范围。 When these prior art means whenever the functional blocks constituting the apparatus of the present invention, it will undoubtedly fall within the scope of the present invention.

[0105] 现有光电鼠标,包括提供照射光的发光二极管,用来感应被物面反射的照射光并将光信号转换成模拟图像信号的图像传感器,将所述模拟图像信号转换成数字图像信号的模数转换器,以及一个接收所述数字图像信号的图像信号处理单元;还包括一用来电连接电脑主机和鼠标按键的微控制单元,该微控制单元通过电连接来控制所述图像传感器、模数转换器或图像信号处理单元,及接收来自所述图像传感器、模数转换器或图像信号处理单元的状态信号;所述图像信号处理单元的数据输出口连接所述微控制单元。 [0105] conventional optical mouse, including light emitting diodes to provide illumination light, the irradiation light for sensing and converting the optical signal reflected by the object plane into an analog image signal of the image sensor, the analog image signal into a digital image signal analog to digital converter, and a signal processing unit receiving said image digital image signal; further comprising a micro control unit for electrically connecting the host computer and the mouse button, the micro control unit controls the image sensor by electrically connecting, the image signal processing unit or analog-digital converter, and receives status signals from the image sensor, an analog processing unit or an image signal; said data output port of the image signal processing unit connected to the micro control unit. 若所述图像信号处理单元采用本发明运动图像位移检测装置,无疑此类光电鼠标也在本发明的保护范围之内。 If the moving image signal processing unit according to the present invention using the image displacement detecting means, certainly is within the scope of the present invention these optical mouse. 此类光电鼠标大大提闻了鼠标导航的精确度,提闻了广品成品率。 Such optical mouse smell greatly enhanced the accuracy of the mouse to navigate, and mention the smell broad product yield. 基于现有芯片技术和封装技术的集成度越来越高,所述图像传感器与所述模数转换器可以是一体的微成像器,所述图像信号处理单元和所述微控制单元可以被集成在一颗芯片内。 Based on integration of the prior art chip and packaging technologies increasingly high, the image sensor and the analog to digital converter may be one of the micro image forming device, the image signal processing unit and the micro control unit may be integrated within a single chip.

Claims (13)

1.一种运动图像位移检测方法,用于分析具有时间关系的图像序列以确定采集该图像序列时光感应器件相对于物体的运动轨迹,包括步骤: A.图像信号处理单元按时间顺序接收所述图像序列; 其特征在于:所述图像序列包括三帧或三帧以上的图像;还包括步骤B.基于所述图像序列,依次对各当前图像进行数据处理,依据与该当前图像相对应的差分图像上的各像素进行计算,按下列公式计算并输出与当前图像对应的位移矢量(ux,uy) 1. A moving image displacement detection method for analyzing a sequence of images having a temporal relationship to determine the image capture time sequence of sensor devices with respect to the trajectory of the object, comprising steps of: A. an image signal processing unit receives the chronologically image sequence; characterized in that: said image comprises a sequence of three or more than three images; B. further comprising the step based on the sequence of images, sequentially for each current image data processing, according to the current image corresponding to the difference each pixel on the image is calculated by the following formula and outputs the current image corresponding to the displacement vector (ux, uy)
Figure CN102243537BC00021
其中Dx,Dy*差分像素的空间导数,代表着差分像素梯度值的两个分量;Dt为差分图像的时间导数;n为该差分图像内的像素数。 Spatial derivative wherein Dx, Dy * pixel difference, the pixel difference represent the two components of the gradient values; Dt of the time derivative of the difference image; n the number of pixels for the differential image.
2.根据权利要求1所述的运动图像位移检测方法,其特征在于,还包括步骤: C.根据步骤B中的各位移矢量及其对应图像间的时间间隔来拟合出所述运动轨迹。 The moving image of the displacement detecting method as claimed in claim 1, characterized in that, further comprising the step of: C. according to the respective displacement vector and corresponding to the time interval between images of the motion trajectory fitting step B.
3.根据权利要求1所述的运动图像位移检测方法,其特征在于,所述步骤B具体为包括步骤: ①根据所述图像序列来产生差分图像序列,即把该图像序列中当前帧图像数据的一后帧图像数据减去当前帧图像数据后作为当前帧的差分图像数据,再设定该后帧图像数据为当前帧图像数据,......依此类推直到获得两帧或两帧以上的差分图像来构成所述差分图像序列; ②利用当前差分图像及其一后帧差分图像来计算当前差分图像中每一像素的所述空间导数Dx,Dy和时间导数Dt ; ③通过两两相乘来运算得到所述公式中的各中间系数Z^2,g,DxDy^ DxDt, DyDt ; ④在当前差分图像范围内计算所述公式中的各中间系数的累加和;并进而计算得到所述位移矢量(ux,uy)。 The moving image displacement detection method according to claim 1, wherein said step B comprises the steps of: generating a difference image sequence according to the sequence of images, i.e., the current frame of the image sequence image data ① after subtracting the image data of a rear frame of the current frame image data as the difference image data of the current frame, then sets the frame image data of the current frame image data, ...... and so on until two or two above frame difference image constituting the differential image sequence; ② difference with the current image frame and a difference image to calculate the current difference image of the spatial derivative of each pixel Dx, Dy, and the time derivative Dt; ③ by two multiplying two coefficients is calculated to give each of the intermediate of the formula Z ^ 2, g, dxDy ^ dxDt, DyDt; ④ the intermediate coefficients calculated for each formula and accumulated within the current range of the differential image; thus calculated the displacement vector (ux, uy).
4.根据权利要求1所述的运动图像位移检测方法,其特征在于,所述步骤B具体为包括步骤: ①根据所述图像序列利用当前图像及其一后帧图像来计算获得当前图像中每一像素的空间导数Ix,Iy和时间导数It; ②把一后帧图像中各像素的空间导数Ix,Iy和时间导数It减去当前帧图像中相应像素的空间导数和时间导数,来产生与当前图像相对应的差分图像的每一像素的所述空间导数Dx, Dy和时间导数Dt ; ③通过两两相乘来运算得到所述公式中的各中间系数DxDy^ DxDt, DyDt ; ④在当前差分图像范围内计算所述公式中的各中间系数的累加和;并进而计算得到所述位移矢量(ux,uy)。 The moving image displacement detection method according to claim 1, wherein the step B specifically comprises: ① From calculated according to the sequence of images using the current image and an image obtained in each of the current image a pixel spatial derivative Ix, Iy and the time derivative It; ② to a rear frame image each pixel of the spatial derivative Ix, Iy and the time derivative It subtracts the corresponding pixel spatial derivative number and the time derivative of the current image to generate the current spatial derivative Dx image corresponding to each pixel of the difference image, Dy, and the time derivative Dt; ③ is calculated by multiplying each intermediate coefficients obtained twenty-two dxDy ^ dxDt, DyDt the formula; ④ current the intermediate coefficients calculated for each formula and accumulated within the range of the differential image; thus the calculated displacement vector (ux, uy).
5.根据权利要求3或4所述的运动图像位移检测方法,其特征在于:所述后帧图像或者为所述图像序列中当前图像的下一帧图像,或者为所述图像序列中当前图像之后若干帧的一帧图像。 The moving image displacement detecting method according to claim 3 or claim 4, wherein: said rear frame image or a sequence of images of the current image of the next frame, or a sequence of images of the current image after an image several frames.
6.根据权利要求1所述的运动图像位移检测方法,其特征在于: 步骤B对各当前图像进行数据处理时,处理区域最大为图像区域的可求导数的最大相关区域。 The moving image displacement detection method according to claim 1, wherein: step B for each current image data processing, the processing region up to the maximum correlation region can be evaluated derivative image region.
7.—种运动图像位移检测装置,用于光电鼠标;包括一个用来接收数字图像信号的数据接口以及一个数据输出口;其特征在于: 还包括串连连接的差分处理模块及求导处理模块,来自所述数据接口的数字图像信号经差分处理模块以帧为单位进行帧间数据的差分运算及求导处理模块的求导运算后,输出与当前帧图像相对应的差分图像的每一像素的空间导数Dx,Dy和时间导数Dt送往全局运算模块,分别计算该差分图像预定范围内所有像素的各个中间系数这,#,DxDy,DxDt,DyDt的累加和;该全局运算模块输出这些累加和往位移矢量计算单元, 以按公式: 7.- kinds of moving image displacement detecting means for optical mouse; comprises a data interface for receiving data and output a digital image signal; characterized in that: the processing module further includes a differential connected in series and a derivation processing module after the digital image signal from said data difference processing interface module in a unit inter-frame difference calculation data processing module and the derivative of the derivative operation, each pixel outputs a difference image corresponding to the current frame image the spatial derivative Dx, Dy and the time derivative Dt sent to the global calculation module calculates the difference image a predetermined range of each intermediate coefficients of all pixels, #, dxDy, dxDt, DyDt cumulative sum; the global operator module outputs accumulated and to the displacement vector calculating unit, according to the formula:
Figure CN102243537BC00031
来计算位移矢量(ux,Uy), 其中η为所述差分图像预定范围内的像素的数量;该位移矢量计算单元输出位移矢量(ux,Uy)往所述数据输出口。 Calculating a displacement vector (ux, Uy), wherein η is the number of pixels within a predetermined range of the differential image; vector calculating unit outputs a displacement of the displacement vector (ux, Uy) to said data output port.
8.根据权利要求7所述的运动图像位移检测装置,其特征在于: 还包括两个相串连的帧延迟器或帧缓存存储器,连接所述数据接口,将所述数字图像信号以帧为单位进行分离;令远离所述数据接口的第二帧延迟器或第二帧缓存存储器输出当前帧图像数据,则第一帧延迟器或第一帧缓存存储器输出的是该当前帧的一帧后帧图像的数据,所述数据接口同时输出的是该后帧图像的一帧后帧图像的数据,这三帧图像数据同时输出往差分处理模块和求导处理模块的串连支路。 8. The moving image of the displacement detecting apparatus according to claim 7, characterized in that: further comprising a series of two phases the frame delay or a frame buffer memory coupled to the data interface, the digital image signal in frame separating units; the data interface so remote from the second frame delay memory or the second frame buffer outputs the current image data frame, the first frame delay or the frame buffer memory of the first output after a current frame of the frame image data, the data output from the data interface on the rear of the rear frame image of a frame image, the three-frame image data while the output branch in series to the differential derivative processing module and the processing module.
9.根据权利要求8所述的运动图像位移检测装置,其特征在于: 所述差分处理模块串连在所述数据接口和求导处理模块之间,所述第二帧延迟器/帧缓存存储器和第一帧延迟器/帧缓存存储器分别提供一路输出送往所述差分处理模块进行帧间差分,从而该差分处理模块同时产生两帧差分图像数据送往所述求导处理模块。 9. The moving image of the displacement detecting apparatus according to claim 8, wherein: said difference processing module is connected in series between the data interface module and the derivation process, the second frame delay / memory frame buffer a first frame delay and frame buffer memory are supplied / sent to one output of the differential processing module interframe difference, so that the difference processing module generates two difference image data to the derivative of the processing module.
10.根据权利要求8所述的运动图像位移检测装置,其特征在于: 所述求导处理模块串连在所述数据接口和差分处理模块之间,所述第二帧延迟器/帧缓存存储器和第一帧延迟器/帧缓存存储器分别提供一路输出送往所述求导处理模块,从而该求导处理模块同时产生两帧空间导数Ix,Iy和时间导数It送往所述差分处理模块进行帧间差分以获得差分后的空间导数Dx,Dy和时间导数Dt。 10. The moving image of the displacement detecting apparatus according to claim 8, wherein: said derivation processing module connected in series between the data interface and the difference processing module, the second frame delay / memory frame buffer a first frame delay and frame buffer memory are supplied / sent all the way to the derivation of an output processing module, so that the derivation processing module generates two spatial derivative Ix, Iy, and It sent to the time derivative of the difference processing module interframe difference to obtain the spatial derivative of the difference Dx, Dy and the time derivative Dt.
11.根据权利要求7所述的运动图像位移检测装置,其特征在于: 还包括一微控制单元,该微控制单元具有用来接收按键信号的按键接口;该微控制单元接收来自所述按键接口的数据和来自所述位移矢量计算单元的位移矢量数据,并将这些数据打包输出往所述数据输出口。 11. The moving image of the displacement detecting apparatus according to claim 7, characterized in that: further comprising a micro control unit, the micro control unit having means for receiving a key signal button interface; from the key interface receives the micro control unit data and vector data from the displacement of the displacement vector calculation means, and outputting the data package to the data output port.
12.—种光电鼠标,包括提供照射光的发光二极管,用来感应被物面反射的照射光并将光信号转换成模拟图像信号的图像传感器,将所述模拟图像信号转换成数字图像信号的模数转换器,以及一个接收所述数字图像信号的图像信号处理单元;还包括一用来电连接电脑主机和鼠标按键的微控制单元;所述图像信号处理单元的数据输出口连接所述微控制单元;其特征在于: 该图像信号处理单元为权利要求7~10中任一项所述的运动图像位移检测装置。 12.- optical mouse species, including light emitting diodes to provide illumination light, the irradiation light for sensing the optical signal reflected by the object plane into the image sensor converts an analog image signal, converting the analog image signal into a digital image signal, analog to digital converter, and an image signal processing unit for receiving the digital image signal; further comprising a micro control unit for electrically connecting a host computer and the mouse button; connecting the micro control data output of the image signal processing unit unit; characterized in that: the image signal processing unit in the moving image displacement detecting apparatus according to any one of claims 7 to 10.
13.根据权利要求12所述的光电鼠标,其特征在于: 所述图像传感器与所述模数转换器为一体的微成像器,或,所述图像信号处理单元和所述微控制单元被集成在一颗芯片内`。 13. The optical mouse as claimed in claim 12, wherein: said image sensor and said analog to digital converter integrated micro imager, or the image signal processing unit and the micro control unit is integrated `within a single chip.
CN 201010176100 2010-05-14 2010-05-14 Method and device for detecting displacement of motion image as well as optical mouse CN102243537B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010176100 CN102243537B (en) 2010-05-14 2010-05-14 Method and device for detecting displacement of motion image as well as optical mouse

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010176100 CN102243537B (en) 2010-05-14 2010-05-14 Method and device for detecting displacement of motion image as well as optical mouse

Publications (2)

Publication Number Publication Date
CN102243537A CN102243537A (en) 2011-11-16
CN102243537B true CN102243537B (en) 2014-01-15

Family

ID=44961624

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010176100 CN102243537B (en) 2010-05-14 2010-05-14 Method and device for detecting displacement of motion image as well as optical mouse

Country Status (1)

Country Link
CN (1) CN102243537B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104340710A (en) * 2013-07-26 2015-02-11 山东新北洋信息技术股份有限公司 Slice medium moving state detection method and device and medium treatment device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1604126A (en) 2003-10-02 2005-04-06 三星电机株式会社 Optical navigation sensor device and image processing method using 2-dimensional sequential image process
CN1981322A (en) 2004-05-21 2007-06-13 硅光机器公司 Cross-reference to related applications

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100545062B1 (en) * 2002-11-22 2006-01-24 삼성전기주식회사 Image Data Processing Method of Optical Mouse
KR100623080B1 (en) * 2002-12-10 2006-09-11 주식회사 애트랩 Optical mouse having improved accuracy and the method for controlling it

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1604126A (en) 2003-10-02 2005-04-06 三星电机株式会社 Optical navigation sensor device and image processing method using 2-dimensional sequential image process
CN1981322A (en) 2004-05-21 2007-06-13 硅光机器公司 Cross-reference to related applications

Also Published As

Publication number Publication date
CN102243537A (en) 2011-11-16

Similar Documents

Publication Publication Date Title
Han et al. Enhanced computer vision with microsoft kinect sensor: A review
Banz et al. Real-time stereo vision system using semi-global matching disparity estimation: Architecture and FPGA-implementation
Jin et al. Real-time feature tracking and outlier rejection with changes in illumination
US9460339B2 (en) Combined color image and depth processing
CN101657825B (en) Modeling of humanoid forms from depth maps
JP2009515493A (en) Determining camera movement
US9922249B2 (en) Super-resolving depth map by moving pattern projector
JP5861949B2 (en) Digital processing method and system for optical flow determination
US8602887B2 (en) Synthesis of information from multiple audiovisual sources
US7239718B2 (en) Apparatus and method for high-speed marker-free motion capture
JP2013544449A (en) Depth camera based on structured light and stereoscopic vision
US9826216B1 (en) Systems and methods for compact space-time stereo three-dimensional depth sensing
Jalal et al. Robust human activity recognition from depth video using spatiotemporal multi-fused features
Li et al. Using kinect for face recognition under varying poses, expressions, illumination and disguise
Woodfill et al. Tyzx deepsea high speed stereo vision system
KR101613133B1 (en) Method and apparatus for processing 3-dimensional image
US10271746B2 (en) Method and system for carrying out photoplethysmography
CN101983389B (en) Moving body detection method and moving body detection device
US8159598B2 (en) Distance estimation apparatus, distance estimation method, storage medium storing program, integrated circuit, and camera
US8773512B1 (en) Portable remote control device enabling three-dimensional user interaction with at least one appliance
Berman et al. Sensors for gesture recognition systems
US8233661B2 (en) Object tracking apparatus and object tracking method
JP2000512790A (en) Data processing system and method
WO2007043036A1 (en) Method and system for object reconstruction
CN102184531A (en) Deep map confidence filtering

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C14 Grant of patent or utility model
C56 Change in the name or address of the patentee