CN102508574A - Projection-screen-based multi-touch detection method and multi-touch system - Google Patents

Projection-screen-based multi-touch detection method and multi-touch system Download PDF

Info

Publication number
CN102508574A
CN102508574A CN2011103535122A CN201110353512A CN102508574A CN 102508574 A CN102508574 A CN 102508574A CN 2011103535122 A CN2011103535122 A CN 2011103535122A CN 201110353512 A CN201110353512 A CN 201110353512A CN 102508574 A CN102508574 A CN 102508574A
Authority
CN
China
Prior art keywords
image
projection
pixel
fingertip
plane
Prior art date
Application number
CN2011103535122A
Other languages
Chinese (zh)
Other versions
CN102508574B (en
Inventor
刘金
李国林
王志华
胡军
谢翔
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Priority to CN 201110353512 priority Critical patent/CN102508574B/en
Publication of CN102508574A publication Critical patent/CN102508574A/en
Application granted granted Critical
Publication of CN102508574B publication Critical patent/CN102508574B/en

Links

Abstract

The invention discloses a projection-screen-based multi-touch detection method, which relates to the technical field of human-computer interaction. The method comprises the following steps of: measuring a projection plane, calculating parameters of the projection plane, and performing background model training on the projection plane to form a background image; projecting a graphical interface to the projection plane, and judging whether to start monitoring touch operation or not; acquiring an image s of the touch operation; performing background segmentation on the image s acquired according to the background image, and extracting a foreground image; extracting fingertips from the foreground image, and filtering a palm, an arm and corresponding shades; judging contact points according to the spatial three-dimensional information of the extracted fingertips; and if the fingertips contact the projection plane, determining that the touch operation is effective, and returning to the position coordinates of the contact points. The invention also discloses a projection-screen-based multi-touch system. A human-computer interaction mode which is convenient, comfortable, accurate, real-time and stable is realized.

Description

基于投影屏幕的多点触控检测方法及多点触控系统技术领域[0001] 本发明涉及人机交互技术领域,特别涉及一种基于投影屏幕的多点触控检测方法及多点触控系统。 Based on multi-touch detecting method of a projection screen and multi-touch TECHNICAL FIELD [0001] The present invention relates to human-computer interaction, and in particular relates to a multi-touch detecting method based on a projection screen and a multi-touch system . 背景技术[0002] 人们渴望随时随地与数字世界进行沟通,因此伴随电子产品微型化的趋势,智能手机、平板电脑等移动便携式数字产品受到了消费者的青睐。 [0002] whenever and wherever people want to communicate with the digital world, and therefore associated with electronic product miniaturization trend, smart phones, tablet PCs and other mobile portable digital products by consumers. 手机、移动电脑的体积越来越小,性能越来越高、功能越来越多。 Mobile phones, mobile computers smaller and smaller, more and more high-performance, more and more powerful. 如今大量的传感器,例如:加速度计、陀螺仪、图像传感器等已经广泛的应用于智能手机、平板电脑中。 Today, a large number of sensors, such as: an accelerometer, a gyroscope, an image sensor has been widely used in smart phones, tablet. 随着微电子技术的高速发展,可以预计将会有越来越多的传感器嵌入到智能手机、平板电脑等便携式微型设备中。 With the rapid development of microelectronics technology, it can be expected there will be more and more sensors embedded in smart phones, tablet computers and other portable devices in miniature. [0003] 如今的电子产品多采用LED显示器、CRT显示器、液晶显示屏(IXD)等作为显示设备,采用物理键盘、触控面板(Touch Panel)等作为输入设备,从而实现人机交互。 [0003] Today's electronic products to use LED display, CRT display, a liquid crystal display (IXD) and the like as a display device using a physical keyboard, a touch panel (Touch Panel) or the like as an input device, thereby achieving human-computer interaction. 前面提到的电子产品的微型化从本质上并不希望看到人机交互界面的微型化。 The aforementioned miniaturization of electronic products essentially do not want to see the miniature man-machine interface. 人机交互界面的小型化在很多应用环境中只会影响人机交互的便捷性、舒适性——比如,人们更愿意在物理键盘上打字、更愿意在大屏幕的显示器上浏览信息。 Miniaturization man-machine interface in many applications environment will only affect human-computer interaction of convenience, comfort - for example, people are more willing to typing on a physical keyboard, more willing to browse information on the display of a large screen. 因此人机交互的便捷性、舒适性要求输入设备适合人手操作、显示界面适合信息浏览。 Therefore, human-computer interaction of convenience, comfort requirements input device for manual operation, the display interface for browsing information. 这与电子产品的微型化成为了一对矛盾。 This miniature into electronic products to a contradiction. 而采用现有的人机交互方式,电子产品的微型化必然会导致交互界面的小型化。 The use of existing human-computer interaction, miniaturization of electronic products is bound to lead to miniaturization of the interactive interface. 例如,现在的手机尺寸过小,导致显示屏幕小,不适合人观察,键盘太小,也非常不方便用户操作。 For example, now the phone size is too small, resulting in a small display screen, unfit for human observation, the keyboard is too small, but also very inconvenient for users to operate. [0004] 为了克服传统输入输出设备的缺陷,近年来出现了一些新的人机交互系统。 [0004] In order to overcome the shortcomings of traditional input and output devices, in recent years there have been some new interactive system. 专利CN 101881921A、专利CN 102063618A以及专利US 2011/0197147A1中采用投影装置显示图像,然后通过投影平面上的手势实现人机交互——系统中定义若干手势,每种手势与指令相对应,存储在指令数据库中。 Patent CN 101881921A, CN 102063618A patents and patent US 2011 / 0197147A1 image projection apparatus employing the display, and then interacting with a computer through gestures on a projection plane - defined in the system a plurality of gestures, each gesture corresponding to the instruction stored in the instruction database. 比如,利用手势可以实现比如翻页、放大、缩小等操作。 For example, gestures can be realized by using such flip, zoom and other operations. 该系统最大的缺陷是手势定义的指令数量有限。 Biggest disadvantage of this system is the limited number of instructions defined by the gesture. 一方面,指令数的增加必然会加大用户的学习难度,很难被普通用户接受;另外一方面,即使指令数量增加到一定程度也很难实现字符、 数字输入的功能。 On the one hand, the number of instructions increases will inevitably increase the user's learning curve, it is difficult to accept the ordinary users; on the other hand, even if the number of instructions to a certain extent it is difficult to realize the function character, numeric input. 最近,赛路恩公司推出了一款镭射虚拟键盘(VKB)。 Recently, ex-way race has launched a virtual laser keyboard (VKB). 该产品采用激光在桌面上投影出一个虚拟键盘,红外传感器判断手指是否敲击键盘。 The product uses a laser to project a virtual keyboard on the desktop, infrared sensors to determine whether the finger tapping the keyboard. 该产品投影出的键盘图像色彩单调,也只能完成字符、数字输入的功能,而不能实现色彩丰富图形界面投影与操作。 The product is projected keyboard image color monotonous, can only fulfill the functions of character, digital inputs, and can not achieve the projected colorful graphical interface and operation. 发明内容[0005](一)要解决的技术问题[0006] 本发明要解决的技术问题是:如何实现微型移动便携式设备的方便、舒适、快速、 准确以及稳定的人机交互。 SUMMARY [0005] (a) To solve the technical problem [0006] The present invention to solve the technical problem is: how to achieve compact mobile convenience of a portable device, comfortable, fast, accurate and stable human-computer interaction. [0007] ( 二)技术方案[0008] 为解决上述技术问题,本发明提供了一种基于投影屏幕的多点触控检测方法,包括以下步骤:[0009] Sl :测定投影平面、计算所述投影平面参数,并对所述投影平面进行背景模型训4练,形成背景图像;[0010] S2:向投影平面投影图形界面后并判断是否开始监测触控操作,若没有触控操作, 则继续监测;[0011] S3 :采集触控操作的图像s ;[0012] S4:依据所述背景图像对步骤S3中采集的图像s进行背景分割,提取出前景图像, 该前景图像中至少包括了指尖、手掌、手臂以及各自形成的阴影;[0013] S5 :从前景图像中提取出指尖,滤除手掌、手臂以及相应的阴影;[0014] S6 :根据提取出的指尖的三维信息判断接触点;[0015] S7:如果指尖与投影平面接触,那么判定为一次有效的触控操作,并返回接触点的位置坐标。 [0007] (ii) Technical Solution [0008] To solve the above problems, the present invention provides a multi-touch detection method based on a projection screen, comprising the steps of: [0009] Sl: Determination of the projection plane, computing the projection plane parameter, the projection plane and the background model training training 4, forming a background image; [0010] S2: the projection plane projected to the graphical interface and determines whether the touch operation start monitoring, if there is no touch operation is continued monitoring; [0011] S3: image acquisition s touch operation; [0012] S4: step S3, the acquired image is divided according to the background s background image, foreground image is extracted, the foreground image including at least a finger sharp, hands, arms, and shadow respectively formed; [0013] S5: extracted from the foreground image of a fingertip, the palm was filtered off, and the corresponding shadow of the arm; [0014] S6: determining three-dimensional information extracted fingertip the contact point; [0015] S7: If the fingertip into contact with the projection plane, then it is determined that a valid touch operation and returns the position coordinates of the contact point. [0016] 其中,所述步骤Sl中的平面参数计算包括如下步骤:[0017] 采用结构光方法检测出投影平面上N个点,N >=3,非N点共线,在图像传感器坐标系的坐标(Xi Yi Zi)T,i = 1,2···,Ν,当N= 3时,通过如下公式计算平面参数:[0018][0019] 否则,通过最小二乘法估计投影平面参数,其中α、β、Y为所述投影平面的方程系数,满足平面方程α χ+ β y+ Υ ζ = 1。 [0016] wherein, in the step Sl plane parameter calculation comprising the steps of: [0017] The method of using structured light detected points on the projection plane N, N> = 3, N non-collinear points in the coordinate system of the image sensor coordinates (Xi Yi Zi) T, i = 1,2 ···, Ν, when N = 3, the parameter plane is calculated by the following equation: [0018] [0019] otherwise, by the least squares method to estimate the projection plane parameters, wherein α, β, Y is the projection plane equation coefficients of the plane equation satisfies α χ + β y + Υ ζ = 1. [0020] 其中,所述步骤S4中分割背景图像的方式为:遍历图像s中每一个像素点,将偏离背景图像统计均值大于第一门限值的像素判断为前景图像;将偏离背景图像统计均值小于第二门限值的像素判定为背景图像,根据该像素值更新背景图像;偏离在第一门限值和第二门限值之间的像素判定为背景图像,但不更新背景图像,所述第一门限值大于所述第二门限值。 [0020] wherein the background image segmentation step S4 of way: traversing each pixel s in the image, the background image is determined to deviate from the pixel is greater than the first threshold value statistical mean of the foreground image; departing from the background image statistics mean pixels smaller than the second threshold value is determined as a background image based on the pixel value of the background image is updated; departing from the pixels between the first threshold and the second threshold value is determined as a background image, the background image is not updated, the first threshold value is greater than the second threshold value. [0021] 其中,所述步骤S4中,在遍历之前通过投影图像四个角的坐标计算采集图像平面上对应的四个坐标,以确定在图像s中的投影图像的区域,遍历时只遍历图像s中的投影图像所在区域。 [0021] wherein, in the step S4, before traversing the four captured image corresponding to the coordinates on the coordinate plane is calculated by the four corners of the projection image to determine areas of the projected image in an image in s, traversing the image only when traversing the area of ​​the projected image s. [0022] 其中,所述步骤S5具体包括:[0023] (1)判断像素点是否为前景图像,若不为前景图像,则结束对该像素点的处理;[0024] (2)若为前景图像,则判断是否为阴影,如果该像素点为阴影,结束对该像素点的处理,否则断定所述像素点为指尖;[0025] (3)按上述(1)42)遍历图像s的每一个像素点。 [0022] wherein the step S5 comprises: [0023] (1) determines whether the pixel is a foreground image, as if the foreground image, the pixel processing is terminated; [0024] (2) If the foreground image, it is determined whether as a shadow, if the pixel is a shadow, the pixel processing is completed, otherwise the pixel is judged fingertip; [0025] (3) according to the above (1) 42) traversing the image s each pixel. [0026] 其中,所述步骤S5具体包括:[0027] (1)判断像素点是否为前景图像,若不为前景图像,则结束对该像素点的处理;[0028] (2)若为前景图像,则判断是否为阴影,如果所述像素点为阴影,则结束对所述像素点的处理;[0029] (3)计算所述像素点的像素斜率;[0030] (4)判断同一斜率上是否有阴影点,如果与所述像素点相同斜率上存在阴影点,则断定所述像素点所在区域不与投影屏幕接触,结束对该像素点的处理,否则断定所述像素点为指尖;[0031] (5)按上述(1)-(4)遍历图像s的每一个像素点。 [0026] wherein the step S5 comprises: [0027] (1) determines whether the pixel is a foreground image, as if the foreground image, the pixel processing is terminated; [0028] (2) If the foreground image, it is determined whether as a shadow, if the pixel is a shadow, the pixel processing is terminated; [0029] (3) calculate the slope of the pixel of the pixel; [0030] (4) determining the slope of the same if the shadow point, if there is a shadow point on the slope of the same pixel, it is judged that the region where the pixel is not in contact with the projection screen, the pixels of the processing ends, otherwise the pixel is judged fingertip ; [0031] (5) above (1) - (4) through each pixel of the image s. [0032] 其中,所述步骤S6具体包括:[0033] 将结构光投影到指尖区域,并捕获失真的结构光图案;[0034] 查找指尖区域中投影结构光图像与失真结构光图案的像素点映射关系;[0035] 计算指尖到投影平面的距离,通过结构光的方法得到指尖在图像传感器坐标系的坐标为(Xi Yi Zi)T,那么指尖与投影平面之间的距离为: [0032] wherein the step S6 comprises: [0033] The structured light projected onto the fingertip area, and distortion of the structured light pattern captured; [0034] Find fingertip area image and the structured light projection distortion of the structured light pattern pixel mapping relation; [0035] calculated from the fingertip to the projection plane, obtained by the method of the structured light fingertip in the coordinate system of the image sensor coordinate (Xi Yi Zi) T, then the distance between the fingertip and the projection plane for:

Figure CN102508574AD00061
Figure CN102508574AD00062

[0037] 判断接触点,如果Cli > thresholcLl,则指尖不与投影平面接触,否则指尖与投影平面接触,其中,threshold_l为预定阈值。 [0037] Analyzing the point of contact, if Cli> thresholcLl, the fingertip is not in contact with the projection plane, or the plane of the fingertip in contact with the projection, wherein, threshold_l the predetermined threshold value. [0038] 其中,所述步骤S6具体包括:[0039] 向指尖区域投影结构光条纹,并捕获结构光图像;[0040] 检测捕获结构光图像中条纹上边缘像素横坐标X1和下边缘像素横坐标& ;[0041] 计算条纹中心^^;[0042] 判断指尖是否与投影平面接触,如果[0043] xc - "^1 二X2 < threshold—2,[0044] 那么断定指尖与投影平面接触,其中&为指尖中心横坐标,threshold_2为预定阈值。[0045] 本发明还提供了一种基于投影屏幕的多点触控系统,包括:投影仪、图像传感器、 I/O端口、输入设备、存储设备及中央控制器,所述投影仪图像传感器、I/O端口、输入设备、 存储设备均与所述中央控制器相连,所述I/O端口用于连接移动终端,所述投影仪用于将移动终端显示的图像和操作界面投影到投影平面;所述图像传感器用于捕获投影屏幕上手势;所述中央控制器用于解析所述手势的触控操作,并将 [0038] wherein the step S6 comprises: [0039] The light pattern projected to the fingertip area, and the light image capture structure; [0040] detected in the structured light image capturing pixel abscissa X1 stripe edge and a lower edge pixels & abscissa; [0041] ^^ calculated center stripe; [0042] Analyzing the fingertip is in contact with the projection plane, if the [0043] xc - "^ 1 two X2 <threshold-2, [0044] it is judged that the projected fingertip a contact plane, wherein the center of a fingertip & abscissa, threshold_2 the predetermined threshold value [0045] the present invention further provides a multi-touch system based on a projection screen, comprising: a projector, an image sensor, I / O ports, an input device, a storage device and a central controller, the projector image sensor, I / O ports, input devices, storage devices are connected to the central controller, the I / O port for connecting the mobile terminal, the a projector for projecting an image and a mobile terminal user interface displayed to the projection plane; the image sensor for capturing a gesture on the projection screen; the central controller for parsing the gesture touch operation, and 析出的触控信息反馈给移动终端,所述输入设备用于输入配置信息及指令。[0046] 其中,所述输入设备包括:感光传感器,用于检测环境光的亮度,所述中央控制器根据感光传感器的输出值调节投影仪的亮度。[0047](三)有益效果[0048] 本发明的基于投影屏幕的多点触控系统及方法中,在背景分割步骤采用了高斯模型对背景建模、实时更新背景模型以及采用差分法提取前景信息。系统能够准确、快速的提取出前景图像,并且适应环境光的变化。[0049] 在指尖检测步骤采用阴影检测以及像素斜率法,滤除不与投影平面接触的指尖区域。优化了后续深度检测步骤中的检测区域,减少了深度检测步骤耗费的时间,加快了系统触控检测的速度。[0050] 另外,在指尖深度检测步骤中,向指尖区域投射结构光,根据反馈信息判断指尖是否与投影平面接触。本发明可采 The precipitated touch information back to the mobile terminal, the input device for inputting configuration information and instructions [0046] wherein, the input apparatus comprising: a photosensitive sensor for detecting the brightness of ambient light, according to the central controller photosensitive sensor output value to adjust the brightness of the projector. [0047] (c) advantageous effects [0048] the present invention system and method for multi-touch screen, a projection-based segmentation step uses a background modeling background Gaussian model , updated in real time background model difference method and the use of extracting the foreground information system can accurately, quickly extracted foreground images, and adapt to changes in ambient light. [0049] the shadow detection pixels slope method and the fingertip detection step, filtering out with the planar contact area of ​​the projection of the fingertip. optimization of the detection region depth subsequent detection step, reducing the time-consuming step of detecting the depth, speeds up the system touch detection. [0050] Further, the depth of the fingertip detection step , the fingertip region projecting structured light, according to the feedback information to determine whether a fingertip into contact with the projection plane. the present invention can be taken 用快速算法,不需要计算指尖空间坐标的具体值,从而降低了深度检测步骤的计算量。附图说明6[0051] 图Ia是本发明实施例的一种基于投影屏幕的多点触控系统结构;[0052] 图Ib是图Ia中系统的一个应用示例图;[0053] 图2是本发明实施例的一种基于投影屏幕的多点触控检测方法的流程图;[0054] 图3是多点触控检测方法的系统初始化流程;[0055] 图4是多点触控检测方法的一种简化的指尖检测流程;[0056] 图5是多点触控检测方法的一种高精确度的指尖检测流程;[0057] 图6是像素斜率的示意图;[0058] 图7是多点触控检测方法的一种指尖深度检测流程;[0059] 图8是多点触控检测方法的一种快速指尖深度检测流程。 A fast algorithm, the specific values ​​necessary to calculate the spatial coordinates of the fingertip, thereby reducing the amount of calculation of the depth of the detecting step. BRIEF DESCRIPTION 6 [0051] FIG. Ia is one embodiment of the present invention is based on multi-point touch-screen projection system configuration; [0052] FIG. Ib is an example of application of the system of FIG. Ia; [0053] FIG 2 is an embodiment of the present invention, the multi-touch detecting method of the flow chart based on a projection screen; [0054] FIG. 3 is a multi-touch detection system initialization process method; [0055] FIG. 4 is a simplified multi-touch detection method of the fingertip detection process; and [0056] FIG. 5 is a multi-touch detection method process fingertip detection of high accuracy; [0057] FIG. 6 is a schematic diagram of a pixel slope; [0058] FIG. 7 is one kind of the multi-depth detection process fingertip touch detection method; [0059] FIG. 8 is a multi-touch control method for detecting one kind of fast fingertip depth detection process. 具体实施方式[0060] 下面结合附图和实施例,对本发明的具体实施方式作进一步详细描述。 DETAILED DESCRIPTION [0060] The following embodiments and the accompanying drawings, specific embodiments of the present invention will be further described in detail. 以下实施例用于说明本发明,但不用来限制本发明的范围。 The following examples serve to illustrate the present invention but are not intended to limit the scope of the present invention. [0061] 本发明实施例公开了一种基于投影屏幕的多点触控系统结构101,如图Ia所示。 [0061] Example embodiments of the present invention discloses a multi-touch system configuration 101 based on the projection screen, as shown in FIG Ia. 该系统结构包括:[0062] 投影仪104,作为系统的输出显示设备。 The system structure comprising: [0062] the projector 104, the output of the system as a display device. 其将图像投影到墙面、桌面等任何易于用户操作和观察的平面,该投影平面称为投影屏幕。 Any plane which is easy to operate and the user to observe the image projection wall, table, etc., the projection plane is called a projection screen. 投影仪104可以采用DLP、LCD、LC0S以及激光投影等技术;[0063] 图像传感器106,用于图像采集,捕获在投影屏幕上的点击、拖动等触控操作。 The projector 104 may be employed DLP, LCD, LC0S and laser projection technology; [0063] image sensor 106, for image acquisition, capture clicking on the projection screen, a drag touch operation and the like. 目前,图像传感器106已经广泛的应用到移动便携式设备中——基本上所有的手机、笔记本电脑上都嵌入了摄像头。 Currently, the image sensor 106 has been widely applied to mobile portable equipment - substantially all of the mobile phones, laptop computers are embedded camera. 主流的图像传感器106主要分为C⑶和CMOS两种技术。 Mainstream image sensor 106 is divided into two techniques C⑶ and CMOS. 在相同分辨率下,CMOS价格比CXD便宜,但是CMOS器件产生的图像质量相比CXD来说要低一些。 In the same resolution, CMOS cheaper price than CXD, but CMOS devices produced image quality is lower compared to CXD it. 另外,CMOS图像传感器的优点之一是电源消耗量比C⑶低;CMOS图像传感器的另一优点,是与周边电路的整合性高,可将ADC与讯号处理器整合在一起,使体积大幅缩小。 Further, one of the advantages of the CMOS image sensor is lower than the power consumption C⑶; Another advantage of the CMOS image sensor, a high integration with peripheral circuits, may be integrated with the ADC signal processor, so that the volume significantly reduced. 优选地,图像传感器106采用CMOS技术的摄像头;[0064] 1/0端口113,外接移动便携式设备。 Preferably, the image sensor 106 CMOS camera technology employed; [0064] 1/0 port 113, external mobile portable device. 1/0端口113可以是USB,串口,VGA等常用接口,也可以是自定义接口。 1/0 port 113 may be a USB, serial, VGA and other common interfaces, it may be a custom interface. 移动便携式设备可以是智能手机、平板电脑、笔记本电脑等。 The portable device may be a smart mobile phones, tablet PCs, notebook computers. 该1/0端口实现了移动便携式设备与中央控制器120的数据与指令通信,通过该端口可以很方便的将投影屏幕的多点触控系统嵌入到其他电子设备中。 The 1/0 port implements data communication with a central command and control movement of the portable device 120, can easily be embedded in multi-touch screen, a projection system to other electronic devices through the port. [0065] 输入设备115,实现用户对系统的配置:输入相关指令,配置工作模式等,是一种辅助的输入方式。 [0065] The input device 115, user configuration of the system to achieve: an input related instructions, configure the working mode, it is an auxiliary input. [0066] 优选地,输入设备可以是自定义键盘、鼠标、触控面板,实现用户对系统的配置:输入相关指令,配置工作模式等;[0067] 优选地,输入设备也可以是感光传感器:采用感光传感器检测环境光的亮度,系统根据感光传感器的输出值调节投影仪的亮度。 [0066] Preferably, the input device may be a custom keyboard, a mouse, a touch panel, the configuration of the system to achieve the user: Enter the relevant instruction, the working mode and the like; [0067] Preferably, the input device may be a photosensing: using the photosensitive sensor for detecting the brightness of ambient light, the brightness of the projector system is adjusted according to the output of the photosensitive sensor. [0068] 存储设备111,用于存储系统参数、缓存图像。 [0068] storage device 111 for storing system parameters, image cache. 在中央控制器120进行图像数据的过程中,需要将图像数据、系统参数进行存储。 In the process of the image data of the central controller 120, the image data, system parameters need to be stored. 存储设备可以是一般的动态存储器或者静态存储设备;[0069] 中央控制器120,分别与投影仪104、图像传感器106、1/0端口113、输入设备115及存储设备111连接,是整个触控系统的核心模块。 General storage device may be dynamic or static memory storage device; [0069] The central controller 120, the projector 104, respectively, the image sensor 106 and 1/0 port 113, an input device 115 and storage device 111 are connected, the entire touch the core module of the system. 模块主要实现以下功能:[0070] 1)从I/O端口113接收移动便携式设备的显示图像,将显示图像发送到投影仪104 ;[0071] 2)接收图像传感器106图像(该图像中包括了裸手在操作界面上的手势信息), 对图像进行解析处理,分析触控信息;[0072] 3)将分析结果通过I/O端口113返回移动便携式设备,然后移动便携设备执行相应操作。 Modules to achieve the following functions: [0070] 1) receives the display image movement of the portable device from the I / O port 113, the display image is sent to the projector 104; [0071] 2) receives the image sensor 106 images included (the image bare hand gesture information on the user interface), the image analysis processing, analyzing touch information; [0072] 3) the analysis results through I / O port 113 to return movement of the portable device, and then move the mobile device to perform a corresponding operation. [0073] 中央控制器120是整个系统的核心部分,其负责系统的数据运算处理以及控制。 [0073] The central controller 120 is the core of the system, the data processing system which is responsible for computing and control. 为了满足系统的性能指标,中央控制器120应保证数据处理的实时性。 In order to meet the system performance, the central controller 120 should guarantee real-time data processing. [0074] 优选地,中央控制器120可以设计成ASIC芯片,以提高整个系统的响应速度。 [0074] Preferably, the central controller 120 may be designed as an ASIC chip, in order to improve the response speed of the entire system. 当然,也可以采用高性能的微处理。 Of course, a high-performance microprocessor can be used. [0075] 投影屏幕的多点触控系统可以应用到智能手机、平板电脑、笔记本电脑、数码相机等移动便携式电子产品中;该触控系统也可以应用于一般的电子设备中,取代传统的LCD 显示屏、鼠标和键盘;该触控系统甚至可以应用于一些恶劣的环境中,比如在海上的设备, 需要防水、防盐雾、防潮,如果没有传统的操控面板和显示面板,将使得系统的三防特性更好,同时也使得设备的设计更加简单和统一。 [0075] projection screen multi-touch system can be applied to smart phones, tablet PCs, notebook computers, digital cameras and other portable mobile electronic products; the touch system can also be applied to general electronic devices to replace the traditional LCD display, mouse and keyboard; the touch system can even be used in some harsh environments, such as in equipment at sea, you need water, salt spray, moisture, if not the traditional control panel and display panel, making the system three anti properties better, but also make these devices more simple and uniform. [0076] 优选地,本实施例将触控系统嵌入到智能手机102中以示说明,如图Ib所示。 [0076] Preferably, the present embodiment will be embedded into smart phones touch system 102 is shown to illustrate, as shown in FIG Ib. [0077] 投影仪104在图像传感器106的正上方位置处。 At [0077] In the projector 104 of the image sensor 106 directly above the location. 这种安装方式,主要为了便于图像传感器检测由投影仪104和指尖产生的阴影。 This installation, in order to facilitate the main sensor detecting a shadow image produced by the projector 104 and the fingertip. 当然,投影仪104和图像传感器106的安装位置也不局限与上述例子。 Of course, the projector 104 and the mounting position of the image sensor 106 is not limited to the above examples. [0078] 投影仪104可以将投影图像或者操作界面投影到任意平面上。 [0078] The projector 104 may be a projection image projected onto the interface, or any plane. 人手112可以在投影图像110上进行相应的触控操作。 112 can make the appropriate hand touch operation on the projection image 110. 比如:点击按钮、拖动滑块等,从这个角度其基本上代替了鼠标的功能。 For example: Click button, drag the slider, etc., from this point of which substantially replaces the mouse. 另外,还可以将虚拟键盘投影出来,可以根据不同用户的特点改变键盘的大小,使得投影的键盘大小适合于人手的大小以及用户的习惯。 Further, the virtual keyboard can be projected, the size of the keyboard can be changed according to the characteristics of different users, such that the size of the projected keyboard is adapted to the size of a human hand and the user's habits. [0079] 示例100中将图像或者界面投影到桌面108上。 [0079] Example 100 or in the image projected onto the desktop interface 108. 这种投影方式的最大好处在于便于用户的操作和观察。 The maximum advantage of this projection is to facilitate the user's operation and observation. 用户将手机102竖立在面前,投影图像显示在手机102和用户之间, 调节投影图像的大小以及亮度到最适合使用的程度。 The user erected in front of the mobile phone 102, the projection image 102 displayed between the phone and the user to adjust the projected image size and brightness to the most suitable extent of their use. 用户可以悠闲的坐在桌前,上网、办公寸。 Users can leisurely sitting at the table, Internet, Office inch. [0080] 投影平面也可以是墙面。 [0080] The projection plane may be a wall. 这种投影方式的最大特点是便于信息共享。 The maximum feature of this projection is to facilitate information sharing. 当用户需要商务报告、与同事讨论问题时,可以将图像投影到墙面上,这样方式相对于几个人围在一台笔记本面前更便于交流。 When the user needs of business reports, discuss issues with colleagues, images can be projected onto the wall, in such a way with respect to a few people around in front of a laptop easier communication. 当然,休息的时候,也可以多个人聚在周围一起欣赏影片。 Of course, the rest of the time, more than one person can gather around to enjoy the movie together. [0081] 本发明实施例中还公开了一种基于投影屏幕的多点触控检测方法200,本实施例中采用上述系统来实现该方法,如图2所示,包括:[0082] 步骤202,测定投影平面、计算所述投影平面参数,并对所述投影平面进行背景模型训练,形成背景图像。 Embodiment [0081] The present invention also discloses a multi-touch detection method based on a projection screen 200, the present embodiment to implement the system using the above method embodiment, shown in Figure 2, comprising: [0082] Step 202 measuring projection plane, the projection plane is calculated parameters, and said projection plane background model training, forming a background image. [0083] 步骤204,向屏幕投影显示图像或者操作界面,并判断是否开始触控检测,如果未开始检测,继续步骤204。 [0083] Step 204, the display image projected onto the screen or user interface, and determines whether to start detecting the touch, if the detection is not started, step 204 continues. [0084] 步骤206,图像传感器采集图像,该图像中包含了裸手的点击、拖动等触控信息。 [0084] Step 206, the image sensor capture an image, the image information contains bare hands touch the click, drag and the like. [0085] 步骤208,背景分割,对步骤206中采集的图像进行背景分割,提取出前景图像,该前景图像中可能包括了指尖、手掌、手臂以及阴影等。 [0085] Step 208, background segmentation on the image acquired in step 206 the background segmentation, extracts the foreground image, the foreground image may include fingers, hands, arms and shadows. [0086] 步骤210,指尖检测,从前景图像中提取出指尖(一般将指尖作为触控操作的有效部位),需要将手掌、手臂以及阴影等无用信息滤除。 [0086] Step 210, the fingertip detection, extracted from the foreground image of a fingertip (fingertip as an active site typically touch operation) needs to be hands, arms, and other garbage shading filtered. [0087] 步骤212,根据指尖的三维信息判断接触点,采用结构光的方法获取指尖的三维信息。 [0087] Step 212, the contact point is determined according to three-dimensional information of the fingertip, using structured light method of obtaining three-dimensional information of the fingertip. 向指尖区域投射结构光条纹,图像传感器106捕获失真的结构光条纹,然后计算指尖的三维信息;结合指尖的三维信息与投影平面的方程,判断指尖是否与投影平面接触。 The fingertip area projected light pattern, the image sensor 106 captures light pattern distortion, and then calculating three-dimensional information of the fingertip; binding equation of three-dimensional information of the projection plane fingertip, the fingertip is in contact with determining the projection plane. [0088] 步骤214,返回触控信息,如果指尖与投影平面接触,那么判定为一次有效的触控操作。 [0088] Step 214, return the touch information, if the fingertip into contact with the projection plane, then it is determined that a valid touch operation. 该系统支持多个指尖同时点击投影屏幕,对应的多个触控点皆有效。 The system supports multiple finger while clicking on the projection screen, corresponding to a plurality of touch points are valid. 完成一次触控检测,返回步骤204。 Detecting a touch complete, returns to step 204. [0089] 步骤202如图3所示,本实施例中,相当于是对上述系统进行初始化,主要过程为检测系统应用环境的相关参数,具体包括:步骤302,投影平面测定、平面参数计算; 步骤304,投影亮度调节; 步骤306,背景模型训练。 [0089] Step 202 shown in Figure 3, the present embodiment is equivalent to the above-described system is initialized, the main process parameters of the detection system application environment, specifically includes: Step 302, the projection plane is determined, parameter calculation plane; a step 304, projection brightness adjustment; step 306, the background model training. 投影平面测定的原理如下,图像传感器106以及投影仪104的成像公式都可以描 Measurement principle as the projection plane, the image sensor 106 and the imaging formula of the projector 104 can be described

Figure CN102508574AD00091
Figure CN102508574AD00092

[0095] 其中等式左边是图像传感器106拍摄图像或者投影仪104图像中的坐标,单位是像素;右边Mint为内参数矩阵,Mrait为外参数矩阵,e是归一化变量,(^^,〜…是世界坐标系的齐次坐标。内参数矩阵的测量已经比较成熟,优选地,本实施例中采用张正友标定法测定内参数矩阵。图像传感器106和投影仪104的内参数在出厂时就已经确定下来,在实施例中作为已知量使用;另外,图像传感器106和投影仪104之间的相对位置及姿态也作为已知量使用。[0096] 可以看到,图像传感器系统的坐标是真实世界坐标的线性映射,但是这并不是一一映射,多个真实世界坐标可能对应同样一个图像传感器系统坐标。但是,如果已知真实世界坐标满足一个特定的条件,即在某个平面上,那么,在上述方程成立的同时,又有世界坐标系的平面方程:[0097] 1 = β γ) yw (2)[0098] 上述平 [0095] The left hand side where the image sensor 106 is captured image or the coordinates of an image projector 104, in pixels; Mint within the right parameter matrix, Mrait outer parameter matrix, e is the normalized variables (^^, ~ ... are homogeneous coordinates of the world coordinate system. measuring the parameter matrix is ​​relatively mature, preferably, the present embodiment uses the measurement Zhang Zhengyou calibration method parameter matrix image sensor 106 and the internal parameters of the projector 104 in the factory has been determined, using a known amount in an embodiment; Further, the relative position and orientation between the image sensor 106 and the projector 104 using a known amount of [0096] can be seen, the image sensor coordinate system Yes. linear mapping real world coordinates, but this is not a one to one mapping, the plurality of real world coordinates may correspond to the same one image sensor coordinate system. However, if it is known real world coordinates satisfy a specific condition, i.e. in a plane, then, the above equations hold at the same time, another plane equation of the world coordinate system: [0097] 1 = β γ) yw (2) [0098] said planar 参数(α β Y)可以采用结构光技术进行测定。 Parameters (α β Y) may employ structured light measurement technique. 结构光技术为一种在3D 图像重建领域广泛使用的方法:其向物体表面投射编码结构光,由于物体表面深度不一,因此结构光会发生形变;捕获该形变结构光,根据形变信息计算物体表面的深度。 Structured light method is a widely used technique in the field of 3D image reconstruction: which is projected onto the object surface light coding structure, since the surface of varying depth, structured light may thus deformed; light trapping structure the strain, the strain is calculated according to the object information surface depth. 本发明中, 采用结构光技术可以计算投影平面上点的世界坐标,从而计算参数(α β y)o[0099] 从方程(1) (2)就可以得到世界坐标与图像传感器系统坐标的一一映射关系,同时由于投影系统和图像传感器系统的相似性,也可以得到世界坐标与投影仪系统坐标的一一映射关系,从而可知图像传感器(摄像机)系统坐标和投影仪系统坐标在投影平面上具有一一映射关系,而且可以用下面的式子线性表示[0100] The present invention, the use of structured light techniques may be calculated world coordinates of points on the projection plane, thereby calculating the parameter (α β y) o [0099] From equation (1) (2) can be obtained in the world coordinates and the image sensor system coordinates a a mapping relationship, and because of the similarity of the projection system and the image sensor system, can be obtained in the world coordinate system and the projector coordinate-one mapping, and thus seen that the image sensor (camera) coordinate system and the projector coordinate system on the projection plane having one mapping, and can be represented by the following linear expression [0100]

Figure CN102508574AD00101

[0101] 其中等式左右分别为图像传感器和投影仪图像的像素坐标,H为3X3变换矩阵, 称为单应性矩阵。 [0101] wherein the pixel coordinates of the left and right respectively equation image sensor and the projector image, H is a 3X3 transformation matrix, called a homography matrix. 而且上式只对于某一平面上的点成立,也就是说,对于不在投影平面上的点,上式一定不成立。 And the only formula for a point on a plane set up, that is, not to a point on the projection plane, the formula must not established. 通过这一性质,可以判断某一个空间点是否在平面上。 By this property, based on a certain point in space is on the plane. [0102] 所述投影平面的测定方法已经比较成熟,优选地,本实施例中采用步骤如下:[0103] (1)投影仪104向投影平面投影的黑白棋盘图(在图像传感器标定技术中最常用的方格棋盘图);[0104] (2)图像传感器106采集投影平面上的棋盘图;[0105] (3)采用角点检测技术检测棋盘角点,角点检测技术已经比较成熟,比如可以采用opencv中的函数直接检测角点;[0106] (4)利用图像传感器图像和投影图像之间的角点对应关系,计算单应性矩阵H,该步骤也可以采用opencv中的函数完成。 [0102] The method of measuring the projection plane is relatively mature, preferred, embodiment, the present embodiment uses the following steps: [0103] (1) The projector 104 projected to the projection plane of FIG Reversing (calibration image sensor technology is the most FIG conventional checkerboard); [0104] (2) the image sensor 106 of FIG capture board on the projection plane; [0105] (3) the corner detection chessboard corner detection, corner detection technology is relatively mature, such as opencv corner directly detecting function may be employed; [0106] (4) the corner between the image sensor and the image projected images corresponding relationship, computing the homography matrix H, the step function may be employed in opencv completed. [0107] 上述步骤302中所述平面参数的计算步骤如下:[0108] (1)采用结构光技术获取投影平面上N(非N点共线,N>= 3)个点在图像传感器坐标系的坐标(Xi Ii Zi)T,其中i = 1,2-,N;[0109] (2)通过上述N个点的坐标值,计算投影平面方程参数。 Said step of calculating the parameters of the plane [0107] The step 302 as follows: [0108] (1) using structured light projection plane techniques obtaining N (a non-collinear points N, N> = 3) dots in the image sensor coordinate system coordinates (Xi Ii Zi) T, where i = 1,2-, N; [0109] (2) through the N point coordinate values, calculating projection plane equation parameters. 当N = 3时,[0110] When N = 3, [0110]

Figure CN102508574AD00102

[0111] 如果,N > 3,那么采用最小二乘法求解方程参数,其中α、β、Y为所述投影平面的方程系数,满足平面方程α χ+ β y+ Υ χ = 1。 [0111] If, N> 3, then solved using least squares parameter equation, wherein α, β, Y is the projection plane equation coefficients of the plane equation satisfies α χ + β y + Υ χ = 1. [0112] 上述投影亮度调节步骤304具体如下:[0113] 多点触控系统会应用于不同的环境,高亮度的环境光会导致投影图像或者操作界面不清晰,因而需要调高投影仪104的亮度,但是此时投影模块处于高耗能模式;低亮度的环境光中,投影仪104不需要投影出特别高亮度的图像,因此,可以调低投影仪104的亮度, 使得投影模块处于低耗能模式。 [0112] The projection luminance adjusting step 304 as follows: [0113] Multi-touch system in different environments, high-luminance ambient light will cause the projected image or user interface is not clear, and therefore the need to increase the projector 104 brightness, but this time the projector module in a high energy consumption mode; low-luminance ambient light, the projector 104 does not need a particularly high brightness projection image, and therefore, can reduce the brightness of the projector 104, the projector module is in low power such that energy mode. 可以手动调节投影仪亮度,使其亮度合适。 You can manually adjust the brightness of the projector, so that appropriate brightness. 优选地,可以在系统中增加亮度传感器——测量环境光的强度,系统根据亮度传感器的返回值自动的调节投影仪104的亮度。 Preferably, the luminance sensor may be increased in the system - measuring brightness intensity of ambient light, the system automatically adjust the projector 104 based on the return value of the luminance sensor. [0114] 上述背景模型训练步骤306具体如下:[0115] 背景建模为一种比较成熟的技术,本发明中优选的采用背景高斯模型。 [0114] The background model training step 306 as follows: [0115] background modeling is a relatively mature technology, the present invention preferably employs a Gaussian background model. 高斯背景模型是假定背景上每一个像素点服从高斯分布。 Gaussian background model assumes that each pixel on the background Gaussian distribution. 在背景模型训练阶段,系统从投影屏幕捕获若干幅图像,统计这些图像每个像素点的均值μ ij与标准差σ,由均值μ ,J组成的图像称为背景图像BackGrouncKBackGround为图像矩阵,矩阵元素图像像素值)。 In the context of the model training phase, the system captures a plurality of images from the projection screen, the mean and standard deviation of μ ij these statistics [sigma] of each pixel in an image, the image by the mean μ, J the composition is referred to as the background image BackGrouncKBackGround image matrix, the matrix elements image pixel value). [0116] 在完成了背景训练之后,可以通过提取出来的背景像素的均值来进行前景的提取和背景的分割。 [0116] After completion of the training background, can be segmented foreground and background extracted by the mean extracted from the background pixels. 不过这里有两点需要注意:可以实时根据环境光照的微弱变化调整背景参数,从而使得算法对于环境光照的缓慢变化不再敏感。 But here there are two points to note: The parameters can be adjusted in real time background small changes in ambient light, so that the algorithm is not sensitive to slowly varying ambient light. 同时,算法能够对于人手的快速移动提供良好的前景分离结果。 At the same time, the algorithm can provide good prospects for the separated result of fast-moving hand. 背景分割步骤208具体如下:[0117] 采用集合FrontArea表示前景区域。 Background segmentation step 208 as follows: [0117] The expressed set FrontArea foreground area. FrontArea为一个widthXheight的二进制CN 102508574 A说明书8/10 页 FrontArea widthXheight a binary description CN 102508574 A page 8/10

Figure CN102508574AD00111
Figure CN102508574AD00112

矩阵,width为图像的宽,height为图像的高。 Matrix, width is the width of the image, height of the image is high. 如果像素点(i,j)为前景,则FrontAreaG, j) = 1 ;否则FrontArea(i, j) = O。 If the pixel (i, j) as a foreground, the FrontAreaG, j) = 1; otherwise, FrontArea (i, j) = O. (i,j)为图像传感器图像平面的像素坐标。 (I, j) is the pixel coordinates in the image plane of the image sensor. [0118] 优选的采用以下前景判断法则进行前景图像的提取和背景模型的更新,依次遍历像素点(i,j)(设图像传感器106捕获图像为s,初始化时,令FrontArea(ZJ) = O, V/J ):[0119] 若偏离背景统计均值Isij-PijI大于门限值Ω1的像素,判定为前景图像,令FrontArea(i, j) = 1,不根据它更新背景图像,Sij为图像s在(i,j)处的值,优选地,捕获图像s为灰度图像,Sij为图像s在(i,j)处的灰度值;[0120] 偏离背景统计均值Isij-PijI于门限值Ω2(Ω1> Ω 2)的像素,判定为背景图像, 根据它进行背景模型的更新BackGroimcKi,j) = Sij0[0121] 在Ω 1和Ω2之间的像素,则认为是背景图像,但是并不根据它更新模型。 [0118] Preferred following foreground decision rule to update extraction and background model of the foreground image, sequentially traversing the pixel (i, j) (provided the image sensor 106 captures an image of s, initialization, so FrontArea (ZJ) = O , V / J): [0119] If the background statistical deviation from the mean Isij-PijI Ω1 pixels greater than the threshold limit value, it is determined that the foreground image, so FrontArea (i, j) = 1, which is not according to the updated background image, an image Sij of the value of s (i, j) at, preferably, s is the image captured gray image, the image Sij of the s (i, j) at the gradation values; [0120] departing from the background statistical mean of the door Isij-PijI limit Ω2 (Ω1> Ω 2) pixels, it is determined that the background image is updated background model based on its BackGroimcKi, j) = Sij0 [0121] in the pixel between Ω 1 and [Omega] 2, is considered to be the background image, but it is not based on updated models. [0122] 优选地,门限值Ω 1取5 σ,门限值Ω 2取3 σ。 [0122] Preferably, the threshold value Ω 1 taken 5 σ, the threshold value Ω 2 taken 3 σ. [0123] 按照上述3个条件依次判断并更新每一个像素点,遍历方式如下表所示:[0124]For(i= 1 ;i<width;i++) For (j=l J <height;j ++)判断像素点㈨)为前景或者背景; EndFor EndFor[0125] 最终得到前景图像FrontArea以及更新之后的背景模型图像BackGround。 [0123] sequence is determined according to the above three criteria and update each pixel, traversal in the following table: [0124] For (i = 1; i <width; i ++) For (j = l J <height; j + +) (ix) is determined pixels) as foreground or background; EndFor EndFor [0125] finally obtained FrontArea foreground image and the background image backGround model after updating. [0126] 特别地,图像传感器的视角一般大于投影仪的视角,因此图像传感器拍摄的图像除了包括投影图像,还一些区域没有投影。 [0126] In particular, the viewing angle of the image sensor is generally greater than the angle of view of the projector, the image captured by the image sensor comprises a projection image in addition, also some regions are not projected. 由于只需关注投影屏幕附近的图像,因此可以根据之前得到的投影屏幕参数信息,只提取屏幕附近的像素点进行处理,以此来加快处理速度和增强稳定性。 Because only concerned about the image near the projection screen, it can be based on a projection screen parameter information previously obtained extract only the vicinity of screen pixels to process, in order to speed up the processing speed and enhanced stability. 该思路的详细过程可以描述如下:[0127] 将投影仪图像的4个角: fl、 r ι 、 'height、 ,height、[0128] αλ = 1 1 、α2 = width 1 、a3 = 1 1 ^a4 = width 1[0129] 带入方程(3),可以计算得到图像传感器图像平面的4个点:[0130] Iy1 = H α[,]ζ = Ha^X3 = Ha^X4 = Ha^4[0131] 设区域Area为由上述图像传感器图像平面上的点^决定的四边形区域。 Detailed process of this idea can be described as follows: [0127] The four corners of the projector image: fl, r ι, 'height,, height, [0128] αλ = 1 1, α2 = width 1, a3 = 1 1 ^ a4 = width 1 [0129] into the equation (3), four points can be calculated image sensor image plane: [0130] Iy1 = H α [,] ζ = Ha ^ X3 = Ha ^ X4 = Ha ^ 4 [ 0131] area area provided by the quadrangular region point on the image plane of the image sensor ^ determined. 投影仪图像上所有的像素点映射到图像传感器平面上,都位于区域Area内。 The image projector all image pixels are mapped to the sensor plane, are located within the area Area. [0132] 因此,所述背景分割就不需要遍历图像传感器采集到的整个图像区域,只需要遍历上述Area,即图像传感器采集到的图像中投影图像区域。 [0132] Thus, segmentation of the background image does not need to traverse the entire area of ​​the image sensor is acquired, only the above-described traverse Area, i.e., the image sensor to capture an image of the projection image area. [0133] 图4和图5示出了两种指尖检测步骤210的流程。 [0133] Figures 4 and 5 illustrate two kinds fingertip detection process of step 210. [0134] 其中,图4描述了一种简化的指尖检测流程210a。 [0134] wherein, Figure 4 depicts a simplified fingertip detection process 210a. 对于在步骤206中捕获的图像s的每一个像素点按照图4的流程进行处理。 For each pixel of the captured image in step 206 s is processed according to the flow of FIG. 如果像素点(i,j)属于指尖区域,那么将其加入集合FingerArea,一开始FingerArea =空集。 If the pixel (i, j) belongs to the fingertip area, then it is added to the collection FingerArea, beginning FingerArea = empty set. 对流程如下:[0135] 步骤402,获取像素点(i,j)的值SiJ。 Of the process is as follows: [0135] Step 402, acquiring pixel (i, j) values ​​SiJ. [0136] 步骤404,判断该像素点是否为前景图像,如果FrontAreaG,j) = 1那么说明像素点(i,j)为前景图像;否则不为前景图像,结束对该像素点的处理。 [0136] Step 404, it is determined whether the pixel is a foreground image, if FrontAreaG, j) = 1 then that pixel (i, j) as a foreground image; otherwise, not the foreground image, the pixel processing is ended. 11[0137] 步骤406,判断是否为阴影,阴影检测技术为已知技术,由于阴影在RGB以及YVU等色彩空间上具有一定的特点,可以根据该特点判断是否为阴影。 11 [0137] Step 406 determines whether the shadow, the shadow detection technique is a known technique, because of the shadow has certain characteristics in the RGB color space and YVU the like, can be determined whether the shading according to the characteristics. 如果该像素点为阴影,结束对该像素点的处理。 If on the pixel shading, pixel processing of the end. [0138] 步骤408,将像素点(i,j)加入集合FingerArea。 [0138] Step 408, the pixel (i, j) to the collection FingerArea. [0139] 简化的指尖检测方法检测出来的结果包含了很多冗余信息:有的指尖在投影屏幕上有阴影,因而可以判断指尖并未与投影屏幕接触,理论上,不需要对这种类型的指尖进行深度测量。 [0139] The simplified method of detecting the fingertip detection result contains a lot of out redundant information: Some fingertips shadow on the projection screen, it can be determined not in contact with the finger projection screen, theoretically, this does not require type of fingertip depth measurement. 但是,这种方法把这种类型的指尖不加区分的加入了集合FingerArea,这加大了后续深度测量的运算量。 However, this approach to this type of fingertip added indiscriminate collection FingerArea, which increases the amount of computation of a subsequent depth measurement. [0140] 针对上述简化的指尖检测方法的缺陷,图5描述了一种精确度更高的指尖检测方法210b。 [0140] For the above-described simplified fingertip defect detection method, Figure 5 depicts a method of detecting the fingertip higher accuracy 210b. 该方法中采用像素斜率检测法将包含阴影的指尖区分出来,减少了集合FingerArea的元素,从而减小了指尖深度测量的计算量。 The method employs a pixel slope detection method comprising the fingertip distinguish the shadow, the elements of the set FingerArea reduced, thereby reducing the amount of computation fingertip depth measurement. 像素斜率定义为:[0141] k = ^^ (5)yo-y:[0142] 如图6所示,其中(χ。,y0)是指极点坐标,(xi; Yi)是指像素点坐标,图像的左上角作为坐标原点。 The slope is defined as the pixel: [0141] k = ^^ (5) yo-y: [0142] shown in Figure 6, wherein (. Χ, y0) coordinates of the pole means, (xi; Yi) refers to the pixel's coordinates the upper left corner of the image as the origin of coordinates. 其中,极点坐标可以通过外参数直接计算得到。 Wherein the pole coordinate can be calculated directly by external parameters. 根据外极线几何的理论知道:如果指尖(设指尖的像素斜率为k)有阴影,该阴影点必然在同一像素斜率k对应的射线上。 According to the theory of known epipolar geometry: if the fingertip (fingertip is set as the pixel slope k) with a shadow, the shadow points necessarily on the same pixel corresponds to the slope k rays. 因此,在判断指尖是否有阴影时,基本思想为:计算出指尖的像素斜率,然后搜索该斜率上是否存在阴影点。 Thus, when determining whether there is a fingertip shadow, the basic idea is: calculated slope fingertip pixel, and then search for the presence or absence of a shadow point on the slope. 如果有阴影点可以断定指尖与投影屏幕不接触;如果不存在阴影点,也不能直接断定指尖与投影屏幕接触——在实际使用过程中,由于阴影亮度过高、手指与平面距离过近等原因,经常会导致部分阴影检测不出来。 If the shadow points can be concluded that there is a fingertip is not in contact with the projection screen; if the shadow point is not present, can not directly determine the fingertip into contact with the projection screen - in actual use, because of the shadow high brightness, too close to the plane of the finger and other reasons, often resulting in partial shadow can not be detected. 完全确定无阴影的指尖是否与投影平面接触,还需要后续的指尖深度检测的结果进行判断。 Determining completely unshaded fingertip is in contact with the projection plane, but also the depth of the fingertip detection result of the subsequent judgment. 精确度更高的指尖检测方法210b的流程如下:[0143] 步骤502,获取像素点(i,j)的值SiJ。 The method of the fingertip detection accuracy higher flow 210b is as follows: [0143] Step 502, acquiring pixel (i, j) values ​​SiJ. [0144] 步骤504,通过步骤208中提取的前景图像,判断该像素点是否为前景图像。 [0144] Step 504, the foreground image extracted in step 208, it is determined whether the pixel is a foreground image. 如果FrontArea(i, j) = 1那么说明像素点(i,j)为前景图像;否则不为前景图像,结束对该像素点的处理。 If FrontArea (i, j) = 1 then that pixel (i, j) as a foreground image; otherwise, not the foreground image, the pixel processing is ended. [0145] 步骤506,通过阴影检测的结果判断该像素点是否为阴影。 [0145] Step 506, the shadow detection result by the judgment whether the pixel is a shadow. 如果该像素点为阴影, 结束对该像素点的处理。 If on the pixel shading, pixel processing of the end. [0146] 步骤508,计算像素点(i,j)的像素斜率k。 [0146] Step 508 calculates the pixel (i, j) pixel slope k. [0147] 步骤510,判断同一斜率上是否有阴影点,如果与像素点相同斜率上存在阴影点, 那么可以断定像素点(i,j)所在区域不与投影屏幕接触,结束对该像素点的处理。 [0147] Step 510 determines whether there is shadow points on the same slope, if the shadow point is present on the same slope pixels, it can be concluded that the pixel (i, j) Area of ​​contact with the projection screen, the end point of the pixel deal with. [0148] 步骤512,将像素点(i,j)加入集合FingerArea。 [0148] Step 512, the pixel (i, j) to the collection FingerArea. [0149] 图7描述了一种指尖深度测量的方法21加。 [0149] FIG 7 describes a method for measuring the depth of the fingertip 21 plus. 结构光测量深度为已知技术,其基本步骤为:[0150] 步骤702,把结构光投影到指尖区域。 Structured light measurement depth of known techniques, the basic steps of: [0150] Step 702, the structured light projected onto the fingertip area. [0151] 步骤704,通过图像传感器106捕获失真的结构光图案。 [0151] Step 704, the image sensor 106 captures the structured light pattern distortion. [0152] 步骤706,查找指尖区域中投影结构光图像与失真结构光图案的像素点映射关系。 [0152] Step 706, to find the pixel mapping relation with the distortion of the projected light image structure structured-light pattern of the fingertip area. 优选的,可以采用灰度编码、颜色编码或者正弦波编码方式获取投影图像和图像传感器图像之间的映射关系。 Preferably, the gray scale code can be used to acquire the mapping relation between the projection image and a color image sensor an image coding or sinusoidal coding. [0153] 步骤708,计算指尖到投影平面的距离,假设通过结构光的方法得到指尖在图像传感器坐标系的坐标为(Xi Yi Zi)1, Zi表示指尖深度,即指尖到图像传感器106的距离,那么指尖与投影平面之间的距离为:[0154] [0153] Step 708, the projection plane is calculated from the fingertip to the assumed structure obtained by the method of the fingertip of the light sensor in the coordinates of the image coordinate system is (Xi Yi Zi) 1, Zi Biaoshi fingertip depth, i.e., the image fingertip the distance sensor 106, the distance between the fingertip and the projection plane as: [0154]

Figure CN102508574AD00131

[0155] 步骤710,判断接触点,如果Cli > thresholcLl,则指尖不与投影平面接触,否则指尖与投影平面接触,其中,thresholcLl是设定的一个阈值,优选的,threshold_l取1cm。 [0155] Step 710, the point of contact is determined, if Cli> thresholcLl, the fingertip is not in contact with the projection plane, or the plane of the fingertip in contact with the projection, wherein, thresholcLl a threshold value is set, preferably, threshold_l taken 1cm. [0156] 图8描述了另一种快速指尖深度检测方法212b。 [0156] Figure 8 depicts another method for rapid detection of the depth of the fingertip 212b. 如果指尖与投影屏幕接触,那么指尖对应的投影仪坐标系坐标与图像传感器坐标系坐标应该满足公式(3)。 If the projection screen in contact with the fingertip, the fingertip corresponding to the coordinate system of the projector coordinate system of the image sensor should satisfy equation (3). 方法212b正是利用这一性质判断指尖是否与投影屏幕接触。 212b is the use of this method of determining the nature of a fingertip is in contact with the projection screen. 设图像s中指尖区域的中心为(χ。,y。)T, 假设指尖与投影屏幕接触,那么根据公式(3),可以计算出指尖在投影仪坐标系中的坐标ΟνΚ,ΐΓζΗίΟ^ι,ΐΓ,Η Up,yp)作为结构光条纹的中心向指尖区域投影条纹。 S center of this image region as a fingertip (χ., Y.) T, assuming the fingertip in contact with the projection screen, then according to equation (3), can calculate the coordinates of the fingertip ΟνΚ in the projector coordinate system, ΐΓζΗίΟ ^ ι, ΐΓ, Η Up, yp) as a center stripe light pattern projected to the fingertip area. 设图像传感器106捕获到结构光条纹的上下边缘分别是X1和&,因此捕获到的结构光的中心为。 The image sensor 106 captures provided light pattern to upper and lower edges and respectively X1 & therefore the captured light to the central structure. 如果中心^^与&相同,那么可以断定指尖与投影屏幕接触,否则指尖不与投影屏幕接触。 If the center of the & ^ ^ same, it can be concluded that the fingertip contact with the projection screen, otherwise the fingertips of contact with the projection screen. 该方法的步骤如下:[0157] 步骤802,指尖区域结构光条纹投影。 The method comprises the following steps: [0157] Step 802, the fingertip region projected light pattern. [0158] 步骤804,捕获结构光图像。 [0158] Step 804, light image capture structure. [0159] 步骤806,检测捕获结构光图像中条纹上边缘像素横坐标^c1和下边缘像素横坐标X2 ο[0160] 步骤808,计算条纹中心^^。 [0159] Step 806, the image capturing light detection structure in the edge stripe pixel abscissa ^ c1 and the lower edge pixel abscissa X2 ο [0160] Step 808 calculates ^^ strip center. [0161] 步骤810,判断指尖是否与投影平面接触,如果[0162] [0161] Step 810, it is determined whether a fingertip in contact with the projection plane, if the [0162]

Figure CN102508574AD00132

[0163] 指尖与投影平面接触,否则不接触。 [0163] contacting the fingertip and the projection plane, or not in contact. 其中,threshold_2是设定的另一个阈值。 Wherein, threshold_2 is another set threshold. 优选的,threshold_2 取2。 Preferably, threshold_2 take 2. [0164] 采用上述快速指尖深度检测方法212b,可以降低判断接触点步骤212的算法复杂度,加快触控检测的速度,算法复杂度低便于集成到硬件上,速度快保证了整个系统触控检测的实时性。 [0164] The above-described method for the rapid detection of the depth of the fingertip 212b, the contact point can be determined to reduce the complexity of the algorithm in step 212, the detected touch speed, low complexity easily integrated into the hardware, to ensure that the speed of the entire system of the touch real-time detection. [0165] 以上实施方式仅用于说明本发明,而并非对本发明的限制,有关技术领域的普通技术人员,在不脱离本发明的精神和范围的情况下,还可以做出各种变化和变型,因此所有等同的技术方案也属于本发明的范畴,本发明的专利保护范围应由权利要求限定。 [0165] the above embodiments are merely illustrative of the present invention, and are not restrictive of the invention, relating to ordinary skill in the art, without departing from the spirit and scope of the present invention, can make various changes and modifications , all equivalent technical solutions also within the scope of the present invention, the scope of the present invention patent is defined by the appended claims.

Claims (10)

1. 一种基于投影屏幕的多点触控检测方法,其特征在于,包括以下步骤:51 :测定投影平面、计算所述投影平面参数,并对所述投影平面进行背景模型训练,形成背景图像;52 :向投影平面投影图形界面后并判断是否开始监测触控操作,若没有触控操作,则继续监测;S3:采集触控操作的图像s;S4:依据所述背景图像对步骤S3中采集的图像s进行背景分割,提取出前景图像,该前景图像中至少包括了指尖、手掌、手臂以及各自形成的阴影;55 :从前景图像中提取出指尖,滤除手掌、手臂以及相应的阴影;56 :根据提取出的指尖的三维信息判断接触点;57 :如果指尖与投影平面接触,那么判定为一次有效的触控操作,并返回接触点的位置坐标。 A multi-touch detection method based on a projection screen, characterized by comprising the steps of: 51: Determination of the projection plane, the projection plane is calculated parameters, and said projection plane background model training, the background image is formed ; 52: projecting a pattern to the projection plane interface and determines whether the touch operation start monitoring, if there is no touch operation is continued monitoring; S3: image acquisition s touch operation; S4: according to the background image in step S3 s acquired image background segmentation, extracts the foreground image, the foreground image including at least a fingertip, palm, arm, and shadow formed by each; 55: extracted from the foreground image of a fingertip, the palm was filtered off, and the corresponding arm shadow; 56: three-dimensional information is determined according to the extracted contact point fingertip; 57: If the fingertip into contact with the projection plane, then it is determined that a valid touch operation and returns the position coordinates of the contact point.
2.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤Sl 中的平面参数计算包括如下步骤:采用结构光方法检测出投影平面上N个点,N >= 3,非N点共线,在图像传感器坐标系的坐标(Xi Yi Zi)T,i = 1,2···,Ν,当N= 3时,通过如下公式计算平面参数: 2. The multi-touch detection method based on the projection screen as claimed in claim 1, wherein, in said step Sl plane parameter calculation comprising the steps of: using an optical method the structure of the N detected points on the projection plane, N> = 3, N non-collinear points in the image sensor coordinate system coordinates (Xi Yi Zi) T, i = 1,2 ···, Ν, when N 3 = when calculated by the following equation plane parameters:
Figure CN102508574AC00021
^3 否则,通过最小二乘法估计投影平面参数,其中α、β、γ为所述投影平面的方程系数,满足平面方程α χ+ β y+ Υ ζ = 1。 Otherwise ^ 3, by least squares parameter estimation projection plane, wherein the α, β, γ to the projection plane equation coefficients of the plane equation satisfies α χ + β y + Υ ζ = 1.
3.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S4 中分割背景图像的方式为:遍历图像s中每一个像素点,将偏离背景图像统计均值大于第一门限值的像素判断为前景图像;将偏离背景图像统计均值小于第二门限值的像素判定为背景图像,根据该像素值更新背景图像;偏离在第一门限值和第二门限值之间的像素判定为背景图像,但不更新背景图像,所述第一门限值大于所述第二门限值。 Traversing each pixel s in the image, the background image statistics deviate: The multi-touch detection method based on a projection screen, characterized in that said 1, the background image segmentation step S4 way claims pixel determination mean is greater than a first threshold value as foreground image; departing from the pixels of the background image is less than the statistical mean value of the second threshold value is determined as a background image based on the updated background image pixel value; first threshold value and offset from the first value between two pixels is determined as the background image, the background image is not updated, the first threshold value is greater than the second threshold value.
4.如权利要求3所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S4 中,在遍历之前通过投影图像四个角的坐标计算采集图像平面上对应的四个坐标,以确定在图像s中的投影图像的区域,遍历时只遍历图像s中的投影图像所在区域。 4. The multi-touch detection method based on a projection screen, characterized in that said 3, the step S4, the calculated before traversing the captured image corresponding to the coordinates of the projection image plane through the four corners of claim coordinates, to determine the region of the projection image in an image in s, traverses only region where the projection image of the image when traversing s.
5.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S5 具体包括:(1)判断像素点是否为前景图像,若不为前景图像,则结束对该像素点的处理;(2)若为前景图像,则判断是否为阴影,如果该像素点为阴影,结束对该像素点的处理, 否则断定所述像素点为指尖;(3)按上述(1)42)遍历图像s的每一个像素点。 (1) whether the pixel is determined as a foreground image, as if the foreground image is terminated: 5. The multi-touch detection method based on a projection screen, characterized in that said 1, comprises the step S5 claim processing the pixel point; (2) if it is the foreground image, it is determined whether as a shadow, if the pixel is a shadow, the pixel processing is completed, otherwise the pixel is judged a fingertip; (3) press traverse (1) 42) above each pixel of the image s.
6.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S5 具体包括:(1)判断像素点是否为前景图像,若不为前景图像,则结束对该像素点的处理;(2)若为前景图像,则判断是否为阴影,如果所述像素点为阴影,则结束对所述像素点的处理;(3)计算所述像素点的像素斜率;(4)判断同一斜率上是否有阴影点,如果与所述像素点相同斜率上存在阴影点,则断定所述像素点所在区域不与投影屏幕接触,结束对该像素点的处理,否则断定所述像素点为指尖;(5)按上述(1)44)遍历图像s的每一个像素点。 (1) whether the pixel is determined as a foreground image, as if the foreground image is terminated: The multi-touch detection method based on a projection screen, characterized in that said 1, comprises the step S5 claim processing the pixel point; (2) if it is the foreground image, it is determined whether as a shadow, if the pixel is a shadow, the pixel processing is ended; (3) calculating a slope of the pixels of the pixel ; (4) determines whether a shadow point on the same slope, if there is a shadow point on the slope of the same pixel, it is judged that the region where the pixel is not in contact with the projection screen, the pixel processing is completed, otherwise determine the pixel fingertip; (5) above (1), 44) traversing each pixel of the image s.
7.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S6 具体包括:将结构光投影到指尖区域,并捕获失真的结构光图案; 查找指尖区域中投影结构光图像与失真结构光图案的像素点映射关系; 计算指尖到投影平面的距离,通过结构光的方法得到指尖在图像传感器坐标系的坐标为(Xi Yi Zi)T,那么指尖与投影平面之间的距离为: ,axi +Pyj+Yzi«, = —ι判断接触点,如果Cli > thresholcLl,则指尖不与投影平面接触,否则指尖与投影平面接触,其中,threshold_l为预定阈值。 7. The multi-touch detection method based on a projection screen, characterized in that said 1, comprises the step S6 claim: structured light projected onto the fingertip area, and captures the structured light pattern distortion; refers to find the projected image pixel structures and the optical distortion of the structured light pattern is a mapping relationship in the tip region; projection plane calculated distance to the fingertip, the fingertip obtained by the method of structured light in the coordinate system of the image sensor coordinate (Xi Yi Zi) T, then the distance between the fingertip and the projection plane as:, axi + Pyj + Yzi «, = -ι point of contact is determined, if Cli> thresholcLl, the fingertip is not in contact with the projection plane, or the plane of the fingertip in contact with the projection, wherein , threshold_l to a predetermined threshold value.
8.如权利要求1所述的基于投影屏幕的多点触控检测方法,其特征在于,所述步骤S6 具体包括:向指尖区域投影结构光条纹,并捕获结构光图像;检测捕获结构光图像中条纹上边缘像素横坐标X1和下边缘像素横坐标& ; 计算条纹中心ip; 判断指尖是否与投影平面接触,如果xc < threshold—2,那么断定指尖与投影平面接触,其中^为指尖中心横坐标,threshold_2为预定阈值。 Detecting light trapping structure; light pattern projected to the fingertip area, and the light image capture structure: 8. The multi-touch detection method based on a projection screen, characterized in that said 1, comprising the step S6 specific requirements image edge stripe pixel abscissa X1 & abscissa and the lower edge pixels; IP computing center stripe; Analyzing the fingertip is in contact with the projection plane, if xc <threshold-2, it is judged that the fingertip in contact with the projection plane, wherein ^ is The abscissa fingertip center, threshold_2 is a predetermined threshold value.
9. 一种基于投影屏幕的多点触控系统,包括:投影仪、图像传感器、I/O端口、输入设备、存储设备及中央控制器,所述投影仪图像传感器、I/O端口、输入设备、存储设备均与所述中央控制器相连,其特征在于,所述I/O端口用于连接移动终端,所述投影仪用于将移动终端显示的图像和操作界面投影到投影平面;所述图像传感器用于捕获投影屏幕上手势; 所述中央控制器用于解析所述手势的触控操作,并将解析出的触控信息反馈给移动终端, 所述输入设备用于输入配置信息及指令。 A projection system based on multi-touch screen, comprising: a projector, an image sensor, I / O ports, input devices, storage devices and a central controller, the projector image sensor, I / O ports, input devices, storage devices are connected to the central controller, wherein said I / O port for connecting the mobile terminal, a projector for projecting an image and a mobile terminal user interface displayed to the projection plane; the said image sensor for capturing a gesture on the projection screen; the central controller for parsing the gesture touch operation and the touch feedback parsed information to the mobile terminal, the input device for inputting configuration information and instructions .
10.如权利要求9所述的基于投影屏幕的多点触控系统,其特征在于,所述输入设备包括:感光传感器,用于检测环境光的亮度,所述中央控制器根据感光传感器的输出值调节投影仪的亮度。 A photosensitive sensor for detecting the brightness of ambient light, the output of the photosensitive sensor in accordance with the central controller: multi-touch system based on a projection screen, characterized in that said 9, said input apparatus comprising as claimed in claim 10 adjusting the brightness value of the projector.
CN 201110353512 2011-11-09 2011-11-09 Projection-screen-based multi-touch detection method and multi-touch system CN102508574B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110353512 CN102508574B (en) 2011-11-09 2011-11-09 Projection-screen-based multi-touch detection method and multi-touch system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110353512 CN102508574B (en) 2011-11-09 2011-11-09 Projection-screen-based multi-touch detection method and multi-touch system

Publications (2)

Publication Number Publication Date
CN102508574A true CN102508574A (en) 2012-06-20
CN102508574B CN102508574B (en) 2014-06-04

Family

ID=46220673

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110353512 CN102508574B (en) 2011-11-09 2011-11-09 Projection-screen-based multi-touch detection method and multi-touch system

Country Status (1)

Country Link
CN (1) CN102508574B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102968979A (en) * 2012-11-12 2013-03-13 广东欧珀移动通信有限公司 Screen brightness scheduling method based on curve fitting
CN103093475A (en) * 2013-01-28 2013-05-08 海信集团有限公司 Image processing method and electronic device
CN103279225A (en) * 2013-05-30 2013-09-04 清华大学 Projection type man-machine interactive system and touch control identification method
CN103336634A (en) * 2013-07-24 2013-10-02 清华大学 Adaptive hierarchical structure light-based touch detection system and method
CN103995592A (en) * 2014-05-21 2014-08-20 上海华勤通讯技术有限公司 Wearable equipment and terminal information interaction method and terminal
CN104090689A (en) * 2014-06-27 2014-10-08 深圳市中兴移动通信有限公司 Mobile terminal and interactive projection method and system thereof
CN104464412A (en) * 2014-12-23 2015-03-25 成都韬睿教育咨询有限公司 Remote education system and implementation method thereof
CN105657307A (en) * 2014-12-02 2016-06-08 霍尼韦尔国际公司 System and method of foreground extraction for digital cameras
CN103955316B (en) * 2014-04-28 2016-09-21 清华大学 One kind of finger touch detection system and method
CN106774904A (en) * 2016-12-20 2017-05-31 哈尔滨拓博科技有限公司 Method for converting finger action signal into PC control signal
CN107092350A (en) * 2017-03-22 2017-08-25 深圳大学 Remote computer control system and method
CN107122083A (en) * 2017-04-25 2017-09-01 上海唱风信息科技有限公司 Touch detection method for projection surface

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106095199A (en) * 2016-05-23 2016-11-09 广州华欣电子科技有限公司 Projection screen based touch positioning method and system
TWI626423B (en) * 2016-09-12 2018-06-11 財團法人工業技術研究院 Tapping detecting device, tapping detecting method and smart projecting system using the same

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1655175A (en) * 2004-02-13 2005-08-17 株式会社日立制作所 Table type information terminal
CN101231450A (en) * 2008-02-25 2008-07-30 陈伟山 Multipoint and object touch panel arrangement as well as multipoint touch orientation method
CN101477427A (en) * 2008-12-17 2009-07-08 明 卫 Contact or non-contact type infrared laser multi-point touch control apparatus

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1655175A (en) * 2004-02-13 2005-08-17 株式会社日立制作所 Table type information terminal
CN101231450A (en) * 2008-02-25 2008-07-30 陈伟山 Multipoint and object touch panel arrangement as well as multipoint touch orientation method
CN101477427A (en) * 2008-12-17 2009-07-08 明 卫 Contact or non-contact type infrared laser multi-point touch control apparatus

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102968979A (en) * 2012-11-12 2013-03-13 广东欧珀移动通信有限公司 Screen brightness scheduling method based on curve fitting
CN103093475B (en) * 2013-01-28 2015-05-13 海信集团有限公司 Image processing method and electronic device
CN103093475A (en) * 2013-01-28 2013-05-08 海信集团有限公司 Image processing method and electronic device
CN104766331B (en) * 2013-01-28 2017-10-13 海信集团有限公司 An image processing method and an electronic device
CN104766332B (en) * 2013-01-28 2017-10-13 海信集团有限公司 An image processing method and an electronic device
CN104766329A (en) * 2013-01-28 2015-07-08 海信集团有限公司 Image processing method and electronic device
CN104766332A (en) * 2013-01-28 2015-07-08 海信集团有限公司 Image processing method and electronic device
CN104766331A (en) * 2013-01-28 2015-07-08 海信集团有限公司 Imaging processing method and electronic device
CN104766329B (en) * 2013-01-28 2018-04-27 海信集团有限公司 An image processing method and an electronic device
CN103279225A (en) * 2013-05-30 2013-09-04 清华大学 Projection type man-machine interactive system and touch control identification method
CN103279225B (en) * 2013-05-30 2016-02-24 清华大学 Projection interactive system and a touch recognition method
CN103336634A (en) * 2013-07-24 2013-10-02 清华大学 Adaptive hierarchical structure light-based touch detection system and method
CN103336634B (en) * 2013-07-24 2016-04-20 清华大学 Adaptive layered structure of the light detection system and method based on a touch
CN103955316B (en) * 2014-04-28 2016-09-21 清华大学 One kind of finger touch detection system and method
CN103995592A (en) * 2014-05-21 2014-08-20 上海华勤通讯技术有限公司 Wearable equipment and terminal information interaction method and terminal
CN104090689A (en) * 2014-06-27 2014-10-08 深圳市中兴移动通信有限公司 Mobile terminal and interactive projection method and system thereof
CN105657307A (en) * 2014-12-02 2016-06-08 霍尼韦尔国际公司 System and method of foreground extraction for digital cameras
CN104464412A (en) * 2014-12-23 2015-03-25 成都韬睿教育咨询有限公司 Remote education system and implementation method thereof
CN106774904A (en) * 2016-12-20 2017-05-31 哈尔滨拓博科技有限公司 Method for converting finger action signal into PC control signal
CN107092350A (en) * 2017-03-22 2017-08-25 深圳大学 Remote computer control system and method
CN107122083A (en) * 2017-04-25 2017-09-01 上海唱风信息科技有限公司 Touch detection method for projection surface

Also Published As

Publication number Publication date
CN102508574B (en) 2014-06-04

Similar Documents

Publication Publication Date Title
CN102402680B (en) Hand and indication point positioning method and gesture confirming method in man-machine interactive system
US9552673B2 (en) Grasping virtual objects in augmented reality
KR101984683B1 (en) Multi display device and method for controlling thereof
CN102799277B (en) Wink action-based man-machine interaction method and system
KR101457777B1 (en) A computing device interface
US9600078B2 (en) Method and system enabling natural user interface gestures with an electronic system
CN104246682B (en) Enhanced virtual touchpad and touchscreen
US20110012848A1 (en) Methods and apparatus for operating a multi-object touch handheld device with touch sensitive display
US20120249422A1 (en) Interactive input system and method
CN103729055B (en) Multi display device, a stylus, a display control method and a multi-multi-display system
US9423876B2 (en) Omni-spatial gesture input
CN102541365B (en) Multi-touch command generating apparatus and method
WO2011106201A2 (en) Projectors and depth cameras for deviceless augmented reality and interaction
CN101673161B (en) A visual entity operable without a touch screen system
US8860675B2 (en) Drawing aid system for multi-touch devices
CN102426480A (en) Man-machine interactive system and real-time gesture tracking processing method for same
CN103353935B (en) 3d dynamic gesture recognition method for an intelligent home system
CN103365410B (en) Gesture sensing apparatus and an electronic system having a gesture input function
CN102436343B (en) To align user interface element based on the touch input
CN101593022A (en) Method for quick-speed human-computer interaction based on finger tip tracking
US8485668B2 (en) 3D interaction for mobile device
Shen et al. Vision-based hand interaction in augmented reality environment
EP2972727A1 (en) Non-occluded display for hover interactions
CN102498454B (en) Determining display gesture
CN1912816A (en) Virtus touch screen system based on camera head

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C14 Grant of patent or utility model