CN102193631A - Wearable three-dimensional gesture interaction system and using method thereof - Google Patents

Wearable three-dimensional gesture interaction system and using method thereof Download PDF

Info

Publication number
CN102193631A
CN102193631A CN 201110114604 CN201110114604A CN102193631A CN 102193631 A CN102193631 A CN 102193631A CN 201110114604 CN201110114604 CN 201110114604 CN 201110114604 A CN201110114604 A CN 201110114604A CN 102193631 A CN102193631 A CN 102193631A
Authority
CN
China
Prior art keywords
step
gesture
go
interaction
color
Prior art date
Application number
CN 201110114604
Other languages
Chinese (zh)
Inventor
凌晨
张文俊
赵凡
陈明
Original Assignee
上海大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海大学 filed Critical 上海大学
Priority to CN 201110114604 priority Critical patent/CN102193631A/en
Publication of CN102193631A publication Critical patent/CN102193631A/en

Links

Abstract

The invention discloses a wearable three-dimensional gesture interaction system and a suing method thereof. The system consists of a micro projector, a common camera, a portable computer, and a group of simple and easy marks. The function of human-computer interaction by using gestures and a three-dimensional graphical interface is realized through simple and easy computer vision computation. In the wearable three-dimensional gesture interaction system, a three-dimensional graphical interface is more vivid and the interaction mode is more flexible through a projection screen marked three-dimensional graphical interface mode and a virtual mouse gesture. The wearable three-dimensional gesture interaction system has simple equipment and low cost and can meet the requirements on mobile equipment.

Description

可穿戴式三维手势交互系统及其使用方法 Wearable dimensional gesture interaction system and method of use

技术领域 FIELD

[0001 ] 本发明涉及的是一种可以穿戴的三维图形界面手势交互技术,具体地说是一种基于单目视觉与标记、使用手势进行人机交互的可穿戴式三维手势交互系统及其使用方法。 [0001] The present invention relates to a three-dimensional graphical interface may wear gesture interaction techniques, in particular based on monocular vision with the labeled, using gestures wearable human interaction system and three-dimensional gesture interaction using method.

[0002] 背景技术 [0002] BACKGROUND OF THE INVENTION

随着人们对自然交互的要求不断提高,通过手势进行人机交互已经成为主流趋势。 As people demand continues to increase natural interaction, human-computer interaction through gesture has become a mainstream trend. 为了达到随时随地都能进行虚拟数据的操作,可穿戴式计算机技术是解决此问题的一种可行的方法。 In order to achieve anytime, anywhere to operate virtual data, wearable computer technology to solve this problem is a feasible method. 随着微型投影仪的发展,微型投影仪成为移动设备标准配置已是不可逆的趋势。 With the development of micro-projectors, pico projectors have become standard on mobile devices is an irreversible trend. 同时,人们也渐渐不满足于移动设备过小的显示屏幕。 At the same time, people have gradually satisfied with mobile devices too small display screen. 所以将界面投影到平面上来扩大屏幕大小成为研究热点。 So the interface projected onto the plane up to expand the size of the screen become a hot topic.

[0003] 对于可穿戴式计算机的研究,如;中国专利:其名称为“穿戴式计算机专用的微小型键盘”,申请号200410043913. 8,此专利提供了一种穿戴式计算机专用的微小型键盘,体积小、结构简单,但是交互方式仍然是传统键盘式交互,不符合自然交互的要求。 [0003] For studies wearable computer, such as; China Patent: entitled "wearable computer dedicated miniature keyboard", Application No. 200410043913.8, this patent provides a wearable computer keyboard dedicated micro , small size, simple structure, but interaction is still the traditional keyboard-based interaction, does not meet the requirements of natural interaction. 中国专利: 其名称为“穿戴式输入装置”,申请号200920178274. 4 ;中国专利:其名称为“一种手套式虚拟输入装置”,申请号200920204864. X。 Chinese patent: entitled "wearable input device", Application No. 200920178274.4; China Patent: which is named as "a glove virtual input device", Application No. 200920204864. X. 这些专利拜托传统鼠标键盘式的硬件束缚,实现设备虚拟化的目的,但是需要特制硬件的支持,不适合移动设备的要求。 These patents Come traditional mouse and keyboard style hardware constraints, achieving the purpose of device virtualization, but need the support of special hardware is not required for mobile devices. 中国专利:其名称为“带有中文LINUX操作系统的可穿戴式多媒体无线微型计算机”,申请号00107607. 8,此专利使用人的自然语言进行人机交互,但是人们更加习惯使用手进行交互,故此专利不适合大规模商用推广。 Chinese patent: its name is "wearable wireless multimedia microcomputer with Chinese LINUX operating system," Application No. 00107607.8, this patent using human natural language human-computer interaction, but people are more accustomed to using the hand to interact, Therefore patent is not suitable for large-scale commercial promotion. 中国专利:其名称为“一种用于穿戴式计算机的人机交互头盔”,申请号200710023167. X,此专利提出一种具有视线跟踪功能,能通过配合语音实现对穿戴式计算机进行人机交互的头盔,能广泛应用于航空、航天、机械制造、电力、军事等不能彻底解放人的双手的应用领域,但是对硬件要求过高。 Chinese patent: its name as "a wearable computer for interactive helmet", application number 200710023167. X, this patent proposes a kind of gaze tracking function, can realize with the voice of human-computer interaction wearable computer helmet, it can be widely used applications can not complete the liberation of human hands aviation, aerospace, machinery manufacturing, power, military, etc., but the hardware requirements are too high.

[0004] 发明内容 [0004] SUMMARY OF THE INVENTION

鉴于以上所述现有技术存在的问题和不足,本发明的目的在于提供一种可穿戴式三维手势交互系统及其使用方法,系统结构简单,用户能通过手势对虚拟三维物体进行交互,三维界面生产具体。 In view of the above problems of the prior art and disadvantages, an object of the present invention is to provide a three-dimensional gesture wearable interactive system and method of use, simple structure, the user can gesture to interact with the virtual three-dimensional object, three-dimensional interface production of concrete. 为达到上述目的,本发明的构思是:根据可穿戴式计算机技术要求,本发明包括便携式电脑(1)、摄像头(2)、微型投影仪(3)和彩色标记(4)。 To achieve the above object, the present invention is a concept: The wearable computer technology requirements, the present invention comprises a portable computer (1), the camera (2), micro projector (3) and the color mark (4). 为了使摄像头拍摄的画面是用户看到的画面,故摄像头应靠近用户眼部,将摄像头戴在头上或者挂在胸前。 To make the picture taken by the camera user to see the screen, so the user should be close to the eye camera, the imaging head wearing or hung on the chest. 为了使投影画面不被手所遮挡,微型投影仪应该戴在头部。 In order that the projection screen is not blocked by the hand, micro projector should be worn on the head. 并且为了使摄像头获得的画面与微型投影仪投出的实际画面,由于两者本身位置之间不同造成的几何失真不影响后面处理与降低程序的复杂性,应该使微型投影仪与摄像头尽量靠近,即两者到投影屏幕的距离与夹角尽量相同。 And in order to screen the mini projector camera obtained cast actual screen geometry due to the difference between self position due to distortion does not affect the complexity of the subsequent processing and reduction program, it should make micro projector with the camera as close as possible, i.e. both the distance and the angle between the projection screen as identical as possible. 微型投影仪与摄像头连接在一台便携式电脑上,此便携式电脑可以使用笔记本电脑,也可以使用智能手机。 Micro projector with the camera attached to a portable computer, this laptop can use a laptop, you can use smart phones. 用户手指上需要带上彩色标记,在本发明中,左手的食指、拇指和小指和右手的食指、拇指需要带上不同的彩色标记。 Colored markers need to bring the user's finger, in the present invention, left index finger, the thumb and little finger and index finger of the right hand, the thumb needs to take different colored markers.

[0005] 微型投影仪与摄像头并无特别要求,目前市面上能够购得的均可。 [0005] projector and the micro-camera is not particularly required, can be commercially available at present available. 将摄像头与微型投影仪安置在帽子上,戴在头上,便携式电脑背在身后,这样便构成了穿戴式三维手势交互方法的硬件系统。 The camera and projector disposed on the micro-hat worn on the head, behind the back of the portable computer, so that they constitute a three-dimensional gesture wearable hardware interaction method. 随着移动设备的发展,当微型投影仪也如同摄像头成为移动设备配置 With the development of mobile devices, micro projector as also when the camera becomes mobile device profile

3的时候,本系统可以简化为单一的移动设备,做到一体化的要求。 When 3, the system can be simplified to a single mobile device, to achieve the required integration.

[0006] 根据上述发明构思,本发明采用下述技术方案: [0006] According to the inventive concept, the present invention adopts the following technical scheme:

一种可穿戴式三维手势交互系统,包括便携式电脑(1)、摄像头(2)、微型投影仪(3)和彩色标记(4),其特征在于摄像头和微型投影仪安置在帽子上,使两者到投影屏幕的距离与夹角尽量相同,分别连接在一台便携式电脑上,并且在用户手指上需要带上彩色标记。 A wearable dimensional gesture interaction system comprising a portable computer (1), the camera (2), micro projector (3) and the color mark (4), characterized in that the camera and micro projector disposed on the cap, so that the two and by the distance as far as possible the same as the angle between the projection screen, each connected to a portable computer, and the need to bring the user's finger on the color marking.

[0007] —种上述可穿戴式三维手势交互系统的使用方法,其特征在于操作步骤为: [0007] - use of the above three-dimensional gesture interaction wearable system, characterized by the steps of:

1)标记识别:通过摄像头(2)获得用户手指上彩色标记(4)的信息,通过图像分割获得其坐标信息; 1) identification tag: (2) Information obtained color mark (4) on the finger of the user through the camera, the coordinate information which is obtained by dividing the image;

2)动作识别:实时计算各个标记的运动向量以及各标记的位置关系,从而分析是何种动作指令; 2) Operation of recognition: a motion vector calculated in real time and the position relation of each marker of each mark, which is to analyze the operation command;

3)响应输出:根据动作指令修改画面,并通过微型投影仪(3)投影到一个平面上,实现人机交互。 3) responsive to the output: an operation command to modify the picture, and by micro projector (3) projected onto a plane, interacting with a computer.

[0008] 上述步骤1的标记识别,通过成熟的基于颜色的计算机图像处理技术,将摄像头拍摄到的实际图像进行分割,获得各个彩色标记的坐标信息。 Mark recognition [0008] Step 1 above, by means of highly color-based computer image processing technology, the camera to take the actual image is divided, to obtain the coordinate information of each color indicia.

[0009] 上述步骤2的动作识别,本发明中的交互方式是手势交互,包括四类手势:多点触控手势、自由手势、混合手势和虚拟鼠标。 [0009] The above step 2 of the identification operation, the alternate method of the present invention gesture interaction, comprising four gestures: multi-touch gestures, gesture free, mixed and virtual mouse gesture. 多点触控手势就是如今最热门的多点触控的一系列手势,通过此类手势,可以自由缩放图像、移动图标等等。 Multi-touch gestures is now the most popular series of multi-touch gestures, through such gestures can zoom images, moving icons and so on. 自由手势就是人们日常生活中的手势,例如左手做出一个“0K”手势,便能实现确认的操作;左手食指与右手食指交叉呈“X”形,就能实现取消的操作;两手撑平,左手在上,右手在下,左手拇指与右手食指相触,左手食指与右手拇指相触,摆出一个“取景”的手势,就可以将当前摄像头获得的画面保存起来,即拍照功能。 Free gesture is a gesture in daily life, for example, to make a left-hand "0K" gesture, an operation can achieve confirmed; intersecting left index finger and the right forefinger was "X" shape, can achieve a cancellation operation; hands hold flat, on the left hand, right lower, left thumb and right index finger touching, right thumb and left index finger touching, put a "finder" gesture, the screen can be saved to obtain the current camera, i.e. camera. 混合手势就是通过左右手配合进行人机交互,左手控制三维变换,例如视角的选择等;右手控制平面变换,如画面的左右移动等。 Gesture is performed by mixing with the right-hand human-computer interaction, a left-handed three-dimensional transformation, e.g. selection perspective; hand control plane conversion, such as the move around the screen and the like. 虚拟鼠标是本发明提出的一种手势。 Is a virtual mouse gestures proposed by the invention. 虚拟鼠标即将左手手掌作为鼠标使用。 Virtual mouse left hand is about to be used as a mouse. 此手势是将左手撑开,手心面对摄像头,除拇指外的四个手指尽量弯曲,使得四个手指的手指甲在一直线上。 This gesture is a distraction to his left hand, palm facing the camera, four fingers except the thumb bent as much as possible, so that the fingernail four fingers in a straight line. 此时左手食指与小指的彩色标记在一直线上,将食指上的标记作为左上角顶点,小指上的标记作为右上角顶点,向下做一正方形,在这正方形区域内即为控制区域,与图形界面上的坐标对应。 At this time, the left hand little finger and index finger of the color marker on a straight line, the indicia on the top left vertex as the index finger, a little finger marks on the top right vertex, to make down a square, in which the control region is the region of the square, and corresponding to the coordinates on the graphical interface. 并且此正方形区域在左手手掌之上,当右手食指在控制区域中移动时,可以接触手掌,即可看作为手掌心为控制区域。 And this square region on the left hand, the right index finger when the movement in the control region, may contact the palm of the hand, the palm can be seen as a control region. 这样设计的优点是右手食指在进行移动的时候可以有借力的地方,能够使移动更稳,达到精确定位的效果。 The advantage of this design is that when the right index finger during the movement can place leveraging can be made to move more stable, precise positioning to achieve the effect. 将在具体实施方式中,结合图进行具体说明。 In a specific embodiment, the binding will be specifically described in FIG.

[0010] 上述步骤3的响应输出,将便携电脑中的图形界面通过微型投影仪投影到墙上、 桌面等物理平面上。 [0010] The step response of the output 3, the portable computer graphical interface by a micro projector onto a physical plane wall, desk or the like. 使用的图形界面不仅可以使用如今常规的二维图形界面,还可以使用三维图形界面。 Use graphical interface can now use not only conventional two-dimensional graphical interface, you can also use three-dimensional graphical interface. 可以兼容如今商业三维界面,例如BumpTop和Real Desktop。 Today, business can be compatible with three-dimensional interface, such as BumpTop and Real Desktop. 本发明也提出一种利用投影屏幕标记实现三维界面的方法。 The present invention also provides a method of using a labeled three-dimensional projection screen interface. 投影屏幕标记即在原有的界面四个顶角上做绘制不同的标记再投射到投影屏幕上,这些标记就称为“投影屏幕标记”。 I.e. projection screen to do different mark drawn on the original four corners interface tag to be projected on a projection screen, these markers are called "projection screen tag." 由于人与投影平面法向量的夹角的不同,投影的画面会产生几何失真,然后再通过摄像头获得实际投影图像。 Since the projection angle man vector method different plane, the projection screen will produce geometric distortions, then the actual projection image obtained through the camera. 根据四个标记和其视觉变换矩阵,计算出投影平面的方向向量,进而渲染出虚拟三维物体并投影到平面上;最后用户能通过手势对虚拟三维物体进行交互。 The four markers which visual transformation matrix and calculate the direction vector projection plane, thereby rendering the virtual three-dimensional object and projected onto a plane; the final user can gesture to interact with a virtual three-dimensional object. 这样随着用户的移动,即用户与投影平面法向量夹角的变化,将渲染不同的虚拟物体,但用户将产生虚拟物体犹如真实存在的感受。 As such a mobile user, i.e. the user plane normal vector projection angle changes, a different rendering of the virtual object, but the user will have a feeling as if real virtual object exists. 并且用户能通过手势对虚拟三维物体进行交互。 And the user can gesture to interact with a virtual three-dimensional object. 将在具体实施方式中,结合图进行具体说明。 In a specific embodiment, the binding will be specifically described in FIG.

[0011] 本发明与现有技术相比较,具有如下显而易见的突出实质性特点和显著优点: [0011] The present invention compared with the prior art, has a protruding apparent substantive features and significant advantages:

本发明通过一种投影屏幕标记的三维图形界面方式和一种虚拟鼠标手势,使三维图形界面更加生动,交互方式更加灵活。 The present invention is achieved by a projection screen labeled and one three-dimensional graphic interface mode virtual mouse gestures, more vivid three-dimensional graphical interface, interactive more flexible. 本发明设备简单、成本低,能满足移动设备的要求。 The present invention is simple equipment, low cost, to meet the requirements of the mobile device.

[0012] 附图说明 [0012] BRIEF DESCRIPTION OF DRAWINGS

图1本发明实施例的示意图; 图2本发明实施例的流程示意图; 图3虚拟鼠标示意图; 图4投影屏幕标记示意图。 Figure 1 is a schematic diagram of the embodiment of the present invention; FIG. 2 embodiment of the process of the present invention a schematic view; 3 a schematic virtual mouse; Figure 4 a schematic view of the projection screen tag.

具体实施方式 Detailed ways

[0013] 以下结合附图对本发明的优选实施例作进一步的详细说明。 [0013] The following embodiments in conjunction with the accompanying drawings of the preferred embodiments of the present invention will be further described in detail.

[0014] 实施例一:如图1所示,可穿戴式三维手势交互系统包括一台便携式电脑(1)、一个摄像头(2 )、一台微型投影仪(3 )和一些彩色标记(4 )。 [0014] Example a: 1, the three-dimensional gesture interaction wearable system comprising a portable computer (1), a camera (2), a mini projector (3) and a number of colored markers (4) . 本交互系统对硬件要求低,微型投影仪和摄像头没有要求,故系统成本低。 This interactive system low hardware requirements, micro projector and camera is not required, so the system cost. 为了使摄像头拍摄的画面是用户看到的画面,并且手部动作不影响投影效果,如图1所示,将微型投影仪和摄像头安置在帽子上。 To make the picture taken by the camera user to see the screen, and the operation of the hand does not affect the projection, as shown in FIG. 1, the micro projectors and cameras disposed in the hat.

[0015] 实施例二:可穿戴式三维手势交互系统的使用方法,如图2所示,其具体步骤如下: Example Two [0015] Embodiment: use a three-dimensional gesture interaction wearable system, shown in Figure 2, the specific steps are as follows:

①、标记识别; ①, identification tag;

②、动作识别; ②, action recognition;

③、响应输出。 ③, responsive to the output.

[0016] 上述步骤①所述的标记识别,是通过摄像头获得用户手指上彩色标记的信息,通过图像分割获得其坐标信息,其具体步骤如下: [0016] The above step ① the identification indicia, is color information obtained by the camera on the user's finger marks, the coordinate information which is obtained by dividing the image, the specific steps are as follows:

(1)、获取关键帧画面。 (1), obtain key frames.

[0017] O)、对关键帧画面中的彩色标记进行图像分割,获得各个彩色标记的坐标,并计算各标记的相对位置作为手势。 [0017] O), the key frames are color-coded image segmentation, to obtain the coordinates of respective color marks, and calculates the relative positions of the marks as gestures.

[0018] 上述步骤②所述的动作识别进行手势交互,其具体步骤如下: [0018] The above steps ② interact gesture operation recognition, the specific steps are as follows:

(3)、是否是多点触控手势? (3), whether it is multi-touch gestures? 是多点触控手势则转步骤(7),否则转步骤(4)。 Multi-touch gestures are then go to step (7), otherwise go to step (4).

[0019] 0)、是否是自由手势? [0019] 0), whether it is free gesture? 是则转步骤(8),否则转步骤(5)。 Yes go to Step (8), otherwise go to step (5).

[0020] (5)、是否是混合手势? [0020] (5), whether it is a mixed gesture? 是则转步骤(9),否则转步骤(6)。 Is the transfer step (9), otherwise go to step (6).

[0021] (6)、是否是虚拟鼠标? [0021] (6), whether it is virtual mouse? 是则转步骤(10),否则转步骤(1)。 Yes go to Step (10), otherwise go to step (1).

[0022] 步骤(6)中一种特殊的手势:虚拟鼠标,如图3所示,左手食指与小指的彩色标记在一直线上,将食指上的标记作为左上角顶点,小指上的标记作为右上角顶点,向下做一正方形,在这正方形区域内即为控制区域,即手掌心,与图形界面上的坐标对应。 [0022] Step (6) in a particular gesture: virtual mouse, shown in Figure 3, the left index finger and little finger of the color marker on a straight line, the indicia on the top left vertex as the index finger, a little finger on the mark top right vertex, to make a square downward, in which the control region is the square region, i.e. palm, and correspond to coordinates on the graphical interface. 这样设计的优点是右手食指在进行移动的时候可以有借力的地方,能够使移动更稳,达到精确定位的效果。 The advantage of this design is that when the right index finger during the movement can place leveraging can be made to move more stable, precise positioning to achieve the effect. 运用虚拟鼠标可以作为虚拟键盘输入文字,也可以在界面上精确地绘图。 Using virtual mouse can be used as a virtual keyboard to enter text, you can accurately draw on screen.

[0023] 上述步骤③所述的响应输出,根据动作指令修改画面,并通过微型投影仪投影到一个平面上,实现人机交互,其具体步骤如下: [0023] The step response of the output ③, modified according to the operation command screen, and by a micro projector onto a projection plane, interacting with a computer, the specific steps are as follows:

(7)、根据彩色标记的不同手势,做出相应的多点触控操作,完成交互后转步骤(1)。 (7), color-coded depending on the gesture, and make the appropriate multi-touch operation, after the completion of transfer interaction in step (1). [0024] (8)、根据彩色标记的不同手势,做出相应的自由手势操作,完成交互后转步骤(1)。 [0024] (8), the color-coded depending on the gesture, the gesture operation accordingly free, after the completion of transfer interaction in step (1).

[0025] (9)、根据彩色标记的不同手势,做出相应的混合手势操作,完成交互后转步骤(1)。 [0025] (9), color-coded depending on the gesture, the gesture operation made corresponding mixed, after the completion of transfer interaction in step (1).

[0026] (10)、根据彩色标记的不同手势,做出相应的虚拟鼠标操作,完成交互后转步骤(1)。 [0026] (10), color-coded depending on the gesture, and make the appropriate virtual mouse operation, after the completion of transfer interaction in step (1).

[0027] 本发明中可以使用普通二维图形界面,也可以使用三维图形界面。 [0027] The present invention can be used in ordinary two-dimensional graphical interface may be a three-dimensional graphical interface. 本实施例给出一种特殊的三维图形界面“投影屏幕标记”,如图4所示,在原有的界面四个顶角上绘制不同的标记再投射到投影屏幕上。 This embodiment provides a special three-dimensional graphic interface "mark projection screen", shown in Figure 4, different mark drawn on the original screen four corners and then onto a projection screen. 由于人与投影平面法向量的夹角的不同,投影的画面将产生几何失真,然后再通过摄像头获得这些实际投影图像。 Because different people angle projection plane vector method, the projected picture will produce geometric distortions, then the actual projection image obtained through the camera. 根据四个标记和其视觉的变换矩阵: The four transformation matrix and its visual marking of:

Figure CN102193631AD00061

其中,Tctr是世界坐标系(Zw,〗ς,Iw )到摄像头坐标系(xc ,Yc, zc)的变换 Wherein, Tctr is the world coordinate system (Zw,〗 ς, Iw) to the camera coordinate system (xc, Yc, zc) conversion

矩阵。 matrix. 在? in? ;-中! ;-in! ^-为选择矩阵,獸3X1为平移矩阵,I3为透视变换矩阵。 ^ - is a selection matrix, the translation matrix is ​​a 3X1 animals, I3 perspective transformation matrix. 计算出投影平面 Calculated projection plane

的方向向量,进而渲染出虚拟三维物体并投影到平面上,最后用户能通过手势对虚拟三维物体进行交互。 Direction vector, and thus rendering the virtual three-dimensional object is projected onto a plane, and, finally the user can gesture to interact with a virtual three-dimensional object. 以往基于标记的增强现实系统是摄像头位置固定,标记位置变动。 Previous tag-based augmented reality system is a fixed camera position, mark the location change. 而本发明中的投影屏幕标记是标记位置,即投影屏幕不动,摄像头移动。 The projection screen of the present invention is marked in the mark position, i.e. the projection screen does not move, the camera moves. 这样随着用户的移动,即用户与投影平面法向量夹角的变化,将渲染不同的虚拟物体,但用户将产生虚拟物体犹如真实存在的感受,比以往三维界面更加生动具体。 As such a mobile user, i.e. the user plane normal vector projection angle changes, a different rendering of the virtual object, but the user will have a feeling as if the virtual object existing real, three-dimensional interface more vivid than conventional concrete. 并且用户能通过手势对虚拟三维物体进行交互。 And the user can gesture to interact with a virtual three-dimensional object.

Claims (5)

1. 一种可穿戴式三维手势交互系统,包括便携式电脑(1)、摄像头(2)、微型投影仪(3) 和彩色标记(4),其特征在于摄像头和微型投影仪安置在帽子上,使两者到投影屏幕的距离与夹角尽量相同,分别连接在一台便携式电脑上,并且在用户手指上需要带上彩色标记。 A wearable dimensional gesture interaction system comprising a portable computer (1), the camera (2), micro projector (3) and the color mark (4), characterized in that the camera and micro projector disposed on the cap, so that both the distance and the angle between the projection screen as identical as possible, are connected to a portable computer, and the need to bring the user's finger on the color marking.
2. 一种根据权利要求1所述的可穿戴式三维手势交互系统的使用方法,其特征在于操作步骤为:①标记识别:通过摄像头(2)获得用户手指上彩色标记(4)的信息,通过图像分割获得其坐标信息;②动作识别:实时计算各个标记的运动向量以及各标记的位置关系,从而分析是何种动作指令;③响应输出:根据动作指令修改画面,并通过微型投影仪(3)投影到一个平面上,实现人机交互。 A method of using a three-dimensional gesture interaction wearable system according to claim 1, characterized in that the steps are: ① identification tag: obtain the user's finger on the color marking (4) information through the camera (2), which is obtained by dividing the image coordinate information; ② motion recognition: the motion vectors calculated in real time the positional relationship between each mark and each of the markers to analyze what kind of operation command; ③ output response: Review screen in accordance with operation command, and by a micro projector ( 3) projected onto a plane, interacting with a computer.
3.根据权利要求2所述的可穿戴式三维手势交互系统的使用方法,其特征在于所述步骤①标记识别的具体操作步骤如下:(1)、获取关键帧画面;O)、对关键帧画面中的彩色标记进行图像分割,获得各个彩色标记的坐标,并计算各标记的相对位置作为手势。 The use of three-dimensional gesture interaction wearable system according to claim 2, wherein said step of identifying marks ① following these steps: (1) obtaining the key frames; O), keyframes screen color image segmentation mark to obtain the coordinates of respective color marks, and calculates the relative positions of the marks as gestures.
4.根据权利要求2所述的可穿戴式三维手势交互系统的使用方法,其特征在于所述步骤②动作识别的具体操作步骤如下:(3)、是否是多点触控手势? The use of three-dimensional gesture interaction wearable system according to claim 2, wherein said step of identifying ② operation steps are as follows: (3), whether the multi-touch gestures? 是多点触控手势则转步骤(7),否则转步骤(4);G)、是否是自由手势? Is the multi-touch gestures go to Step (7), otherwise go to step (4); G), whether it is free gesture? 是则转步骤(8),否则转步骤(5);(5)、是否是混合手势? It is go to Step (8), otherwise go to step (5); (5) whether the gesture is a hybrid? 是则转步骤(9),否则转步骤(6);(6)、是否是虚拟鼠标? It is go to Step (9), otherwise go to step (6); (6), whether it is virtual mouse? 是则转步骤(10),否则转步骤(1);步骤(6)中一种特殊的手势:虚拟鼠标,左手食指与小指的彩色标记在一直线上,将食指上的标记作为左上角顶点,小指上的标记作为右上角顶点,向下做一正方形,在这正方形区域内即为控制区域,即手掌心,与图形界面上的坐标对应;这样设计的优点是右手食指在进行移动的时候可以有借力的地方,能够使移动更稳,达到精确定位的效果;C3)、是否是多点触控手势? Yes go to Step (10), otherwise go to step (1); in step (6) in a particular gesture: virtual mouse, the left index finger and little finger of the color marker on a straight line, the indicia on the top left vertex as the forefinger , as little finger marks on the top right vertex, to make down a square, in which the control region is the square region, i.e. palm, corresponding to the coordinates on the graphical interface; the advantage of this is that during the movement of the right index finger when the can have a place to leveraging, enabling the movement more stable, achieve precise positioning; C3), whether it is multi-touch gestures? 是多点触控手势则转步骤(7),否则转步骤(4)。 Multi-touch gestures are then go to step (7), otherwise go to step (4).
5.根据权利要求2所述的可穿戴式三维手势交互系统的使用方法,其特征在于所述步骤③响应输出的具体操作步骤如下:(7)、根据彩色标记的不同手势,做出相应的多点触控操作,完成交互后转步骤(1);(8)、根据彩色标记的不同手势,做出相应的自由手势操作,完成交互后转步骤(1);(9)、根据彩色标记的不同手势,做出相应的混合手势操作,完成交互后转步骤(1);(10)、根据彩色标记的不同手势,做出相应的虚拟鼠标操作,完成交互后转步骤(1)。 The use of three-dimensional gesture interaction wearable system according to claim 2, wherein said step ③ responsive to the output of the specific steps are as follows: (7), depending on the color-coded gesture, accordingly multi-touch operation, after the completion of transfer interaction in step (1); (8), the color-coded depending on the gesture, the gesture operation accordingly free, after the completion of transfer interaction in step (1); (9) a color marking different gestures, gesture operation corresponding mixed to make, after the completion of transfer interaction in step (1); (10), color-coded depending on the gesture, and make the appropriate virtual mouse operation, after the completion of transfer interaction in step (1).
CN 201110114604 2011-05-05 2011-05-05 Wearable three-dimensional gesture interaction system and using method thereof CN102193631A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110114604 CN102193631A (en) 2011-05-05 2011-05-05 Wearable three-dimensional gesture interaction system and using method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110114604 CN102193631A (en) 2011-05-05 2011-05-05 Wearable three-dimensional gesture interaction system and using method thereof

Publications (1)

Publication Number Publication Date
CN102193631A true CN102193631A (en) 2011-09-21

Family

ID=44601809

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110114604 CN102193631A (en) 2011-05-05 2011-05-05 Wearable three-dimensional gesture interaction system and using method thereof

Country Status (1)

Country Link
CN (1) CN102193631A (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014012486A1 (en) * 2012-07-17 2014-01-23 Gao Shouqian Wearable wireless intelligent electronic device having removable and freely-combinable functional modules
CN103995592A (en) * 2014-05-21 2014-08-20 上海华勤通讯技术有限公司 Wearable equipment and terminal information interaction method and terminal
CN104063038A (en) * 2013-03-18 2014-09-24 联想(北京)有限公司 Information processing method and device and electronic equipment
CN104102071A (en) * 2013-04-12 2014-10-15 王旭东 Full-automatic photographing equipment
CN104345802A (en) * 2013-08-08 2015-02-11 派布勒斯有限公司 Method and device for controlling a near eye display
CN104460988A (en) * 2014-11-11 2015-03-25 陈琦 Input control method of intelligent cell phone virtual reality device
CN104461277A (en) * 2013-09-23 2015-03-25 Lg电子株式会社 Mobile terminal and method of controlling therefor
CN104660941A (en) * 2013-11-22 2015-05-27 北京弘天智达科技有限公司 Micro display communication device, system and method
CN104866103A (en) * 2015-06-01 2015-08-26 联想(北京)有限公司 Relative position determining method, wearable electronic equipment and terminal equipment
WO2015165181A1 (en) * 2014-04-28 2015-11-05 京东方科技集团股份有限公司 Method and apparatus for controlling projection of wearable device, and wearable device
CN105302337A (en) * 2013-06-04 2016-02-03 李文傑 High resolution and high sensitivity three-dimensional (3D) cursor maneuvering system, device and motion detection method
CN106293078A (en) * 2016-08-02 2017-01-04 福建数博讯信息科技有限公司 Virtual reality exchange method based on photographic head and device
CN104090465B (en) * 2014-06-17 2017-01-11 福建水立方三维数字科技有限公司 Three-dimensional interactive projection imaging method
WO2017107182A1 (en) * 2015-12-25 2017-06-29 深圳市柔宇科技有限公司 Head-mounted display device
CN107085467A (en) * 2017-03-30 2017-08-22 北京奇艺世纪科技有限公司 A kind of gesture identification method and device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101077232A (en) * 2007-06-07 2007-11-28 南京航空航天大学 Human-computer interaction helmet for type computer

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101077232A (en) * 2007-06-07 2007-11-28 南京航空航天大学 Human-computer interaction helmet for type computer

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《北京理工大学学报》 20051231 柳阳等 基于穿戴视觉的人手跟踪与手势识别方法 第1083-1086页 1-2 第25卷, 第12期 *
《计算机辅助设计与图形学学报》 20110430 李岩等 一种手部实时跟踪与定位的虚实碰撞检测方法 第713-718页 1-2 第23卷, 第4期 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014012486A1 (en) * 2012-07-17 2014-01-23 Gao Shouqian Wearable wireless intelligent electronic device having removable and freely-combinable functional modules
CN104063038A (en) * 2013-03-18 2014-09-24 联想(北京)有限公司 Information processing method and device and electronic equipment
CN104102071A (en) * 2013-04-12 2014-10-15 王旭东 Full-automatic photographing equipment
CN105302337B (en) * 2013-06-04 2019-10-18 李文傑 High-resolution and three-dimensional (3D) the mouse mobile control system of high sensitive, mobile controller part and its motion detection method
CN105302337A (en) * 2013-06-04 2016-02-03 李文傑 High resolution and high sensitivity three-dimensional (3D) cursor maneuvering system, device and motion detection method
CN104345802B (en) * 2013-08-08 2019-03-22 脸谱公司 For controlling the devices, systems, and methods of near-to-eye displays
CN104345802A (en) * 2013-08-08 2015-02-11 派布勒斯有限公司 Method and device for controlling a near eye display
CN104461277A (en) * 2013-09-23 2015-03-25 Lg电子株式会社 Mobile terminal and method of controlling therefor
CN104660941A (en) * 2013-11-22 2015-05-27 北京弘天智达科技有限公司 Micro display communication device, system and method
WO2015165181A1 (en) * 2014-04-28 2015-11-05 京东方科技集团股份有限公司 Method and apparatus for controlling projection of wearable device, and wearable device
US9872002B2 (en) 2014-04-28 2018-01-16 Boe Technology Group Co., Ltd. Method and device for controlling projection of wearable apparatus, and wearable apparatus
CN103995592A (en) * 2014-05-21 2014-08-20 上海华勤通讯技术有限公司 Wearable equipment and terminal information interaction method and terminal
CN104090465B (en) * 2014-06-17 2017-01-11 福建水立方三维数字科技有限公司 Three-dimensional interactive projection imaging method
CN104460988B (en) * 2014-11-11 2017-12-22 陈琦 A kind of input control method of smart mobile phone virtual reality device
CN104460988A (en) * 2014-11-11 2015-03-25 陈琦 Input control method of intelligent cell phone virtual reality device
CN104866103A (en) * 2015-06-01 2015-08-26 联想(北京)有限公司 Relative position determining method, wearable electronic equipment and terminal equipment
CN104866103B (en) * 2015-06-01 2019-12-24 联想(北京)有限公司 Relative position determining method, wearable electronic device and terminal device
WO2017107182A1 (en) * 2015-12-25 2017-06-29 深圳市柔宇科技有限公司 Head-mounted display device
CN106293078A (en) * 2016-08-02 2017-01-04 福建数博讯信息科技有限公司 Virtual reality exchange method based on photographic head and device
CN107085467A (en) * 2017-03-30 2017-08-22 北京奇艺世纪科技有限公司 A kind of gesture identification method and device

Similar Documents

Publication Publication Date Title
Hilliges et al. HoloDesk: direct 3d interactions with a situated see-through display
US8669939B2 (en) Spatial, multi-modal control device for use with spatial operating system
CN103443742B (en) For staring the system and method with gesture interface
Balakrishnan et al. Exploring interactive curve and surface manipulation using a bend and twist sensitive input strip
US10095030B2 (en) Shape recognition device, shape recognition program, and shape recognition method
JP6116064B2 (en) Gesture reference control system for vehicle interface
JP2006506737A (en) Body-centric virtual interactive device and method
US9471148B2 (en) Control system for navigating a principal dimension of a data space
US10296099B2 (en) Operating environment with gestural control and multiple client devices, displays, and users
US8866740B2 (en) System and method for gesture based control system
US20130076617A1 (en) Adaptive tracking system for spatial input devices
Malik et al. Interacting with large displays from a distance with vision-tracked multi-finger gestural input
US9471147B2 (en) Control system for navigating a principal dimension of a data space
EP3382510A1 (en) Visibility improvement method based on eye tracking, machine-readable storage medium and electronic device
US8681098B2 (en) Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes
US10235412B2 (en) Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes
Harrison et al. OmniTouch: wearable multitouch interaction everywhere
Benko et al. Cross-dimensional gestural interaction techniques for hybrid immersive environments
CN103246351B (en) A kind of user interactive system and method
Izadi et al. C-Slate: a multi-touch and object recognition system for remote collaboration using horizontal surfaces
US8537112B2 (en) Control system for navigating a principal dimension of a data space
Grossman et al. Multi-finger gestural interaction with 3d volumetric displays
Mistry et al. WUW-wear Ur world: a wearable gestural interface
CN102822862B (en) Calculation element interface
US20120013613A1 (en) Tools for Use within a Three Dimensional Scene

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
C02 Deemed withdrawal of patent application after publication (patent law 2001)