CN103279188A - Method for operating and controlling PPT in non-contact mode based on Kinect - Google Patents

Method for operating and controlling PPT in non-contact mode based on Kinect Download PDF

Info

Publication number
CN103279188A
CN103279188A CN2013102071721A CN201310207172A CN103279188A CN 103279188 A CN103279188 A CN 103279188A CN 2013102071721 A CN2013102071721 A CN 2013102071721A CN 201310207172 A CN201310207172 A CN 201310207172A CN 103279188 A CN103279188 A CN 103279188A
Authority
CN
China
Prior art keywords
kinect
palm
ppt
depth
center
Prior art date
Application number
CN2013102071721A
Other languages
Chinese (zh)
Inventor
路飞
田国会
李健
刘志勇
Original Assignee
山东大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 山东大学 filed Critical 山东大学
Priority to CN2013102071721A priority Critical patent/CN103279188A/en
Publication of CN103279188A publication Critical patent/CN103279188A/en

Links

Abstract

The invention relates to a method for operating and controlling a PPT in a non-contact mode based on Kinect. The method comprises the following steps that the Kinect is accessed to a computer system; the position information and the depth information of a palm are acquired by starting the Kinect to track the palm; the two-dimension plane area where the palm is located at a position where the palm is 1000 mm deep away from the center of the Kinect is divided into several functional arrears, namely a laser pen area, an isolating area and a control area; corresponding PPT operating commands at different positions of each area of the palm are defined; finally corresponding operation is carried out on the PPT by a computer according to gesture control commands. According to the method for operating and controlling the PPT in the non-contact mode based on the Kinect, non-contact operation and control of the demonstration of PPT software is achieved, the degree of demonstration freedom is improved, and operation accuracy is high.

Description

一种基于Kinect无接触性操作控制PPT的方法 Kinect one kind of non-contacting operation control method based on PPT

一、技术领域 First, the technical field

[0001] 本发明涉及智能化会议中控制PPT演示软件的方法,具体是无接触性操作控制PPT演示软件的方法。 [0001] The present invention relates to a method for controlling intelligent conference PPT presentation software, the operation control method without particular presentation software PPT contact.

二、背景技术 Second, the technical background

[0002] 近年来,随着会议智能化程度的普遍提高,几乎所有行业的正式会议都会使用Power Point来进行辅助演示。 [0002] In recent years, along with a general increase in intelligence meetings, official meetings in almost all industries will be aided using Power Point presentations. 随着PPT已经成为各种正式会议的必备演示工具,该软件的使用为用户提供了很大的方便,同时很多研究人员也对PPT控制进行了相关开发,其中包括无线接收器激光笔、声音控制PPT、手势控制PPT。 With the PPT presentation has become an essential tool for a variety of formal meetings, the use of the software provides users with a great convenience, while many researchers to control the PPT related development, including wireless receivers laser pointer, sound control PPT, gesture control PPT.

[0003] 虽然一方面上述研究成果简化了人机之间的接触性操作,但是还没有使演讲人员完全从接触性的操作中解放出来。 [0003] Although the aspect of the above findings simplifies the operation of contact between man and machine, but also did not make a speech completely liberated from personnel operating in contact. PPT本身就是微软以方便用户为目的而开发的,接触性操作PPT软件的演示限制了操作人员演示会议内容过程中的自由度。 PPT itself is a user-friendly Microsoft developed for the purpose of operating PPT presentation contact the software limits the operator to demonstrate the degree of freedom during the meeting content. 另一方面现有的声音控制PPT、手势控制PPT的方法灵敏度,准确度较差,并不能满足演讲者应用的需求。 On the other hand PPT conventional sound control, gesture control method of PPT sensitivity, accuracy is poor, and can not meet the needs of the application speakers.

[0004] Kinect是微软在2010年6月14日推出的新型3D体感摄像设备,它能对周围场景进行深度成像,且能获取人体骨骼信息,可用于人体肢体语言的识别。 [0004] Kinect is Microsoft in June 14, 2010 launch of the new 3D somatosensory camera equipment that can image the surrounding scene depth, and can obtain information about the human skeleton can be used to identify the human body language. Kinect成本低廉,功能强大,其本身及配套软件并没有提供无接触性操作控制PPT的功能,但是其具有的特性完全适用于将其用于智能会议中,实现无接触性操作控制PPT演示软件的目的。 Kinect low-cost, powerful, and supporting software itself does not provide non-contact operation control PPT functions, but it has the characteristics of fully applicable to it for a smart conference, contact-free operation control PPT presentation software purpose.

三、发明内容 III. SUMMARY OF THE INVENTION

[0005] 本发明所解决的问题,就是现有PPT的操控多为接触性操控,不能使演讲者脱离接触操作工具,灵活性差;现有无接触性操控PPT的方法如声音控制、某些手势控制,多为灵敏度低,准确度较低的操作,不能满足讲演过程中演讲者的应用需要。 [0005] The problem addressed by the present invention is mostly conventional PPT manipulation control contact, the speaker can not disengage the operating tool, inflexible; conventional non-contact manipulation method such as a sound control PPT, certain gestures control, mostly low sensitivity, less accurate operation, can not meet the application needs during the lecture the lecturer.

[0006] 为了解决以上问题,本发明利用Kinect摄像头的摄像和手心跟踪功能,提供一种基于Kinect无接触性操作控制PPT的方法,包括以下步骤: [0006] In order to solve the above problem, the present invention utilizes an imaging camera Kinect palm and tracking, there is provided a method for non-contacting operation Kinect PPT-based control, comprising the steps of:

[0007] a、将Kinect接入计算机系统; [0007] a, Kinect access to the computer system;

[0008] b、通过启动Kinect对手心的跟踪,获得手心的位置信息,以及手心的深度信息,在手心距离Kinect深度传感器中心IOOOmm深度的位置,对手心所在的二维平面区域进行功能区域的划分,将其划分为激光笔区域、隔离区域、控制区域,并定义手心在各区域中不同位置相对应的PPT操作的指令,并作以下规定:标准距离是位于Kinect正前方I米处的距离;标准平面是位于标准距离处,手心移动的全部范围所在的平面;标准区域是手心在标准平面移动时在Kinect上的图像范围,并将其规定为矩形区域;挥动中心是使用者在胸前伸出右手时,手心在整个挥动范围中所处的的位置,可在不同深度信息下;基准区域是手心移动范围在Kinect图像上的中心,即人在垂直于Kinect方向的中心位置所产生的图像上的挥手区域; [0008] b, by tracking the heart starting Kinect opponents, the position information of the palm, the palm and the depth information, the depth sensor Kinect IOOOmm center depth position from the palm of the hand, a two-dimensional planar area of ​​the heart where the opponent divided functional areas , it is divided into laser pointer region, the isolation region, the control region, and the definition instruction palm different positions in each region corresponding to the PPT operation, with the following provisions: the standard distance is a distance in front of the Kinect I meters located; standard plane is located at a standard distance, the full range of plane movement of the palm is located; standard palm area is in the standard range in the image plane shift Kinect, and it is defined as a rectangular area; swing center extends in the chest user when the right hand, in which the position of the palm of the entire swing range, the information may be at different depths; reference region is centered on the palm of the hand moving range Kinect image, i.e. the image to the person at the center position in the vertical direction generated Kinect waving the region;

[0009] C、计算机根据手势控制指令对PPT执行相应操作。 [0009] C, PPT computer to perform a corresponding operation according to a control command gesture. 实现了无接触性操作全自动控制PPT演示过程的效果,该方法避开了对手势轨迹的研究,简化了识别难度,并提高识别率。 To achieve the effect of non-contacting operation PPT presentation automatic control process, which avoids the study of gesture track, which simplifies the identification more difficult, and improve the recognition rate.

[0010] 本发明的有益效果是:演讲者可以实现无接触性操作控制PPT软件的演示过程,将演讲者从接触性的操作中解放出来,提高其演示的自由度,并且该方法避开了对手势轨迹的研究,简化了识别难度,并提高识别率、准确率和灵敏度,同时本发明具有简单实用、扩展性好以及便于普及的优点。 [0010] Advantageous effects of the present invention are: speakers during the presentation can be achieved without contacting operation control software PPT, liberated from the speaker of the operation of the contact, increasing the degree of freedom of presentation, and the method avoids the Study gesture trajectory simplify identification difficult, and improve the recognition rate, accuracy and sensitivity, while the present invention has a simple, practical, and easy to extend the advantages of good popularity.

[0011] 在标准平面中,手心在图片中移动可以成功的控制PPT的正常播放以及产生激光笔效果。 [0011] In the standard plane, the palm movement in the picture can be successfully controlled and generation of normal-play PPT laser pointer effect. 但在实际应用中,演讲者的手不是始终都在标准平面内,所以对应在二维图片上的区域也会产生差异。 However, in practice, the speaker's hand is not always within the standard plane, corresponding to the two-dimensional picture of the area will also have differences. 演讲者手心的深度移动范围是固定的,但是距离越远,手心在图片上的移动范围会减小,为了解决以上问题,作为本发明的进一步改进,本发明还包括在b步骤中,当演讲者手心距离Kinect深度传感器中心的深度不是IOOOmm时,还包括将手心距离Kinect深度传感器中心不同深度位置的二维信息转换到手心距离Kinect深度传感器中心IOOOmm深度的二维信息的过程,可选择通过以下步骤实现:首先,采用最小二乘法确定手心在任意深度信息下的移动范围;然后,采用比例的方式根据深度数值把手心的二维数据转换到手心距离Kinect深度传感器中心IOOOmm深度的二维数据。 Depth palm movement range speaker is fixed, but the greater the distance, the moving range of the palm on the image will be reduced, in order to solve the above problems, the present invention As a further improvement, the present invention further comprises in step b, when the speech by palms of depth from the depth sensor Kinect center not IOOOmm, further comprising converting the two-dimensional information from the palm center Kinect depth sensor different depth positions of two-dimensional information to process the palm from the center of the depth sensor Kinect IOOOmm depth, by selectively implements the following: first, determine the range of movement of the palm at any depth information of the least squares method; then, a proportional way to convert from the palm center IOOOmm Kinect depth sensor according to the depth of two-dimensional data the two-dimensional depth value of the handle of the heart.

[0012] 标准区域内使用者站在Kinect的中心位置,控制局限性较大,为了使操作者能在垂直于Kinect左右的方向上有一定范围的移动,作为本发明的进一步改进,本发明还包括在于在b步骤中,当演讲者的手心偏离Kinect深度传感器中心位置时,还包括将偏离Kinect深度传感器中心位置的手心二维信息校正到中心位置的手心二维信息的过程,并可以通过以下步骤实现:首先,计算出校正前后挥动中心的距离差值;然后通过差值把实 The [0012] user stands on the center position of the standard region Kinect control a great limitation, to enable the operator to have a certain range in the direction perpendicular to the movement of the left and right Kinect, as a further development of the invention, the present invention further wherein in step b it comprises, when the palm of the speaker deviates from the center position Kinect depth sensor, further comprising a two-dimensional information will deviate from the palm sensor Kinect depth correction center position to the center position of the two-dimensional information palm process, and by the following implements the following: first, calculate the distance difference correcting wave front center; then, by difference, the real

际手心坐标校正到基准区域。 Inter palms coordinate correction to the reference area. 校正公式为 Correction formula is

Figure CN103279188AD00051

其中(Xmiddle,ymiddle) Wherein (Xmiddle, ymiddle)

为校正前的挥动中心,(Xci, yd为校正后的挥动中心,(Xbefra^ ybrfOTe)为转换前手心的位置,(Xnow.Ynow)为转换后的手心位置。 The center of swing before the correction, (Xci, yd of the center of the corrected wave, (Xbefra ^ ybrfOTe) is a palm position before conversion, (Xnow.Ynow) conversion of the palm position.

[0013] 为了达到确定不同操作者身份的效果,作为本发明的进一步改进,在通过手势指令控制PPT操作之前,还包括对于PPT操作者身份识别的过程,此步骤可以通过人脸识别的方式实现,人脸识别步骤包括人脸检测和人脸数据库的建立、确定操作者身份的过程。 [0013] In order to achieve a different identity of the operator to determine the effect, as a further improvement of the present invention, before the control operation by the gesture command PPT, further comprising a process for the identification of the operator PPT, this step may be achieved by way of face recognition , face recognition steps include the establishment of face detection and face database to determine the identity of the operator of the process. 所述的人脸数据库的建立可采用主成分分析法,把收集到的人脸图片转换成能代表其主要区别的特征脸集:首先通过获取每个像素的平均值,生成这些图片的平均人脸图片,然后将特征人脸和平均人脸比较,最后把所有训练图片的比率、平均人脸和特征人脸集存储作为数据库。 First, the average person by obtaining an average value for each pixel, to generate these images: the face database may be used to establish the principal component analysis, converting the collected image into face to represent the main difference is the set of facial feature pictures of faces and facial features and compares the average face, and finally all the training ratio picture, the average face and facial feature set is stored as a database. 所述的确定操作者身份的过程包括:首先载入数据库中平均人脸、特征脸集以及特征值矩阵,然后利用摄像头采集图像,对图像进行处理并做人脸检测,将输入的图片投影到主成分分析子空间,最后查找原始的训练图片,找出拥有最相似比率的图片,确定身份。 Determining the identity of the operator of the process comprising: a first loaded database average face, facial characteristics, and set the eigenvalue matrix, and then using the camera image acquisition, image processing and man face detection, the input image is projected onto the main subspace component analysis, the final look of the original training images to identify with the most similar ratio of the picture, to determine the identity. 所述的找出最拥有相似比率的图片的步骤采用欧几里得距离:首先计算输入图片与每张训练图片的欧氏距离,然后比较计算的距离值,距离越小表明越相似,并引入可信度公式: Most of the identified images with similar step using Euclidean distance ratio: first calculating the input image and each training image Euclidean distance, and then compares the calculated distance value, the more similar the smaller the distance, and the introduction of the credibility of the formula:

Figure CN103279188AD00052

,其中pConfidence 为识别结果的可信度,范围为O Wherein pConfidence recognition result confidence, ranging O

到I之间;nTrainFaces为数据库中参加训练的人脸的个数。 Between I; nTrainFaces the number of persons who participated in the training database in the face. 如果pConfidence > 0.5贝丨J认为识别成功,反之识别结果不可信。 If pConfidence> 0.5 J Pui Shu that the recognition is successful, otherwise the recognition result credible. [0014] 为了实现更为准确的确定操作者身份的目的,作为本发明的进一步改进,所述的确定操作者身份的过程采用多次采样分析的方法,即在一定时间内,统计单次识别可信度超过0.5的结果的次数,并只有当识别为同一人的次数达到输出门槛时,该识别结果才作为最终的识别结果,所述输出门槛是指识别为同一人的次数占全部识别次数的比例。 [0014] In order to achieve a more accurate determination of the identity of the operator of the object, as a further development of the invention, the process of determining the identity of the operator of multiple samples according to the method of analysis, i.e., within a certain time, a single statistical identification the results of more than 0.5 times the reliability of, and only when the number is identified as the same person when the threshold reaches the output, as a final recognition result before the recognition result, the output of the threshold refers to the total number of times the identification is recognized as representing the same person proportion.

四、附图说明 IV BRIEF DESCRIPTION

[0015] 图1是本发明的操作流程示意图。 [0015] FIG. 1 is a schematic view of the operation flow of the present invention.

[0016] 图2是Kinect外观图。 [0016] FIG. 2 is an external view of Kinect.

[0017] 图3是手心三维位置解释图。 [0017] FIG. 3 is a three-dimensional position palm explained in FIG.

[0018] 图4是手心在标准平面位置移动时在Kinect上的图像范围的区域划分图。 [0018] FIG. 4 is a palm in the standard position a plane view of an image area division on the Kinect range.

[0019] 图5是检测到手心的位置位于激光笔效果区域时的手心位置转换图。 [0019] FIG. 5 is a view of the palm position into a position of the detected palm region is located the laser pen effect.

[0020] 图6是距离Kinect深度传感器中心位置不同深度时手心移动范围图。 [0020] FIG. 6 is a view of the palm movement range from the center position of the depth sensor Kinect different depths.

[0021] 图7是将偏离Kinect深度传感器中心位置的手心二维信息校正到中心位置的手心二维信息的校正原理图。 [0021] FIG. 7 is a two-dimensional information departing from the palm of a depth sensor Kinect corrected to the center position of the center position of the correction schematic two-dimensional information of the palm.

[0022] 图8是人脸识别步骤的示意图。 [0022] FIG. 8 is a schematic diagram of the recognition step.

[0023] 图9是平均人脸图。 [0023] FIG. 9 is a face view of the average person.

[0024] 图10是特征人脸集。 [0024] FIG. 10 is a set of features of a face.

五、具体实施方式 V. DETAILED DESCRIPTION

[0025] 下面结合附图就本发明的实施做出详细的说明 [0025] DRAWINGS made in detail to embodiments of the present invention will be described

[0026] 图1为本发明的操作流程示意图。 [0026] FIG. 1 is a schematic view of the operation flow of the present invention. 如图1所示,首先将Kinect与计算机连接组成本发明的操作系统,该系统硬件部分为计算机和Kinect设备,软件部分主要是在windows环境下通过MFC编程实现,包括OpenCV的调用和MPC动态内存绘图技术,其中PPT文件可以提前存储于计算机,也可以通过云数据库从网络上自动下载。 1, the first connected to a computer operating system Kinect composition of the present invention, the computer system hardware and Kinect equipment, software part is mainly achieved by programming MFC in the windows environment, including calling OpenCV dynamic memory and MPC drawing technique, which may be advanced PPT files stored on a computer may be automatically downloaded from the network through the cloud database.

[0027] 图2为Kinect外观图。 [0027] FIG. 2 is an external view of Kinect. 如图2所示:左边镜头为红外线发射器,中间的镜头是一般常见的RGB彩色摄像机,右边镜头是红外线CMOS摄像头所构成的3D深度传感器。 Shown in Figure 2: Left Lens infrared emitter, the intermediate lens is general common RGB color camera, the right lens is a 3D depth sensor CMOS infrared cameras constituted. 传感器以每秒30帧的速度生成深度图像流,实时3D的再现周围环境。 Flow sensor generates a depth image at a rate of 30 frames per second, the real-time 3D playback surroundings. Kinect传感器可以同时获取图像RGB和深度图像数据,支持实时的全身和骨骼跟踪,并能识别一系列的动作。 Kinect sensor can simultaneously acquire images RGB and depth image data, real-time tracking of the whole body and bone, and can identify a series of actions.

[0028] 通过启动Kinect对手心的跟踪,应用程序通过OpenNI的API函数可以获得手心在整幅图片中的位置信息,以及手心的深度信息。 [0028] can get location information in the palm of the whole picture, and the palm of the depth information through the API function OpenNI by tracking, application launch Kinect heart of the opponent. 如图3所示:a,b代表手心位置在二维图像中的位置。 3: a, b in the position representative of palm two-dimensional image. c代表手心距离深度传感器的深度信息。 c represents a palm depth information from a depth sensor. 这是通过回调函数HancLUpdate O返回的三维数据。 This three-dimensional data is returned by the callback function HancLUpdate O.

[0029] 首先在深度信息为IOOOmm的二维平面中,本发明将整个区域分为两大部分:激光笔区域和控制区域。 [0029] First IOOOmm depth information in a two-dimensional plane, the whole area of ​​the present invention will be divided into two parts: a laser pointer region and the control region. 激光笔区域主要是利用手在空间中的位置,把位置投影到屏幕上产生类似激光笔效果辅助PPT讲解部分;控制区域则是根据手心的位置判断是否控制PPT前翻页或后翻页等操作。 Pen position of the hand area is the use in space, the position of the projection screen to produce a similar effect Pen PPT explain auxiliary portion; control region is the palm of determining whether the position control or the front page turning operations such as PPT . 根据上述两大区域的划分,作以下规定:标准距离是位于Kinect正前方I米处的距离;标准平面是位于标准距离处,手心移动的全部范围所在的平面;标准区域是手心在标准平面移动时在Kinect上的图像范围,规定为矩形区域;挥动中心是使用者在胸前伸出右手时,手心在整个挥动范围中所处的的位置,可在不同深度信息下;基准区域是手心移动范围在Kinect图像上的中心,即人在垂直于Kinect方向的中心位置所产生的图像上的挥手区域。 The division of these two areas, as specified below: standard distance is located in front of Kinect I m ​​from; standard plane is located at a standard distance, the full range of plane movement of the palm is located; standard region is a standard plane shift palms the image on the Kinect range, defined as a rectangular area; swing center position of the user when the chest is right hand, in which the palm of the entire swing range, the information may be at different depths; reference region is moved palm Kinect range centered on the image, i.e. the image of the person waving area at the central position perpendicular to the direction of the generated Kinect.

[0030] 将标准区域划分为如图4所示的区域。 [0030] The standard area divided into regions shown in FIG. 如图4所示:整幅图片分为位于中心的激光笔效果区域和四周的控制区域两部分。 4: the whole image is divided into two portions in the center region of the laser pointer effect and the control area around.

[0031] 如检测到手心的位置位于激光笔效果区域A,系统会根据手心在图中的位置,如图5所示,通过公式I把位置信息转换到电脑全屏时屏幕中的位置,并利用MFC图像编程技术 [0031] The position of the detected palm region located Pen effect A, the system according to the palm of the hand position in the figure, as shown in FIG. 5, the I by converting the position information into the equation the position of the screen when the computer is full screen, and using MFC programming technology image

在屏幕中实现红色光点移动类似于激光笔的效果。 Achieve red light spot moving on the screen similar to a laser pointer effect.

Figure CN103279188AD00071

[0032] 在控制区域,为了防止手心在两区域交界处产生误操作,在A区域四周划分出隔离区域,隔离区域效果为:如果手心位于隔离区域则不产生任何操作或激光笔效果。 [0032] In order to prevent generation palm region in the control region at the junction of the two erroneous, divided in the A zone isolation region around isolation region effect: if any action or effect laser pointer is located in the palm isolation region is produced. 除了在隔离区域,当手心的位置在最左侧“开始展示”位置时,在智能空间下,智能灯光系统和自动窗帘系统会自动·关灯关窗帘配合PPT开始展示;也可以将手心移至“结束展示”来关闭PPT ;当手心位于左上角或右上角时,系统会自动为PPT翻页;此外在讲解过程中,演讲者可将手心移动至上侧或下侧随时播放或关闭视频;演讲者还可以通过将手心移至左下角在会议讲解中途进行休息,或移至右下角停止休息继续讲解。 In addition to the isolation region, when the position of the palm of the left most "Display Start" position, in a smart space, automatic curtain intelligent lighting system and the system will automatically turn off the lights-off fitting curtain began to show PPT; palm can also be moved "end show" to close the PPT; when the palm is located in the upper left or upper right corner, the system will automatically flip PPT; in addition to explain the process, the speaker can be moved to the upper side of the palm of the hand or lower side or off at any time to play video; presentations who can also move to the lower left corner of the palm rest were in the middle of the meeting to explain, or explain the move to the lower right corner to continue the rest stop. 其中所有功能触发后都会有一定的休眠时间来防止重复引发操作。 After all the features which will trigger a certain time to prevent duplication of sleep caused by the operation. 最后PPT的COM接口控制是通过对MSDN中例程的修改实现。 Finally PPT COM interface is controlled by modifying the routines implemented on MSDN. 通过上述控制区域划分,在标准平面上,手势操作基本能满足PPT演讲者的需求。 By dividing the control area in the standard plane, a gesture operation to meet the basic needs of PPT speaker.

[0033] 在标准平面中,手心在图片中移动可以成功的控制PPT的正常播放以及产生激光笔效果。 [0033] In the standard plane, the palm movement in the picture can be successfully controlled and generation of normal-play PPT laser pointer effect. 但在实际应用中,演讲者的手不是始终都在标准平面内,所以对应在二维图片上的区域也会产生差异。 However, in practice, the speaker's hand is not always within the standard plane, corresponding to the two-dimensional picture of the area will also have differences. 演讲者手心的移动范围是固定的,但是距离越远,手心在图片上的移动范围会减小,如图6所示。 Palm movement range of the speaker is fixed, but the greater the distance, the moving range of the palm on the image will be reduced, as shown in FIG.

[0034] 为了解决上述问题,根据手心位置的深度信息,利用对测量数据的分析把手心在不同深度位置的二维信息通过最小二乘法统一转换到标准区域中。 [0034] In order to solve the above problems, according to the depth information of the position of the palm, the use of the two-dimensional information analysis center handle different depth positions of the measurement data converted by the least squares method to the standard unified area. 通过深度信息转换可以保证演讲者在任意深度做出手势与在标准区域中的控制效果相同。 Depth conversion information by the speaker can be guaranteed in any depth effects make the same gesture control in the standard region. 如表I所示,测量数据是由测试者在Kinect中心位置在不同深度挥动手所测量的边界数据,表中的值为测量数据的平均值。 As shown in Table I, the measured data is waving hand boundary data measured by the tester at the central position Kinect different depths, the average value of the measurement table data.

[0035] 表I不同深度信息下的手心移动数据 [0035] The palm movement data in Table I, different depth information

Figure CN103279188AD00072

[0037] 通过表中数据可观察到四组边界的数据与深度信息h近似成线性关系。 [0037] The data in the table may be observed through the boundary into four sets of data and depth information h is approximately linear. 本发明根据数据通过最小二乘法求出四组边界的最佳经验公式:四组线性经验公式表示 According to the present invention to obtain an optimum empirical data four boundary by the least square method: Linear empirical formula represented by four

为f(h) = a+bh。 To f (h) = a + bh. a, b 的解如公式2,3 所示: a, b, solutions of Equation 2 and 3:

Figure CN103279188AD00073
Figure CN103279188AD00081

:参一一⑶但必须指出只有当h和f(h)之间存在线性关系时,拟合的 : Reference eleven ⑶ It must be noted that only when there is a linear relationship between h and f (h), fitted

直线才有意义,为此引入一个参量:相关系数r,它定义为: Straight makes sense, for the introduction of a parameter: the correlation coefficient r, which is defined as:

Figure CN103279188AD00082

(4) ο当r的绝对值越接近I时,说明线性关系越好。 When (4) ο r when the absolute value closer to I, the better the linearity. 求出a, b带入公式f (h) = a+bh得出最佳经验公式。 Obtaining a, b into the equation f (h) = a + bh optimum empirical formula. 通过表I共计算出四组公式,计算得到的四组系数和相关系数如表2所示: Table I total is calculated by equation four, and four sets of correlation coefficients calculated as shown in Table 2:

[0038] 表2最小二乘法计算结果 The results [0038] Table 2 Least Squares

[0039] [0039]

Figure CN103279188AD00083

[0040] 通过相关系数可知,每一组数据都具有线性相关性。 [0040] understood by the correlation coefficient, each set of data having a linear correlation. 而手心移动范围的长宽可由 Length and width of the palm by movement range

最右侧减去最左侧以及最下侧减最上侧得出。 Subtracting the right of the leftmost and lowermost side Save uppermost side draw. 通过上表可得出手心的移动范围与深度h Table may be derived by the palm of the movement range of the depth h

Xif [f'«j: X = —α 100917Λ + 740.8714的关系:y/j-_: y = —0.093771A +574.5143 (幻通过上述公式可计算出,在深度为IOOOmm时 When y = -0.093771A +574.5143 (phantom calculated by the above equation, at a depth IOOOmm: Xif [f ' «j: the relationship X = -α 100917Λ + 740.8714 is: y / j-_

与实际测量值的误差小于2px,完全满足手势控制的需要。 And the actual measurement value error is less than 2px, fully meet the needs of the gesture control. 手心在任意深度信息下的移动范围已经确定,下面将采用比例的方式根据深度h把手心的二维数据转换到标准区域中, Palm movement at any depth range information has been determined, the following manner proportional to the converted data in the standard region in accordance with the two-dimensional depth h of the handle heart,

转换公式6如下所不 Conversion formula 6 below are not

Figure CN103279188AD00084

愈 其中XMW,yn„为手心的真头位直, The more which XMW, yn "true head position straight palm,

xfinal, Yfinal为转换后对应到640*480的标准区域的坐标。 xfinal, Yfinal is converted to the corresponding coordinates of the standard area of ​​640 * 480. 与上节相同,通过对转换以后的坐标进行分析,就可以完成手心对PPT的控制。 The same upper section, by the subsequent conversion of the coordinate analysis can be completed in the palm of the hand control of the PPT.

[0041 ] 上述结果是演讲者站在Kinect的中心位置所得出,控制局限性较大,为了使用户能在垂直于Kinect左右的方向上有一定范围的移动,本实施方式对公式6中参数做以下修正。 [0041] The above results speaker stood derived Kinect center position, a great limitation control, to enable a user to have a certain range in the direction perpendicular to the movement of the left and right Kinect, embodiments of the present parameters do Formula 6 the following amendments. 由于演讲者的挥动中心可能会变化,所以首先计算出校正前后挥动中心的距离差值,再通过差值把实际手心坐标校正到基准区域。 Since the swing center speaker may vary, so the first correction is calculated from the difference before and after the center of the swing, and then by the difference between the actual correction to the reference coordinates palm area. 校正前的挥动中心可通过规定演讲者开始使用时举起右手后追踪到的稳定位置来获得,记为(Xffliddle, Yffliddle)。 After he raised his right hand while waving before correction center can be started by specifying the track to the speaker's stable position is obtained, denoted by (Xffliddle, Yffliddle). 而校正后的中心可通过深度信息获取,因为校正后的挥动中心就是基准区域中手心移动范围的中心,记为: And the center of the corrected depth information may be obtained by, as the corrected reference swing center is the center of the palm region of the range of movement, referred to as:

[0042] [0042]

Figure CN103279188AD00085

[0044] 通过两个中心计算出差值,可进行校正,校正原理图如图7所示。 [0044] The difference value is calculated by the two centers, can be corrected, the correction diagram shown in Figure 7.

[0045] 通过上述坐标表示,可以通过以下校正公式求得(xMW,yn(J: [0045] represented by the coordinate, the correction can be obtained by the following equation (xMW, yn (J:

Figure CN103279188AD00086

其中(before, Ybefore)为转换前手心的位置,(xnow, ynow)为转换后的手心位置,经过转换后,再利用公式6就可实现在即使人相对中心位置有移动也能实现同样效果的目的。 Wherein (before, Ybefore) for the location of the palm before conversion, (xnow, ynow) is a palm position after conversion, after the conversion, then there can be achieved using Equation 6 can be moved to achieve the same effect even if the center position of the person relative purpose.

[0047] 为了实现对于不同演讲者身份的识别,在演讲者使用手势控制PPT之前,还可以包括对演讲者人脸识别的过程。 [0047] In order to achieve recognition for different speaker identities, prior to using the gesture control speaker PPT, may further include a process of speaker recognition. 系统利用Kinect摄像头通过主成分分析的方法对演讲人进行人脸识别,确定其姓名,自动为其打开与之相对应的Power point文稿。 Kinect camera system using face recognition on the speaker by the method of principal component analysis, to determine their names, for automatically opening corresponding thereto Power point document.

[0048] 人脸识别技术主要分为以下三部分:(I)建立人脸的数据库;(2)对人脸进行实时检测;(3)用当前获得的脸部数据与数据库进行比对,得到识别结果。 [0048] The face recognition technology is divided into the following three parts: (I) establishing face database; (2) real-time detection of the face; (3) facial data for comparison with a database currently obtained, to give recognition results.

[0049] 人脸检测是人脸数据库和人脸对比识别的前提,本发明采用OpenCV提供的分类器(haarcascade_frontalface_alt.xml)来进行人脸检测。 [0049] Face detection is a face comparison and face recognition database premise, the present invention provides uses OpenCV classifier (haarcascade_frontalface_alt.xml) to perform face detection. 为了增大数据的存储量和提高计算效率,全部采用灰度图像计算。 In order to increase the amount of data storage and computational efficiency, all using the calculated grayscale images. 首先通过Kinect的摄像头获得图像,然后把图像转换为灰度图像,并以此灰度图像作为输入并利用OpenCV分类器和函数cvHaarDetectObjectsO进行人脸识别。 First, the image obtained by the camera Kinect, then the image is converted to grayscale images, and as a grayscale image as an input and using OpenCV cvHaarDetectObjectsO classifiers and face recognition function. 如果得到人脸区域则继续对此数据进行处理,反之则重新获得图像再次进行检测。 If you get face region continue to process this data, and vice versa regain the image again detected. 人脸检测的流程如图8所示。 Face detection process shown in Figure 8.

[0050] 得到人脸区域后,根据采集的人脸建立数据库。 [0050] After the obtained face region, in accordance with established database collected from a human face. 首先对人脸图像调整成固定的维度,然后应用直方图均衡化来实现固定的亮度和对比度。 First, the face image is adjusted to a fixed dimension, histogram equalization is then applied to achieve a fixed brightness and contrast. 最后得到经过预处理的人脸图片。 Finally get through the pre-treatment of facial images. 例如,为了建立数据库,共对四人进行人脸采集,每人采集20张图片并对不同人的不同照片通过重命名做好标记。 For example, to establish a database, a total of four for face capture, capture 20 images per person and different photos of different people and marked by renaming. 在此基础上采用主成分分析的方法建立脸部数据库。 On the basis of the principal component analysis method for establishing facial database.

[0051] 主成分分析(即特征脸)的方法主要思路是:把收集的80张训练图片转换成能代表这些训练图片主要区别的“特征脸”集(eigenface)。 The main idea of ​​the method [0051] Principal component analysis (ie, features face) are: to convert collected 80 training images to represent the main difference of these training picture "feature face" set (eigenface). 首先通过获取每个像素的平均值,生成这些图片的“平均人脸图片”(averageface)。 First, by taking the average of each pixel to produce these pictures "average face picture" (averageface). 然后将特征脸与“平均人脸”比较。 Then compare the features of the face and "average face." 第一个特征脸是最主要的脸部区别,第二个特征脸是第二重要的脸部区别,以此类推……计算出了训练集中图片的主要区别,并且用这些“区别”的组合来代表每幅训练图片。 The first feature of the face is the most important difference between the face, the second face is the second most important characteristic facial differences, and so on ...... calculate the main differences in the training set of pictures, and these "differences" in combination to represent each piece of training images. 一张训练图片可能是如下的组成:(averageface) + (12.l%of eigenfaceO) - (25.5%ofeigenfacel) + (8.0%of eigenface2) +...+ (0.0%of eigenface79)。 A training picture may be composed as follows: (averageface) + (12.l% of eigenfaceO) - (25.5% ofeigenfacel) + (8.0% of eigenface2) + ... + (0.0% of eigenface79). 该训练图片所占的比率可表示为{12.1, -25.5,8.0,……,0.0}。 The occupied ratio of the training image may be expressed as {12.1, -25.5,8.0, ......, 0.0}. 由于排在后面的一些特征脸是图像噪声或者不会对图片有太大作用,因此这个比率表可以被降维到只剩下最主要的部分,只取前30个。 Since some of the features at the back face of the image or noise will not have much effect on the image, so that dimension reduction ratio table may be the most important part to the left, only the leading 30. 最后把所有训练图片的比率、平均人脸和特征脸集存储作为数据库。 Finally, the ratio of all the training images, the average face and facial feature set is stored as a database. 平均人脸和特征脸集如图9和图10所示。 Average face and the face feature sets 9 and 10 shown in FIG.

[0052] 在人脸识别部分,首先载入数据库中平均人脸、特征脸集以及特征值矩阵。 [0052] In the recognition section, the database is first loaded average face, facial characteristics, and set the eigenvalue matrix. 然后利用摄像头采集图像,对图像进行处理并做人脸检测。 Then using the camera image acquisition, image processing and face detection man. 当检测到人脸区域后,根据上文对人脸图像进行预处理。 Upon detecting the face region, the face image according to the above for preprocessing. 使用OpenCV的cvEigenDecomposite O函数,将输入的图片投影到主成分分析子空间,以得到此照片的特征值。 Using the OpenCV cvEigenDecomposite O function, the input image is projected onto the subspace of principal component analysis, to obtain a characteristic value of this photo. 最后查找原始的训练图片,找出拥有最相似比率的图片,确定身份。 Finally, find the original training images to identify with the most similar ratio of the picture, to determine the identity.

[0053] 在查询拥有最相似比率的训练图片时,采用“欧几里得距离”。 [0053] When the query has the most similar to the ratio of train pictures, the "Euclidean distance." 首先计算输入图片与每张训练图片的欧氏距离,然后比较计算的距离值,距离越小表明越相似。 First calculate the input images and pictures of each training Euclidean distance, and then compares the calculated distance value, the smaller the distance, the more similar. 距离计算公式如式8所示。 Distance is calculated as shown in Equation 8. .V=IiEigens-1 .V = IiEigens-1

[0054] [0054]

Figure CN103279188AD00091

其中TestFace 为被检测图像的比率,TrainFacej为第j个训练图像的比率。 Wherein TestFace as being the ratio of the detected image, TrainFacej ratio of the j-th training image. nEigens为特征脸的个数。 nEigens is the number of features of the face. 通过上述计算结果,可找到距离最小的图片,但不能认定该图片与被检测的是一个人。 By the above calculation result, a minimum distance images can be found, but the picture can not be identified to be detected is a person. 为了表示识别结果的可信度,引入可信度计算公式9:pConfidence = 1.0- In order to express the reliability of the recognition result, the reliability is calculated introducing 9: pConfidence = 1.0-

[0055] (9)其中pConfidence 为払、力Ij [0055] (9) where is a partial payment pConfidence force Ij

结果的可信度,范围为O到I之间;nTrainFaces为数据库中参加训练的人脸的个数。 The credibility of the results, ranging between O to I; nTrainFaces number of participants for the training database of faces. 如果pConfidence > 0.5则认为识别成功,反之识别结果不可信。 If pConfidence> 0.5 is considered successful identification, whereas the recognition result credible.

[0056] 为了进一步提高识别结果的准确度,消除识别过程中的因某些偶然因素对结果造成的干扰,系统采用多次采样分析的方法,即在一定时间内,统计单次识别可信度超过0.5的结果的次数,并只有当识别为同一人的次数达到输出门槛(即识别为同一人的次数占全部识别次数的比例)时,该识别结果才作为最终的识别结果。 [0056] To further improve the recognition accuracy of the results, to eliminate the interference due to some accidental factors resulting in the identification process, the system uses a multiple sample analysis, i.e., within a certain time, to identify a single statistical confidence the results of more than 0.5 times, and only when the identification number of the same person as the output reaches a threshold (i.e., identified as the same person identification number of the total number ratio) when only the recognition result as a final recognition result.

[0057] 根据以上的识别结果,系统会找到相应的PPT文件并为其打开准备播放,其中PPT文件可以是提前存储的也可以通过云数据库从网络上自动下载。 [0057] According to the above recognition result, the system will find the PPT file and open ready for play, which PPT files can be stored in advance can be automatically downloaded from the Internet via cloud database. 文件打开后,系统启动Kinect对人的手部进行动态捕捉,最后根据动态数据确定相应动作来控制PPT。 After the file system startup Kinect human hand for motion capture, and finally determine the appropriate action to control the dynamic data PPT.

Claims (10)

1.一种基于Kinect无接触性操作控制PPT的方法,其特征在于包括以下步骤: a、将Kinect接入计算机系统; b、通过启动Kinect对手心的跟踪,获得手心的位置信息,以及手心的深度信息,在手心距离Kinect深度传感器中心IOOOmm深度的位置,对手心所在的二维平面区域进行功能区域的划分,将其划分为激光笔区域、隔离区域、控制区域,并定义手心在各区域中不同位置相对应的PPT操作的指令,并作以下规定:标准距离是位于Kinect正前方I米处的距离;标准平面是位于标准距离处,手心移动的全部范围所在的平面;标准区域是手心在标准平面移动时在Kinect上的图像范围,并将其规定为矩形区域;挥动中心是使用者在胸前伸出右手时,手心在整个挥动范围中所处的的位置,可在不同深度信息下;基准区域是手心移动范围在Kinect图像上的中心,即人在垂直于Kinect方向的 1. A method for controlling Kinect PPT-based non-contact operation, characterized by comprising the steps of: a, Kinect access the computer system; B, by actuating the tracking Kinect opponents heart, obtain location information of the palm, and palm depth information from the palm of the hand in the center of the depth sensor Kinect IOOOmm depth position, where the two-dimensional plane region of the opponent's heart function divided areas, be divided into laser pointer region, the isolation region, the control region, and each region is defined in the palms different positions corresponding instruction PPT operation, with the following provisions: standard distance is located Kinect front of I m from; standard plane is located at a standard distance, a plane full scope palm movement is located; standard region is palms standard image plane is moved in the range of Kinect, and is defined as a rectangular area; swing center position of the user when the right hand, in which the palm of the entire swing range in the chest, the depth information may be different ; reference region is centered on the palm of the hand moving range Kinect image, i.e., a direction perpendicular to the person's Kinect 心位置所产生的图像上的挥手区域; C、计算机根据手势控制指令对PPT执行相应操作。 Waving position of the center region on the image produced; C, the control computer instructions to perform a corresponding operation according to a gesture PPT.
2.根据权利要求1所述的基于Kinect无接触性操作控制PPT的方法,其特征在于在b步骤中,当演讲者手心距离Kinect深度传感器中心的深度不是IOOOmm时,还包括将手心距离Kinect深度传感器中心不同深度位置的二维信息转换到手心距离Kinect深度传感器中心IOOOmm深度的二维信息的过程。 2. The method Kinect non-contacting operation control based on PPT, characterized in that said step b 1, the depth of the palm when the speaker is not IOOOmm Kinect depth sensor from the center of the palm further comprising depth distance claim Kinect two-dimensional information of different depths from the center of the sensor switch to palm Kinect process two-dimensional depth information, a depth sensor center IOOOmm.
3.根据权利要求2所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的将手心在不同深度位置的二维信息转换到手心距离Kinect中心IOOOmm深度的二维信息的过程可以通过以下步骤实现:首先,采用最小二乘法确定手心在任意深度信息下的移动范围;然后,采用比例的方式根据深度数值把手心的二维数据转换到手心距离Kinect中心IOOOmm深度的二维数据。 3. The method of controlling Kinect PPT-based non-contacting operation according to claim 2, wherein said transition to the palm from the palm center Kinect process IOOOmm two-dimensional information in a two-dimensional depth information of different depth positions can be achieved by the following steps: first, determine the moving range of the palm of the hand in the depth information of any least squares method; then, a proportional way to convert two-dimensional data palm IOOOmm depth data from the two-dimensional center Kinect depth value of the center of the handle .
4.根据权利要求1一3任意一项所述的基于Kinect无接触性操作控制PPT的方法,其特征在于在b步骤中,当演讲者的手心偏离Kinect深度传感器中心位置时,还包括将偏离Kinect深度传感器中心位置的手心二维信息校正到中心位置的手心二维信息的过程。 1 according to claim 3 a method of non-contacting operation Kinect PPT-based control, wherein in step b, when the palm of the speaker deviates from the center position Kinect depth sensor, further comprising offset from any one of palm two-dimensional information Kinect depth sensor is corrected to the center position of the palm of the process of two-dimensional information of the center position.
5.根据权利要求4所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的将偏离Kinect深度传感器中心位置的手心二维信息校正到中心位置的手心二维信息的过程可以通过以下步骤实现:首先,计算出校正前后挥动中心的距离差值;然后Xamr =Xhelan +(Xa通过差值把实际手心坐标校正到基准区域,校正公式为P —v ,, J其中J 9(Xmiddle, Ymiddle )为校正前的挥动中心,(X0, Y0)为校正后的挥动中心,(Xfaefore, Ybefore)为转换前手心的位置,(xnOT,yMW)为转换后的手心位置。 5. The method of non-contacting operation Kinect PPT-based control, wherein said two-dimensional information will deviate from the palm sensor Kinect depth correction center position to the center position of the two-dimensional information of the process can palm claim 4 achieved by the following steps: first, calculate the distance difference correcting wave front center; then Xamr = Xhelan + (Xa by the difference between the actual coordinates of the palm region to the reference correction, the correction formula is P -v ,, J wherein J 9 ( Xmiddle, Ymiddle) as the center of swing before the correction, (X0, Y0) of the center of the corrected wave, (Xfaefore, Ybefore) to position the palm before conversion, (xnOT, yMW) is a palm position after conversion.
6.根据权利要求1、2、4、5中任意一项所述的基于Kinect无接触性操作控制PPT的方法,其特征在于通过手势指令控制PPT操作之前,还包括对于PPT操作者身份识别的过程,并通过人脸识别来实现,人脸识别步骤包括人脸检测和人脸数据库的建立、确定演讲者身份的过程。 The PPT-based method for controlling non-contacting operation 1,2,4,5 Kinect in any one of the preceding claims, characterized in that before the control operation by the gesture command PPT, further comprising an operator identification to the PPT process and is achieved by face recognition, face recognition steps include the establishment of face detection and face database, the process of determining the identity of the speaker.
7.根据权利要求6所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的人脸数据库的建立采用主成分分析法,把收集到的人脸图片转换成能代表其主要区别的特征脸集:首先通过获取每个像素的平均值,生成这些图片的平均人脸图片,然后将特征人脸和平均人脸比较,最后把所有训练图片的比率、平均人脸和特征人脸集存储作为数据库。 According to claim Kinect based contactless method of controlling operation of the PPT, wherein establishing said face database using principal component analysis, the collected image of the human face can be converted into its main representative of claim 6 distinguishing feature face set: first, by taking the average of the average person for each pixel, the images generated pictures of faces and facial features then the average face comparison, the ratio of the last images of all the training, the average characteristics of the human face and face set stored as a database.
8.根据权利要求6所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的确定操作者身份的过程包括:首先载入数据库中平均人脸、特征脸集以及特征值矩阵,然后利用摄像头采集图像,对图像进行处理并做人脸检测,将输入的图片投影到主成分分析子空间,最后查找原始的训练图片,找出拥有最相似比率的图片,确定身份。 A method according to claim Kinect non-contacting operation control based on PPT, characterized in that the operation of determining the identity of the process of claim 6 comprising: first loaded database average face, the face feature value and the feature set matrix and then use the camera image acquisition, image processing and face detection man, will enter the picture projected onto the subspace principal component analysis, the final look of the original training images to identify with the most similar ratio of the picture, to determine the identity.
9.根据权利要求8所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的找出最拥有相似比率的图片的步骤采用欧几里得距离:首先计算输入图片与每张训练图片的欧氏距离,然后比较计算的距离值,距离越小表明越相似,并引入可信度公式: A method according to claim Kinect PPT non-contacting operation based control, wherein said image to identify the most have a similar ratio of the step using Euclidean distance of 8: first calculating each input picture and training pictures Euclidean distance, and then compares the calculated distance value, the smaller the distance, the more similar, and the introduction of the credibility of the formula:
Figure CN103279188AC00031
为识别结果的可信度,范围为O到I之间;nTrainFaces为数据库中参加训练的人脸的个数。 Reliability for the recognition result, the range is between O to I; nTrainFaces number of people participating in the training database face. 如果pConfidence > 0.5,则认为识别成功,反之识别结果不可信。 If pConfidence> 0.5, is considered successful identification, whereas the recognition result credible.
10.根据权利要求9所述的基于Kinect无接触性操作控制PPT的方法,其特征在于所述的确定操作者身份的过程采用多次采样分析的方法,即在一定时间内,统计单次识别可信度超过0.5的结果的次数,并只有当识别为同一人的次数达到输出门槛时,该识别结果才作为最终的识别结果,所述输出门槛即识别为同一人的次数占全部识别次数的比例。 10. The method of Kinect process based non-contact operation control of the PPT according to claim 9, wherein the operation of determining the identity of multiple samples using the method of analysis, i.e., within a certain time, a single statistical identification 0.5 times more than the reliability of the results, and only when the number is identified as the same person reaches the output threshold, the recognition result before the recognition result as a final, i.e., the number of the output threshold is identified as the same person to identify the total number of proportion.
CN2013102071721A 2013-05-29 2013-05-29 Method for operating and controlling PPT in non-contact mode based on Kinect CN103279188A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2013102071721A CN103279188A (en) 2013-05-29 2013-05-29 Method for operating and controlling PPT in non-contact mode based on Kinect

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2013102071721A CN103279188A (en) 2013-05-29 2013-05-29 Method for operating and controlling PPT in non-contact mode based on Kinect

Publications (1)

Publication Number Publication Date
CN103279188A true CN103279188A (en) 2013-09-04

Family

ID=49061747

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2013102071721A CN103279188A (en) 2013-05-29 2013-05-29 Method for operating and controlling PPT in non-contact mode based on Kinect

Country Status (1)

Country Link
CN (1) CN103279188A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104714649A (en) * 2015-03-31 2015-06-17 王子强 Kinect-based naked-eye 3D UI interaction method
CN104750253A (en) * 2015-03-11 2015-07-01 苏州佳世达电通有限公司 Electronic device for motion sensing input conducted by user
CN105204609A (en) * 2014-06-17 2015-12-30 立普思股份有限公司 Depth camera system
CN105975072A (en) * 2016-04-29 2016-09-28 乐视控股(北京)有限公司 Method, device and system for identifying gesture movement
CN106557172A (en) * 2016-11-25 2017-04-05 上海第二工业大学 A kind of keyboard with data glove and using method
CN106845405A (en) * 2017-01-20 2017-06-13 武汉仟品网络科技有限公司 A kind of method, device and electronic equipment that identity is recognized by Biological imaging
CN107015651A (en) * 2017-04-05 2017-08-04 上海心勍信息科技有限公司 A kind of dining interaction systems based on recognition of face
CN108089715A (en) * 2018-01-19 2018-05-29 赵然 A kind of demonstration auxiliary system based on depth camera

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101689244A (en) * 2007-05-04 2010-03-31 格斯图尔泰克股份有限公司 Camera-based user input for compact devices
CN101810003A (en) * 2007-07-27 2010-08-18 格斯图尔泰克股份有限公司 enhanced camera-based input
CN102129292A (en) * 2010-01-15 2011-07-20 微软公司 Recognizing user intent in motion capture system
CN102323859A (en) * 2011-09-08 2012-01-18 昆山市工业技术研究院有限责任公司 Lecture note playing system and method based on gesture control

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101689244A (en) * 2007-05-04 2010-03-31 格斯图尔泰克股份有限公司 Camera-based user input for compact devices
CN101810003A (en) * 2007-07-27 2010-08-18 格斯图尔泰克股份有限公司 enhanced camera-based input
CN102129292A (en) * 2010-01-15 2011-07-20 微软公司 Recognizing user intent in motion capture system
CN102323859A (en) * 2011-09-08 2012-01-18 昆山市工业技术研究院有限责任公司 Lecture note playing system and method based on gesture control

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105204609B (en) * 2014-06-17 2018-01-26 立普思股份有限公司 Depth camera chain
CN105204609A (en) * 2014-06-17 2015-12-30 立普思股份有限公司 Depth camera system
CN104750253B (en) * 2015-03-11 2018-10-12 苏州佳世达电通有限公司 A kind of electronic device carrying out body-sensing input for user
CN104750253A (en) * 2015-03-11 2015-07-01 苏州佳世达电通有限公司 Electronic device for motion sensing input conducted by user
CN104714649A (en) * 2015-03-31 2015-06-17 王子强 Kinect-based naked-eye 3D UI interaction method
CN105975072A (en) * 2016-04-29 2016-09-28 乐视控股(北京)有限公司 Method, device and system for identifying gesture movement
CN106557172A (en) * 2016-11-25 2017-04-05 上海第二工业大学 A kind of keyboard with data glove and using method
CN106845405A (en) * 2017-01-20 2017-06-13 武汉仟品网络科技有限公司 A kind of method, device and electronic equipment that identity is recognized by Biological imaging
CN107015651A (en) * 2017-04-05 2017-08-04 上海心勍信息科技有限公司 A kind of dining interaction systems based on recognition of face
CN108089715A (en) * 2018-01-19 2018-05-29 赵然 A kind of demonstration auxiliary system based on depth camera

Similar Documents

Publication Publication Date Title
CN102591459B (en) Posture state classification
CN102656542B (en) Camera navigation for presentations
Zollhöfer et al. Automatic reconstruction of personalized avatars from 3D face scans
US8593402B2 (en) Spatial-input-based cursor projection systems and methods
US9667860B2 (en) Photo composition and position guidance in a camera or augmented reality system
JP5567206B2 (en) Computing device interface
JP2015222591A (en) Human-computer interaction system, method for point positioning hand and hand instruction, and finger gesture determination method
CN101339607B (en) Human face recognition method and system, human face recognition model training method and system
US9129155B2 (en) Systems and methods for initializing motion tracking of human hands using template matching within bounded regions determined using a depth map
CN102662476B (en) Gaze estimation method
US20140211991A1 (en) Systems and methods for initializing motion tracking of human hands
CN104159032B (en) A kind of real-time adjustment camera is taken pictures the method and device of U.S. face effect
CN102830797B (en) A human-machine interaction method and system for determination based on the line of sight
CN101593022B (en) Method for quick-speed human-computer interaction based on finger tip tracking
US9697414B2 (en) User authentication through image analysis
US9224037B2 (en) Apparatus and method for controlling presentation of information toward human object
CN101697199B (en) Detection method of head-face gesture and disabled assisting system using same to manipulate computer
CN103914152B (en) Multi-point touch and the recognition methods and system that catch gesture motion in three dimensions
TWI454968B (en) Three-dimensional interactive device and operation method thereof
Li et al. Real time hand gesture recognition using a range camera
CN103353935B (en) A kind of 3D dynamic gesture identification method for intelligent domestic system
JP2001282456A (en) Man-machine interface system
CN101874234A (en) User interface device, user interface method, and recording medium
CN102982557A (en) Method for processing space hand signal gesture command based on depth camera
CN102426480A (en) Man-machine interactive system and real-time gesture tracking processing method for same

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)