CN104679246A - Wearable type equipment based on interactive interface human hand roaming control and interactive interface human hand roaming control method - Google Patents

Wearable type equipment based on interactive interface human hand roaming control and interactive interface human hand roaming control method Download PDF

Info

Publication number
CN104679246A
CN104679246A CN201510076982.7A CN201510076982A CN104679246A CN 104679246 A CN104679246 A CN 104679246A CN 201510076982 A CN201510076982 A CN 201510076982A CN 104679246 A CN104679246 A CN 104679246A
Authority
CN
China
Prior art keywords
action
roaming
hand
interactive interface
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510076982.7A
Other languages
Chinese (zh)
Other versions
CN104679246B (en
Inventor
徐向民
许望
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Bo Wei Intelligent Technology Co Ltd
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201510076982.7A priority Critical patent/CN104679246B/en
Publication of CN104679246A publication Critical patent/CN104679246A/en
Application granted granted Critical
Publication of CN104679246B publication Critical patent/CN104679246B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • User Interface Of Digital Computer (AREA)

Abstract

本发明公开了一种交互界面中人手漫游控制的穿戴式设备及控制方法,可穿戴设备呈环状,使用时需佩戴在用户手腕处,该设备具体包含人手运动信息获取模块、数据处理模块、控制权调配模块和近距无线收发模块。一种交互界面人手漫游控制的方法为,设备与主机建立无线通讯,用户通过抬手和连续两次或以上快速转动手腕动作或左右晃动手腕动作来获取设备控制权,系统实时采集人手运动信息并在交互界面上显示为模拟光标的移动,用户通过弹指或单指点击或多指点击动作选中交互界面上所需功能图标。本发明提供的设备为低成本低功耗可穿戴设备,且本发明提供的方法能提供给用户舒适自然且操作精确的人机交互体验。

The invention discloses a wearable device and a control method for hand roaming control in an interactive interface. The wearable device is ring-shaped and needs to be worn on the user's wrist when in use. The device specifically includes a human hand movement information acquisition module, a data processing module, The control rights allocation module and the short-range wireless transceiver module. A method for hand roaming control on an interactive interface is that the device establishes wireless communication with the host, and the user obtains the control right of the device by raising the hand and rotating the wrist two or more times in a row or shaking the wrist left and right. It is displayed on the interactive interface as simulating the movement of the cursor, and the user selects the desired function icon on the interactive interface by flicking or single-finger clicking or multi-finger clicking. The device provided by the present invention is a wearable device with low cost and low power consumption, and the method provided by the present invention can provide users with a comfortable, natural and precise man-machine interaction experience.

Description

一种交互界面中人手漫游控制的穿戴式设备及控制方法A wearable device and control method for human hand roaming control in an interactive interface

技术领域technical field

本发明涉及人机交互的研究领域,特别涉及一种交互界面中人手漫游控制的穿戴式设备及控制方法。The invention relates to the research field of human-computer interaction, in particular to a wearable device and a control method for manual roaming control in an interactive interface.

背景技术Background technique

随着计算机技术的快速发展,人机交互技术也得到了一定的发展。传统的人机交互大多是基于鼠标键盘的交互方式,为使人们在操作计算机时有更好的体验,鼠标也经历了许多革命性的发展。从机械鼠标到光电鼠标,再到如今的无线鼠标,都显示着人们对鼠标的重视,即人们对人机交互体验的重视。然而,上述鼠标在操作时都需要一个支撑平面来使鼠标完成精确的操作,但这也暴露出了一些不方便,人们总希望能不受约束的与计算机设备进行交互操作。空中鼠标,可以实现对鼠标的精确控制而不需要放在任何平面上,可以实现点击甚至文字输入操作,但它像传统鼠标一样无法释放双手来操作屏幕。With the rapid development of computer technology, human-computer interaction technology has also been developed to a certain extent. Most of the traditional human-computer interaction is based on the interaction mode of mouse and keyboard. In order to make people have a better experience when operating computers, the mouse has also undergone many revolutionary developments. From mechanical mouse to optical mouse, to today's wireless mouse, it shows that people attach importance to the mouse, that is, people attach importance to the human-computer interaction experience. However, the above-mentioned mice all need a support plane to enable the mouse to complete precise operations during operation, but this also exposes some inconveniences, and people always hope to perform interactive operations with computer equipment without restraint. The air mouse can realize precise control of the mouse without placing it on any flat surface, and can realize click and even text input operations, but it cannot release both hands to operate the screen like a traditional mouse.

人们希望有一种释放双手的操作方式,于是基于视觉的手势控制能完全释放双手,可以实现鼠标漫游和点击功能,但大屏幕下基于视觉的手势交互不够稳定,不能精确控制鼠标,并且在家居环境下的摄像头也存在侵犯隐私的风险,难以普及。手指鼠标因生活中有许多不适宜佩戴指环的活动,例如洗手、洗菜等,导致用户不能长时间连续佩戴指环,因此用户体验不佳;,最新的臂环基于肌肉感应手势则成本高技术复杂,佩戴不适。如今的手环基本都已具备健康监测,管理手机等功能,人们也越来越接受佩戴手环这一可穿戴设备,同时,手环基本都配备有加速度传感器、陀螺仪,并且成本较低,用手环来控制鼠标是我们提出的新技术,将手势的动作映射为交互界面上模拟光标的移动,可以实现稳定精确操作交互界面,具有操作精确、释放双手、成本低、技术成熟、功耗小等优势,能够满足用户需求。People hope to have an operation method that releases both hands, so vision-based gesture control can completely release hands and realize mouse roaming and clicking functions. The camera under the camera also has the risk of violating privacy and is difficult to popularize. Finger mouse has many activities that are not suitable for wearing rings in life, such as washing hands, washing vegetables, etc., so users cannot wear rings continuously for a long time, so the user experience is not good; the latest armbands based on muscle-sensing gestures are expensive and technically complicated , uncomfortable to wear. Today's wristbands basically have functions such as health monitoring and mobile phone management, and people are increasingly accepting wearing wristbands as a wearable device. At the same time, wristbands are basically equipped with acceleration sensors and gyroscopes, and the cost is relatively low. Using a wristband to control the mouse is a new technology proposed by us. It maps gesture movements to the movement of the simulated cursor on the interactive interface, which can realize stable and precise operation of the interactive interface. It has the advantages of precise operation, free hands, low cost, mature technology, low power consumption Small and other advantages, to meet user needs.

发明内容Contents of the invention

本发明的主要目的在于克服现有技术的缺点与不足,提供一种操作精确、能解放人的双手且成本低的交互界面中人手漫游控制的穿戴式设备。The main purpose of the present invention is to overcome the disadvantages and deficiencies of the prior art, and provide a wearable device with precise operation, freedom of hands and low cost for manual roaming control in an interactive interface.

本发明的另一目的在于,提供一种上述交互界面中人手漫游控制的穿戴式设备的控制方法。Another object of the present invention is to provide a control method of a wearable device controlled by human hand roaming in the above-mentioned interactive interface.

为了达到上述第一目的,本发明采用以下技术方案:In order to achieve the above-mentioned first purpose, the present invention adopts the following technical solutions:

一种交互界面中人手漫游控制的穿戴式设备,包括设备本体与设于设备本体内的电路模块,设备为环状,使用时佩戴在用户手腕处;A wearable device controlled by human hand roaming in an interactive interface, including a device body and a circuit module installed in the device body, the device is ring-shaped, and is worn on the user's wrist when in use;

所述电路模块包括人手运动信息获取模块、数据处理模块、控制权调配模块、近距无线收发模块,其中:The circuit module includes a manual movement information acquisition module, a data processing module, a control right allocation module, and a short-range wireless transceiver module, wherein:

人手运动信息获取模块:用于获取手势漫游时人手在空间坐标系中的运动信息,并发送至数据处理模块;Human hand movement information acquisition module: used to acquire the movement information of the human hand in the spatial coordinate system during gesture roaming, and send it to the data processing module;

数据处理模块:用于将所述运动信息进行处理分析,并分类识别其漫游手势动作,将其转化为控制信号并发送至主机;Data processing module: used to process and analyze the motion information, classify and recognize its roaming gestures, convert them into control signals and send them to the host;

控制权调配模块:根据人手运动情况调配控制权,触发或停止交互界面漫游操作功能;Control rights allocation module: allocate control rights according to the movement of human hands, trigger or stop the roaming operation function of the interactive interface;

近距无线收发模块:用于与主机连接,建立无线通讯,并在建立通讯的基础上将控制信息发送至主机。Short-distance wireless transceiver module: used to connect with the host, establish wireless communication, and send control information to the host on the basis of establishing communication.

优选的,所述人手运动信息获取模块包括:Preferably, the human hand movement information acquisition module includes:

三轴加速度传感器,用于获取手势漫游时人手在空间坐标系中的线性平移加速度A=(ax,ay,az);The three-axis acceleration sensor is used to obtain the linear translation acceleration A=(a x , a y , a z ) of the human hand in the space coordinate system when the gesture is roaming;

三轴陀螺仪,用于获取手势漫游时人手在空间坐标系中的轴向转动角速度Ω=(wx,wy,wz)。The three-axis gyroscope is used to obtain the axial rotation angular velocity Ω=(w x , w y , w z ) of the human hand in the space coordinate system during gesture roaming.

优选的,所述无线通信方式为Bluetooth、ZigBee、WiFi无线通讯方式中的一种。Preferably, the wireless communication method is one of Bluetooth, ZigBee, and WiFi wireless communication methods.

为了达到上述第二目的,本发明采用以下技术方案:In order to achieve the above-mentioned second purpose, the present invention adopts the following technical solutions:

一种交互界面中人手漫游控制的穿戴设备的控制方法,包括下述步骤:A control method for a wearable device controlled by human hand roaming in an interactive interface, comprising the following steps:

S1、当设备与主机距离为第一预定范围时,近距无线收发模块自动发送无线信息,与主机进行连接,建立无线通讯,系统进入待机状态;S1. When the distance between the device and the host is within the first predetermined range, the short-range wireless transceiver module automatically sends wireless information, connects with the host, establishes wireless communication, and the system enters the standby state;

S2、人手运动信息获取模块实时获取人手在空间坐标系中的运动信息,所述运动信息包括加速度信息和角速度信息,并发送至数据处理模块;S2. The movement information acquisition module of the human hand acquires the movement information of the human hand in the spatial coordinate system in real time, the movement information includes acceleration information and angular velocity information, and sends it to the data processing module;

S3、当检测到用户抬手和连续两次或两次以上快速转动手腕超过第一预定角度时,则控制权调配模块为其分配控制权,设备进入工作状态,交互界面显示模拟光标;S3. When it is detected that the user raises his hand and rotates his wrist twice or more in a row to exceed the first predetermined angle, the control right allocation module assigns the control right to him, the device enters the working state, and the interactive interface displays a simulated cursor;

S4、人手运动信息获取模块实时获取人手漫游时在空间坐标系中的运动信息,数据处理模块将所述运动信息进行处理分析,并分类识别其为漫游动作,并将识别结果数据发送至主机;S4. The hand movement information acquisition module acquires the movement information in the spatial coordinate system when the hand is roaming in real time, and the data processing module processes and analyzes the movement information, classifies and recognizes it as a roaming action, and sends the recognition result data to the host;

S5、主机接收所述识别结果数据后,将Ω=(ωxyz)通过卡尔曼滤波器处理得到并将其转化为交互界面上模拟光标的模拟坐标并同步显示模拟光标的移动;S5. After the host receives the recognition result data, it processes Ω=(ω xyz ) through a Kalman filter to obtain And convert it into the simulated coordinates of the simulated cursor on the interactive interface Simultaneously display the movement of the analog cursor;

S6、当光标漫游至用户所需功能图标时,用户做出弹指动作,人手运动信息获取模块采集人手此时的动作信息;S6. When the cursor roams to the function icon required by the user, the user makes a flick action, and the hand movement information acquisition module collects the movement information of the hand at this time;

S7、数据处理模块对上述运动信息进行分类识别为选中动作,交互界面上即显示为用户所需功能图标被选中;S7. The data processing module classifies and recognizes the above motion information as a selected action, and the interactive interface displays that the function icon required by the user is selected;

S8、重复S2至S7步骤,直至检测到用户手臂自然放下动作以及鼠标处于交互界面边缘超过第一预定时间,或者在第二预定时间内未检测到人手动作,则控制权调配模块取消设备控制权,系统进入待机状态;或者直至检测到步骤S3中所述抬手和连续两次或以上快速转动或左右晃动手腕动作,系统再次进入工作状态。S8. Repeat steps S2 to S7 until it detects that the user's arm is naturally lowered and the mouse is on the edge of the interactive interface for more than the first predetermined time, or no human hand movement is detected within the second predetermined time, then the control right allocation module cancels the device control right , the system enters the standby state; or until it detects the raising of the hand and two or more consecutive quick rotations or shaking the wrist left and right described in step S3, the system enters the working state again.

优选的,步骤S3中,获取设备控制权的方式为:手腕被抬起,且连续两次或以上快速转动手腕超过第一预定角度,一次转动定义为手腕由初始状态沿某一方向轴向转动,再沿相反方向将手腕转动回初始状态。Preferably, in step S3, the way to obtain the control right of the device is: the wrist is lifted, and the wrist is rotated twice or more rapidly to exceed the first predetermined angle, and one rotation is defined as the wrist rotates axially in a certain direction from the initial state , and then turn the wrist back to the original position in the opposite direction.

优选的,数据处理模块将所述运动信息进行处理分析,并分类识别其为漫游动作,具体过程为:Preferably, the data processing module processes and analyzes the motion information, and classifies and identifies it as a roaming action. The specific process is:

S41、数据处理模块将所述运动信息进行A/D采样转化为数字信号,并进行卡尔曼滤波融合处理得到人手运动的九维数据信息,所述九维数据信息包括:三轴加速度信息A=(ax,ay,az),三轴角速度信息Ω=(ωxyz)和三轴姿态角信息Θ=(θxyz);S41. The data processing module performs A/D sampling on the motion information and converts it into a digital signal, and performs Kalman filter fusion processing to obtain nine-dimensional data information of hand motion. The nine-dimensional data information includes: three-axis acceleration information A= (a x , a y , a z ), three-axis angular velocity information Ω=(ω xyz ) and three-axis attitude angle information Θ=(θ xyz );

S42、数据处理模块根据上述九维数据信息构建多阶特征张量 x = [ A , Ω , Θ , A · , Ω · , Θ · ] ; S42. The data processing module constructs a multi-order feature tensor according to the above nine-dimensional data information x = [ A , Ω , Θ , A &Center Dot; , Ω &Center Dot; , Θ &Center Dot; ] ;

S43、在短时分析窗口内,基于单帧多阶张量特征xt构建时序动作识别特征Xt=[xt-n,...,xt],其中n为短时窗口长度;S43. In the short-term analysis window, construct a time-series action recognition feature X t =[x tn ,...,x t ] based on the single-frame multi-level tensor feature x t , where n is the length of the short-term window;

S44、对时序动作识别特征做主成分分析,其中包括去中心化,特征值分解与PCA白化,公式为:S44. Perform principal component analysis on time series action recognition features, including decentralization, eigenvalue decomposition and PCA whitening, the formula is:

PP tt == (( Xx tt -- Xx ‾‾ tt )) TT WW Mm ×× NN PCAPCA

其中,为低维子空间映射矩阵,M为时序特征Xt的维度,N为低维子空间映射的维度;in, is the low-dimensional subspace mapping matrix, M is the dimension of the time series feature X t , and N is the dimension of the low-dimensional subspace mapping;

S45、采用多分类支持向量机(SVM)实现对漫游动作、弹指动作和点击动作的分类识别,基于结构风险最小原理训练得到漫游/弹指SVM分类器参数<W1 SVM,b1>,漫游/点击SVM参数<W2 SVM,b1>S45. Using a multi-category support vector machine (SVM) to realize the classification and recognition of roaming actions, flicking actions and clicking actions, and training based on the principle of minimum structural risk to obtain roaming/fingering SVM classifier parameters <W 1 SVM ,b 1 >, roaming/ Click on the SVM parameter <W 2 SVM ,b 1 >

gi(Pt)=PtWi SVM+bi g i (P t )=P t W i SVM +b i

优选的,步骤S7中,所述选中动作为弹指动作,即非拇指的手指在大拇指的按压下快速向外弹出。Preferably, in step S7, the selection action is a flick action, that is, the non-thumb fingers quickly pop out under the pressure of the thumb.

优选的,所述步骤S3,控制权获取动作还可以为:手腕左右晃动动作,即手腕在自然放松状态下,沿同一水平线左右晃动。Preferably, in step S3, the action of obtaining the control right may also be: shaking the wrist left and right, that is, shaking the wrist left and right along the same horizontal line in a naturally relaxed state.

优选的,所述步骤S7中的选中动作,方式还可以为,单指点击动作,即除食指外其余手指做握住状,食指做上下敲击动作。Preferably, the selection action in step S7 can also be a single-finger click action, that is, the rest of the fingers except the index finger perform a holding motion, and the index finger performs an up and down tapping action.

优选的,所述步骤S7中的选中动作,方式还可以为,双指点击动作或多指点击动作,即手腕在自然放松状态下,五指自然张开,食指与中指交替做出点击动作,或除拇指外,其余手指交替做出点击动作。Preferably, the selection action in step S7 can also be a two-finger click action or a multi-finger click action, that is, the wrist is in a naturally relaxed state, the five fingers are naturally opened, and the index finger and middle finger alternately make a click action, or Except for the thumb, the rest of the fingers alternately make clicking movements.

本发明与现有技术相比,具有如下优点和有益效果:Compared with the prior art, the present invention has the following advantages and beneficial effects:

1)本发明一种交互界面中人手漫游控制的穿戴式设备采用加速度传感器与陀螺仪采集人手的运动信息,使用户手势在交互界面上漫游时能得到精确稳定的漫游结果。1) A wearable device controlled by hand roaming in an interactive interface of the present invention uses an acceleration sensor and a gyroscope to collect motion information of the human hand, so that accurate and stable roaming results can be obtained when user gestures roam on the interactive interface.

2)本发明一种交互界面中人手漫游控制的穿戴式设备为非手持式设备,使用时只需佩戴在手腕处即可,因此使用户摆脱了手持交互设备的束缚,解放了人的双手。2) The wearable device controlled by hand roaming in the interactive interface of the present invention is a non-handheld device, which only needs to be worn on the wrist when in use, thus freeing the user from the shackles of the handheld interactive device and liberating the human hands.

3)本发明一种交互界面中人手漫游控制的方法是基于人们在进行人机交互操作时的习惯设计而成,使用户能自然的完成交互操作,不容易出现疲劳。3) A method for manual roaming control in an interactive interface of the present invention is designed based on people's habits in man-machine interactive operations, so that users can naturally complete interactive operations without fatigue.

附图说明Description of drawings

图1为本发明一种交互界面中人手漫游控制的穿戴式设备的结构示意图;Fig. 1 is a schematic structural diagram of a wearable device controlled by hand roaming in an interactive interface of the present invention;

图2为本发明一种交互界面中人手漫游控制的穿戴式设备的系统结构示意图;Fig. 2 is a schematic diagram of the system structure of a wearable device controlled by hand roaming in an interactive interface of the present invention;

图3为本发明一种交互界面中人手漫游控制的方法的流程示意图;3 is a schematic flowchart of a method for manual roaming control in an interactive interface of the present invention;

图4为本发明一种交互界面中人手漫游控制的方法的控制权获取动作示意图;Fig. 4 is a schematic diagram of a control right acquisition action of a method for manual roaming control in an interactive interface of the present invention;

图5为本发明一种交互界面中人手漫游控制的方法的弹指动作示意图;Fig. 5 is a schematic diagram of flicking actions of a method for manual roaming control in an interactive interface of the present invention;

图6为本发明一种交互界面中人手漫游控制的方法的单指点击动作示意图;6 is a schematic diagram of a single-finger click action of a method for manual roaming control in an interactive interface of the present invention;

图7为本发明一种交互界面中人手漫游控制的方法的双指点击动作示意图;7 is a schematic diagram of a two-finger click action of a method for manual roaming control in an interactive interface of the present invention;

图8为本发明一种交互界面中人手漫游控制的方法的多指点击动作示意图。FIG. 8 is a schematic diagram of a multi-finger click action of a method for manual roaming control in an interactive interface according to the present invention.

具体实施方式Detailed ways

下面结合实施例及附图对本发明作进一步详细的描述,但本发明的实施方式不限于此。The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

实施例Example

本发明的一种交互界面中人手漫游控制的穿戴式设备,包括设备本体与设于设备本体内的电路模块,其中,如图1所示,设备为环状,使用时需佩戴在用户手腕处。A wearable device controlled by hand roaming in an interactive interface of the present invention includes a device body and a circuit module installed in the device body, wherein, as shown in Figure 1, the device is ring-shaped and needs to be worn on the user's wrist when in use .

如图2所示,所述电路模块包括人手运动信息获取模块21、数据处理模块22、控制权调配模块23、近距无线收发模块24,其中:As shown in Figure 2, the circuit module includes a hand movement information acquisition module 21, a data processing module 22, a control right allocation module 23, and a short-range wireless transceiver module 24, wherein:

人手运动信息获取模块21:用于获取手势漫游时人手在空间坐标系中的运动信息,并发送至数据处理模块;Hand movement information acquisition module 21: used to acquire movement information of hands in the spatial coordinate system during gesture roaming, and send it to the data processing module;

数据处理模块22:将所述运动信息进行处理分析,并分类识别其漫游手势动作,将其转化为控制信号并发送至主机;Data processing module 22: process and analyze the motion information, classify and identify its roaming gestures, convert them into control signals and send them to the host;

控制权调配模块23:根据人手运动情况调配控制权,触发或停止漫游操作功能;Control right allocation module 23: Allocate the control right according to the movement of human hands, trigger or stop the roaming operation function;

近距无线收发模块24:用于与主机连接,建立无线通讯,并在建立通讯的基础上将控制信息发送至主机。Short-distance wireless transceiver module 24: used to connect with the host, establish wireless communication, and send control information to the host on the basis of establishing communication.

具体的,所述主机可以为计算机、智能电视等具有较大显示屏的设备,且主机也应有近距无线收发模块以跟本发明穿戴式设备建立无线通讯。Specifically, the host computer can be a device with a large display screen such as a computer, a smart TV, etc., and the host computer should also have a short-range wireless transceiver module to establish wireless communication with the wearable device of the present invention.

进一步的,所述人手运动信息获取模块21包括:Further, the human hand movement information acquisition module 21 includes:

三轴加速度传感器211,用于获取手势漫游时人手在空间坐标系中的线性平移加速度A=(ax,ay,az);The three-axis acceleration sensor 211 is used to obtain the linear translation acceleration A=(a x , a y , a z ) of the human hand in the space coordinate system during gesture roaming;

三轴陀螺仪212,用于获取手势漫游时人手在空间坐标系中的轴向转动角速度Ω=(wx,wy,wz)。The three-axis gyroscope 212 is used to acquire the axial rotation angular velocity Ω=(w x , w y , w z ) of the human hand in the space coordinate system during gesture roaming.

进一步的,所述无线通迅,方式可以为Bluetooth、ZigBee、WiFi等无线通讯方式。Further, the wireless communication method may be Bluetooth, ZigBee, WiFi and other wireless communication methods.

如图3所示,一种交互界面中人手漫游控制的方法,其中,包括以下步骤:As shown in Figure 3, a method for manual roaming control in an interactive interface, which includes the following steps:

S1当设备与主机距离为第一预定范围时,近距无线收发模块24自动发送无线信息,与主机进行连接,建立无线通讯,系统进入待机状态;S1 When the distance between the device and the host is within the first predetermined range, the short-range wireless transceiver module 24 automatically sends wireless information, connects with the host, establishes wireless communication, and the system enters a standby state;

具体应用场景可以如下,本发明穿戴式设备的近距无线收发模块24不断检测附近5米内(假设第一预定范围阈值为5米)是否存在有无线收发模块开启工作的主机,若有,则穿戴式设备的近距无线收发模块24主动与主机的近距无线收发模块连接,建立无线通讯,本发明设备进入待机状态。The specific application scenario can be as follows. The short-range wireless transceiver module 24 of the wearable device of the present invention continuously detects whether there is a host computer with the wireless transceiver module turned on within 5 meters (assuming that the first predetermined range threshold is 5 meters), and if so, wear The short-range wireless transceiver module 24 of the type device actively connects with the short-range wireless transceiver module of the host computer to establish wireless communication, and the device of the present invention enters a standby state.

S2人手运动信息获取模块21实时获取人手在空间坐标系中的运动信息(ax,ay,az和wx,wy,wz),并发送至数据处理模块;S2 The hand movement information acquisition module 21 acquires the movement information (a x , a y , a z and w x , w y , w z ) of the hand in the space coordinate system in real time, and sends it to the data processing module;

S3当检测到用户抬手和连续两次(或以上)快速转动手腕超过第一预定角度时(如图4所示),则控制权调配模块23为其分配控制权,设备进入工作状态,交互界面显示模拟光标;S3 When it is detected that the user raises his hand and rotates his wrist twice (or more) rapidly to exceed the first predetermined angle (as shown in Figure 4), the control right allocation module 23 assigns the control right to it, and the device enters the working state, and the interaction The interface displays an analog cursor;

具体应用场景可以如下,用户想要用本发明穿戴式设备在交互界面上进行漫游时,则只需要将手从自然下垂状态抬起到一舒适的位置,并连续两次(或以上)快速转动手腕超过90°(假设第一预定角度阈值为90°),人手运动信息获取模块21采集到此动作信息后发送至数据处理模块22,数据处理模块02对所述动作信息进行处理分析,并分类识别其为控制权获取动作,并将识别结果发送至控制权调配模块23为该穿戴式设备分配控制权,设备进入工作状态,交互界面中心显示初始模拟光标,如图4所示。The specific application scenario can be as follows. When the user wants to use the wearable device of the present invention to roam on the interactive interface, he only needs to lift his hand from the natural drooping state to a comfortable position, and rotate it twice (or more) quickly When the wrist exceeds 90° (assuming that the first predetermined angle threshold is 90°), the movement information acquisition module 21 of the human hand collects the movement information and sends it to the data processing module 22. The data processing module 02 processes and analyzes the movement information and classifies it. It is recognized as a control right acquisition action, and the recognition result is sent to the control right allocation module 23 to assign control rights to the wearable device, the device enters the working state, and the initial simulation cursor is displayed in the center of the interactive interface, as shown in FIG. 4 .

若用户只抬手而未转动手腕,或者用户只转动手腕一次,或者用户每次转动手腕都不超过90°,则系统都认为不是控制权获取动作,因此也不会分配控制权给当前设备。If the user only raises the hand but does not turn the wrist, or the user only turns the wrist once, or the user does not turn the wrist more than 90° each time, the system considers it is not a control right acquisition action, so the control right will not be assigned to the current device.

S4人手运动信息获取模块21实时获取人手漫游时在空间坐标系中的运动信息(ax,ay,az和wx,wy,wz),数据处理模块22将所述运动信息进行处理分析,并分类识别其为漫游动作,并将识别结果数据发送至主机;S4 The hand movement information acquisition module 21 acquires the movement information (a x , a y , a z and w x , w y , w z ) in the spatial coordinate system when the hand roams in real time, and the data processing module 22 performs the movement information on the Process and analyze, classify and identify it as a roaming action, and send the identification result data to the host;

S5主机接收所述识别结果数据后,将其转化为交互界面上模拟光标的模拟坐标,并同步显示模拟光标的移动;After receiving the recognition result data, the S5 host converts it into the simulated coordinates of the simulated cursor on the interactive interface, and simultaneously displays the movement of the simulated cursor;

S6当光标漫游至用户所需功能图标时,用户做出弹指动作,如图5所示,人手运动信息获取模块21采集人手此时的动作信息;S6 When the cursor roams to the desired function icon of the user, the user makes a flicking action, as shown in Figure 5, the movement information of the hand movement information acquisition module 21 collects the action information of the hand at this moment;

S7数据处理模块22对上述运动信息进行分类识别为选中动作,交互界面上即显示为用户所需功能图标被选中;S7 data processing module 22 classifies and recognizes the above-mentioned motion information as a selected action, and the user's required function icon is selected on the interactive interface;

S8重复S2至S7步骤,直至检测到用户手臂自然放下动作以及鼠标处于交互界面边缘超过第一预定时间,或者在第二预定时间内未检测到人手动作,则控制权调配模块23取消设备控制权,系统进入待机状态,直至检测到步骤S3中所述抬手和连续两次(或以上)快速转动手腕动作,系统再次进入工作状态;S8 Repeat steps S2 to S7 until it detects that the user's arm is naturally lowered and the mouse is on the edge of the interactive interface for more than the first predetermined time, or no human hand motion is detected within the second predetermined time, then the control right allocation module 23 cancels the device control right , the system enters the standby state until it detects the raising of the hand and two consecutive (or more) quick rotations of the wrist described in step S3, and the system enters the working state again;

具体应用场景可以如下,用户由于某一因素,例如想要详细浏览交互界面内容,暂时不需要光标漫游与选中操作,则可以将手臂自然放下,设备检测到这一动作之后判断其为用户暂时不需要漫游操作与选中操作;同时,若用户快速将模拟光标移动至交互界面边缘超过3秒(假设第一预定时间阈值为3秒),或者设备在15秒(假设第二预定时间阈值为15秒)内未检测到用户手的动作,则设备判断用户暂时不需要漫游与选中操作,则系统进入待机状态。若用户需要再次获得控制权,则应该做抬起手腕动作,且连续两次(或以上)快速转动手腕超过90°,设备检测到这一动作之后则系统再次进入工作状态。The specific application scenario can be as follows. Due to a certain factor, such as wanting to browse the content of the interactive interface in detail, the user does not need the cursor roaming and selection operations for the time being, so the user can put down the arm naturally. After detecting this action, the device judges that the user is not temporarily Roaming and selection operations are required; at the same time, if the user quickly moves the simulated cursor to the edge of the interface for more than 3 seconds (assuming the first predetermined time threshold is 3 seconds), or the device is within 15 seconds (assuming the second predetermined time threshold is 15 seconds) ), the device judges that the user does not need roaming and selection operations for the time being, and the system enters the standby state. If the user needs to gain control again, he should raise his wrist and turn his wrist more than 90° two times (or more) in a row. After the device detects this action, the system will enter the working state again.

需要说明的是,上述记载的第一预定时间阈值为3秒和第二预定时间阈值为15秒仅仅作为一种示例,并不构成不当的限定,本发明实施例中第一预定时间阈值和第二预定时间阈值可以为相同,如都是15秒。It should be noted that the first predetermined time threshold value of 3 seconds and the second predetermined time threshold value of 15 seconds described above are only an example and do not constitute an improper limitation. In the embodiment of the present invention, the first predetermined time threshold value and the second predetermined time threshold value The two predetermined time thresholds may be the same, for example, both are 15 seconds.

进一步的,获取设备控制权的方式为:手腕被抬起,且连续两次(或以上)快速转动手腕超过第一预定角度,一次转动定义为手腕由初始状态沿某一方向轴向转动,再沿相反方向将手腕转动回初始状态。Further, the way to obtain the control right of the device is as follows: the wrist is lifted, and the wrist is rotated twice (or more) quickly to exceed the first predetermined angle. One rotation is defined as the wrist rotates axially in a certain direction from the initial state, and then Turn your wrist back to the original position in the opposite direction.

进一步的,所述步骤S5,即数据处理模块02对上述运动信息进行分类识别,具体过程为:Further, the step S5, that is, the data processing module 02 classifies and recognizes the above-mentioned motion information, and the specific process is as follows:

S51数据处理模块02将所述运动信息进行A/D采样转化为数字信号,并进行卡尔曼滤波融合处理得到人手运动的九维数据信息(三轴加速度信息A=(ax,ay,az),三轴角速度信息Ω=(wx,wy,wz)和三轴姿态角信息Θ=(θx,θy,θz));S51 Data processing module 02 performs A/D sampling on the motion information and converts it into a digital signal, and performs Kalman filter fusion processing to obtain nine-dimensional data information of human hand motion (three-axis acceleration information A=(a x , a y , a z ), three-axis angular velocity information Ω=(w x , w y , w z ) and three-axis attitude angle information Θ=(θ x , θ y , θ z ));

S52数据处理模块02根据上述九维数据信息构建多阶特征张量 x = [ A , &Theta; , &Omega; , A &CenterDot; , &Theta; &CenterDot; , &Omega; &CenterDot; ] ; S52 data processing module 02 constructs a multi-level feature tensor according to the above nine-dimensional data information x = [ A , &Theta; , &Omega; , A &Center Dot; , &Theta; &Center Dot; , &Omega; &CenterDot; ] ;

S53在短时分析窗口内,基于单帧多阶张量特征xt构建时序动作识别特征Xt=[xt-n,...,xt]其中n为短时窗口长度;S53 In the short -term analysis window, construct a time-series action recognition feature X t = [ x tn , .

S54对时序动作识别特征做主成分分析(PCA),其中包括去中心化,特征值分解与PCA白化。S54 performs principal component analysis (PCA) on the temporal action recognition features, including decentralization, eigenvalue decomposition and PCA whitening.

PP tt == (( Xx tt -- Xx tt &OverBar;&OverBar; )) TT WW Mm &times;&times; NN

其中,M为时序特征Xt的维度,N为低维子空间映射的维度;Among them, M is the dimension of time series feature X t , and N is the dimension of low-dimensional subspace mapping;

S55采用支持向量机(Kernel SVM)实现对漫游动作、弹指动作和点击动作的分类识别;S55 adopts support vector machine (Kernel SVM) to realize the classification and recognition of roaming action, flick action and click action;

ythe y tt (( ww TT PP tt ++ bb )) == &gamma;&gamma; ^^ ii &GreaterEqual;&Greater Equal; &gamma;&gamma; ^^ ,, ii == 11 ,, .. .. .. ,, nno

maxmax 11 || || ww || || sthe s .. tt .. ythe y tt (( ww TT PP tt ++ bb )) &GreaterEqual;&Greater Equal; 11 ,, ii == 11 ,, .. .. .. ,, nno

进一步的,所述选中动作为弹指动作,如图5所示,食指(或其余非拇指的手指)在大拇指的按压下快速向外弹出。Further, the selection action is a flick action, as shown in FIG. 5 , the index finger (or other non-thumb fingers) quickly pops out under the pressure of the thumb.

进一步的,所述步骤S3,控制权获取动作还可以为,手腕左右晃动动作,即手腕在自然放松状态下,沿同一水平线左右晃动。Further, in the step S3, the action of obtaining the control right may also be an action of shaking the wrist left and right, that is, shaking the wrist left and right along the same horizontal line in a naturally relaxed state.

进一步的,所述步骤S8中选中动作,方式还可以为,单指点击动作,即除食指外其余手指做握住状,食指做上下敲击动作,如图6所示。Further, the action of selecting in step S8 may also be a single-finger click action, that is, the other fingers except the index finger perform a holding motion, and the index finger performs an up and down tapping action, as shown in FIG. 6 .

进一步的,所述步骤S8中选中动作,方式还可以为,双指点击动作或多指点击动作,即手腕在自然放松状态下,五指自然张开,食指与中指交替做出点击动作,或除拇指外,其余手指交替做出点击动作。Further, the action selected in step S8 can also be a two-finger click action or a multi-finger click action, that is, the wrist is in a naturally relaxed state, the five fingers are naturally opened, and the index finger and middle finger alternately make a click action, or except Except for the thumb, the rest of the fingers alternately make clicking movements.

综上,本发明公开了一种交互界面中人手漫游控制的穿戴式设备和方法。可穿戴设备呈环状,使用时需佩戴在用户手腕处,该设备具体包含人手运动信息获取模块、数据处理模块、控制权调配模块和近距无线收发模块。一种交互界面人手漫游控制的方法为,设备与主机建立无线通讯,用户通过抬手和连续两次(或以上)快速转动手腕动作或左右晃动手腕动作来获取设备控制权,系统实时采集人手运动信息并在交互界面上显示为模拟光标的移动,用户通过弹指或单指点击或多指点击动作选中交互界面上所需功能图标,如图7、图8所示。In summary, the present invention discloses a wearable device and method for manual roaming control in an interactive interface. The wearable device is ring-shaped and needs to be worn on the user's wrist when in use. The device specifically includes a human hand movement information acquisition module, a data processing module, a control right allocation module and a short-range wireless transceiver module. A method for hand roaming control of an interactive interface is that the device establishes wireless communication with the host, and the user obtains the control right of the device by raising the hand and turning the wrist twice (or more) quickly or shaking the wrist left and right, and the system collects the movement of the human hand in real time The information is displayed on the interactive interface as simulating the movement of the cursor, and the user selects the desired function icon on the interactive interface by flicking or single-finger clicking or multi-finger clicking, as shown in Figure 7 and Figure 8.

本发明实施方式提供的一种交互界面中人手漫游控制的穿戴式设备和方法,至少具有如下有益效果:A wearable device and method for manual roaming control in an interactive interface provided by an embodiment of the present invention has at least the following beneficial effects:

1)本发明一种交互界面中人手漫游控制的穿戴式设备采用加速度传感器与陀螺仪采集人手的运动信息,使用户手势在交互界面上漫游时能得到精确稳定的漫游结果;1) A wearable device controlled by hand roaming in an interactive interface of the present invention uses an acceleration sensor and a gyroscope to collect motion information of the human hand, so that user gestures can obtain accurate and stable roaming results when roaming on the interactive interface;

2)本发明一种交互界面中人手漫游控制的穿戴式设备为非手持式设备,使用时只需佩戴在手腕处即可,因此使用户摆脱了手持交互设备的束缚,解放了人的双手;2) The wearable device controlled by human hand roaming in an interactive interface of the present invention is a non-handheld device, which only needs to be worn on the wrist when in use, so that the user is freed from the shackles of the handheld interactive device and liberates the human hands;

3)本发明一种交互界面中人手漫游控制的方法是基于人们在进行人机交互操作时的习惯设计而成,使用户能自然的完成交互操作,不容易出现疲劳;3) The method for manual roaming control in an interactive interface of the present invention is designed based on people's habits when performing human-computer interactive operations, so that users can naturally complete interactive operations without fatigue;

应注意的是上述实施例对本发明进行说明而不是对本发明进行限制,并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中,不应将位于括号之间的任何参考符号构成对权利要求的限制。单词“包括”不排除存在未列在权利要求中的器件与步骤。本领域的技术人员在不脱离本发明精神和范围的前提下可作若干的更动与润饰,本发明所主张的保护范围应以权利要求书所述为准。It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements and steps not listed in a claim. Those skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention, and the scope of protection claimed by the present invention should be based on the claims.

Claims (10)

1. A wearable device for controlling hand roaming in an interactive interface comprises a device body and a circuit module arranged in the device body, and is characterized in that the device is annular and is worn on the wrist of a user when in use;
the circuit module comprises a hand movement information acquisition module, a data processing module, a control right allocation module and a short-distance wireless transceiving module, wherein:
the human hand movement information acquisition module: the system is used for acquiring the motion information of the hand in the space coordinate system when the gesture roams and sending the motion information to the data processing module;
a data processing module: the device is used for processing and analyzing the motion information, classifying and identifying the roaming gesture action of the motion information, converting the roaming gesture action into a control signal and sending the control signal to the host;
a control right allocating module: allocating control right according to the motion condition of the human hand, and triggering or stopping the roaming operation function of the interactive interface;
short-range wireless transceiver module: the wireless communication system is used for being connected with the host, establishing wireless communication and sending control information to the host on the basis of establishing communication.
2. The wearable device for controlling hand roaming in the interactive interface of claim 1, wherein the hand movement information obtaining module comprises:
the three-axis acceleration sensor is used for acquiring linear translation acceleration A ═ a (of the human hand in a space coordinate system) when the gesture roamsx,ay,az);
The three-axis gyroscope is used for acquiring the axial rotation angular speed omega (w) of the hand in the space coordinate system when the hand moves in a gesture roaming processx,wy,wz)。
3. The wearable device for controlling roaming of human hands in the interaction interface of claim 1, wherein the wireless communication mode is one of Bluetooth, ZigBee and WiFi wireless communication modes.
4. The method for controlling the wearable device for human hand roaming control in the interactive interface of claim 1, comprising the following steps:
s1, when the distance between the device and the host is a first preset range, the short-distance wireless transceiving module automatically sends wireless information, is connected with the host, establishes wireless communication, and the system enters a standby state;
s2, the human hand motion information acquisition module acquires motion information of the human hand in a space coordinate system in real time, wherein the motion information comprises acceleration information and angular velocity information and is sent to the data processing module;
s3, when detecting that the user raises his hand and rotates the wrist twice or more than twice quickly to exceed the first preset angle, the control right allocating module allocates the control right to the user, the device enters the working state, and the interactive interface displays the simulation cursor;
s4, the hand motion information acquisition module acquires motion information in a space coordinate system in real time when a hand roams, the data processing module processes and analyzes the motion information, classifies and identifies the motion information as a roaming action, and sends identification result data to the host;
s5, the host computer receives the recognition result data, and then changes Ω to (ω)xyz) Processed by Kalman filter to obtainAnd converting the coordinates into simulation coordinates of a simulation cursor on the interactive interfaceAnd synchronously displaying the movement of the simulation cursor;
s6, when the cursor roams to the functional icon required by the user, the user makes a finger flicking action, and the hand movement information acquisition module acquires the action information of the hand at the moment;
s7, the data processing module classifies and identifies the motion information as a selected action, and a functional icon required by the user is selected as displayed on the interactive interface;
s8, repeating the steps S2 to S7 until the natural arm lowering action of the user and the situation that the mouse is positioned at the edge of the interactive interface and exceeds the first preset time are detected, or the action of the human hand is not detected in the second preset time, cancelling the equipment control right by the control right allocating module, and enabling the system to enter a standby state; or until the hand raising and two or more continuous quick rotations or wrist left-right shaking motions are detected in the step S3, the system enters the working state again.
5. The control method according to claim 4, wherein in step S3, the device control authority is acquired by: the wrist is lifted and the wrist is rapidly rotated twice or more than twice continuously beyond a first predetermined angle, one rotation being defined as the wrist being axially rotated in one direction from an initial state and then rotated back in the opposite direction to the initial state.
6. The control method according to claim 4, wherein in step S4, the data processing module processes and analyzes the motion information, and classifies and identifies the motion information as a roaming action, which includes:
s41, the data processing module performs A/D sampling on the motion information to convert the motion information into digital signals, and performs Kalman filtering fusion processing to obtain nine-dimensional data information of human hand motion, wherein the nine-dimensional data information comprises: three-axis acceleration information a ═ ax,ay,az) The three-axis angular velocity information Ω ═ ω (ω ═ ω)xyz) And the three-axis attitude angle information theta ═ (theta)xyz);
S42, the data processing module constructs the multi-order feature tensor according to the nine-dimensional data information <math> <mrow> <mi>x</mi> <mo>=</mo> <mo>[</mo> <mi>A</mi> <mo>,</mo> <mi>&Omega;</mi> <mo>,</mo> <mi>&Theta;</mi> <mo>,</mo> <mover> <mi>A</mi> <mo>&CenterDot;</mo> </mover> <mo>,</mo> <mover> <mi>&Omega;</mi> <mo>&CenterDot;</mo> </mover> <mo>,</mo> <mover> <mi>&Theta;</mi> <mo>&CenterDot;</mo> </mover> <mo>]</mo> <mo>;</mo> </mrow> </math>
S43, in the short time analysis window, based on the single-frame multi-order tensor feature xtConstructing temporal action recognition features Xt=[xt-n,...,xt]Where n is the short time window length;
s44, performing principal component analysis on the time sequence action recognition characteristics, wherein the principal component analysis comprises decentralization, eigenvalue decomposition and PCA whitening, and the formula is as follows:
<math> <mrow> <msub> <mi>P</mi> <mi>t</mi> </msub> <mo>=</mo> <msup> <mrow> <mo>(</mo> <msub> <mi>X</mi> <mi>t</mi> </msub> <mo>-</mo> <msub> <mover> <mi>X</mi> <mo>&OverBar;</mo> </mover> <mi>t</mi> </msub> <mo>)</mo> </mrow> <mi>T</mi> </msup> <msubsup> <mi>W</mi> <mrow> <mi>M</mi> <mo>&times;</mo> <mi>N</mi> </mrow> <mi>PCA</mi> </msubsup> </mrow> </math>
wherein,for the low-dimensional subspace mapping matrix, M is the timing characteristic XtN is the dimension of the low-dimensional subspace mapping;
s45, classifying and identifying the roaming action, the pinball action and the click action by adopting a multi-classification Support Vector Machine (SVM), and training based on the principle of minimum structural risk to obtain the parameters of the SVM classifier of the roaming/pinballRoaming/click SVM parameter
gi(Pt)=PtWi SVM+bi
7. The control method according to claim 4, wherein in step S7, the selected action is a finger-flicking action, i.e. a finger other than the thumb flicks out quickly under the pressure of the thumb.
8. The control method according to claim 4, wherein in step S3, the control right acquiring action further comprises: the wrist shakes left and right, i.e. the wrist shakes left and right along the same horizontal line in a natural relaxed state.
9. The control method according to claim 4, wherein the selecting action in step S7 is performed by a single-finger click action, i.e. a holding action by the fingers except the index finger and a tap action by the index finger.
10. The control method according to claim 4, wherein the action selected in step S7 is selected by a two-finger click action or a multi-finger click action, i.e. the wrist is in a natural relaxed state, five fingers are naturally opened, and the index finger and the middle finger are alternatively used for making click actions, or the fingers except the thumb are alternatively used for making click actions.
CN201510076982.7A 2015-02-11 2015-02-11 The Wearable and control method of human hand Roaming control in a kind of interactive interface Active CN104679246B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510076982.7A CN104679246B (en) 2015-02-11 2015-02-11 The Wearable and control method of human hand Roaming control in a kind of interactive interface

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510076982.7A CN104679246B (en) 2015-02-11 2015-02-11 The Wearable and control method of human hand Roaming control in a kind of interactive interface

Publications (2)

Publication Number Publication Date
CN104679246A true CN104679246A (en) 2015-06-03
CN104679246B CN104679246B (en) 2017-10-20

Family

ID=53314440

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510076982.7A Active CN104679246B (en) 2015-02-11 2015-02-11 The Wearable and control method of human hand Roaming control in a kind of interactive interface

Country Status (1)

Country Link
CN (1) CN104679246B (en)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105068462A (en) * 2015-09-02 2015-11-18 肖继林 Pull rod lifting motor control mechanism
CN105204579A (en) * 2015-09-27 2015-12-30 孙博 Modular integrated-type host device
CN105259885A (en) * 2015-11-18 2016-01-20 马群毅 Internet type workshop equipment operating system
CN105425632A (en) * 2015-11-18 2016-03-23 马群毅 Internet man-machine interaction system
CN106293131A (en) * 2016-08-16 2017-01-04 广东小天才科技有限公司 expression input method and device
CN106354378A (en) * 2015-07-13 2017-01-25 阿里巴巴集团控股有限公司 Method and device for quickly selecting multiple targets
WO2017012362A1 (en) * 2015-07-22 2017-01-26 深圳多新哆技术有限责任公司 Method and apparatus for adjusting attitude angle of virtual object in virtual space
WO2017028527A1 (en) * 2015-07-22 2017-02-23 深圳多新哆技术有限责任公司 Method and apparatus for determining touch event in virtual space
CN106502416A (en) * 2016-11-09 2017-03-15 华南理工大学广州学院 A kind of driving simulation system of Intelligent Recognition bimanual input and its control method
CN106512391A (en) * 2016-11-09 2017-03-22 华南理工大学广州学院 Two-hand gesture recognition method, and simulation driving system and method based on two-hand gesture recognition method
CN106990841A (en) * 2017-03-30 2017-07-28 无锡云瞳信息科技有限公司 Gesture identification method and the wearable video camera of intelligence based on motion sensor
WO2018161906A1 (en) * 2017-03-09 2018-09-13 腾讯科技(深圳)有限公司 Motion recognition method, device, system and storage medium
CN108900698A (en) * 2018-05-31 2018-11-27 努比亚技术有限公司 Method, wearable device, terminal and the computer storage medium of controlling terminal
CN109074709A (en) * 2016-05-05 2018-12-21 摩托罗拉解决方案公司 Message is shown using multiple body worn electronic display units
CN109409427A (en) * 2018-10-25 2019-03-01 珠海市君天电子科技有限公司 A kind of key detecting method and device
CN109976518A (en) * 2019-03-12 2019-07-05 合肥工业大学 A kind of man-machine interaction method based on PVDF sensor array
CN110046009A (en) * 2019-02-19 2019-07-23 阿里巴巴集团控股有限公司 Method for recording, device, server and readable storage medium storing program for executing
CN111367356A (en) * 2018-12-24 2020-07-03 群光电子股份有限公司 Method for controlling electronic device by tapping and tapping control group
CN112346559A (en) * 2020-09-29 2021-02-09 广东乐芯智能科技有限公司 Application interface control system
CN112346560A (en) * 2020-09-29 2021-02-09 广东乐芯智能科技有限公司 Method for controlling application interface by bracelet
WO2022067963A1 (en) * 2020-09-29 2022-04-07 泰州翔升科技服务有限公司 Computer control method and wearable device
CN117786643A (en) * 2024-02-22 2024-03-29 清华大学 User identity authentication method and related equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060028429A1 (en) * 2004-08-09 2006-02-09 International Business Machines Corporation Controlling devices' behaviors via changes in their relative locations and positions
US20100124949A1 (en) * 2008-11-14 2010-05-20 Sony Ericsson Mobile Communications Ab Portable communication device and remote motion input device
US20120127070A1 (en) * 2010-11-22 2012-05-24 Electronics And Telecommunications Research Institute Control signal input device and method using posture recognition
CN103186256A (en) * 2011-12-31 2013-07-03 北京德信互动网络技术有限公司 Human-machine interaction system
CN103513788A (en) * 2013-09-13 2014-01-15 广东欧珀移动通信有限公司 Gesture recognition method and system based on gyroscope sensor and mobile terminal
CN103984416A (en) * 2014-06-10 2014-08-13 北京邮电大学 Gesture recognition method based on acceleration sensor

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060028429A1 (en) * 2004-08-09 2006-02-09 International Business Machines Corporation Controlling devices' behaviors via changes in their relative locations and positions
US20100124949A1 (en) * 2008-11-14 2010-05-20 Sony Ericsson Mobile Communications Ab Portable communication device and remote motion input device
US20120127070A1 (en) * 2010-11-22 2012-05-24 Electronics And Telecommunications Research Institute Control signal input device and method using posture recognition
CN103186256A (en) * 2011-12-31 2013-07-03 北京德信互动网络技术有限公司 Human-machine interaction system
CN103513788A (en) * 2013-09-13 2014-01-15 广东欧珀移动通信有限公司 Gesture recognition method and system based on gyroscope sensor and mobile terminal
CN103984416A (en) * 2014-06-10 2014-08-13 北京邮电大学 Gesture recognition method based on acceleration sensor

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106354378A (en) * 2015-07-13 2017-01-25 阿里巴巴集团控股有限公司 Method and device for quickly selecting multiple targets
WO2017012362A1 (en) * 2015-07-22 2017-01-26 深圳多新哆技术有限责任公司 Method and apparatus for adjusting attitude angle of virtual object in virtual space
WO2017028527A1 (en) * 2015-07-22 2017-02-23 深圳多新哆技术有限责任公司 Method and apparatus for determining touch event in virtual space
CN105068462A (en) * 2015-09-02 2015-11-18 肖继林 Pull rod lifting motor control mechanism
CN105204579A (en) * 2015-09-27 2015-12-30 孙博 Modular integrated-type host device
CN105259885A (en) * 2015-11-18 2016-01-20 马群毅 Internet type workshop equipment operating system
CN105425632A (en) * 2015-11-18 2016-03-23 马群毅 Internet man-machine interaction system
CN109074709B (en) * 2016-05-05 2019-11-15 摩托罗拉解决方案公司 Message is shown using multiple body worn electronic display units
CN109074709A (en) * 2016-05-05 2018-12-21 摩托罗拉解决方案公司 Message is shown using multiple body worn electronic display units
CN106293131A (en) * 2016-08-16 2017-01-04 广东小天才科技有限公司 expression input method and device
CN106512391A (en) * 2016-11-09 2017-03-22 华南理工大学广州学院 Two-hand gesture recognition method, and simulation driving system and method based on two-hand gesture recognition method
CN106502416A (en) * 2016-11-09 2017-03-15 华南理工大学广州学院 A kind of driving simulation system of Intelligent Recognition bimanual input and its control method
CN106512391B (en) * 2016-11-09 2019-07-12 华南理工大学广州学院 A kind of bimanual input recognition methods and the driving simulation system based on it, method
CN106502416B (en) * 2016-11-09 2019-04-02 华南理工大学广州学院 A kind of driving simulation system and its control method of intelligent recognition bimanual input
WO2018161906A1 (en) * 2017-03-09 2018-09-13 腾讯科技(深圳)有限公司 Motion recognition method, device, system and storage medium
CN106990841A (en) * 2017-03-30 2017-07-28 无锡云瞳信息科技有限公司 Gesture identification method and the wearable video camera of intelligence based on motion sensor
CN108900698A (en) * 2018-05-31 2018-11-27 努比亚技术有限公司 Method, wearable device, terminal and the computer storage medium of controlling terminal
CN109409427A (en) * 2018-10-25 2019-03-01 珠海市君天电子科技有限公司 A kind of key detecting method and device
CN111367356A (en) * 2018-12-24 2020-07-03 群光电子股份有限公司 Method for controlling electronic device by tapping and tapping control group
CN110046009A (en) * 2019-02-19 2019-07-23 阿里巴巴集团控股有限公司 Method for recording, device, server and readable storage medium storing program for executing
CN110046009B (en) * 2019-02-19 2022-08-23 创新先进技术有限公司 Recording method, recording device, server and readable storage medium
CN109976518A (en) * 2019-03-12 2019-07-05 合肥工业大学 A kind of man-machine interaction method based on PVDF sensor array
CN112346559A (en) * 2020-09-29 2021-02-09 广东乐芯智能科技有限公司 Application interface control system
CN112346560A (en) * 2020-09-29 2021-02-09 广东乐芯智能科技有限公司 Method for controlling application interface by bracelet
WO2022067963A1 (en) * 2020-09-29 2022-04-07 泰州翔升科技服务有限公司 Computer control method and wearable device
CN117786643A (en) * 2024-02-22 2024-03-29 清华大学 User identity authentication method and related equipment
CN117786643B (en) * 2024-02-22 2024-05-07 清华大学 A user identity authentication method and related equipment

Also Published As

Publication number Publication date
CN104679246B (en) 2017-10-20

Similar Documents

Publication Publication Date Title
CN104679246B (en) The Wearable and control method of human hand Roaming control in a kind of interactive interface
CN113608665B (en) Application for multi-touch input detection
US11543887B2 (en) User interface control of responsive devices
US11567573B2 (en) Neuromuscular text entry, writing and drawing in augmented reality systems
Moazen et al. AirDraw: Leveraging smart watch motion sensors for mobile human computer interactions
KR102407071B1 (en) Multi-device multi-user sensor correlation for pen and computing device interaction
EP3090331B1 (en) Systems with techniques for user interface control
US10042457B2 (en) Watch type terminal
CN106462341B (en) Sensor correlation for pen and touch sensitive computing device interaction
US10289239B2 (en) Application programming interface for multi-touch input detection
CN104780217B (en) Detect method, system and the client of user job efficiency
EP2933709A2 (en) Haptic information management method and electronic device supporting the same
US20150323998A1 (en) Enhanced user interface for a wearable electronic device
US20160179210A1 (en) Input supporting method and input supporting device
US20160124579A1 (en) Controlling multiple devices with a wearable input device
CN106164808A (en) Equipment and the method for equipment is calculated for ring
US20170010695A1 (en) Enhanced multi-touch input detection
CN107390867A (en) A kind of man-machine interactive system based on Android wrist-watch
US20170010733A1 (en) User-identifying application programming interface (api)
JP2010086367A (en) Positional information inputting device, positional information inputting method, program, information processing system, and electronic equipment
JP5605281B2 (en) WRITE INFORMATION DISPLAY SYSTEM, WRITE INFORMATION DISPLAY DEVICE, AND WRITE INFORMATION DISPLAY METHOD
JP6523509B1 (en) Game program, method, and information processing apparatus
Sajjadi et al. Airdraw: Leveraging smart watch motion sensors for mobile human computer interactions
CN113849066A (en) Information barrier-free interaction device, system, method and storage medium
Wu et al. Infofinder: just-in-time information interface from the combination of an hwd with a smartwatch

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20181213

Address after: 510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province

Patentee after: Guangzhou Boguanwen Language Technology Co., Ltd.

Address before: 510640 No. five, 381 mountain road, Guangzhou, Guangdong, Tianhe District

Patentee before: South China University of Technology

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20190926

Address after: 510000 820, room 8, 8, 116 Heng Road, Dongguan, Guangzhou, Guangdong.

Patentee after: Guangzhou Bo Wei Intelligent Technology Co., Ltd.

Address before: 510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province

Patentee before: Guangzhou Boguanwen Language Technology Co., Ltd.

TR01 Transfer of patent right