CN104679246A - Wearable type equipment based on interactive interface human hand roaming control and interactive interface human hand roaming control method - Google Patents
Wearable type equipment based on interactive interface human hand roaming control and interactive interface human hand roaming control method Download PDFInfo
- Publication number
- CN104679246A CN104679246A CN201510076982.7A CN201510076982A CN104679246A CN 104679246 A CN104679246 A CN 104679246A CN 201510076982 A CN201510076982 A CN 201510076982A CN 104679246 A CN104679246 A CN 104679246A
- Authority
- CN
- China
- Prior art keywords
- action
- roaming
- hand
- interactive interface
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000033001 locomotion Effects 0.000 claims abstract description 79
- 210000000707 wrist Anatomy 0.000 claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000004891 communication Methods 0.000 claims abstract description 20
- 230000003993 interaction Effects 0.000 claims abstract description 8
- 230000009471 action Effects 0.000 claims description 72
- 210000003811 finger Anatomy 0.000 claims description 25
- 230000001133 acceleration Effects 0.000 claims description 14
- 238000012706 support-vector machine Methods 0.000 claims description 13
- 210000004247 hand Anatomy 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 210000003813 thumb Anatomy 0.000 claims description 9
- 238000000513 principal component analysis Methods 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000002087 whitening effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 2
- 230000002123 temporal effect Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims 1
- 241000699666 Mus <mouse, genus> Species 0.000 description 15
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010079 rubber tapping Methods 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- 241000699670 Mus sp. Species 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 235000013311 vegetables Nutrition 0.000 description 1
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
本发明公开了一种交互界面中人手漫游控制的穿戴式设备及控制方法,可穿戴设备呈环状,使用时需佩戴在用户手腕处,该设备具体包含人手运动信息获取模块、数据处理模块、控制权调配模块和近距无线收发模块。一种交互界面人手漫游控制的方法为,设备与主机建立无线通讯,用户通过抬手和连续两次或以上快速转动手腕动作或左右晃动手腕动作来获取设备控制权,系统实时采集人手运动信息并在交互界面上显示为模拟光标的移动,用户通过弹指或单指点击或多指点击动作选中交互界面上所需功能图标。本发明提供的设备为低成本低功耗可穿戴设备,且本发明提供的方法能提供给用户舒适自然且操作精确的人机交互体验。
The invention discloses a wearable device and a control method for hand roaming control in an interactive interface. The wearable device is ring-shaped and needs to be worn on the user's wrist when in use. The device specifically includes a human hand movement information acquisition module, a data processing module, The control rights allocation module and the short-range wireless transceiver module. A method for hand roaming control on an interactive interface is that the device establishes wireless communication with the host, and the user obtains the control right of the device by raising the hand and rotating the wrist two or more times in a row or shaking the wrist left and right. It is displayed on the interactive interface as simulating the movement of the cursor, and the user selects the desired function icon on the interactive interface by flicking or single-finger clicking or multi-finger clicking. The device provided by the present invention is a wearable device with low cost and low power consumption, and the method provided by the present invention can provide users with a comfortable, natural and precise man-machine interaction experience.
Description
技术领域technical field
本发明涉及人机交互的研究领域,特别涉及一种交互界面中人手漫游控制的穿戴式设备及控制方法。The invention relates to the research field of human-computer interaction, in particular to a wearable device and a control method for manual roaming control in an interactive interface.
背景技术Background technique
随着计算机技术的快速发展,人机交互技术也得到了一定的发展。传统的人机交互大多是基于鼠标键盘的交互方式,为使人们在操作计算机时有更好的体验,鼠标也经历了许多革命性的发展。从机械鼠标到光电鼠标,再到如今的无线鼠标,都显示着人们对鼠标的重视,即人们对人机交互体验的重视。然而,上述鼠标在操作时都需要一个支撑平面来使鼠标完成精确的操作,但这也暴露出了一些不方便,人们总希望能不受约束的与计算机设备进行交互操作。空中鼠标,可以实现对鼠标的精确控制而不需要放在任何平面上,可以实现点击甚至文字输入操作,但它像传统鼠标一样无法释放双手来操作屏幕。With the rapid development of computer technology, human-computer interaction technology has also been developed to a certain extent. Most of the traditional human-computer interaction is based on the interaction mode of mouse and keyboard. In order to make people have a better experience when operating computers, the mouse has also undergone many revolutionary developments. From mechanical mouse to optical mouse, to today's wireless mouse, it shows that people attach importance to the mouse, that is, people attach importance to the human-computer interaction experience. However, the above-mentioned mice all need a support plane to enable the mouse to complete precise operations during operation, but this also exposes some inconveniences, and people always hope to perform interactive operations with computer equipment without restraint. The air mouse can realize precise control of the mouse without placing it on any flat surface, and can realize click and even text input operations, but it cannot release both hands to operate the screen like a traditional mouse.
人们希望有一种释放双手的操作方式,于是基于视觉的手势控制能完全释放双手,可以实现鼠标漫游和点击功能,但大屏幕下基于视觉的手势交互不够稳定,不能精确控制鼠标,并且在家居环境下的摄像头也存在侵犯隐私的风险,难以普及。手指鼠标因生活中有许多不适宜佩戴指环的活动,例如洗手、洗菜等,导致用户不能长时间连续佩戴指环,因此用户体验不佳;,最新的臂环基于肌肉感应手势则成本高技术复杂,佩戴不适。如今的手环基本都已具备健康监测,管理手机等功能,人们也越来越接受佩戴手环这一可穿戴设备,同时,手环基本都配备有加速度传感器、陀螺仪,并且成本较低,用手环来控制鼠标是我们提出的新技术,将手势的动作映射为交互界面上模拟光标的移动,可以实现稳定精确操作交互界面,具有操作精确、释放双手、成本低、技术成熟、功耗小等优势,能够满足用户需求。People hope to have an operation method that releases both hands, so vision-based gesture control can completely release hands and realize mouse roaming and clicking functions. The camera under the camera also has the risk of violating privacy and is difficult to popularize. Finger mouse has many activities that are not suitable for wearing rings in life, such as washing hands, washing vegetables, etc., so users cannot wear rings continuously for a long time, so the user experience is not good; the latest armbands based on muscle-sensing gestures are expensive and technically complicated , uncomfortable to wear. Today's wristbands basically have functions such as health monitoring and mobile phone management, and people are increasingly accepting wearing wristbands as a wearable device. At the same time, wristbands are basically equipped with acceleration sensors and gyroscopes, and the cost is relatively low. Using a wristband to control the mouse is a new technology proposed by us. It maps gesture movements to the movement of the simulated cursor on the interactive interface, which can realize stable and precise operation of the interactive interface. It has the advantages of precise operation, free hands, low cost, mature technology, low power consumption Small and other advantages, to meet user needs.
发明内容Contents of the invention
本发明的主要目的在于克服现有技术的缺点与不足,提供一种操作精确、能解放人的双手且成本低的交互界面中人手漫游控制的穿戴式设备。The main purpose of the present invention is to overcome the disadvantages and deficiencies of the prior art, and provide a wearable device with precise operation, freedom of hands and low cost for manual roaming control in an interactive interface.
本发明的另一目的在于,提供一种上述交互界面中人手漫游控制的穿戴式设备的控制方法。Another object of the present invention is to provide a control method of a wearable device controlled by human hand roaming in the above-mentioned interactive interface.
为了达到上述第一目的,本发明采用以下技术方案:In order to achieve the above-mentioned first purpose, the present invention adopts the following technical solutions:
一种交互界面中人手漫游控制的穿戴式设备,包括设备本体与设于设备本体内的电路模块,设备为环状,使用时佩戴在用户手腕处;A wearable device controlled by human hand roaming in an interactive interface, including a device body and a circuit module installed in the device body, the device is ring-shaped, and is worn on the user's wrist when in use;
所述电路模块包括人手运动信息获取模块、数据处理模块、控制权调配模块、近距无线收发模块,其中:The circuit module includes a manual movement information acquisition module, a data processing module, a control right allocation module, and a short-range wireless transceiver module, wherein:
人手运动信息获取模块:用于获取手势漫游时人手在空间坐标系中的运动信息,并发送至数据处理模块;Human hand movement information acquisition module: used to acquire the movement information of the human hand in the spatial coordinate system during gesture roaming, and send it to the data processing module;
数据处理模块:用于将所述运动信息进行处理分析,并分类识别其漫游手势动作,将其转化为控制信号并发送至主机;Data processing module: used to process and analyze the motion information, classify and recognize its roaming gestures, convert them into control signals and send them to the host;
控制权调配模块:根据人手运动情况调配控制权,触发或停止交互界面漫游操作功能;Control rights allocation module: allocate control rights according to the movement of human hands, trigger or stop the roaming operation function of the interactive interface;
近距无线收发模块:用于与主机连接,建立无线通讯,并在建立通讯的基础上将控制信息发送至主机。Short-distance wireless transceiver module: used to connect with the host, establish wireless communication, and send control information to the host on the basis of establishing communication.
优选的,所述人手运动信息获取模块包括:Preferably, the human hand movement information acquisition module includes:
三轴加速度传感器,用于获取手势漫游时人手在空间坐标系中的线性平移加速度A=(ax,ay,az);The three-axis acceleration sensor is used to obtain the linear translation acceleration A=(a x , a y , a z ) of the human hand in the space coordinate system when the gesture is roaming;
三轴陀螺仪,用于获取手势漫游时人手在空间坐标系中的轴向转动角速度Ω=(wx,wy,wz)。The three-axis gyroscope is used to obtain the axial rotation angular velocity Ω=(w x , w y , w z ) of the human hand in the space coordinate system during gesture roaming.
优选的,所述无线通信方式为Bluetooth、ZigBee、WiFi无线通讯方式中的一种。Preferably, the wireless communication method is one of Bluetooth, ZigBee, and WiFi wireless communication methods.
为了达到上述第二目的,本发明采用以下技术方案:In order to achieve the above-mentioned second purpose, the present invention adopts the following technical solutions:
一种交互界面中人手漫游控制的穿戴设备的控制方法,包括下述步骤:A control method for a wearable device controlled by human hand roaming in an interactive interface, comprising the following steps:
S1、当设备与主机距离为第一预定范围时,近距无线收发模块自动发送无线信息,与主机进行连接,建立无线通讯,系统进入待机状态;S1. When the distance between the device and the host is within the first predetermined range, the short-range wireless transceiver module automatically sends wireless information, connects with the host, establishes wireless communication, and the system enters the standby state;
S2、人手运动信息获取模块实时获取人手在空间坐标系中的运动信息,所述运动信息包括加速度信息和角速度信息,并发送至数据处理模块;S2. The movement information acquisition module of the human hand acquires the movement information of the human hand in the spatial coordinate system in real time, the movement information includes acceleration information and angular velocity information, and sends it to the data processing module;
S3、当检测到用户抬手和连续两次或两次以上快速转动手腕超过第一预定角度时,则控制权调配模块为其分配控制权,设备进入工作状态,交互界面显示模拟光标;S3. When it is detected that the user raises his hand and rotates his wrist twice or more in a row to exceed the first predetermined angle, the control right allocation module assigns the control right to him, the device enters the working state, and the interactive interface displays a simulated cursor;
S4、人手运动信息获取模块实时获取人手漫游时在空间坐标系中的运动信息,数据处理模块将所述运动信息进行处理分析,并分类识别其为漫游动作,并将识别结果数据发送至主机;S4. The hand movement information acquisition module acquires the movement information in the spatial coordinate system when the hand is roaming in real time, and the data processing module processes and analyzes the movement information, classifies and recognizes it as a roaming action, and sends the recognition result data to the host;
S5、主机接收所述识别结果数据后,将Ω=(ωx,ωy,ωz)通过卡尔曼滤波器处理得到并将其转化为交互界面上模拟光标的模拟坐标并同步显示模拟光标的移动;S5. After the host receives the recognition result data, it processes Ω=(ω x ,ω y ,ω z ) through a Kalman filter to obtain And convert it into the simulated coordinates of the simulated cursor on the interactive interface Simultaneously display the movement of the analog cursor;
S6、当光标漫游至用户所需功能图标时,用户做出弹指动作,人手运动信息获取模块采集人手此时的动作信息;S6. When the cursor roams to the function icon required by the user, the user makes a flick action, and the hand movement information acquisition module collects the movement information of the hand at this time;
S7、数据处理模块对上述运动信息进行分类识别为选中动作,交互界面上即显示为用户所需功能图标被选中;S7. The data processing module classifies and recognizes the above motion information as a selected action, and the interactive interface displays that the function icon required by the user is selected;
S8、重复S2至S7步骤,直至检测到用户手臂自然放下动作以及鼠标处于交互界面边缘超过第一预定时间,或者在第二预定时间内未检测到人手动作,则控制权调配模块取消设备控制权,系统进入待机状态;或者直至检测到步骤S3中所述抬手和连续两次或以上快速转动或左右晃动手腕动作,系统再次进入工作状态。S8. Repeat steps S2 to S7 until it detects that the user's arm is naturally lowered and the mouse is on the edge of the interactive interface for more than the first predetermined time, or no human hand movement is detected within the second predetermined time, then the control right allocation module cancels the device control right , the system enters the standby state; or until it detects the raising of the hand and two or more consecutive quick rotations or shaking the wrist left and right described in step S3, the system enters the working state again.
优选的,步骤S3中,获取设备控制权的方式为:手腕被抬起,且连续两次或以上快速转动手腕超过第一预定角度,一次转动定义为手腕由初始状态沿某一方向轴向转动,再沿相反方向将手腕转动回初始状态。Preferably, in step S3, the way to obtain the control right of the device is: the wrist is lifted, and the wrist is rotated twice or more rapidly to exceed the first predetermined angle, and one rotation is defined as the wrist rotates axially in a certain direction from the initial state , and then turn the wrist back to the original position in the opposite direction.
优选的,数据处理模块将所述运动信息进行处理分析,并分类识别其为漫游动作,具体过程为:Preferably, the data processing module processes and analyzes the motion information, and classifies and identifies it as a roaming action. The specific process is:
S41、数据处理模块将所述运动信息进行A/D采样转化为数字信号,并进行卡尔曼滤波融合处理得到人手运动的九维数据信息,所述九维数据信息包括:三轴加速度信息A=(ax,ay,az),三轴角速度信息Ω=(ωx,ωy,ωz)和三轴姿态角信息Θ=(θx,θy,θz);S41. The data processing module performs A/D sampling on the motion information and converts it into a digital signal, and performs Kalman filter fusion processing to obtain nine-dimensional data information of hand motion. The nine-dimensional data information includes: three-axis acceleration information A= (a x , a y , a z ), three-axis angular velocity information Ω=(ω x ,ω y ,ω z ) and three-axis attitude angle information Θ=(θ x ,θ y ,θ z );
S42、数据处理模块根据上述九维数据信息构建多阶特征张量
S43、在短时分析窗口内,基于单帧多阶张量特征xt构建时序动作识别特征Xt=[xt-n,...,xt],其中n为短时窗口长度;S43. In the short-term analysis window, construct a time-series action recognition feature X t =[x tn ,...,x t ] based on the single-frame multi-level tensor feature x t , where n is the length of the short-term window;
S44、对时序动作识别特征做主成分分析,其中包括去中心化,特征值分解与PCA白化,公式为:S44. Perform principal component analysis on time series action recognition features, including decentralization, eigenvalue decomposition and PCA whitening, the formula is:
其中,为低维子空间映射矩阵,M为时序特征Xt的维度,N为低维子空间映射的维度;in, is the low-dimensional subspace mapping matrix, M is the dimension of the time series feature X t , and N is the dimension of the low-dimensional subspace mapping;
S45、采用多分类支持向量机(SVM)实现对漫游动作、弹指动作和点击动作的分类识别,基于结构风险最小原理训练得到漫游/弹指SVM分类器参数<W1 SVM,b1>,漫游/点击SVM参数<W2 SVM,b1>S45. Using a multi-category support vector machine (SVM) to realize the classification and recognition of roaming actions, flicking actions and clicking actions, and training based on the principle of minimum structural risk to obtain roaming/fingering SVM classifier parameters <W 1 SVM ,b 1 >, roaming/ Click on the SVM parameter <W 2 SVM ,b 1 >
gi(Pt)=PtWi SVM+bi g i (P t )=P t W i SVM +b i
优选的,步骤S7中,所述选中动作为弹指动作,即非拇指的手指在大拇指的按压下快速向外弹出。Preferably, in step S7, the selection action is a flick action, that is, the non-thumb fingers quickly pop out under the pressure of the thumb.
优选的,所述步骤S3,控制权获取动作还可以为:手腕左右晃动动作,即手腕在自然放松状态下,沿同一水平线左右晃动。Preferably, in step S3, the action of obtaining the control right may also be: shaking the wrist left and right, that is, shaking the wrist left and right along the same horizontal line in a naturally relaxed state.
优选的,所述步骤S7中的选中动作,方式还可以为,单指点击动作,即除食指外其余手指做握住状,食指做上下敲击动作。Preferably, the selection action in step S7 can also be a single-finger click action, that is, the rest of the fingers except the index finger perform a holding motion, and the index finger performs an up and down tapping action.
优选的,所述步骤S7中的选中动作,方式还可以为,双指点击动作或多指点击动作,即手腕在自然放松状态下,五指自然张开,食指与中指交替做出点击动作,或除拇指外,其余手指交替做出点击动作。Preferably, the selection action in step S7 can also be a two-finger click action or a multi-finger click action, that is, the wrist is in a naturally relaxed state, the five fingers are naturally opened, and the index finger and middle finger alternately make a click action, or Except for the thumb, the rest of the fingers alternately make clicking movements.
本发明与现有技术相比,具有如下优点和有益效果:Compared with the prior art, the present invention has the following advantages and beneficial effects:
1)本发明一种交互界面中人手漫游控制的穿戴式设备采用加速度传感器与陀螺仪采集人手的运动信息,使用户手势在交互界面上漫游时能得到精确稳定的漫游结果。1) A wearable device controlled by hand roaming in an interactive interface of the present invention uses an acceleration sensor and a gyroscope to collect motion information of the human hand, so that accurate and stable roaming results can be obtained when user gestures roam on the interactive interface.
2)本发明一种交互界面中人手漫游控制的穿戴式设备为非手持式设备,使用时只需佩戴在手腕处即可,因此使用户摆脱了手持交互设备的束缚,解放了人的双手。2) The wearable device controlled by hand roaming in the interactive interface of the present invention is a non-handheld device, which only needs to be worn on the wrist when in use, thus freeing the user from the shackles of the handheld interactive device and liberating the human hands.
3)本发明一种交互界面中人手漫游控制的方法是基于人们在进行人机交互操作时的习惯设计而成,使用户能自然的完成交互操作,不容易出现疲劳。3) A method for manual roaming control in an interactive interface of the present invention is designed based on people's habits in man-machine interactive operations, so that users can naturally complete interactive operations without fatigue.
附图说明Description of drawings
图1为本发明一种交互界面中人手漫游控制的穿戴式设备的结构示意图;Fig. 1 is a schematic structural diagram of a wearable device controlled by hand roaming in an interactive interface of the present invention;
图2为本发明一种交互界面中人手漫游控制的穿戴式设备的系统结构示意图;Fig. 2 is a schematic diagram of the system structure of a wearable device controlled by hand roaming in an interactive interface of the present invention;
图3为本发明一种交互界面中人手漫游控制的方法的流程示意图;3 is a schematic flowchart of a method for manual roaming control in an interactive interface of the present invention;
图4为本发明一种交互界面中人手漫游控制的方法的控制权获取动作示意图;Fig. 4 is a schematic diagram of a control right acquisition action of a method for manual roaming control in an interactive interface of the present invention;
图5为本发明一种交互界面中人手漫游控制的方法的弹指动作示意图;Fig. 5 is a schematic diagram of flicking actions of a method for manual roaming control in an interactive interface of the present invention;
图6为本发明一种交互界面中人手漫游控制的方法的单指点击动作示意图;6 is a schematic diagram of a single-finger click action of a method for manual roaming control in an interactive interface of the present invention;
图7为本发明一种交互界面中人手漫游控制的方法的双指点击动作示意图;7 is a schematic diagram of a two-finger click action of a method for manual roaming control in an interactive interface of the present invention;
图8为本发明一种交互界面中人手漫游控制的方法的多指点击动作示意图。FIG. 8 is a schematic diagram of a multi-finger click action of a method for manual roaming control in an interactive interface according to the present invention.
具体实施方式Detailed ways
下面结合实施例及附图对本发明作进一步详细的描述,但本发明的实施方式不限于此。The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.
实施例Example
本发明的一种交互界面中人手漫游控制的穿戴式设备,包括设备本体与设于设备本体内的电路模块,其中,如图1所示,设备为环状,使用时需佩戴在用户手腕处。A wearable device controlled by hand roaming in an interactive interface of the present invention includes a device body and a circuit module installed in the device body, wherein, as shown in Figure 1, the device is ring-shaped and needs to be worn on the user's wrist when in use .
如图2所示,所述电路模块包括人手运动信息获取模块21、数据处理模块22、控制权调配模块23、近距无线收发模块24,其中:As shown in Figure 2, the circuit module includes a hand movement information acquisition module 21, a data processing module 22, a control right allocation module 23, and a short-range wireless transceiver module 24, wherein:
人手运动信息获取模块21:用于获取手势漫游时人手在空间坐标系中的运动信息,并发送至数据处理模块;Hand movement information acquisition module 21: used to acquire movement information of hands in the spatial coordinate system during gesture roaming, and send it to the data processing module;
数据处理模块22:将所述运动信息进行处理分析,并分类识别其漫游手势动作,将其转化为控制信号并发送至主机;Data processing module 22: process and analyze the motion information, classify and identify its roaming gestures, convert them into control signals and send them to the host;
控制权调配模块23:根据人手运动情况调配控制权,触发或停止漫游操作功能;Control right allocation module 23: Allocate the control right according to the movement of human hands, trigger or stop the roaming operation function;
近距无线收发模块24:用于与主机连接,建立无线通讯,并在建立通讯的基础上将控制信息发送至主机。Short-distance wireless transceiver module 24: used to connect with the host, establish wireless communication, and send control information to the host on the basis of establishing communication.
具体的,所述主机可以为计算机、智能电视等具有较大显示屏的设备,且主机也应有近距无线收发模块以跟本发明穿戴式设备建立无线通讯。Specifically, the host computer can be a device with a large display screen such as a computer, a smart TV, etc., and the host computer should also have a short-range wireless transceiver module to establish wireless communication with the wearable device of the present invention.
进一步的,所述人手运动信息获取模块21包括:Further, the human hand movement information acquisition module 21 includes:
三轴加速度传感器211,用于获取手势漫游时人手在空间坐标系中的线性平移加速度A=(ax,ay,az);The three-axis acceleration sensor 211 is used to obtain the linear translation acceleration A=(a x , a y , a z ) of the human hand in the space coordinate system during gesture roaming;
三轴陀螺仪212,用于获取手势漫游时人手在空间坐标系中的轴向转动角速度Ω=(wx,wy,wz)。The three-axis gyroscope 212 is used to acquire the axial rotation angular velocity Ω=(w x , w y , w z ) of the human hand in the space coordinate system during gesture roaming.
进一步的,所述无线通迅,方式可以为Bluetooth、ZigBee、WiFi等无线通讯方式。Further, the wireless communication method may be Bluetooth, ZigBee, WiFi and other wireless communication methods.
如图3所示,一种交互界面中人手漫游控制的方法,其中,包括以下步骤:As shown in Figure 3, a method for manual roaming control in an interactive interface, which includes the following steps:
S1当设备与主机距离为第一预定范围时,近距无线收发模块24自动发送无线信息,与主机进行连接,建立无线通讯,系统进入待机状态;S1 When the distance between the device and the host is within the first predetermined range, the short-range wireless transceiver module 24 automatically sends wireless information, connects with the host, establishes wireless communication, and the system enters a standby state;
具体应用场景可以如下,本发明穿戴式设备的近距无线收发模块24不断检测附近5米内(假设第一预定范围阈值为5米)是否存在有无线收发模块开启工作的主机,若有,则穿戴式设备的近距无线收发模块24主动与主机的近距无线收发模块连接,建立无线通讯,本发明设备进入待机状态。The specific application scenario can be as follows. The short-range wireless transceiver module 24 of the wearable device of the present invention continuously detects whether there is a host computer with the wireless transceiver module turned on within 5 meters (assuming that the first predetermined range threshold is 5 meters), and if so, wear The short-range wireless transceiver module 24 of the type device actively connects with the short-range wireless transceiver module of the host computer to establish wireless communication, and the device of the present invention enters a standby state.
S2人手运动信息获取模块21实时获取人手在空间坐标系中的运动信息(ax,ay,az和wx,wy,wz),并发送至数据处理模块;S2 The hand movement information acquisition module 21 acquires the movement information (a x , a y , a z and w x , w y , w z ) of the hand in the space coordinate system in real time, and sends it to the data processing module;
S3当检测到用户抬手和连续两次(或以上)快速转动手腕超过第一预定角度时(如图4所示),则控制权调配模块23为其分配控制权,设备进入工作状态,交互界面显示模拟光标;S3 When it is detected that the user raises his hand and rotates his wrist twice (or more) rapidly to exceed the first predetermined angle (as shown in Figure 4), the control right allocation module 23 assigns the control right to it, and the device enters the working state, and the interaction The interface displays an analog cursor;
具体应用场景可以如下,用户想要用本发明穿戴式设备在交互界面上进行漫游时,则只需要将手从自然下垂状态抬起到一舒适的位置,并连续两次(或以上)快速转动手腕超过90°(假设第一预定角度阈值为90°),人手运动信息获取模块21采集到此动作信息后发送至数据处理模块22,数据处理模块02对所述动作信息进行处理分析,并分类识别其为控制权获取动作,并将识别结果发送至控制权调配模块23为该穿戴式设备分配控制权,设备进入工作状态,交互界面中心显示初始模拟光标,如图4所示。The specific application scenario can be as follows. When the user wants to use the wearable device of the present invention to roam on the interactive interface, he only needs to lift his hand from the natural drooping state to a comfortable position, and rotate it twice (or more) quickly When the wrist exceeds 90° (assuming that the first predetermined angle threshold is 90°), the movement information acquisition module 21 of the human hand collects the movement information and sends it to the data processing module 22. The data processing module 02 processes and analyzes the movement information and classifies it. It is recognized as a control right acquisition action, and the recognition result is sent to the control right allocation module 23 to assign control rights to the wearable device, the device enters the working state, and the initial simulation cursor is displayed in the center of the interactive interface, as shown in FIG. 4 .
若用户只抬手而未转动手腕,或者用户只转动手腕一次,或者用户每次转动手腕都不超过90°,则系统都认为不是控制权获取动作,因此也不会分配控制权给当前设备。If the user only raises the hand but does not turn the wrist, or the user only turns the wrist once, or the user does not turn the wrist more than 90° each time, the system considers it is not a control right acquisition action, so the control right will not be assigned to the current device.
S4人手运动信息获取模块21实时获取人手漫游时在空间坐标系中的运动信息(ax,ay,az和wx,wy,wz),数据处理模块22将所述运动信息进行处理分析,并分类识别其为漫游动作,并将识别结果数据发送至主机;S4 The hand movement information acquisition module 21 acquires the movement information (a x , a y , a z and w x , w y , w z ) in the spatial coordinate system when the hand roams in real time, and the data processing module 22 performs the movement information on the Process and analyze, classify and identify it as a roaming action, and send the identification result data to the host;
S5主机接收所述识别结果数据后,将其转化为交互界面上模拟光标的模拟坐标,并同步显示模拟光标的移动;After receiving the recognition result data, the S5 host converts it into the simulated coordinates of the simulated cursor on the interactive interface, and simultaneously displays the movement of the simulated cursor;
S6当光标漫游至用户所需功能图标时,用户做出弹指动作,如图5所示,人手运动信息获取模块21采集人手此时的动作信息;S6 When the cursor roams to the desired function icon of the user, the user makes a flicking action, as shown in Figure 5, the movement information of the hand movement information acquisition module 21 collects the action information of the hand at this moment;
S7数据处理模块22对上述运动信息进行分类识别为选中动作,交互界面上即显示为用户所需功能图标被选中;S7 data processing module 22 classifies and recognizes the above-mentioned motion information as a selected action, and the user's required function icon is selected on the interactive interface;
S8重复S2至S7步骤,直至检测到用户手臂自然放下动作以及鼠标处于交互界面边缘超过第一预定时间,或者在第二预定时间内未检测到人手动作,则控制权调配模块23取消设备控制权,系统进入待机状态,直至检测到步骤S3中所述抬手和连续两次(或以上)快速转动手腕动作,系统再次进入工作状态;S8 Repeat steps S2 to S7 until it detects that the user's arm is naturally lowered and the mouse is on the edge of the interactive interface for more than the first predetermined time, or no human hand motion is detected within the second predetermined time, then the control right allocation module 23 cancels the device control right , the system enters the standby state until it detects the raising of the hand and two consecutive (or more) quick rotations of the wrist described in step S3, and the system enters the working state again;
具体应用场景可以如下,用户由于某一因素,例如想要详细浏览交互界面内容,暂时不需要光标漫游与选中操作,则可以将手臂自然放下,设备检测到这一动作之后判断其为用户暂时不需要漫游操作与选中操作;同时,若用户快速将模拟光标移动至交互界面边缘超过3秒(假设第一预定时间阈值为3秒),或者设备在15秒(假设第二预定时间阈值为15秒)内未检测到用户手的动作,则设备判断用户暂时不需要漫游与选中操作,则系统进入待机状态。若用户需要再次获得控制权,则应该做抬起手腕动作,且连续两次(或以上)快速转动手腕超过90°,设备检测到这一动作之后则系统再次进入工作状态。The specific application scenario can be as follows. Due to a certain factor, such as wanting to browse the content of the interactive interface in detail, the user does not need the cursor roaming and selection operations for the time being, so the user can put down the arm naturally. After detecting this action, the device judges that the user is not temporarily Roaming and selection operations are required; at the same time, if the user quickly moves the simulated cursor to the edge of the interface for more than 3 seconds (assuming the first predetermined time threshold is 3 seconds), or the device is within 15 seconds (assuming the second predetermined time threshold is 15 seconds) ), the device judges that the user does not need roaming and selection operations for the time being, and the system enters the standby state. If the user needs to gain control again, he should raise his wrist and turn his wrist more than 90° two times (or more) in a row. After the device detects this action, the system will enter the working state again.
需要说明的是,上述记载的第一预定时间阈值为3秒和第二预定时间阈值为15秒仅仅作为一种示例,并不构成不当的限定,本发明实施例中第一预定时间阈值和第二预定时间阈值可以为相同,如都是15秒。It should be noted that the first predetermined time threshold value of 3 seconds and the second predetermined time threshold value of 15 seconds described above are only an example and do not constitute an improper limitation. In the embodiment of the present invention, the first predetermined time threshold value and the second predetermined time threshold value The two predetermined time thresholds may be the same, for example, both are 15 seconds.
进一步的,获取设备控制权的方式为:手腕被抬起,且连续两次(或以上)快速转动手腕超过第一预定角度,一次转动定义为手腕由初始状态沿某一方向轴向转动,再沿相反方向将手腕转动回初始状态。Further, the way to obtain the control right of the device is as follows: the wrist is lifted, and the wrist is rotated twice (or more) quickly to exceed the first predetermined angle. One rotation is defined as the wrist rotates axially in a certain direction from the initial state, and then Turn your wrist back to the original position in the opposite direction.
进一步的,所述步骤S5,即数据处理模块02对上述运动信息进行分类识别,具体过程为:Further, the step S5, that is, the data processing module 02 classifies and recognizes the above-mentioned motion information, and the specific process is as follows:
S51数据处理模块02将所述运动信息进行A/D采样转化为数字信号,并进行卡尔曼滤波融合处理得到人手运动的九维数据信息(三轴加速度信息A=(ax,ay,az),三轴角速度信息Ω=(wx,wy,wz)和三轴姿态角信息Θ=(θx,θy,θz));S51 Data processing module 02 performs A/D sampling on the motion information and converts it into a digital signal, and performs Kalman filter fusion processing to obtain nine-dimensional data information of human hand motion (three-axis acceleration information A=(a x , a y , a z ), three-axis angular velocity information Ω=(w x , w y , w z ) and three-axis attitude angle information Θ=(θ x , θ y , θ z ));
S52数据处理模块02根据上述九维数据信息构建多阶特征张量
S53在短时分析窗口内,基于单帧多阶张量特征xt构建时序动作识别特征Xt=[xt-n,...,xt]其中n为短时窗口长度;S53 In the short -term analysis window, construct a time-series action recognition feature X t = [ x tn , .
S54对时序动作识别特征做主成分分析(PCA),其中包括去中心化,特征值分解与PCA白化。S54 performs principal component analysis (PCA) on the temporal action recognition features, including decentralization, eigenvalue decomposition and PCA whitening.
其中,M为时序特征Xt的维度,N为低维子空间映射的维度;Among them, M is the dimension of time series feature X t , and N is the dimension of low-dimensional subspace mapping;
S55采用支持向量机(Kernel SVM)实现对漫游动作、弹指动作和点击动作的分类识别;S55 adopts support vector machine (Kernel SVM) to realize the classification and recognition of roaming action, flick action and click action;
进一步的,所述选中动作为弹指动作,如图5所示,食指(或其余非拇指的手指)在大拇指的按压下快速向外弹出。Further, the selection action is a flick action, as shown in FIG. 5 , the index finger (or other non-thumb fingers) quickly pops out under the pressure of the thumb.
进一步的,所述步骤S3,控制权获取动作还可以为,手腕左右晃动动作,即手腕在自然放松状态下,沿同一水平线左右晃动。Further, in the step S3, the action of obtaining the control right may also be an action of shaking the wrist left and right, that is, shaking the wrist left and right along the same horizontal line in a naturally relaxed state.
进一步的,所述步骤S8中选中动作,方式还可以为,单指点击动作,即除食指外其余手指做握住状,食指做上下敲击动作,如图6所示。Further, the action of selecting in step S8 may also be a single-finger click action, that is, the other fingers except the index finger perform a holding motion, and the index finger performs an up and down tapping action, as shown in FIG. 6 .
进一步的,所述步骤S8中选中动作,方式还可以为,双指点击动作或多指点击动作,即手腕在自然放松状态下,五指自然张开,食指与中指交替做出点击动作,或除拇指外,其余手指交替做出点击动作。Further, the action selected in step S8 can also be a two-finger click action or a multi-finger click action, that is, the wrist is in a naturally relaxed state, the five fingers are naturally opened, and the index finger and middle finger alternately make a click action, or except Except for the thumb, the rest of the fingers alternately make clicking movements.
综上,本发明公开了一种交互界面中人手漫游控制的穿戴式设备和方法。可穿戴设备呈环状,使用时需佩戴在用户手腕处,该设备具体包含人手运动信息获取模块、数据处理模块、控制权调配模块和近距无线收发模块。一种交互界面人手漫游控制的方法为,设备与主机建立无线通讯,用户通过抬手和连续两次(或以上)快速转动手腕动作或左右晃动手腕动作来获取设备控制权,系统实时采集人手运动信息并在交互界面上显示为模拟光标的移动,用户通过弹指或单指点击或多指点击动作选中交互界面上所需功能图标,如图7、图8所示。In summary, the present invention discloses a wearable device and method for manual roaming control in an interactive interface. The wearable device is ring-shaped and needs to be worn on the user's wrist when in use. The device specifically includes a human hand movement information acquisition module, a data processing module, a control right allocation module and a short-range wireless transceiver module. A method for hand roaming control of an interactive interface is that the device establishes wireless communication with the host, and the user obtains the control right of the device by raising the hand and turning the wrist twice (or more) quickly or shaking the wrist left and right, and the system collects the movement of the human hand in real time The information is displayed on the interactive interface as simulating the movement of the cursor, and the user selects the desired function icon on the interactive interface by flicking or single-finger clicking or multi-finger clicking, as shown in Figure 7 and Figure 8.
本发明实施方式提供的一种交互界面中人手漫游控制的穿戴式设备和方法,至少具有如下有益效果:A wearable device and method for manual roaming control in an interactive interface provided by an embodiment of the present invention has at least the following beneficial effects:
1)本发明一种交互界面中人手漫游控制的穿戴式设备采用加速度传感器与陀螺仪采集人手的运动信息,使用户手势在交互界面上漫游时能得到精确稳定的漫游结果;1) A wearable device controlled by hand roaming in an interactive interface of the present invention uses an acceleration sensor and a gyroscope to collect motion information of the human hand, so that user gestures can obtain accurate and stable roaming results when roaming on the interactive interface;
2)本发明一种交互界面中人手漫游控制的穿戴式设备为非手持式设备,使用时只需佩戴在手腕处即可,因此使用户摆脱了手持交互设备的束缚,解放了人的双手;2) The wearable device controlled by human hand roaming in an interactive interface of the present invention is a non-handheld device, which only needs to be worn on the wrist when in use, so that the user is freed from the shackles of the handheld interactive device and liberates the human hands;
3)本发明一种交互界面中人手漫游控制的方法是基于人们在进行人机交互操作时的习惯设计而成,使用户能自然的完成交互操作,不容易出现疲劳;3) The method for manual roaming control in an interactive interface of the present invention is designed based on people's habits when performing human-computer interactive operations, so that users can naturally complete interactive operations without fatigue;
应注意的是上述实施例对本发明进行说明而不是对本发明进行限制,并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中,不应将位于括号之间的任何参考符号构成对权利要求的限制。单词“包括”不排除存在未列在权利要求中的器件与步骤。本领域的技术人员在不脱离本发明精神和范围的前提下可作若干的更动与润饰,本发明所主张的保护范围应以权利要求书所述为准。It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements and steps not listed in a claim. Those skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention, and the scope of protection claimed by the present invention should be based on the claims.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510076982.7A CN104679246B (en) | 2015-02-11 | 2015-02-11 | The Wearable and control method of human hand Roaming control in a kind of interactive interface |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510076982.7A CN104679246B (en) | 2015-02-11 | 2015-02-11 | The Wearable and control method of human hand Roaming control in a kind of interactive interface |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104679246A true CN104679246A (en) | 2015-06-03 |
CN104679246B CN104679246B (en) | 2017-10-20 |
Family
ID=53314440
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510076982.7A Active CN104679246B (en) | 2015-02-11 | 2015-02-11 | The Wearable and control method of human hand Roaming control in a kind of interactive interface |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104679246B (en) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105068462A (en) * | 2015-09-02 | 2015-11-18 | 肖继林 | Pull rod lifting motor control mechanism |
CN105204579A (en) * | 2015-09-27 | 2015-12-30 | 孙博 | Modular integrated-type host device |
CN105259885A (en) * | 2015-11-18 | 2016-01-20 | 马群毅 | Internet type workshop equipment operating system |
CN105425632A (en) * | 2015-11-18 | 2016-03-23 | 马群毅 | Internet man-machine interaction system |
CN106293131A (en) * | 2016-08-16 | 2017-01-04 | 广东小天才科技有限公司 | expression input method and device |
CN106354378A (en) * | 2015-07-13 | 2017-01-25 | 阿里巴巴集团控股有限公司 | Method and device for quickly selecting multiple targets |
WO2017012362A1 (en) * | 2015-07-22 | 2017-01-26 | 深圳多新哆技术有限责任公司 | Method and apparatus for adjusting attitude angle of virtual object in virtual space |
WO2017028527A1 (en) * | 2015-07-22 | 2017-02-23 | 深圳多新哆技术有限责任公司 | Method and apparatus for determining touch event in virtual space |
CN106502416A (en) * | 2016-11-09 | 2017-03-15 | 华南理工大学广州学院 | A kind of driving simulation system of Intelligent Recognition bimanual input and its control method |
CN106512391A (en) * | 2016-11-09 | 2017-03-22 | 华南理工大学广州学院 | Two-hand gesture recognition method, and simulation driving system and method based on two-hand gesture recognition method |
CN106990841A (en) * | 2017-03-30 | 2017-07-28 | 无锡云瞳信息科技有限公司 | Gesture identification method and the wearable video camera of intelligence based on motion sensor |
WO2018161906A1 (en) * | 2017-03-09 | 2018-09-13 | 腾讯科技(深圳)有限公司 | Motion recognition method, device, system and storage medium |
CN108900698A (en) * | 2018-05-31 | 2018-11-27 | 努比亚技术有限公司 | Method, wearable device, terminal and the computer storage medium of controlling terminal |
CN109074709A (en) * | 2016-05-05 | 2018-12-21 | 摩托罗拉解决方案公司 | Message is shown using multiple body worn electronic display units |
CN109409427A (en) * | 2018-10-25 | 2019-03-01 | 珠海市君天电子科技有限公司 | A kind of key detecting method and device |
CN109976518A (en) * | 2019-03-12 | 2019-07-05 | 合肥工业大学 | A kind of man-machine interaction method based on PVDF sensor array |
CN110046009A (en) * | 2019-02-19 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Method for recording, device, server and readable storage medium storing program for executing |
CN111367356A (en) * | 2018-12-24 | 2020-07-03 | 群光电子股份有限公司 | Method for controlling electronic device by tapping and tapping control group |
CN112346559A (en) * | 2020-09-29 | 2021-02-09 | 广东乐芯智能科技有限公司 | Application interface control system |
CN112346560A (en) * | 2020-09-29 | 2021-02-09 | 广东乐芯智能科技有限公司 | Method for controlling application interface by bracelet |
WO2022067963A1 (en) * | 2020-09-29 | 2022-04-07 | 泰州翔升科技服务有限公司 | Computer control method and wearable device |
CN117786643A (en) * | 2024-02-22 | 2024-03-29 | 清华大学 | User identity authentication method and related equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060028429A1 (en) * | 2004-08-09 | 2006-02-09 | International Business Machines Corporation | Controlling devices' behaviors via changes in their relative locations and positions |
US20100124949A1 (en) * | 2008-11-14 | 2010-05-20 | Sony Ericsson Mobile Communications Ab | Portable communication device and remote motion input device |
US20120127070A1 (en) * | 2010-11-22 | 2012-05-24 | Electronics And Telecommunications Research Institute | Control signal input device and method using posture recognition |
CN103186256A (en) * | 2011-12-31 | 2013-07-03 | 北京德信互动网络技术有限公司 | Human-machine interaction system |
CN103513788A (en) * | 2013-09-13 | 2014-01-15 | 广东欧珀移动通信有限公司 | Gesture recognition method and system based on gyroscope sensor and mobile terminal |
CN103984416A (en) * | 2014-06-10 | 2014-08-13 | 北京邮电大学 | Gesture recognition method based on acceleration sensor |
-
2015
- 2015-02-11 CN CN201510076982.7A patent/CN104679246B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060028429A1 (en) * | 2004-08-09 | 2006-02-09 | International Business Machines Corporation | Controlling devices' behaviors via changes in their relative locations and positions |
US20100124949A1 (en) * | 2008-11-14 | 2010-05-20 | Sony Ericsson Mobile Communications Ab | Portable communication device and remote motion input device |
US20120127070A1 (en) * | 2010-11-22 | 2012-05-24 | Electronics And Telecommunications Research Institute | Control signal input device and method using posture recognition |
CN103186256A (en) * | 2011-12-31 | 2013-07-03 | 北京德信互动网络技术有限公司 | Human-machine interaction system |
CN103513788A (en) * | 2013-09-13 | 2014-01-15 | 广东欧珀移动通信有限公司 | Gesture recognition method and system based on gyroscope sensor and mobile terminal |
CN103984416A (en) * | 2014-06-10 | 2014-08-13 | 北京邮电大学 | Gesture recognition method based on acceleration sensor |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106354378A (en) * | 2015-07-13 | 2017-01-25 | 阿里巴巴集团控股有限公司 | Method and device for quickly selecting multiple targets |
WO2017012362A1 (en) * | 2015-07-22 | 2017-01-26 | 深圳多新哆技术有限责任公司 | Method and apparatus for adjusting attitude angle of virtual object in virtual space |
WO2017028527A1 (en) * | 2015-07-22 | 2017-02-23 | 深圳多新哆技术有限责任公司 | Method and apparatus for determining touch event in virtual space |
CN105068462A (en) * | 2015-09-02 | 2015-11-18 | 肖继林 | Pull rod lifting motor control mechanism |
CN105204579A (en) * | 2015-09-27 | 2015-12-30 | 孙博 | Modular integrated-type host device |
CN105259885A (en) * | 2015-11-18 | 2016-01-20 | 马群毅 | Internet type workshop equipment operating system |
CN105425632A (en) * | 2015-11-18 | 2016-03-23 | 马群毅 | Internet man-machine interaction system |
CN109074709B (en) * | 2016-05-05 | 2019-11-15 | 摩托罗拉解决方案公司 | Message is shown using multiple body worn electronic display units |
CN109074709A (en) * | 2016-05-05 | 2018-12-21 | 摩托罗拉解决方案公司 | Message is shown using multiple body worn electronic display units |
CN106293131A (en) * | 2016-08-16 | 2017-01-04 | 广东小天才科技有限公司 | expression input method and device |
CN106512391A (en) * | 2016-11-09 | 2017-03-22 | 华南理工大学广州学院 | Two-hand gesture recognition method, and simulation driving system and method based on two-hand gesture recognition method |
CN106502416A (en) * | 2016-11-09 | 2017-03-15 | 华南理工大学广州学院 | A kind of driving simulation system of Intelligent Recognition bimanual input and its control method |
CN106512391B (en) * | 2016-11-09 | 2019-07-12 | 华南理工大学广州学院 | A kind of bimanual input recognition methods and the driving simulation system based on it, method |
CN106502416B (en) * | 2016-11-09 | 2019-04-02 | 华南理工大学广州学院 | A kind of driving simulation system and its control method of intelligent recognition bimanual input |
WO2018161906A1 (en) * | 2017-03-09 | 2018-09-13 | 腾讯科技(深圳)有限公司 | Motion recognition method, device, system and storage medium |
CN106990841A (en) * | 2017-03-30 | 2017-07-28 | 无锡云瞳信息科技有限公司 | Gesture identification method and the wearable video camera of intelligence based on motion sensor |
CN108900698A (en) * | 2018-05-31 | 2018-11-27 | 努比亚技术有限公司 | Method, wearable device, terminal and the computer storage medium of controlling terminal |
CN109409427A (en) * | 2018-10-25 | 2019-03-01 | 珠海市君天电子科技有限公司 | A kind of key detecting method and device |
CN111367356A (en) * | 2018-12-24 | 2020-07-03 | 群光电子股份有限公司 | Method for controlling electronic device by tapping and tapping control group |
CN110046009A (en) * | 2019-02-19 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Method for recording, device, server and readable storage medium storing program for executing |
CN110046009B (en) * | 2019-02-19 | 2022-08-23 | 创新先进技术有限公司 | Recording method, recording device, server and readable storage medium |
CN109976518A (en) * | 2019-03-12 | 2019-07-05 | 合肥工业大学 | A kind of man-machine interaction method based on PVDF sensor array |
CN112346559A (en) * | 2020-09-29 | 2021-02-09 | 广东乐芯智能科技有限公司 | Application interface control system |
CN112346560A (en) * | 2020-09-29 | 2021-02-09 | 广东乐芯智能科技有限公司 | Method for controlling application interface by bracelet |
WO2022067963A1 (en) * | 2020-09-29 | 2022-04-07 | 泰州翔升科技服务有限公司 | Computer control method and wearable device |
CN117786643A (en) * | 2024-02-22 | 2024-03-29 | 清华大学 | User identity authentication method and related equipment |
CN117786643B (en) * | 2024-02-22 | 2024-05-07 | 清华大学 | A user identity authentication method and related equipment |
Also Published As
Publication number | Publication date |
---|---|
CN104679246B (en) | 2017-10-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104679246B (en) | The Wearable and control method of human hand Roaming control in a kind of interactive interface | |
CN113608665B (en) | Application for multi-touch input detection | |
US11543887B2 (en) | User interface control of responsive devices | |
US11567573B2 (en) | Neuromuscular text entry, writing and drawing in augmented reality systems | |
Moazen et al. | AirDraw: Leveraging smart watch motion sensors for mobile human computer interactions | |
KR102407071B1 (en) | Multi-device multi-user sensor correlation for pen and computing device interaction | |
EP3090331B1 (en) | Systems with techniques for user interface control | |
US10042457B2 (en) | Watch type terminal | |
CN106462341B (en) | Sensor correlation for pen and touch sensitive computing device interaction | |
US10289239B2 (en) | Application programming interface for multi-touch input detection | |
CN104780217B (en) | Detect method, system and the client of user job efficiency | |
EP2933709A2 (en) | Haptic information management method and electronic device supporting the same | |
US20150323998A1 (en) | Enhanced user interface for a wearable electronic device | |
US20160179210A1 (en) | Input supporting method and input supporting device | |
US20160124579A1 (en) | Controlling multiple devices with a wearable input device | |
CN106164808A (en) | Equipment and the method for equipment is calculated for ring | |
US20170010695A1 (en) | Enhanced multi-touch input detection | |
CN107390867A (en) | A kind of man-machine interactive system based on Android wrist-watch | |
US20170010733A1 (en) | User-identifying application programming interface (api) | |
JP2010086367A (en) | Positional information inputting device, positional information inputting method, program, information processing system, and electronic equipment | |
JP5605281B2 (en) | WRITE INFORMATION DISPLAY SYSTEM, WRITE INFORMATION DISPLAY DEVICE, AND WRITE INFORMATION DISPLAY METHOD | |
JP6523509B1 (en) | Game program, method, and information processing apparatus | |
Sajjadi et al. | Airdraw: Leveraging smart watch motion sensors for mobile human computer interactions | |
CN113849066A (en) | Information barrier-free interaction device, system, method and storage medium | |
Wu et al. | Infofinder: just-in-time information interface from the combination of an hwd with a smartwatch |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20181213 Address after: 510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province Patentee after: Guangzhou Boguanwen Language Technology Co., Ltd. Address before: 510640 No. five, 381 mountain road, Guangzhou, Guangdong, Tianhe District Patentee before: South China University of Technology |
|
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20190926 Address after: 510000 820, room 8, 8, 116 Heng Road, Dongguan, Guangzhou, Guangdong. Patentee after: Guangzhou Bo Wei Intelligent Technology Co., Ltd. Address before: 510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province Patentee before: Guangzhou Boguanwen Language Technology Co., Ltd. |
|
TR01 | Transfer of patent right |