WO2017084253A1 - 一种应用于头戴设备的控制方法及头戴设备 - Google Patents

一种应用于头戴设备的控制方法及头戴设备 Download PDF

Info

Publication number
WO2017084253A1
WO2017084253A1 PCT/CN2016/081916 CN2016081916W WO2017084253A1 WO 2017084253 A1 WO2017084253 A1 WO 2017084253A1 CN 2016081916 W CN2016081916 W CN 2016081916W WO 2017084253 A1 WO2017084253 A1 WO 2017084253A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
motion track
action
cursor
light
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/081916
Other languages
English (en)
French (fr)
Inventor
陈相金
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Original Assignee
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Le Holdings Beijing Co Ltd, Leshi Zhixin Electronic Technology Tianjin Co Ltd filed Critical Le Holdings Beijing Co Ltd
Publication of WO2017084253A1 publication Critical patent/WO2017084253A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the present invention relates to the field of device control technologies, and in particular, to a control method and a head mounted device applied to a head mounted device.
  • a headset is at least a head-mounted display device that gives the user a virtual reality experience.
  • other features can be integrated on the headset.
  • the game application as an example, the game console, audio, etc. can all be integrated into the headset.
  • a keyboard, a mouse, a remote controller, and the like are generally used for control. Specifically, the cursor is moved on the display interface of the display screen of the head wear device through a keyboard, a mouse, a remote controller, and the like, and operations such as dragging and clicking are performed.
  • an unexpected result may be generated by inputting an incorrect command, such as pressing an incorrect button, causing an abnormality in the current application (for example, a currently running game), in severe cases. May cause the system to run incorrectly.
  • the embodiment of the invention provides a control method and a head-wearing device applied to a head-mounted device, which are used to solve the problem that the user needs to explore and operate the control in the prior art, and the risk of misoperation is high, which may result in The current application or even the system is running incorrectly.
  • An embodiment of the present invention provides a control method applied to a headset, including:
  • gesture action is a control action
  • determining that the cursor is located in a display area to which the control action acts according to the cursor motion trajectory performing a control operation corresponding to the control action.
  • An embodiment of the present invention provides a headset device, including:
  • a gesture information sensor for acquiring gesture information
  • a processor configured to determine a gesture motion and a gesture motion track according to the gesture information acquired by the gesture information sensor; determine a cursor motion track on the display interface of the display screen according to the gesture motion track; if the gesture motion is a control action, and according to The cursor movement trajectory determines that the cursor is located in a display area in which the control action is applied, and performs a control operation corresponding to the control action;
  • Display for providing a display interface.
  • control method and the head-wearing device applied to the head-wearing device provided by the embodiment of the present invention because of the control by the gesture recognition technology, do not require the user to explore the operation, thereby reducing the risk of misoperation, thereby improving the system performance.
  • FIG. 1 is a flow chart of an embodiment of a control method applied to a headwear device according to the present invention
  • FIG. 2 is a schematic structural view of an embodiment of a headset device according to the present invention.
  • FIG. 3 is a schematic diagram of a working principle of a gesture information sensor according to an embodiment of the present invention.
  • FIG. 1 is a schematic diagram of a control method applied to a headset device according to an embodiment of the present invention, where the method includes:
  • Step 100 Determine a gesture motion and a gesture motion track according to the acquired gesture information.
  • a head-mounted device is a component of a game device or a game device. Then, the user wearing the headset makes gestures during the game to manipulate the game application, such as adjusting the volume, controlling the motion of the game character, and the like.
  • the gesture information corresponding to the gesture action made by the user during the game can be obtained by using the gesture recognition technology. In the implementation of the step, the gesture gesture and the gesture gesture made by the user during the game can be determined according to the acquired gesture information. Track.
  • the gesture information corresponding to the gesture motion corresponding to the user's right-to-left swipe gesture can be obtained by the gesture recognition technology, and the gesture can be performed according to the gesture in step 100.
  • the information determines the gesture action of sliding, and determines that the gesture motion trajectory is a right-to-left trajectory.
  • Step 110 Determine a cursor motion track on the display interface of the headset according to the gesture motion track.
  • the cursor on the display interface can be controlled to move according to the determined motion track.
  • those skilled in the art can easily set the correspondence between the gesture motion track and the cursor motion track on the display interface.
  • the most intuitive way is that when the gesture motion trajectory is a right-to-left trajectory, then in step 110, the cursor motion trajectory on the display interface of the headset is determined to be a right-to-left trajectory on the screen.
  • the display trajectory can also be set to a left to right or top to bottom trajectory.
  • the shape of the cursor on the display interface of the headwear device is not limited.
  • the shape of the cursor can be an arrow, a blinking line segment, a shape of a hand, and the like.
  • Step 120 If the gesture action is a control action, and determining that the cursor is located in the display area affected by the control action according to the cursor motion track, performing a control operation corresponding to the control action.
  • the cursor motion track is determined only according to the gesture motion track Trace, and then control the cursor movement.
  • each control action corresponds to a display area.
  • the display area is also referred to as a display area to which the control action acts.
  • each control action corresponds to one control operation.
  • the control method applied to the headset device provided by the embodiment of the invention because of the control by the gesture recognition technology, does not require the user to explore the operation, reduces the risk of misoperation, and further improves the system performance.
  • the manner of representing the gesture information is various, for example, it can be represented by the depth information image of the gesture.
  • the implementation of the foregoing step 100 may be: determining a gesture action and a gesture motion track by performing gesture recognition on the depth information image of the acquired gesture.
  • the coordinates of the key point in the three-dimensional coordinate system are acquired from the depth information image of each frame of the gesture, and the gesture motion and the gesture motion track are determined according to the coordinates of the acquired key point.
  • the key points can be, but are not limited to, including: the coordinates of each finger, the palm coordinates, the arm coordinates, and the like.
  • whether the gesture action is completed can be determined by determining whether the coordinate change of the same key point in the depth information image within the predetermined number is within the set threshold.
  • the depth information image of the gesture may be obtained by analyzing, by the light receiver, the received light intensity, and the light received by the light receiver is emitted by the light emitter and reflected by the occlusion object. It should be noted that the depth information image of the gesture may also be acquired by a multi-view stereo camera such as a binocular stereo camera.
  • the light receiver can be either a laser receiver or an infrared receiver.
  • the determined gesture motion track may be valid before determining the cursor motion track on the display interface of the headset according to the gesture motion track.
  • the gesture motion trajectory template is matched; if the matching is successful, the above step 110 is to determine a cursor motion trajectory on the display interface of the headset according to the gesture motion track that is successfully matched. If the match is not successful, the corresponding action is not triggered.
  • the specific content and quantity of the effective motion trajectory template can be set according to actual needs, which is not limited by the present invention.
  • template matching is taken as an example because the method is relatively easy to implement, so the template matching is taken as an example here; however, as long as the implementation method capable of recognizing the invalid gesture motion track is applicable to this embodiment
  • the embodiments of the present invention are not limited to the manner of template matching.
  • the manner of template matching is only used to teach the person skilled in the art how to implement the present invention, but it does not mean that only the method can be used, and the implementation process can be determined in combination with practical needs. The corresponding way.
  • the determined gesture action can be matched with the control action template, and if matched, the gesture action is a control action.
  • control action template can be set according to actual needs, which is not limited by the present invention.
  • template matching is taken as an example because the method is relatively easy to implement, so template matching is taken as an example here; however, any implementation that can recognize the control action is applicable to the implementation of the present invention.
  • confirmation by means of template matching the manner of template matching is only used to teach those skilled in the art how to implement the present invention in detail, but it does not mean that only this method can be used, and the implementation process can be combined with practical needs to determine corresponding the way. .
  • the embodiment of the present invention further provides a head-wearing device, as shown in FIG. 2, including:
  • a gesture information sensor 201 configured to acquire gesture information
  • the processor 202 is configured to determine a gesture motion and a gesture motion track according to the gesture information acquired by the gesture information sensor 201, and determine a cursor motion track on the display interface of the display screen 203 according to the gesture motion track; if the gesture action is a control action And determining, according to the cursor motion trajectory, that the cursor is located in a display area in which the control action is applied, performing a control operation corresponding to the control action;
  • the display screen 203 is configured to provide a display interface.
  • the headset device provided by the embodiment of the present invention does not require the user to explore the operation because the gesture recognition technology is used for control, thereby reducing the risk of misoperation, thereby improving system performance.
  • the gesture information sensor may be a light emitter and a light receiver.
  • the light emitter is used to emit light;
  • the light receiver is used to analyze the received light intensity to obtain a depth information image of the gesture, and the light received by the light receiver is emitted by the light emitter and reflected by the obstruction.
  • the light emitter is a laser emitter
  • the light receiver is a laser receiver
  • the light emitter is an infrared emitter
  • the light receiver is an infrared receiver.
  • the light emitter is composed of a light source and a grating disposed in a light exiting direction of the light source.
  • gesture information sensor can also be implemented by other devices capable of acquiring gesture information, such as a multi-view camera.
  • the gesture information sensor and the display screen are located on the same side of the headset.
  • an infrared emitter and an infrared receiver are disposed on the headset and on the same side of the display screen.
  • the infrared emitter is composed of an infrared light source and a grating.
  • the memory of the headset stores a correspondence between a plurality of control actions and a display area to which it acts, and a correspondence relationship with the control operation.
  • the action of the finger press acts on the display area where all the clickable button icons are located, and corresponds to the click operation
  • the action of the finger lift acts on the display area where all the clickable button icons are located, and corresponds to the operation of releasing the button.
  • a plurality of effective gesture motion track templates are also stored in the memory of the headset.
  • a plurality of control action templates are also stored in the memory of the headset.
  • the infrared emitter emits infrared light
  • the human body will reflect on the optical fiber in the infrared light projection range.
  • the infrared receiver collects light of different intensity, thereby generating Depth information image. It is assumed that the infrared receiver generates a frame of depth information image every millisecond in units of 1 millisecond.
  • the processor receives the depth information image generated by the infrared receiver, and determines the gesture motion and the gesture motion trajectory accordingly. Specifically, the processor analyzes the coordinates of the key points in the depth information image of each frame, and determines the gesture action and the gesture track according to the change of the coordinates of the key points.
  • the processor compares the determined gesture motion trajectory with each effective gesture motion trajectory template Match. If the matching is successful, for example, matching the effective gesture motion track template of the linear motion, the subsequent process is continued, otherwise the motion track is not responded to.
  • the processor determines a cursor motion trajectory on the display interface of the display screen according to the gesture motion track that matches the success. Wherein, if the shape of the cursor is the shape of the hand, the action of the hand displayed on the display interface can also be controlled according to the recognized gesture action.
  • the processor matches the determined gesture action with each control action template. If the matching is successful, for example, matching the control action template pressed by the finger, and the cursor is located in the display area to which the gesture action is applied, the control operation corresponding to the gesture action is performed, such as clicking the button icon. If the match is unsuccessful, or the cursor is not located in the display area where the gesture action is applied, the gesture action is not responded.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

一种应用于头戴设备的控制方法及头戴设备。包括：根据获取到的手势信息确定手势动作和手势运动轨迹；根据所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹；如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作。由于本发明通过手势识别技术进行控制，不需要用户摸索操作，降低了误操作的风险，进而提高了系统性能。

Description

一种应用于头戴设备的控制方法及头戴设备
本申请要求在2015年11月20日提交中国专利局、申请号为201510811932.9、发明名称为“一种应用于头戴设备的控制方法及头戴设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及设备控制技术领域，尤其涉及一种应用于头戴设备的控制方法及头戴设备。
背景技术
头戴设备至少是指头戴式显示设备，使用户有虚拟实境的体验。当然，头戴设备上还可以整合其他功能。以游戏应用为例，可以将游戏主机、音响等全部集成到头戴设备上。
目前所知的基于头戴设备的实现方案中，为了实现对软件及硬件的控制，通常使用的是键盘、鼠标、遥控器等等实现控制。具体的，通过键盘、鼠标、遥控器等在头戴设备的显示屏的显示界面上移动光标并进行拖动、点击等操作。
用户在使用头戴设备的过程中，所能看到的只是头戴设备的显示屏上显示的画面，而无法看到实际环境。仍以游戏应用中的头戴设备，并通过键盘实现控制为例，如果用户希望对游戏的音量进行调整，但由于用户无法看到键盘，因此用户只能摸索操作，这无疑会影响用户的体验。
进一步的，由于用户是通过摸索键盘进行操作的，因此可能会因输入错误的指令而产生意外的结果，例如按下错误的按键，导致当前应用(例如当前运行的游戏)出现异常，严重情况下，可能导致系统运行错误。
发明内容
本发明实施例提供一种应用于头戴设备的控制方法及头戴设备，用以解决现有技术中用户需要摸索操作实现控制，误操作风险较高、可能导致 当前应用甚至系统运行错误的问题。
本发明实施例提供一种应用于头戴设备的控制方法，包括：
根据获取到的手势信息确定手势动作和手势运动轨迹；
根据所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹；
如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作。
本发明实施例提供一种头戴设备，包括：
手势信息传感器，用于获取手势信息；
处理器，用于根据手势信息传感器获取的手势信息确定手势动作和手势运动轨迹；根据所述手势运动轨迹确定显示屏的显示界面上的光标运动轨迹；如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作；
显示屏，用于提供显示界面。
本发明实施例提供的应用于头戴设备的控制方法及头戴设备，由于通过手势识别技术进行控制，不需要用户摸索操作，降低了误操作的风险，进而提高了系统性能。
附图说明
图1为本发明应用于头戴设备的控制方法实施例流程图；
图2为本发明头戴设备实施例结构示意图；
图3为本发明实施例手势信息传感器工作原理示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述， 显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
图1所示为本发明实施例提供的应用于头戴设备的控制方法，该方法包括：
步骤100、根据获取到的手势信息确定手势动作和手势运动轨迹。
以游戏应用为例，头戴设备作为游戏设备或游戏设备的组成部件。那么，佩戴头戴设备的用户在游戏过程中作出手势动作以操控游戏应用，例如调整音量、控制游戏人物的动作等等。相应的，通过手势识别技术便可以获取用户在游戏过程中作出的手势动作所对应的手势信息，在该步骤实施中即可根据获取的手势信息确定用户在游戏过程中作出的手势动作和手势动作轨迹。例如，假设用户做出了由右至左滑动的手势动作，那么通过手势识别技术便可以获取用户作出的由右至左滑动的手势动作所对应的手势信息，在步骤100中即可根据该手势信息确定出滑动这一手势动作，并确定出该手势动作轨迹为由右至左的轨迹。
本领域技术人员容易理解，在头戴设备的其他应用场景(例如视频播放等等)中，本发明实施例提供的方法同样适用。
步骤110、根据上述手势运动轨迹确定头戴设备的显示界面上的光标运动轨迹。
在确定了光标的运动轨迹后，就可以控制显示界面上的光标按照确定的运动轨迹移动。显然，本领域技术人员容易设定手势运动轨迹与显示界面上的光标运动轨迹之间的对应关系。例如，最直观的方式是当手势运动轨迹是由右至左的轨迹，那么，该步骤110中确定头戴设备的显示界面上的光标运动轨迹为在屏幕上由右至左的轨迹。但显然，本领域技术人员容易理解，也可将显示轨迹设定为由左至右或由上至下的轨迹。
本发明实施例中，不对头戴设备的显示界面上的光标形状进行限定。例如，光标的形状可以是箭头，闪烁的线段，手的形状等等。
步骤120、如果上述手势动作为控制动作，且根据上述光标运动轨迹确定光标位于该控制动作所作用的显示区域时，执行该控制动作所对应的控制操作。
如果该手势动作不是控制动作，则仅根据手势运动轨迹确定光标运动轨 迹，进而控制光标移动即可。
本发明实施例中，每个控制动作均对应一个显示区域，当光标位于该显示区域时，才会执行相应的控制动作，因此，这个显示区域又称为该控制动作所作用的显示区域。另外，每个控制动作均对应一个控制操作。
例如，手指按下这一手势动作，对应“点击”这一控制操作，其所作用的显示区域为可以被点击的各个按钮图标的显示区域。
本发明实施例提供的应用于头戴设备的控制方法，由于通过手势识别技术进行控制，不需要用户摸索操作，降低了误操作的风险，进而提高了系统性能。
本发明实施例中，手势信息的表示方式有多种，例如，可以通过手势的深度信息图像表示。相应的，上述步骤100的实现方式可以是：通过对获取的手势的深度信息图像进行手势识别，确定手势动作和手势运动轨迹。
其具体实现方式有多种，下面仅以一例进行说明如何实施，其他可行的实现方式本领域技术人员根据实践需要做出设计即可。例如，从每帧手势的深度信息图像中获取关键点在三维坐标系的坐标，根据获取的关键点的坐标确定手势动作和手势运动轨迹。
其中，关键点可以但不仅限于包括：每个手指的坐标、掌心坐标、手臂坐标等等。
其中，可以通过判断预定数量内的深度信息图像中相同关键点的坐标变化是否在设定阈值内，来判断一个手势动作是否完成。
本发明实施例中，手势的深度信息图像可以是由光线接收器对接收到的光线强度进行分析得到的，该光线接收器接收到的光线是由光线发射器发出并被遮挡物反射的。应当指出的是，手势的深度信息图像也可以是通过多目立体相机(例如双目立体相机)获取。
如果通过光线接收器获取深度信息图像。光线接收器可以为激光接收器，也可以为红外接收器。
基于上述任意方法实施例，为了避免无意义的手势动作或手势轨迹对控制造成干扰，可以在根据手势运动轨迹确定头戴设备的显示界面上的光标运动轨迹之前，将确定的手势运动轨迹与有效手势运动轨迹模板进行匹配；如果匹配成功，上述步骤110是根据匹配成功的手势运动轨迹确定头戴设备的显示界面上的光标运动轨迹。如果匹配不成功，则不触发相应的动作。
通过上述实现方式，可以避免无意义的手势动作或手势轨迹对控制造成干扰。例如，用户只是挥动一下手臂，并不需要控制光标移动。在检测到这一手势运动轨迹后，由于没有正确匹配有效手势运动轨迹模板，因此就不会据此确定光标运动轨迹。
其中，有效运动轨迹模板的具体内容和数量可以根据实际需要设定，本发明对此不作限定。
应当指出的是，在实施中以模板匹配为例是因为该方式是较为容易实现的方式，所以这里以模板匹配为例；但是，只要是能够识别出无效手势运动轨迹的实现方式均适用于本发明实施例，而不仅限于通过模板匹配的方式确认，模板匹配的方式仅用于教导本领域技术人员具体如何实施本发明，但不意味仅能使用该方式，实施过程中可以结合实践需要来确定相应的方式。
基于上述任意方法实施例，可以将确定的手势动作与控制动作模板进行匹配，如果匹配，则该手势动作为控制动作。
其中，控制动作模板的具体内容和数量可以根据实际需要设定，本发明对此不作限定。
应当指出的是，在实施中以模板匹配为例是因为该方式是较为容易实现的方式，所以这里以模板匹配为例；但是，只要是能够识别出控制动作的实现方式均适用于本发明实施例，而不仅限于通过模板匹配的方式确认，模板匹配的方式仅用于教导本领域技术人员具体如何实施本发明，但不意味仅能使用该方式，实施过程中可以结合实践需要来确定相应的方式。。
基于与方法同样的发明构思，本发明实施例还提供了一种头戴设备，如图2所示，包括：
手势信息传感器201，用于获取手势信息；
处理器202，用于根据手势信息传感器201获取的手势信息确定手势动作和手势运动轨迹；根据所述手势运动轨迹确定显示屏203的显示界面上的光标运动轨迹；如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作；
显示屏203，用于提供显示界面。
本发明实施例提供的头戴设备，由于通过手势识别技术进行控制，不需要用户摸索操作，降低了误操作的风险，进而提高了系统性能。
其中，头戴设备各个单元的具体工作方式可以参照上述方法实施例的描述，此处不再赘述。
本发明实施例中，手势信息传感器可以光线发射器和光线接收器。其中，光线发射器用于发出光线；光线接收器用于对接收到的光线强度进行分析，得到手势的深度信息图像，光线接收器接收到的光线是由光线发射器发出并被遮挡物反射的。
其中，光线发射器为激光发射器，光线接收器为激光接收器；或者，光线发射器为红外发射器，光线接收器为红外接收器。
可选的，光线发射器由光源和设置于光源的出光方向的光栅构成。
应当指出的是，手势信息传感器也可以由其他能够获取手势信息的装置实现，例如多目相机。
基于上述任意头戴设备的实施例，优选的，手势信息传感器与显示屏位于头戴设备的同一面。
下面以游戏用头戴设备为例，对本发明实施例提供的技术方案进行详细说明。
本实施例中，头戴设备上、与显示屏同侧设置有红外发射器和红外接收器。其中，红外发射器由红外光源和光栅构成。
头戴设备的存储器中保存有多个控制动作与其所作用的显示区域的对应关系，以及与控制操作的对应关系。例如，手指按下的动作作用于所有可以点击的按钮图标所在的显示区域，并对应点击操作；手指抬起的动作作用于所有可以点击的按钮图标所在的显示区域，并对应释放按钮的操作。
头戴设备的存储器中还保存有多个有效手势运动轨迹模板。
头戴设备的存储器中还保存有多个控制动作模板。
在工作状态下，如图3所示，红外发射器发出红外光，在红外光投射范围内人体会对光纤产生反射，由于人体各部位距离不同，红外接收器采集到不同强度的光线，进而生成深度信息图像。假设红外接收器以1毫秒为单位，每毫秒产生一帧深度信息图像。
处理器接收红外接收器生成的深度信息图像，并据此确定手势动作和手势运动轨迹。具体的，处理器分析每帧深度信息图像中的关键点的坐标，根据关键点的坐标的变化确定手势动作和手势轨迹。
处理器将确定的手势运动轨迹分别与各个有效手势运动轨迹模板进行匹 配。如果匹配成功，例如与直线运动这一有效手势运动轨迹模板匹配，则继续后续流程，否则不响应该手势运动轨迹。
处理器根据匹配成功的手势运动轨迹确定显示屏的显示界面上的光标运动轨迹。其中，如果光标的形状是手的形状，还可以根据识别到的手势动作，控制显示界面上显示的手的动作。
处理器将确定的手势动作与各个控制动作模板进行匹配。如果匹配成功，例如与手指按下这一控制动作模板匹配，并且光标位于该手势动作所作用的显示区域，则执行该手势动作对应的控制操作，例如点击按钮图标。如果匹配不成功，或者光标没有位于手势动作所作用的显示区域，则不响应该手势动作。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种应用于头戴设备的控制方法，其特征在于，包括：
    根据获取到的手势信息确定手势动作和手势运动轨迹；
    根据所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹；
    如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作。
  2. 根据权利要求1所述的方法，其特征在于，所述根据获取到的手势信息确定手势动作和手势运动轨迹，包括：
    通过对获取的手势的深度信息图像进行手势识别，确定手势动作和手势运动轨迹。
  3. 根据权利要求2所述的方法，其特征在于，所述手势的深度信息图像是由光线接收器对接收到的光线强度进行分析得到的，所述光线接收器接收到的光线是由光线发射器发出并被遮挡物反射的。
  4. 根据权利要求3所述的方法，其特征在于，所述光线接收器为激光接收器，或者，所述光线接收器为红外接收器。
  5. 根据权利要求1～4任一项所述的方法，其特征在于，所述根据所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹之前，该方法还包括：将确定的手势运动轨迹与有效手势运动轨迹模板进行匹配；
    根据所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹，包括：根据匹配成功的所述手势运动轨迹确定所述头戴设备的显示界面上的光标运动轨迹。
  6. 根据权利要求1～4任一项所述的方法，其特征在于，该方法还包括：
    将确定的手势动作与控制动作模板进行匹配，如果匹配，则所述手势动作为控制动作。
  7. 一种头戴设备，其特征在于，包括：
    手势信息传感器，用于获取手势信息；
    处理器，用于根据手势信息传感器获取的手势信息确定手势动作和手势运动轨迹；根据所述手势运动轨迹确定显示屏的显示界面上的光标运动轨迹；如果所述手势动作为控制动作，且根据所述光标运动轨迹确定所述光标位于 所述控制动作所作用的显示区域时，执行所述控制动作所对应的控制操作；
    显示屏，用于提供显示界面。
  8. 根据权利要求7所述的头戴设备，其特征在于，所述手势信息传感器包括光线发射器和光线接收器；
    所述光线发射器用于发出光线；
    所述光线接收器用于对接收到的光线强度进行分析，得到手势的深度信息图像，所述光线接收器接收到的光线是由光线发射器发出并被遮挡物反射的。
  9. 根据权利要求8所述的头戴设备，其特征在于，所述光线发射器为激光发射器，光线接收器为激光接收器；
    或者，所述光线发射器为红外发射器，所述光线接收器为红外接收器。
  10. 根据权利要求7～9任一项所述的头戴设备，其特征在于，所述手势信息传感器与所述显示屏位于头戴设备的同一面。
PCT/CN2016/081916 2015-11-20 2016-05-12 一种应用于头戴设备的控制方法及头戴设备 Ceased WO2017084253A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510811932.9A CN105892636A (zh) 2015-11-20 2015-11-20 一种应用于头戴设备的控制方法及头戴设备
CN201510811932.9 2015-11-20

Publications (1)

Publication Number Publication Date
WO2017084253A1 true WO2017084253A1 (zh) 2017-05-26

Family

ID=57002864

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/081916 Ceased WO2017084253A1 (zh) 2015-11-20 2016-05-12 一种应用于头戴设备的控制方法及头戴设备

Country Status (2)

Country Link
CN (1) CN105892636A (zh)
WO (1) WO2017084253A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025194917A1 (zh) * 2024-03-22 2025-09-25 华为技术有限公司 防误触方法和电子设备

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108268124A (zh) * 2016-12-30 2018-07-10 成都虚拟世界科技有限公司 基于头戴式显示设备的手势识别方法及装置
WO2018184233A1 (zh) * 2017-04-07 2018-10-11 深圳市柔宇科技有限公司 一种手势识别方法及相关装置
CN107242882A (zh) * 2017-06-05 2017-10-13 上海瓴舸网络科技有限公司 一种b超显示辅助设备及其控制方法
CN109410691A (zh) * 2018-12-17 2019-03-01 深圳市中智仿真科技有限公司 一种手势控制功能的汽车驾培模拟机
CN112083796A (zh) * 2019-06-12 2020-12-15 Oppo广东移动通信有限公司 控制方法、头戴设备、移动终端和控制系统
CN110119209A (zh) * 2019-06-13 2019-08-13 漫谷科技股份公司 音频设备控制方法和装置
CN111601129B (zh) * 2020-06-05 2022-04-01 北京字节跳动网络技术有限公司 控制方法、装置、终端及存储介质
CN112162631B (zh) * 2020-09-18 2023-05-16 聚好看科技股份有限公司 一种交互设备、数据处理方法及介质
CN112988107A (zh) * 2021-04-25 2021-06-18 歌尔股份有限公司 一种音量调节方法、系统及头戴式设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150042680A1 (en) * 2013-08-08 2015-02-12 Pebbles Ltd. Method and device for controlling a near eye display
CN104571510A (zh) * 2014-12-30 2015-04-29 青岛歌尔声学科技有限公司 一种3d场景中输入手势的系统和方法
CN105045398A (zh) * 2015-09-07 2015-11-11 哈尔滨市一舍科技有限公司 一种基于手势识别的虚拟现实交互设备
CN204790857U (zh) * 2015-07-24 2015-11-18 贺杰 一种手势控制器及一种虚拟现实系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9069164B2 (en) * 2011-07-12 2015-06-30 Google Inc. Methods and systems for a virtual input device
US9389420B2 (en) * 2012-06-14 2016-07-12 Qualcomm Incorporated User interface interaction for transparent head-mounted displays
CN110083202B (zh) * 2012-07-27 2023-09-19 诺基亚技术有限公司 与近眼显示器的多模交互
CN104076907A (zh) * 2013-03-25 2014-10-01 联想(北京)有限公司 一种控制方法、装置和穿戴式电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150042680A1 (en) * 2013-08-08 2015-02-12 Pebbles Ltd. Method and device for controlling a near eye display
CN104571510A (zh) * 2014-12-30 2015-04-29 青岛歌尔声学科技有限公司 一种3d场景中输入手势的系统和方法
CN204790857U (zh) * 2015-07-24 2015-11-18 贺杰 一种手势控制器及一种虚拟现实系统
CN105045398A (zh) * 2015-09-07 2015-11-11 哈尔滨市一舍科技有限公司 一种基于手势识别的虚拟现实交互设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025194917A1 (zh) * 2024-03-22 2025-09-25 华为技术有限公司 防误触方法和电子设备

Also Published As

Publication number Publication date
CN105892636A (zh) 2016-08-24

Similar Documents

Publication Publication Date Title
WO2017084253A1 (zh) 一种应用于头戴设备的控制方法及头戴设备
US11020165B2 (en) Touch free operation of ablator workstation by use of depth sensors
US11294475B1 (en) Artificial reality multi-modal input switching model
US10394334B2 (en) Gesture-based control system
US10025390B2 (en) Enhanced input using recognized gestures
CN103890696B (zh) 经认证的手势辨识
CN103347437B (zh) 3d映射环境中的凝视检测
US9400548B2 (en) Gesture personalization and profile roaming
US9625993B2 (en) Touch free operation of devices by use of depth sensors
US11188145B2 (en) Gesture control systems
CN107589827B (zh) 一种可识别手势的虚拟现实头盔及其手势识别方法
US10156938B2 (en) Information processing apparatus, method for controlling the same, and storage medium
US20150234467A1 (en) Method and apparatus for gesture detection and display control
US20240281070A1 (en) Simultaneous Controller and Touch Interactions
US20240385692A1 (en) Two-handed gesture interpretation
KR20240036582A (ko) 물리적 객체와의 사용자 인터페이스에 대한 상호작용들을 관리하기 위한 방법 및 디바이스
US20210103315A1 (en) Method and apparatus for determining interaction position
CN206411612U (zh) 一种虚拟现实系统的交互控制装置及虚拟现实设备
US11307762B2 (en) Operating user interfaces
CN111309153A (zh) 人机交互的控制方法和装置、电子设备和存储介质
WO2021241038A1 (ja) 情報処理装置及びユーザの入力操作に基づく情報処理方法並びに該方法を実行するためのコンピュータプログラム
JP2022032984A (ja) 客体制御を支援する方法、システムおよび非一過性のコンピュータ読み取り可能記録媒体
Nagy et al. Evaluation of ai-supported input methods in augmented reality environment
CN113498029B (zh) 交互式广播
KR20130076992A (ko) 영역 별로 임계치를 다르게 적용한 마스크 이미지 생성 장치 및 그 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16865435

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16865435

Country of ref document: EP

Kind code of ref document: A1