WO2017084253A1

WO2017084253A1 - Control method applied to head-mounted device and head-mounted device

Info

Publication number: WO2017084253A1
Application number: PCT/CN2016/081916
Authority: WO
Inventors: 陈相金
Original assignee: 乐视控股（北京）有限公司; 乐视致新电子科技（天津）有限公司
Priority date: 2015-11-20
Filing date: 2016-05-12
Publication date: 2017-05-26
Also published as: CN105892636A

Abstract

A control method applied to a head-mounted device and a head-mounted device. The method comprises: determining a gesture action and a gesture motion track according to acquired gesture information; determining a cursor motion track on a display interface of the head-mounted device according to the gesture motion track; and if the gesture action is a control action and it is determined that a cursor is located in a display region on which the control action acts according to the cursor motion track, executing a control operation corresponding to the control action. In the present invention, control is performed through a gesture recognition technology without a user groping operation, thus reducing the risk of a faulty operation, and thereby improving the system performance.

Description

Control method and headwear device applied to headwear

This application claims priority to Chinese Patent Application No. 201510811932.9, entitled "A Control Method for Headset Equipment and Headset Equipment", filed on November 20, 2015, the entire contents of which are hereby incorporated by reference. This is incorporated herein by reference.

Technical field

The present invention relates to the field of device control technologies, and in particular, to a control method and a head mounted device applied to a head mounted device.

Background technique

A headset is at least a head-mounted display device that gives the user a virtual reality experience. Of course, other features can be integrated on the headset. Take the game application as an example, the game console, audio, etc. can all be integrated into the headset.

In the currently known implementation of the headset-based device, in order to implement control of software and hardware, a keyboard, a mouse, a remote controller, and the like are generally used for control. Specifically, the cursor is moved on the display interface of the display screen of the head wear device through a keyboard, a mouse, a remote controller, and the like, and operations such as dragging and clicking are performed.

In the process of using the headset, what the user can see is only the screen displayed on the display of the headset, and the actual environment cannot be seen. Still taking the headset in the game application and controlling it through the keyboard as an example. If the user wants to adjust the volume of the game, but the user cannot see the keyboard, the user can only explore the operation, which will undoubtedly affect the user experience. .

Further, since the user operates by groping the keyboard, an unexpected result may be generated by inputting an incorrect command, such as pressing an incorrect button, causing an abnormality in the current application (for example, a currently running game), in severe cases. May cause the system to run incorrectly.

Summary of the invention

The embodiment of the invention provides a control method and a head-wearing device applied to a head-mounted device, which are used to solve the problem that the user needs to explore and operate the control in the prior art, and the risk of misoperation is high, which may result in The current application or even the system is running incorrectly.

An embodiment of the present invention provides a control method applied to a headset, including:

Determining a gesture action and a gesture motion track according to the acquired gesture information;

Determining a cursor movement track on the display interface of the headset device according to the gesture motion track;

And if the gesture action is a control action, and determining that the cursor is located in a display area to which the control action acts according to the cursor motion trajectory, performing a control operation corresponding to the control action.

An embodiment of the present invention provides a headset device, including:

a gesture information sensor for acquiring gesture information;

a processor, configured to determine a gesture motion and a gesture motion track according to the gesture information acquired by the gesture information sensor; determine a cursor motion track on the display interface of the display screen according to the gesture motion track; if the gesture motion is a control action, and according to The cursor movement trajectory determines that the cursor is located in a display area in which the control action is applied, and performs a control operation corresponding to the control action;

Display for providing a display interface.

The control method and the head-wearing device applied to the head-wearing device provided by the embodiment of the present invention, because of the control by the gesture recognition technology, do not require the user to explore the operation, thereby reducing the risk of misoperation, thereby improving the system performance.

DRAWINGS

1 is a flow chart of an embodiment of a control method applied to a headwear device according to the present invention;

2 is a schematic structural view of an embodiment of a headset device according to the present invention;

FIG. 3 is a schematic diagram of a working principle of a gesture information sensor according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is apparent that the described embodiments are a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

FIG. 1 is a schematic diagram of a control method applied to a headset device according to an embodiment of the present invention, where the method includes:

Step 100: Determine a gesture motion and a gesture motion track according to the acquired gesture information.

Taking a game application as an example, a head-mounted device is a component of a game device or a game device. Then, the user wearing the headset makes gestures during the game to manipulate the game application, such as adjusting the volume, controlling the motion of the game character, and the like. Correspondingly, the gesture information corresponding to the gesture action made by the user during the game can be obtained by using the gesture recognition technology. In the implementation of the step, the gesture gesture and the gesture gesture made by the user during the game can be determined according to the acquired gesture information. Track. For example, if the user performs a gesture action that is swiped from right to left, the gesture information corresponding to the gesture motion corresponding to the user's right-to-left swipe gesture can be obtained by the gesture recognition technology, and the gesture can be performed according to the gesture in step 100. The information determines the gesture action of sliding, and determines that the gesture motion trajectory is a right-to-left trajectory.

It is easy for those skilled in the art to understand that the methods provided by the embodiments of the present invention are equally applicable to other application scenarios of the headset (such as video playback, etc.).

Step 110: Determine a cursor motion track on the display interface of the headset according to the gesture motion track.

After the motion track of the cursor is determined, the cursor on the display interface can be controlled to move according to the determined motion track. Obviously, those skilled in the art can easily set the correspondence between the gesture motion track and the cursor motion track on the display interface. For example, the most intuitive way is that when the gesture motion trajectory is a right-to-left trajectory, then in step 110, the cursor motion trajectory on the display interface of the headset is determined to be a right-to-left trajectory on the screen. However, it will be apparent to those skilled in the art that the display trajectory can also be set to a left to right or top to bottom trajectory.

In the embodiment of the present invention, the shape of the cursor on the display interface of the headwear device is not limited. For example, the shape of the cursor can be an arrow, a blinking line segment, a shape of a hand, and the like.

Step 120: If the gesture action is a control action, and determining that the cursor is located in the display area affected by the control action according to the cursor motion track, performing a control operation corresponding to the control action.

If the gesture action is not a control action, the cursor motion track is determined only according to the gesture motion track Trace, and then control the cursor movement.

In the embodiment of the present invention, each control action corresponds to a display area. When the cursor is located in the display area, the corresponding control action is performed. Therefore, the display area is also referred to as a display area to which the control action acts. In addition, each control action corresponds to one control operation.

For example, the finger presses this gesture action, corresponding to the "click" control operation, and the display area it acts on is the display area of each button icon that can be clicked.

The control method applied to the headset device provided by the embodiment of the invention, because of the control by the gesture recognition technology, does not require the user to explore the operation, reduces the risk of misoperation, and further improves the system performance.

In the embodiment of the present invention, the manner of representing the gesture information is various, for example, it can be represented by the depth information image of the gesture. Correspondingly, the implementation of the foregoing step 100 may be: determining a gesture action and a gesture motion track by performing gesture recognition on the depth information image of the acquired gesture.

There are a plurality of specific implementation manners. The following describes how to implement only one example. Other feasible implementation manners may be made by those skilled in the art according to practical needs. For example, the coordinates of the key point in the three-dimensional coordinate system are acquired from the depth information image of each frame of the gesture, and the gesture motion and the gesture motion track are determined according to the coordinates of the acquired key point.

Among them, the key points can be, but are not limited to, including: the coordinates of each finger, the palm coordinates, the arm coordinates, and the like.

Wherein, whether the gesture action is completed can be determined by determining whether the coordinate change of the same key point in the depth information image within the predetermined number is within the set threshold.

In the embodiment of the present invention, the depth information image of the gesture may be obtained by analyzing, by the light receiver, the received light intensity, and the light received by the light receiver is emitted by the light emitter and reflected by the occlusion object. It should be noted that the depth information image of the gesture may also be acquired by a multi-view stereo camera such as a binocular stereo camera.

If the depth information image is obtained through the light receiver. The light receiver can be either a laser receiver or an infrared receiver.

Based on any of the foregoing method embodiments, in order to prevent the meaningless gesture action or the gesture track from causing interference to the control, the determined gesture motion track may be valid before determining the cursor motion track on the display interface of the headset according to the gesture motion track. The gesture motion trajectory template is matched; if the matching is successful, the above step 110 is to determine a cursor motion trajectory on the display interface of the headset according to the gesture motion track that is successfully matched. If the match is not successful, the corresponding action is not triggered.

Through the above implementation manner, it is possible to prevent the meaningless gesture action or the gesture track from causing interference to the control. For example, the user just flicks his arm and does not need to control the cursor movement. After detecting the motion track of the gesture, since the effective gesture motion track template is not correctly matched, the cursor motion track is not determined accordingly.

The specific content and quantity of the effective motion trajectory template can be set according to actual needs, which is not limited by the present invention.

It should be noted that in the implementation, template matching is taken as an example because the method is relatively easy to implement, so the template matching is taken as an example here; however, as long as the implementation method capable of recognizing the invalid gesture motion track is applicable to this embodiment The embodiments of the present invention are not limited to the manner of template matching. The manner of template matching is only used to teach the person skilled in the art how to implement the present invention, but it does not mean that only the method can be used, and the implementation process can be determined in combination with practical needs. The corresponding way.

Based on any of the above method embodiments, the determined gesture action can be matched with the control action template, and if matched, the gesture action is a control action.

The specific content and quantity of the control action template can be set according to actual needs, which is not limited by the present invention.

It should be noted that in the implementation, template matching is taken as an example because the method is relatively easy to implement, so template matching is taken as an example here; however, any implementation that can recognize the control action is applicable to the implementation of the present invention. For example, and not limited to, confirmation by means of template matching, the manner of template matching is only used to teach those skilled in the art how to implement the present invention in detail, but it does not mean that only this method can be used, and the implementation process can be combined with practical needs to determine corresponding the way. .

Based on the same inventive concept as the method, the embodiment of the present invention further provides a head-wearing device, as shown in FIG. 2, including:

a gesture information sensor 201, configured to acquire gesture information;

The processor 202 is configured to determine a gesture motion and a gesture motion track according to the gesture information acquired by the gesture information sensor 201, and determine a cursor motion track on the display interface of the display screen 203 according to the gesture motion track; if the gesture action is a control action And determining, according to the cursor motion trajectory, that the cursor is located in a display area in which the control action is applied, performing a control operation corresponding to the control action;

The display screen 203 is configured to provide a display interface.

The headset device provided by the embodiment of the present invention does not require the user to explore the operation because the gesture recognition technology is used for control, thereby reducing the risk of misoperation, thereby improving system performance.

For the specific working mode of each unit of the headwear device, reference may be made to the description of the foregoing method embodiment, and details are not described herein again.

In the embodiment of the present invention, the gesture information sensor may be a light emitter and a light receiver. The light emitter is used to emit light; the light receiver is used to analyze the received light intensity to obtain a depth information image of the gesture, and the light received by the light receiver is emitted by the light emitter and reflected by the obstruction.

Wherein, the light emitter is a laser emitter, the light receiver is a laser receiver; or the light emitter is an infrared emitter, and the light receiver is an infrared receiver.

Optionally, the light emitter is composed of a light source and a grating disposed in a light exiting direction of the light source.

It should be noted that the gesture information sensor can also be implemented by other devices capable of acquiring gesture information, such as a multi-view camera.

Based on the embodiment of any of the above-described headsets, preferably, the gesture information sensor and the display screen are located on the same side of the headset.

The technical solution provided by the embodiment of the present invention is described in detail below by taking the game wearing device as an example.

In this embodiment, an infrared emitter and an infrared receiver are disposed on the headset and on the same side of the display screen. Among them, the infrared emitter is composed of an infrared light source and a grating.

The memory of the headset stores a correspondence between a plurality of control actions and a display area to which it acts, and a correspondence relationship with the control operation. For example, the action of the finger press acts on the display area where all the clickable button icons are located, and corresponds to the click operation; the action of the finger lift acts on the display area where all the clickable button icons are located, and corresponds to the operation of releasing the button.

A plurality of effective gesture motion track templates are also stored in the memory of the headset.

A plurality of control action templates are also stored in the memory of the headset.

In the working state, as shown in FIG. 3, the infrared emitter emits infrared light, and the human body will reflect on the optical fiber in the infrared light projection range. Due to the different distances of various parts of the human body, the infrared receiver collects light of different intensity, thereby generating Depth information image. It is assumed that the infrared receiver generates a frame of depth information image every millisecond in units of 1 millisecond.

The processor receives the depth information image generated by the infrared receiver, and determines the gesture motion and the gesture motion trajectory accordingly. Specifically, the processor analyzes the coordinates of the key points in the depth information image of each frame, and determines the gesture action and the gesture track according to the change of the coordinates of the key points.

The processor compares the determined gesture motion trajectory with each effective gesture motion trajectory template Match. If the matching is successful, for example, matching the effective gesture motion track template of the linear motion, the subsequent process is continued, otherwise the motion track is not responded to.

The processor determines a cursor motion trajectory on the display interface of the display screen according to the gesture motion track that matches the success. Wherein, if the shape of the cursor is the shape of the hand, the action of the hand displayed on the display interface can also be controlled according to the recognized gesture action.

The processor matches the determined gesture action with each control action template. If the matching is successful, for example, matching the control action template pressed by the finger, and the cursor is located in the display area to which the gesture action is applied, the control operation corresponding to the gesture action is performed, such as clicking the button icon. If the match is unsuccessful, or the cursor is not located in the display area where the gesture action is applied, the gesture action is not responded.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, optical discs, etc., include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A control method applied to a headwear device, comprising:

Determining a gesture action and a gesture motion track according to the acquired gesture information;

Determining a cursor movement track on the display interface of the headset device according to the gesture motion track;

And if the gesture action is a control action, and determining that the cursor is located in a display area to which the control action acts according to the cursor motion trajectory, performing a control operation corresponding to the control action.
The method according to claim 1, wherein the determining the gesture action and the gesture motion track according to the acquired gesture information comprises:

The gesture action and the gesture motion track are determined by performing gesture recognition on the depth information image of the acquired gesture.
The method according to claim 2, wherein the depth information image of the gesture is obtained by analyzing, by the light receiver, the received light intensity, and the light received by the light receiver is emitted by the light emitter Emitted and reflected by the obstruction.
The method of claim 3 wherein said light receiver is a laser receiver or said light receiver is an infrared receiver.
The method according to any one of claims 1 to 4, wherein before the determining a cursor motion track on the display interface of the headset device according to the gesture motion track, the method further comprises: determining the determined The gesture motion track is matched with the effective gesture motion track template;

Determining a cursor motion track on the display interface of the headset device according to the gesture motion track includes: determining a cursor motion track on the display interface of the headset device according to the gesture motion track that is successfully matched.
The method according to any one of claims 1 to 4, further comprising:

The determined gesture action is matched with the control action template, and if matched, the gesture action is a control action.
A head mounted device, comprising:

a gesture information sensor for acquiring gesture information;

a processor, configured to determine a gesture motion and a gesture motion track according to the gesture information acquired by the gesture information sensor; determine a cursor motion track on the display interface of the display screen according to the gesture motion track; if the gesture motion is a control action, and according to The cursor motion trajectory determines that the cursor is located When the display area to which the action is controlled is performed, a control operation corresponding to the control action is performed;

Display for providing a display interface.
The headgear according to claim 7, wherein the gesture information sensor comprises a light emitter and a light receiver;

The light emitter is for emitting light;

The light receiver is configured to analyze the received light intensity to obtain a depth information image of the gesture, and the light received by the light receiver is emitted by the light emitter and reflected by the obstruction.
The headgear according to claim 8, wherein the light emitter is a laser emitter and the light receiver is a laser receiver;

Alternatively, the light emitter is an infrared emitter and the light receiver is an infrared receiver.
The headgear according to any one of claims 7 to 9, wherein the gesture information sensor and the display screen are located on the same side of the head mounted device.