CN111796671A

CN111796671A - Gesture recognition and control method for head-mounted device and storage medium

Info

Publication number: CN111796671A
Application number: CN202010442052.XA
Authority: CN
Inventors: 刘德建; 陈丛亮; 郭玉湖; 陈宏�
Original assignee: Fujian TQ Digital Co Ltd
Current assignee: Fujian TQ Digital Co Ltd
Priority date: 2020-05-22
Filing date: 2020-05-22
Publication date: 2020-10-20
Anticipated expiration: 2040-05-22
Also published as: CN111796671B

Abstract

The invention provides a gesture recognition and control method and a storage medium of a head-mounted device, wherein the method comprises the following steps: within a preset first time length, acquiring unused RGB values according to the RGB values of pixels of each frame of image, and sending the unused RGB values to light-emitting equipment; within a preset second time length, identifying the coordinate position of light output by the light-emitting device according to the received RGB value in the display; starting timing in turn seamlessly between the first time length and the second time length; the head-mounted device reads the gravity sensing data from the light-emitting device through the proxy application. The invention can improve the accuracy and the recognition efficiency of single-camera gesture recognition, supports the gravity sensing control through the light-emitting device, improves the operation convenience and the control accuracy, enriches the control modes and functions and greatly improves the product usability.

Description

Gesture recognition and control method for head-mounted device and storage medium

Technical Field

The invention relates to the field of gesture recognition, in particular to a method for recognizing and controlling gestures of head-mounted equipment and a storage medium.

Background

The head-mounted equipment in the prior art is worn on the head, so that the operation control is difficult to be carried out by using a mode such as touch control of a mobile phone screen. Although some head-mounted devices are already capable of supporting gesture control. However, since the head-mounted device is generally configured with only a single camera, gesture recognition through the single camera generally has the problems of complicated calculation, low recognition rate, low operation sensitivity, insufficient convenience in operation, and the like.

Therefore, it is necessary to provide a gesture recognition and control method of a head-mounted device, a storage medium, which can overcome the above problems at the same time.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the gesture recognition and control method and the storage medium of the head-mounted equipment are provided, so that the gesture recognition accuracy and recognition efficiency of a single camera can be improved at the same time; and the quick gesture control is supported, and the operation convenience is improved.

In order to solve the technical problems, the invention adopts the technical scheme that:

a method of gesture recognition and control of a head-mounted device, comprising:

within a preset first time length, acquiring unused RGB values according to the RGB values of pixels of each frame of image, and sending the unused RGB values to light-emitting equipment;

within a preset second time length, identifying the coordinate position of light output by the light-emitting device according to the received RGB value in the display;

starting timing in turn seamlessly between the first time length and the second time length;

the head-mounted device reads the gravity sensing data from the light-emitting device through the proxy application.

The invention provides another technical scheme as follows:

a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is able to carry out the steps involved in the above-mentioned method of gesture recognition and control of a head-mounted device.

The invention has the beneficial effects that: the invention controls the light-emitting equipment to output the light of the unused RGB value by acquiring the unused RGB value; and then, acquiring the movement track of the touch gesture by identifying the movement track of the display corresponding to the light. Therefore, the conventional mode that the operation track of the real hand of the user can be recognized only by performing complex calculation on all image pixels is changed into the recognition mode that the gesture of the user can be quickly acquired only by simply analyzing the pixels with the specific RGB values. Therefore, the invention not only greatly reduces the calculation complexity of the recognition, but also improves the recognition efficiency; the accuracy of identification can be ensured at the same time; especially, the efficiency and the accuracy of gesture recognition of the head-mounted equipment with a single camera can be remarkably improved. Furthermore, the gravity sensing control of the head-mounted equipment by the light-emitting equipment is supported, so that the gravity sensing application can be supported, and the usability of the product is improved; the control is realized through the light-emitting equipment, so that the accuracy of the control of the head-mounted equipment and the convenience in operation can be improved; furthermore, the problem of application compatibility can be solved well through proxy application, and the cost caused by the fact that the application needs to be customized and developed again for the docking control equipment is avoided.

Drawings

Fig. 1 is a schematic flowchart of a gesture recognition and control method for a head-mounted device according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of a gesture recognition and control method of a head-mounted device according to an embodiment of the present invention.

Detailed Description

In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.

The most key concept of the invention is as follows: the user gesture can be quickly acquired only by simply analyzing the pixels with the specific RGB values; meanwhile, gravity sensing control through the light-emitting device is supported.

Referring to fig. 1, the present invention provides a gesture recognition and control method for a head-mounted device, comprising:

From the above description, the beneficial effects of the present invention are:

1. the speed and the accuracy of gesture recognition can be effectively improved.

2. The gravity sensing device can support gravity sensing application and improve accuracy and operation convenience of control of the head-mounted device.

3. The compatibility of the application well solved by the proxy application program avoids the need for an application developer to re-customize and develop the docking equipment, and can better reuse the existing android ecology.

Further, still include:

and receiving a click signal sent by the light-emitting equipment, wherein the click signal corresponds to the coordinate position of the light currently output by the light-emitting equipment in the display.

As can be seen from the above description, the mouse can also be matched with gestures to simulate the function of mouse clicking.

Further, the obtaining of the unused RGB values according to the RGB values of each pixel of each frame image specifically includes:

presetting more than two groups respectively corresponding to different RGB value ranges;

acquiring the RGB value of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

calculating the number of pixel points of each group to obtain the group with the least number of pixel points;

and determining the RGB value in the RGB value range corresponding to the group with the least number of pixel points as an unused RGB value.

As can be seen from the above description, the way of dividing the RGB value ranges of each group according to the color values is helpful to lock one or a few groups in a centralized manner when analyzing the unused pixel colors in the image captured in the first time period, so as not to disperse into multiple groups, thereby improving the accuracy and efficiency of subsequent analysis and calculation.

Further, if the unused RGB values correspond to two or more groups, the sending is performed to a light emitting device, specifically:

respectively calculating RGB difference values of more than two groups corresponding to the unused RGB values and other groups;

acquiring an RGB value range corresponding to the group with the largest difference value with other groups;

and sending the RGB value range to a light-emitting device.

As can be seen from the above description, if the unused RGB values are dispersed in two or more groups, the group with the largest difference from other groups is further selected, and the corresponding RGB value range is used as the standard of the light output by the light-emitting device, so that the recognition degree of the light output by the light-emitting device in the display screen of the head portrait device can be further improved, and the recognition accuracy is improved again.

Further, the RGB values of the light output by the light emitting device are randomly chosen from the range of received RGB values.

As can be seen from the above description, the light emitting device can be freely selected from a given range, and the matching degree with the light emitting device is improved, ensuring that it can output light of RGB values meeting the requirements.

Further, the different RGB value ranges are RGB value ranges corresponding to respective colors.

As can be seen from the above description, grouping is directly performed according to the color values corresponding to the colors, so that the available value and intuitiveness of the pixel grouping result can be improved.

Further, the identifying the coordinate position of the light output by the light emitting device according to the received RGB values in the display specifically includes:

controlling a light emitting device to emit light corresponding to the received RGB values;

searching pixel points corresponding to the RGB values sent to the light-emitting equipment in the current frame image, and acquiring coordinate positions of the pixel points;

and acquiring a moving track of the display corresponding to the light in the second duration according to the coordinate position of each frame of image in the second duration.

As can be seen from the above description, by locating a specific RGB value in an image and combining the RGB values in time sequence, a control gesture made by a user through a light-emitting device can be obtained.

Further, still include:

and the head-mounted equipment executes the corresponding gravity sensing event according to the acquired gravity sensing data.

As can be seen from the above description, the head-mounted device can directly implement the corresponding gravity sensing control according to the gravity sensing data of the light-emitting device.

Further, the head-mounted device reads gravity sensing data from the light-emitting device through the agent application, specifically:

the head-mounted device is in communication connection with the light-emitting device through the agent application program in the head-mounted device;

the agent application program loads a target application in the application of the agent application program;

the head-mounted equipment calls a gravity sensing data acquisition request of the target application;

and the agent application program loads the request into the memory and carries out byte code replacement on the gravity sensing acquisition interface so as to acquire the gravity sensing data of the light-emitting equipment.

According to the description, the problem that the gravity sensing data of the external equipment cannot be acquired is solved by loading the target application and modifying the byte codes of the gravity sensing related functions through the proxy application program, the gravity sensing data of the external light-emitting equipment can be simply acquired by the head-mounted equipment, the self application is controlled according to the gravity sensing data, and the problems that the head-mounted equipment is inconvenient to operate in gravity sensing control and low in control precision are solved.

The invention provides another technical scheme as follows:

a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is capable of carrying out the steps included in a gesture recognition and control method for a head-mounted device:

Further, still include:

acquiring the RGB value of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

and sending the RGB value range to a light-emitting device.

Further, still include:

As can be understood from the above description, those skilled in the art can understand that all or part of the processes in the above technical solutions can be implemented by instructing related hardware through a computer program, where the program can be stored in a computer-readable storage medium, and when executed, the program can include the processes of the above methods. The program can also achieve advantageous effects corresponding to the respective methods after being executed by a processor.

The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

Example one

Referring to fig. 2, the present embodiment provides a gesture recognition and control method for a head-mounted device, which can significantly improve gesture recognition efficiency and accuracy of gesture recognition; meanwhile, the gravity sensing control of the head-mounted equipment is realized by the light-emitting equipment, the control function is enriched, and the accuracy and the operation convenience of the gravity sensing control are improved. The light emitting device may be any device capable of emitting light corresponding to a specified RGB value, such as a ring or bracelet configured with LEDs.

The method comprises the following steps:

s1: presetting more than two groups respectively corresponding to different RGB value ranges;

that is, one group corresponds to one RGB value range. Preferably a set corresponds to a range of colour values. For example, the color image may be divided into 9 groups of red, orange, yellow, green, blue, purple, and black, wherein the "red" group corresponds to a cube having a length, width, and height of 50 and 55, which is a rectangular solid space having RGB values ranging from (200, 50, 50) to (255, 0, 0).

Of course, the grouping may also be finer, such as grouping each small RGB value interval.

Red: (200, 50, 50) - (255, 0, 0);

orange: (200, 100, 50) - (255, 50, 0);

yellow: (200, 150, 50) - (255, 100, 0);

green: (0, 255, 0) to (50, 200, 50);

cyan: (0, 255, 50) - (50, 200, 100);

blue: (0, 0, 255) to (50, 50, 200);

purple: (50, 0, 255) - (100, 50, 200);

preferably, the partition of red, orange, yellow, green, blue-violet may also be a range classified in an LAB-wise manner in polar coordinates and then converted into RGB.

S2: presetting a first time length and a second time length;

preferably, the first and second time periods are equal, such as 100 ms.

S3: in the first time period, the head-mounted device obtains unused RGB values according to the RGB values of the pixels of each frame of image and sends the unused RGB values to the light-emitting device.

Specifically, the step includes:

s31: and acquiring the RGB value of each pixel of each frame of image. Namely, the RGB value of each pixel in each frame of image shot by the camera in the first duration is obtained.

S32: and dividing each pixel into a corresponding group according to the RGB value. That is, each pixel acquired in step S31 is grouped according to its RGB value and classified into a group corresponding to a range of RGB values.

S33: and calculating the number of pixel points of each group to obtain the group with the minimum number of pixel points. That is, the number of pixels included in each group is calculated, and the group with the smallest number of pixels is obtained, and if the obtained number of groups is one, the group can be considered to have the largest difference from other groups.

In another specific example, the number of the groups with the smallest number of pixels finally obtained in the step S33 is two or more, and the identification degree of the light output by the light emitting device can be improved by further calculating one group with the largest difference from all other groups.

For example, if statistics shows that the pixel points included in the 3 groups of "red", "orange" and "yellow" are all 0 or close to 0, the RGB value ranges corresponding to the three groups are the unused RGB values.

In accordance with another embodiment, the most different group can be determined by:

s34: RGB difference values of two or more groups corresponding to the unused RGB values acquired at S33 and all other groups are calculated, respectively, to determine.

Taking the above 9 groups of red, orange, yellow, green, blue, purple and black as an example, and determining that the three groups of "red", "orange" and "yellow" have the least and equal pixel points after the step of S33, the unused RGB values correspond to the RGB values of the three groups of "red", "orange" and "yellow". In order to further improve the discrimination of the light emitted by the light emitting device. Then the difference between the three groups "red", "orange" and "yellow" and the 9 groups red, orange, yellow, green, blue, purple and black can be calculated. The calculation process may be: the RGB values of three groups of "red", "orange" and "yellow" are subtracted from other groups having pixel points (6 groups of cyan, blue, violet, black and white) to obtain a group having the largest difference among the three components (R, G, B). The formula is as follows: group 1 and group 2 had a difference d12 ═ 2+ (R1-R2)2+ (G1-G2)2+ (B1-B2) 2; the difference value dij is obtained for each of the red orange yellow 3 group and the green blue violet 4 group. Wherein the numbers of red, orange, yellow, green, blue and purple are 1234567 respectively; the maximum difference value is max (min (d14, d15, d16, d17), min (d24, d25, d26, d27), min (d34, d35, d36, d 37)).

If the maximum value d14 is found, the red group is defined.

S35: if the group corresponding to the unused RGB value is only one group, the RGB values within the range of RGB values corresponding to the group can be directly sent to the light emitting device.

If the group with the least number of pixels corresponds to more than two groups, the corresponding RGB value range can be directly sent to the light-emitting equipment, one RGB value is selected by the light-emitting equipment to carry out light sending accidents, and the group with the highest identification degree can be screened from the more than two groups and then the RGB value range is sent to the light-emitting equipment.

That is, regardless of the number of groups to which unused RGB values correspond, it is preferable to transmit the RGB values having the largest difference to the light emitting devices; of course, it is also possible to choose to send all unused RGB values to the light emitting device.

It should be noted that the head-mounted device sends the RGB value range to the lighting device via the lighting device connected to its bluetooth.

S4: during the second time period, the head-mounted device identifies the coordinate position of the light in the display that the light-emitting device outputs according to the RGB value range it receives.

The method specifically comprises the following steps:

s41: and controlling the light-emitting equipment to emit corresponding light according to the received RGB value range in the second time length.

Preferably, if the unused RGB values correspond to more than two color values (i.e. two grouped ranges of RGB values), the light-emitting device may randomly select the RGB values from the unused RGB values for output.

In a specific example, if the maximum difference group is found as "red" group, only one group of corresponding RGB values is obtained. If the "red" group corresponds to a cuboid space having RGB values in the ranges of (200, 50, 50) to (255, 0, 0), i.e., a cuboid having a length, width and height of 50 and 55, the light emitting device may randomly output the colors of (200, 50, 50) to (255, 0, 0).

S42: the head-mounted device searches pixel points of the RGB values which are correspondingly sent to the light-emitting device in the current frame image according to the image shot by the camera at present (the image is still shot by the camera in real time within the second duration), and obtains the coordinate position of the pixel points.

S43: and acquiring the movement track of the display corresponding to the light in the second time length according to the coordinate position determined in the last step of each frame image in the second time length.

That is, the head-mounted device only needs to identify the pixel points corresponding to the RGB values sent to the light-emitting device in the image shot in real time, and locate the coordinate positions of the pixel points in the image; and then connecting the coordinate positions according to the time sequence, so that the gesture of the light output by the light-emitting device in the second time length corresponding to the screen of the head-mounted device, namely the gesture made by the user through the light-emitting device, can be obtained.

S5: and starting timing in turn seamlessly between the first time length and the second time length.

Correspondingly, S3 and S4 are executed alternately, and the first time period is the beginning. Specifically, in the first time period, the light emitting device stops outputting any light, and only when receiving the RGB value range sent by the head-mounted device, the corresponding light is output.

That is, in the first duration, the head-mounted device calculates and transmits the calculation result to the light-emitting device; in a second time period, the light-emitting device outputs corresponding light; and starting the timing of the first time length again, stopping outputting light by the light-emitting device, and repeatedly calculating and sending the light to the light-emitting device by the head-mounted device.

The whole process is that the user utilizes the light-emitting device to simulate a human hand, a mouse or other control devices to make gestures, and the head-mounted device obtains the user control gestures by identifying the position of the screen corresponding to the light emitted by the light-emitting device.

In a specific example, based on the above, it is already possible to set various specific gestures and corresponding manipulation methods in the head-mounted device in advance, and then, after recognizing the gestures, directly execute the manipulation methods corresponding to the gestures. For example, a preset "stroke left" gesture corresponds to "go back to previous page"; the preset 'hooking' gesture corresponds to 'closing the current interface', and the like.

In particular, more complex gesture manipulations can be provided, such as simulating mouse clicks of such functions.

The concrete implementation is as follows:

s6: and receiving a click signal sent by the light-emitting equipment, wherein the click signal corresponds to the coordinate position of the light currently output by the light-emitting equipment in the display.

Presetting a 'click' or 'touch' button on the light-emitting equipment in advance; then, in the process of executing the gesture, the button can be triggered to send a click signal to the head-mounted device (in a Bluetooth or infrared mode, etc.); then the head-mounted equipment instantly determines the position of the current screen corresponding to the light according to the received click signal; and finally triggering the function corresponding to the position. Which may be understood as a click mouse function.

For example, the camera shooting range is a coordinate range from (0, 0) to (1920, 1080), and the corresponding head-mounted display is 1920 × 1080 pixels; if the center positions of the plurality of pixel points of the light output by the identified light-emitting device are (100,200), displaying that a mouse pointer is suspended at the position of the coordinates (100,200) in the display screen of the head-mounted device; and receiving a click signal sent by the light-emitting device at the moment, and clicking the mouse corresponding to the coordinates.

Through the above, the realization can wear the light emitting equipment through the user and realize carrying out the gesture to the head mounted device and control to can show ground and improve the head mounted device, especially nearly dispose the gesture recognition speed of the head mounted device of single camera, and can guarantee the accuracy of discernment simultaneously.

On the basis, the embodiment also supports a method for directly utilizing the light-emitting device equipped with the gravity sensing device to perform gravity sensing control on the head-mounted device, so that the light-emitting device is fully utilized, and the problems of inconvenient operation, low control precision and the like in the process of directly utilizing the head-mounted device to perform gravity sensing control are solved.

Specifically, the method of the present embodiment includes:

s7: the head-mounted device is in communication connection with the light-emitting device through the agent application program, and is used for receiving gravity sensing data of the light-emitting device.

The proxy application program corresponds to an application split app or an android virtual machine; the target application corresponds to a WeChat, a gaming application, or other application.

For the more understandable example of comparing the head-mounted device to the mobile phone, the reason is that the normal app is to read the gravity sensing data of the mobile phone by default, but not the gravity sensing data of the external light-emitting device; the app may be given access to gravity sensing data of the light emitting device through the proxy application.

S8: the head-mounted device reads gravity sensing data from the light-emitting device through the agent application program;

specifically, this step may be realized by the following substeps:

s81: the agent application program loads a target application in the application of the agent application program;

s82: the head-mounted equipment calls a gravity sensing data acquisition request of the target application;

s83: and the agent application program loads the request into the memory and carries out byte code replacement on the gravity sensing acquisition interface so as to acquire the gravity sensing data of the light-emitting equipment.

Here, the installing of the target application by the proxy application program is similar to installing of a WeChat or game application for an application-based app, and may also be understood as creating a virtual mobile phone in the mobile phone, loading the app in the virtual mobile phone, and then intercepting and acquiring the gravity sensing data and replacing the gravity sensing data with the gravity sensing data of the light-emitting device when the app program is executed. Specifically, android app compiles java codes into Dalvik byte codes, the agent application program directly loads the Dalvik byte codes into a memory to execute, and replaces the byte codes of the gravity sensing related functions with function byte codes of the gravity sensing function of the light-emitting device during execution, so that the gravity sensing data obtained by the target application is directly the gravity sensing data of the light-emitting device. Wherein, the replacement process specifically comprises the following steps: and modifying to obtain a function byte code of the gravity sensing of the light-emitting device by searching the byte code of the sensormanager.

Accordingly, the app which does not originally support reading the gravity sensing device from the external device can read the gravity sensing data of the external device.

The embodiment can not only effectively improve the speed and the accuracy of gesture recognition; the application requiring gravity sensing can be perfectly simulated, and the defect that the head-mounted equipment cannot flexibly operate gravity sensing like a hand is overcome, so that the head-mounted equipment can support external gravity sensing control, and the control convenience and accuracy are improved; furthermore, the problem of application compatibility can be well solved through the proxy application program, and an application developer is not required to customize and develop the docking equipment again, so that the existing android ecology can be better reused.

Example two

This embodiment provides a corresponding embodiment one, and provides a specific application scenario:

1. the RGB values of pixels of each frame of image of the camera are obtained, firstly, the RGB values are grouped through manual presetting, and each group has an RGB range. For example: the groups can be divided into 9 groups of red, orange, yellow, green, blue, purple and black (the groups can also be divided into one group for each smaller rgb value interval). Then, the number of pixel points of each group is calculated according to the image acquired by the camera. For example, the 3 groups of pixels of red, orange and yellow are statistically found to be 0, or the three groups are the unused rgb values when the pixel is close to 0.

2. The most different color is output through a led ring or a led watch (here assumed to be a led ring) worn on the finger and turned off at a fixed frequency, such as 100 milliseconds. The maximum difference group was found from the red orange yellow group as the red group above. For example, if the red rgb ranges from (200, 00, 00) to (255, 00, 00), the led ring randomly outputs the colors of (200, 00, 00) to (255, 00, 00).

3. The camera also calculates the rgb range of the maximum value of rgb difference red in the led off output period according to the same frequency as (200, 00, 00) to (255, 00, 00), then the led outputs the color of the maximum difference value, and the upper led position is obtained when the led is on. After the Led is closed, the color to be displayed by the Led ring is calculated, after the Led is opened, the camera searches the positions of the pixel points in the Led color range, for example, the coordinate range of (0, 0) to (1920, 1080) is shot by the camera, a plurality of identified pixel points of the Led color are searched, 3 coordinates are initialized at random by obtaining the current Led number of 3, the plurality of pixel points are classified into 3 types by a k-means algorithm, and the central positions of the 3 types are respectively (100,200), (150, 200) (200 ).

4. Then, assuming that the head mounted display is also 1920 × 1080 pixels, the display touch point floats on the (100,200), (150, 200) (200 ) coordinates.

5. Pressing the button of the led device is equivalent to clicking a mouse.

6. The QQ sound and video application on the head-mounted device obtains the gravity sensing data in the led ring as 'direction (x1, y1, z1) and acceleration (x2, y2, z 2)', and accordingly, the device is judged to be in the range of the horizontal screen and played in a full screen mode.

The method specifically comprises the following steps:

6.1QQ sound and video calling;

getdefaultsensor (sensor, type _ ACCELEROMETER), which is a function of the target application to acquire the gravity sensing value, is redirected to the led device by replacing the byte code of the function to acquire the gravity sensing data from the led ring, since the byte code to acquire the gravity sensing is replaced by the proxy application program, the received gravity sensing data is not the gravity sensing data of the head-mounted device itself, but the gravity sensing data of the led ring.

6.2 the agent application program executes the method of acquiring the gravity sensing from the led device and returns the direction and the acceleration to the QQ video application;

and 6.3, confirming the corresponding gravity sensing event and executing the gravity sensing event by the QQ video and audio application according to the received gravity sensing data.

For example, if the screen needs to be horizontally and vertically screened, the screen can be prevented from being horizontally screened only by twisting the head, and the screen can be controlled by the hand-held LED equipment.

EXAMPLE III

The present embodiment provides a computer-readable storage medium, on which a computer program is stored, where the computer program is capable of implementing the steps included in the gesture recognition and control method for a head-mounted device according to one or two of the above embodiments when the computer program is executed by a processor. The specific steps are not repeated here, and refer to the description of the first embodiment or the second embodiment for details.

In summary, the gesture recognition and control method and the storage medium of the head-mounted device provided by the invention can improve the accuracy and recognition efficiency of single-camera gesture recognition, support the gravity sensing control through the light-emitting device, improve the convenience and control accuracy, enrich the control modes and functions, and greatly improve the product availability. Particularly, the method is applied to the head-mounted equipment with a single camera, and has remarkable effect; furthermore, the equipment (light-emitting equipment) required to be matched has the characteristics of simple structure, lightness, portability and the like, so that the scheme also has the characteristics of strong practicability and easiness in implementation.

The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.

Claims

1. A gesture recognition and control method for a head-mounted device, comprising:

2. The method of gesture recognition and control of a head-mounted device according to claim 1, further comprising:

3. The method for recognizing and controlling gestures of a head-mounted device according to claim 1, wherein the unused RGB values are obtained according to the RGB values of the pixels of each frame of image, specifically:

acquiring the RGB value of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

4. The method according to claim 3, wherein if the unused RGB values correspond to more than two groups, the unused RGB values are sent to a light emitting device, specifically:

and sending the RGB value range to a light-emitting device.

5. The method of claim 4, wherein the RGB values of the light output by the light emitting device are randomly selected from a range of received RGB values.

6. The method of gesture recognition and control of a head-mounted device according to claim 3, wherein the different RGB value ranges are RGB value ranges corresponding to respective colors.

7. The method for recognizing and controlling gestures of a head-mounted device according to claim 1, wherein the recognizing light-emitting device outputs light according to the received RGB values at coordinate positions in the display, specifically:

8. The method of gesture recognition and control of a head-mounted device according to claim 1, further comprising:

9. The gesture recognition and control method of the head-mounted device according to claim 1, wherein the head-mounted device reads the gravity sensing data from the light-emitting device through the proxy application, specifically:

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, is able to carry out the steps involved in the method for gesture recognition and control of a head-mounted device according to any one of the claims 1 to 9.