CN111796671B

CN111796671B - Gesture recognition and control method of head-mounted equipment and storage medium

Info

Publication number: CN111796671B
Application number: CN202010442052.XA
Authority: CN
Inventors: 刘德建; 陈丛亮; 郭玉湖; 陈宏�
Original assignee: Fujian TQ Digital Co Ltd
Current assignee: Fujian TQ Digital Co Ltd
Priority date: 2020-05-22
Filing date: 2020-05-22
Publication date: 2023-04-28
Anticipated expiration: 2040-05-22
Also published as: CN111796671A

Abstract

The invention provides a gesture recognition and control method and a storage medium of a head-mounted device, wherein the method comprises the following steps: acquiring unused RGB values according to the RGB values of each pixel of each frame of image within a preset first duration, and sending the unused RGB values to a light emitting device; within a preset second duration, recognizing the coordinate position of the light output by the light emitting device according to the received RGB values in the display; the first time length and the second time length are seamlessly and alternately started to time; the head-mounted device reads the gravity sensing data from the light emitting device through the agent application. The gesture recognition method and the gesture recognition device not only can improve the accuracy and recognition efficiency of gesture recognition of the single camera, but also support gravity sensing control through the light-emitting equipment, improve the operation convenience and the control accuracy, enrich the control modes and functions, and greatly improve the usability of products.

Description

Gesture recognition and control method of head-mounted equipment and storage medium

Technical Field

The invention relates to the field of gesture recognition, in particular to a gesture recognition and control method and a storage medium of head-mounted equipment.

Background

The prior art head-mounted device is worn on the head, so that operation control is difficult to perform by using a touch control mode such as a mobile phone screen. Although some headsets are already capable of supporting gesture control. However, since the headset is generally configured with only a single camera, gesture recognition by the single camera generally has the problems of complex calculation, low recognition rate, low operation sensitivity, insufficient convenience in operation and the like.

Accordingly, there is a need for providing a method of gesture recognition and control of a headset device, and a storage medium, which can overcome the above problems at the same time.

Disclosure of Invention

The technical problems to be solved by the invention are as follows: the gesture recognition and control method and the storage medium of the head-mounted equipment are provided, so that the accuracy and recognition efficiency of single-camera gesture recognition can be improved simultaneously; and the shortcut gesture control is supported, so that the operation convenience is improved.

In order to solve the technical problems, the invention adopts the following technical scheme:

the gesture recognition and control method of the head-mounted equipment comprises the following steps:

acquiring unused RGB values according to the RGB values of each pixel of each frame of image within a preset first duration, and sending the unused RGB values to a light emitting device;

within a preset second duration, recognizing the coordinate position of the light output by the light emitting device according to the received RGB values in the display;

the first time length and the second time length are seamlessly and alternately started to time;

the head-mounted device reads the gravity sensing data from the light emitting device through the agent application.

The other technical scheme provided by the invention is as follows:

a computer readable storage medium having stored thereon a computer program which, when executed by a processor, is capable of carrying out the steps comprised in the gesture recognition and control method of a head-mounted device as described above.

The invention has the beneficial effects that: the invention controls the light emitting device to output the light with the unused RGB value by acquiring the unused RGB value; and then, the movement track of the touch gesture is acquired by identifying the movement track of the display corresponding to the light. Therefore, the existing mode that the operation track of the real hand of the user can be identified by carrying out complex calculation on all the image pixels is changed into the identification mode that the gesture of the user can be quickly obtained by simply analyzing the pixels with specific RGB values. Therefore, the invention not only greatly reduces the computational complexity of recognition, but also improves the recognition efficiency; and the accuracy of identification can be ensured at the same time; particularly, aiming at the head-mounted equipment with a single camera, the gesture recognition efficiency and accuracy of the head-mounted equipment can be remarkably improved. Furthermore, the invention also supports gravity sensing control of the head-mounted equipment through the light-emitting equipment, so that the head-mounted equipment can support gravity sensing application, and the usability of the product is improved; the control accuracy and the operation convenience of the head-mounted equipment can be improved by controlling the light-emitting equipment; still further, the problem of application compatibility can be well solved by proxy application, and the cost brought by the need of re-customizing and developing the docking control equipment by the application is avoided.

Drawings

FIG. 1 is a schematic flow chart of a method for gesture recognition and control of a headset according to an embodiment of the present invention;

fig. 2 is a flowchart of a gesture recognition and control method of a headset according to an embodiment of the present invention.

Detailed Description

In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.

The most critical concept of the invention is as follows: the gesture of the user can be quickly obtained by simply analyzing the pixels with specific RGB values; meanwhile, gravity sensing control is supported to be realized through the light emitting device.

Referring to fig. 1, the present invention provides a gesture recognition and control method for a headset device, which is characterized by comprising:

From the above description, the beneficial effects of the invention are as follows:

1. the speed and the accuracy of gesture recognition can be effectively improved.

2. Gravity sensing application can be supported, and accuracy and operation convenience of head-mounted equipment control are improved.

3. The compatibility of the application is well solved by the agent application program, the need of customizing and developing the docking equipment by an application developer is avoided, and the existing android ecology can be better reused.

Further, the method further comprises the following steps:

and receiving a click signal sent by the light-emitting device, wherein the click signal corresponds to the coordinate position of the light currently output by the light-emitting device in the display.

From the above description, it is also possible to simulate the mouse click function in cooperation with gestures.

Further, the obtaining the unused RGB values according to the RGB values of each pixel of each frame of image specifically includes:

presetting more than two groups respectively corresponding to different RGB value ranges;

acquiring RGB values of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

calculating the pixel points of each group, and obtaining the group with the minimum pixel points;

and determining the RGB value in the RGB value range corresponding to the group with the minimum pixel point number as an unused RGB value.

As can be seen from the above description, the manner of dividing the RGB value ranges of each group according to the color values is helpful for centralizing one or a few groups without being dispersed into a plurality of groups when analyzing the unused pixel colors in the image captured in the first time period, so as to improve the accuracy and efficiency of the subsequent analysis and calculation.

Further, if the unused RGB values correspond to more than two groups, the sending to the light emitting device specifically includes:

respectively calculating RGB difference values of more than two groups corresponding to the unused RGB values and other groups;

acquiring an RGB value range corresponding to the group with the largest difference value with other groups;

and sending the RGB value range to a light emitting device.

As can be seen from the above description, if the unused RGB values are dispersed in more than two groups, the group with the largest difference from other groups is further selected, and the corresponding RGB value range is used as the standard of the light output by the light emitting device, so that the recognition degree of the light output by the light emitting device in the head portrait device display screen can be further improved, and the recognition accuracy is improved again.

Further, the RGB values of the light output by the light emitting device are randomly selected from the received RGB value range.

From the above description, the light emitting device can be freely selected from a given range, so that the matching degree with the light emitting device is improved, and the light emitting device can output light meeting the required RGB value.

Further, the different RGB value ranges are RGB value ranges corresponding to respective colors.

From the above description, it can be seen that the grouping is directly performed according to the color values corresponding to the respective colors, so as to improve the usability and intuitiveness of the pixel grouping result.

Further, the identifying the coordinate position of the light output by the light emitting device according to the received RGB values in the display specifically includes:

controlling a light emitting device to emit light corresponding to the received RGB values;

searching pixel points in the current frame image, which correspond to RGB values sent to the light emitting equipment, and acquiring coordinate positions of the pixel points;

and acquiring the movement track of the display corresponding to the light in the second time according to the coordinate position of each frame of image in the second time.

As can be seen from the above description, by locating specific RGB values in the images and comparing the time sequence combination, the control gesture made by the user through the light emitting device can be obtained.

Further, the method further comprises the following steps:

the head-mounted device executes a corresponding gravity sensing event according to the acquired gravity sensing data.

From the above description, the head-mounted device can directly implement corresponding gravity sensing control according to the gravity sensing data of the light-emitting device.

Further, the head-mounted device reads gravity sensing data from the light-emitting device through the agent application program, specifically:

the head-mounted device is in communication connection with the light-emitting device through a proxy application program in the head-mounted device;

the agent application program loads a target application in the application;

the head-mounted equipment invokes a gravity sensing data acquisition request of the target application;

and the agent application program loads the request into the memory, and replaces the byte codes of the gravity sensing acquisition interface to acquire the gravity sensing data of the light emitting device.

According to the description, the problem that the acquisition of the gravity sensing data of the external equipment is not supported is solved by loading the target application and modifying the byte code of the gravity sensing related function through the proxy application program, the problem that the head-mounted equipment can simply acquire the gravity sensing data of the external light-emitting equipment and control the application according to the gravity sensing data is solved, and the problems that the head-mounted equipment is inconvenient to perform gravity sensing control operation and low in control precision are solved.

The other technical scheme provided by the invention is as follows:

a computer readable storage medium having stored thereon a computer program which, when executed by a processor, is capable of implementing the steps comprised in a method of gesture recognition and control of a headset device:

Further, the method further comprises the following steps:

acquiring RGB values of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

and sending the RGB value range to a light emitting device.

Further, the method further comprises the following steps:

the agent application program loads a target application in the application;

From the foregoing description, it will be appreciated by those skilled in the art that the foregoing embodiments may be implemented, in whole or in part, by hardware, by a computer program, where the program may be stored on a computer readable storage medium, where the program, when executed, may include the steps of the methods described above. After the program is executed by the processor, the beneficial effects corresponding to the methods can be realized.

The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.

Example 1

Referring to fig. 2, the present embodiment provides a gesture recognition and control method for a headset device, which can significantly improve gesture recognition efficiency and gesture recognition accuracy; meanwhile, the gravity sensing control of the light-emitting device on the head-mounted device is supported, the control function is enriched, and meanwhile, the accuracy and the operation convenience of the gravity sensing control are improved. The light emitting device may be any device capable of emitting light corresponding to a specified RGB value, such as a ring or a bracelet provided with LEDs.

The method comprises the following steps:

s1: presetting more than two groups respectively corresponding to different RGB value ranges;

that is, one group corresponds to one RGB value range. Preferably, a group corresponds to a range of color values. For example, the three-dimensional image can be divided into 9 groups of red, orange, yellow, green, blue, violet, and black, wherein the "red" group corresponds to a cuboid space of which the RGB values range from (200, 50, 50) to (255, 0), namely a cuboid of which the length and width are 50 and the height is 55.

Of course, the grouping may also be finer, such as grouping each small RGB value interval.

Red color: (200, 50, 50) to (255, 0);

orange: (200, 100, 50) to (255, 50, 0);

yellow: (200, 150, 50) to (255, 100, 0);

green: (0, 255, 0) to (50, 200, 50);

and (3) green: (0, 255, 50) to (50, 200, 100);

blue: (0, 255) to (50, 50, 200);

purple: (50, 0, 255) to (100, 50, 200);

preferably, the red-orange-yellow-green-blue-violet partition can also be a range classified by polar LAB and reconverted to RGB.

S2: presetting a first time length and a second time length;

preferably the first time period and the second time period are equal, such as 100ms.

S3: and in the first time period, the head-mounted device acquires unused RGB values according to the RGB values of each pixel of each frame of image and sends the unused RGB values to the light-emitting device.

Specifically, the method comprises the following steps:

s31: RGB values for each pixel of each frame of image are acquired. The RGB value of each pixel in each frame of image shot by the camera in the first time period is obtained.

S32: each pixel is divided into a corresponding group according to the RGB values. That is, each pixel acquired in step S31 is grouped according to its RGB value, and is divided into groups corresponding to the RGB value ranges.

S33: and calculating the pixel points of each group, and obtaining the group with the minimum pixel points. That is, the number of pixels included in each group is calculated, and a group having the smallest number of pixels is acquired, and if the acquired number of groups is one, the group may be regarded as the largest difference from the other groups.

In another embodiment, the number of groups with the smallest number of pixel points finally obtained in the step S33 is more than two, and the discrimination of the light output by the light emitting device can be improved by further calculating one group with the largest difference from all other groups.

For example, if the pixels included in the group 3 of "red", "orange", "yellow" are all 0 or close to 0, the RGB value ranges corresponding to the three groups are unused RGB values.

Corresponding to the other specific example described above, the group with the largest difference can be determined therefrom by:

s34: and (3) respectively calculating the RGB difference values of more than two groups corresponding to the unused RGB values acquired in the step (S33) and all other groups to determine.

Taking the above 9 groups of red, orange, yellow, green, blue, violet, black and white as an example, and determining that the three groups of red, orange and yellow have the least pixels and are equal after the step S33, the unused RGB values are corresponding to the RGB values of the three groups of red, orange and yellow. In order to further improve the discrimination of the light emitted by the light emitting device. The one of the three groups "red", "orange", "yellow" which is most different from the 9 groups of red, orange, yellow, green, blue, violet, black and white can be further calculated. The calculation process may be: the RGB values of the three groups "red", "orange", "yellow" are subtracted from the other groups having pixels (6 groups of cyan, magenta and black) to obtain the group having the largest difference in the three components (R, G, B). The formula is: difference values d12= (R1-R2) 2+ (G1-G2) 2+ (B1-B2) 2 of group 1, group 2; the difference value dij is obtained for the red, orange, yellow 3 groups and the green, blue and purple 4 groups. Wherein, the numbers of red, orange, yellow, green, blue and purple are respectively 1234567; the maximum difference value=max (min (d 14, d15, d16, d 17), min (d 24, d25, d26, d 27), min (d 34, d35, d36, d 37)).

The red group is assumed to be the maximum value d 14.

S35: if only one group corresponds to the unused RGB values, the RGB values in the RGB value range corresponding to the group may be directly transmitted to the light emitting device.

If the group with the least pixel points corresponds to more than two groups, the corresponding RGB value range can be directly sent to the light-emitting device, one RGB value of the corresponding RGB value range is selected for light sending accidents, and one group with the highest identification degree can be screened out of the more than two groups and then the RGB value range is sent to the light-emitting device.

That is, regardless of the number of groups to which the unused RGB values correspond, it is preferable to transmit the RGB value having the largest difference to the light emitting device; of course, it is also possible to choose to send all unused RGB values to the light emitting device.

It should be noted that the head-mounted device transmits the RGB value range to the light emitting device through the light emitting device connected to the bluetooth.

S4: during the second period of time, the headset identifies the coordinate position in the display of the light output by the light emitting device in accordance with the RGB value range it receives.

The method specifically comprises the following steps:

s41: and in the second time period, controlling the light emitting device to emit corresponding light according to the received RGB value range.

Preferably, if the unused RGB values correspond to more than two color values (i.e., two grouped RGB value ranges), the light emitting device may randomly select the RGB value for output.

In a specific example, if the maximum difference set is "red" set, i.e. only one set of corresponding RGB values is obtained. The light emitting device can also randomly output the colors (200, 50, 50) to (255, 0) from the light emitting device if the RGB value range corresponding to the "red" group is a cuboid space, i.e., a cuboid with the length and width of 50 and the height of 55, of (200, 50, 50) to (255, 0).

S42: the head-mounted device searches pixel points corresponding to RGB values sent to the light-emitting device in the current frame image according to the image shot by the camera (the camera still shoots in real time in the second time duration), and acquires the coordinate positions of the pixel points.

S43: and acquiring the movement track of the display corresponding to the light in the second time according to the coordinate position determined in the last step of each frame image in the second time.

That is, the head-mounted device only needs to identify the pixel points corresponding to the RGB values transmitted to the light-emitting device in the image photographed in real time and locate the coordinate positions thereof in the figure; and then connecting the coordinate positions according to the time sequence, so that the gesture of the light output by the light-emitting device in the second time period corresponding to the screen of the head-mounted device, namely the gesture made by the user through the light-emitting device, can be obtained.

S5: and starting timing in turn of the first time length and the second time length in a seamless mode.

Correspondingly, the above S3 and S4 are alternately executed, and the first time period is started. In particular, during the first period of time, the light emitting device will cease outputting any light, and will output the corresponding light only when the RGB value range transmitted by the head-mounted device is received.

That is, during the first period of time, the head-mounted device calculates and transmits the calculation result to the light-emitting device; outputting corresponding light by the light emitting device in a second time period; the timing of the first time period is started again, the light emitting device stops outputting light, and the head-mounted device performs calculation again and sends the calculation to the light emitting device, and the cycle is repeated.

The whole process is a process that a user makes a gesture by using the light-emitting device to simulate a human hand, a mouse or other control devices, and the head-mounted device acquires the user control gesture by identifying the position of light emitted by the light-emitting device corresponding to a screen.

In a specific example, based on the above, setting various specific gestures and corresponding control methods of the specific gestures on the headset device in advance can be realized, and then after the gestures are identified, the control methods corresponding to the gestures are directly executed. For example, a gesture of preset "swipe left" corresponds to "back to previous page"; the gesture of presetting "hooking" corresponds to "closing the current interface", and so on.

In particular, more complex gesture manipulations, such as simulating mouse clicks, can also be provided.

The specific implementation is as follows:

s6: and receiving a click signal sent by the light-emitting device, wherein the click signal corresponds to the coordinate position of the light currently output by the light-emitting device in the display.

Presetting a click or touch button on the light-emitting device in advance; then, in the process of executing the gesture, the button can be triggered to send a click signal to the head-mounted device (in a Bluetooth or infrared mode, etc.); then the head-mounted equipment determines the position of the current light corresponding to the screen according to the received click signal in real time; and finally triggering the function corresponding to the position. It is understood as a function of clicking a mouse.

For example, the camera shooting range is a coordinate range of (0, 0) to (1920, 1080), and the corresponding head-mounted display is 1920×1080 pixels; the central position of the plurality of pixel points of the light output by the identified light emitting device is (100, 200), and the display mouse pointer is suspended at the position of the coordinates (100, 200) in the display screen of the head-mounted device; when the clicking signal sent by the light-emitting device is received at the moment, the mouse is clicked corresponding to the coordinates.

Through the method, the user can wear the light-emitting device to realize gesture control on the head-mounted device, the gesture recognition speed of the head-mounted device, particularly the gesture recognition speed of the head-mounted device near which the single camera is arranged, can be obviously improved, and the recognition accuracy can be guaranteed at the same time.

On the basis of the above, the embodiment also supports a method for carrying out gravity sensing control on the head-mounted device by directly utilizing the light-emitting device provided with the gravity sensing device, thereby not only fully utilizing the light-emitting device, but also overcoming the problems of inconvenient operation, low control precision and the like existing in the gravity sensing control by directly utilizing the head-mounted device.

Specifically, the method of the present embodiment includes:

s7: the head-mounted device is in communication connection with the light-emitting device through a proxy application program therein and is used for receiving gravity sensing data of the light-emitting device.

The proxy application program corresponds to an application splitting app or an android virtual machine; the target application corresponds to a letter, a gaming application, or other application.

Taking the example of a headset device as a mobile phone, which is easier to understand, because the normal app is the gravity sensing data of the default read mobile phone, but not the gravity sensing data of the external light emitting device; the app can be given access to gravity sensing data of the lighting device by the proxy application.

S8: the head-mounted device reads gravity sensing data from the light-emitting device through the agent application program;

specifically, this step may be achieved by the following sub-steps:

s81: the agent application program loads a target application in the application;

s82: the head-mounted equipment invokes a gravity sensing data acquisition request of the target application;

s83: and the agent application program loads the request into the memory, and replaces the byte codes of the gravity sensing acquisition interface to acquire the gravity sensing data of the light emitting device.

Here, the proxy application installation target application is similar to an application splitting app installation WeChat or game application, and can be understood as creating a virtual mobile phone in the mobile phone, loading the app in the virtual mobile phone, and then intercepting and acquiring gravity sensing data to replace the gravity sensing data of the light emitting device when executing the app. Specifically, the android app compiles java codes into Dalvik byte codes, and the agent application program directly loads the Dalvik byte codes into the memory for execution, and replaces the byte codes of the gravity sensing related functions with function byte codes for acquiring gravity sensing of the light emitting device during execution, so that gravity sensing data acquired by the target application are directly gravity sensing data of the light emitting device. The replacing process specifically comprises the following steps: the function byte code for acquiring the gravity sensing of the lighting device is modified by searching the byte code of the sensor manager.

Accordingly, the app which does not support the reading of the gravity sensing device from the external device can read the gravity sensing data of the external device.

The embodiment not only can effectively improve the speed and accuracy of gesture recognition; the application needing gravity sensing can be perfectly simulated, and the defect that the head-mounted equipment cannot flexibly operate the gravity sensing like hands is overcome, so that the head-mounted equipment can support the outside to perform gravity sensing control, and the control convenience and accuracy are improved; furthermore, the problem of application compatibility can be well solved through the proxy application program, and the need of an application developer to re-customize and develop the docking device is avoided, so that the existing android ecology can be better reused.

Example two

The present embodiment provides a corresponding embodiment, and provides a specific application scenario:

1. by acquiring RGB values of pixels of each frame of image of a camera, firstly, manually preset groups are arranged, and each group has an RGB range. For example: can be divided into 9 groups of red, orange, yellow, green, blue, violet, black and white (groups can also be divided into groups of finer each small rgb value interval). Then, the image acquired by the camera is respectively calculated to the pixel point number of each group. For example, if the statistics find that the pixel points of the 3 groups are 0, or are close to 0, the three groups are unused rgb value groups.

2. The most different colors are output by the led ring or the led watch (here, assuming an led ring) worn on the finger and turned off at a fixed frequency, for example, 100ms. The maximum difference group was found to be the red group from the red-orange group above. For example, red, with an rgb range of (200, 00, 00) to (255, 00, 00), the led ring outputs colors of (200, 00, 00) to (255, 00, 00) randomly.

3. The camera calculates the rgb range of the maximum value red of the rgb difference in the led off output period to be (200, 00, 00) to (255, 00, 00) according to the same frequency, then the led outputs the color of the maximum difference value, and the upper led position is acquired when the led is on. After the Led is closed, calculating the color which the Led ring should display, after the Led is opened, searching the positions of the pixel points in the Led color range by using a camera, for example, the coordinate ranges of (0, 0) to (1920, 1080) of the camera shooting range, identifying a plurality of pixel points of the Led color, randomly initializing 3 coordinates by acquiring the current Led number 3, dividing the plurality of pixel points into 3 types by a k-means algorithm, and obtaining the central positions of the 3 types to be (100, 200), (150, 200) (200 ) respectively.

4. Then assuming the head mounted display is also 1920x1080 pixels, the display touch point floats on the (100, 200), (150, 200) (200 ) coordinates.

5. Pressing the button of the led device corresponds to clicking the mouse.

6. The QQ audio and video application on the head-mounted device judges that the device is in a horizontal screen range and plays full screen by acquiring the gravity sensing data in the led ring as ' directions (x 1, y1, z 1) ', and accelerations (x 2, y2, z 2) '.

The method specifically comprises the following steps:

6.1QQ sound and video call;

the gravity sensing is obtained by the sensor manager, the type-ACCELERROMETER, which is a function of the gravity sensing value obtained by the target application, the redirection into the led device is achieved by replacing this function bytecode, and the gravity sensing data received is not the gravity sensing data of the head-mounted device itself, but the gravity sensing data of the led ring, since the bytecode for obtaining the gravity sensing is replaced by the proxy application by the bytecode of the function for obtaining the gravity sensing data from the led ring.

6.2, the agent application program executes a method for acquiring gravity sensing from the led equipment, and returns a direction and acceleration to the QQ audio-visual application;

and 6.3. The QQ video and audio application confirms the corresponding gravity sensing event according to the received gravity sensing data and executes the gravity sensing event.

For example, a user can turn the head to turn the screen horizontally and vertically, and the screen can be controlled through the led equipment on the hand.

Example III

The present embodiment provides a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, is capable of implementing the steps involved in the gesture recognition and control method of a headset device described in the first or second embodiment. The specific steps will not be repeated here, and the details will be described in the first or second embodiment.

In summary, the gesture recognition and control method and the storage medium for the head-mounted device provided by the invention can improve the accuracy and recognition efficiency of single-camera gesture recognition, support gravity sensing control through the light-emitting device, improve the operation convenience and control accuracy, enrich the control modes and functions, and greatly improve the product availability. Particularly, the device is applied to the head-mounted equipment with a single camera, and has remarkable effect; further, the equipment (light-emitting equipment) for realizing the needed matching has the characteristics of simple structure, light weight, portability and the like, so that the scheme also has the characteristics of strong practicability and easiness in realization.

The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.

Claims

1. The gesture recognition and control method of the head-mounted equipment is characterized by comprising the following steps of:

the head-mounted device reads gravity sensing data from the lighting device through the agent application program:

the agent application program replaces the function byte codes of the gravity sensing related function by the function byte codes for acquiring the gravity sensing of the light emitting device, so that the gravity sensing data acquired by the target application is the gravity sensing data of the light emitting device;

the unused RGB values are obtained according to the RGB values of each pixel of each frame of image, specifically:

acquiring RGB values of each pixel of each frame of image;

dividing each pixel into a corresponding group according to the RGB value;

determining RGB values in an RGB value range corresponding to the group with the minimum pixel points as unused RGB values;

the coordinate position of the light output by the identification light emitting device according to the received RGB value in the display is specifically:

2. The method of gesture recognition and control of a headset of claim 1, further comprising:

3. The method for gesture recognition and control of a headset according to claim 1, wherein if the unused RGB values correspond to more than two groups, the sending to the light emitting device is specifically:

and sending the RGB value range to a light emitting device.

4. A method of gesture recognition and control of a headset as claimed in claim 3, wherein the RGB values of the light output by the lighting device are randomly selected from a range of RGB values received.

5. The method of gesture recognition and control of a headset of claim 1, wherein the different RGB value ranges are RGB value ranges corresponding to respective colors.

6. The method of gesture recognition and control of a headset of claim 1, further comprising:

7. The method for gesture recognition and control of a headset according to claim 1, wherein the headset reads gravity sensing data from the lighting device through a proxy application program, specifically:

the agent application program loads a target application in the application;

8. A computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, is capable of implementing the steps comprised in the gesture recognition and control method of a head-mounted device according to any of the preceding claims 1-7.