WO2019000817A1

WO2019000817A1 - Control method and electronic equipment for hand gesture recognition

Info

Publication number: WO2019000817A1
Application number: PCT/CN2017/112312
Authority: WO
Inventors: 丁琦城; 吴梦
Original assignee: 联想(北京)有限公司
Priority date: 2017-06-29
Filing date: 2017-11-22
Publication date: 2019-01-03
Also published as: CN107273869B; CN107273869A

Abstract

Disclosed are a control method and an electronic equipment for hand gesture recognition. The hand gesture recognition control method is applicable to the electronic equipment, wherein the method comprises: acquiring a first image from when hand gesture input is carried out, the first image comprising an image of a hand and at least part of an arm; on the basis of the first image, determining whether the hand gesture input is performed by a specific user; and if the determination result is yes, then performing recognition on the hand gesture input; if not, ignoring the hand gesture input.

Description

Gesture recognition control method and electronic device

Technical field

The present invention relates to the technical field of gesture recognition, and more particularly to a gesture recognition control method capable of distinguishing gesture input of a specific user and an electronic device to which the method is applied.

Background technique

Currently, gesture recognition modules have been added to more and more electronic devices. The user's experience can be improved by triggering the corresponding instruction by the recognized gesture. During use of such an electronic device, it may only be desirable to identify the gesture input for a particular user. However, in addition to being able to acquire a hand image for a gesture input by a specific user, it is often possible to acquire a hand image of a user other than a specific user. Thus, there is a case where the electronic device regards the gesture of another person as a gesture of a specific user, thereby causing an erroneous reaction.

As a possible solution, data sources for specific users are indicated by using data gloves or controllers with sensors. However, the disadvantage of this scheme is that although it can effectively distinguish whether it is a gesture of a specific user, more accessories (gloves or controllers) are added, which increases the cost on the one hand and reduces the convenience of the user on the other hand. Overall experience.

Summary of the invention

In view of the above circumstances, it is desirable to provide a gesture recognition control method and apparatus capable of distinguishing gestures of specific users and other users without adding redundant equipment or increasing user burden.

According to an aspect of the present invention, a gesture recognition control method is provided for an electronic device, the method comprising: acquiring a first image when a gesture input is performed, the first image including a hand and at least a part of an arm An image; based on the first image, determining whether the gesture input is made by a specific user; and if the determination result is yes, performing recognition on the gesture input, otherwise ignoring the gesture input.

Preferably, the method according to an embodiment of the present invention may further include: pre-storing a three-dimensional model of a specific user's arm for performing a gesture operation; wherein based on the first image, determining whether the gesture input is made by a specific user The step further includes: determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image, extracting features of different parts; and acquiring features of different parts of the arm A comparison is made with features in the corresponding three-dimensional model and it is determined whether the gesture input is made by the particular user.

Preferably, in the method according to the embodiment of the present invention, the step of pre-storing a three-dimensional model of a specific user's arm for performing a gesture operation further comprises: acquiring a plurality of arms of the specific user in a plurality of gesture operations Image For each of the plurality of arm images, different parts of the arm are decomposed and features of different parts are extracted; and a plurality of features of the same part are fused for the same arm, and a three-dimensional model for the arm is obtained.

Preferably, in the method according to the embodiment of the invention, the step of comparing the acquired features of the different parts of the arm with the features in the corresponding three-dimensional model further comprises: determining the size and the different parts of the arm in the first image Whether the difference between the corresponding sizes in the three-dimensional model is less than a predetermined threshold, obtaining a first determination result; and/or determining whether an angle between the forearm and the boom in the first image is determined based on the three-dimensional model Within a predetermined range, obtaining a second determination result, wherein determining whether the gesture input is made by the specific user comprises: determining whether the gesture input is based on the first determination result and/or the second determination result Made by the specific user.

Preferably, after determining whether the gesture input is made by a specific user based on the first image, the method according to an embodiment of the present invention may further include: after determining that the gesture input is made by a specific user, determining Whether the user's undo operation is received; after determining that the gesture input is made by a specific user, determining whether the recognized gesture input is completely unrelated to the current task being executed by the electronic device; determining whether the first image is There are other people appearing; after determining that the gesture input is not made by a specific user, determining whether to receive the repeated input of the gesture; based on at least one of the above determination results, determining whether the gesture input is made by a specific user Whether the determination is correct to update the corresponding three-dimensional model with the arm features in the first image.

According to another aspect of the present invention, an electronic device includes: an image acquisition module, configured to acquire a first image when a gesture input is performed, the first image including an image of a hand and at least a portion of an arm; and a memory And a processor configured to, when executing the program stored on the memory, implement a function of: determining, based on the first image, whether the gesture input is made by a specific user; and if the determination result is Yes, the recognition is performed on the gesture input, otherwise the gesture input is ignored.

Preferably, in the device according to an embodiment of the present invention, the memory is further configured to pre-store a three-dimensional model of an arm of a specific user for performing a gesture operation; wherein the processor is configured to execute the program to further Implementing a function of determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image, extracting features of different parts; and acquiring features of different parts of the arm A comparison is made with features in the corresponding three-dimensional model and it is determined whether the gesture input is made by the particular user.

Preferably, in the device according to an embodiment of the invention, the processor is configured to execute the program to further implement the following functions:

Separating different parts of the arm and extracting features of different parts for each of the plurality of arm images of the specific user acquired by the image acquisition unit in the plurality of gesture operations;

A plurality of features of the same portion are fused for the same arm, and a three-dimensional model for the arm is obtained.

Preferably, in the device according to an embodiment of the invention, the processor is configured to execute the program to further implement The function of determining whether a difference between a size of a different part of the arm in the first image and a corresponding size in the three-dimensional model is less than a predetermined threshold, obtaining a first determination result; and/or determining the first image Whether the angle between the middle forearm and the boom is within a predetermined range determined based on the three-dimensional model, obtaining a second determination result; and determining the gesture input based on the first determination result and/or the second determination result Whether it is made by the specific user.

Preferably, in the device according to an embodiment of the present invention, the processor is configured to execute the program to further implement a function of: determining whether the user's revocation is received after determining that the gesture input is made by a specific user After determining that the gesture input is made by a specific user, determining whether the recognized gesture input is completely unrelated to the current task being executed by the electronic device; determining whether another person appears in the first image; Determining whether the gesture input is not made by a specific user, determining whether a repeated input of the gesture is received; and determining, based on at least one of the above determination results, whether the determination as to whether the gesture input is made by a specific user is correct And updating the corresponding three-dimensional model with the arm feature in the first image.

In the gesture recognition control method and electronic device according to the present invention, the accuracy of a specific user's own gesture recognition is improved by image processing of the entire arm instead of a single palm. Moreover, the modeling process is completed and perfected by using the data determined as the arm of the specific user during use, so that the more the three-dimensional model of the arm is used, the higher the accuracy. In addition, in the gesture recognition control method and the electronic device according to the present invention, no unnecessary device is added or the user's use burden is increased (for example, the user is actively selected whether the gesture is made by himself), thereby reducing the cost and improving the cost. user experience.

DRAWINGS

1 is a flowchart illustrating a procedure of a gesture recognition control method according to an embodiment of the present invention;

FIG. 2 is a functional block diagram illustrating a configuration of an electronic device according to an embodiment of the present invention.

Detailed ways

Various preferred embodiments of the present invention will now be described with reference to the accompanying drawings. The following description with reference to the accompanying drawings will be understood to It includes various specific details to help understanding, but they can only be considered as exemplary. Accordingly, it will be appreciated by those skilled in the art that various modifications and changes may be made to the embodiments described herein without departing from the scope and spirit of the invention. Further, detailed descriptions of well-known functions and constructions in the art are omitted for clarity and conciseness.

The terminology used herein is for the purpose of describing the particular embodiments, The use of the terms "comprising", "comprising" or "an"

All terms (including technical and scientific terms) used herein have the meaning commonly understood by one of ordinary skill in the art. Unless otherwise defined. It should be noted that the terms used herein are to be interpreted as having a meaning consistent with the context of the present specification and should not be interpreted in an ideal or too rigid manner.

Where an expression similar to "at least one of A, B, and C, etc." is used, it should generally be interpreted in accordance with the meaning of the expression as commonly understood by those skilled in the art (for example, "having A, B, and C" "Systems of at least one of" shall include, but are not limited to, systems having A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ). Where an expression similar to "at least one of A, B or C, etc." is used, it should generally be interpreted according to the meaning of the expression as commonly understood by those skilled in the art (for example, "having A, B or C" "Systems of at least one of" shall include, but are not limited to, systems having A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ). Those skilled in the art will also appreciate that transitional conjunctions and/or phrases that are arbitrarily arbitrarily representing two or more optional items, whether in the specification, claims, or drawings, are to be construed as The possibility of one of the projects, either or both of these projects. For example, the phrase "A or B" should be understood to include the possibility of "A" or "B", or "A and B."

Some block diagrams and/or flowcharts are shown in the drawings. It will be understood that some blocks or combinations of the block diagrams and/or flowcharts can be implemented by computer program instructions. These computer program instructions may be provided to a general purpose computer, a special purpose computer or a processor of other programmable data processing apparatus such that when executed by the processor, the instructions may be used to implement the functions illustrated in the block diagrams and/or flowcharts. / operating device.

Thus, the techniques of this disclosure may be implemented in the form of hardware and/or software (including firmware, microcode, etc.). Additionally, the techniques of this disclosure may take the form of a computer program product on a computer readable medium storing instructions for use by or in connection with an instruction execution system. In the context of the present disclosure, a computer readable medium can be any medium that can contain, store, communicate, propagate or transport the instructions. For example, a computer readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the computer readable medium include: a magnetic storage device such as a magnetic tape or a hard disk (HDD); an optical storage device such as a compact disk (CD-ROM); a memory such as a random access memory (RAM) or a flash memory; and/or a wired /Wireless communication link.

First, a gesture recognition control method according to the present invention will be described with reference to FIG. 1. The gesture recognition control method is applied to an electronic device. For example, the electronic device can be a cell phone, a tablet, a smart TV, or a wearable device. As shown in FIG. 1, the gesture recognition control method includes the following steps.

First, in step S101, a first image when a gesture input is performed is acquired, the first image including an image of a hand and at least a part of an arm.

Then, in step S102, based on the first image, it is determined whether the gesture input is made by a specific user.

If the result of the determination in step S102 is YES, the process proceeds to step S103. In step S103, the gesture is lost Into the execution identification. Then, based on the recognized gesture, the corresponding function is activated. On the other hand, if the result of the determination in step S102 is NO, the process proceeds to step S104. At step S104, the gesture input is ignored. That is, the gesture input is not recognized and responded.

It can be seen that in the gesture recognition control method according to the present invention, when the user performs gesture input, not only the hand image but also at least part of the arm image is acquired, compared with the prior art. Depending on the user's arm posture and/or the positional relationship of the arm to the image acquisition module (eg, depth camera), it may be possible to capture images of all of the arms, or only images that include part of the arm. Thus, the gesture recognition control method according to the present invention can determine whether a gesture is made for a specific user based on the arm image, avoiding the erroneous recognition of gestures of other users that should not be recognized.

For example, in an application scenario where the electronic device is a mobile phone, a tablet computer, a smart TV, or the like, in the gesture recognition method according to the prior art, a specific user can be verified by additional facial recognition, and only the hand is collected after the verification is passed. Image for gesture recognition. That is to say, in the gesture recognition method according to the related art, it is possible to confirm whether it is a specific user by face recognition, but it is not possible to confirm whether the current gesture is made by the identified specific user. In contrast, with the gesture recognition control method according to the present invention, it is possible to confirm whether the gesture input is made by a specific user, thereby avoiding a misjudgment caused when a specific user's face is recognized but the gesture is made by another person.

In addition, with the gradual maturity of virtual reality and augmented reality technology, head-mounted display devices will be the most promising interactive interface rendering devices in augmented reality display devices. In an application scenario where the electronic device is such a head mounted display device, in addition to being able to acquire a hand image of the wearer for gesture recognition, it is also possible to acquire the hand of another user other than the wearer. image. Therefore, there is a case where the head-mounted display device regards the gesture of another person as the gesture of the wearer, thereby causing an erroneous reaction. In the scene where many people wear head-mounted display devices at the same time in the future, the frequency and severity of such misrecognition will be higher. With the gesture recognition control method according to the present invention, it can be confirmed whether the gesture input is made by the wearer, and the gesture of mistaking the gesture of the wearer of the other head mounted display device as the wearer of the current head mounted display device is effectively avoided. Case.

Based on the first image, determining whether the gesture input is made by a specific user may include, but is not limited to, the following manners: (1) determining whether the gesture is satisfied by a specific user based on the position of the arm in the first image. Rationality; (2) Based on the three-dimensional model of the arm to determine whether the similarity with the gesture input by the specific user is satisfied.

Next, the two methods will be explained separately.

(1) judging whether the satisfaction of the specific user making the gesture input is based on the position of the arm in the first image

For example, in the case where the electronic device is a mobile phone, a tablet computer, a smart TV, etc., step S102 in FIG. 1 may further include: confirming whether it is a specific user based on a relative relationship between the arm and the verified face in the first image. The gesture made. If the distance between the arm and the face exceeds a predetermined threshold, or if the angle between the arm and the face exceeds a predetermined angular range, then the relative relationship between the arm and the face is unreasonable, and it can be determined that the gesture input is not Made by a specific user. On the other hand, if it is not judged that the relative relationship between the arm and the face is unreasonable, then the gesture input is temporarily considered to be made by a specific user.

For example, in the case where the electronic device is a head mounted display device, it is confirmed based on the position of the arm in the first image whether it is a gesture made by a specific user (in this case, a wearer). Specifically, since the image of the wearer's arm is photographed by the camera worn on the head mounted display device of the head, in the obtained first image, the left arm image may not be located in the right side region, and the right arm The image cannot be in the left area. Therefore, step S102 in FIG. 1 may further include first determining whether the arm in which the gesture input is performed in the first image is the left arm or the right arm. If it is determined to be the left arm, it is determined whether the left arm appears in the right area, and if so, the position of the arm is unreasonable, and it can be determined that the gesture input is not made by a specific user. On the other hand, if it is not judged that the positional relationship of the arm is unreasonable, it is temporarily considered that the gesture input is made by a specific user. Similarly, if it is determined to be the right arm, it is determined whether the right arm is present in the left area, and if so, the position of the arm is unreasonable, and it can be determined that the gesture input is not made by a specific user. On the other hand, if it is not judged that the positional relationship of the arm is unreasonable, it is temporarily considered that the gesture input is made by a specific user.

(2) Based on the three-dimensional model of the arm to determine whether the similarity with the gesture input by the specific user is satisfied

In this way, it is judged whether the similarity with the gesture input by the specific user is satisfied, and it is necessary to store in advance the three-dimensional model of the arm of the specific user for performing the gesture operation. Here, two three-dimensional models of the arms of the specific user may be stored in advance so that the gesture of any arm of the user can be recognized. Alternatively, it is also possible to store only the three-dimensional model of one arm of the specific user, so that only the gesture of the specific arm of the user can be recognized.

Wherein the step of pre-storing the three-dimensional model for the arm of the specific user further comprises: acquiring a plurality of arm images of the specific user in the multiple gesture operation; and decomposing the arm for each of the plurality of arm images Different parts, and extracting features of different parts; and merging multiple features of the same part for the same arm, and obtaining a three-dimensional model for the arm.

In the gesture recognition control method according to the present invention, the recognition range of the depth camera and the machine learning algorithm are used to record the characteristics of the specific user's arm at various angles, and the whole body is learned and modeled, not only It is only the recognition of finger movements. Here, it should be noted that in the process of obtaining a gesture operation using the depth camera, since the position and direction in which the arm is placed are different, the degree of integrity of the arm that can be obtained is different. However, as the number of gestures is accumulated, the three-dimensional data of all the arms is gradually obtained. After a certain amount of data learning, that is, after acquiring a sufficient arm image of a specific user in multiple gesture operations, a model for a specific user's arm can be established. Different parts of the arm include: hand, forearm, and big arm. The three-dimensional model of the arm includes: size information of the forearm and the big arm, angular range of the forearm and the big arm, and characteristic images on the forearm and the big arm (eg, personalized joints, bumps, etc.). When a new arm position appears, the model can be used to determine if it is a particular user's own gesture.

Step S102 in FIG. 1 may further include: determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image, extracting features of different parts; and acquiring The features of the different parts of the arm are compared to the features in the corresponding three-dimensional model and it is determined whether the gesture input is made by the particular user.

The step of comparing the acquired features of different parts of the arm with the features in the corresponding three-dimensional model further includes:

Determining whether a difference between a size of a different part of the arm in the first image and a corresponding size in the three-dimensional model is less than a predetermined threshold, obtaining a first determination result; and/or

Determining whether the angle between the forearm and the boom in the first image is within a predetermined range determined based on the three-dimensional model, obtaining a second determination result.

Wherein, determining whether the gesture input is made by the specific user comprises determining whether the gesture input is made by the specific user based on the first determination result and/or the second determination result. Specifically, if there is a judgment result of the first judgment result and/or the second judgment result, it is determined that the gesture input is not made by the specific user, otherwise it is determined that the gesture input is performed by the specific user. Out.

The two methods described above for determining whether the gesture input is made by a particular user may be used separately. Alternatively, these two methods can also be used serially. For example, first judge in terms of rationality and then judge in terms of similarity. Since the calculation amount of the rationality judgment is small, the rationality is used as the screening condition, and the calculation amount based on the judgment of the three-dimensional model of the arm can be effectively reduced.

As can be seen from the above description, in the gesture recognition control method according to the present invention, the principle of "doubt is never" is adopted. That is, as long as it is not certain that the gesture input is not made by a particular user, then the gesture input is temporarily considered to be made by a particular user. For example, when it is not judged to be unreasonable based on the first image, or when it is not possible to determine whether the gesture input is made by a specific user based on the current arm three-dimensional model, then the gesture input is temporarily considered to be made by a specific user.

Therefore, for example, when other users who have an arm size similar to the arm size of a specific user perform gesture input, a misjudgment may occur. In order to avoid the occurrence of such a situation and further improve the accuracy of the determination, in the gesture recognition control method according to the present invention, after determining whether the gesture input is made by a specific user based on the first image, Further includes a verification step.

Specifically, the verifying step may include the following processing:

(1) After determining that the gesture input is made by a specific user, it is determined whether the user's undo operation is received. In general, if the user's undo operation is received after determining that the gesture input is made by a particular user, then the gesture input is most likely not made by a particular user, ie, the probability of a false positive is greater.

(2) After determining that the gesture input is made by a specific user, it is determined whether the recognized gesture input is completely unrelated to the current task being executed by the electronic device. In general, if it is determined that the gesture input made by a particular user is completely unrelated to the current task being performed by the electronic device, then the gesture input is likely not to be made by a particular user, ie, the probability of a false positive is greater than Big.

(3) determining whether another person appears in the first image. In general, if there are other people in the first image, the probability that the gesture input is not made by a particular user increases, that is, the probability of false positives increases. However, since other people in the first image do not necessarily have a misjudgment, the judgment can only serve as a supplementary reference for other conditions, and cannot be used alone as a basis for misjudgment.

(4) After determining that the gesture input is not made by a specific user, it is determined whether a repeated input of the gesture is received. In general, if a repeated input of the gesture is received after determining that the gesture input is not made by a particular user, then the gesture input is likely to be made by a particular user, ie, the probability of a false positive is greater.

Based on at least one of the above determination results, it is determined whether the determination as to whether the gesture input is made by a specific user is correct to update the corresponding three-dimensional model with the arm feature in the first image.

In particular, if it is determined whether the determination as to whether the gesture input is made by a particular user is correct, the arm features in the first image are added to the corresponding three-dimensional model.

On the other hand, if a determination is made as to whether the gesture input is made by a particular user, the corresponding three-dimensional model is modified based on the arm features in the first image. The corresponding three-dimensional model is modified to exclude the arm feature in the three-dimensional model, that is, if the arm feature is detected again, a gesture that is not made by a specific user is determined.

By using the data determined to be the arm of a particular user during use to complete the process of modeling, the more the three-dimensional model is used, the higher its accuracy.

In the above, the specific process of the gesture recognition control method according to the present invention has been described in detail with reference to FIG. Next, an electronic device to which the gesture recognition control method according to the present invention is applied will be described with reference to FIG.

As shown in FIG. 2, the electronic device 200 includes an image acquisition module 201, a memory 202, and a processor 203.

The image acquisition module 201 is configured to acquire a first image when the gesture input is performed, wherein the first image includes an image of the hand and at least a part of the arm. Generally, the image acquisition module 201 can be implemented by a depth camera.

The memory 202 is used to store programs.

The processor 203 is configured to implement the following functions when executing the program stored on the memory 202:

Determining whether the gesture input is made by a specific user based on the first image;

If the result of the determination is yes, recognition is performed on the gesture input, otherwise the gesture input is ignored.

Based on the first image, determining whether the gesture input is made by a specific user may include, but is not limited to, the following manners: (1) determining whether the gesture is satisfied by a specific user based on the position of the arm in the first image. Rationality (2) Based on the three-dimensional model of the arm, it is judged whether the similarity with the gesture input by the specific user is satisfied.

Next, the two methods will be explained separately.

For example, in the case where the electronic device is a mobile phone, a tablet, a smart TV, etc., the processor 203 can execute the program to further implement a function of confirming based on a relative relationship between the arm and the verified face in the first image. Whether it is a gesture made by a specific user. If the distance between the arm and the face exceeds a predetermined threshold, or if the angle between the arm and the face exceeds a predetermined angular range, then the relative relationship between the arm and the face is unreasonable, and it can be determined that the gesture input is not by a specific user. Made. On the other hand, if it is not judged that the relative relationship between the arm and the face is unreasonable, then the gesture input is temporarily considered to be made by a specific user.

For example, in the case where the electronic device is a head mounted display device, it is confirmed based on the position of the arm in the first image whether it is a gesture made by a specific user (in this case, a wearer). Specifically, since the image of the wearer's arm is photographed by the camera worn on the head mounted display device of the head, in the obtained first image, the left arm image may not be located in the right side region, and the right arm The image cannot be in the left area. Accordingly, the processor 203 can execute the program to further implement a function of first determining whether the arm in which the gesture input is made in the first image is the left arm or the right arm. If it is determined to be the left arm, it is determined whether the left arm appears in the right area, and if so, the position of the arm is unreasonable, and it can be determined that the gesture input is not made by a specific user. On the other hand, if it is not judged that the positional relationship of the arm is unreasonable, it is temporarily considered that the gesture input is made by a specific user. Similarly, if it is determined to be the right arm, it is determined whether the right arm is present in the left area, and if so, the position of the arm is unreasonable, and it can be determined that the gesture input is not made by a specific user. On the other hand, if it is not judged that the positional relationship of the arm is unreasonable, it is temporarily considered that the gesture input is made by a specific user.

In this manner, it is judged whether the similarity with the gesture input by the specific user is satisfied, and the memory is further configured to pre-store a three-dimensional model of the arm of the specific user for performing the gesture operation. Here, two three-dimensional models of the arms of the specific user may be stored in advance so that the gesture of any arm of the user can be recognized. Alternatively, it is also possible to store only the three-dimensional model of one arm of the specific user, so that only the gesture of the specific arm of the user can be recognized.

Wherein the processor is configured to execute the program to further implement a function of: decomposing an arm of each of the plurality of arm images of the specific user acquired in the plurality of gesture operations by the image acquisition unit Different parts, and extracting features of different parts; and merging multiple features of the same part for the same arm, and obtaining a three-dimensional model for the arm.

When a new arm position appears, the model can be used to determine if it is a particular user's own gesture. Specifically, the processor is configured to execute the program to further implement the following functions:

Determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image to extract features of different parts;

The acquired features of different parts of the arm are compared with features in the corresponding three-dimensional model, and it is determined whether the gesture input is made by the specific user.

Wherein, comparing the acquired features of the different parts of the arm with the features in the corresponding three-dimensional model, and determining whether the gesture input is processed by the specific user further comprises:

Determining whether an angle between the forearm and the boom in the first image is within a predetermined range determined based on the three-dimensional model, obtaining a second determination result;

Based on the first determination result and/or the second determination result, it is determined whether the gesture input is made by the specific user. Specifically, if there is a judgment result of the first judgment result and/or the second judgment result, it is determined that the gesture input is not made by the specific user, otherwise it is determined that the gesture input is performed by the specific user. Out.

As can be seen from the above description, in the electronic device according to the present invention, the principle of "doubt is never" is taken. For example, when other users who have an arm size similar to the arm size of a particular user make a gesture input, a misjudgment may occur. In order to avoid this occurrence and further improve the accuracy of the determination, in the electronic device according to the present invention, the processor is configured to execute the program to further implement the following functions:

(1) determining whether the user's revocation operation is received after determining that the gesture input is made by a specific user;

(2) after determining that the gesture input is made by a specific user, determining whether the recognized gesture input is completely unrelated to the current task being executed by the electronic device;

(3) determining whether another person appears in the first image;

(4) determining whether a repeated input of the gesture is received after determining that the gesture input is not made by a specific user;

Based on at least one of the above determination results, it is determined whether the determination as to whether the gesture input is made by a specific user is correct.

Processor 203 may include a general purpose microprocessor, an instruction set processor, and/or a related chipset, in accordance with an embodiment of the present disclosure. And/or a dedicated microprocessor (eg, an application specific integrated circuit (ASIC)), and the like. Processor 203 may also include an onboard memory for caching purposes. The processor 203 may be a single processing unit or a plurality of processing units for performing different actions of the method flow according to the embodiment of the present disclosure described with reference to FIG.

Memory 202, for example, can be any medium that can contain, store, communicate, propagate, or transport the instructions. For example, memory 202 can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the memory 202 include: a magnetic storage device such as a magnetic tape or a hard disk (HDD); an optical storage device such as a compact disk (CD-ROM); a memory such as a random access memory (RAM) or a flash memory; and/or a wired/wireless Communication link. Since the configuration of the electronic device according to the present invention completely corresponds to the steps in the previously described gesture recognition control method, in order to avoid redundancy, in the description for the electronic device, many details are not developed. However, those skilled in the art will appreciate that the content described for the gesture recognition control method can be applied analogously to electronic devices.

Heretofore, the gesture recognition control method and electronic apparatus according to the present invention have been described in detail with reference to FIGS. 1 and 2. In the gesture recognition control method and electronic device according to the present invention, the accuracy of a specific user's own gesture recognition is improved by image processing of the entire arm instead of a single palm. Moreover, the modeling process is completed and perfected by using the data determined as the arm of the specific user during use, so that the more the three-dimensional model of the arm is used, the higher the accuracy. In addition, in the gesture recognition control method and the electronic device according to the present invention, no unnecessary redundancy is added or the user's use burden is increased (for example, the user is actively selected whether the gesture is made by himself), thereby reducing costs and improving The user experience.

It should be noted that, in this specification, the terms "including", "comprising", or any other variations thereof are intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a And also includes other elements not explicitly listed, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising", without limiting the invention, does not exclude the presence of additional elements in the process, method, article, or device.

Finally, it should also be noted that the series of processes described above include not only processes that are performed in time series in the order described herein, but also processes that are performed in parallel or separately, rather than in chronological order.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary hardware platform, and of course, all can be implemented by software. Based on such understanding, all or part of the technical solution of the present invention contributing to the background art may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM, a magnetic disk, an optical disk, or the like. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or in some portions of the embodiments.

The present invention has been described in detail above, and the principles and embodiments of the present invention have been described with reference to specific examples. The description of the above embodiments is only for helping to understand the method of the present invention and its core ideas; The description of the present invention is not intended to limit the scope of the present invention.

Claims

A gesture recognition control method is applied to an electronic device, and the method includes:

Obtaining a first image when the gesture input is performed, the first image including an image of the hand and at least a portion of the arm;

Determining whether the gesture input is made by a specific user based on the first image;

If the result of the determination is yes, recognition is performed on the gesture input, otherwise the gesture input is ignored.
The method of claim 1 wherein:

The method further includes pre-storing a three-dimensional model of a specific user's arm for performing a gesture operation;

The determining, based on the first image, whether the gesture input is made by a specific user comprises:

Determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image to extract features of different parts;

The acquired features of different parts of the arm are compared with features in the corresponding three-dimensional model, and it is determined whether the gesture input is made by the specific user.
The method according to claim 2, wherein said three-dimensional model of an arm for performing a gesture operation of a specific user in advance comprises:

Obtaining a plurality of arm images of the specific user in a plurality of gesture operations;

Decomposing different parts of the arm for each of the plurality of arm images and extracting features of the different parts;

A plurality of features of the same portion are fused for the same arm, and a three-dimensional model for the arm is obtained.
The method of claim 2 wherein:

The comparing the acquired features of different parts of the arm with the features in the corresponding three-dimensional model includes:

Determining whether a difference between a size of a different part of the arm in the first image and a corresponding size in the three-dimensional model is less than a predetermined threshold, obtaining a first determination result; and/or

Determining whether an angle between the forearm and the boom in the first image is within a predetermined range determined based on the three-dimensional model, obtaining a second determination result,

Determining whether the gesture input is made by the specific user comprises:

Based on the first determination result and/or the second determination result, it is determined whether the gesture input is made by the specific user.
The method of claim 1, wherein the determining, after the gesture input is made by a specific user based on the first image, further comprises:

After determining that the gesture input is made by a specific user, determining whether the user's undo operation is received;

After determining that the gesture input is made by a specific user, determining whether the recognized gesture input is related to the electronic device The current task being executed is completely irrelevant;

Determining whether another person appears in the first image;

After determining that the gesture input is not made by a specific user, determining whether a repeated input of the gesture is received;

Based on at least one of the above determination results, it is determined whether the determination as to whether the gesture input is made by a specific user is correct to update the corresponding three-dimensional model with the arm feature in the first image.
An electronic device comprising:

An image acquisition module, configured to acquire a first image when the gesture input is performed, where the first image includes an image of the hand and at least a portion of the arm;

a memory for storing programs;

a processor for performing the following functions when executing a program stored on the memory:

Determining whether the gesture input is made by a specific user based on the first image;

If the result of the determination is yes, recognition is performed on the gesture input, otherwise the gesture input is ignored.
The device of claim 6 wherein:

The memory is further configured to pre-store a three-dimensional model of an arm of a specific user for performing a gesture operation;

The processor is configured to execute the program to further implement the following functions:

Determining whether the arm included in the first image is a left arm or a right arm, and decomposing different parts of the arm in the first image to extract features of different parts;

The acquired features of different parts of the arm are compared with features in the corresponding three-dimensional model, and it is determined whether the gesture input is made by the specific user.
The apparatus of claim 7, wherein the processor is configured to execute the program to further implement the following functions:

Separating different parts of the arm and extracting features of different parts for each of the plurality of arm images of the specific user acquired by the image acquisition unit in the plurality of gesture operations;

A plurality of features of the same portion are fused for the same arm, and a three-dimensional model for the arm is obtained.
The apparatus of claim 7, wherein the processor is configured to execute the program to further implement the following functions:

Determining whether a difference between a size of a different part of the arm in the first image and a corresponding size in the three-dimensional model is less than a predetermined threshold, obtaining a first determination result; and/or

Determining whether an angle between the forearm and the boom in the first image is within a predetermined range determined based on the three-dimensional model, obtaining a second determination result;

Based on the first determination result and/or the second determination result, it is determined whether the gesture input is made by the specific user.
The apparatus of claim 6, the processor being configured to execute the program to further implement the following functions:

After determining that the gesture input is made by a specific user, determining whether the user's undo operation is received;

After determining that the gesture input is made by a specific user, determining whether the recognized gesture input is completely unrelated to the current task being executed by the electronic device;

Determining whether another person appears in the first image;

After determining that the gesture input is not made by a specific user, determining whether a repeated input of the gesture is received;

Based on at least one of the above determination results, it is determined whether the determination as to whether the gesture input is made by a specific user is correct to update the corresponding three-dimensional model with the arm feature in the first image.