CN115525140A

CN115525140A - Gesture recognition method, gesture recognition apparatus, and storage medium

Info

Publication number: CN115525140A
Application number: CN202110709488.5A
Authority: CN
Inventors: 井维强; 刘楠
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2021-06-25
Filing date: 2021-06-25
Publication date: 2022-12-27

Abstract

The present disclosure relates to a gesture recognition method, a gesture recognition apparatus, and a storage medium. The method comprises the following steps: acquiring a gesture image and acquisition scene information of the gesture image; recognizing the gesture image according to the acquisition scene information of the gesture image; wherein the collecting scene information at least comprises one of the following: acquiring the current posture of the terminal equipment of the gesture image; and acquiring the environment of the terminal equipment when the gesture image is acquired.

Description

Gesture recognition method, gesture recognition apparatus, and storage medium

Technical Field

The present disclosure relates to the field of terminal technologies, and in particular, to a gesture recognition method, a gesture recognition apparatus, and a storage medium.

Background

With the rapid development of terminal technology, the screen size of the terminal device is larger and larger, and the interactive operation realized through the touch screen causes inconvenience for the user in some scenes (for example, a scene in which the terminal device is held by one hand to perform self-shooting).

Therefore, in the related art, the problem that the operation of a user is inconvenient in partial scenes of the large-screen terminal device can be solved in a mode of space gesture operation. However, for contactless air gesture operation, how to accurately identify the air gesture operation is a technical problem to be solved at present.

Disclosure of Invention

The present disclosure provides a gesture recognition method, a gesture recognition apparatus, and a storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a gesture recognition method, including:

acquiring a gesture image and acquisition scene information of the gesture image;

recognizing the gesture image according to the acquisition scene information of the gesture image; wherein the collecting scene information at least comprises one of the following:

acquiring the current posture of the terminal equipment when the gesture image is acquired;

and acquiring the environment of the terminal equipment when the gesture image is acquired.

Optionally, the recognizing the gesture image according to the acquired scene information of the gesture image includes:

if the acquired scene information comprises the current posture of the terminal equipment when the gesture image is acquired, determining whether the current posture is a preset posture;

when the current posture is not the preset posture, adjusting the gesture image based on a posture difference between the current posture and the preset posture of the terminal equipment;

and performing gesture recognition on the adjusted gesture image.

Optionally, the adjusting the gesture image based on a gesture difference between the current gesture and a preset gesture of the terminal device includes:

determining the rotation angle of the terminal equipment switched from the current posture to the preset posture;

and based on the angle, carrying out rotation processing on the acquired gesture image to obtain an adjusted gesture image.

Optionally, the recognizing the gesture image according to the collection scene information of the gesture image includes:

if the acquired scene information comprises the environment where the terminal equipment is located when the gesture image is acquired, acquiring environment information of the environment where the terminal equipment is located when the gesture image is acquired; wherein the environment information includes at least one of: ambient brightness, ambient background color;

and determining a target model of gesture recognition according to the environment information, and recognizing the gesture image based on the target model.

Optionally, the determining a target model of gesture recognition according to the environment information, and recognizing the gesture image based on the target model includes:

if the environment brightness in the environment information is larger than a first brightness threshold, determining that the target model is a first model, extracting a first brightness area with the brightness value of a pixel smaller than the first brightness threshold from the gesture image based on the first model, and determining a target gesture type corresponding to the gesture image based on the first brightness area;

and/or if the ambient brightness in the ambient information is smaller than the first brightness threshold and larger than a second brightness threshold, determining that the target model is a second model, extracting second image features from the whole gesture image based on the second model, and determining a target gesture type corresponding to the gesture image according to the second image features;

and/or if the environment brightness in the environment information is smaller than the second brightness threshold, determining that the target model is a third model, extracting a second brightness area with the brightness value of the pixel larger than the second brightness threshold from the gesture image based on the third model, and determining the target gesture type corresponding to the gesture image based on the second brightness area.

Optionally, the method further comprises:

and if the environment brightness in the environment information of the gesture image is within the illumination abnormal brightness range, outputting alarm prompt information.

According to a second aspect of the embodiments of the present disclosure, there is provided a gesture recognition apparatus, the apparatus including:

the acquisition module is used for acquiring a gesture image and acquisition scene information of the gesture image;

the recognition module is used for recognizing the gesture image according to the acquisition scene information of the gesture image; wherein the collecting scene information at least comprises one of the following: acquiring the current posture of the terminal equipment when the gesture image is acquired; and acquiring the environment of the terminal equipment when the gesture image is acquired.

Optionally, the identification module is configured to:

and performing gesture recognition on the adjusted gesture image.

Optionally, the identification module is further configured to:

Optionally, the identification module is configured to:

Optionally, the identification module is further configured to:

and/or if the environmental brightness in the environmental information is smaller than the first brightness threshold and larger than a second brightness threshold, determining that the target model is a second model, extracting a second image feature from the whole gesture image based on the second model, and determining a target gesture type corresponding to the gesture image according to the second image feature;

Optionally, the apparatus further comprises:

and the output module is used for outputting alarm prompt information if the environment brightness in the environment information of the gesture image is within the illumination abnormal brightness range.

According to a third aspect of the embodiments of the present disclosure, there is provided a gesture recognition apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the executable instructions, when executed, implement the steps in the method according to the first aspect of the embodiments of the present disclosure.

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer readable storage medium having instructions which, when executed by a processor of a gesture recognition apparatus, enable the gesture recognition apparatus to perform the steps of the method according to the first aspect of embodiments of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

according to the gesture recognition method provided by the embodiment of the disclosure, by determining the current gesture of the terminal device when the gesture image is acquired and/or the environment of the terminal device, when the gesture image is subjected to gesture recognition, the gesture recognition process of the gesture image is optimized in consideration of the influence of the current gesture of the terminal device and/or the environment of the terminal device on the gesture recognition process, so that the problems of false recognition and recognition failure caused by the current gesture of the terminal device and/or the environment of the terminal device are reduced, the recognition rate and accuracy of gesture recognition are improved, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a flowchart illustrating a gesture recognition method according to an exemplary embodiment.

Fig. 2 is a schematic diagram of a terminal device shown according to an example embodiment.

Fig. 3 is a flowchart illustrating details of step S102 according to an exemplary embodiment.

Fig. 4 is a detailed flowchart two of step S102 shown according to an exemplary embodiment.

FIG. 5 is a flow diagram illustrating a gesture recognition method according to an exemplary embodiment.

Fig. 6 is a schematic diagram illustrating a pose of a terminal device according to an exemplary embodiment.

Fig. 7 is a schematic diagram illustrating a structure of a gesture recognition apparatus according to an exemplary embodiment.

FIG. 8 is a block diagram illustrating a gesture recognition apparatus according to an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

The embodiment of the disclosure provides a gesture recognition method. Fig. 1 is a flowchart illustrating a gesture recognition method according to an exemplary embodiment, where as shown in fig. 1, the method includes the following steps:

step S101, acquiring a gesture image and acquisition scene information of the gesture image;

step S102, recognizing the gesture image according to the acquisition scene information of the gesture image; wherein the collecting scene information at least comprises one of the following: acquiring the current posture of the terminal equipment when the gesture image is acquired; and acquiring the environment of the terminal equipment when the gesture image is acquired.

In the embodiment of the disclosure, the gesture recognition method can be applied to a mobile terminal and can also be applied to a server. When the gesture recognition method is applied to the server, the mobile terminal can send the acquired gesture image and the acquired scene information of the gesture image to the server, the server processes the gesture image through the steps from S101 to S102, and the gesture recognition result is sent to the mobile terminal.

Taking the application of the gesture recognition method to the mobile terminal as an example, the mobile terminal may be: smart phones, tablet computers, wearable electronic devices, or the like; the mobile terminal comprises an image acquisition module, wherein the image acquisition module generally refers to a device capable of finishing a photographing function in the mobile terminal, and comprises a camera, a necessary processing module and a necessary storage module so as to finish acquisition and transmission of fingerprint images, and the mobile terminal can also comprise some processing function modules. The image acquisition module can be a front camera or a rear camera in a mobile phone.

In step S101, when the starting operation of the image capturing module is detected, the terminal device may capture the gesture image through the image capturing module, and monitor a change of a parameter of a sensor associated with the terminal device, so as to obtain capture scene information of the gesture image.

In some embodiments, if the acquiring the scene information includes: acquiring the current posture of the terminal equipment when the gesture image is acquired; the method comprises the steps that inclination angle data collected by a horizontal sensor in the terminal equipment can be obtained; and detecting the current posture of the terminal equipment when acquiring the gesture image based on the inclination angle data acquired by the horizontal sensor.

In the embodiment of the present disclosure, the terminal device is configured with a level sensor, and the level sensor is configured to collect inclination angle data between the terminal device and a horizontal plane, where the horizontal plane refers to a horizontal plane of a shooting environment. Specifically, the horizontal sensor can generate different piezoelectric signals for different inclination angles under the action of gravity, and then the acquisition of inclination angle data is realized.

The current posture of the terminal equipment can be represented by an included angle between the terminal equipment and the horizontal plane. In order to determine the angle between the terminal device and the horizontal plane, a reference line of the terminal device needs to be defined.

For example, as shown in fig. 2, fig. 2 is a schematic diagram of a terminal device according to an exemplary embodiment, and reference numeral 21 indicates a horizontal axis direction of the terminal device; reference numeral 22 indicates the longitudinal direction of the terminal device. And determining the horizontal axis direction of the terminal device as the reference line of the terminal device, or determining the vertical axis direction of the terminal device as the reference line of the terminal device. After the reference line of the terminal equipment is determined, the included angle between the terminal equipment and the horizontal plane is calculated according to the coordinate relation between the reference line and the inclination angle data.

It should be noted that the reference line may also be selected in other manners, for example, the direction of the symmetry axis of the terminal display screen, and the present disclosure does not specifically limit the manner of determining the terminal reference line.

In some embodiments of the present disclosure, the terminal device may be configured with a gravitational acceleration sensor, where the gravitational acceleration sensor is configured to obtain a motion acceleration of the terminal device, and determine a current posture of the terminal device according to the motion acceleration of the terminal device.

The motion acceleration value of the terminal device acquired by the gravitational acceleration sensor is a value obtained by subtracting a superposition of gravitational accelerations from an acceleration of the terminal device using the earth as a reference object. When the terminal device is in a weightless state when the terminal device makes free-fall movement to the ground with the gravity acceleration, the gravity acceleration sensor takes the state as 0 of the acceleration. When the terminal equipment is in a static state or a uniform motion state relative to the ground, in order to resist the trend of free fall motion, an upward acceleration vertical to the ground exists, namely the motion acceleration of the terminal equipment; determining the direction of the terminal equipment relative to the ground according to the motion acceleration of the terminal equipment; and then determining the current posture of the terminal equipment.

For example, the motion acceleration of the terminal device may be obtained by a gravitational acceleration sensor of the terminal device, and the orientation of the terminal device with respect to the ground is determined based on the motion acceleration such that the upper left corner point of the terminal device is higher than the upper right corner point of the terminal device, thereby determining the attitude of the terminal device.

In some other embodiments, if the collecting scene information includes: the environment of the terminal equipment is acquired when the gesture image is acquired, the environment information of the acquired gesture image can be acquired through a sensor, and the environment of the terminal equipment is determined when the gesture image is acquired.

Here, the environment information may include at least one of:

temperature, humidity, geographical location, weather conditions, lighting information.

The terminal equipment can acquire gesture images through the image acquisition module in different photographing environments, wherein the photographing environment can comprise an indoor photographing environment or an outdoor photographing environment; but also dark photographing environment (night photographing environment) or bright photographing environment (day photographing environment); and may also include a sunny photographing environment or a cloudy photographing environment, and the embodiments of the present disclosure are not limited.

It can be understood that the gesture images acquired in different photographing environments may have a large difference in image quality, and therefore, when performing gesture recognition on the gesture images, the influence of the environment where the terminal device is located on the recognition of the gesture images needs to be considered.

In the step S102, when the gesture image is recognized, the gesture image may be adjusted according to the collected scene information.

The brightness of the gesture image can be adjusted according to the illumination information in the collected scene information, and gesture recognition can be performed according to the gesture image after the brightness adjustment.

In some embodiments, after the operation instruction corresponding to the gesture image is recognized, the parameter of the operation instruction is adjusted according to the collected scene information.

Exemplarily, if the operation instruction corresponding to the gesture image is an instruction for navigating to a preset target address, the preset target address includes a company address and a house address preset by a user; and after the gesture image is recognized, determining the navigation target position according to the geographic position in the acquisition scene information. If the geographic position indicates that the current position of the user is a company address, determining that the target position of the navigation is a house address preset by the user; and if the geographic position indicates that the current position of the user is a residential address, determining that the target position of the navigation is a company address preset by the user.

And illustratively, according to the fact that the operation instruction corresponding to the gesture image is a photographing instruction, after the gesture image is recognized, adjusting a photographing mode of the terminal device according to the illumination information in the collected scene information. If the illumination information indicates that the environment where the terminal equipment is located is a dark light environment, a shooting model of the terminal equipment can be switched to a night shooting mode; if the illumination information indicates that the environment where the terminal equipment is located is a bright environment, the photographing model of the terminal equipment can be switched to a daytime photographing mode.

Optionally, fig. 3 is a first flowchart illustrating details of the method of step S102 according to an exemplary embodiment, and as shown in fig. 3, recognizing the gesture image according to the capture scene information of the gesture image in step S102 may include:

step S1021, if the collected scene information comprises the current posture of the terminal equipment when the gesture image is collected, determining whether the current posture is a preset posture;

step S1022, when the current posture is not the preset posture, adjusting the gesture image based on a posture difference between the current posture and the preset posture of the terminal device;

and step S1023, performing gesture recognition on the adjusted gesture image.

In step 1021, the preset gesture may be a reference gesture of the terminal device, where the preset gesture may be a default gesture, or may be a gesture when the gesture image template is entered into the terminal device.

In some embodiments of the present disclosure, the preset posture refers to that an included angle between the terminal device and a horizontal plane is within a preset angle range.

It can be understood that in the process of acquiring the gesture image, the image is not required to be a standard horizontal image, and in consideration of the convenience of shooting by the user, a certain error allowance mechanism should be provided when the acquired gesture image level is ensured. Therefore, the embodiment of the disclosure can determine an angle range in advance, and if the included angle between the terminal device and the horizontal plane is within the preset angle range, determine that the current state of the terminal device is the preset posture.

In the embodiment of the present disclosure, the determination of the preset angle range is related to the determination of the reference line of the terminal device; for example, if the horizontal axis direction of the terminal device is determined as the reference line of the terminal device, it may be determined that the preset angle range is greater than or equal to minus 10 degrees and less than or equal to plus 10 degrees; if the longitudinal axis mode of the terminal device is determined as the reference line of the terminal device, it may be determined that the preset angle range is greater than or equal to 80 degrees and less than or equal to 100 degrees.

It should be noted that the preset angle range may be set according to an actual application situation, and the setting of the preset angle range is not limited in the embodiment of the present disclosure.

In step S1022, comparing the current posture of the terminal device with a preset posture, and if the current posture of the terminal device is not the preset posture, obtaining a posture difference between the current posture and the preset posture; and adjusting the acquired gesture image according to the gesture difference so that the adjusted gesture image can restore the real gesture of the user.

It can be understood that the physical position of the camera in the image acquisition module of the terminal device relative to the terminal device is fixed; the position change between the position of the camera when the terminal equipment is in the current posture and the position of the camera when the terminal equipment is in the preset posture can be determined according to the posture difference between the current posture and the preset posture when the terminal equipment acquires the gesture image; and then according to the position change, correspondingly adjusting the gesture image acquired by the camera, so that the adjusted gesture image is not influenced by the posture change of the terminal equipment.

In some embodiments, since the current posture of the terminal device can be represented by an included angle between the terminal device and a horizontal plane, the current posture of the terminal device can be represented by acquiring a first included angle corresponding to the current posture of the terminal device and a second included angle corresponding to the preset posture; and adjusting the gesture image according to the angle difference between the first included angle and the second included angle to obtain an adjusted gesture image.

In other embodiments of the present disclosure, if the camera in the image capturing module of the terminal device is a movable camera relative to the terminal device; the method further comprises the following steps:

acquiring the current posture and the initial posture of an image acquisition module of the terminal equipment;

determining whether the current posture of the image acquisition model is the initial posture of the image acquisition module;

if the current posture of the image acquisition model is the initial posture, adjusting the gesture image based on a first posture difference between the current posture of the terminal equipment and the preset posture;

if the current posture of the image acquisition model is not the initial posture, the gesture image is adjusted based on a first posture difference between the current posture of the terminal equipment and the preset posture and a second posture difference between the current posture of the image acquisition module and the initial posture.

In an embodiment of the present disclosure, the initial pose may be a reference pose of the image capturing module; here, the initial posture may be a default posture of the image capturing module, or may be a posture when the gesture image template is entered into the image capturing module.

The first posture difference can be used for indicating a first angle difference and an adjustment direction corresponding to the current posture and the preset posture of the terminal equipment; the second posture difference can be used for indicating a second angle difference and an adjustment direction corresponding to the current posture and the preset posture of the image acquisition module.

When terminal equipment acquires a gesture image by using an image acquisition module, acquiring the current posture of the image acquisition module and the current posture of the terminal equipment; if the current posture of the image acquisition module is the initial posture of the image acquisition module, the camera in the image acquisition module is not moved, and the gesture image can be directly adjusted according to the first posture difference between the current posture of the terminal equipment and the preset posture.

For example, if the current posture of the camera of the terminal device is detected to be the same as the initial posture, comparing the current posture of the terminal device with a preset posture, and determining that the first angle difference between the current posture and the preset posture of the terminal device is 90 degrees, wherein the adjusting direction is clockwise adjustment; the collected gesture images can be rotated 90 degrees clockwise to obtain the adjusted gesture images.

If the current posture of the image acquisition module is not the initial posture of the image acquisition module, the camera in the image acquisition module is moved, a first posture difference between the current posture of the terminal equipment and the preset posture and a second posture difference between the current posture of the image acquisition module and the initial posture are determined, and the gesture image is adjusted according to a superposition result of the first posture difference and the second posture difference.

For example, if the current posture of a camera of the terminal device is detected to be different from the initial posture, comparing the current posture of the camera with the initial posture, determining that a second angle difference between the current posture and the initial posture of the camera is 45 degrees, and adjusting the direction to be anticlockwise adjustment; according to the current posture and the preset posture of the terminal equipment, a first angle difference between the current posture and the preset posture of the terminal equipment is determined to be 90 degrees, and the adjusting direction is clockwise adjustment. And determining an adjustment strategy of the acquired gesture image based on the first angle difference and the adjustment direction corresponding to the first angle difference and the adjustment method corresponding to the second angle difference, namely, clockwise rotating the acquired gesture image by 45 degrees.

In step S1023, the gesture recognition result may be determined by comparing the adjusted gesture image with the pre-stored gesture image templates one by one.

In some embodiments of the present disclosure, the similarity between the adjusted gesture image and the pre-stored gesture image templates may be determined, and if the similarity between the adjusted gesture image and any one of the gesture image templates exceeds a preset similarity threshold, it is determined that the adjusted gesture image and the gesture image template are successfully matched, and the matched gesture image template is determined as the target gesture.

In other embodiments of the present disclosure, a gesture image template with the highest similarity may be determined as the target gesture by determining similarities between the adjusted gesture image and a plurality of gesture image templates stored in advance.

And when the adjusted gesture image is successfully matched with any pre-stored gesture image template, determining the matched gesture image template as a target gesture, and acquiring an operation instruction corresponding to the target gesture so that the terminal equipment can execute the operation instruction corresponding to the target gesture.

It should be noted that, the terminal device stores in advance a corresponding relationship between the gesture image templates and the operation instructions, where the corresponding relationship describes the operation instructions corresponding to multiple gesture image templates, and the operation instructions corresponding to different gesture image templates are different; and after the adjusted gesture image is successfully matched with any gesture image template, inquiring the corresponding relation, acquiring an operation instruction corresponding to the gesture image template, and executing the operation instruction.

It can be understood that the corresponding relation between the gesture image template and the operation instruction can be set by user according to the user requirement; any application within the terminal device may be associated with at least one gesture image template. When a user sets a gesture image template associated with an application program, a plurality of gesture image templates can be set in the same application program to be associated and responded with the same according to needs; therefore, the user can operate the application program conveniently, and the user experience is improved.

Illustratively, in an audio playing scene, the correspondence between the gesture image template and the operation instruction pre-stored in the terminal device records that the operation instruction corresponding to the left slide gesture image template is to switch the previous audio, and the operation instruction corresponding to the right slide gesture image template is to switch the next audio. When the adjusted gesture image is successfully matched with the right slide gesture image template, determining that the adjusted gesture image is a right slide gesture; and determining that the operation instruction corresponding to the right slide gesture is the next audio switching operation by inquiring the corresponding relation, wherein at the moment, the terminal equipment can execute the operation instruction, namely, the next audio of the current playlist terminal is played.

And when the adjusted gesture image is not matched with a pre-stored gesture image template, determining that gesture recognition fails, wherein the adjusted gesture image is an invalid gesture image, and outputting prompt information of the gesture recognition failure by the terminal equipment without responding to the invalid gesture image.

Optionally, in step S1022, adjusting the gesture image based on a gesture difference between the current gesture and a preset gesture of the terminal device includes:

determining the angle of rotation of the terminal equipment from the current posture to the preset posture;

In the embodiment of the present disclosure, when it is determined that the terminal device is switched from the current posture to the preset posture by the angle, a first rotation angle or a second rotation angle may be determined.

Here, the first rotation angle is an angle required when the terminal device rotates clockwise from the current posture to the preset posture; the second rotation angle is an angle required when the terminal device rotates anticlockwise from the current posture to the preset posture.

For example, if the terminal device rotates clockwise from the current posture to the preset posture, the required first rotation angle is 90 degrees; the second rotation angle required when the terminal rotates counterclockwise from the current posture to the preset posture is 270 degrees.

And after the first rotation angle or the second rotation angle is determined, rotating the acquired gesture image at a corresponding angle and in a corresponding direction to obtain an adjusted gesture image.

Optionally, fig. 4 is a flowchart illustrating a second method detail of step S102 according to an exemplary embodiment, where the recognizing the gesture image according to the captured scene information of the gesture image in step S102 includes:

step S1024, if the acquired scene information comprises the environment where the terminal device is located when the gesture image is acquired, acquiring environment information of the environment where the terminal device is located when the gesture image is acquired; wherein the environment information includes at least one of: ambient brightness, ambient background color;

and S1025, determining a target model of gesture recognition according to the environment information, and recognizing the gesture image based on the target model.

In the embodiment of the present disclosure, the ambient brightness refers to a luminous flux (illumination intensity in a unit area) received by the image capturing module in a unit area, and may be used to reflect a brightness of an illuminated degree of the terminal device.

Under terminal equipment is in the environment of shooing of difference, the luminous flux that receives on the image acquisition module unit area is different among the terminal equipment, and then the ambient brightness of environment is different when terminal equipment gathers gesture image. For example, the ambient brightness determined when the terminal device collects the gesture image outdoors is greater than the ambient brightness determined when the terminal device collects the gesture image indoors.

In addition, in the process of determining the ambient brightness when the terminal device collects the gesture image, the positions of the light sources in the environment where the terminal device is located are different, and the ambient brightness when the corresponding terminal device collects the gesture image is also different. For example, the ambient brightness of the image capturing module of the terminal device at the position away from the light source is less than the ambient brightness of the image capturing module of the terminal device at the position facing the light source.

It can be understood that, because the environmental luminances corresponding to different photographing environments may also be different, and the image quality difference of the gesture images acquired under different environmental luminances is large, if the gesture images acquired under different environmental luminances are subjected to gesture recognition by using the same recognition model, the accuracy of gesture recognition may be reduced.

Based on this, the embodiment of the disclosure determines the ambient brightness when the gesture image is acquired, classifies the gesture image according to the ambient brightness, and determines the target model for performing gesture recognition on the gesture image according to the ambient brightness of the gesture image of the same category.

Here, the object models corresponding to the gesture images with different ambient brightness are different. According to the gesture recognition method and device, gesture recognition is carried out by adopting different target models for the gesture images with different ambient brightness, so that in the gesture recognition process, the recognition model which is more adaptive to the gesture images with different ambient brightness is adopted to carry out gesture recognition according to the characteristics of the gesture images with different ambient brightness, the accuracy of gesture recognition can be effectively improved, and the user experience is improved.

In some embodiments, the obtaining of the environment information of the environment in which the terminal device is located when the gesture image is acquired may include:

if the environment information comprises environment brightness, determining the environment brightness according to the statistical characteristics of the brightness values of the pixels contained in the gesture image;

alternatively, the first and second electrodes may be,

and sensing and acquiring the ambient brightness of the gesture image by using a light sensor.

In an embodiment of the present disclosure, the determining the ambient brightness according to a statistical feature of brightness values of pixels included in the gesture image includes:

determining a brightness mean value of the gesture image by obtaining brightness values of all pixel points in the gesture image; and taking the brightness mean value of the gesture image as the representation of the environment brightness when the gesture image is collected.

In other embodiments of the present disclosure, the gesture area may be determined by determining a corresponding gesture area in the gesture image; acquiring the brightness value of each pixel point in the gesture area; determining a brightness mean value of the gesture area; and taking the brightness mean value of the gesture area as the representation of the environment brightness when the gesture image is collected.

In this disclosed embodiment, terminal equipment's image acquisition module can include: the camera is used for collecting gesture images, and the light sensor is used for detecting the ambient brightness of the environment where the camera collects the gesture images.

The direction of the camera is the same as that of the light sensor, the direction is indicated as the acquisition direction of the camera for acquiring gesture images, and the direction is the same as the detection direction of the light sensor for detecting ambient light.

In the embodiment of the disclosure, the ambient brightness is collected by the light sensor with the same orientation as the camera, so that the ambient brightness detected by the light sensor is close to the ambient light when the gesture image is collected by the camera, and the ambient brightness of the environment where the image collection module is located when the gesture image is collected can be accurately obtained.

In other embodiments of the present disclosure, identification information of a camera of the image acquisition module may be obtained; determining the orientation of the camera according to the identification information of the camera; and reading the ambient brightness from a light sensor which is contained in the image acquisition module and has the same direction as the camera according to the direction.

In the embodiment of the disclosure, after the terminal device receives the start request for the image acquisition module, the camera corresponding to the camera identification information is started to acquire the gesture image according to the camera identification information carried in the start request for the image acquisition module.

It should be noted that, the image capturing module of the terminal device has at least two cameras, and the at least two cameras have different identification information. Based on the identification information of the different cameras, the orientation of the cameras can be determined. The orientation of the camera can comprise a screen facing the terminal equipment (namely a front camera); a back shell facing the terminal device (i.e., a rear camera) may also be included. The back shell of the terminal equipment and the screen of the terminal equipment are arranged on two opposite surfaces of the terminal equipment.

In the embodiment of the present disclosure, the cameras in different orientations are correspondingly provided with different light sensors. For example, the front camera has a first light sensor for sensing light coming from the direction of the screen, and the rear camera has a second light sensor for sensing light coming from the direction of the back shell.

In some embodiments, reading the ambient brightness from a light sensor included in the image capture module oriented in the same direction as the camera according to the orientation comprises:

when the orientation of the camera is the same as that of a first light sensor contained in the image acquisition module, reading the ambient brightness acquired by the first light sensor;

when the orientation of the camera is the same as that of a second light sensor contained in the image acquisition module, the ambient brightness acquired by the second light sensor is read.

It can be understood that, under the same light source, the first light sensor and the second light sensor corresponding to the cameras in different directions have different detected ambient brightness. The embodiment of the disclosure can rotationally select different first light sensors or second light sensors to collect the ambient brightness according to different orientations of the camera, and can obtain more accurate ambient brightness which can most reflect the environment where the camera collecting the gesture image is located.

The environment background color refers to color components contained in light in the environment where the image acquisition module is located.

Under terminal equipment is in the different environment of shooing, the colour composition that the light in the environment that the image acquisition module was located among the terminal equipment contained is different, and then the environment background colour of environment is also different when terminal equipment gathered the gesture image. For example, the background color of the environment determined by the terminal device collecting the gesture image under the white light is different from the background color of the environment determined by the terminal device collecting the gesture image under the warm light.

In some embodiments, a color temperature sensor may be utilized to obtain an ambient background color of an environment in which the gesture image was captured.

In the embodiment of the disclosure, a color temperature sensor may be disposed on the terminal device, the color temperature value of the environment where the terminal device is located is detected by the color temperature sensor, and the environment background color of the environment where the terminal device is located when the terminal device collects the gesture image is determined according to the color temperature value.

It will be appreciated that the different colors of light have different color temperature values, for example, blue light has a color temperature value of about 9000 Kelvin and orange light has a color temperature value of about 2800 Kelvin.

In other embodiments, the background color of the environment in which the gesture image is captured may be determined by the color distribution of the background area in the gesture image.

After determining the environment background color when the gesture image is collected, selecting a target model corresponding to the environment background color according to the environment background color in the embodiment of the disclosure; recognizing the gesture image based on the target model.

Here, the gesture recognition models corresponding to different environment background colors are different, and it can be understood that, when the gesture recognition models perform model training, the environment background colors of training sample images required by the gesture recognition models corresponding to different environment background colors are also different; the gesture recognition model obtained by training according to the training sample images with different environment background colors can be used for gesture recognition of the gesture image more suitable for the environment background colors, the accuracy of gesture recognition can be effectively improved, and user experience is improved.

In the embodiment of the present disclosure, the first brightness threshold and the second brightness threshold may be set by default, or may be set or modified by a user according to a user requirement, and the first brightness threshold is greater than the second brightness threshold.

If the environment brightness is larger than the first brightness threshold value, the light of the environment where the gesture image is collected is considered to be brighter, and the collected gesture image is a bright image. A first model corresponding to the bright image may be determined as a target model for recognizing the gesture image.

If the environment brightness is smaller than the first brightness threshold value and larger than the second brightness threshold value, the light of the environment where the gesture image is collected is considered to be normal, and the collected gesture image is a normal illumination image. A second model corresponding to the normal illumination image may be determined as a target model for recognizing the gesture image.

And if the environment brightness is smaller than the second brightness threshold value, the light of the environment where the gesture image is collected is considered to be dark, and the gesture image is collected to be a dim light image. A third model corresponding to the dim-light image may be determined as a target model for recognizing the gesture image.

In the embodiment of the disclosure, according to the brightness value of each pixel point in the gesture image, the pixel point with the brightness value smaller than the first brightness threshold value is extracted; forming a first brightness region based on the pixel points with the brightness values smaller than the first brightness threshold; it is understood that the first brightness region is a normally illuminated region in the gesture image. And performing gesture recognition on a first brightness area in the gesture image based on a first model.

In some embodiments of the present disclosure, the method further comprises:

determining whether the number of pixel points with brightness values smaller than a first brightness threshold value in the gesture image meets a first pixel number condition required by gesture recognition;

if the number of the pixel points with the brightness values smaller than the first brightness threshold value does not meet the first pixel number condition required by gesture recognition, determining that the gesture recognition fails;

and if the number of the pixel points with the brightness values smaller than the first brightness threshold meets a first pixel number condition required by gesture recognition, extracting a first brightness area with the brightness values of the pixels smaller than the first brightness threshold from the adjusted gesture image.

Here, the first pixel number condition is a minimum pixel number required in the gesture recognition process based on the first model; the first pixel number condition may be determined according to actual requirements.

If the number of the pixel points of which the brightness values are smaller than the first brightness threshold in the gesture image meets the minimum number required by the first pixel number condition, it is indicated that the gesture image can be accurately represented based on the first brightness region formed by the pixel points of which the brightness values are smaller than the first brightness threshold; and matching the first brightness region in the gesture image with a gesture image template pre-stored in a first model, so that the gesture recognition of the gesture image can be realized.

If the number of the pixel points with the brightness values smaller than the first brightness threshold in the gesture image does not meet the minimum number required by the first pixel number condition, it is indicated that gesture recognition cannot be accurately performed on the gesture image based on the first brightness area formed by the pixel points with the brightness values smaller than the first brightness threshold, warning prompt information can be output to a user to prompt the user that gesture recognition fails, and the gesture image is shot again.

In other embodiments of the present disclosure, the determining a target gesture type corresponding to the gesture image based on the first luminance region includes:

performing feature extraction on the first brightness region to obtain a first image feature;

determining similarity between the first image feature and a plurality of first gesture feature templates pre-stored by the first model;

and determining the gesture type corresponding to the first gesture feature template with the similarity exceeding a first similarity threshold as the target gesture type corresponding to the gesture image.

In the embodiment of the disclosure, a first model is utilized to perform feature extraction on a first brightness area in the gesture image to obtain a first image feature; and extracting a plurality of first gesture feature templates from a plurality of gesture image templates pre-stored in the first model, comparing the first image features with the plurality of first gesture feature templates one by one, and if the similarity between the first image features and a certain first gesture feature template is greater than or equal to a first similarity threshold value, determining the gesture type corresponding to the first gesture feature template as the target gesture type of the gesture image.

Here, the first similarity threshold may be any fixed value, and the fixed value may be set according to a user requirement; or the number of the first image features; if the number of the first image features is N, the first similarity threshold is N/2; the embodiments of the present disclosure are not particularly limited in this regard.

In some embodiments, similarity matching may be performed between the first image feature and a plurality of first gesture feature templates pre-stored in the first model, and if the number of the same image features between the first image feature and one of the first gesture feature templates reaches a first similarity threshold, a gesture corresponding to the first gesture feature template is determined as a target gesture type corresponding to the gesture image.

It should be noted that, because the first model is used to perform gesture recognition on the bright image, and the bright image may cause partial image information of the gesture image to be lost due to overexposure. In order to improve the accuracy of the first model in recognizing the brightness image, the gesture image template pre-stored by the first model can also be set as a bright gesture image template.

In the embodiment of the disclosure, the second model can be used for performing feature extraction on the gesture image to obtain a second image feature; performing gesture recognition based on the second image characteristics; determining a target gesture type corresponding to the gesture image according to a gesture recognition result; and acquiring an operation instruction corresponding to the target gesture type so as to enable the terminal equipment to execute the operation instruction.

In some embodiments, the determining, according to the second image feature, a target gesture type corresponding to the gesture image includes:

determining similarity between the second image feature and a plurality of second gesture feature templates pre-stored by the second model;

and determining the gesture type corresponding to the second gesture feature template with the similarity exceeding a second similarity threshold as the target gesture type corresponding to the gesture image.

Here, the second similarity threshold may be any fixed value, and the fixed value may be set according to a user requirement; or the number of the second image features; if the number of the second image features is N, the second similarity threshold is N/2; the embodiments of the present disclosure are not particularly limited in this regard.

Similarity matching can be carried out between the second image characteristics and a plurality of gesture characteristic templates pre-stored in a second model, and if the number of the same image characteristics between the second image characteristics and one gesture characteristic template reaches a second similarity threshold value, a gesture corresponding to the gesture characteristic template is determined as a target gesture type corresponding to the gesture image.

It should be noted that, because the second model is used for performing gesture recognition on the normal illumination image, the feature extraction can be directly performed on the whole gesture image to obtain all second image features of the gesture image; compared with the first model which performs feature extraction on a partial region (namely a first brightness region) in the gesture image, the number of the second image features may be more than that of the first image features; on the basis, in order to improve the accuracy of gesture recognition, the gesture image template pre-stored by the second model in the embodiment of the disclosure may also be set as a normal illumination gesture image template; moreover, the second similarity threshold may also be greater than the first similarity threshold, for example, the first similarity threshold is N/2, and the second similarity threshold is 2N/3.

In the embodiment of the disclosure, according to the brightness value of each pixel point in the gesture image, the pixel point with the brightness value larger than the second brightness threshold value is extracted; forming a second brightness area based on the pixel points of which the brightness values are greater than the second brightness threshold; it is understood that the second brightness region is a normally illuminated region in the gesture image. And performing gesture recognition on a second brightness area in the gesture image based on a third model.

In other embodiments of the present disclosure, the method further comprises:

determining whether the number of pixel points with the brightness values larger than a second brightness threshold value in the gesture image meets a second pixel number condition required by gesture recognition;

if the number of the pixel points with the brightness values larger than the second brightness threshold value does not meet the second pixel number condition required by the gesture recognition, determining that the gesture recognition fails;

and if the number of the pixel points with the brightness values larger than the second brightness threshold meets a second pixel number condition required by gesture recognition, extracting a second brightness area with the brightness values of the pixels larger than the second brightness threshold from the gesture image.

In the embodiment of the present disclosure, the second pixel number condition is a minimum pixel number required in the gesture recognition process based on the third model; the second pixel number condition may be determined according to actual requirements.

If the number of the pixel points of which the brightness values are larger than the second brightness threshold value in the gesture image meets the minimum number required by the second pixel number condition, it is indicated that the gesture image can be accurately represented based on a second brightness area formed by the pixel points of which the brightness values are larger than the second brightness threshold value; and matching the second brightness region in the gesture image with a gesture image template pre-stored in a third model, so that the gesture recognition of the gesture image can be realized.

If the number of the pixel points of which the brightness values are greater than the second brightness threshold in the gesture image does not meet the minimum number required by the second pixel number condition, it is indicated that the gesture image cannot be accurately gesture-recognized based on the second brightness area formed by the pixel points of which the brightness values are greater than the second brightness threshold, an alarm prompt message can be output to a user to prompt the user that the gesture recognition fails, and the gesture image is shot again.

In some embodiments, the determining, based on the second brightness region, a target gesture type corresponding to the gesture image includes:

performing feature extraction on the second brightness area to obtain a third image feature;

determining similarity between the third image feature and a plurality of third gesture feature templates pre-stored by the third model;

and determining the gesture type corresponding to the third gesture feature template with the similarity exceeding a third similarity threshold as the target gesture type corresponding to the gesture image.

In the embodiment of the disclosure, a third model is used for performing feature extraction on a second brightness area in the gesture image to obtain a third image feature; and extracting a plurality of third gesture feature templates from a plurality of gesture image templates prestored in the third model, comparing the third image features with the plurality of third gesture feature templates one by one, and if the similarity between the third image features and a certain third gesture feature template is greater than or equal to a third similarity threshold, determining the gesture type corresponding to the third gesture feature template as the target gesture type of the gesture image.

Here, the third similarity threshold may be any fixed value, and the fixed value may be set according to a user requirement; or the number of the third image features; if the number of the third image features is N, the third similarity threshold is N/2; the embodiment of the present disclosure is not particularly limited to this.

In other embodiments, similarity matching may be performed between a third image feature and a plurality of third gesture feature templates pre-stored in a third model, and if the number of the same image features between the third image feature and a certain third gesture feature template reaches a third similarity threshold, a gesture corresponding to the third gesture feature template is determined as the target gesture type corresponding to the gesture image.

It should be noted that, because the third model is used for performing gesture recognition on a dim image, the dim image may cause loss of detail information of the gesture image due to underexposure. In order to improve the accuracy of the third model in recognizing the brightness image, a gesture image template pre-stored by the third model can also be set as a dim light gesture image template; and the third similarity threshold is smaller than the second similarity threshold, for example, the third similarity threshold is N/2, and the second similarity threshold is 2N/3.

Optionally, the method further comprises:

In the embodiment of the present disclosure, the illumination abnormal brightness range may be set by default, or may be set or modified by a user according to a user requirement.

In some embodiments of the present disclosure, if the brightness value of the gesture image is greater than the third brightness threshold, or less than the fourth brightness threshold, it is determined that the gesture image is within the illumination abnormal brightness range.

Here, the third luminance threshold value is larger than the first luminance threshold value, and the fourth luminance threshold value is smaller than the second luminance threshold value.

When the brightness value of the adjusted gesture image is within the illumination abnormal brightness range, the terminal device outputs alarm prompt information to prompt that the current working environment of the terminal device is abnormal.

In some embodiments, the gesture recognition of the gesture image further comprises:

acquiring characteristic information of a first user;

performing identity recognition on the first user based on the characteristic information;

if the first user passes the identity recognition, matching the gesture image with a plurality of gesture image templates corresponding to the first user to obtain a gesture recognition result;

and if the first user fails in the identity recognition, determining that the gesture recognition fails.

In the embodiment of the disclosure, the terminal device may store a set of gesture image templates of a plurality of different users in advance.

Before gesture recognition is carried out on the gesture image, identity recognition needs to be carried out on a first user; after the identity information of the user is determined, a gesture image template set corresponding to the first user is obtained, and the gesture image is matched with a plurality of gesture image templates in the gesture image template set corresponding to the first user, so that a gesture recognition result is obtained.

The feature information of the first user may include: biometric information and/or behavioral characteristic information.

Here, the biometric information may include at least one of: fingerprint information, palm print information, vein information (e.g., palm veins and/or finger veins); the behavior feature information may include at least one of: holding position, holding posture and holding force.

In some embodiments, the first user may be identified based on the biometric information of the first user by obtaining the biometric information of the user from the gesture image.

After the gesture image is obtained, extracting the biological feature information of the user from the gesture image, matching the biological feature information of the user with the biological feature information of a plurality of pre-stored users, and determining the identity recognition result of the first user according to the matching result.

In other embodiments of the present disclosure, the identity of the user may be identified according to behavior feature information of the user by acquiring the behavior feature information.

In the embodiment of the disclosure, a preset sensor may be arranged at a specific position of the terminal device, and behavior feature information of a user is acquired based on the preset sensor.

Here, the specific location may be set according to a user's demand; for example, if what gather is first user's the dynamics of gripping, can set up pressure sensor at two sides of terminal equipment and detect first user's the dynamics of gripping.

It should be noted that, because different users have different usage habits or different palm sizes of different users, and the holding posture, holding position, and holding strength of the user holding the terminal device are different, the behavior feature information of the first user is obtained, and the behavior feature information of the first user is matched with the behavior feature information of the plurality of users stored in advance, and the identification result of the first user is determined according to the matching result.

On one hand, the safety of the terminal equipment can be effectively improved by carrying out identity recognition on the user and then carrying out gesture recognition on the gesture image of the user after the user passes the identity recognition; on the other hand, after the user passes through the identity recognition, the gesture image of the user is recognized based on the gesture image template set corresponding to the user, so that when the same terminal device is used by different users, corresponding operation instructions can be executed according to the gesture operation set by each user, and the user experience is improved.

The present disclosure also provides the following embodiments:

fig. 5 is a flowchart illustrating a gesture recognition method according to an exemplary embodiment, where as shown in fig. 3, the method includes:

step S201, determining the current posture of the terminal equipment when acquiring a gesture image;

in this example, the gyroscope may be provided in the terminal device, and the gyroscope may be used to assist in determining the current attitude of the terminal device.

When the starting operation aiming at the image acquisition module is detected, the terminal equipment can determine the current posture when the terminal equipment acquires the gesture image through monitoring the parameter change of the gyroscope.

Step S202, determining whether the current posture is a preset posture;

in this example, the preset gesture is a gesture when the terminal device inputs a gesture image template; the current gesture when the terminal equipment collects the gesture image can be compared with the preset gesture of the gesture image template input by the terminal equipment, and whether the collected gesture image needs to be adjusted or not is determined according to the comparison result.

It should be noted that, because the physical position of the image acquisition module in the terminal device is relatively fixed with respect to the terminal device, if the user inputs the gesture image template in a preset gesture, and during the later use, if the gesture image is acquired, the current gesture of the terminal device is different from the preset gesture, which may result in the gesture being unrecognizable or the recognition failing.

Exemplarily, as shown in fig. 6, fig. 6 is a schematic pose diagram of a terminal device according to an exemplary embodiment. Wherein, the reference numeral 61 is the upright posture of the device; reference numeral 62 denotes the upside down posture of the apparatus, reference numeral 63 denotes the leftward lying posture of the apparatus, and reference numeral 64 denotes the rightward lying posture of the apparatus. If the user enters the gesture image template in the upright posture of the device as shown by the reference numeral 61, and in the later use process, the gesture image is captured in the posture as shown by the reference numeral 62, the reference numeral 63 or the reference numeral 64, and if the captured gesture image is not processed, the gesture cannot be recognized or the recognition may fail.

For example, for a glide gesture, when a user enters a glide gesture image template in a device normal posture, a camera detects a top-down glide, and the glide gesture is recognized; if the user acquires the gesture image in the inverted posture of the device in the later stage, the acquired gesture image may be recognized as a slide-up gesture or recognition failure.

Step S203, when the current posture is not the preset posture, determining the rotation angle of the terminal equipment switched from the current posture to the preset posture; based on the angle, the collected gesture image is rotated to obtain an adjusted gesture image;

in this example, if the current posture of the terminal device when acquiring the gesture image is different from the preset posture of the gesture image template input by the terminal device, the rotation angle required for switching the terminal posture from the current posture to the preset posture can be determined according to the current posture and the preset posture; and the collected gesture images are rotated, so that the selected gesture images can be correctly identified.

For example, when a user performs gesture recognition, the terminal device acquires a gesture image through the camera, reads that the current gesture of the terminal device is the device inverted gesture through the gyroscope, and inputs a preset gesture when a gesture image template is input as the device upright gesture; according to the comparison between the current posture and the preset posture, the current posture when the gesture image is collected is different from the preset posture of the gesture image template input by the terminal equipment, and the rotation angle required for switching from the current posture to the preset posture is 180 degrees; through carrying out 180 degrees rotation processing to the gesture image that gathers, the real gesture of user can effectively be reduced to the gesture image after the adjustment, promotes gesture recognition's accuracy.

Step S204, determining the ambient brightness when the gesture image is acquired; determining a target model of gesture recognition according to the environment brightness; performing gesture recognition on the adjusted gesture image based on the target model;

in this example, the ambient brightness of the gesture image may be determined according to the brightness information of the acquired gesture image. For example, determining a brightness mean value of the gesture image according to brightness values of all pixel points in the gesture image; and determining the brightness mean value of the gesture image as the environment brightness of the gesture image.

It should be noted that the difference of image quality of gesture images acquired by a camera of the terminal device under different ambient light is large; for example, a gesture image acquired in a dark light environment may have insufficient detail information due to insufficient exposure of the image; the gesture image collected in the bright environment may cause the information of the gesture image to be lost due to the overexposure of the image.

Thus, after the gesture image is acquired, determining the ambient brightness of the gesture image; and classifying the gesture images according to the environment brightness, and recognizing the gestures by adopting different recognition models aiming at different types of gesture images, thereby improving the accuracy of gesture recognition.

Here, the types of the gesture image may include: a dim light image, a normal light image, and a bright light image. The type of the gesture image can be determined based on the corresponding ambient brightness ranges of different types of gesture images and the ambient brightness of the gesture images.

In step S205, if the adjusted ambient brightness of the gesture image is within the illumination abnormal brightness range, an alarm prompt message is output.

In this example, the illumination abnormal brightness range may be set by default, or may be set or modified by a user according to user requirements.

And when the adjusted environment brightness of the gesture image is in the illumination abnormal brightness range, the terminal equipment outputs alarm prompt information for prompting that the current working environment of the terminal equipment is abnormal.

The embodiment of the disclosure also provides a gesture recognition device. Fig. 7 is a schematic structural diagram illustrating a gesture recognition apparatus according to an exemplary embodiment, and as shown in fig. 7, the gesture recognition apparatus 100 includes:

the acquisition module 101 is configured to acquire a gesture image and acquisition scene information of the gesture image;

the recognition module 102 is configured to recognize the gesture image according to the acquisition scene information of the gesture image; wherein the collecting scene information at least comprises one of the following: acquiring the current posture of the terminal equipment when the gesture image is acquired; and acquiring the environment of the terminal equipment when the gesture image is acquired.

Optionally, the identifying module 102 is configured to:

and performing gesture recognition on the adjusted gesture image.

Optionally, the identifying module 102 is further configured to:

Optionally, the identifying module 102 is configured to:

Optionally, the identifying module 102 is further configured to:

The device further comprises:

and the output module 103 is configured to output alarm prompt information if the ambient brightness in the ambient information of the gesture image is within the illumination abnormal brightness range.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

FIG. 8 is a block diagram illustrating a gesture recognition apparatus according to an example embodiment. For example, the device 200 may be a mobile phone, a mobile computer, or the like.

Referring to fig. 8, the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, an input/output (I/O) interface 212, a sensor component 214, and a communication component 216.

The processing component 202 generally controls overall operation of the device 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 202 may include one or more processors 220 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 202 can include one or more modules that facilitate interaction between the processing component 202 and other components. For example, the processing component 202 can include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

Memory 204 is configured to store various types of data to support operation at device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 206 provides power to the various components of the device 200. The power components 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 200 is in an operational mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 210 is configured to output and/or input audio signals. For example, audio component 210 includes a Microphone (MIC) configured to receive external audio signals when apparatus 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 804 or transmitted via the communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

The I/O interface 212 provides an interface between the processing component 202 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor component 214 includes one or more sensors for providing various aspects of status assessment for the device 200. For example, the sensor component 214 may detect an open/closed state of the device 200, the relative positioning of components, such as a display and keypad of the apparatus 200, the sensor component 214 may also detect a change in position of the apparatus 200 or a component of the apparatus 200, the presence or absence of user contact with the apparatus 200, orientation or acceleration/deceleration of the apparatus 200, and a change in temperature of the apparatus 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 216 is configured to facilitate wired or wireless communication between the apparatus 200 and other devices. The device 200 may access a wireless network based on a communication standard, such as Wi-Fi,2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as memory 204, comprising instructions executable by processor 220 of device 200 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A method of gesture recognition, the method comprising:

2. The method according to claim 1, wherein the recognizing the gesture image according to the captured scene information of the gesture image comprises:

and performing gesture recognition on the adjusted gesture image.

3. The method according to claim 2, wherein the adjusting the gesture image based on the gesture difference between the current gesture and the preset gesture of the terminal device comprises:

4. The method according to claim 1, wherein the recognizing the gesture image according to the captured scene information of the gesture image comprises:

5. The method according to claim 4, wherein the determining a target model for gesture recognition according to the environment information, and the recognizing the gesture image based on the target model comprises:

if the environmental brightness in the environmental information is larger than a first brightness threshold, determining that the target model is a first model, extracting a first brightness area with the brightness value of pixels smaller than the first brightness threshold from the gesture image based on the first model, and determining a target gesture type corresponding to the gesture image based on the first brightness area;

6. The method of claim 4, further comprising:

7. A gesture recognition apparatus, the apparatus comprising:

8. The apparatus of claim 7, wherein the identification module is configured to:

and performing gesture recognition on the adjusted gesture image.

9. The apparatus of claim 8, wherein the identification module is further configured to:

10. The apparatus of claim 7, wherein the identification module is configured to:

11. The apparatus of claim 10, wherein the identification module is further configured to:

12. The apparatus of claim 10, further comprising:

13. A gesture recognition apparatus, comprising:

a processor;

a memory for storing executable instructions;

wherein the processor is configured to: the method of gesture recognition as claimed in any one of claims 1 to 6 when executed by executable instructions stored in the memory.

14. A non-transitory computer readable storage medium, instructions in which, when executed by a processor of a control device, enable the control device to perform the gesture recognition method of any one of claims 1 to 6.