WO2017113794A1 - 手势动作识别方法、控制方法和装置以及腕式设备 - Google Patents

手势动作识别方法、控制方法和装置以及腕式设备 Download PDF

Info

Publication number
WO2017113794A1
WO2017113794A1 PCT/CN2016/093226 CN2016093226W WO2017113794A1 WO 2017113794 A1 WO2017113794 A1 WO 2017113794A1 CN 2016093226 W CN2016093226 W CN 2016093226W WO 2017113794 A1 WO2017113794 A1 WO 2017113794A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
image
preset
contact
contacts
Prior art date
Application number
PCT/CN2016/093226
Other languages
English (en)
French (fr)
Inventor
张霄
Original Assignee
北京体基科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京体基科技有限公司 filed Critical 北京体基科技有限公司
Publication of WO2017113794A1 publication Critical patent/WO2017113794A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures

Definitions

  • the present invention relates to the field of smart wearable devices, and in particular, to a gesture motion recognition method, a control method and apparatus, and a wrist device.
  • the integration of wrist smart devices is getting higher and higher, and functions are becoming more and more abundant.
  • a large proportion of mobile phone functions can pass smart watches and smart hands.
  • Ring implementation greatly simplifies the way users receive and deliver information.
  • the wrist smart device is limited to the small-size display screen.
  • the user cannot use the touch screen or the button to complete the operation of the related function, which is easy to cause misoperation.
  • a smart watch is worn on one hand, it must be operated. Except for wake-up, sleep, etc., the simple operation does not require the other hand to operate. The other complicated operations are completed by the other hand and cannot be used.
  • the smart watch is operated independently with one hand, and therefore, the smart watch still has great defects in content display and operation.
  • Cipheral Patent Application CN104756045A discloses a wearable sensing device for performing gesture-based control of a computing device, the wearable sensing device comprising: a camera for capturing the sensing An image of the articulated portion of the wearer's body of the device; a tracking module arranged to use the captured image to follow in real time Tracing a 3D articulated model of the articulated body part without wearing a marker on the articulated body part; a communication interface arranged to track the 3D The articulated model is sent to a computing device to control the computing device in accordance with the 3D articulation of the articulated body part.
  • the device can perform 3D modeling on the wearer's hand, and then use the 3D model to reflect the wearer's hand movement, and control the device according to the hand movement.
  • the 3D modeling operation adopted by the device relies on a plurality of relatively complicated auxiliary devices such as the above-mentioned camera and tracking module, which has higher requirements on hardware performance, higher power consumption, and poor portability of the device.
  • the technical problem to be solved by the present invention is that the wrist device control scheme in the prior art has high performance requirements on hardware devices and high power consumption.
  • the present invention provides a gesture motion recognition method, including: acquiring a hand image; identifying each hand specific region in the hand image; and monitoring a position of the identified specific region of the hand, at least When the distance between the two hand-specific regions is less than the preset threshold, it is determined that the portions corresponding to the at least two hand-specific regions are in contact.
  • the present invention also provides a gesture motion control method, comprising: acquiring a hand image; identifying each hand specific region in the hand image; and monitoring a position of the identified specific region of the hand, when at least 2 hands When the distance between the specific regions is less than the preset threshold, determining that the portions corresponding to the at least two hand-specific regions are in contact; recording the contact time of the contact portion and/or the number of contacts within the preset time; The preset action is performed by the contact time of the contacted portion and/or the number of contacts within the preset time.
  • the performing the preset action according to the contact time of the contacted portion comprises: determining whether the contact duration of the contacted portion reaches the first preset time; and when the first preset time is reached, executing the first pre-preparation And setting an action, when the first preset time is not reached, performing a second preset action different from the first preset action.
  • the performing the preset action according to the number of contacts in the preset time of the contact portion includes: counting the number of contacts of the part in the second preset time; performing execution is associated with the number of contacts Preset action.
  • the method further includes: identifying a contact portion; performing the preset action according to the contact time of the contact portion and/or the contact time within the preset time comprises: contacting time according to the contact portion and/or pre- The preset action is performed by setting the number of contacts in the set time and the mark information associated with the contacted portion.
  • the present invention also provides another gesture motion control method, including: acquiring a hand image; identifying each hand specific region in the hand image; and monitoring the position of the identified specific region of the hand, when at least 2 When the distance between the specific areas of the hand is less than a preset threshold, determining that the corresponding parts of the at least two hand-specific areas are in contact; identifying the contacted parts; performing presets according to the marked information associated with the contacted parts action.
  • the method further includes: recording a contact time of the contact portion and/or a contact time within the preset time; and performing the preset action according to the mark information associated with the contact portion includes: contacting according to the contacted portion The preset action is performed at the time and/or the number of contacts in the preset time and the tag information associated with the contacted portion.
  • the specific area of the hand is a fingertip area.
  • the identifying a specific area of the hand in the hand image comprises: removing a foreground and/or a background image from the hand image; identifying a hand contour in the hand image after the background image is removed The fingertip area is identified based on the curvature of the hand contour.
  • the removing the foreground and/or the background image from the hand image comprises: performing color space conversion processing on the hand image; and binarizing the hand image subjected to color space conversion processing The foreground and/or background image is removed from the binarized hand image.
  • the removing the foreground and/or the background image from the hand image comprises: acquiring a depth value of each pixel point in the hand image; and setting a depth value of the each pixel point with a preset depth range The values are compared to determine a finger image, foreground and/or background image from the hand image; the foreground and/or background image is removed.
  • the present invention provides a gesture recognition device, comprising: an acquisition unit, Obtaining a hand image; a feature recognition unit configured to identify each hand specific region in the hand image; and a determining unit configured to monitor the position of the identified specific region of the hand, when at least two hand specific When the distance between the regions is less than the preset threshold, it is determined that the portions corresponding to the at least two hand-specific regions are in contact.
  • the present invention also provides a gesture motion control apparatus, comprising: an acquisition unit for acquiring a hand image; a feature recognition unit for identifying each hand specific region in the hand image; and a determination unit for monitoring Determining the position of the specific region of the hand, when the distance between the at least two hand-specific regions is less than a preset threshold, determining that the portions corresponding to the at least two hand-specific regions are in contact; the recording unit is used The contact time of the contact portion and/or the number of contacts in the preset time period are recorded; the execution unit is configured to perform the preset action according to the contact time of the contact portion and/or the contact time within the preset time.
  • the executing unit includes: a determining subunit, configured to determine whether a contact duration of the contacted portion reaches a first preset time; and the first executing subunit is configured to execute when the first preset time is reached The first preset action, when the first preset time is not reached, performing a second preset action different from the first preset action.
  • the execution unit includes: a statistical subunit for counting the number of contacts of the part in the second preset time; and a second executing subunit executing a preset action associated with the number of contacts.
  • the method further includes: a part identification unit for identifying a contact portion; the execution unit is configured to be associated with the contact time of the contacted portion and/or the number of contacts within the preset time and the contact portion
  • the tag information performs a preset action.
  • the present invention also provides another gesture motion control apparatus, including: an acquisition unit for acquiring a hand image; a feature recognition unit for identifying each hand specific region in the hand image; and a determining unit for Monitoring the position of the identified specific area of the hand, when the distance between the at least two hand-specific areas is less than a preset threshold, determining that the corresponding parts of the at least two hand-specific areas are in contact; the part identification unit And a unit for identifying a contact; the executing unit is configured to perform a preset action according to the marker information associated with the contacted portion.
  • the method further includes: a recording unit for recording contact time of the contacted portion and/or The number of contacts in the preset time; the execution unit is configured to perform a preset action according to the contact time of the contacted portion and/or the number of contacts in the preset time and the mark information associated with the contacted portion.
  • the specific area of the hand is a fingertip area.
  • the feature recognition unit comprises: a background removal subunit for removing foreground and/or background images from the hand image; and a contour recognition subunit for use in the hand image after the background image is removed Identifying a hand contour; a fingertip recognition subunit for identifying a fingertip region based on the curvature of the hand contour.
  • the background removal subunit includes: a color space conversion unit configured to perform color space conversion processing on the hand image; and a binarization unit configured to perform a color image on the hand image after the color space conversion process Value processing; a background removal unit for removing foreground and/or background images from the binarized hand image.
  • the background removal subunit includes: a depth value acquisition subunit, configured to acquire a depth value of each pixel point in the hand image; and an image determination subunit, configured to: depth values of the respective pixel points The preset depth range values are compared to determine a finger image, foreground and/or background image from the hand image; an image removal subunit for removing the foreground and/or background image.
  • the present invention also provides a wrist device, comprising: an image capturing device for collecting a wearer's hand image along the wrist of the wearer toward the palm; a processor for receiving the collected hand image and for the hand The image is processed.
  • the processor uses the above method to recognize a gesture action with a hand image acquired by the camera.
  • the processor performs gesture action control with the hand image acquired by the camera device by using the above method.
  • the wrist device is a smart watch
  • the camera device is disposed on a watchband
  • the processor is disposed at a dial
  • the camera device and the processor are connected by a connecting component disposed in the watchband .
  • the gesture motion recognition method and apparatus described above by identifying each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into an area in the two-dimensional screen. Then, by judging the position and distance of the area in the two-dimensional picture, the gesture action of the wearer's hand part can be identified. It can be seen that the present invention does not require high-performance hardware to perform three-dimensional modeling of the human hand, and the motion of the hand can be judged only by the two-dimensional image of the hand, thereby reducing the requirement for hardware performance, and the data. Smaller calculations This method is easier to implement and more practical.
  • the first gesture motion control method and apparatus described above by identifying each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into a region in the two-dimensional image, and then pass the two-dimensional image. In the judgment of the position and distance of the area in the middle, the gesture action of the wearer's hand part can be recognized, and then the control of the smart watch can be realized according to the duration of the contact of the hand part and the number of contacts in a certain period of time.
  • the scheme has lower requirements on hardware performance and less data calculation, and its convenience and practicability are strong.
  • the second gesture motion control method and apparatus described above by identifying each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into a region in the two-dimensional image, and then pass the two-dimensional image.
  • the gesture action of the hand part can be recognized, and then the contact parts are further judged, and various control operations on the smart watch are realized according to different part contact combinations.
  • the hardware performance requirements are lower, the data calculation amount is smaller, and the convenience and practicability are strong.
  • the wrist device can use the camera device to collect the image of the wearer's hand along the wearer's wrist in the direction of the palm of the hand, and the captured image can display the image of the user's finger, and then the processor can analyze and judge the image of the opponent, and the like.
  • the hand image recognizes the user's gesture action, which in turn enables control of the device.
  • FIG. 1 is a schematic structural diagram of a wrist device according to an embodiment of the present invention.
  • FIG. 2 is a hand image captured by an image capturing apparatus of a smart watch according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a smart watch according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of a gesture motion recognition method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a fingertip region identified by using a gesture recognition method provided by an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of converting the image shown in FIG. 2 by using the gesture motion recognition method provided by the embodiment of the present invention
  • FIG. 7 is a schematic diagram of converting the image shown in FIG. 5 by using the gesture motion recognition method provided by the embodiment of the present invention.
  • FIG. 8 is a flowchart of a gesture action control method according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of another gesture action control method according to an embodiment of the present invention.
  • FIG. 10 is a flowchart of a third gesture action control method according to an embodiment of the present invention.
  • FIG. 11 is a structural diagram of a gesture motion recognition apparatus according to an embodiment of the present invention.
  • FIG. 12 is a structural diagram of a gesture action control apparatus according to an embodiment of the present invention.
  • FIG. 13 is a structural diagram of another gesture motion control apparatus according to an embodiment of the present invention.
  • connection or integral connection; may be mechanical connection or electrical connection; may be directly connected, may also be indirectly connected through an intermediate medium, or may be internal communication of two components, may be wireless connection, or may be wired connection.
  • connection or integral connection; may be mechanical connection or electrical connection; may be directly connected, may also be indirectly connected through an intermediate medium, or may be internal communication of two components, may be wireless connection, or may be wired connection.
  • the embodiment of the invention provides a smart wrist device. As shown in FIG. 1 , the device includes:
  • the imaging device 11 is configured to collect the image of the wearer's hand along the wrist of the wearer in the direction of the palm.
  • the arrangement of the imaging device 11 is different for different types of wrist devices, but the angle with the wearer's arm is relatively fixed.
  • the image pickup apparatus 11 set in this manner can collect the hand image as shown in FIG. 2.
  • the processor 12 is configured to receive and process the image of the hand, and the processor 12 can perform various processing on the image, for example, identifying a hand motion in the image, controlling the device according to the hand motion reflected by the image, etc., specifically This will be described in detail in the subsequent embodiments.
  • the device can use the camera 11 to collect the image of the wearer's hand along the wrist of the wearer in the direction of the palm of the wearer.
  • the captured image can display the image of the user's finger, and then the processor 12 can perform analysis and judgment on the image of the opponent, and the like.
  • the hand image recognizes the user's gesture action, which in turn enables control of the device.
  • the wrist device may be a smart watch, and the camera device 11 may be disposed at the dial.
  • the user needs to wear the dial on the inner side of the wrist, which obviously does not conform to the general wearing habit.
  • the above-mentioned camera device 11 can be disposed on a wristband.
  • the processor 12 is disposed as a processing core of the smart watch, and is disposed at the dial.
  • the connecting component of the camera 11 and the processor 12 is disposed in the strap.
  • the connecting component may be a flexible circuit board.
  • the embodiment of the present invention provides a gesture action recognition method, which may be performed by the processor 12 in Embodiment 1. As shown in FIG. 4, the method includes the following steps:
  • a hand image is acquired, which may be an image acquired by the image pickup device 11 in the first embodiment.
  • each hand-specific area is identified in the hand image.
  • the specific area may be an area corresponding to each part such as each finger, palm, and fingertip.
  • the palm and the finger can be recognized according to the hand skin texture in the image, and the finger and the fingertip can be recognized according to the contour of the hand in the image.
  • the image can be pre-processed, for example, the image is first subjected to color space conversion processing to distinguish the hand from the background, and then the hand is recognized, and finally the specific region is identified according to the contour and the skin texture feature.
  • a specified target for example, it can be identified according to the characteristics of the line, or it can be recognized by using a neural network model or the like.
  • the fingertip area that can be identified may be a pixel area as shown in FIG. 5, that is, one pixel area may be captured at the position of each fingertip.
  • step S3 the position of the identified specific region of the hand is monitored.
  • the distance between the at least two hand-specific regions is less than a preset threshold, it is determined that the portions corresponding to the at least two hand-specific regions are in contact.
  • each specific area has a set of two-dimensional coordinate values, and the respective areas move as the wearer's hand moves, whereby the coordinate values of the specific areas are released in real time.
  • the present invention is not limited to monitoring the contact action of two fingertip regions, and it is feasible to monitor more joints of the fingertip regions (for example, the thumb, the index finger and the middle finger are in contact with one point. action). Thereafter, various operations can be further implemented by contact events, which can be used, for example, to control a smart watch to contact events to trigger various functions of the watch.
  • the gesture motion recognition method described above by recognizing each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into an area in the two-dimensional screen, and then pass the position of the region in the two-dimensional image.
  • the judgment of the distance can recognize the gesture action of the wearer's hand part. It can be seen that the present invention does not require high-performance hardware to perform three-dimensional modeling of the human hand, and the motion of the hand can be judged only by the two-dimensional image of the hand, thereby reducing the requirement for hardware performance, and the data. Smaller calculations This method is easier to implement and more practical.
  • the environment in which the wearer is located will affect the difficulty of recognizing the hand portion in the image, and the hand image obtained may be pre-processed in order to facilitate identification of a particular portion in the image.
  • fingertip touch is more suitable as a control operation; from the perspective of recognition difficulty, the characteristics of the fingertip are more prominent and easier to be recognized. Therefore, the above-described hand specific region is preferably a fingertip region of each finger.
  • step S2 may include the following steps:
  • Step S21 removing foreground and/or background images from the hand image, and determining a foreground or background image from the image. Since the skin color of the human body has a certain range, in the image, the hand region is The RGB values of the pixels should be within a certain range, so that the content in the image can be judged according to the preset RGB value range, and the target image and the background image can be selected; or the sharpness value or depth of the image can be selected according to the image. The value is used to judge and remove the content in the image, and the existing removal methods are feasible.
  • the hand contour is recognized in the hand image after the foreground and/or the background is removed, and specifically, the edge of the hand region can be recognized to obtain the hand contour. Only the skin area remains in the hand image after the background image is removed, and the area can be considered as the hand area.
  • the Canny operator can be used to extract the edge contour of the hand region. The Canny operator measures the signal-to-noise ratio and the positioning product through the optimization method. The method performs approximation and gets edge information.
  • Gaussian filtering of the image is first required to smooth the image noise to reduce the influence of noise on the detection result.
  • the Gaussian kernel function is as follows:
  • the fingertip region is identified based on the curvature of the hand contour. After the edge of the finger is obtained, the fingertip portion can be extracted by the finger shape. It can be known from the analysis of the shape of the fingertip portion that the edge of the fingertip has a sudden change in curvature, that is, the line on both sides of the finger is more consistent, and the fingertip line is more curved and close to a 180 degree turn.
  • the edges of the image are first sampled and the image edge lines are vectorized to form a feature line having length and trend statistics.
  • vectorization the direction is obtained according to the position between the pixels and the first-order difference.
  • the point multiplication results of these vectors are calculated to obtain the angle between the vector lines.
  • find all straight straight segments for all edges for example, the average angle is no more than 25 degrees.
  • the straight line segments are arranged in order, and the change of the trend of the curved segments between the straight segments is calculated. For example, if the change of the strike is greater than 140 degrees and the distance is greater than a certain threshold, the corresponding fingertip is determined. The corresponding noise and the repeated result are removed and determined as the fingertip portion.
  • the above described preferred scheme has higher recognition efficiency and accuracy.
  • step S21 may further include the following sub-steps:
  • step S211a the image of the hand is subjected to color space conversion processing, and the human skin is composed of a dermis layer and a thin skin layer covering the skin layer, and light is absorbed by melanin in the skin layer, and absorption and scattering occur simultaneously in the dermis layer.
  • the skin color difference of different individuals is mainly manifested by the change of brightness caused by the difference of melanin concentration in the epidermis layer.
  • the optical properties of the dermis layer are basically the same, and the skin color of the same ethnic group has strong commonality, and is obviously different from most backgrounds. Color, which forms a small, compact cluster in the color space. Thus, skin based detection based on color is feasible.
  • the image captured by the camera is an RGB image.
  • the overlap between the skin color and the non-skin tone is more, and it is seriously affected by the brightness;
  • the HSV color space due to the good separation of hue, color saturation and brightness, There is less overlap with non-skinning points;
  • the CbCr subspace in the YCbCr color space the skin color is well concentrated in an ellipse-like range, and the distribution on the Cb and Cr components is also concentrated. Therefore, it is feasible to convert the hand image from the RGB space to the YCbCr color space or the HSV color space.
  • the image as shown in Fig. 2 can be processed into an image as shown in Fig. 6. Color empty There are many ways to convert between them, and it is feasible to use existing conversion methods.
  • Step S212a performing binarization processing on the hand image subjected to the color space conversion processing, and converting the image shown in FIG. 6 into a line graph having only two colors of black and white as shown in FIG. 7;
  • step S213a the background image is removed from the hand image after the binarization process.
  • the above preferred solution can further improve the accuracy of identifying the fingertip area.
  • step S21 may include the following steps:
  • Step S211b acquiring a depth value of each pixel in the hand image
  • Step S212b comparing the depth value of each pixel point with a preset depth range value to determine a finger image, a foreground, and/or a background image from the hand image;
  • the part of the finger to be imaged is about 10-15 cm away from the camera device, so the focus point of the camera device can be fixed, and only the focus within 10-15 cm is required to be clear; at the same time, imaging Other objects in the range (foreground and background) are usually closer or farther from the hand, not within the range of 10-15 cm, so the foreground and background are out of focus, and the ambiguity algorithm can easily distinguish the front background. Thereby, it is possible to determine the content (foreground image) that is too close to the imaging device 11 and the content (background image) that is too far from the imaging device 11.
  • Step S213b removing the foreground and/or background image.
  • the above preferred solution removes both the foreground and background images according to the depth of field information, and only retains the scene at the front of the finger, and further recognizes the fingertip region in the scene, thereby further improving the recognition efficiency.
  • the embodiment of the present invention provides a gesture action control method, which can be performed by the processor 12 in Embodiment 1, as shown in FIG. 8, the method includes the following steps:
  • Step S1a acquiring a hand image
  • Step S2a identifying each hand specific area in the hand image
  • Step S3a monitoring the position of the identified specific area of the hand, and determining that the at least two hand-specific areas correspond to when the distance between the at least two hand-specific areas is less than a preset threshold The parts are in contact.
  • step S4a the contact time of the contacted portion and/or the number of contacts in the preset time are recorded. Specifically, taking two fingers (thumbs and forefingers) that are in contact as an example, when contacting, the processor 12 can record the duration of the current contact; and can also record the interval between the two contacts before and after, and further count the predetermined time. Number of contacts.
  • step S5a the preset action is performed according to the contact time of the contacted portion and/or the number of contacts in the preset time.
  • a number of control instructions may be pre-stored, each control instruction associated with time information and/or number of times information.
  • a control command can be determined based on the recorded contact time and/or the number of contacts with the pre-stored control command, and then executed to implement the preset action.
  • Each preset action can be associated with a unique contact time and/or number of contacts within a preset time. For example, a single finger contact time of more than 3s can control the device to shut down and touch the finger twice within 0.5s, and the selected operation can be performed in the device interface.
  • judging the contact time can also effectively prevent the occurrence of misoperations.
  • the gesture motion control method by recognizing each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into an area in the two-dimensional screen, and then to position the region in the two-dimensional screen. And the distance judgment can identify the gesture action of the wearer's hand part, and then further control the smart watch according to the duration of the contact of the hand part and the number of contacts within a certain time, the hardware performance of the solution The requirements are lower, the amount of data calculation is smaller, and the convenience and practicability are stronger.
  • the step S5a can be divided into three cases, a case in which only the contact time is considered, a case in which only the number of contacts is considered, and a case in which the above two factors are simultaneously considered.
  • step S5a may include:
  • Step S51a determining whether the contact duration of the contacted portion reaches the first preset time
  • Step S52a When the first preset time is reached, the first preset action is performed, and when the first preset time is not reached, the second preset action different from the first preset action is performed.
  • a short contact is a "selected” action
  • a long contact is an "exit” action, whereby different control actions can be performed according to the contact time.
  • the above preferred solution can realize the judgment of the short contact and the long contact, thereby performing different preset actions according to the judgment result, and the solution can enable the smart wrist device to support a richer gesture control action.
  • step S5a may include:
  • Step S51b counting the number of contacts of the part in the second preset time, specifically starting the recording time after the current contact action is completed, until the next contact action occurs, determining whether the interval time is less than the second preset time, and then possibly More contact, that is, double-click, three-shot, etc. in n seconds can be counted.
  • step S52b a preset action associated with the number of contacts is performed.
  • a click is a "check” action
  • a double click is an "exit” action, whereby different control actions can be performed according to the number of contacts.
  • the above preferred solution can perform statistics on the number of multiple combos, and perform different preset actions according to the statistical result.
  • the solution can enable the smart wrist device to support richer gesture control actions.
  • the above two factors can also be considered at the same time, and the combination of the above two factors is many, thereby providing more gesture actions, thereby correlating more preset actions, and further improving the richness of the control operation.
  • the embodiment of the present invention provides another gesture action control method, which is different from Embodiment 3 in that the preset action is determined by the combined contact situation of different fingertip regions. As shown in FIG. 9, the method includes the following steps:
  • Step S1b acquiring a hand image
  • Step S2b identifying each specific region of the hand in the hand image
  • step S3b the position of the identified specific region of the hand is monitored.
  • the distance between the at least two hand-specific regions is less than a preset threshold, it is determined that the portions corresponding to the at least two hand-specific regions are in contact.
  • each specific area can be marked when a specific area is recognized.
  • a (thumb) and B can be respectively marked.
  • index finger index finger
  • C middle finger
  • D ringless finger
  • E small finger 5 marks, assuming that the wearer's thumb and forefinger are in contact, the corresponding fingertip area A and the fingertip area B are in contact.
  • the preset action can be associated with the above tag information, and different preset actions are associated with different tag combinations.
  • step S5b a preset action is performed according to the tag information associated with the contacted portion, and a plurality of control commands may be stored in advance, and each control command is associated with a different preset action. That is, in the case of ignoring factors such as contact time, the control command is directly determined according to the mark of the specific area, thereby performing the preset action.
  • the thumb is in contact with the other four fingers, respectively, to output four different signals, respectively. Therefore, it is necessary to determine which finger is in contact with the finger.
  • the four finger tips in the image that is, the finger order determined by the horizontal direction (the little finger, the ring finger, the middle finger, the index finger) are determined according to the finger model. Then, based on the input video image, the change in the position of the finger movement and the number of fingers in the image are detected.
  • the gesture motion control method described above by recognizing each specific region in the hand image, it is possible to convert the human hand portion in the three-dimensional space into an area in the two-dimensional screen, and then to position the region in the two-dimensional image.
  • the judgment of the distance can recognize the gesture action of the contact of the hand part, and then further judge the contact parts, and realize various control operations on the smart watch according to different contact combinations, and the hardware performance requirements of the solution are compared. Low, small amount of data calculation, and its convenience and practicability.
  • Embodiments 3 and 4 can be combined, that is, the preset action can be determined simultaneously according to the above-mentioned contact duration, number of contacts, and part contact combination.
  • the present embodiment provides a gesture action control method, as shown in FIG. 10, the method includes:
  • Step S1c acquiring a hand image
  • Step S2c identifying each hand specific area in the hand image
  • Step S3c monitoring the position of the identified specific region of the hand, and determining that the portions corresponding to the at least two hand-specific regions are in contact when the distance between the at least two hand-specific regions is less than a preset threshold;
  • Step S4c identifying the parts that are in contact
  • Step S5c recording the contact time of the contacted portion and/or the number of contacts within the preset time
  • step S6c the preset action is performed according to the contact time of the finger in the contact portion and/or the number of contacts in the preset time and the mark information associated with the finger in the contact portion.
  • the preset action “turning the page to the right” can be associated with the fingertip region.
  • a and fingertip area B, and preset contact time 1s; the preset action “jump to the right to the final page” can be associated with the fingertip area A and the fingertip area B, and the preset contact time 2s, assuming that the wearer's When the thumb and forefinger touch each other, it is determined by performing the "page to the right” or "to the right to the final page” by judging the contact time.
  • the above preferred solution further determines the combined contact situation of the specific area on the basis of determining the contact time and the number of contacts, thereby performing more different preset actions according to the judgment result, and the solution can enable the smart watch wrist device to support More rich gesture control action.
  • the present invention provides a gesture recognition device, as shown in FIG. 11, the device includes: an acquisition unit 101 for acquiring a hand image; and a feature recognition unit 102 for identifying each hand specific in the hand image a determining unit 103, configured to monitor a position of the identified specific area of the hand, and determine, when the distance between the at least two hand-specific areas is less than a preset threshold, determining that the at least two hand-specific areas correspond to The parts are in contact.
  • the gesture gesture recognition device can convert a human hand portion in a three-dimensional space into an area in a two-dimensional image by recognizing each specific region in the hand image, and then pass the position and distance of the region in the two-dimensional image. Judging, the gesture action of the wearer's hand part can be recognized. It can be seen that the present invention does not require high-performance hardware to perform three-dimensional modeling on the human hand, and the hand motion can be judged only by the two-dimensional image of the hand, thereby reducing the hardware. Performance requirements, and the amount of data calculation is small. This method is easier to implement and more practical.
  • the above-described hand specific region may preferably be a fingertip region.
  • the feature recognition unit 102 includes:
  • a background removal subunit for removing foreground and/or background images from the hand image
  • a contour recognition subunit for recognizing a hand contour in a hand image after the background image is removed
  • a fingertip recognition subunit for identifying a fingertip region based on the curvature of the hand contour.
  • the background removal subunit may include:
  • a color space conversion unit configured to perform color space conversion processing on the hand image
  • a binarization unit for performing binarization processing on the hand image subjected to color space conversion processing
  • a background removing unit for removing foreground and/or background images from the binarized hand image.
  • the above preferred solution can further improve the accuracy of identifying the fingertip area.
  • the background removal subunit may include:
  • a depth value obtaining subunit configured to acquire a depth value of each pixel in the hand image
  • An image determining subunit configured to compare a depth value of each pixel point with a preset depth range value to determine a finger image, a foreground, and/or a background image from the hand image;
  • An image removal subunit for removing the foreground and/or background image.
  • the above preferred solution removes both the foreground and background images according to the depth of field information, and only retains the scene at the front of the finger, and further recognizes the fingertip region in the scene, thereby further improving the recognition efficiency.
  • the present invention also provides a gesture motion control apparatus.
  • the apparatus includes: an acquisition unit 111 for acquiring a hand image; and a feature recognition unit 112 for identifying each hand in the hand image. a specific area; a determining unit 113, configured to monitor a position of the identified specific area of the hand, and when the distance between the at least two hand-specific areas is less than a preset threshold, the determining unit The at least two portions corresponding to the specific area of the hand are in contact; the recording unit 114 is configured to record the contact time of the contact portion and/or the number of contacts within the preset time;
  • the executing unit 115 is configured to perform a preset action according to the contact time of the contacted portion and/or the number of contacts within the preset time.
  • the gesture motion control device can convert a human hand portion in a three-dimensional space into an area in a two-dimensional image by recognizing each specific region in the hand image, and then pass the position and distance to the region in the two-dimensional image.
  • the judgment can recognize the gesture action of the wearer's hand part, and then further control the smart watch according to the duration of the contact of the hand part and the number of contacts within a certain time, the hardware performance requirements of the solution The lower the amount of data calculation, the more convenient and practical.
  • the executing unit 115 includes: a determining subunit, configured to determine whether a contact duration of the contacted portion reaches a first preset time; and a first executing subunit, when the first preset time is reached, Performing a first preset action, when the first preset time is not reached, performing a second preset action different from the first preset action.
  • a determining subunit configured to determine whether a contact duration of the contacted portion reaches a first preset time
  • a first executing subunit when the first preset time is reached, Performing a first preset action, when the first preset time is not reached, performing a second preset action different from the first preset action.
  • the executing unit 115 includes: a statistical subunit for counting the number of contacts of the part in the second preset time; and a second executing subunit executing a preset action associated with the number of contacts.
  • the above preferred solution can perform statistics on the number of multiple combos, and perform different preset actions according to the statistical result.
  • the solution can enable the smart wrist device to support richer gesture control actions.
  • the method further includes: a part identification unit for identifying a contact portion; the execution unit is configured to be associated with the contact time of the contacted portion and/or the number of contacts within the preset time and the contact portion
  • the tag information performs a preset action.
  • the above preferred solution further determines the combined contact situation of the specific area on the basis of determining the contact time and the number of contacts, thereby performing more different preset actions according to the judgment result, and the solution can enable the smart watch wrist device to support More rich gesture control action.
  • the present invention further provides another gesture motion control apparatus.
  • the apparatus includes: an acquisition unit 121 for acquiring a hand image; and a feature recognition unit 122 for identifying each hand in the hand image. a specific area; a determining unit 123, configured to monitor a position of the identified specific area of the hand, and determine the at least two hand-specific areas when the distance between the at least two hand-specific areas is less than a preset threshold The corresponding parts are in contact; the part identification unit 124 is configured to identify the contacted parts; and the execution unit 125 is configured to perform the preset action according to the mark information associated with the contacted parts.
  • the gesture motion control device can convert a human hand portion in a three-dimensional space into an area in a two-dimensional image by recognizing each specific region in the hand image, and then pass the position and distance of the region in the two-dimensional image. Judging, the gesture action of the hand part can be recognized, and then the contact parts are further judged, and various control operations on the smart watch are realized according to different contact combinations, and the hardware performance requirement of the solution is low.
  • the amount of data calculation is small, and its convenience and practicability are strong.
  • the method further includes: a recording unit for recording a contact time of the contact portion and/or a contact time within a preset time; the execution unit for contacting the contact time and/or the preset time according to the contacted portion The number of contacts within and the tag information associated with the portion in contact with each other performs a preset action.
  • the above preferred solution further determines the contact time and the number of contacts on the basis of identifying the combined contact conditions of different specific areas, thereby performing more different preset actions according to the judgment result, and the scheme can make the smart watch support more Rich gesture control action.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种手势动作识别方法、控制方法和装置以及腕式设备，所述手势动作识别方法包括：获取手部图像（S1）；在所述手部图像中识别出各个手部特定区域（S2）；监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触（S3）。

Description

手势动作识别方法、控制方法和装置以及腕式设备
交叉引用
本申请要求在2015年12月31日提交中国专利局、申请号为201511030898.8、发明名称为“手势动作识别方法、控制方法和装置以及腕式设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及智能穿戴设备技术领域，具体涉及手势动作识别方法、控制方法和装置以及腕式设备。
背景技术
随着软硬件相关科技的快速发展，腕式智能装置，如智能手表，智能手环等的集成度越来越高，功能越来越丰富，很大比例的手机功能可以通过智能手表，智能手环实现，大大简化用户接收和传递信息的方法。但和传统智能手机比较，腕式智能装置受限于小尺寸显示屏幕，一方面，使用者在使用时无法很好地利用触屏或者按键完成相关功能的操作，易造成误操作，另一方面，当智能手表佩戴在一只手上时，要对其进行操作，除了唤醒，休眠等简单操作不需要另一只手操作外，其余的较为复杂的操作都有另一只手完成，无法使用单手独立对智能手表进行操作，因此，智能手表在内容显示和操作上仍存在很大的缺陷。
针对上述问题，中国专利申请CN104756045A公开了一种用于对计算设备进行基于姿势的控制的可佩戴感测设备，所述可佩戴感测设备包括：相机，所述相机用于捕捉所述感测设备的佩戴者的身体的由关节连接的部位的图像；跟踪模块，所述跟踪模块被安排成使用捕捉到的图像来实时地跟 踪所述由关节连接的身体部位的3D的由关节连接的模型，而无需在所述由关节连接的身体部位上佩戴标记；通信接口，所述通信接口被安排成将跟踪到的3D的由关节连接的模型发送给计算设备，以便根据所述由关节连接的身体部位的3D关节连接来控制所述计算设备。该设备可以对佩戴者的手部进行3D建模，然后利用3D模型反映佩戴者手部动作，根据手部动作实现对设备的控制。但是该设备所采用的3D建模操作需要依赖诸如上述摄像机和跟踪模块等多个比较复杂的辅助设备，该方案对硬件性能的要求较高，功耗较大，并且该设备便携性较差。
发明内容
因此，本发明要解决的技术问题在于现有技术中的腕式设备控制方案对硬件设备性能要求高且功耗大。
有鉴于此，本发明提供一种手势动作识别方法，包括：获取手部图像；在所述手部图像中识别出各个手部特定区域；监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。
本发明还提供一种手势动作控制方法，包括：获取手部图像；在所述手部图像中识别出各个手部特定区域；监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；记录相接触的部位的接触时间和/或预设时间内的接触次数；根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。
优选地，所述根据相接触的部位的接触时间执行预设动作，包括：判断相接触的部位的接触持续时间是否达到第一预设时间；当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。
优选地，所述根据相接触的部位预设时间内的接触次数执行预设动作，包括：统计第二预设时间内所述部位的接触次数；执行与接触次数相关联 的预设动作。
优选地，还包括：识别相接触的部位；所述根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作包括：根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
本发明还提供另一种手势动作控制方法，包括：获取手部图像；在所述手部图像中识别出各个手部特定区域；监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；识别相接触的部位；根据相接触的部位所关联的标记信息执行预设动作。
优选地，还包括：记录相接触的部位的接触时间和/或预设时间内的接触次数；所述根据相接触的部位所关联的标记信息执行预设动作包括：根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
优选地，所述手部特定区域是指尖区域。
优选地，所述在所述手部图像中识别出手部特定区域，包括：从所述手部图像中去除前景和/或背景图像；在去除了背景图像后的手部图像中识别手部轮廓；根据所述手部轮廓的曲率识别出指尖区域。
优选地，所述从所述手部图像中去除前景和/或背景图像，包括：对所述手部图像进行色彩空间转换处理；对经过色彩空间转换处理后的手部图像进行二值化处理；在经过二值化处理后的手部图像中去除前景和/或背景图像。
优选地，所述从所述手部图像中去除前景和/或背景图像，包括：获取所述手部图像中各个像素点的深度值；将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；去除所述前景和/或背景图像。
相应地，本发明提供一种手势动作识别装置，包括：获取单元，用于 获取手部图像；特征识别单元，用于在所述手部图像中识别出各个手部特定区域；判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。
本发明还提供一种手势动作控制装置，包括：获取单元，用于获取手部图像；特征识别单元，用于在所述手部图像中识别出各个手部特定区域；判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；记录单元，用于记录相接触的部位的接触时间和/或预设时间内的接触次数；执行单元，用于根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。
优选地，所述执行单元包括：判定子单元，用于判断相接触的部位的接触持续时间是否达到第一预设时间；第一执行子单元，用于当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。
优选地，所述执行单元包括：统计子单元，用于统计第二预设时间内所述部位的接触次数；第二执行子单元，执行与接触次数相关联的预设动作。
优选地，还包括：部位识别单元，用于识别相接触的部位；所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
本发明还提供另一种手势动作控制装置，包括：获取单元，用于获取手部图像；特征识别单元，用于在所述手部图像中识别出各个手部特定区域；判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；部位识别单元，用于识别相接触的部位；执行单元，用于根据相接触的部位所关联的标记信息执行预设动作。
优选地，还包括：记录单元，用于记录相接触的部位的接触时间和/或 预设时间内的接触次数；所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
优选地，所述手部特定区域是指尖区域。
优选地，所述特征识别单元包括：背景去除子单元，用于从所述手部图像中去除前景和/或背景图像；轮廓识别子单元，用于在去除了背景图像后的手部图像中识别手部轮廓；指尖识别子单元，用于根据所述手部轮廓的曲率识别出指尖区域。
优选地，所述背景去除子单元包括：色彩空间转换单元，用于对所述手部图像进行色彩空间转换处理；二值化单元，用于对经过色彩空间转换处理后的手部图像进行二值化处理；背景去除单元，用于在经过二值化处理后的手部图像中去除前景和/或背景图像。
优选地，所述背景去除子单元包括：深度值获取子单元，用于获取所述手部图像中各个像素点的深度值；图像确定子单元，用于将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；图像去除子单元，用于去除所述前景和/或背景图像。
本发明还提供一种腕式设备，包括：摄像装置，用于沿佩戴者手腕向手心方向采集佩戴者手部图像；处理器，用于接收所采集的手部图像，并对所述手部图像进行处理。
优选地，所述处理器利用上述方法以所述摄像装置采集的手部图像识别手势动作。
优选地，所述处理器利用上述方法以所述摄像装置采集的手部图像进行手势动作控制。
优选地，所述腕式设备为智能手表，所述摄像装置设置在表带上，所述处理器设置在表盘处，所述摄像装置与所述处理器通过设置在表带内的连接部件连接。
根据上述手势动作识别方法及装置，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域， 然后通过对二维画面中区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作。由此可见，本发明不需要使用高性能的硬件对人体手部进行三维建模，仅通过手部的二维图像即可判断手部的动作，由此可以降低对硬件性能的要求，并且数据计算量较小本方法更易于实现，且实用性更强。
根据上述第一种手势动作控制方法及装置，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中的区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作，然后进一步根据手部部位相接触的持续时间以及一定时间内的接触次数，实现对智能手表的控制，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
根据上述第二种手势动作控制方法及装置，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中区域的位置和距离的判断，即可识别手部部位相接触的手势动作，然后进一步对相接触的部位进行判断，根据不同的部位接触组合实现各种对智能手表的控制操作，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
上述腕式设备可利用其摄像装置沿佩戴者手腕向手心方向采集佩戴者手部图像，其采集的图像可以展现出用户手指的影像，然后其处理器可对手部图像进行分析判断等处理，通过手部图像识别出用户的手势动作，进而可实现对设备的控制。
附图说明
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的腕式设备的结构示意图；
图2为本发明实施例提供的智能手表的摄像装置所采集的手部图像；
图3为本发明实施例提供的智能手表的结构示意图；
图4为本发明实施例提供的手势动作识别方法的流程图；
图5为利用本发明实施例提供的手势动作识别方法识别出的指尖区域示意图；
图6为利用本发明实施例提供的手势动作识别方法对图2所示图像进行转换后的示意图；
图7为利用本发明实施例提供的手势动作识别方法对图5所示图像进行转换后的示意图；
图8为本发明实施例提供的一种手势动作控制方法的流程图；
图9为本发明实施例提供的另一种手势动作控制方法的流程图；
图10为本发明实施例提供的第三种手势动作控制方法的流程图；
图11为本发明实施例提供的手势动作识别装置的结构图；
图12为本发明实施例提供的一种手势动作控制装置的结构图；
图13为本发明实施例提供的另一种手势动作控制装置的结构图。
具体实施方式
下面将结合附图对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本发明的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附 图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
在本发明的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，还可以是两个元件内部的连通，可以是无线连接，也可以是有线连接。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本发明中的具体含义。
此外，下面所描述的本发明不同实施方式中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
实施例1
本发明实施例提供一种智能腕式设备，如图1所示，该设备包括：
摄像装置11，用于沿佩戴者手腕向手心方向采集佩戴者手部图像，针对不同种类的腕式设备，摄像装置11的设置方式不同，但其与佩戴者手臂的角度是比较固定的。按此方式设置的摄像装置11可采集到如图2所示的手部图像。
处理器12，用于接收并对手部图像进行处理，处理器12可以对图像进行多种处理，例如在图像中识别手部动作、根据图像反映出的手部动作对设备进行控制等，具体将在后续实施例中进行详细介绍。
上述设备可利用其摄像装置11沿佩戴者手腕向手心方向采集佩戴者手部图像，其采集的图像可以展现出用户手指的影像，然后其处理器12可对手部图像进行分析判断等处理，通过手部图像识别出用户的手势动作，进而可实现对设备的控制。
上述腕式设备可以是智能手表，摄像装置11可设置在表盘处，如此设置则需使用者将表盘佩戴在手腕内侧，这显然不符合一般的佩戴习惯，所 以作为一个优选的实施方式，如图3所示上述摄像装置11可设置在表带上，用户佩戴手表时使摄像装置11贴在手腕内侧朝向手部，其角度和方向恰好可拍到手指，此结构不需要用户调整摄像装置11的位置，便于用户佩戴。处理器12作为智能手表的处理核心，可设置在表盘处，摄像装置11与所述处理器12的连接部件设置在所述表带内，该连接部件可以是柔性电路板。
实施例2
本发明实施例提供一种手势动作识别方法，该方法可以由实施例1中的处理器12执行，如图4所示该方法包括如下步骤：
步骤S1，获取手部图像，该图像可以是由实施例1中的摄像装置11采集到的图像。
步骤S2，在手部图像中识别出各个手部特定区域。在这里，特定区域可以是各个手指、手掌、各个指尖等各种部位对应的区域。
具体地，可以根据图像中的手部皮肤纹路对手掌、手指进行识别，也可以根据图像中的手部轮廓对手指、指尖进行识别。并且在识别前还可以图像进行预处理，例如首先对图像进行色彩空间转换处理，以区分出手部与背景，然后识别手部，最终根据轮廓、皮肤纹路特征识别特定区域。本领域技术人员可以理解，识别指定目标的方法有多种，例如可以根据线条的特点进行识别，或者利用神经网络模型等方式进行识别都是可行的。其中，以指尖为例，可以识别出的指尖区域可以是如图5所示的像素区域，即在每个指尖的位置都可以捕捉到一个像素区域。
步骤S3，监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。具体地，各个特定区域都有一组二维坐标值，随着佩戴者手部的活动，上述各个区域会发生移动，由此各个特定区域的坐标值会实时放生变化。以2个手指的指尖为例，当佩戴者的2个手指(例如拇指和食指)的指尖相互靠近时，相应的指尖区域会相互靠近，当该2个指尖区域之间的距离小于预设阈值时(预设阈值可以为0)，则判定佩戴者的2个 手指相接触。需要说明的是，本发明并不限于监测2个指尖区域的接触动作，监测更多的指尖区域相接处的动作都是可行的(例如拇指、食指和中指3手指相接触于一点的动作)。之后，可以通过接触事件进一步实现各种操作，接触事件例如可以用于控制智能手表，以接触事件来触发手表的各种功能。
根据上述手势动作识别方法，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作。由此可见，本发明不需要使用高性能的硬件对人体手部进行三维建模，仅通过手部的二维图像即可判断手部的动作，由此可以降低对硬件性能的要求，并且数据计算量较小本方法更易于实现，且实用性更强。
佩戴者所处的环境将影响在图像中识别手部部位的难度，为了便于在图像中识别特定部位，可以对获取到的手部图像进行预处理。并且，从人为操作习惯的角度考虑，指尖相触更适合作为控制操作；从识别难度的角度考虑，指尖部位的特点更突出，更容易被识别。因此，上述手部特定区域优选为各个手指的指尖区域。
作为一个优选的实施方式，上述步骤S2可以包括如下步骤：
步骤S21，从所述手部图像中去除前景和/或背景图像，从图像中确定前景或背景图像的方法有多种，由于人体的皮肤颜色具有一定范围，所以在图像中，手部区域的像素点的RGB值都应当在某一范围内，由此可以根据预设的RGB值范围对图像中的内容进行判断，可筛选出目标图像和背景图像；也可以根据图像的锐度值或深度值对图像中的内容进行判断和去除，现有的去除方法都是可行的。
步骤S22，在去除了前景和/或背景后的手部图像中识别手部轮廓，具体可识别手部区域的边缘即可得到手部轮廓。去除了背景图像后的手部图像中只保留有皮肤区域，可以认为该区域为手的区域。为了识别手指部分，需依据手指的形态特征进行判别。因而，可采用Canny算子提取手部区域的边缘轮廓。Canny算子通过对信噪比与定位乘积进行测度，通过最优化方 法进行逼近，得到边缘信息。
具体地址，首先需要对图像进行高斯滤波平滑图像噪声，以减小噪声对检测结果的影响，高斯核函数如下：
Figure PCTCN2016093226-appb-000001
然后计算图像灰度值的梯度，即做两个方向的一阶差分。计算每个像素点的梯度幅度及方向：
Figure PCTCN2016093226-appb-000002
相应的强度与方向为：
Figure PCTCN2016093226-appb-000003
θ[x，y]＝arctan(Gx(x，y)/Gy(x，y))。
得到整个图像中每个点的梯度幅度与方向后，计算局部最大值，保留相应的像素点。最后，根据双阈值计算应该保留的像素点，对于保留下的像素点进行边界追踪，完成边缘提取。步骤S23，根据手部轮廓的曲率识别出指尖区域。得到手指边缘后，可以利用手指形态进行指尖部位的提取。通过对于指尖部分形态的分析可以知道，手指指尖边缘具有曲率突变的情况，即手指两侧线条走向较为一致，而指尖线条弯曲程度较大，且接近180度转弯。
基于上述特性，首先对图像边缘进行采样并矢量化图像边缘线，以形成具有长度与走向统计的特征线。矢量化时，依据像素点间的位置求距离以及一阶差分得到方向走向。然后，计算这些矢量的点乘结果，得到矢量线间的夹角大小。而后，针对所有边缘寻找所有的较直的直线段(例如平均夹角不大于25度)。按顺序排列这些直线段，计算直线段间曲线段的走向变化，例如将走向变化大于140度，且距离大于一定阈值，则判定为相应的指尖。去除相应噪声以及重复的结果，判定为指尖部位。上述优选方案的识别效率以及准确性更高。
由于佩戴者所处的环境可能比较复杂，由此可能给背景图像的去除操作带来一定难度，为解决此问题，上述步骤S21可进一步包括如下子步骤：
步骤S211a，对手部图像进行色彩空间转换处理，人体皮肤由真皮层和覆盖其上的较薄的表皮层构成，光在表皮层中被黑色素吸收，而在真皮层中则同时发生吸收和散射。不同个体的肤色差异主要表现为由表皮层中黑色素的浓度不同所引起的亮度变化，其真皮层光学特性则基本相同，而且同种族的个体肤色具有较强的共性，并明显区别于大多数背景颜色，在颜色空间中形成一个小而紧致的聚簇。因而，基于颜色进行皮肤的检测是可行的。
进行肤色检测需要选择恰当的彩色空间，在此空间中肤色能团簇、聚合在一起，并且与非肤色的重叠部分要尽可能少。摄像头采集的图像是RGB图像，在RGB彩色空间中，肤色与非肤色的重叠部分较多，且会受亮度的影响严重；在HSV彩色空间中由于色调、色饱和度及亮度很好的分离，与非肤色点重叠的较少；在YCbCr彩色空间中的CbCr子空间上，肤色很好的聚集在一个类椭圆范围内，而且在Cb、Cr分量上的分布也比较集中。因此，将手部图像由RGB空间转换为YCbCr彩色空间或HSV彩色空间都是可行的。
RGB至HSV的转换公式为：
Figure PCTCN2016093226-appb-000004
RGB至YCbCr的转换公式为：
Y＝0.257R′+0.504G′+0.098B′+16
Cb＝-0.148R′-0.291G′+0.439B′+128
Cr＝0.439R′-0.368G′-0.071B′+128。
经过转换，可将如图2所示的图像处理为如图6所示的图像。色彩空 间转换的方法有多种，利用现有的转换方法都是可行的。
步骤S212a，对经过色彩空间转换处理后的手部图像进行二值化处理，经过转换，可将如图6所示的图像转换为如图7所示的只有黑白两种颜色的线条图；
步骤S213a，在经过二值化处理后的手部图像中去除背景图像。
上述优选方案可以进一步提高识别指尖区域的准确性。
作为另一个优选的实施方式，上述步骤S21可以包括如下步骤：
步骤S211b，获取所述手部图像中各个像素点的深度值；
步骤S212b，将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；
由于腕式设备的特殊性，需成像的手指部分距离摄像装置约为10-15cm左右，因此摄像装置的对焦点可以是固定的，仅需保证10-15cm内的对焦清晰即可；同时，成像范围内的其他物体(前景和背景)通常距离手部较近或较远，不在10-15cm距离范围内，因此前景和背景失焦，通过模糊度的算法可以容易的分辨前背景。由此即可判断出距离摄像装置11过近的内容(前景图像)和距离摄像装置11过远的内容(背景图像)。
步骤S213b，去除所述前景和/或背景图像。
上述优选方案根据景深信息将前景和背景图像均去除掉，只保留手指前部的景象，在该景象中进一步识别指尖区域，由此可进一步提高识别效率。
实施例3
本发明实施例提供一种手势动作控制方法，该方法可以由实施例1中的处理器12执行，如图8所示，该方法包括如下步骤：
步骤S1a，获取手部图像；
步骤S2a，在所述手部图像中识别出各个手部特定区域；
步骤S3a，监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应 的部位相接触。
步骤S4a，记录相接触的部位的接触时间和/或预设时间内的接触次数。具体地，以2个相接触的手指(拇指和食指)为例，当相接触时，处理器12可记录当前接触的持续时间；还可以记录前后两次接触的间隔时间，进一步统计预定时间内的接触次数。
步骤S5a，根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。可以预先存储若干控制指令，每一个控制指令关联时间信息和/或次数信息。由此，可以根据所记录的接触时间和/或接触次数与预存控制指令的关联关系确定一个控制指令，然后执行该指令实现预设动作。预设动作可以有多种，例如关机、在软件界面中的选中、退出、选择等，每一种预设动作都可以关联唯一的接触时间和/或预设时间内的接触次数。例如，单次手指相接触时间超过3s，可控制设备关机、0.5s内手指相接触2次，可在设备界面中执行选定操作。此外，对接触时间进行判断还可以有效的避免误操作的情况发生。
根据上述手势动作控制方法，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中的区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作，然后进一步根据手部部位相接触的持续时间以及一定时间内的接触次数，实现对智能手表的控制，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
如上所述，步骤S5a可分为三种情况，分别为只考虑接触时间的情况、只考虑接触次数的情况和同时考虑上述两种因素的情况。
对于只考虑接触时间的情况，上述步骤S5a可包括：
步骤S51a，判断相接触的部位的接触持续时间是否达到第一预设时间；
步骤S52a，当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。例如短接触是“选中”动作、长接触是“退出”动作，由此可根据接触时间进行不同的控制动作。
上述优选方案可实现对短接触和长接触进行判断，从而根据判断结果执行不同的预设动作，该方案可以使智能腕式设备支持更丰富的手势控制动作。
对于只考接触次数的情况，上述步骤S5a可包括：
步骤S51b，统计第二预设时间内所述部位的接触次数，具体可以在当前接触动作完毕后开始记录时间，直到下一次接触动作出现，判断间隔时间是否小于第二预设时间，之后还可能有更多次的接触，即n秒内的双击、三连击等操作都是可以被统计到的。
步骤S52b，执行与接触次数相关联的预设动作。例如单击是“选中”动作、双击是“退出”动作，由此可根据接触次数进行不同的控制动作。
上述优选方案可实现对多次连击的次数进行统计，从而根据统计结果执行不同的预设动作，该方案可以使智能腕式设备支持更丰富的手势控制动作。
实际应用中还可以同时考虑上述两种因素，上述两种因素的组合方式很多，由此可以提供更多的手势动作，从而关联更多的预设动作，进一步提高控制操作的丰富性。
实施例4
本发明实施例提供另一种手势动作控制方法，与实施例3的区别在于本实施例通过不同的指尖区域的组合接触情况,来确定预设动作。如图9所示，该方法包括如下步骤：
步骤S1b，获取手部图像；
步骤S2b，在手部图像中识别出各个手部特定区域；
步骤S3b，监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定至少2个手部特定区域所对应的部位相接触。
步骤S4b,识别相接触的部位。具体可以在识别出特定区域之时对各个特定区域赋予标记，例如对于5个指尖区域，可以分别标记A(拇指)、B (食指)、C(中指)、D(无名指)、E(小指)5个标记，假设佩戴者拇指和食指相接触，则相应的指尖区域A和指尖区域B相触。预设动作可以关联上述标记信息，且不同的预设动作关联不同的标记组合。
步骤S5b,根据相接触的部位所关联的标记信息执行预设动作，可以预先存储若干控制指令，每一个控制指令关联不同的预设动作。即在忽略接触时间等因素的情况下，直接根据特定区域的标记确定控制指令，进而执行预设动作。
在一个具体的实施例中，假设需要在大拇指分别与其他四个指头进行接触时进行判定，以分别输出四个不同的信号。因而，需要判定大拇指同何手指进行接触。判定时，首先依据手指模型判定图像中的四个手指尖，即由水平方向决定的手指顺序(小指、无名指、中指、食指)。然后依据输入的视频图像，检测手指运动位置的变化以及图像中的手指数量。若检测出两手指位置“相撞”且保持一定时间，同时还可以参考图像中剩余手指个数产生变化(认为手指间点击会造成图像中可识别的指尖个数减少)，则认为产生了点击动作。
根据上述手势动作控制方法，通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中区域的位置和距离的判断，即可识别手部部位相接触的手势动作，然后进一步对相接触的部位进行判断，根据不同的部位接触组合实现各种对智能手表的控制操作，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
实施例5
为了实现更多复杂的控制，可将实施例3、4结合，即可以同时根据上述接触持续时间、接触次数、部位接触组合来确定预设动作。具体地，本实施提供一种手势动作控制方法，如图10所示，该方法包括：
步骤S1c，获取手部图像；
步骤S2c，在所述手部图像中识别出各个手部特定区域；
步骤S3c，监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；
步骤S4c，识别相接触的部位；
步骤S5c，记录相接触的部位的接触时间和/或预设时间内的接触次数；
步骤S6c，根据相接触的部位手指的接触时间和/或预设时间内的接触次数以及相接触的部位手指所关联的标记信息执行预设动作。
由此，在考虑接触时间和/或预设时间内的接触次数的基础上，还可以同时考虑相接触的部位所关联的标记，例如，预设动作“向右翻页”可关联指尖区域A和指尖区域B、以及预设接触时间1s；预设动作“向右跳转到最终页面”可关联指尖区域A和指尖区域B、以及预设接触时间2s，假设当佩戴者的拇指和食指相触时，再通过对接触时间进行判断，即可确定执行“向右翻页”或“向右跳转到最终页面”。
上述优选方案在判断出接触时间和接触次数的基础上，进一步对特定区域的组合接触情况进行判断，从而根据判断结果执行更多不同的预设动作，该方案可以使智能腕表腕式设备支持更丰富的手势控制动作。
实施例6
本发明提供一种手势动作识别装置，如图11所示该装置包括：获取单元101，用于获取手部图像；特征识别单元102，用于在所述手部图像中识别出各个手部特定区域；判定单元103，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。
上述手势动作识别装置通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作。由此可见，本发明不需要使用高性能的硬件对人体手部进行三维建模，仅通过手部的二维图像即可判断手部的动作，由此可以降低对硬件 性能的要求，并且数据计算量较小本方法更易于实现，且实用性更强。
如实施例2所述，上述手部特定区域可优选为指尖区域。
优选地，所述特征识别单元102包括：
背景去除子单元，用于从所述手部图像中去除前景和/或背景图像；
轮廓识别子单元，用于在去除了背景图像后的手部图像中识别手部轮廓；
指尖识别子单元，用于根据所述手部轮廓的曲率识别出指尖区域。
上述优选方案的识别效率以及准确性更高。
优选地，所述背景去除子单元可以包括：
色彩空间转换单元，用于对所述手部图像进行色彩空间转换处理；
二值化单元，用于对经过色彩空间转换处理后的手部图像进行二值化处理；
背景去除单元，用于在经过二值化处理后的手部图像中去除前景和/或背景图像。
上述优选方案可以进一步提高识别指尖区域的准确性。
优选地，所述背景去除子单元可以包括：
深度值获取子单元，用于获取所述手部图像中各个像素点的深度值；
图像确定子单元，用于将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；
图像去除子单元，用于去除所述前景和/或背景图像。
上述优选方案根据景深信息将前景和背景图像均去除掉，只保留手指前部的景象，在该景象中进一步识别指尖区域，由此可进一步提高识别效率。
实施例7
本发明还提供一种手势动作控制装置，如图12所示该装置包括：获取单元111，用于获取手部图像；特征识别单元112，用于在所述手部图像中识别出各个手部特定区域；判定单元113，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所 述至少2个手部特定区域所对应的部位相接触；记录单元114，用于记录相接触的部位的接触时间和/或预设时间内的接触次数；
执行单元115，用于根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。
上述手势动作控制装置通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中的区域的位置和距离的判断，即可识别佩戴者手部部位相接触的手势动作，然后进一步根据手部部位相接触的持续时间以及一定时间内的接触次数，实现对智能手表的控制，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
优选地，所述执行单元115包括：判定子单元，用于判断相接触的部位的接触持续时间是否达到第一预设时间；第一执行子单元，用于当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。上述优选方案可实现对短接触和长接触进行判断，从而根据判断结果执行不同的预设动作，该方案可以使智能腕式设备支持更丰富的手势控制动作。
优选地，所述执行单元115包括：统计子单元，用于统计第二预设时间内所述部位的接触次数；第二执行子单元，执行与接触次数相关联的预设动作。上述优选方案可实现对多次连击的次数进行统计，从而根据统计结果执行不同的预设动作，该方案可以使智能腕式设备支持更丰富的手势控制动作。
优选地，还包括：部位识别单元，用于识别相接触的部位；所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
上述优选方案在判断出接触时间和接触次数的基础上，进一步对特定区域的组合接触情况进行判断，从而根据判断结果执行更多不同的预设动作，该方案可以使智能腕表腕式设备支持更丰富的手势控制动作。
实施例8
本发明还提供另一种手势动作控制装置，如图13所示该装置包括：获取单元121，用于获取手部图像；特征识别单元122，用于在所述手部图像中识别出各个手部特定区域；判定单元123，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；部位识别单元124，用于识别相接触的部位；执行单元125，用于根据相接触的部位所关联的标记信息执行预设动作。
上述手势动作控制装置通过在手部图像中识别出各个特定区域，可实现将三维空间中的人体手部部位转化为二维画面中的区域，然后通过对二维画面中区域的位置和距离的判断，即可识别手部部位相接触的手势动作，然后进一步对相接触的部位进行判断，根据不同的部位接触组合实现各种对智能手表的控制操作，本方案对硬件性能的要求较低、数据计算量较小，其便利性和实用性较强。
优选地，还包括：记录单元，用于记录相接触的部位的接触时间和/或预设时间内的接触次数；所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
上述优选方案在识别出不同的特定区域的组合接触情况的基础上，进一步对接触时间和接触次数进行判断，从而根据判断结果执行更多不同的预设动作，该方案可以使智能腕表支持更丰富的手势控制动作。
显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本发明创造的保护范围之中。

Claims (26)

  1. 一种手势动作识别方法，其特征在于，包括：
    获取手部图像；
    在所述手部图像中识别出各个手部特定区域；
    监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。
  2. 一种手势动作控制方法，其特征在于，包括：
    获取手部图像；
    在所述手部图像中识别出各个手部特定区域；
    监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；
    记录相接触的部位的接触时间和/或预设时间内的接触次数；
    根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。
  3. 根据权利要求2所述的方法，其特征在于，所述根据相接触的部位的接触时间执行预设动作，包括：
    判断相接触的部位的接触持续时间是否达到第一预设时间；
    当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。
  4. 根据权利要求2所述的方法，其特征在于，所述根据相接触的部位预设时间内的接触次数执行预设动作，包括：
    统计第二预设时间内所述部位的接触次数；
    执行与接触次数相关联的预设动作。
  5. 根据权利要求2-4中任一项所述的方法，其特征在于，还包括：识别相接触的部位；
    所述根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作包括：根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
  6. 一种手势动作控制方法，其特征在于，包括：
    获取手部图像；
    在所述手部图像中识别出各个手部特定区域；
    监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；
    识别相接触的部位；
    根据相接触的部位所关联的标记信息执行预设动作。
  7. 根据权利要求6所述的方法，其特征在于，还包括：记录相接触的部位的接触时间和/或预设时间内的接触次数；
    所述根据相接触的部位所关联的标记信息执行预设动作包括：根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
  8. 根据权利要求1-7中任一项所述的方法，其特征在于，所述手部特定区域是指尖区域。
  9. 根据权利要求8所述的方法，其特征在于，所述在所述手部图像中识别出手部特定区域，包括：
    从所述手部图像中去除前景和/或背景图像；
    在去除了前景和/或背景图像后的手部图像中识别手部轮廓；
    根据所述手部轮廓的曲率识别出指尖区域。
  10. 根据权利要求9所述的方法，其特征在于，所述从所述手部图像中去除前景和/或背景图像，包括：
    对所述手部图像进行色彩空间转换处理；
    对经过色彩空间转换处理后的手部图像进行二值化处理；
    在经过二值化处理后的手部图像中去除前景和/或背景图像。
  11. 根据权利要求9所述的方法，其特征在于，所述从所述手部图像中去除前景和/或背景图像，包括：
    获取所述手部图像中各个像素点的深度值；
    将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；
    去除所述前景和/或背景图像。
  12. 一种手势动作识别装置，其特征在于，包括：
    获取单元，用于获取手部图像；
    特征识别单元，用于在所述手部图像中识别出各个手部特定区域；
    判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触。
  13. 一种手势动作控制装置，其特征在于，包括：
    获取单元，用于获取手部图像；
    特征识别单元，用于在所述手部图像中识别出各个手部特定区域；
    判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；
    记录单元，用于记录相接触的部位的接触时间和/或预设时间内的接触次数；
    执行单元，用于根据相接触的部位的接触时间和/或预设时间内的接触次数执行预设动作。
  14. 根据权利要求13所述的装置，其特征在于，所述执行单元包括：
    判定子单元，用于判断相接触的部位的接触持续时间是否达到第一预设时间；
    第一执行子单元，用于当达到第一预设时间时，执行第一预设动作，当未达到第一预设时间时，执行与所述第一预设动作不同的第二预设动作。
  15. 根据权利要求13所述的装置，其特征在于，所述执行单元包括：
    统计子单元，用于统计第二预设时间内所述部位的接触次数；
    第二执行子单元，执行与接触次数相关联的预设动作。
  16. 根据权利要求13-15中任一项所述的装置，其特征在于，还包括：
    部位识别单元，用于识别相接触的部位；
    所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
  17. 一种手势动作控制装置，其特征在于，包括：
    获取单元，用于获取手部图像；
    特征识别单元，用于在所述手部图像中识别出各个手部特定区域；
    判定单元，用于监测所识别出的手部特定区域的位置，当至少2个手部特定区域之间的距离小于预设阈值时，判定所述至少2个手部特定区域所对应的部位相接触；
    部位识别单元，用于识别相接触的部位；
    执行单元，用于根据相接触的部位所关联的标记信息执行预设动作。
  18. 根据权利要求17所述的装置，其特征在于，还包括：
    记录单元，用于记录相接触的部位的接触时间和/或预设时间内的接触次数；
    所述执行单元用于根据相接触的部位的接触时间和/或预设时间内的接触次数以及相接触的部位所关联的标记信息执行预设动作。
  19. 根据权利要求12-18中任一项所述的装置，其特征在于，所述手部特定区域是指尖区域。
  20. 根据权利要求19所述的装置，其特征在于，所述特征识别单元包括：
    背景去除子单元，用于从所述手部图像中去除前景和/或背景图像；
    轮廓识别子单元，用于在去除了背景图像后的手部图像中识别手部轮廓；
    指尖识别子单元，用于根据所述手部轮廓的曲率识别出指尖区域。
  21. 根据权利要求20所述的装置，其特征在于，所述背景去除子单元 包括：
    色彩空间转换单元，用于对所述手部图像进行色彩空间转换处理；
    二值化单元，用于对经过色彩空间转换处理后的手部图像进行二值化处理；
    背景去除单元，用于在经过二值化处理后的手部图像中去除前景和/或背景图像。
  22. 根据权利要求20所述的装置，其特征在于，所述背景去除子单元包括：
    深度值获取子单元，用于获取所述手部图像中各个像素点的深度值；
    图像确定子单元，用于将所述各个像素点的深度值与预设深度范围值进行比较，以从所述手部图像中确定手指图像、前景和/或背景图像；
    图像去除子单元，用于去除所述前景和/或背景图像。
  23. 一种腕式设备，其特征在于，包括：
    摄像装置，用于沿佩戴者手腕向手心方向采集佩戴者手部图像；
    处理器，用于接收所采集的手部图像，并对所述手部图像进行处理。
  24. 根据权利要求23所述的腕式设备，其特征在于，所述处理器利用权利要求1、8-11中任一项所述的方法根据所述摄像装置采集的手部图像识别手势动作。
  25. 根据权利要求23所述的腕式设备，其特征在于，所述处理器利用权利要求2-11中任一项所述的方法根据所述摄像装置采集的手部图像进行手势动作控制。
  26. 根据权利要求23-25中任一项所述的腕式设备，其特征在于，所述腕式设备为智能手表，所述摄像装置设置在表带上，所述处理器设置在表盘处，所述摄像装置与所述处理器通过设置在表带内的连接部件连接。
PCT/CN2016/093226 2015-12-31 2016-08-04 手势动作识别方法、控制方法和装置以及腕式设备 WO2017113794A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201511030898.8 2015-12-31
CN201511030898.8A CN106933340B (zh) 2015-12-31 2015-12-31 手势动作识别方法、控制方法和装置以及腕式设备

Publications (1)

Publication Number Publication Date
WO2017113794A1 true WO2017113794A1 (zh) 2017-07-06

Family

ID=59224445

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/093226 WO2017113794A1 (zh) 2015-12-31 2016-08-04 手势动作识别方法、控制方法和装置以及腕式设备

Country Status (2)

Country Link
CN (1) CN106933340B (zh)
WO (1) WO2017113794A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491763A (zh) * 2017-08-24 2017-12-19 歌尔科技有限公司 基于深度图像的手指区域分割方法及装置
CN108985191A (zh) * 2018-06-28 2018-12-11 广东技术师范学院 一种基于移动设备手势识别的轮廓提取方法
CN111443831A (zh) * 2020-03-30 2020-07-24 北京嘉楠捷思信息技术有限公司 一种手势识别方法及装置
CN111949134A (zh) * 2020-08-28 2020-11-17 深圳Tcl数字技术有限公司 人机交互方法、设备及计算机可读存储介质
CN117788510A (zh) * 2024-02-02 2024-03-29 北京惠朗时代科技有限公司 一种用于图像数据读取的背景去除处理系统
CN117788510B (zh) * 2024-02-02 2024-05-31 北京惠朗时代科技有限公司 一种用于图像数据读取的背景去除处理系统

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107450672B (zh) * 2017-09-19 2024-03-29 曾泓程 一种高识别率的腕式智能装置
CN108748139A (zh) * 2018-04-18 2018-11-06 四川文理学院 基于体感式的机器人控制方法及装置
CN109902547B (zh) 2018-05-29 2020-04-28 华为技术有限公司 动作识别方法和装置
TWI683237B (zh) * 2018-07-19 2020-01-21 宏碁股份有限公司 使用仿生韌帶之手勢感應系統
TWI672611B (zh) * 2018-07-23 2019-09-21 宏碁股份有限公司 使用仿生韌帶之觸覺回饋系統
CN110858404B (zh) * 2018-08-22 2023-07-07 瑞芯微电子股份有限公司 一种基于区域偏移的识别方法及终端
CN111466882A (zh) * 2020-04-23 2020-07-31 上海祉云医疗科技有限公司 一种智能中医手诊分析系统及方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005301583A (ja) * 2004-04-09 2005-10-27 Nara Institute Of Science & Technology タイピング入力装置
CN202584010U (zh) * 2012-04-06 2012-12-05 寇传阳 一种腕上手势操控系统
CN102915111A (zh) * 2012-04-06 2013-02-06 寇传阳 一种腕上手势操控系统和方法
CN105027030A (zh) * 2012-11-01 2015-11-04 艾卡姆有限公司 用于三维成像、映射、建网和界面连接的无线腕式计算和控制设备和方法
CN105184268A (zh) * 2015-09-15 2015-12-23 北京国承万通信息科技有限公司 手势识别设备、手势识别方法及虚拟现实系统
CN205608659U (zh) * 2015-12-31 2016-09-28 北京体基科技有限公司 一种腕式设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005301583A (ja) * 2004-04-09 2005-10-27 Nara Institute Of Science & Technology タイピング入力装置
CN202584010U (zh) * 2012-04-06 2012-12-05 寇传阳 一种腕上手势操控系统
CN102915111A (zh) * 2012-04-06 2013-02-06 寇传阳 一种腕上手势操控系统和方法
CN105027030A (zh) * 2012-11-01 2015-11-04 艾卡姆有限公司 用于三维成像、映射、建网和界面连接的无线腕式计算和控制设备和方法
CN105184268A (zh) * 2015-09-15 2015-12-23 北京国承万通信息科技有限公司 手势识别设备、手势识别方法及虚拟现实系统
CN205608659U (zh) * 2015-12-31 2016-09-28 北京体基科技有限公司 一种腕式设备

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491763A (zh) * 2017-08-24 2017-12-19 歌尔科技有限公司 基于深度图像的手指区域分割方法及装置
CN108985191A (zh) * 2018-06-28 2018-12-11 广东技术师范学院 一种基于移动设备手势识别的轮廓提取方法
CN111443831A (zh) * 2020-03-30 2020-07-24 北京嘉楠捷思信息技术有限公司 一种手势识别方法及装置
CN111949134A (zh) * 2020-08-28 2020-11-17 深圳Tcl数字技术有限公司 人机交互方法、设备及计算机可读存储介质
CN117788510A (zh) * 2024-02-02 2024-03-29 北京惠朗时代科技有限公司 一种用于图像数据读取的背景去除处理系统
CN117788510B (zh) * 2024-02-02 2024-05-31 北京惠朗时代科技有限公司 一种用于图像数据读取的背景去除处理系统

Also Published As

Publication number Publication date
CN106933340A (zh) 2017-07-07
CN106933340B (zh) 2024-04-26

Similar Documents

Publication Publication Date Title
WO2017113794A1 (zh) 手势动作识别方法、控制方法和装置以及腕式设备
JP4711885B2 (ja) 遠隔操作装置及び方法
US9412001B2 (en) Method and computer-readable recording medium for recognizing object using captured image
TWI499966B (zh) 互動式操作方法
CN102402289B (zh) 一种基于机器视觉的手势鼠标识别方法
CN107491755B (zh) 用于手势识别的方法及装置
CN106547356B (zh) 智能交互方法和装置
WO2015149712A1 (zh) 一种指向交互方法、装置及系统
CN110045823A (zh) 一种基于动作捕捉的动作指导方法和装置
CN103995595A (zh) 一种基于手势的游戏体感控制方法
US20190384419A1 (en) Handheld controller, tracking method and system using the same
CN111367415B (zh) 一种设备的控制方法、装置、计算机设备和介质
CN105335711A (zh) 一种复杂环境下指尖检测方法
JP3657463B2 (ja) 動作認識システムおよび動作認識プログラムを記録した記録媒体
JP2007052609A (ja) 手領域検出装置及び手領域検出方法、並びにプログラム
CN106933341B (zh) 一种在图像中确定手指所处区域的方法、装置及腕式设备
Gu et al. Hand gesture interface based on improved adaptive hand area detection and contour signature
CN111860082A (zh) 信息处理方法、装置以及系统
CN205608659U (zh) 一种腕式设备
Qin et al. Real-time markerless hand gesture recognition with depth camera
JP5464661B2 (ja) 情報端末装置
KR102526951B1 (ko) 전자 장치에서 생체 정보 측정 방법 및 장치
WO2019244496A1 (ja) 情報処理装置、ウエアラブル機器、情報処理方法及びプログラム
JP7384157B2 (ja) 情報処理装置、ウエアラブル機器、情報処理方法及びプログラム
KR20140026898A (ko) 가상 키보드를 위한 시스템 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16880589

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15.10.18)

122 Ep: pct application non-entry in european phase

Ref document number: 16880589

Country of ref document: EP

Kind code of ref document: A1