WO2017113793A1

WO2017113793A1 - Method and apparatus for determining area of finger in image, and a wrist-type device

Info

Publication number: WO2017113793A1
Application number: PCT/CN2016/093225
Authority: WO
Inventors: 张霄
Original assignee: 北京体基科技有限公司
Priority date: 2015-12-31
Filing date: 2016-08-04
Publication date: 2017-07-06
Also published as: CN106933341B; CN106933341A

Abstract

A method and an apparatus for determining an area of a finger in an image, and a wrist-type device. The method comprises: obtaining a first image and a second image (S1), wherein the first image is an image taken in a direction towards the back of a hand along a wrist, and the second image is an image taken in a direction towards the palm of the hand along the wrist; determining a direction of the finger according to the first image (S2); and determining an area of the finger in the second image according to the direction of the finger (S3).

Description

Method, device and wrist device for determining an area where a finger is located in an image

cross reference

This application claims priority to Chinese Patent Application filed on Dec. 31, 2015, the Chinese Patent Office, Application No. 201511031085.0, entitled "A Method, Apparatus, and Wrist Device for Determining the Area Where a Finger Is Located in an Image" The entire content of which is incorporated herein by reference.

Technical field

The present invention relates to the field of smart wearable devices, and in particular, to a method, device and wrist device for determining an area where a finger is located in an image.

Background technique

With the rapid development of software and hardware related technologies, the integration of wrist smart devices, such as smart watches and smart bracelets, is getting higher and higher, and functions are becoming more and more abundant. A large proportion of mobile phone functions can pass smart watches and smart hands. Ring implementation greatly simplifies the way users receive and deliver information. However, compared with the traditional smart phone, the wrist smart device is limited to the small-size display screen. On the one hand, the user cannot use the touch screen or the button to complete the operation of the related function, which is easy to cause misoperation. When a smart watch is worn on one hand, it must be operated. Except for wake-up, sleep, etc., the simple operation does not require the other hand to operate. The other complicated operations are completed by the other hand and cannot be used. The smart watch is operated independently with one hand, and therefore, the smart watch still has great defects in content display and operation.

At present, some wrist devices have captured the action of the user's fingers through a camera disposed on the device, and the device is controlled according to the action of the finger. The camera of the device is usually disposed inside the wrist, and the finger image is taken toward the user's palm. . However, due to the complexity of the environment in which the user is located, it is susceptible to ambient light when identifying specific areas such as fingers or joints in the image. The interference of factors such as background is taken, and thus the accuracy of finding the area where the finger is located in the image is low.

Summary of the invention

Therefore, the technical problem to be solved by the present invention is that the accuracy of finding the area where the finger is located in the hand image is low in the prior art.

In view of this, the present invention provides a method for determining an area in which a finger is located in an image, comprising the steps of: acquiring a first image and a second image, wherein the first image is an image taken along a wrist toward a back of the hand. The second image is an image taken along the wrist toward the palm of the hand; determining a direction in which the finger is located according to the first image; determining an area in which the finger is located in the second image according to a direction in which the finger is located .

Preferably, the determining, according to the first image, a direction in which the finger is located comprises: identifying, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger and the little finger and the connecting joint of the back of the hand; The apex of each of the joints fits a straight line; a direction at a predetermined angle with the straight line is taken as a direction in which the finger is located.

Preferably, the identifying, in the first image, the apex of at least two of the index finger, the middle finger, the ring finger and the little finger and the connecting joint of the back of the hand comprises: removing the foreground and/or background image from the first image And identifying, in the first image after removing the foreground and/or background image, an outline of a joint joint of at least two of the index finger, the middle finger, the ring finger and the little finger and the back of the hand; and identifying a vertex of the joint according to the curvature of the contour.

Preferably, the removing the foreground and/or the background image from the first image comprises: performing color space conversion processing on the first image; performing binarization processing on the first image subjected to color space conversion processing Removing the foreground and/or background image in the first image after binarization.

Preferably, the removing the foreground and/or the background image from the first image comprises: acquiring a depth value of each pixel point in the first image; and setting a depth value of the each pixel point with a preset depth range The values are compared to determine a finger image, foreground and/or background image from the first image; the foreground and/or background image is removed.

Accordingly, the present invention also provides an apparatus for determining an area in which a finger is located in an image, comprising:

An acquiring unit, configured to acquire a first image and a second image, wherein the first image is an image taken along a wrist toward a back of the hand, and the second image is an image taken along a direction of the wrist toward a palm; the direction determining unit And determining, according to the first image, a direction in which the finger is located; and an area determining unit, configured to determine an area where the finger is located in the second image according to a direction in which the finger is located.

Preferably, the direction determining unit comprises: a vertex determining unit, configured to identify, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger and the little finger and the connecting joint of the back of the hand; the fitting unit, A straight line is fitted using the vertices of the respective joints; an angular direction determining unit for taking a direction at a predetermined angle with the straight line as a direction in which the finger is located.

Preferably, the vertex determining unit comprises: a background removing unit for removing foreground and/or background images from the first image; and a contour identifying unit for first after removing the foreground and/or background image An outline of a joint joint of at least two of the index finger, the middle finger, the ring finger and the little finger and the back of the hand is recognized in the image; a vertex recognition unit is configured to recognize the vertex of the joint according to the curvature of the contour.

Preferably, the background removing unit includes: a color space converting unit configured to perform color space conversion processing on the first image; and a binarization processing unit configured to perform second color image processing on the first image a value processing process; a background processing unit, configured to remove the foreground and/or background image in the first image after the binarization process.

Preferably, the background removing unit includes: a depth value acquiring unit, configured to acquire a depth value of each pixel point in the first image; and an image determining unit, configured to use the depth value of the each pixel point and a preset depth The range values are compared to determine a finger image, foreground and/or background image from the first image; an image removal unit for removing the foreground and/or background image.

The present invention also provides a wrist device, comprising: a wristband; a first camera device disposed on the wristband; and a second camera device disposed on the wristband and opposite to the first camera device And the second camera lens is directed in the same direction as the first camera device; and the processor is configured to process the hand images collected by the first camera device and the second camera device.

Preferably, the first imaging device is configured to capture a first image along the wrist toward the back of the hand, the second imaging device is configured to capture a second image along the wrist toward the palm of the hand, and the processor determines the finger by using the above method. The area in which the second image is located.

Preferably, the wrist device is a smart watch, the processor is disposed in a dial, and the first camera device and the second camera device are respectively disposed on the dial and the strap, and the camera is disposed on the strap. The device is coupled to the processor by a connecting member disposed within the strap.

Preferably, the connecting member is a flexible circuit board.

The present invention also provides another wrist device, comprising: a wristband; a first camera device disposed on the wristband; and a second camera device disposed on the wristband and opposite to the first camera device And the second camera lens is directed in the same direction as the first camera device; the first processor is configured to process the hand image collected by the first camera device; the second processor is configured to use Processing the hand image acquired by the second camera according to the processing result of the first processor.

Preferably, the first imaging device is configured to capture a first image along a wrist to the back of the hand; the second imaging device is configured to capture a second image along a direction of the wrist to the palm; the first processor is configured to The first image determines a direction in which the finger is located; the second processor is configured to determine an area in which the finger is located in the second image according to a direction in which the finger is located.

Preferably, the wrist device is a smart watch, the first processor and the second processor are respectively disposed at a dial and a watchband, and the first camera device and the second camera device are respectively disposed at the dial and On the strap, the processor disposed on the strap and the processor disposed in the dial are connected by a connecting member disposed in the strap.

Preferably, the connecting member is a flexible circuit board.

The method and device for determining the area where the finger is located in the image provided by the present invention can obtain the direction in which the finger is located by analyzing the image taken along the wrist to the back of the hand, and then according to the direction, the direction along the wrist to the palm of the hand. In the image, the area where the finger is located can be determined, and then the activity of the finger can be judged in the area, and finally various control operations can be realized according to the activity of the specific part of the finger. By using the scheme to identify a specific part of the finger, it is not necessary to find a specific part in the entire image, and it is only necessary to find a specific part of the finger in the determined area, thereby improving the scheme. Finding the efficiency of the area in which the finger is located in the image improves the efficiency of the finger activity recognition operation.

The wrist device provided by the invention can use the two camera devices to respectively collect the image of the wearer's hand along the wrist of the wearer toward the palm and the back of the hand, and the captured image can display the image of the user's hand, and then the processor can be opponent. The image of the part is subjected to analysis and judgment processing, and the gesture image of the user is recognized by the hand image, thereby enabling control of the device.

DRAWINGS

In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings to be used in the specific embodiments or the description of the prior art will be briefly described below, and obviously, the attached in the following description The drawings are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative work.

1 is a schematic structural diagram of a wrist device according to Embodiment 1 of the present invention;

2 is a schematic diagram of a hand image collected by an imaging device of a wrist device according to an embodiment of the present invention;

3 is a schematic diagram of a hand image collected by another camera device of a wrist device according to an embodiment of the present invention;

4 is a schematic structural diagram of a wrist device according to Embodiment 2 of the present invention;

FIG. 5 is a flowchart of a method for determining an area where a finger is located in an image according to an embodiment of the present invention;

6 is a schematic diagram of a direction in which a finger is determined by a method for determining an area where a finger is located in an image according to an embodiment of the present invention;

7 is a schematic diagram of an area where a finger is determined by a method for determining an area where a finger is located in an image according to an embodiment of the present invention;

8 is a schematic diagram of a specific scene for determining the direction in which the finger is located;

FIG. 9 is a schematic structural diagram of an apparatus for determining an area where a finger is located in an image according to an embodiment of the present invention.

detailed description

The technical solutions of the present invention will be clearly and completely described in the following with reference to the accompanying drawings. It is obvious that the described embodiments are a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In the description of the present invention, it is to be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. The orientation or positional relationship of the indications is based on the orientation or positional relationship shown in the drawings, and is merely for the convenience of the description of the invention and the simplified description, rather than indicating or implying that the device or component referred to has a specific orientation, in a specific orientation. The construction and operation are therefore not to be construed as limiting the invention. Moreover, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

In the description of the present invention, it should be noted that the terms "installation", "connected", and "connected" are to be understood broadly, and may be fixed or detachable, for example, unless otherwise explicitly defined and defined. Connection, or integral connection; may be mechanical connection or electrical connection; may be directly connected, may also be indirectly connected through an intermediate medium, or may be internal communication of two components, may be wireless connection, or may be wired connection. The specific meaning of the above terms in the present invention can be understood in a specific case by those skilled in the art.

Further, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not constitute a conflict with each other.

Example 1

The embodiment of the present invention provides a wrist device. As shown in FIG. 1 , the device includes: a wristband 10, a first camera device 11, and a second camera device 12; wherein, the first camera device 11 and the second camera device 12 are both Provided on the wristband 10, the first camera 11 and the second camera 12 are disposed opposite to each other, and the directions in which they are directed are the same.

After the user wears, the first imaging device 11 can capture the hand image along the wrist of the wearer in the direction of the back of the hand, and the second imaging device 12 can capture the hand image in the direction of the palm of the hand along the wrist of the wearer. For different types of wrist devices, the camera is set differently, but it is at the corner of the wearer's arm. Degree is relatively fixed. The first camera 11 and the second camera 12 disposed in this manner can respectively acquire hand images as shown in FIGS. 2 and 3.

The processor 13 is connected to the first imaging device 11 and the second imaging device 12 for processing the hand images collected by the first imaging device 11 and the second imaging device 12. The processor 13 can perform various processing on the image, such as recognizing a hand motion in the image, controlling the device according to the hand motion reflected by the image, etc., which will be described in detail in the following embodiments.

The above device can use the two camera devices to collect the image of the wearer's hand along the wrist of the wearer toward the palm and the back of the hand, and the captured image can display the image of the user's hand, and then the processor 13 can analyze the image of the hand. By judging and the like, the gesture image of the user is recognized by the hand image, and the control of the device can be realized.

The wrist device may be a smart watch. When the user wears the watch, the dial is usually located outside the wrist, and the strap surrounds the wrist. Therefore, the first camera 11 can be disposed at the dial, and the second camera 12 can be disposed on the strap. The first camera device 11 is disposed here toward the hand, and its angle and direction are just enough to capture the back of the hand; the second camera device 12 is disposed on the strap (for example, near the strap buckle or on the strap buckle) toward the hand, The angle and direction just capture the palm and fingers. This structure does not require the user to adjust the position of the camera to make it easier for the user to wear. Moreover, it is also possible to exchange the positions of the above two cameras. The processor 13 is disposed as a processing core of the smart watch, and is disposed at the dial. The connecting component of the second camera 12 and the processor 13 is disposed in the watchband, and the connecting component may be a flexible circuit board.

Example 2

The embodiment of the present invention provides a wrist device. As shown in FIG. 4, the device includes: a wristband 20, a first camera device 21, and a second camera device 22; wherein the first camera device 21 and the second camera device 22 are both Provided on the wristband 20, the first camera 21 and the second camera 22 are disposed opposite to each other and the directions in which they are directed are the same.

The first camera 21 and the second camera 22, which are disposed in this manner, can be collected into the hand images as shown in FIGS. 2 and 3, respectively.

The first processor 23 is connected to the first camera 21 for processing the hand image collected by the first camera 21;

The second processor 24 is connected to the second imaging device 22 for processing the hand image acquired by the second imaging device 22 according to the processing result of the first processor 23.

The first processor 23 and the second processor 24 can perform various processing on the respective received images, for example, identifying a hand motion in the image, controlling the device according to the hand motion reflected by the image, etc., specifically This will be described in detail in the subsequent embodiments.

The above device can use the two camera devices to collect the image of the wearer's hand along the wrist of the wearer in the direction of the palm and the back of the hand, and the captured image can display the image of the user's hand, and then the two processors can respectively image the hand. The processing such as analysis and judgment is performed, and the gesture motion of the user is recognized by the hand image, and the control of the device can be realized.

The wrist device may be a smart watch, and the first camera device 21 and the second camera device 22 are respectively disposed at the dial and the watchband, and the corresponding processor may be disposed nearby, for example, the first camera device 21 and the first process may be The device 23 is disposed at the dial, and the second camera 22 and the second processor 24 are disposed on the strap at a position close to the strap buckle, and the processor disposed on the strap and the processor disposed in the dial are disposed on the table. The connecting members in the belt are connected, and the connecting member is preferably a flexible circuit board. In the example shown in FIG. 4, the first processor 23 and the second processor 24 are separately provided, but are not limited thereto, and the two processors may be set together.

Example 3

An embodiment of the present invention provides a method for determining an area where a finger is located in an image, and the method may be performed by the processor 13 in Embodiment 1 or the first processor 23 and the second processor 24 in Embodiment 2, As shown in FIG. 5, the method includes the following steps:

S1. Acquire a first image and a second image, wherein the first image is, for example, an image as shown in FIG. 2, and the second image is, for example, an image as shown in FIG.

S2, determining the direction in which the finger is located according to the first image. It should be noted that, in the normal state, the palm of the wearer is naturally bent inward (semi-claw shape), so the first camera 11 generally does not collect the finger image. In this step, only one direction is estimated based on the image shown in Figure 2. information.

Specifically, an image of the back of the wearer's hand is displayed in the image shown in FIG. 2, and characteristic information such as the outline of the back of the hand, the tilt angle of the image content, and the tilt state of the hand joint can be obtained by analyzing and processing the image. Then, the orientation of the wearer's finger can be estimated based on the above feature information. It will be understood by those skilled in the art that the direction in which the finger is located is not necessarily perpendicular to the lower edge of the image. Due to factors such as the wearing state of the user and the state of the hand joint, as indicated by the arrow in FIG. 6, the direction is usually The lower edge of the image forms a certain angle.

S3, determining an area where the finger is located in the second image according to the direction in which the finger is located, wherein the area where the finger is located is the area where the fingertip and the finger joint are located. Normally, the second imaging device 12 can collect an image containing a finger, so the position of the fingertip can be found in the image shown in FIG. 3 according to the direction estimated in step S2. Specifically, the area can be determined in the image according to a preset length and the above direction. Further, by setting the preset distance, more specific regions such as fingertips and joints can be determined. Taking the fingertip area as an example, according to the above method, an area where the fingertip may be present as shown in FIG. 7 may be determined; or the image may be divided into a plurality of segments according to a preset scale value and the above direction information, and then a segment is determined. Or multiple sections.

For the area other than the area where the finger is located, it may be directly ignored or given a lower weight. When the finger motion is recognized, the specific part may be preferentially found in the above determined area, thereby reducing the calculation amount of the recognition operation. .

According to the above scheme, by analyzing the image taken along the wrist to the back of the hand, the direction in which the finger is located can be obtained, and then according to the direction, the image of the finger can be determined in the image taken along the wrist to the palm, and then the In this area, the activity of the finger is judged, and finally various control operations are performed according to the activity of the specific part of the finger. By using the scheme to identify a specific part of the finger, it is not necessary to find a specific part in the entire image, and it is only necessary to find a specific part of the finger in the determined area, thereby improving the area where the finger is located in the image. Efficiency, which improves the efficiency of finger activity recognition operations.

As a preferred implementation manner, the foregoing step S2 may include:

S21, identifying, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger, and the little finger and the connecting joint of the back of the hand, which may be specifically identified according to the shape of the hand in the image. There are many ways to find a part with obvious features from an image, and it is feasible to use existing feature recognition algorithms. Thereby, the vertices of the four joints of the index finger, the middle finger, the ring finger and the little finger as shown in FIG. 8 can be identified, and in this embodiment, it is preferable to use four joint vertices for subsequent processing, and the result obtained thereby is the most accurate. However, the invention is not limited to the use of 4 vertices, and fitting a straight line with 2 or 3 vertices is possible.

S22, the straight line is fitted by the vertices of each joint, and there are various algorithms for fitting the straight line by using points, and the existing algorithms are all feasible;

S23, a direction at a predetermined angle with the straight line is taken as a direction in which the finger is located. Since the finger is necessarily in the downward direction of the apex of the joint, it can be determined that the direction downward with the straight line is the direction in which the finger is located, wherein the preset angle can be 90 degrees. The above preferred solution determines the direction of the finger based on the position of the joint, which has a high accuracy.

As described above, there are a plurality of ways to find a portion having a distinct feature from the image. As a preferred embodiment, the above step S21 may include:

S211, removing the foreground and/or background image from the first image. There are various methods for determining the foreground and background images from the image. Since the skin color data of the human body has a certain range of values, in the image, the RGB values of the pixel points of the hand region should be within a certain range, thereby The content in the image can be judged according to the preset RGB value range, and the target image and the background image can be selected; the content in the image can be judged and removed according to the sharpness value or the depth value of the image, and the existing background is Removal methods are all feasible.

S212. Identify, in the first image after the foreground and/or the background image is removed, an outline of a joint joint of at least two of the index finger, the middle finger, the ring finger, and the little finger and the back of the hand, and only the hand image after the background image is removed. There is a skin area that can be considered as a hand area. In order to identify the finger portion, it is necessary to discriminate based on the morphological characteristics of the finger. Thus, the Canny operator can be used to extract the edge contour of the hand region. The Canny operator measures the signal-to-noise ratio and the positioning product, and approximates it by the optimization method to obtain the edge information.

For the specific address, Gaussian filtering of the image is first required to smooth the image noise to reduce the influence of noise on the detection result. The Gaussian kernel function is as follows:

Then calculate the gradient of the gray value of the image, that is, make the first order difference in both directions. Calculate the gradient magnitude and direction of each pixel.

f' _x (x,y)≈G _x =[f(x+1,y)-f(x,y)+f(x+1,y+1)-f(x,y+1)]/ 2

f' _y (x,y)≈G _y =[f(x,y+1)-f(x,y)+f(x+1,y+1)-f(x+1,y)]/ 2

The corresponding strength and direction are:

θ[x,y]=arctan(G _x (x,y)/Gy(x,y));

After obtaining the gradient magnitude and direction of each point in the entire image, the local maximum is calculated and the corresponding pixel points are retained. Finally, the pixel points that should be reserved are calculated according to the double threshold, and the boundary tracking is performed for the remaining pixel points to complete the edge extraction.

S213, the apex of the joint is recognized according to the curvature of the contour, and after the back edge of the hand is obtained, the joint convex portion can be extracted by the shape of the back of the hand. By analyzing the shape of the convex part of the joint, it can be known that the convex edge of the joint has a sudden change in curvature, that is, the line on both sides of the joint convex is relatively uniform, and the convex line is curved to a greater extent and is close to a 180 degree turn.

Based on the above characteristics, the edges of the image are first sampled and the image edge lines are vectorized to form a feature line having length and trend statistics. In vectorization, the direction is obtained according to the position between the pixels and the first-order difference. Then, the point multiplication results of these vectors are calculated to obtain the angle between the vector lines. Then, find all straight straight segments for all edges (for example, the average angle is no more than 25 degrees). The straight line segments are arranged in order, and the change of the trend of the curved segments between the straight segments is calculated. For example, if the change of the strike is greater than 140 degrees and the distance is greater than a certain threshold, the corresponding joint is determined. The corresponding noise and the repeated result are removed, and it is determined as a convex portion of the joint.

As a preferred implementation manner, the foregoing step S211 may include:

S2111a, performing color space conversion processing on the first image; the human skin is composed of a dermis layer and a thin skin layer covering the skin layer, and light is absorbed by melanin in the epidermis layer, and absorption and scattering occur simultaneously in the dermis layer. The difference in skin color of different individuals is mainly manifested by the black layer in the epidermis The change in brightness caused by the difference in the concentration of the pigment, the optical properties of the dermis layer are basically the same, and the individual skin color of the same race has strong commonality, and is clearly different from most background colors, forming a small and compact in the color space. Clustering. Thus, skin based detection based on color is feasible.

To perform skin color detection, you need to select the appropriate color space in which skin tones can be clustered, aggregated, and overlapped with non-skinned skin as little as possible. The image captured by the camera is an RGB image. In the RGB color space, the overlap between the skin color and the non-skin tone is more, and it is seriously affected by the brightness; in the HSV color space, due to the good separation of hue, color saturation and brightness, There is less overlap with non-skinning points; in the CbCr subspace in the YCbCr color space, the skin color is well concentrated in an ellipse-like range, and the distribution on the Cb and Cr components is also concentrated. Therefore, it is feasible to convert the hand image from the RGB space to the YCbCr color space or the HSV color space.

The conversion formula for RGB to HSV is:

The conversion formula for RGB to YCbCr is:

Y=0.257R'+0.504G'+0.098B'+16

Cb=0.148R'-0.291G'+0.439B'+128

Cr = 0.439 R' - 0.368 G' - 0.071 B' + 128.

S2112a, performing binarization processing on the first image subjected to color space conversion processing, and converting the image into a line drawing having only two colors of black and white;

S2113a, removing foreground and/or background images in the first image after binarization. The above preferred solution can further improve the accuracy of the recognition operation.

As another preferred implementation manner, the foregoing step S211 may include:

S2111b, obtaining a depth value of each pixel in the first image

S2112b, comparing the depth value of each pixel point with a preset depth range value to determine a finger image, a foreground, and/or a background image from the first image. Due to the particularity of the wrist device, the part of the imaging device to be imaged is about 10-15 cm, so the focus point of the camera device can be fixed, only need to ensure the focus within 10-15 cm is clear; at the same time, when lifting When the wrist is operated, other objects (foreground and background) in the imaging range are usually closer or farther from the hand, not within the range of 10-15 cm, so the foreground and background are out of focus, and the algorithm can be easily distinguished by the algorithm of ambiguity. background.

S212b, the foreground and/or background image is removed, wherein the content that is too close to the first camera 11 is the foreground image, and the content that is too far from the first camera 11 is the background image.

The above preferred solution removes both the foreground and background images based on the depth of field information, retaining only the front of the back of the hand, and then further identifying the joints in the scene, thereby further improving the recognition efficiency.

Example 4

An embodiment of the present invention provides a device for determining an area where a finger is located in an image, as shown in FIG.

The acquiring unit 91 is configured to acquire the first image and the second image, wherein the first image is an image taken along the wrist toward the back of the hand, and the second image is an image taken along the wrist toward the palm direction;

a direction determining unit 92, configured to determine a direction in which the finger is located according to the first image;

The area determining unit 93 is configured to determine an area where the finger is located in the second image according to a direction in which the finger is located.

According to the above scheme, by analyzing the image taken along the wrist to the back of the hand, the direction in which the finger is located can be obtained, and then according to the direction, the image of the finger can be determined in the image taken along the wrist to the palm, and then the In this area, the activity of a specific part of the finger is judged, and finally various control operations are performed according to the activity of the specific part of the finger. By using the scheme to identify a specific part of the finger, it is not necessary to find a specific part in the entire image, and it is only necessary to find a specific part of the finger in the determined area, thereby improving the area where the finger is located in the image. Effect Rate, which improves the efficiency of finger activity recognition operations.

Preferably, the direction determining unit 92 includes:

a vertex determining unit, configured to identify, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger, and the little finger and the connecting joint of the back of the hand;

a fitting unit for fitting a straight line with vertices of each of the joints;

An angle direction determining unit for taking a direction at a predetermined angle with the straight line as a direction in which the finger is located.

The above preferred solution determines the direction of the finger based on the position of the joint, which has a high accuracy.

Preferably, the vertex determining unit comprises:

a background removing unit, configured to remove a background image from the first image;

a contour identifying unit, configured to identify an outline of a joint joint of each finger and the back of the hand in the first image after the background image is removed;

A vertex recognition unit for identifying a vertex of the joint according to the curvature of the contour.

Preferably, the background removal unit comprises:

a color space conversion unit, configured to perform color space conversion processing on the first image;

a binarization processing unit, configured to perform binarization processing on the first image subjected to color space conversion processing;

And a background processing unit, configured to remove the background image in the first image after the binarization process.

The above preferred solution can further improve the accuracy of the recognition operation.

Preferably, the vertex determining unit comprises:

An image determining unit, configured to determine, according to a depth value of each pixel point in the first image and a preset depth range value, an image, a foreground, and/or a background image of a joint joint of each finger and the back of the hand from the first image ;

An image removing unit, configured to remove the foreground and/or background image;

a contour recognition unit, configured to identify an outline of a joint joint of each finger and the back of the hand in the first image after the foreground and/or the background image is removed;

The above preferred solution removes both the foreground and background images according to the depth of field information, leaving only the back of the hand The front scene can then further identify the joints in the scene, thereby further improving the recognition efficiency.

It is apparent that the above-described embodiments are merely illustrative of the examples, and are not intended to limit the embodiments. Other variations or modifications of the various forms may be made by those skilled in the art in light of the above description. There is no need and no way to exhaust all of the implementations. Obvious changes or variations resulting therefrom are still within the scope of the invention.

Claims

A method for determining an area in which a finger is located in an image, comprising the steps of:

Acquiring the first image and the second image, wherein the first image is an image taken along the wrist toward the back of the hand, and the second image is an image taken along the wrist toward the palm of the hand;

Determining a direction in which the finger is located according to the first image;

The area in which the finger is located in the second image is determined according to the direction in which the finger is located.
The method according to claim 1, wherein the determining the direction in which the finger is located according to the first image comprises:

Identifying, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger, and the little finger and the connecting joint of the back of the hand;

Fitting a straight line with the vertices of each of the joints;

A direction at a predetermined angle with the straight line is taken as a direction in which the finger is located.
The method according to claim 2, wherein the identifying, in the first image, the apex of at least two of the index finger, the middle finger, the ring finger and the little finger and the connecting joint of the back of the hand comprises:

Removing foreground and/or background images from the first image;

Identifying an outline of a joint joint of at least two of the index finger, the middle finger, the ring finger, and the little finger with the back of the hand in the first image after the foreground and/or background image is removed;

The vertices of the joint are identified based on the curvature of the contour.
The method of claim 3 wherein said removing foreground and/or background images from said first image comprises:

Performing a color space conversion process on the first image;

Performing binarization processing on the first image subjected to color space conversion processing;

The foreground and/or background image is removed in the first image after binarization.
The method of claim 3 wherein said removing foreground and/or background images from said first image comprises:

Obtaining a depth value of each pixel in the first image;

Comparing the depth values of the respective pixel points with a preset depth range value to determine a finger image, a foreground, and/or a background image from the first image;

The foreground and/or background image is removed.
A device for determining an area in which a finger is located in an image, comprising:

An acquiring unit, configured to acquire a first image and a second image, wherein the first image is an image taken along a wrist toward a back of the hand, and the second image is an image taken along a direction of the wrist toward a palm;

a direction determining unit, configured to determine a direction in which the finger is located according to the first image;

And an area determining unit, configured to determine an area where the finger is located in the second image according to a direction in which the finger is located.
The apparatus according to claim 6, wherein the direction determining unit comprises:

a vertex determining unit, configured to identify, in the first image, an apex of at least two of the index finger, the middle finger, the ring finger, and the little finger and the connecting joint of the back of the hand;

a fitting unit for fitting a straight line with vertices of each of the joints;

An angle direction determining unit for taking a direction at a predetermined angle with the straight line as a direction in which the finger is located.
The apparatus according to claim 7, wherein said vertex determining unit comprises:

a background removing unit, configured to remove a foreground and/or a background image from the first image;

a contour identifying unit, configured to identify an outline of a joint joint of at least two of the index finger, the middle finger, the ring finger and the little finger and the back of the hand in the first image after the foreground and/or background image is removed;

A vertex recognition unit for identifying a vertex of the joint according to the curvature of the contour.
The apparatus according to claim 8, wherein the background removal unit comprises:

a color space conversion unit, configured to perform color space conversion processing on the first image;

a binarization processing unit, configured to perform binarization processing on the first image subjected to color space conversion processing;

a background processing unit, configured to remove foreground and/or in the first image after binarization Background image.
The apparatus according to claim 8, wherein the background removal unit comprises:

a depth value obtaining unit, configured to acquire a depth value of each pixel in the first image;

An image determining unit, configured to compare a depth value of each pixel point with a preset depth range value to determine a finger image, a foreground, and/or a background image from the first image;

An image removal unit for removing the foreground and/or background image.
A wrist device, comprising:

Wristband

a first camera device disposed on the wristband;

a second camera device disposed on the wristband and opposite to the first camera device, and the lens of the second camera device is pointed in the same direction as the first camera device;

And a processor, configured to process the hand images collected by the first camera device and the second camera device.
The wrist device according to claim 11, wherein the first imaging device is configured to capture a first image along the wrist toward the back of the hand, and the second imaging device is configured to shoot along the wrist toward the palm A second image, the processor determining the region in which the finger is located in the second image, using the method of any of claims 1-5.
The wrist device according to claim 11 or 12, wherein the wrist device is a smart watch, the processor is disposed in a dial, and the first camera device and the second camera device are respectively disposed at At the dial and on the strap, the camera mounted on the strap is coupled to the processor via a connecting member disposed within the strap.
The wrist device according to claim 13, wherein the connecting member is a flexible circuit board.
A wrist device, comprising:

Wristband

a first camera device disposed on the wristband;

a second camera device disposed on the wristband and opposite to the first camera device, and the lens of the second camera device is pointed in the same direction as the first camera device;

a first processor, configured to process a hand image collected by the first camera device;

And a second processor, configured to process the hand image collected by the second camera according to the processing result of the first processor.
A wrist device according to claim 15, wherein

The first imaging device is configured to capture a first image along the wrist to the back of the hand;

The second imaging device is configured to capture a second image along the wrist toward the palm of the hand;

The first processor is configured to determine a direction in which the finger is located according to the first image;

The second processor is configured to determine an area in which the finger is located in the second image according to a direction in which the finger is located.
The wrist device according to claim 15 or 16, wherein the wrist device is a smart watch, and the first processor and the second processor are respectively disposed at a dial and a watchband, wherein the A camera device and a second camera device are respectively disposed at the dial and the watch band, and the processor disposed on the watch band and the processor disposed in the dial are connected by a connecting member disposed in the watch band.
The wrist device according to claim 17, wherein the connecting member is a flexible circuit board.