WO2023273227A1 - 指甲识别方法、装置、设备及存储介质 - Google Patents

指甲识别方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023273227A1
WO2023273227A1 PCT/CN2021/140473 CN2021140473W WO2023273227A1 WO 2023273227 A1 WO2023273227 A1 WO 2023273227A1 CN 2021140473 W CN2021140473 W CN 2021140473W WO 2023273227 A1 WO2023273227 A1 WO 2023273227A1
Authority
WO
WIPO (PCT)
Prior art keywords
nail
image
key point
detection
frame
Prior art date
Application number
PCT/CN2021/140473
Other languages
English (en)
French (fr)
Inventor
刘昕
谢符宝
刘文韬
Original Assignee
北京市商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京市商汤科技开发有限公司 filed Critical 北京市商汤科技开发有限公司
Publication of WO2023273227A1 publication Critical patent/WO2023273227A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present application relates to the technical field of image recognition, and in particular to a nail recognition method, device, equipment and storage medium.
  • Nail recognition has important application prospects in mobile entertainment, virtual fitting, virtual reality VR, augmented reality AR and other fields.
  • segmentation is usually used to obtain the nail region in the image or video.
  • this method cannot obtain the semantic information of the nail outline, which limits the use of the recognition results in various application scenarios.
  • An embodiment of the present disclosure provides a nail recognition solution.
  • a nail recognition method comprising: acquiring a detection result of at least one nail in the first image, the detection result including a first nail detection frame and a classification result of the nail, wherein The classification result indicates the finger type to which the nail belongs; the image area corresponding to the nail in the first image is obtained according to the first nail detection frame; and the image corresponding to the nail is obtained according to the finger type to which the nail belongs Multiple first keypoints for the nail in the region.
  • the category information of the nails and the complete semantic features of the nail outline can be obtained, which facilitates the application of the nail recognition results in various scenarios.
  • the obtaining multiple first key points of the nail in the image area corresponding to the nail according to the type of finger to which the nail belongs includes: from the first image Cut out the image area corresponding to the nail; input the cropped image area into the first key point detection network corresponding to the finger type to which the nail belongs, and obtain multiple first key points of the nail.
  • the method further includes: acquiring a binary classification result of each pixel in the image region corresponding to the nail, the binary classification result indicating that the pixel is a foreground pixel or a background pixel; A pixel indicated as a background pixel in the above binary classification result is set as the first pixel value.
  • the method further includes: according to the position information of at least two first key points of the multiple first key points of the nail in the image area, determining the direction.
  • the method further includes: acquiring a sample image; wherein, the sample image has annotation information, and the annotation information indicates a first key point corresponding to the finger type to which the sample image belongs;
  • the sample image is input to the first key point detection network to obtain a key point detection result; according to the difference between the key point detection result and the label information, the network of the first key point detection network Parameters are adjusted.
  • the first key point detection network can realize the recognition of the nail of the specified finger type.
  • the first image is a frame in an image sequence
  • the method further includes: for a second image after the first image, according to a previous frame of the second image A plurality of first key points of the nail in the frame, determine a second nail detection frame in the second image; obtain in the image area corresponding to the second nail detection frame in the second image, the nail's Multiple second keys.
  • the amount of data processing can be reduced, and the speed and efficiency of nail key point detection can be improved.
  • the determining the second nail detection frame in the second image according to the multiple first key points in the previous frame of the second image includes: according to the previous frame A plurality of first key points of the nail in the frame to obtain a circumscribing rectangle of the nail; according to the position information of the circumscribing rectangle in the previous frame, map the circumscribing rectangle to the In the second image, as the second nail detection frame in the second image.
  • the second nail detection frame in the second image obtained by the above method is closer to the real area of the nail and contains less parts other than the nail, which is beneficial to improve the key Accuracy of point detection.
  • the obtaining multiple second key points of the nail in the image area corresponding to the second nail detection frame in the second image includes: cutting out the The image area corresponding to the second nail detection frame in the second image; the cropped image area is input to the second key point detection network to obtain the second key point of the nail.
  • the second key point detection network can detect nail key points based on regression key points. Compared with the first key point detection network based on heat map for nail key point detection, the network structure is simpler, the number of layers is smaller, and the processing speed is faster. , reducing the time-consuming for nail key point detection.
  • the cropped image is Rotation processing.
  • the nail key point detection on the rotated image can improve the efficiency of detection on the one hand, and improve the accuracy of detection on the other hand.
  • the method further includes: acquiring a second image when the second key point of the nail is not detected or the second key point of the nail does not meet the set requirements
  • a detection result of at least one nail the detection result includes a first nail detection frame and a classification result of the nail, the classification result indicates the finger type to which the nail belongs;
  • the first nail detection frame is obtained according to the first nail detection frame
  • An image area corresponding to the nail in the image; according to the finger type to which the nail belongs, a plurality of first key points of the nail in the image area corresponding to the nail are obtained.
  • the nail recognition By judging the second key point of the nail tracked, if it is not detected or does not meet the set requirements, the nail recognition will be performed again. On the one hand, it ensures the consistency of the nail recognition results, and on the other hand, it also ensures The accuracy of nail recognition results.
  • a nail recognition device including: a first acquisition unit, configured to acquire a detection result of at least one nail in the first image, the detection result including the first nail detection frame and the nail's A classification result, the classification result indicating the finger type to which the nail belongs; a second acquisition unit, configured to obtain an image area corresponding to the nail in the first image according to the first nail detection frame; an identification unit, configured to According to the finger type to which the nail belongs, a plurality of first key points of the nail in the image area corresponding to the nail are obtained.
  • the identification unit is specifically configured to: crop out the image area corresponding to the nail from the first image; input the cropped image area into the In the first key point detection network of , multiple first key points of the nail are obtained.
  • the device further includes a filtering unit, configured to: obtain a binary classification result of each pixel in the image region corresponding to the nail, the binary classification result indicating that the pixel is a foreground pixel or Background pixels: setting the pixels indicated as background pixels in the binary classification result as the first pixel value.
  • a filtering unit configured to: obtain a binary classification result of each pixel in the image region corresponding to the nail, the binary classification result indicating that the pixel is a foreground pixel or Background pixels: setting the pixels indicated as background pixels in the binary classification result as the first pixel value.
  • the device further includes an orientation unit configured to: according to the position information of at least two first key points among the plurality of first key points of the nail in the image area , to determine the orientation of the nail.
  • the device further includes a training unit configured to: acquire a sample image; wherein, the sample image has annotation information, and the annotation information indicates the finger type corresponding to the sample image.
  • the first key point input the sample image to the first key point detection network to obtain the key point detection result; according to the difference between the key point detection result and the label information, the first key point
  • the network parameters of the point detection network are adjusted.
  • the first image is a frame in an image sequence
  • the device further includes a tracking unit configured to: for a second image following the first image, according to the first image A plurality of first key points of the nail in the previous frame of the second image, determine a second nail detection frame in the second image; obtain in the image area corresponding to the second nail detection frame in the second image , a plurality of second key points of the nail.
  • the tracking unit when used to determine the second nail detection frame in the second image according to the multiple first key points in the previous frame of the second image, specifically It is used to: obtain the circumscribed rectangular frame of the nail according to the multiple first key points of the nail in the previous frame; and obtain the circumscribed rectangular frame according to the position information of the circumscribed rectangular frame in the previous frame.
  • the circumscribed rectangular frame is mapped to the second image as a second nail detection frame in the second image.
  • the tracking unit when used to obtain multiple second key points of the nail in the image region corresponding to the second nail detection frame in the second image, It is specifically used for: cutting out the image area corresponding to the second nail detection frame in the second image; inputting the cropped image area into the second key point detection network to obtain the second key point of the nail.
  • the device further includes a rotation unit, configured to, before inputting the cropped image area into the second key point detection network, according to the nail in the previous frame direction, and rotate the cropped image.
  • the device further includes a judging unit configured to: if the second key point of the nail is not detected or the second key point of the nail does not meet the set requirements , acquiring a detection result of at least one nail in the second image, the detection result including a first nail detection frame and a classification result of the nail, the classification result indicating the finger type to which the nail belongs; according to the first nail detection Obtain an image area corresponding to the nail in the second image; and obtain a plurality of first key points of the nail in the image area corresponding to the nail according to the finger type to which the nail belongs.
  • an electronic device the device includes a memory and a processor, the memory is used to store computer instructions executable on the processor, and the processor is used to execute the computer instructions Implement the nail recognition method described in any implementation manner provided by the present disclosure.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the nail recognition method described in any implementation manner provided by the present disclosure is implemented.
  • a computer program product including a computer program, and when the program is executed by a processor, the nail recognition method described in any implementation manner provided in the present disclosure is implemented.
  • FIG. 1 is a flow chart of a nail recognition method proposed by at least one embodiment of the present disclosure
  • Fig. 2 is a schematic diagram of the first key point of the nail in the nail recognition method proposed by at least one embodiment of the present disclosure
  • Fig. 3 is a flowchart of another nail recognition method proposed by at least one embodiment of the present disclosure.
  • Fig. 4 is a schematic structural diagram of a nail recognition device proposed by at least one embodiment of the present disclosure
  • Fig. 5 is a schematic structural diagram of an electronic device proposed by at least one embodiment of the present disclosure.
  • first, second, third, etc. may be used in this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be called second information, and similarly, second information may also be called first information.
  • first information may also be called second information, and similarly, second information may also be called first information.
  • word “if” as used herein may be interpreted as “at” or "when” or "in response to a determination”.
  • the recognition of nails in the image still stays at the recognition of the nail area.
  • the most commonly used method is to segment the model, that is, to detect each pixel in the image and combine the pixels belonging to the nail as Test results for nails.
  • this nail detection method can identify the area belonging to the nail in the image, which finger, which hand the nail belongs to, and the orientation of the nail cannot be determined by the above-mentioned nail recognition method, which greatly Limits the use of recognition results in various application scenarios.
  • At least one embodiment of the present disclosure provides a nail recognition method, which can be executed by electronic devices such as terminal devices or servers, and the terminal devices can be fixed terminals or mobile terminals, such as mobile phones, tablet computers, game machine, desktop computer, advertising machine, all-in-one machine, vehicle-mounted terminal, etc., and the server includes a local server or a cloud server, etc., and the method can also be realized by calling a computer-readable instruction stored in a memory by a processor.
  • FIG. 1 shows a flowchart of a nail recognition method according to at least one embodiment of the present disclosure. As shown in FIG. 1 , the method includes steps 101 to 104 .
  • step 101 a detection result of at least one nail in a first image is acquired.
  • the first image may be a still image or a video image captured in real time, or may be a still image or a video image acquired from a memory or other media.
  • the nails in the embodiments of the present disclosure may be the nails of the hand or the nails of the feet, which is not limited in the present disclosure.
  • the first image may be a separate hand image, or a human body image including a hand area; it may also be a partial hand including one or more nails image.
  • a nail detection network may be used to perform nail detection on the first image to obtain a detection result of at least one nail in the first image.
  • the nail detection network is a deep learning network, such as RCNN, Fast RCNN, Faster RCNN or the like.
  • the detection result may include the first nail detection frame, the position information of the first nail detection frame, the classification result of the nail, etc., wherein the classification result indicates the finger type to which the nail belongs.
  • the finger type to which the nail belongs indicates which finger the nail belongs to, or which finger of which hand the nail belongs to. For example, if the classification result indicates the index finger, it indicates that the nail belongs to the index finger; for another example, if the classification result indicates the left thumb, it indicates that the nail belongs to the left thumb.
  • a classification result of the nails can be obtained while the nails are detected.
  • the nail detection network can be trained by using the following sample image: the sample image is marked with a bounding box of at least one nail contained therein, and the type of the nail in the bounding box is marked.
  • step 102 an image area corresponding to the nail in the first image is obtained according to the first nail detection frame.
  • the image area surrounded by the detection frame of the nail is the image area corresponding to the nail.
  • step 103 according to the finger type to which the nail belongs, a plurality of first key points of the nail in the image area corresponding to the nail are obtained.
  • first key points of the nail are used to represent different position points of the nail outline, wherein each first key point is used to represent a specific position point of the nail.
  • the outline of the nail has certain characteristics, and the outline of the nail can be outlined by specific position points. Therefore, when multiple first key points of the nail are detected, the outline of the nail can be determined to obtain edge information of the nail.
  • one nail can correspond to any number of first key points within the range of 12 to 32.
  • one fingernail corresponds to 16 first key points.
  • the first key point P1 is used to characterize the leftmost point at the bottom of the nail outline
  • the first key point P5 is used to characterize the rightmost point at the bottom of the nail outline
  • the first key point P11 is used to characterize the point located in the topmost middle of the nail contour.
  • the number of first key points corresponding to nails of different fingers and the position of each first key point on the nail outline may be the same or different.
  • the nail area of the thumb is larger than that of the little finger, and so is the outline length. Therefore, the nail of the thumb can correspond to a larger number of first key points, such as 32; relatively, the nail of the little finger can correspond to A smaller number of first key points, eg 12.
  • first key number is only an example, which is not limited by the embodiments of the present disclosure.
  • a corresponding first key point detection network may be constructed for the type of finger to which each nail belongs, for performing nail key point detection on nails belonging to the finger type. For example, when the classification result of the nail indicates that the nail belongs to the left thumb, the first key point detection network of the left thumb is called to perform nail key point detection on the image area corresponding to the nail, and the left hand big Multiple first keys on the nail of the thumb.
  • a detection result of at least one nail in the first image is acquired, the detection result includes a first nail detection frame and a classification result of the nail, the classification result indicates the finger type to which the nail belongs; then Obtain the image area corresponding to the nail in the first image according to the first nail detection frame; and obtain a plurality of first images of the nail in the image area corresponding to the nail according to the finger type to which the nail belongs. key point.
  • the image area corresponding to the nail can be cropped first, and then the cropped image area can be input into the first key point detection network corresponding to the finger type to which the nail belongs to obtain multiple images of the nail.
  • the first key point can be cropped first, and then the cropped image area can be input into the first key point detection network corresponding to the finger type to which the nail belongs to obtain multiple images of the nail. The first key point.
  • the binary classification result of each pixel in the image area corresponding to the nail is obtained, and the binary classification result indicates that the pixel is a foreground pixel or a background pixel, the foreground pixel is the pixel corresponding to the nail area, and the background pixel is That is, pixels corresponding to areas other than the nail area.
  • the pixels indicated as background pixels in the binary classification result can be set as the first pixel value, wherein the first pixel value can be 0, or 255, or other values, and the first pixel value The value of the value is determined according to the setting of the background color.
  • the image area judged as the background in the image area corresponding to the nail can be filtered out, so that only the real nail corresponding to the nail remains in the image area corresponding to the nail. Area.
  • the pixels indicated as background pixels may be set as the first pixel value.
  • the direction of the nail may be determined according to position information of at least two first key points of the plurality of first key points of the nail in the image area.
  • each first key point of the nail represents a specific position point of the nail
  • the direction of the nail can be represented according to at least two first key points.
  • the direction indicated by the normal line of the nail can be determined as the direction of the nail.
  • a line between the first key point located in the middle of the bottommost end of the nail outline and the first key point located in the middle of the topmost end forms the first key point of the nail.
  • the normal therefore, the direction of the nail can be determined by the direction indicated by the normal in the first image, or in the image coordinate system.
  • the line connecting the first key point P3 and P11 may be used as the normal line of the nail.
  • the normal line formed by P3 and P11 indicates the vertical direction of the first image, so it can be determined that the direction of the nail in FIG. 2 is the vertical direction of the first image.
  • the direction of the nail may also be determined according to other key points of the multiple key points of the nail, which is not limited in the present disclosure.
  • the first key point detection network can be trained by the following method.
  • sample image Acquiring a sample image; wherein, the sample image has annotation information, and the annotation information indicates a first key point corresponding to the finger type to which the sample image belongs.
  • the number of sample key points marked in the sample image may be any number within the range of 12 to 32.
  • the sample image includes the nail of the index finger, and 16 sample key points are marked on the edge of the nail of the index finger.
  • each sample key point has a serial number, as shown in Figure 2, the point at the bottom and left of the nail contour is the No. 1 sample key point, denoted as P1, and the bottom and right point is No. 5 Sample keypoints, denoted as P5, etc.
  • the sample image is input to the first key point detection network to obtain a key point detection result.
  • the number of predicted first key points in the key point detection result is the same as the number of labeled sample key points, and the predicted first key points also have serial numbers.
  • the network parameters of the first key point detection network are adjusted according to the difference between the key point detection result and the annotation information. That is, the network parameters of the first key point detection network are adjusted according to the difference between each sample key point and the corresponding predicted first key point.
  • the difference is less than the set threshold, or the iteration reaches the set number of times, the training is stopped, and the first key point detection network that has completed the training is obtained.
  • the first key point detection network can realize the recognition of the nail of the specified finger type.
  • the first key point detection network may perform nail key point detection based on a heat map.
  • the heat map of the first key point is a probability distribution map of possible locations of the first key point in the first image.
  • the coordinates of the first key point in the first image may be determined.
  • the position of the first key point of each nail in the first image can be accurately determined according to the key point heat map.
  • an embodiment of the present disclosure proposes a nail key point tracking method.
  • the first image is a video image, that is, the first image is a frame in an image sequence for the same scene, for any frame image (second image) after the first image.
  • the following methods can be used for nail key point tracking.
  • a second nail detection frame in the second image is determined according to a plurality of first key points of the nail in a previous frame of the second image.
  • the nail in the current frame (second image) can be determined The second nail detection frame of .
  • the circumscribed rectangular frame of the nail can be obtained according to a plurality of first key points of the nail in the previous frame; according to the position of the circumscribed rectangular frame in the previous frame information, mapping the circumscribed rectangular frame to the second image, that is, placing the circumscribed rectangular frame at the same position in the second image as in the previous frame, as the first frame in the second image 2.
  • the second nail detection frame in the second image obtained by the above method is closer to the real area of the nail and contains less parts other than the nail, which is beneficial to improve the key Accuracy of point detection.
  • the image area corresponding to the second nail detection frame in the second image can be cropped; the cropped image area is input to the second key point detection network to obtain the second key point of the nail point.
  • the second key point detection network has the same function as the first key point detection network, both of which can be used to detect nail key points from the input image; and the second key point detection
  • the training method of the network may also be the same as that of the first key point detection network.
  • the coordinates of each second key point in the input image may be obtained based on key point regression.
  • the network structure is simpler, the number of layers is smaller, and the processing speed is faster. Fast, reducing the time-consuming for nail key point detection.
  • the cropped image before the cropped image is input to the second key point detection network, the cropped image may be rotated according to the direction of the nail in the previous frame.
  • the cropped image can be rotated counterclockwise by 5 degrees according to the direction, so that the The direction of the nail is that the normal points to the vertical direction.
  • the nail key point detection on the rotated image can improve the efficiency of detection on the one hand, and improve the accuracy of detection on the other hand.
  • the amount of data processing can be reduced, and the speed and efficiency of nail key point detection can be improved.
  • the method of detecting nail key points on the first image is still used to detect nail key points on the second image, which specifically includes: obtaining the detection result of at least one nail in the second image, and The detection result includes a first nail detection frame and a classification result of the nail, and the classification result indicates the finger type to which the nail belongs; according to the first nail detection frame, an image corresponding to the nail in the second image is obtained Area: According to the finger type to which the nail belongs, a plurality of first key points of the nail in the image area corresponding to the nail are obtained.
  • the nail recognition By judging the second key point of the tracked nail, if it is not detected or does not meet the set requirements, the nail recognition will be performed again. On the one hand, it ensures the consistency of the nail recognition results, and on the other hand, it also ensures The accuracy of nail recognition results.
  • nail recognition can be performed on video images containing hands in the following manner. As shown in FIG. 3 , the method may include steps 301 to 309 .
  • the nail detection network may be used to perform nail detection on the first image to obtain at least one nail in the first image.
  • the first nail detection frame, and the classification result of the nail indicates the finger type to which the nail belongs.
  • the first image is the first frame image in a scene.
  • step 302 the image area corresponding to the first nail detection frame is cut out to obtain a first nail area image.
  • step 303 the binary classification result of each pixel in the nail region image is obtained, the binary classification result indicates that the pixel is a foreground pixel or a background pixel; the pixel indicated as a background pixel in the binary classification result is set as The first pixel value.
  • step 304 the nail region image processed in step 303 is input to the first key point detection network to obtain a plurality of first key points of the nail.
  • step 305 for the second image following the first image, according to a plurality of first key points of the nail in the previous frame of the second image, a circumscribed rectangular frame of the nail is obtained; according to The location information of the circumscribed rectangular frame in the previous frame is used to map the circumscribed rectangular frame into the second image as the second nail detection frame in the second image.
  • step 306 the image area corresponding to the second nail detection frame is cut out to obtain a second nail area image.
  • step 307 the second nail region image is rotated according to the direction of the nail in the previous frame to obtain a rotated image.
  • the direction of the nail is determined according to position information of at least two first key points of the plurality of first key points of the nail in the previous frame.
  • step 308 the rotated image is input to the second key point detection network to obtain multiple second key points of the nail.
  • step 309 the key point detection result obtained in step 308 is judged, and if the plurality of second key points of the nail meet the set requirements, it is judged that the tracking of the second image is successful, and the process returns to step 305 , continue to track the next frame of image; if the second key point is not detected, or if multiple second key points of the nail do not meet the set requirements, it is determined that the tracking is unsuccessful, and then return to the step 301. Process the second image as the first image.
  • Fig. 4 is a schematic structural diagram of a nail recognition device proposed by at least one embodiment of the present disclosure.
  • the device may include: a first acquisition unit 401, configured to acquire a detection result of at least one nail in the first image, so The detection result includes a first nail detection frame and a classification result of the nail, and the classification result indicates the finger type to which the nail belongs; the second acquiring unit 402 is configured to obtain the first nail detection frame according to the first nail detection frame. An image area corresponding to the nail in the image; an identification unit 403 configured to obtain a plurality of first key points of the nail in the image area corresponding to the nail according to the finger type to which the nail belongs.
  • the identification unit is specifically configured to: crop out the image area corresponding to the nail from the first image; input the cropped image area into the In the first key point detection network of , multiple first key points of the nail are obtained.
  • the device further includes a filtering unit, configured to: obtain a binary classification result of each pixel in the image region corresponding to the nail, the binary classification result indicating that the pixel is a foreground pixel or Background pixels: setting the pixels indicated as background pixels in the binary classification result as the first pixel value.
  • a filtering unit configured to: obtain a binary classification result of each pixel in the image region corresponding to the nail, the binary classification result indicating that the pixel is a foreground pixel or Background pixels: setting the pixels indicated as background pixels in the binary classification result as the first pixel value.
  • the device further includes an orientation unit configured to: according to the position information of at least two first key points among the plurality of first key points of the nail in the image area , to determine the orientation of the nail.
  • the device further includes a training unit configured to: acquire a sample image; wherein, the sample image has annotation information, and the annotation information indicates the finger type corresponding to the sample image.
  • the first key point input the sample image to the first key point detection network to obtain the key point detection result; according to the difference between the key point detection result and the label information, the first key point
  • the network parameters of the point detection network are adjusted.
  • the first image is a frame in an image sequence
  • the device further includes a tracking unit configured to: for a second image following the first image, according to the first image A plurality of first key points of the nail in the previous frame of the second image, determine a second nail detection frame in the second image; obtain in the image area corresponding to the second nail detection frame in the second image , a plurality of second key points of the nail.
  • the tracking unit when used to determine the second nail detection frame in the second image according to the multiple first key points in the previous frame of the second image, specifically It is used to: obtain the circumscribed rectangular frame of the nail according to the multiple first key points of the nail in the previous frame; and obtain the circumscribed rectangular frame according to the position information of the circumscribed rectangular frame in the previous frame.
  • the circumscribed rectangular frame is mapped to the second image as a second nail detection frame in the second image.
  • the tracking unit when used to obtain multiple second key points of the nail in the image region corresponding to the second nail detection frame in the second image, It is specifically used for: cutting out the image area corresponding to the second nail detection frame in the second image; inputting the cropped image area into the second key point detection network to obtain the second key point of the nail.
  • the device further includes a rotation unit, configured to, before inputting the cropped image area into the second key point detection network, according to the nail in the previous frame direction, and rotate the cropped image.
  • the device further includes a judging unit configured to: if the second key point of the nail is not detected or the second key point of the nail does not meet the set requirements , acquiring a detection result of at least one nail in the second image, the detection result including a first nail detection frame and a classification result of the nail, the classification result indicating the finger type to which the nail belongs; according to the first nail detection Obtain an image area corresponding to the nail in the second image; and obtain a plurality of first key points of the nail in the image area corresponding to the nail according to the finger type to which the nail belongs.
  • At least one embodiment of the present disclosure also provides an electronic device. As shown in FIG. 5 , the device includes a memory 501 and a processor 502. The computer instructions implement the image processing method described in any embodiment of the present disclosure.
  • At least one embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the image processing method described in any embodiment of the present disclosure is implemented.
  • At least one embodiment of the present disclosure further provides a computer program product, including a computer program, when the program is executed by a processor, the image processing method described in any embodiment of the present disclosure is implemented.
  • one or more embodiments of this specification may be provided as a method, system or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may employ a computer program embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The form of the product.
  • each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments.
  • the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment.
  • Embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or in A combination of one or more of .
  • Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of data processing apparatus. Multiple modules.
  • the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for transmission by the data
  • the processing means executes.
  • a computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
  • the processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output.
  • the processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).
  • FPGA Field Programmable Gate Array
  • ASIC Application Specific Integrated Circuit
  • Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit.
  • a central processing unit will receive instructions and data from a read only memory and/or a random access memory.
  • the essential components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data.
  • a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic or magneto-optical disks, or optical disks, to receive data therefrom or to It transmits data, or both.
  • mass storage devices for storing data, such as magnetic or magneto-optical disks, or optical disks, to receive data therefrom or to It transmits data, or both.
  • a computer is not required to have such a device.
  • a computer may be embedded in another device such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a device such as a Universal Serial Bus (USB) ) portable storage devices like flash drives, to name a few.
  • PDA personal digital assistant
  • GPS Global Positioning System
  • USB Universal Serial Bus
  • Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks.
  • semiconductor memory devices such as EPROM, EEPROM, and flash memory devices
  • magnetic disks such as internal hard disks or removable disks
  • magneto-optical disks and CD ROM and DVD-ROM disks.
  • the processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Processing (AREA)

Abstract

公开了一种指甲识别方法、装置、设备及存储介质，所述方法包括：获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。

Description

指甲识别方法、装置、设备及存储介质
相关申请的交叉引用
本公开要求于2021年6月30日提交的、申请号为202110736401.3、发明名称为“一种指甲识别方法、装置、设备及存储介质”的中国专利申请的优先权，该中国专利申请公开的全部内容以引用的方式并入本文中。
技术领域
本申请涉及图像识别技术领域，尤其涉及一种指甲识别方法、装置、设备及存储介质。
背景技术
指甲识别在移动互娱、虚拟试装、虚拟现实VR、增强现实AR等领域具有重要的应用前景。相关技术中，通常利用分割模型(segmentation)得到图像或视频中的指甲区域，然而这种方法并不能得到指甲轮廓的语义信息，使得识别结果在各个应用场景中的使用受到了限制。
发明内容
本公开实施例提供一种指甲识别方案。
根据本公开的一方面，提供一种指甲识别方法，所述方法包括：获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
通过获取指甲的分类结果，并根据指甲所属手指类型获得相应的指甲对应的第一关键点，可以获得指甲的类别信息和指甲轮廓的完整语义特征，便于指甲识别结果在各个场景下的应用。
结合本公开提供的任一实施方式，所述根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中，所述指甲的多个第一关键点，包括：从所述第一图像中裁剪出所述指甲对应的图像区域；将裁剪出的图像区域输入至所述指甲所属手指类型对应的第一关键点检测网络中，得到所述指甲的多个第一关键点。
通过裁剪出所述指甲对应的图像区域，并根据裁剪出的图像进行指甲关键点检测，可以提高指甲关键点检测的效率和准确度。
结合本公开提供的任一实施方式，所述方法还包括：获取所述指甲对应的图像区域中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素；将所述二分类结果中指示为背景像素的像素设置为第一像素值。
通过对所述指甲对应的图像区域中，或者裁剪出的图像中的背景像素进行滤除操作，只保留真实指甲对应的区域，可以减小指甲关键点误检的概率。
结合本公开提供的任一实施方式，所述方法还包括：依据所述指甲的多个 第一关键点中的至少两个第一关键点在所述图像区域中的位置信息，确定所述指甲的方向。
通过获取所述指甲的方向，便于指甲识别结果在各个场景下的应用，例如在为图像中指甲添加美甲特效的场景中，在获知指甲的方向的情况下，可以更方便地为指甲添加美甲特效。
结合本公开提供的任一实施方式，所述方法还包括：获取样本图像；其中，所述样本图像具有标注信息，所述标注信息指示与所述样本图像所属手指类型对应的第一关键点；将所述样本图像输入至所述第一关键点检测网络，得到关键点检测结果；根据所述关键点检测结果与所述标注信息之间的差异，对所述第一关键点检测网络的网络参数进行调整。
通过根据样本图像中指甲所述手指类型对指甲进行信息标注，并利用所述样本图像对第一关键点检测网络进行训练，可以实现第一关键点检测网络针对指定手指类型的指甲的识别。
结合本公开提供的任一实施方式，所述第一图像是图像序列中的一帧，所述方法还包括：对于所述第一图像之后的第二图像，根据所述第二图像的前一帧中所述指甲的多个第一关键点，确定第二图像中的第二指甲检测框；获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点。
通过根据图像序列中前一帧的关键点检测结果，得到当前帧中的关键点检测结果，可以减小数据处理量，提高指甲关键点检测的速度和效率。
结合本公开提供的任一实施方式，所述根据所述第二图像的前一帧中的多个第一关键点，确定第二图像中的第二指甲检测框，包括：根据所述前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，作为所述第二图像中的第二指甲检测框。
通过上述方法得到的第二图像中的第二指甲检测框，相较于通过指甲检测到的第一指甲检测框，更接近指甲的真实区域，且包含更少指甲以外的部分，有利于提高关键点检测的精度。
结合本公开提供的任一实施方式，所述获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点，包括：裁剪出所述第二图像中所述第二指甲检测框对应的图像区域；将裁剪出的图像区域输入至第二关键点检测网络，得到所述指甲的第二关键点。
第二关键点检测网络可以基于回归关键点进行指甲关键点检测，相较于基于热度图进行指甲关键点检测的第一关键点检测网络，网络结构更简单、层数较小、处理速度更快，减少了进行指甲关键点检测的耗时。
结合本公开提供的任一实施方式，在将所述裁剪出的图像区域输入至第二关键点检测网络之前，根据所述前一帧中所述指甲的方向，对所述裁剪出的图像进行旋转处理。
对旋转后的图像进行指甲关键点检测，一方面可以提高检测的效率，一方面也可以提高检测的精度。
结合本公开提供的任一实施方式，所述方法还包括：在未检测到所述指甲的第二关键点或所述指甲的第二关键点不符合设定要求的情况下，获取第二图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第二图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获 得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
通过对追踪得到的指甲的第二关键点进行判定，在未检测到或者不符合设定要求的情况下，则重新进行指甲识别，一方面保证了指甲识别结果的连贯性，另一方面也保证了指甲识别结果的准确性。
根据本公开的一方面，提供一种指甲识别装置，包括：第一获取单元，用于获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；第二获取单元，用于根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；识别单元，用于根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
结合本公开提供的任一实施方式，所述识别单元具体用于：从所述第一图像中裁剪出所述指甲对应的图像区域；将裁剪出的图像区域输入至所述指甲所属手指类型对应的第一关键点检测网络中，得到所述指甲的多个第一关键点。
结合本公开提供的任一实施方式，所述装置还包括过滤单元，用于：获取所述指甲对应的图像区域中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素；将所述二分类结果中指示为背景像素的像素设置为第一像素值。
结合本公开提供的任一实施方式，所述装置还包括定向单元，用于：依据所述指甲的多个第一关键点中的至少两个第一关键点在所述图像区域中的位置信息，确定所述指甲的方向。
结合本公开提供的任一实施方式，所述装置还包括训练单元，用于：获取样本图像；其中，所述样本图像具有标注信息，所述标注信息指示与所述样本图像所属手指类型对应的第一关键点；将所述样本图像输入至所述第一关键点检测网络，得到关键点检测结果；根据所述关键点检测结果与所述标注信息之间的差异，对所述第一关键点检测网络的网络参数进行调整。
结合本公开提供的任一实施方式，所述第一图像是图像序列中的一帧，所述装置还包括追踪单元，用于：对于所述第一图像之后的第二图像，根据所述第二图像的前一帧中所述指甲的多个第一关键点，确定第二图像中的第二指甲检测框；获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点。
结合本公开提供的任一实施方式，所述追踪单元在用于根据所述第二图像的前一帧中的多个第一关键点，确定第二图像中的第二指甲检测框时，具体用于：根据所述前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，作为所述第二图像中的第二指甲检测框。
结合本公开提供的任一实施方式，所述追踪单元在用于获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点时，具体用于：裁剪出所述第二图像中所述第二指甲检测框对应的图像区域；将裁剪出的图像区域输入至第二关键点检测网络，得到所述指甲的第二关键点。
结合本公开提供的任一实施方式，所述装置还包括旋转单元，用于在将所述裁剪出的图像区域输入至第二关键点检测网络之前，根据所述前一帧中所述指甲的方向，对所述裁剪出的图像进行旋转处理。
结合本公开提供的任一实施方式，所述装置还包括判定单元，用于：在未检测到所述指甲的第二关键点或所述指甲的第二关键点不符合设定要求的情况 下，获取第二图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第二图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
根据本公开的一方面，提供一种电子设备，所述设备包括存储器、处理器，所述存储器用于存储可在处理器上运行的计算机指令，所述处理器用于在执行所述计算机指令时实现本公开提供的任一实施方式所述的指甲识别方法。
根据本公开的一方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现本公开提供的任一实施方式所述的指甲识别方法。
根据本公开的一方面，提供一种计算机程序产品，包括计算机程序，所述程序被处理器执行时实现本公开提供的任一实施方式所述的指甲识别方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本说明书。
附图说明
为了更清楚地说明本说明书一个或多个实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本说明书一个或多个实施例中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本公开至少一个实施例提出的一种指甲识别方法的流程图；
图2是本公开至少一个实施例提出的指甲识别方法中指甲的第一关键点示意图；
图3是本公开至少一个实施例提出的另一种指甲识别方法的流程图；
图4是本公开至少一个实施例提出的指甲识别装置的结构示意图；
图5是本公开至少一个实施例提出的电子设备的结构示意图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本说明书相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本说明书的一些方面相一致的装置和方法的例子。
在本说明书使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本说明书。在本说明书和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本说明书可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本说明书范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如 果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
相关技术中，在图像中识别出指甲还停留于对指甲区域的识别，最常用的方法就是分割模型，即，对图像中的每个像素点进行检测，将属于指甲的像素点组合起来，作为针对指甲的检测结果。这种指甲检测的方式，虽然能够在图像中将属于指甲的区域识别出来，但指甲是哪个手指的、哪个手的，以及指甲的朝向，都是上述指甲识别的方式所不能确定的，这大大限制了识别结果在各个应用场景中的使用，例如，在移动互娱、虚拟试妆、虚拟现实VR、增强现实AR等场景下，用户想要为指甲添加具有方向的特效，或者为不同的指甲添加不同的特效，那么就需要在识别指甲时，能够识别出指甲的类别，或者能够识别出指甲的朝向等。
鉴于上述问题，本公开至少一个实施例提供了一种指甲识别方法，该方法可以由终端设备或服务器等电子设备执行，所述终端设备可以是固定终端或移动终端，例如手机、平板电脑、游戏机、台式机、广告机、一体机、车载终端等等，所述服务器包括本地服务器或云端服务器等，所述方法还可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
图1示出根据本公开至少一个实施例的指甲识别方法的流程图，如图1所示，所述方法包括步骤101至步骤104。
在步骤101中，获取第一图像中至少一个指甲的检测结果。
其中，所述第一图像可以是实时拍摄的静态图像或者视频图像，也可以是从存储器或者其他介质中获取的静态图像或者视频图像。并且，本公开实施例中的指甲可以是手部的指甲，也可以是脚部的指甲，本公开对此不进行限制。以所述指甲为手部的指甲为例，所述第一图像可以是单独的手部图像，或者是包含了手部区域的人体图像；也可以是包含了一个或多个指甲的局部手部图像。
在本公开实施例中，可以利用指甲检测网络对所述第一图像进行指甲检测，得到所述第一图像中至少一个指甲的检测结果。其中，所述指甲检测网络为深度学习网络，例如RCNN、Fast RCNN、Faster RCNN等等。所述检测结果可以包含第一指甲检测框、第一指甲检测框的位置信息、所述指甲的分类结果等，其中，所述分类结果指示所述指甲所属手指类型。指甲所属手指类型表示该指甲是哪个手指的指甲，或者该指甲是哪个手的哪个手指的指甲。例如，所述分类结果指示食指，则表明该指甲是食指的指甲；又例如，所述分类结果指示左手大拇指，则表明该指甲是左手大拇指的指甲。
在利用指甲检测网络对所述第一图像进行多目标检测的情况下，则可以在检测出指甲的同时，还得到所述指甲的分类结果。
该指甲检测网络可以利用如下的样本图像进行训练：该样本图像标注了所包含的至少一个指甲每个的包围框，并且标注了该包围框中该指甲所属类型。
在步骤102中，根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域。其中，所述指甲的检测框所包围的图像区域，即为所述指甲对应的图像区域。
在步骤103中，根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
其中，指甲的多个第一关键点用于表征指甲轮廓的不同位置点，其中每个第一关键点用于表征指甲的特定位置点。指甲的轮廓具有一定的特点，通过特定位置点可勾勒出指甲的轮廓。因此，在检测出所述指甲的多个第一关键点的情况下，则可以确定出所述指甲的轮廓，得到所述指甲的边缘信息。
在通常情况下，一个指甲可以对应12至32范围内任意数目的第一关键点。 以图2所示的第一关键点示意图为例，一个指甲对应于16个第一关键点。如图2所示，第一关键点P1用于表征位于指甲轮廓最底部的最左侧的点，第一关键点P5用于表征位于指甲轮廓最底部的最右侧的点，第一关键点P11用于表征位于指甲轮廓的最顶部中间的点。本领域技术人员应当了解，图2所示的指甲对应于16个第一关键点仅用于示例，本公开对此不进行限制。
不同手指的指甲对应的第一关键点数目以及各个第一关键点在指甲轮廓上的位置，可以相同，也可以不同。一般情况下，大拇指的指甲面积要大于小拇指的指甲面积，轮廓长度也是如此，因此，大拇指的指甲可以对应于较多数目的第一关键点，例如32个；相对地，小拇指的指甲可以对较少数目的第一关键点，例如12个。本领域技术人员应当理解，以上所述的第一关键数目仅为示例，本公开实施例对此不进行限制。
在本公开实施例中，可以针对每个指甲所属手指类型构建相应的第一关键点检测网络，用于对属于该手指类型的指甲进行指甲关键点检测。例如，在所述指甲的分类结果指示所述指甲属于左手大拇指的情况下，则调用左手大拇指的第一关键点检测网络对所述指甲对应的图像区域进行指甲关键点检测，得到左手大拇指的指甲上的多个第一关键点。
在本公开实施例中，获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；之后根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；并根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。通过获取指甲的分类结果，并根据指甲所属手指类型获得相应的指甲对应的第一关键点，可以获得指甲的类别信息和指甲轮廓的完整语义特征，便于指甲识别结果在各个场景下的应用。
在一些实施方式中，可以首先裁剪出所述指甲对应的图像区域，再将裁剪出的图像区域输入至所述指甲所属手指类型对应的第一关键点检测网络中，得到所述指甲的多个第一关键点。
通过裁剪出所述指甲对应的图像区域，并根据裁剪出的图像进行指甲关键点检测，可以提高指甲关键点检测的效率和准确度。
在一些实施方式中，获取所述指甲对应的图像区域中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素，前景像素即为指甲区域对应的像素，背景像素即为指甲区域以外的区域对应的像素。接下来，可以将所述二分类结果中指示为背景像素的像素设置为第一像素值，其中，所述第一像素值可以为0，或者255，也可以为其他数值，所述第一像素值的取值根据背景颜色的设置具体确定。通过将指示为背景像素的像素设置为第一像素值，可以将所述指甲对应的图像区域中被判断为背景的图像区域过滤掉，使得所述指甲对应的图像区域中只保留了真实指甲对应的区域。
在一个示例中，也可以针对裁剪出的图像，根据所述图像中各个像素的二分类结果，将指示为背景像素的像素设置为第一像素值。
通过对所述指甲对应的图像区域中，或者裁剪出的图像中的背景像素进行滤除操作，只保留真实指甲对应的区域，可以减小指甲关键点误检的概率。
在一些实施方式中，可以依据所述指甲的多个第一关键点中的至少两个第一关键点在所述图像区域中的位置信息，确定所述指甲的方向。
由于所述指甲的每个第一关键点表征一个指甲的特定位置点，因此，根据至少两个第一关键点则可以表示出所述指甲的方向。
通常，可以将指甲的法线所指示的方向、即指甲的生长方向，确定为所述指甲的方向。一般情况下，在所述指甲的多个第一关键点中，位于指甲轮廓最底端中间的第一关键点与位于最顶端中间的第一关键点之间的连线，形成所述指甲的法线，因此，通过所述法线在所述第一图像中，或者在图像坐标系中所指示的方向，可以确定所述指甲的方向。
如图2所示，可以将第一关键点P3与P11之间的连线，作为所述指甲的法线。在图2中，P3与P11所形成的法线指示所述第一图像的垂直方向，因此可以确定图2中指甲的方向为所述第一图像的垂直方向。
也可以根据所述指甲的多个关键点中的其他关键点来确定所述指甲的方向，本公开对此不进行限制。
通过获取所述指甲的方向，便于指甲识别结果在各个场景下的应用，例如在为图像中指甲添加美甲特效的场景中，在获知指甲的方向的情况下，可以更方便地为指甲添加美甲特效。
在一些实施方式中，可以通过以下方法对所述第一关键点检测网络进行训练。
获取样本图像；其中，所述样本图像具有标注信息，所述标注信息指示与所述样本图像所属手指类型对应的第一关键点。样本图像中标注的样本关键点的数目可以是12至32范围内的任意数目。例如，所述样本图像中包含食指的指甲，并且在所述食指的指甲边缘标注了16个样本关键点。其中，每个样本关键点具有序号，如图2所示，位于指甲轮廓最底部最左侧的点为第1号样本关键点，表示为P1，在最底部最右侧的点为第5号样本关键点，表示为P5，等等。
将所述样本图像输入至所述第一关键点检测网络，得到关键点检测结果。所述关键点检测结果中预测的第一关键点的数目与所标注的样本关键点数目相同，并且所述预测的第一关键点同样具有序号。
根据所述关键点检测结果与所述标注信息之间的差异，对所述第一关键点检测网络的网络参数进行调整。也即，根据各个样本关键点与对应的预测的第一关键点之间的差异，调整所述第一关键点检测网络的网络参数。在差异小于设定阈值，或者迭代达到设定次数的情况下，停止训练，得到完成训练的第一关键点检测网络。
通过根据样本图像中指甲所属手指类型对指甲进行信息标注，并利用所述样本图像对第一关键点检测网络进行训练，可以实现第一关键点检测网络针对指定手指类型的指甲的识别。
在本公开实施例中，所述第一关键点检测网络可以基于热度图进行指甲关键点检测。
首先，生成所述第一图像中各个第一关键点的热度图。所述第一关键点的热度图是该第一关键点在所述第一图像中可能存在位置的概率分布图。
进而，根据所述第一关键点的热度图，可以确定所述第一关键点在所述第一图像中的坐标。
根据关键点热度图可以准确地确定第一图像中各个指甲的第一关键点的位置。
然而，由于基于热度图进行指甲关键点检测的第一关键点检测网络结构复杂、层数较多，并且耗时较大，本公开实施例提出了一种指甲关键点追踪方法。在所述第一图像为视频图像，也即所述第一图像是针对同一场景的图像序列中的一帧的情况下，对于所述第一图像之后的任一帧图像(第二图像)，可以采用以 下方法进行指甲关键点追踪。
首先，根据第二图像的前一帧中所述指甲的多个第一关键点，确定所述第二图像中的第二指甲检测框。
由于同一场景的图像序列中，连续两帧图像中指甲的位置变化通常较小，因此，根据前一帧中一指甲的多个第一关键点，可以确定当前帧(第二图像)中该指甲的第二指甲检测框。
在一些实施方式中，可以根据所述前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，也即将所述外接矩形框放置于所述第二图像中与前一帧中相同的位置，作为所述第二图像中的第二指甲检测框。
通过上述方法得到的第二图像中的第二指甲检测框，相较于通过指甲检测到的第一指甲检测框，更接近指甲的真实区域，且包含更少指甲以外的部分，有利于提高关键点检测的精度。
在一些实施方式中，可以裁剪出所述第二图像中所述第二指甲检测框对应的图像区域；将裁剪出的图像区域输入至第二关键点检测网络，得到所述指甲的第二关键点。
在本公开实施例中，所述第二关键点检测网络与所述第一关键点检测网络的作用相同，都可以用于从输入图像中检测出指甲关键点；并且所述第二关键点检测网络的训练方法也可以与所述第一关键点检测网络相同。然而，在所述第二检测框包含指甲以外的部分更少的情况下，可以基于关键点回归的方式，得到各个第二关键点在输入图像中的坐标。
由于基于关键点回归进行指甲关键点检测的第二关键点检测网络，相较于基于热度图进行指甲关键点检测的第一关键点检测网络，网络结构更简单、层数较小、处理速度更快，减少了进行指甲关键点检测的耗时。
在一些实施方式中，在将所述裁剪出的图像输入至第二关键点检测网络之前，可以根据所述前一帧中所述指甲的方向，对所述裁剪出的图像进行旋转处理。
例如，在所述指甲的方向为法线与图像坐标系中的水平方向夹角为85度时，则可以根据该方向，将所述裁剪出来的图像沿逆时针方向旋转5度，以使得所述指甲的方向为法线指向竖直方向。
对旋转后的图像进行指甲关键点检测，一方面可以提高检测的效率，一方面也可以提高检测的精度。
在本公开实施例中，通过根据图像序列中前一帧的关键点检测结果，得到当前帧中的关键点检测结果，可以减小数据处理量，提高指甲关键点检测的速度和效率。
然而，在未检测到所述指甲的第二关键点或所述指甲的第二关键点不符合设定要求，例如，检测出的第二关键点超出裁剪出的图像区域的范围的情况下，则判定追踪失败，仍然采用与对所述第一图像进行指甲关键点检测的方法，对所述第二图像进行指甲关键点检测，具体包括：获取第二图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第二图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
通过对追踪得到的指甲的第二关键点进行判定，在未检测到或者不符合设定要求的情况下，则重新进行指甲识别，一方面保证了指甲识别结果的连贯性， 另一方面也保证了指甲识别结果的准确性。
在一些实施方式中，可以通过以下方式对包含手部的视频图像进行指甲识别。如图3所示，该方法可以包括步骤301至309。
在步骤301中，对于所述视频图像所包含的图像序列中的任一帧第一图像，可以利用指甲检测网络对所述第一图像进行指甲检测，得到所述第一图像中至少一个指甲的第一指甲检测框，以及所述指甲的分类结果。其中，所述分类结果指示所述指甲所属手指类型。
在通常情况下，所述第一图像为一个场景下的第一帧图像。
在步骤302中，将所述第一指甲检测框对应的图像区域剪裁出来，得到第一指甲区域图像。
在步骤303中，获取所述指甲区域图像中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素；将所述二分类结果中指示为背景像素的像素设置为第一像素值。
在步骤304中，将经步骤303处理的指甲区域图像输入至第一关键点检测网络，得到所述指甲的多个第一关键点。
在步骤305中，针对所述第一图像之后的第二图像，根据所述第二图像的前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，作为所述第二图像中的第二指甲检测框。
在步骤306中，将所述第二指甲检测框对应的图像区域剪裁出来，得到第二指甲区域图像。
在步骤307中，根据所述前一帧中所述指甲的方向，对所述第二指甲区域图像进行旋转，得到旋转后的图像。其中，所述指甲的方向根据所述指甲的多个第一关键点中的至少两个第一关键点在所述前一帧中的位置信息确定。
在步骤308中，将旋转后的图像输入至第二关键点检测网络，得到所述指甲的多个第二关键点。
在步骤309中，对步骤308得到的关键点检测结果进行判定，在所述指甲的多个第二关键点满足设定要求的情况下，判定对所述第二图像追踪成功，返回至步骤305中，对下一帧图像继续进行追踪；在未检测到第二关键点，或在所述指甲的多个第二关键点不满足设定要求的情况下，判定追踪不成功，则返回至步骤301，将所述第二图像作为第一图像进行处理。
图4是本公开至少一个实施例提出的指甲识别装置的结构示意图，如图4所示，该装置可以包括：第一获取单元401，用于获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；第二获取单元402，用于根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；识别单元403，用于根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
结合本公开提供的任一实施方式，所述识别单元具体用于：从所述第一图像中裁剪出所述指甲对应的图像区域；将裁剪出的图像区域输入至所述指甲所属手指类型对应的第一关键点检测网络中，得到所述指甲的多个第一关键点。
结合本公开提供的任一实施方式，所述装置还包括过滤单元，用于：获取所述指甲对应的图像区域中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素；将所述二分类结果中指示为背景像素的像素设置为第 一像素值。
结合本公开提供的任一实施方式，所述装置还包括定向单元，用于：依据所述指甲的多个第一关键点中的至少两个第一关键点在所述图像区域中的位置信息，确定所述指甲的方向。
结合本公开提供的任一实施方式，所述装置还包括训练单元，用于：获取样本图像；其中，所述样本图像具有标注信息，所述标注信息指示与所述样本图像所属手指类型对应的第一关键点；将所述样本图像输入至所述第一关键点检测网络，得到关键点检测结果；根据所述关键点检测结果与所述标注信息之间的差异，对所述第一关键点检测网络的网络参数进行调整。
结合本公开提供的任一实施方式，所述第一图像是图像序列中的一帧，所述装置还包括追踪单元，用于：对于所述第一图像之后的第二图像，根据所述第二图像的前一帧中所述指甲的多个第一关键点，确定第二图像中的第二指甲检测框；获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点。
结合本公开提供的任一实施方式，所述追踪单元在用于根据所述第二图像的前一帧中的多个第一关键点，确定第二图像中的第二指甲检测框时，具体用于：根据所述前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，作为所述第二图像中的第二指甲检测框。
结合本公开提供的任一实施方式，所述追踪单元在用于获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点时，具体用于：裁剪出所述第二图像中所述第二指甲检测框对应的图像区域；将裁剪出的图像区域输入至第二关键点检测网络，得到所述指甲的第二关键点。
结合本公开提供的任一实施方式，所述装置还包括旋转单元，用于在将所述裁剪出的图像区域输入至第二关键点检测网络之前，根据所述前一帧中所述指甲的方向，对所述裁剪出的图像进行旋转处理。
结合本公开提供的任一实施方式，所述装置还包括判定单元，用于：在未检测到所述指甲的第二关键点或所述指甲的第二关键点不符合设定要求的情况下，获取第二图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；根据所述第一指甲检测框得到所述第二图像中所述指甲对应的图像区域；根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
本公开至少一个实施例还提供了一种电子设备，如图5所示，所述设备包括存储器501、处理器502，存储器用于存储可在处理器上运行的计算机指令，处理器用于在执行所述计算机指令时实现本公开任一实施例所述的图像处理方法。
本公开至少一个实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现本公开任一实施例所述的图像处理方法。
本公开至少一个实施例还提供了一种计算机程序产品，包括计算机程序，所述程序被处理器执行时实现本公开任一实施例所述的图像处理方法。
本领域技术人员应明白，本说明书一个或多个实施例可提供为方法、系统或计算机程序产品。因此，本说明书一个或多个实施例可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且，本说明书一个或多个实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用 存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于数据处理设备实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的行为或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
本说明书中描述的主题及功能操作的实施例可以在以下中实现：数字电子电路、有形体现的计算机软件或固件、包括本说明书中公开的结构及其结构性等同物的计算机硬件、或者它们中的一个或多个的组合。本说明书中描述的主题的实施例可以实现为一个或多个计算机程序，即编码在有形非暂时性程序载体上以被数据处理装置执行或控制数据处理装置的操作的计算机程序指令中的一个或多个模块。可替代地或附加地，程序指令可以被编码在人工生成的传播信号上，例如机器生成的电、光或电磁信号，该信号被生成以将信息编码并传输到合适的接收机装置以由数据处理装置执行。计算机存储介质可以是机器可读存储设备、机器可读存储基板、随机或串行存取存储器设备、或它们中的一个或多个的组合。
本说明书中描述的处理及逻辑流程可以由执行一个或多个计算机程序的一个或多个可编程计算机执行，以通过根据输入数据进行操作并生成输出来执行相应的功能。所述处理及逻辑流程还可以由专用逻辑电路—例如FPGA(现场可编程门阵列)或ASIC(专用集成电路)来执行，并且装置也可以实现为专用逻辑电路。
适合用于执行计算机程序的计算机包括，例如通用和/或专用微处理器，或任何其他类型的中央处理单元。通常，中央处理单元将从只读存储器和/或随机存取存储器接收指令和数据。计算机的基本组件包括用于实施或执行指令的中央处理单元以及用于存储指令和数据的一个或多个存储器设备。通常，计算机还将包括用于存储数据的一个或多个大容量存储设备，例如磁盘、磁光盘或光盘等，或者计算机将可操作地与此大容量存储设备耦接以从其接收数据或向其传送数据，抑或两种情况兼而有之。然而，计算机不是必须具有这样的设备。此外，计算机可以嵌入在另一设备中，例如移动电话、个人数字助理(PDA)、移动音频或视频播放器、游戏操纵台、全球定位系统(GPS)接收机、或例如通用串行总线(USB)闪存驱动器的便携式存储设备，仅举几例。
适合于存储计算机程序指令和数据的计算机可读介质包括所有形式的非易失性存储器、媒介和存储器设备，例如包括半导体存储器设备(例如EPROM、EEPROM和闪存设备)、磁盘(例如内部硬盘或可移动盘)、磁光盘以及CD ROM和DVD-ROM盘。处理器和存储器可由专用逻辑电路补充或并入专用逻辑电路中。
虽然本说明书包含许多具体实施细节，但是这些不应被解释为限制任何发明的范围或所要求保护的范围，而是主要用于描述特定发明的具体实施例的特征。本说明书内在多个实施例中描述的某些特征也可以在单个实施例中被组合实施。另一方面，在单个实施例中描述的各种特征也可以在多个实施例中分开实施或以任何合适的子组合来实施。此外，虽然特征可以如上所述在某些组合中起作用并 且甚至最初如此要求保护，但是来自所要求保护的组合中的一个或多个特征在一些情况下可以从该组合中去除，并且所要求保护的组合可以指向子组合或子组合的变型。
类似地，虽然在附图中以特定顺序描绘了操作，但是这不应被理解为要求这些操作以所示的特定顺序执行或顺次执行、或者要求所有例示的操作被执行，以实现期望的结果。在某些情况下，多任务和并行处理可能是有利的。此外，上述实施例中的各种系统模块和组件的分离不应被理解为在所有实施例中均需要这样的分离，并且应当理解，所描述的程序组件和系统通常可以一起集成在单个软件产品中，或者封装成多个软件产品。
由此，主题的特定实施例已被描述。其他实施例在所附权利要求书的范围以内。在某些情况下，权利要求书中记载的动作可以以不同的顺序执行并且仍实现期望的结果。此外，附图中描绘的处理并非必需所示的特定顺序或顺次顺序，以实现期望的结果。在某些实现中，多任务和并行处理可能是有利的。
以上所述仅为本说明书一个或多个实施例的较佳实施例而已，并不用以限制本说明书一个或多个实施例，凡在本说明书一个或多个实施例的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本说明书一个或多个实施例。

Claims (13)

  1. 一种指甲识别方法，其特征在于，所述方法包括：
    获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；
    根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；
    根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中，所述指甲的多个第一关键点，包括：
    从所述第一图像中裁剪出所述指甲对应的图像区域；
    将所述裁剪出的图像区域输入至所述指甲所属手指类型对应的第一关键点检测网络中，得到所述指甲的多个第一关键点。
  3. 根据权利要求1或2所述的方法，其特征在于，所述方法还包括：
    获取所述指甲对应的图像区域中各个像素的二分类结果，所述二分类结果指示所述像素为前景像素或背景像素；
    将所述二分类结果中指示为背景像素的像素设置为第一像素值。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，所述方法还包括：
    依据所述指甲的多个第一关键点中的至少两个第一关键点在所述图像区域中的位置信息，确定所述指甲的方向。
  5. 根据权利要求2至4任一项所述的方法，其特征在于，所述方法还包括：
    获取样本图像；其中，所述样本图像具有标注信息，所述标注信息指示与所述样本图像所属手指类型对应的第一关键点；
    将所述样本图像输入至所述第一关键点检测网络，得到关键点检测结果；
    根据所述关键点检测结果与所述标注信息之间的差异，对所述第一关键点检测网络的网络参数进行调整。
  6. 根据权利要求1至5任一项所述的方法，其特征在于，所述第一图像是图像序列中的一帧，所述方法还包括：
    对于所述第一图像之后的第二图像，根据所述第二图像的前一帧中所述指甲的多个第一关键点，确定所述第二图像中的第二指甲检测框；
    获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述第二图像的前一帧中的多个第一关键点，确定所述第二图像中的第二指甲检测框，包括：
    根据所述前一帧中的所述指甲的多个第一关键点，得到所述指甲的外接矩形框；
    根据所述外接矩形框在所述前一帧中的位置信息，将所述外接矩形框映射至所述第二图像中，作为所述第二图像中的第二指甲检测框。
  8. 根据权利要求6或7所述的方法，其特征在于，所述获得在所述第二图像中所述第二指甲检测框对应的图像区域中，所述指甲的多个第二关键点，包括：
    裁剪出所述第二图像中所述第二指甲检测框对应的图像区域；
    将所述裁剪出的图像区域输入至第二关键点检测网络，得到所述指甲的第二关键点。
  9. 根据权利要求8所述的方法，其特征在于，在将所述裁剪出的图像区域输入至第二关键点检测网络之前，根据所述前一帧中所述指甲的方向，对所述裁剪出的图像进行旋转处理。
  10. 根据权利要求6至8任一项所述的方法，其特征在于，所述方法还包括：
    在未检测到所述指甲的第二关键点或所述指甲的第二关键点不符合设定要求的情况下，获取所述第二图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测 框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；
    根据所述第一指甲检测框得到所述第二图像中所述指甲对应的图像区域；
    根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
  11. 一种指甲识别装置，其特征在于，所述装置包括：
    第一获取单元，用于获取第一图像中至少一个指甲的检测结果，所述检测结果包含第一指甲检测框以及所述指甲的分类结果，所述分类结果指示所述指甲所属手指类型；
    第二获取单元，用于根据所述第一指甲检测框得到所述第一图像中所述指甲对应的图像区域；
    识别单元，用于根据所述指甲所属手指类型，获得在所述指甲对应的图像区域中所述指甲的多个第一关键点。
  12. 一种电子设备，其特征在于，所述设备包括存储器、处理器，所述存储器用于存储可在处理器上运行的计算机指令，所述处理器用于在执行所述计算机指令时实现权利要求1至10任一项所述的方法。
  13. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述程序被处理器执行时实现权利要求1至10任一所述的方法。
PCT/CN2021/140473 2021-06-30 2021-12-22 指甲识别方法、装置、设备及存储介质 WO2023273227A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110736401.3A CN113486761B (zh) 2021-06-30 2021-06-30 一种指甲识别方法、装置、设备及存储介质
CN202110736401.3 2021-06-30

Publications (1)

Publication Number Publication Date
WO2023273227A1 true WO2023273227A1 (zh) 2023-01-05

Family

ID=77937059

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140473 WO2023273227A1 (zh) 2021-06-30 2021-12-22 指甲识别方法、装置、设备及存储介质

Country Status (3)

Country Link
CN (1) CN113486761B (zh)
TW (1) TW202303451A (zh)
WO (1) WO2023273227A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113486761B (zh) * 2021-06-30 2024-08-13 北京市商汤科技开发有限公司 一种指甲识别方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170154214A1 (en) * 2015-11-27 2017-06-01 Holition Limited Locating and tracking fingernails in images
CN111047526A (zh) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
CN111739028A (zh) * 2020-05-26 2020-10-02 华南理工大学 指甲区域图像获取方法、系统、计算设备及存储介质
CN112183388A (zh) * 2020-09-30 2021-01-05 北京字节跳动网络技术有限公司 图像处理方法、装置、设备和介质
CN113486761A (zh) * 2021-06-30 2021-10-08 北京市商汤科技开发有限公司 一种指甲识别方法、装置、设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109829463A (zh) * 2019-01-23 2019-05-31 深圳市邻友通科技发展有限公司 一种指甲识别方法、装置、美甲机及存储介质
SG10201912990QA (en) * 2019-12-23 2020-11-27 Sensetime Int Pte Ltd Gesture Recognition Method And Apparatus, Electronic Device, And Storage Medium
CN112200183A (zh) * 2020-09-30 2021-01-08 北京字节跳动网络技术有限公司 图像处理的方法、装置、设备和计算机可读介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170154214A1 (en) * 2015-11-27 2017-06-01 Holition Limited Locating and tracking fingernails in images
CN111047526A (zh) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
CN111739028A (zh) * 2020-05-26 2020-10-02 华南理工大学 指甲区域图像获取方法、系统、计算设备及存储介质
CN112183388A (zh) * 2020-09-30 2021-01-05 北京字节跳动网络技术有限公司 图像处理方法、装置、设备和介质
CN113486761A (zh) * 2021-06-30 2021-10-08 北京市商汤科技开发有限公司 一种指甲识别方法、装置、设备及存储介质

Also Published As

Publication number Publication date
TW202303451A (zh) 2023-01-16
CN113486761A (zh) 2021-10-08
CN113486761B (zh) 2024-08-13

Similar Documents

Publication Publication Date Title
US10936911B2 (en) Logo detection
CN109584276B (zh) 关键点检测方法、装置、设备及可读介质
WO2020211624A1 (zh) 对象追踪方法、追踪处理方法、相应的装置、电子设备
US11704357B2 (en) Shape-based graphics search
US8442307B1 (en) Appearance augmented 3-D point clouds for trajectory and camera localization
WO2023010758A1 (zh) 一种动作检测方法、装置、终端设备和存储介质
JP7106742B2 (ja) 顔認識方法、装置、電子機器及びコンピュータ不揮発性読み取り可能な記憶媒体
US20120154638A1 (en) Systems and Methods for Implementing Augmented Reality
CN110197149B (zh) 耳部关键点检测方法、装置、存储介质及电子设备
US11941838B2 (en) Methods, apparatuses, devices and storage medium for predicting correlation between objects
CN113348465B (zh) 图像中对象的关联性预测方法、装置、设备和存储介质
WO2019100348A1 (zh) 图像检索方法和装置以及图像库的生成方法和装置
KR20220098312A (ko) 이미지 내 관련 대상 검출 방법, 장치, 디바이스 및 기록 매체
CN112241736B (zh) 一种文本检测的方法及装置
WO2023273227A1 (zh) 指甲识别方法、装置、设备及存储介质
CN117953581A (zh) 动作识别的方法、装置、电子设备及可读存储介质
WO2023109086A1 (zh) 文字识别方法、装置、设备及存储介质
CN116309643A (zh) 人脸遮挡分确定方法、电子设备及介质
CN110910478A (zh) Gif图生成方法、装置、电子设备及存储介质
CN113705643B (zh) 一种目标物检测方法、装置以及电子设备
Yang et al. Keyframe-based camera relocalization method using landmark and keypoint matching
CN115210758A (zh) 运动模糊稳健的图像特征匹配
CN111079472A (zh) 图像对比方法和装置
CN116259072B (zh) 一种动物识别方法、装置、设备以及存储介质
Plósz et al. Practical aspects of visual recognition for indoor mobile positioning

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21948152

Country of ref document: EP

Kind code of ref document: A1