CN113222993A - Image processing method, device, equipment and storage medium - Google Patents

Image processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN113222993A
CN113222993A CN202110713524.5A CN202110713524A CN113222993A CN 113222993 A CN113222993 A CN 113222993A CN 202110713524 A CN202110713524 A CN 202110713524A CN 113222993 A CN113222993 A CN 113222993A
Authority
CN
China
Prior art keywords
nail
processed
key point
key
point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110713524.5A
Other languages
Chinese (zh)
Inventor
苏柳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to CN202110713524.5A priority Critical patent/CN113222993A/en
Publication of CN113222993A publication Critical patent/CN113222993A/en
Priority to PCT/CN2021/140353 priority patent/WO2022267402A1/en
Priority to TW110148370A priority patent/TW202301179A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Disclosed are an image processing method, apparatus, device and storage medium, the method comprising: acquiring first key points of the fingernails in an image to be processed containing at least one fingernail, wherein each fingernail corresponds to a plurality of first key points; determining a region of the nail to be treated according to a plurality of first key points of the nail; acquiring a nail material, wherein a plurality of second key points are marked on the nail material; and rendering the nail material into the area to be processed according to the corresponding relation between the first key point and the second key point to obtain a processed image.

Description

Image processing method, device, equipment and storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to an image processing method, apparatus, device, and storage medium.
Background
Currently, cosmetic makeup or beauty treatment performed on an image or video taken by a user is mostly focused on a face, and beauty treatment on a limb portion, for example, a hand, such as a nail beautifying operation, is lacking. The nail area coverage area of the hand nail is small, the detection difficulty is high, the nail material is difficult to be accurately applied to the nail area, and the good beautifying effect is realized.
Disclosure of Invention
The embodiment of the disclosure provides an image processing scheme.
According to an aspect of the present disclosure, there is provided an image processing method, the method including: acquiring first key points of the fingernails in an image to be processed containing at least one fingernail, wherein each fingernail corresponds to a plurality of first key points; determining a region of the nail to be treated according to a plurality of first key points of the nail; acquiring a nail material, wherein a plurality of second key points are marked on the nail material; and rendering the nail material into the area to be processed according to the corresponding relation between the first key point and the second key point to obtain a processed image.
In combination with any one of the embodiments provided in the present disclosure, the acquiring a first keypoint of at least one nail in an image to be processed containing said nail comprises: inputting the image to be processed into a nail key point detection network to obtain a nail key point detection result; and obtaining a first key point of the nail according to the key point detection result of the nail, or carrying out interpolation processing on the key point detection result of the nail and obtaining the first key point of the nail according to the interpolated result.
In combination with any one of the embodiments provided by the present disclosure, the interpolating the detection result of the key points of the nail includes: performing curve interpolation on adjacent key points in the key point detection result of the fingernail; and/or; and carrying out linear interpolation on symmetrical key points in the key point detection result of the fingernail.
In combination with any embodiment provided by the present disclosure, the method further comprises: determining a target key point in a plurality of first key points of the fingernail, wherein the target key point is a first key point between a connecting line between set symmetrical key points and the first key point corresponding to the top end of the fingertip; moving the target keypoint by a set distance along a normal direction of the nail, wherein the normal direction is determined according to a plurality of first keypoints of the nail; and updating the first key point by using the moved target key point.
In combination with any one of the embodiments provided by the present disclosure, the set distance is determined according to a first distance between the target key point and the connection line, a second distance between the first key point corresponding to the fingertip tip and the connection line, and a length of the connection line, where the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connection line.
In combination with any embodiment provided by the present disclosure, the rendering the nail material to the to-be-processed region according to the corresponding relationship between the first key point and the second key point to obtain a processed image includes: acquiring texture coordinates in the nail material corresponding to a coordinate point to be processed according to the corresponding relation between the first key point and the second key point, wherein the coordinate point to be processed is located in the area to be processed; acquiring a first pixel value of the nail material at the texture coordinate; and obtaining the processed image according to the first pixel value of the coordinate point to be processed.
In combination with any one embodiment provided by the present disclosure, the coordinate point to be processed includes: the processing method comprises the steps of obtaining a first key point and a first interpolation point, wherein the first interpolation point is located in the region to be processed and is obtained by interpolation according to the first key point; and/or, the vertex of the triangular mesh and a second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertex of the triangular mesh, and the triangular mesh is constructed according to a plurality of first key points of the fingernail.
In combination with any embodiment provided by the present disclosure, the method further comprises: acquiring a second pixel value of the coordinate point to be processed in the image to be processed; the obtaining the processed image according to the first pixel value includes: according to a set fusion coefficient, carrying out weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed to obtain a target pixel value of the coordinate point to be processed; and obtaining the processed image according to the target pixel value of the coordinate point to be processed.
According to an aspect of the present disclosure, there is provided an image processing apparatus, the apparatus including: a first acquisition unit, configured to acquire first key points of at least one nail in an image to be processed, wherein each of the nails corresponds to a plurality of the first key points; a determination unit configured to determine a region to be treated of the nail based on a plurality of first key points of the nail; the second acquisition unit is used for acquiring a nail material, and a plurality of second key points are marked on the nail material; and the rendering unit is used for rendering the nail material into the to-be-processed area according to the corresponding relation between the first key point and the second key point to obtain a processed image.
In combination with any embodiment provided by the present disclosure, the first obtaining unit is specifically configured to: inputting the image to be processed into a nail key point detection network to obtain a nail key point detection result; and obtaining a first key point of the nail according to the key point detection result of the nail, or carrying out interpolation processing on the key point detection result of the nail and obtaining the first key point of the nail according to the interpolated result.
In combination with any one of the embodiments provided by the present disclosure, the interpolating the detection result of the key points of the nail includes: performing curve interpolation on adjacent key points in the key point detection result of the fingernail; and/or; and carrying out linear interpolation on symmetrical key points in the key point detection result of the fingernail.
In combination with any one of the embodiments provided by the present disclosure, the apparatus further includes a deforming unit for: determining a target key point in a plurality of first key points of the fingernail, wherein the target key point is a first key point between a connecting line between set symmetrical key points and the first key point corresponding to the top end of the fingertip; moving the target keypoint by a set distance along a normal direction of the nail, wherein the normal direction is determined according to a plurality of first keypoints of the nail; and updating the first key point by using the moved target key point.
In combination with any one of the embodiments provided by the present disclosure, the set distance is determined according to a first distance between the target key point and the connection line, a second distance between the first key point corresponding to the fingertip tip and the connection line, and a length of the connection line, where the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connection line.
In combination with any embodiment provided by the present disclosure, the rendering unit is specifically configured to: acquiring texture coordinates in the nail material corresponding to a coordinate point to be processed according to the corresponding relation between the first key point and the second key point, wherein the coordinate point to be processed is located in the area to be processed; acquiring a first pixel value of the nail material at the texture coordinate; and obtaining the processed image according to the first pixel value of the coordinate point to be processed.
In combination with any one embodiment provided by the present disclosure, the coordinate point to be processed includes: the processing method comprises the steps of obtaining a first key point and a first interpolation point, wherein the first interpolation point is located in the region to be processed and is obtained by interpolation according to the first key point; and/or, the vertex of the triangular mesh and a second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertex of the triangular mesh, and the triangular mesh is constructed according to a plurality of first key points of the fingernail.
In combination with any one of the embodiments provided by the present disclosure, the apparatus further comprises a fusion unit configured to: according to a set fusion coefficient, carrying out weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed to obtain a target pixel value of the coordinate point to be processed; and obtaining the processed image according to the target pixel value of the coordinate point to be processed.
According to an aspect of the present disclosure, there is provided an electronic device, the device comprising a memory for storing computer instructions executable on a processor, the processor being configured to implement the image processing method according to any one of the embodiments provided in the present disclosure when executing the computer instructions.
According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image processing method according to any one of the embodiments provided in the present disclosure.
According to an aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the image processing method of any one of the embodiments provided in the present disclosure.
The image processing method, the device, the equipment and the computer readable storage medium of one or more embodiments of the present disclosure, first key points of a nail in an image to be processed containing at least one nail are obtained, and a region to be processed of the nail is determined according to a plurality of first key points of the nail; and finally, rendering the nail material to the to-be-processed area according to the corresponding relation between the first key point and the second key point to obtain a processed image, and accurately rendering the nail material to the to-be-processed area of the nail in the to-be-processed image to realize special effect processing and beautifying effect processing on the nail in the to-be-processed image.
Drawings
In order to more clearly illustrate one or more embodiments or technical solutions in the prior art in the present specification, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in one or more embodiments of the present specification, and other drawings can be obtained by those skilled in the art without inventive exercise.
Fig. 1 is a flowchart of an image processing method proposed by at least one embodiment of the present disclosure;
fig. 2 is a schematic diagram of a first key point of a nail in an image processing method according to at least one embodiment of the present disclosure;
fig. 3 is a schematic view of a nail material in an image processing method according to at least one embodiment of the present disclosure;
fig. 4A and 4B are schematic diagrams of a hand image obtained by an image processing method proposed by at least one embodiment of the present disclosure;
FIG. 5 is a flow chart of a method for image processing of a hand image as set forth in at least one embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an image processing apparatus according to at least one embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to at least one embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
At least one embodiment of the present disclosure provides an image processing method, which may be performed by an electronic device such as a terminal device or a server, where the terminal device may be a fixed terminal or a mobile terminal, such as a mobile phone, a tablet computer, a game machine, a desktop computer, an advertisement machine, a kiosk, a vehicle-mounted terminal, and the like, and the server includes a local server or a cloud server, and the method may also be implemented by a manner in which a processor calls a computer-readable instruction stored in a memory.
Fig. 1 illustrates a flowchart of an image processing method according to at least one embodiment of the present disclosure, which includes steps 101 to 104, as illustrated in fig. 1.
In step 101, a first keypoint of a nail in an image to be processed comprising at least one nail is acquired.
The image to be processed may be a still image or a video image captured in real time, or may be a still image or a video image acquired from a memory or other media. The fingernails in the embodiment of the present disclosure may be fingernails of hands or may be fingernails of feet, which is not limited by the present disclosure. Taking the fingernail as the fingernail of the hand as an example, the image to be processed can be a single hand image or a human body image containing a hand area; or a partial hand image containing one or more nails.
In one example, the image to be processed may be subjected to keypoint detection to obtain keypoints of at least one nail included in the image to be processed, where each nail corresponds to a plurality of keypoints. In order to distinguish the key points acquired from the image to be processed from the key points acquired subsequently, the key points acquired in this step are referred to as first key points.
The first keypoints are generally located on the edge of the nail, each first keypoint being intended to characterize a specific location point of the nail. In general, one nail may correspond to any number of first keypoints ranging from 12 to 32. Taking the schematic diagram of the first keypoints shown in fig. 2 as an example, one nail corresponds to 16 first keypoints. As shown in FIG. 2, first keypoint P0 is used to characterize the leftmost point of the bottom-most portion of the nail contour, first keypoint P4 is used to characterize the rightmost portion of the bottom-most portion of the nail contour, and first keypoint P10 is used to characterize the middle point of the top-most portion of the nail contour. Those skilled in the art will appreciate that the fingernails shown in fig. 2 correspond to 16 first keypoints for example only, and that one fingernail may correspond to other numbers of first keypoints as well, as the present disclosure is not limited thereto.
In step 102, a region of the nail to be treated is determined based on a plurality of first keypoints of the nail.
In the embodiment of the present disclosure, since the first key point is located on the edge of the nail, the region to be treated of the nail can be obtained according to a plurality of first key points corresponding to the nail.
In step 103, a nail material is obtained, and a plurality of second key points are marked on the nail material.
The nail material can be a material which can add special effects or beautify nails, the nail material can be pure color, can also contain patterns, and can also be other types, and the embodiment of the disclosure does not limit the types of the nail material.
In the embodiment of the present disclosure, the nail material is marked with a second key point. The second keypoints are similar to the first keypoints and are also located on the edge of the nail material, and each second keypoint is used for characterizing a specific location point of the nail material. In an embodiment of the present disclosure, all or part of the first keypoints of the nail correspond one-to-one to a plurality of second keypoints of the nail material. For example, the nail material includes 16 second keypoints, and the nail also includes 16 first keypoints, and the 16 second keypoints are in one-to-one correspondence with the 16 first keypoints. For another example, the nail material includes 12 second key points, the nail includes 16 first key points, and the 12 second key points are in one-to-one correspondence with 12 first key points in the 16 first key points. The nail materials can correspond to any number of second key points within the range of 12-32.
As shown in fig. 3, the nail material is marked with second key points S0 to S15. The second key points S0-S15 correspond to the first key points P0-P15 of the fingernails in the image to be processed in a one-to-one mode.
In the embodiment of the present disclosure, the neural network for detecting nail key points on nail materials and the neural network for detecting nail key points on the image to be processed may be the same neural network or a neural network obtained by training using the same sample set.
In step 104, rendering the nail material to the region to be processed according to the corresponding relationship between the first key point and the second key point, so as to obtain a processed image.
For one of the nails in the image to be processed, after the first key point corresponding to the nail is obtained, according to the corresponding relation between the plurality of first key points of the nail and the plurality of second key points of the nail material, the corresponding relation between the area to be processed of the nail and the nail material can be determined, so that the color and/or the pattern of the nail material can be rendered into the area to be processed of the nail, the nail is made to present the same color and/or pattern as the nail material, and the special effect processing or the nail beautifying effect processing is realized.
In the embodiment of the disclosure, first key points of a nail in an image to be processed containing at least one nail are firstly acquired, and a region to be processed of the nail is determined according to a plurality of first key points of the nail; and finally, rendering the nail material to the to-be-processed area according to the corresponding relation between the first key point and the second key point to obtain a processed image, and accurately rendering the nail material to the to-be-processed area of the nail in the to-be-processed image to realize special effect processing and beautifying effect processing on the nail in the to-be-processed image.
In the embodiment of the present disclosure, the image to be processed may be input to a keypoint detection network, so as to perform keypoint detection on the image to be processed.
In some implementations, the first keypoint of the nail may be obtained according to a nail detection result output by the keypoint detection network.
In some implementations, interpolation processing may be performed on the nail detection result output by the nail key detection network, and a first key point of the nail may be obtained according to the interpolated result.
In one example, curve interpolation may be performed for adjacent keypoints of the nail in the nail detection result. Taking fig. 2 as an example, for adjacent key points, such as P7 and P8, P8 and P9, etc., by performing curve interpolation, the arrangement of key points at the edge of the nail can be more dense, so that the region to be treated surrounded by the key points maintains the rounded shape of the nail and is more consistent with the shape of the real nail.
In one example, the symmetric key points of the nail in the nail detection result may be linearly interpolated. Still taking fig. 2 as an example, for symmetric key points, such as P5 and P15, P7 and P13, etc., by performing linear interpolation, the middle region of the nail can also have the first key point, which is beneficial to the subsequent construction of triangular mesh according to the first key point.
In one example, symmetric keypoints may be linearly interpolated while adjacent keypoints of the nail in the nail detection result are curve interpolated, so that the first keypoints located at the edge of the nail are more densely arranged and have the first keypoints in the middle region of the nail.
Since the area of the nail material is usually larger than that of the real nail, the embodiment of the disclosure provides a method for deforming the key point, and the deformed area to be processed is slightly longer than the real nail, so as to obtain a special effect and a nail beautifying effect which are closer to the real nail.
Firstly, determining a target key point in a plurality of first key points of the fingernail, wherein the target key point is a first key point between a connecting line between the set symmetrical key points and the first key point corresponding to the top end of the fingertip. The target key points comprise the set symmetrical key points and first key points corresponding to the top ends of the fingertips.
In this step, the determined target keypoint is also the first keypoint to be deformed subsequently. Taking the first keypoint shown in fig. 2 as an example, the first keypoint located on the upper half of the nail may be determined as the target keypoint. For example, all first keypoints between a connecting line between P5 and P15 and P10 corresponding to the top end, including P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, and P15, may be used as target keypoints.
Next, the target key point is moved by a set distance in a normal direction of the nail, that is, in a growth direction of the nail. Wherein the normal direction is determined from a plurality of first keypoints of the nail.
In one example, a direction of a line connecting first key points located at a lowermost middle point and first key points located at a uppermost middle point among the first key points of the nail may be taken as a normal direction of the nail. As shown in fig. 2, a line connecting the first key point P2 and P10 may be taken as a normal direction of the nail.
The set distance of each target key point moving along the normal direction of the nail may be set by the user, or may be determined according to the position relationship between the target key point and the other first key points and the position information.
In one example, the distance may be determined according to a first distance between the target key point and the connection line, a second distance between the first key point corresponding to the fingertip tip and the connection line, and a length of the connection line, wherein the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connection line.
Taking the example of calculating the set distance corresponding to the target keypoint P8 in fig. 2, the set distance can be determined according to a first distance l3 between the P8 and the line connecting the P5 and the P15, a second distance l2 between the P10 and the line connecting the P5 and the P15, and a length l1 of the line connecting the P5 and the P15. The formula of the set distance Δ m is, for example:
Figure BDA0003134511100000101
the a is a set coefficient and can be adjusted according to the required effect, and the value range of the a can be 0.1-0.4. The setting coefficients of the target key points may be the same or different. For example, the setting coefficient a of all the target key points may be set to 0.2.
And finally, updating the first key point by using the moved target key point. That is, the updated first keypoint is obtained according to the moved target keypoint and the first keypoint other than the target keypoint in the first keypoint.
In the embodiment of the disclosure, according to the target key point moving along the normal direction of the nail, the to-be-processed area corresponding to the nail is deformed, and the length of the to-be-processed area is increased in the normal direction compared with the length of the real nail. The deformed area to be processed is more consistent and matched with the shape of common nail materials, so that the nail special effect and nail beautifying effect which are more approximate to reality can be obtained.
In some embodiments, the rendering of nail material to the area to be processed may be accomplished in the following manner.
Firstly, acquiring texture coordinates in the nail material corresponding to the coordinate point to be processed contained in the area to be processed according to the corresponding relation between the first key point and the second key point. The coordinate points to be processed may include a first key point and other coordinate points obtained based on the first key point, and the position relationship between the other coordinate points and the first key point is known.
And then, acquiring texture coordinates in the nail material corresponding to the coordinate point to be processed according to the corresponding relation between the first key point and the second key point, and acquiring a first pixel value of the nail material at the texture coordinates.
All or part of the first key points of the fingernails correspond to the plurality of second key points of the fingernail material one by one, so that the texture coordinates of the second key points corresponding to the first key points in the fingernail material can be determined according to the texture coordinates of the second key points corresponding to the first key points in the fingernail material, and corresponding first pixel values can be obtained according to the texture coordinates.
For other coordinate points obtained according to the first key point, because the relative position relationship between the other coordinate points and the first key point is known, texture coordinates in the nail material corresponding to the other coordinate points can be determined according to the corresponding relationship between the first key point and the second key point, and corresponding first pixel values can be obtained according to the texture coordinates.
By the method, the first pixel value in the nail material corresponding to each coordinate point to be processed in the area to be processed of the nail can be obtained, so that the color and/or the pattern of the nail material can be rendered in the area to be processed, and special effect processing and beautifying effect processing of the nail can be realized.
In the embodiment of the disclosure, the rendering of the nail material to the area to be processed is realized through the corresponding relationship between the first key point and the second key point, and the nail material can be adaptively changed according to the area to be processed, so that a vivid nail special effect and a beautification effect are obtained.
In some implementation manners, the coordinate point to be processed includes a first key point and a first interpolation point, where the first interpolation point is located in the region to be processed and is obtained by performing interpolation according to the first key point. That is, each coordinate point to be processed included in the region to be processed may be obtained by performing interpolation processing on the first key point.
In some embodiments, a triangular mesh may be constructed from the plurality of first keypoints of the nail, and a region to be treated of the nail may be determined from the constructed triangular mesh. The first key point can be obtained by interpolating the detection result output by the nail key point detection network, and the edge key points positioned on the nail outline are more dense by interpolating the detection result, so that the region to be processed formed by the constructed triangular mesh keeps the round shape of the nail and is more consistent with the shape of the real nail.
In a case where a triangular mesh is constructed according to the plurality of first key points of the nail, the coordinate point to be processed may include a vertex of the triangular mesh and a second interpolation point, where the second interpolation point is located in the triangular mesh and is obtained by interpolating according to the vertex of the triangular mesh, and the triangular mesh is constructed according to the plurality of first key points of the nail. That is, the vertices of the triangular mesh are formed by the first keypoints, and the second interpolation points located in the triangular mesh are obtained by interpolating the first keypoints.
In some implementations, a second pixel value of the coordinate point to be processed in the image to be processed, that is, a color of a nail in the image to be processed, may also be obtained. And then, according to a set fusion coefficient, carrying out weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed to obtain a target pixel value of the coordinate point to be processed, and finally obtaining the processed image according to the target pixel value of the coordinate point to be processed. The above process of fusion can be expressed according to formula (2):
Cnew＝C1*s1+C2*(1-s1) (2)
wherein, CnewRepresenting the target pixel value, C1Representing a first pixel value, C2Representing a second pixel value, s1Indicating a set fusion coefficient that may be specifically set by the user as desired for the implementation. In the embodiment of the present disclosure, the color of the nail material is fused with the color of the nail itself in the image to be processed, so that the nail effect and the beautifying effect are more natural and closer to the real nail beautifying effect, as shown in fig. 4A and 4B.
In some embodiments, the hand image may be image processed in the following manner to achieve a nail beautification effect of the hand image. As shown in FIG. 5, the method may include steps 501-506.
In step 501, a hand image is acquired, and a plurality of first key points corresponding to respective nails in the hand image are acquired. Wherein, the hand image can be collected in real time or acquired from a storage medium.
In step 502, the first keypoints are interpolated, and the first keypoints are updated by using new first keypoints obtained by interpolation.
The interpolation of the first key points comprises curve interpolation of adjacent key points in the first key points and/or linear interpolation of symmetrical key points in the first key points.
In step 503, the first key is deformed so that the nail region corresponding to the deformed first key is longer than the length of the nail in the hand image, and the first key is updated by using the deformed first key.
In this step, the first key point located on the upper half of the nail may be moved by a set distance in the normal direction of the nail. Wherein, the set distance moved by each first keypoint can be calculated according to formula (1).
In step 504, a triangular mesh is constructed according to the first key point, and a region to be processed, in which a nail art effect needs to be achieved, is obtained according to the triangular mesh.
In step 505, the color of the pixel of the nail material is extracted according to the texture coordinate on the nail material corresponding to the first key point.
In this step, a second key point corresponding to the first key point may be marked on the nail material, and a texture coordinate on the nail material corresponding to the first key is obtained according to a correspondence between the first key point and the second key point; texture coordinates on the nail material corresponding to the first key can also be obtained according to other modes.
In step 506, according to the set fusion coefficient, the color of the nail material corresponding to the first key point and the color of the original pixel in the hand image are fused to obtain a new color. Specifically, the fusion can be performed only in the manner indicated by formula (2).
In this step, a new color corresponding to each coordinate point in the region to be processed can be obtained by interpolating the first key point, so that the nail art effect on the hand image is realized.
In the embodiment of the disclosure, interpolation and deformation processing are performed on the first key points corresponding to the fingernails, so that the to-be-processed area can keep the round and smooth shape of the fingernails and is more consistent with the shape of the real fingernails; and the color of the nail material is fused with the color of the original pixel in the hand image, so that the realized nail beautifying effect is more natural and is closer to the real nail beautifying effect.
Fig. 6 illustrates a schematic structural diagram of an image processing apparatus according to at least one embodiment of the present disclosure, and as shown in fig. 6, the apparatus may include: a first acquiring unit 601, configured to acquire first key points of at least one nail in an image to be processed, where each nail corresponds to a plurality of first key points; a determining unit 602, configured to determine a region to be processed of the nail according to a plurality of first key points of the nail; a second obtaining unit 603, configured to obtain a nail material, where a plurality of second key points are marked on the nail material; a rendering unit 604, configured to render the nail material into the to-be-processed region according to a corresponding relationship between the first key point and the second key point, so as to obtain a processed image.
In some embodiments, the first obtaining unit is specifically configured to: inputting the image to be processed into a nail key point detection network to obtain a nail key point detection result; and obtaining a first key point of the nail according to the key point detection result of the nail, or carrying out interpolation processing on the key point detection result of the nail and obtaining the first key point of the nail according to the interpolated result.
In some embodiments, the interpolating the key point detection result of the nail includes: performing curve interpolation on adjacent key points in the key point detection result of the fingernail; and/or; and carrying out linear interpolation on symmetrical key points in the key point detection result of the fingernail.
In some embodiments, the apparatus further comprises a deformation unit for: determining a target key point in a plurality of first key points of the fingernail, wherein the target key point is a first key point between a connecting line between set symmetrical key points and the first key point corresponding to the top end of the fingertip; moving the target keypoint by a set distance along a normal direction of the nail, wherein the normal direction is determined according to a plurality of first keypoints of the nail; and updating the first key point by using the moved target key point.
In some embodiments, the set distance is determined according to a first distance between the target key point and the connection line, a second distance between the first key point corresponding to the fingertip tip and the connection line, and a length of the connection line, wherein the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connection line.
In some embodiments, the rendering unit is specifically configured to: acquiring texture coordinates in the nail material corresponding to a coordinate point to be processed according to the corresponding relation between the first key point and the second key point, wherein the coordinate point to be processed is located in the area to be processed; acquiring a first pixel value of the nail material at the texture coordinate; and obtaining the processed image according to the first pixel value of the coordinate point to be processed.
In some embodiments, the coordinate points to be processed include: the processing method comprises the steps of obtaining a first key point and a first interpolation point, wherein the first interpolation point is located in the region to be processed and is obtained by interpolation according to the first key point; and/or, the vertex of the triangular mesh and a second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertex of the triangular mesh, and the triangular mesh is constructed according to a plurality of first key points of the fingernail.
In some embodiments, the apparatus further comprises a fusion unit for: according to a set fusion coefficient, carrying out weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed to obtain a target pixel value of the coordinate point to be processed; and obtaining the processed image according to the target pixel value of the coordinate point to be processed. At least one embodiment of the present disclosure further provides an electronic device, as shown in fig. 7, the device includes a memory for storing computer instructions executable on a processor, and the processor is configured to implement the image processing method according to any one of the embodiments of the present disclosure when executing the computer instructions.
At least one embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the image processing method according to any one of the embodiments of the present disclosure.
At least one embodiment of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the image processing method according to any one of the embodiments of the present disclosure.
As will be appreciated by one skilled in the art, one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the data processing apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Embodiments of the subject matter and the functional operations described in this specification can be implemented in: digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a tangible, non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode and transmit information to suitable receiver apparatus for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASI C (application-specific integrated circuit).
Computers suitable for executing computer programs include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and/or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Moreover, a computer may be embedded in another device, e.g., a mobile telephone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash drive, to name a few.
Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., an internal hard disk or a removable disk), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. In other instances, features described in connection with one embodiment may be implemented as discrete components or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Further, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.
The above description is only for the purpose of illustrating the preferred embodiments of the one or more embodiments of the present disclosure, and is not intended to limit the scope of the one or more embodiments of the present disclosure, and any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the one or more embodiments of the present disclosure should be included in the scope of the one or more embodiments of the present disclosure.

Claims (12)

1. An image processing method, characterized in that the method comprises:
acquiring first key points of the fingernails in an image to be processed containing at least one fingernail, wherein each fingernail corresponds to a plurality of first key points;
determining a region of the nail to be treated according to a plurality of first key points of the nail;
acquiring a nail material, wherein a plurality of second key points are marked on the nail material;
and rendering the nail material into the area to be processed according to the corresponding relation between the first key point and the second key point to obtain a processed image.
2. The method of claim 1, wherein said obtaining a first keypoint of at least one nail of an image to be processed containing said nail comprises:
inputting the image to be processed into a nail key point detection network to obtain a nail key point detection result;
and obtaining a first key point of the nail according to the key point detection result of the nail, or carrying out interpolation processing on the key point detection result of the nail and obtaining the first key point of the nail according to the interpolated result.
3. The method according to claim 2, wherein the interpolating the key point detection result of the nail includes:
performing curve interpolation on adjacent key points in the key point detection result of the fingernail; and/or;
and carrying out linear interpolation on symmetrical key points in the key point detection result of the fingernail.
4. The method according to any one of claims 1 to 3, further comprising:
determining a target key point in a plurality of first key points of the fingernail, wherein the target key point is a first key point between a connecting line between set symmetrical key points and the first key point corresponding to the top end of the fingertip;
moving the target keypoint by a set distance along a normal direction of the nail, wherein the normal direction is determined according to a plurality of first keypoints of the nail;
and updating the first key point by using the moved target key point.
5. The method according to claim 4, wherein the set distance is determined according to a first distance between the target key point and the connection line, a second distance between the first key point corresponding to the fingertip tip and the connection line, and a length of the connection line, wherein the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connection line.
6. The method according to any one of claims 1 to 5, wherein the rendering the nail material into the region to be processed according to the correspondence between the first key point and the second key point to obtain a processed image comprises:
acquiring texture coordinates in the nail material corresponding to a coordinate point to be processed according to the corresponding relation between the first key point and the second key point, wherein the coordinate point to be processed is located in the area to be processed;
acquiring a first pixel value of the nail material at the texture coordinate;
and obtaining the processed image according to the first pixel value of the coordinate point to be processed.
7. The method according to claim 6, wherein the coordinate points to be processed comprise:
the processing method comprises the steps of obtaining a first key point and a first interpolation point, wherein the first interpolation point is located in the region to be processed and is obtained by interpolation according to the first key point; and/or the presence of a gas in the gas,
the nail identification method comprises vertexes of a triangular mesh and a second interpolation point, wherein the second interpolation point is located in the triangular mesh and obtained by interpolation according to the vertexes of the triangular mesh, and the triangular mesh is constructed according to a plurality of first key points of the nail.
8. The method according to claim 6 or 7, characterized in that the method further comprises:
acquiring a second pixel value of the coordinate point to be processed in the image to be processed;
the obtaining the processed image according to the first pixel value includes:
according to a set fusion coefficient, carrying out weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed to obtain a target pixel value of the coordinate point to be processed;
and obtaining the processed image according to the target pixel value of the coordinate point to be processed.
9. An image processing apparatus, characterized in that the apparatus comprises:
a first acquisition unit, configured to acquire first key points of at least one nail in an image to be processed, wherein each of the nails corresponds to a plurality of the first key points;
a determination unit configured to determine a region to be treated of the nail based on a plurality of first key points of the nail;
the second acquisition unit is used for acquiring a nail material, and a plurality of second key points are marked on the nail material;
and the rendering unit is used for rendering the nail material into the to-be-processed area according to the corresponding relation between the first key point and the second key point to obtain a processed image.
10. An electronic device, comprising a memory for storing computer instructions executable on a processor, the processor being configured to implement the method of any one of claims 1 to 8 when executing the computer instructions.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the program, when executed by a processor, implements the method of any one of claims 1 to 8.
CN202110713524.5A 2021-06-25 2021-06-25 Image processing method, device, equipment and storage medium Pending CN113222993A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202110713524.5A CN113222993A (en) 2021-06-25 2021-06-25 Image processing method, device, equipment and storage medium
PCT/CN2021/140353 WO2022267402A1 (en) 2021-06-25 2021-12-22 Image processing method and apparatus, device, and storage medium
TW110148370A TW202301179A (en) 2021-06-25 2021-12-23 Image processing methods, apparatuses, devices and storage media

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110713524.5A CN113222993A (en) 2021-06-25 2021-06-25 Image processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113222993A true CN113222993A (en) 2021-08-06

Family

ID=77081408

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110713524.5A Pending CN113222993A (en) 2021-06-25 2021-06-25 Image processing method, device, equipment and storage medium

Country Status (3)

Country Link
CN (1) CN113222993A (en)
TW (1) TW202301179A (en)
WO (1) WO2022267402A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114880057A (en) * 2022-04-22 2022-08-09 北京三快在线科技有限公司 Image display method, image display device, terminal, server, and storage medium
WO2022267402A1 (en) * 2021-06-25 2022-12-29 北京市商汤科技开发有限公司 Image processing method and apparatus, device, and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047526A (en) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 Image processing method and device, electronic equipment and storage medium
CN112150580A (en) * 2019-06-28 2020-12-29 腾讯科技（深圳）有限公司 Image processing method and device, intelligent terminal and storage medium
CN112767288A (en) * 2021-03-19 2021-05-07 北京市商汤科技开发有限公司 Image processing method and device, electronic equipment and storage medium
CN112991358A (en) * 2020-09-30 2021-06-18 北京字节跳动网络技术有限公司 Method for generating style image, method, device, equipment and medium for training model

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10976829B1 (en) * 2019-06-03 2021-04-13 Facebook, Inc. Systems and methods for displaying augmented-reality objects
CN112183388A (en) * 2020-09-30 2021-01-05 北京字节跳动网络技术有限公司 Image processing method, apparatus, device and medium
CN113222993A (en) * 2021-06-25 2021-08-06 北京市商汤科技开发有限公司 Image processing method, device, equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112150580A (en) * 2019-06-28 2020-12-29 腾讯科技（深圳）有限公司 Image processing method and device, intelligent terminal and storage medium
CN111047526A (en) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 Image processing method and device, electronic equipment and storage medium
CN112991358A (en) * 2020-09-30 2021-06-18 北京字节跳动网络技术有限公司 Method for generating style image, method, device, equipment and medium for training model
CN112767288A (en) * 2021-03-19 2021-05-07 北京市商汤科技开发有限公司 Image processing method and device, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022267402A1 (en) * 2021-06-25 2022-12-29 北京市商汤科技开发有限公司 Image processing method and apparatus, device, and storage medium
CN114880057A (en) * 2022-04-22 2022-08-09 北京三快在线科技有限公司 Image display method, image display device, terminal, server, and storage medium

Also Published As

Publication number Publication date
WO2022267402A1 (en) 2022-12-29
TW202301179A (en) 2023-01-01

Similar Documents

Publication Publication Date Title
US10043308B2 (en) Image processing method and apparatus for three-dimensional reconstruction
CN108875523B (en) Human body joint point detection method, device, system and storage medium
CN110072046B (en) Image synthesis method and device
CN109754396B (en) Image registration method and device, computer equipment and storage medium
KR100997616B1 (en) Rolled fingerprint acquisiton apparatus and fingerprint acquisiton method using registration and synthesis
CN113222993A (en) Image processing method, device, equipment and storage medium
CN110287775B (en) Palm image clipping method, palm image clipping device, computer equipment and storage medium
CN109583509B (en) Data generation method and device and electronic equipment
CN107644423B (en) Scene segmentation-based video data real-time processing method and device and computing equipment
CN107784321B (en) Method and system for quickly identifying digital picture books and computer readable storage medium
CN109978077B (en) Visual recognition method, device and system and storage medium
CN107590811B (en) Scene segmentation based landscape image processing method and device and computing equipment
CN107563357B (en) Live-broadcast clothing dressing recommendation method and device based on scene segmentation and computing equipment
CN110619334B (en) Portrait segmentation method based on deep learning, architecture and related device
CN107766803B (en) Video character decorating method and device based on scene segmentation and computing equipment
CN111357034A (en) Point cloud generation method, system and computer storage medium
CN107481280A (en) The antidote and computing device of a kind of skeleton point
CN110084309A (en) Characteristic pattern amplification method, device and equipment and computer readable storage medium
CN111199169A (en) Image processing method and device
CN110910326B (en) Image processing method and device, processor, electronic equipment and storage medium
CN108960012B (en) Feature point detection method and device and electronic equipment
JP4339221B2 (en) Image construction method, fingerprint image construction apparatus and program
CN113298870A (en) Object posture tracking method and device, terminal equipment and storage medium
CN110858409A (en) Animation generation method and device
CN110728172B (en) Point cloud-based face key point detection method, device and system and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40049968

Country of ref document: HK