WO2022267402A1

WO2022267402A1 - Image processing method and apparatus, device, and storage medium

Info

Publication number: WO2022267402A1
Application number: PCT/CN2021/140353
Authority: WO
Inventors: 苏柳
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2021-06-25
Filing date: 2021-12-22
Publication date: 2022-12-29
Also published as: CN113222993A; TW202301179A

Abstract

Disclosed are an image processing method and apparatus, a device, and a storage medium. The method comprises: obtaining first key points of a fingernail in an image to be processed comprising at least one fingernail, wherein each fingernail corresponds to a plurality of first key points; according to the plurality of first key points of the fingernail, determining an area to be processed of the fingernail; obtaining a fingernail material on which a plurality of second key points is marked; and according to the correspondences between the first key points and the second key points, rendering the fingernail material into said area to obtain a processed image.

Description

Image processing method, device, device and storage medium

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application with the application number 202110713524.5 and the title of the invention "image processing method, device, equipment and storage medium" submitted on June 25, 2021. The entire content of the Chinese patent application is disclosed in Incorporated herein by reference.

technical field

The present disclosure relates to the field of computer technology, and in particular to an image processing method, device, equipment and storage medium.

Background technique

At present, most of the beautification or beautification processing of images or videos taken by users focuses on the face, but lacks beautification processing on body parts, such as hands, such as manicure operations. This is due to the fact that the hand nail area covers a small area and is difficult to detect. It is difficult to accurately apply nail materials to the nail area to achieve a better beautification effect.

Contents of the invention

The embodiment of the present disclosure provides an image processing solution.

According to an aspect of the present disclosure, there is provided an image processing method, the method comprising: acquiring a first key point of the nail in a to-be-processed image containing at least one nail, wherein each of the nails corresponds to a plurality of first key points A key point; according to a plurality of first key points of the nail, determine the area to be treated of the nail; obtain a nail material, and the nail material is marked with a plurality of second key points; according to the first key point and the corresponding relationship between the second key point, the nail material is rendered into the region to be processed, and a processed image is obtained.

In combination with any embodiment provided by the present disclosure, the acquiring the first key point of the nail in the image to be processed including at least one nail includes: inputting the image to be processed into a nail key point detection network to obtain the nail The key point detection result of the nail; the first key point of the nail is obtained according to the key point detection result of the nail, or, the key point detection result of the nail is interpolated, and the nail is obtained according to the interpolated result The first key point.

In combination with any embodiment provided in the present disclosure, the interpolation processing of the key point detection result of the nail includes: performing curve interpolation on adjacent key points in the key point detection result of the nail; and/or; Linear interpolation is performed on the symmetrical key points in the key point detection results of the nail.

In combination with any implementation manner provided by the present disclosure, the method further includes: determining a target key point among the plurality of first key points of the nail, and the target key point is setting a connection line between the symmetrical key points and The first key point between the first key points corresponding to the top of the nail or the top of the fingertip; move the target key point along the normal direction of the nail for a set distance, wherein the normal direction is based on the Multiple first key points of the nail are determined; the corresponding first key points are updated by using the moved target key points.

In combination with any implementation manner provided by the present disclosure, the set distance corresponds to the first distance between the target key point and the line between the set symmetrical key point, the top of the nail or the top of the fingertip. The second distance between the first key point and the connecting line, and the length of the connecting line are determined, wherein the set distance is positively correlated with the first distance and negatively correlated with the second distance , which is negatively correlated with the length of the link.

In combination with any implementation manner provided in the present disclosure, according to the correspondence between the first key point and the second key point, the nail material is rendered into the area to be processed, and the processed The image includes: according to the correspondence between the first key point and the second key point, acquiring the texture coordinates in the nail material corresponding to the coordinate points to be processed, wherein the coordinate points to be processed are located at the within the area to be processed; acquire the first pixel value of the nail material at the texture coordinate; and obtain the processed image according to the first pixel value of the coordinate point to be processed.

In combination with any implementation manner provided by the present disclosure, the coordinate points to be processed include: a first key point and a first interpolation point, wherein the first interpolation point is located in the area to be processed and is based on the first obtained by interpolating the key points; and/or, the vertices of the triangular mesh and the second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertices of the triangular mesh Yes, the triangular mesh is constructed according to a plurality of first key points of the nail.

In combination with any implementation manner provided by the present disclosure, the method further includes: obtaining a second pixel value of the coordinate point to be processed in the image to be processed; obtaining the processed pixel value according to the first pixel value The image includes: according to the set fusion coefficient, weighting the first pixel value and the second pixel value of the coordinate point to be processed to obtain the target pixel value of the coordinate point to be processed; according to the coordinate point to be processed The target pixel value of the coordinate point is processed to obtain the processed image.

According to an aspect of the present disclosure, there is provided an image processing device, the device comprising: a first acquisition unit, configured to acquire a first key point of the nail in an image to be processed containing at least one nail, wherein each of the The nail corresponds to a plurality of first key points; the determination unit is configured to determine the area to be processed of the nail according to the plurality of first key points of the nail; the second acquisition unit is configured to acquire a nail material, the A plurality of second key points are marked on the nail material; a rendering unit is configured to render the nail material into the area to be processed according to the correspondence between the first key point and the second key point , to get the processed image.

In combination with any implementation manner provided in the present disclosure, the first acquisition unit is specifically configured to: input the image to be processed into the nail key point detection network to obtain the key point detection result of the nail; according to the key point detection result of the nail Obtaining the first key point of the nail, or performing interpolation processing on the detection result of the key point of the nail, and obtaining the first key point of the nail according to the interpolated result.

In combination with any implementation manner provided by the present disclosure, the device further includes a deformation unit configured to: determine a target key point among the multiple first key points of the nail, and the target key point is one of the set symmetrical key points The first key point between the connecting line and the first key point corresponding to the top of the nail or fingertip; move the target key point along the normal direction of the nail for a set distance, wherein the normal The direction is determined according to a plurality of first key points of the nail; the corresponding first key points are updated by using the moved target key points.

In combination with any of the implementations provided in the present disclosure, the rendering unit is specifically configured to: acquire, according to the correspondence between the first key point and the second key point, the nail material corresponding to the coordinate point to be processed. texture coordinates, wherein, the coordinate point to be processed is located in the area to be processed; the first pixel value of the nail material at the texture coordinate is acquired; according to the first pixel value of the coordinate point to be processed, the The processed image.

In combination with any embodiment provided in the present disclosure, the device further includes a fusion unit configured to: perform weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed according to a set fusion coefficient, The target pixel value of the coordinate point to be processed is obtained; the processed image is obtained according to the target pixel value of the coordinate point to be processed.

According to an aspect of the present disclosure, there is provided an electronic device, the device includes a memory and a processor, the memory is used to store computer instructions executable on the processor, and the processor is used to execute the computer instructions The image processing method described in any implementation manner provided by the present disclosure is realized.

According to an aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the image processing method described in any implementation manner provided in the present disclosure is implemented.

According to an aspect of the present disclosure, a computer program product is provided, including a computer program, and when the program is executed by a processor, the image processing method described in any implementation manner provided in the present disclosure is implemented.

According to the image processing method, device, device and computer-readable storage medium of one or more embodiments of the present disclosure, first, the first key point of the nail in the image to be processed including at least one nail is obtained, and according to the number of the nails A first key point, determine the area to be processed of the nail; then obtain the nail material, the nail material is marked with a plurality of second key points, and finally according to the first key point and the second key point The corresponding relationship between them, the nail material is rendered into the area to be processed, and the processed image is obtained. By accurately rendering the nail material to the area to be processed of the nail in the image to be processed, the Special effect treatment and beautification effect treatment of nails.

Description of drawings

In order to more clearly illustrate the technical solutions of one or more embodiments of this specification, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only one or more For some embodiments described in the multiple embodiments, for those skilled in the art, other drawings can also be obtained according to these drawings on the premise of not paying creative efforts.

FIG. 1 is a flowchart of an image processing method proposed by at least one embodiment of the present disclosure;

Fig. 2 is a schematic diagram of the first key point of the nail in the image processing method proposed by at least one embodiment of the present disclosure;

Fig. 3 is a schematic diagram of nail material in the image processing method proposed by at least one embodiment of the present disclosure;

4A and 4B are schematic diagrams of hand images obtained through the image processing method proposed by at least one embodiment of the present disclosure;

FIG. 5 is a flow chart of a method for image processing a hand image proposed by at least one embodiment of the present disclosure;

Fig. 6 is a schematic structural diagram of an image processing device proposed by at least one embodiment of the present disclosure;

Fig. 7 is a schematic structural diagram of an electronic device proposed by at least one embodiment of the present disclosure.

detailed description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

At least one embodiment of the present disclosure provides an image processing method, the image processing method can be executed by electronic devices such as terminal devices or servers, and the terminal devices can be fixed terminals or mobile terminals, such as mobile phones, tablet computers, game consoles , desktop computer, advertising machine, all-in-one machine, vehicle-mounted terminal, etc., the server includes a local server or a cloud server, etc., and the method can also be implemented by calling a computer-readable instruction stored in a memory by a processor.

FIG. 1 shows a flowchart of an image processing method according to at least one embodiment of the present disclosure. As shown in FIG. 1 , the method includes steps 101 to 104 .

In step 101, a first key point of the nail in an image to be processed containing at least one nail is acquired, wherein each of the nails corresponds to a plurality of first key points.

Wherein, the image to be processed may be a still image or a video image captured in real time, or may be a still image or a video image acquired from a memory or other media. Moreover, the nails in the embodiments of the present disclosure may be the nails of the hand or the nails of the feet, which is not limited in the present disclosure. Taking the nail as the nail of the hand as an example, the image to be processed can be a separate hand image, or a human body image including the hand area; it can also be a partial hand including one or more nails image.

In an example, key point detection may be performed on the image to be processed to obtain key points of at least one nail contained in the image to be processed, wherein each nail corresponds to multiple key points. In order to distinguish the key points obtained from the image to be processed from the key points obtained subsequently, the key points obtained in this step are called the first key points.

The first key points are usually located on the edge of the nail, and each first key point is used to represent a specific position of the nail. In general, one nail can correspond to any number of first key points within the range of 12-32. Taking the schematic diagram of the first key point shown in FIG. 2 as an example, one fingernail corresponds to 16 first key points. As shown in Figure 2, the first key point P0 is used to characterize the leftmost point at the bottom of the nail outline, the first key point P4 is used to characterize the rightmost point at the bottom of the nail outline, and the first key point P10 was used to characterize the point located in the topmost middle of the nail outline. Those skilled in the art should understand that the nails shown in FIG. 2 corresponding to 16 first key points are only for example, and one nail may also correspond to other numbers of first key points, which is not limited in the present disclosure.

In step 102, a region to be treated of the nail is determined according to a plurality of first key points of the nail.

In the embodiment of the present disclosure, since the first key point is located on the edge of the nail, the region to be treated of the nail can be obtained according to the plurality of first key points corresponding to the nail.

In step 103, the nail material is acquired, and the nail material is marked with a plurality of second key points.

Wherein, the nail material is a material that can add special effects or beautify the nails (for example, a marked picture of the nail, etc.), and the nail material can be a solid color, or it can contain a pattern, or it can be It is other types, and the embodiment of the present disclosure does not limit the type of nail material.

In the embodiment of the present disclosure, the second key point is marked on the nail material. Similar to the first key point, the second key point is also located on the edge of the nail material, and each second key point is used to characterize a specific point of the nail material. In an embodiment of the present disclosure, all or part of the first key points of the nail correspond one-to-one to multiple second key points of the nail material. For example, the nail material includes 16 second key points, and the nail in the image to be processed also includes 16 first key points, and the 16 second key points are in one-to-one correspondence with the 16 first key points. For another example, the nail material includes 12 second key points, and the nail in the image to be processed includes 16 first key points, and the 12 second key points are the same as the 12th key points in the above-mentioned 16 first key points. There is a one-to-one correspondence between key points. Wherein, the nail material may correspond to any number of second key points within the range of 12-32.

As shown in FIG. 3 , the second key points S0 - S15 are marked on the nail material. Wherein, the second key points S0-S15 are in one-to-one correspondence with the first key points P0-P15 of the nail in the image to be processed.

In the embodiment of the present disclosure, the neural network used to detect nail key points on the nail material and the neural network used to detect nail key points on the image to be processed may be the same neural network, or use the same The neural network obtained by training the sample set.

In step 104, according to the corresponding relationship between the first key point and the second key point, the nail material is rendered into the region to be processed to obtain a processed image.

For one of the nails in the image to be processed, after obtaining the first key point corresponding to the nail, according to the relationship between the multiple first key points of the nail and the multiple second key points of the nail material The corresponding relationship between the to-be-processed area of the nail can be determined and the corresponding relationship between the nail material can be determined, so that the color and/or pattern of the nail material can be rendered into the to-be-processed area of the nail, so that all The nails present the same color and/or pattern as the nail material, thereby realizing the special effect treatment of the nails or the beautification treatment of the nails.

In an embodiment of the present disclosure, the first key point of the nail in the image to be processed including at least one nail is first obtained, and the area to be processed of the nail is determined according to the plurality of first key points of the nail; then Acquiring nail material, the nail material is marked with a plurality of second key points, and finally according to the correspondence between the first key point and the second key point, rendering the nail material to the pending In the region, the processed image is obtained, and by accurately rendering the nail material to the region to be processed of the nail in the image to be processed, special effect processing and beautification effect processing of the nail in the image to be processed can be realized.

In the embodiment of the present disclosure, the image to be processed may be input into a key point detection network, so as to perform key point detection on the image to be processed.

In some implementation manners, the first key point of the nail may be obtained according to the nail detection result output by the key point detection network.

In some implementation manners, an interpolation process may be performed on the nail detection result output by the nail key detection network, and the first key point of the nail is obtained according to the interpolated result.

In an example, curve interpolation may be performed on adjacent key points of the nail in the nail detection result. Taking Figure 2 as an example, for adjacent key points, such as P7 and P8, P8 and P9, etc., by performing curve interpolation, the key points on the edge of the nail can be arranged more densely, so that the key points are surrounded by The area to be treated maintains the round shape of the nail, which is more consistent with the shape of the real nail.

In an example, linear interpolation may be performed on the symmetric key points of the nail in the nail detection result. Still taking Figure 2 as an example, for symmetrical key points, such as P5 and P15, P7 and P13, etc., by performing linear interpolation, the middle area of the nail (non-nail edge) can also have the first key point, which is beneficial to Subsequently, a triangular mesh is constructed according to the first key point.

In an example, linear interpolation may be performed on the symmetrical key points while curve interpolation is performed on the adjacent key points of the nail in the nail detection result, so that the arrangement of the first key points on the edge of the nail is denser, and There is also a first key in the middle area of the nail.

Since the nail contained in the image to be processed represents the real nail, and the area of the nail material is usually larger than the area of the real nail, the embodiment of the present disclosure proposes a method for deforming the key points. The area is slightly longer than the real nail, resulting in more realistic nail effects and nail beautification effects.

First, determine the target key point among the plurality of first key points of the nail, the target key point is between the line between the symmetrical key points and the first key point corresponding to the top of the nail or fingertip the first key point. Wherein, the target key point includes the set symmetry key point and the first key point corresponding to the top of the nail or fingertip.

In this step, the determined target key point is also the first key point for subsequent deformation. Taking the first key point shown in FIG. 2 as an example, the first key point located in the upper half of the nail may be determined as the target key point. For example, all the first key points between the line between P5 and P15 and the corresponding P10 at the top, including P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, and P15, can be used as target key points.

Next, the target key point is moved by a set distance along the normal direction of the nail, that is, along the growth direction of the nail by a set distance. Wherein, the normal direction is determined according to a plurality of first key points of the nail.

In an example, among the first key points of the nail, the direction of the line between the first key point at the bottommost middle and the first key point at the topmost middle can be used as the normal direction of the nail. line direction. As shown in FIG. 2 , the direction of the line connecting the first key point P2 and P10 may be taken as the normal direction of the nail.

The set distance for each target key point to move along the normal direction of the nail can be set by the user, or can be determined according to the positional relationship and position information between the target key point and other first key points.

In an example, according to the first distance between the target key point and the line between the set symmetrical key point, the first key point corresponding to the top of the nail or the top of the fingertip and the line The second distance between the lines and the length of the connecting line are determined, wherein the set distance is positively correlated with the first distance, negatively correlated with the second distance, and negatively correlated with the length of the connecting line relevant.

Taking the calculation of the set distance corresponding to the target key point P8 in Figure 2 as an example, it can be based on the first distance l3 between P8 and the line connecting P5 and P15, the second distance l2 between P10 and the line connecting P5 and P15, and P5 The length l1 of the connection with P15 is determined. The formula of the set distance Δm is, for example:

Wherein, a is a setting coefficient, which can be adjusted according to the required effect, and the value range of a can be 0.1-0.4. The setting coefficients of each target key point can be the same or different. For example, the setting coefficient a of all target keypoints can be set to 0.2.

The corresponding first key point is updated by using the moved target key point. That is, an updated first key point is obtained according to the moved target key point and first key points other than the target key point.

Finally, the area to be processed of the nail is updated according to the updated first key point of the nail.

In the embodiment of the present disclosure, according to moving the target key point along the normal direction of the nail, the area to be treated corresponding to the nail is deformed, and the length of the normal direction of the nail is increased compared with that of the real nail. The deformed area to be processed is more consistent and matches the shape of common nail materials, so that the nail special effects and nail beautification effects closer to reality can be obtained.

In some embodiments, the rendering of the nail material to the area to be processed may be implemented in the following manner.

Firstly, according to the correspondence between the first key point and the second key point, the texture coordinates in the nail material corresponding to the coordinate points to be processed included in the area to be processed are acquired. Wherein, the coordinate points to be processed may include the first key point and other coordinate points obtained based on the first key point, and the positional relationship between the other coordinate points and the first key point is known. The size of the nail material is usually fixed, and the texture coordinates of the key points in the nail material can be stored in advance.

Next, according to the correspondence between the first key point and the second key point, the texture coordinates in the nail material corresponding to the coordinate points to be processed are obtained, and all the texture coordinates at the texture coordinates are obtained. Describes the first pixel value of the nail material.

Since all or part of the first key point of the nail is in one-to-one correspondence with multiple second key points of the nail material, according to the second key point corresponding to the first key point in the nail texture coordinates in the material, then the texture coordinates in the nail material corresponding to the first key point can be determined, and the corresponding first pixel value can be obtained according to the texture coordinates.

For other coordinate points obtained according to the first key point, since the relative position relationship between the other coordinate points and the first key point is known, according to the first key point and the second key point The corresponding relationship of the points may determine the texture coordinates in the nail material corresponding to the other coordinate points, and obtain the corresponding first pixel value according to the texture coordinates.

Through the above method, the first pixel value in the nail material corresponding to each coordinate point to be processed in the to-be-processed area of the nail can be obtained, so that the color and/or pattern of the nail material can be rendered to the to-be-processed area. In the processing area, special effect processing and beautifying effect processing of nails are realized.

In the embodiment of the present disclosure, the rendering of the nail material to the area to be processed is realized through the corresponding relationship between the first key point and the second key point, and the nail material can be realized according to the area to be processed Adaptive changes are made for realistic nail effects and beautification.

In some implementations, the coordinate points to be processed include a first key point and a first interpolation point, wherein the first interpolation point is located in the area to be processed, and is obtained by interpolating according to the first key point of. That is to say, each coordinate point to be processed included in the area to be processed can be obtained by interpolating the first key point.

In some implementations, a triangular mesh may be constructed according to a plurality of first key points of the nail, and an area to be treated of the nail may be determined according to the constructed triangular mesh. Wherein, the first key point can be obtained by interpolating the detection result output by the nail key point detection network, and the interpolation is performed through the detection result, so that the key points located at the edge of the nail outline are denser, so that the constructed triangular mesh The resulting treated area maintains the round shape of the nail and is more consistent with the shape of the real nail.

In the case that a triangular mesh is constructed according to a plurality of first key points of the nail, the coordinate points to be processed may include vertices of the triangular mesh and second interpolation points, wherein the second interpolation points Located in the triangular mesh, it is obtained by interpolating according to the vertices of the triangular mesh, and the triangular mesh is constructed according to a plurality of first key points of the nail. That is to say, the vertices of the triangular mesh are constituted by the first key point, and the second interpolation point located in the triangular mesh is obtained by using the first key point as the vertices of the triangular mesh obtained by interpolation.

In some implementation manners, the second pixel value of the coordinate point to be processed in the image to be processed may also be obtained, that is, the color of the nail itself in the image to be processed may be obtained. Next, according to the set fusion coefficient, the first pixel value and the second pixel value of the coordinate point to be processed are weighted to obtain the target pixel value of the coordinate point to be processed, and finally according to the coordinate point to be processed The target pixel value of the coordinate point is obtained in the processed image. The above fusion process can be expressed according to formula (2):

C _new ＝C ₁ *s ₁ +C ₂ *(1-s ₁ ) (2)

Wherein, C _new represents the target pixel value, C ₁ represents the first pixel value, C ₂ represents the second pixel value, s ₁ represents the set fusion coefficient, and the set fusion coefficient may be specifically set by the user according to the desired implementation. In the embodiment of the present disclosure, by combining the color of the nail material with the color of the nail itself in the image to be processed, the achieved nail special effects and beautification effects are more natural and closer to the real nail art effect, see FIG. 4A and the processed image shown in Figure 4B.

In some implementation manners, image processing may be performed on the hand image in the following manner, so as to realize the nail art effect of the hand image. As shown in FIG. 5 , the method may include steps 501 to 506 .

In step 501, a hand image is acquired, and multiple first key points corresponding to each nail in the hand image are acquired. Wherein, the hand image may be collected in real time, or may be obtained from a storage medium.

In step 502, the first key point is interpolated, and the corresponding first key point is updated with the new first key point obtained by interpolation.

Wherein, performing interpolation on the first key point includes performing curve interpolation on adjacent key points in the first key point, and/or performing linear interpolation on symmetrical key points in the first key point.

In step 503, deform the first key point so that the nail region corresponding to the deformed first key point is longer than the length of the nail in the hand image, and use the deformed first key point to update the corresponding The first key point of .

In this step, the first key point located at the upper half of the nail can be moved by a set distance along the normal direction of the nail. Wherein, the set distance moved by each first key point can be calculated according to the formula (1).

In step 504, a triangular mesh is constructed according to the first key point, and a region to be processed that needs to realize the nail art effect is obtained according to the triangular mesh.

In step 505, the pixel color of the nail material is extracted according to the texture coordinates on the nail material corresponding to the first key point.

In this step, the nail material may be marked with a second key point corresponding to the first key point, and the first key point is obtained according to the correspondence between the first key point and the second key point The texture coordinates on the corresponding nail material; the texture coordinates on the nail material corresponding to the first key point can also be acquired in other ways.

In step 506, according to the set fusion coefficient, the pixel color of the nail material corresponding to the first key point is fused with the color of the original pixel in the hand image to obtain a new color. Specifically, fusion can be performed according to the manner indicated by formula (2).

In this step, the new color corresponding to each coordinate point in the area to be processed can be obtained by interpolating the first key point, so as to realize the nail art effect on the hand image.

In the embodiment of the present disclosure, by performing interpolation and deformation processing on the first key point corresponding to the nail, the area to be treated can maintain the round shape of the nail, which is more consistent with the shape of the real nail; Blending the colors of the original pixels in the external image can make the nail art effect more natural and closer to the real nail art effect.

Fig. 6 shows a schematic structural diagram of an image processing device according to at least one embodiment of the present disclosure. As shown in Fig. 6, the device may include: a first acquisition unit 601, configured to acquire an image to be processed containing at least one nail A first key point of the nail, wherein each of the nails corresponds to a plurality of first key points; a determination unit 602, configured to determine the area to be treated of the nail according to the plurality of first key points of the nail; The second acquiring unit 603 is used to acquire the nail material marked with a plurality of second key points; the rendering unit 604 is used to obtain the nail material according to the correspondence between the first key point and the second key point relationship, rendering the nail material into the area to be processed to obtain a processed image.

In some embodiments, the first acquisition unit is specifically configured to: input the image to be processed into the nail key point detection network to obtain the key point detection result of the nail; obtain the nail key point detection result according to the nail key point detection result. The first key point of the nail, or, perform interpolation processing on the key point detection result of the nail, and obtain the first key point of the nail according to the interpolated result.

In some embodiments, the interpolation processing of the key point detection results of the nails includes: performing curve interpolation on adjacent key points in the key point detection results of the nails; and/or; Linear interpolation is performed on the symmetric keypoints in the keypoint detection results.

In some embodiments, the device further includes a deformation unit, configured to: determine a target key point among the plurality of first key points of the nail, and the target key point is to set a connecting line between symmetrical key points A first key point between the first key points corresponding to the top of the nail or the top of the fingertip; moving the target key point along the normal direction of the nail for a set distance, wherein the normal direction is based on the specified determined by multiple first key points of the nail; using the moved target key points to update the corresponding first key points.

In some implementations, the set distance is based on the first distance between the target key point and the line between the set symmetric key point and the first key corresponding to the top of the nail or fingertip. The second distance between the point and the connecting line, and the length of the connecting line are determined, wherein the set distance is positively correlated with the first distance, negatively correlated with the second distance, and correlated with the The length of the link is negatively correlated.

In some implementations, the rendering unit is specifically configured to: acquire the texture coordinates in the nail material corresponding to the coordinate points to be processed according to the correspondence between the first key point and the second key point, Wherein, the coordinate point to be processed is located in the area to be processed; the first pixel value of the nail material at the texture coordinate is obtained; the processed Image.

In some implementations, the coordinate points to be processed include: a first key point and a first interpolation point, wherein the first interpolation point is located in the area to be processed, and is interpolated according to the first key point obtained; and/or, the vertices of the triangular mesh and the second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertices of the triangular mesh, the A triangular mesh is constructed from a plurality of first keypoints of the nail.

In some embodiments, the device further includes a fusion unit, configured to: perform weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed according to the set fusion coefficient to obtain the Processing the target pixel value of the coordinate point; obtaining the processed image according to the target pixel value of the coordinate point to be processed.

At least one embodiment of the present disclosure also provides an electronic device. As shown in FIG. 7 , the device includes a memory 701 and a processor 702. The image processing method described in any embodiment of the present disclosure is implemented when the computer instructions are executed.

At least one embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the image processing method described in any embodiment of the present disclosure is implemented.

At least one embodiment of the present disclosure further provides a computer program product, including a computer program, when the program is executed by a processor, the image processing method described in any embodiment of the present disclosure is implemented.

Those skilled in the art should understand that one or more embodiments of this specification may be provided as a method, system or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may employ a computer program embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The form of the product.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data processing device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, please refer to the part of the description of the method embodiment.

The foregoing describes specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or in A combination of one or more of . Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of data processing apparatus. Multiple modules. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for transmission by the data The processing means executes. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory and/or a random access memory. The essential components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic or magneto-optical disks, or optical disks, to receive data therefrom or to It transmits data, or both. However, a computer is not required to have such a device. In addition, a computer may be embedded in another device such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a device such as a Universal Serial Bus (USB) ) portable storage devices like flash drives, to name a few.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as primarily describing features of particular embodiments of particular inventions. Certain features that are described in this specification in multiple embodiments can also be implemented in combination in a single embodiment. On the other hand, various features that are described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may function in certain combinations as described above and even be initially so claimed, one or more features from a claimed combination may in some cases be removed from that combination and the claimed A protected combination can point to a subcombination or a variant of a subcombination.

Similarly, while operations are depicted in the figures in a particular order, this should not be construed as requiring that those operations be performed in the particular order shown, or sequentially, or that all illustrated operations be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can often be integrated together in a single software product in, or packaged into multiple software products.

Thus, certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

The above descriptions are only preferred embodiments of one or more embodiments of this specification, and are not intended to limit one or more embodiments of this specification. Within the spirit and principles of one or more embodiments of this specification, Any modification, equivalent replacement, improvement, etc. should be included in the scope of protection of one or more embodiments of this specification.

Claims

An image processing method, characterized in that the method comprises:

Acquiring a first key point of the nail in the image to be processed containing at least one nail, wherein each of the nails corresponds to a plurality of first key points;

determining a region to be treated of the nail according to a plurality of first key points of the nail;

Acquiring nail material, where multiple second key points are marked on the nail material;

According to the corresponding relationship between the first key point and the second key point, the nail material is rendered into the area to be processed to obtain a processed image.
The method according to claim 1, wherein the acquiring the first key point of the nail in the image to be processed comprising at least one nail comprises:

Input the image to be processed into the nail key point detection network to obtain the key point detection result of the nail;

Obtain the first key point of the nail according to the key point detection result of the nail, or perform interpolation processing on the key point detection result of the nail, and obtain the first key point of the nail according to the interpolated result.
The method according to claim 2, wherein the interpolation processing of the key point detection results of the nails includes:

Curve interpolation is performed on adjacent key points in the key point detection results of the nail; and/or;

Linear interpolation is performed on the symmetrical key points in the key point detection results of the nail.
The method according to any one of claims 1 to 3, wherein the method further comprises:

Determine the target key point among the plurality of first key points of the nail, the target key point is the first key point between the line between the symmetrical key points and the first key point corresponding to the top of the fingertip ;

moving the target key point by a set distance along the normal direction of the nail, wherein the normal direction is determined according to a plurality of first key points of the nail;

The corresponding first key point is updated by using the moved target key point.
The method according to claim 4, wherein the set distance is based on the first distance between the connecting line between the target key point and the set symmetric key point, the top of the fingertip corresponds to The second distance between the first key point and the connecting line, and the length of the connecting line are determined, wherein the set distance is positively correlated with the first distance and negatively correlated with the second distance , which is negatively correlated with the length of the link.
The method according to any one of claims 1 to 5, characterized in that, according to the corresponding relationship between the first key point and the second key point, the nail material is rendered to the In the processing area, the processed image is obtained, including:

According to the correspondence between the first key point and the second key point, the texture coordinates in the nail material corresponding to the coordinate points to be processed are obtained, wherein the coordinate points to be processed are located in the area to be processed Inside;

Acquiring the first pixel value of the nail material at the texture coordinate;

The processed image is obtained according to the first pixel value of the coordinate point to be processed.
The method according to claim 6, wherein the coordinate points to be processed comprise:

A first key point and a first interpolation point, wherein the first interpolation point is located in the area to be processed and is obtained by interpolation according to the first key point; and/or,

The vertices of the triangular mesh and the second interpolation point, wherein the second interpolation point is located in the triangular mesh and is obtained by interpolation according to the vertices of the triangular mesh, and the triangular mesh is obtained according to the Nails are built with multiple first key points.
The method according to claim 6 or 7, characterized in that the method further comprises:

Acquiring a second pixel value of the coordinate point to be processed in the image to be processed;

The obtaining the processed image according to the first pixel value includes:

Perform weighting processing on the first pixel value and the second pixel value of the coordinate point to be processed according to the set fusion coefficient to obtain the target pixel value of the coordinate point to be processed;

The processed image is obtained according to the target pixel value of the coordinate point to be processed.
An image processing device, characterized in that the device comprises:

A first acquisition unit, configured to acquire a first key point of the nail in the image to be processed including at least one nail, wherein each of the nails corresponds to a plurality of first key points;

A determination unit, configured to determine the area to be treated of the nail according to the plurality of first key points of the nail;

The second acquiring unit is used to acquire the nail material, and the nail material is marked with a plurality of second key points;

A rendering unit, configured to render the nail material into the region to be processed according to the correspondence between the first key point and the second key point, to obtain a processed image.
An electronic device, characterized in that the device comprises a memory and a processor, the memory is used to store computer instructions executed on the processor, and the processor is used to implement claims 1 to 1 when executing the computer instructions. 8. The method described in any one.
A computer-readable storage medium, on which a computer program is stored, wherein, when the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
A computer program product, comprising a computer program, characterized in that, when the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.