WO2022095860A1 - 指甲特效的添加方法及装置 - Google Patents

指甲特效的添加方法及装置 Download PDF

Info

Publication number
WO2022095860A1
WO2022095860A1 PCT/CN2021/128261 CN2021128261W WO2022095860A1 WO 2022095860 A1 WO2022095860 A1 WO 2022095860A1 CN 2021128261 W CN2021128261 W CN 2021128261W WO 2022095860 A1 WO2022095860 A1 WO 2022095860A1
Authority
WO
WIPO (PCT)
Prior art keywords
nail
area
information
image
region
Prior art date
Application number
PCT/CN2021/128261
Other languages
English (en)
French (fr)
Inventor
武珊珊
赵松涛
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2022095860A1 publication Critical patent/WO2022095860A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm

Definitions

  • the present disclosure relates to the technical field of adding special effects, and in particular, to a method, device, electronic device and storage medium for adding special effects to nails.
  • various special effects can be added to the captured image, such as adding nail effects to the nail area in the image.
  • the current method for adding nail special effects is generally to add nail material to the nail area determined in the image.
  • the present disclosure provides a method, device, electronic device and storage medium for adding special effects to nails.
  • the present disclosure provides a method for adding nail special effects, comprising:
  • the nail area is the unoccluded nail area in the character image
  • the determining of the nail region in the character image includes:
  • the nail region in the character image is determined according to the nail region semantic information, the nail key point information and the nail visibility information.
  • the obtaining of the nail region semantic information, nail key point information and nail visibility information based on the hand region image includes:
  • the hand region images are respectively input into the nail semantic segmentation model, the nail key point regression model and the nail visibility classification model;
  • the nail region semantic information, the nail key point information and the nail visibility information are determined.
  • the adding the nail material to the nail area includes:
  • the nail material is added to the nail area according to the nail direction.
  • the adding the nail material to the nail area further includes:
  • the nail material determines the nail material corresponding to the finger information from the nail material
  • the nail material corresponding to the finger information is added to the nail area.
  • the adding the nail material corresponding to the finger information to the nail area includes:
  • the nail material corresponding to the finger information is added to the nail area.
  • the nail regions include nail regions corresponding to different hands, and the nail materials include nail materials corresponding to different hands;
  • the adding the nail material to the nail area further includes:
  • the nail material corresponding to each of the hands is added to the corresponding nail area of the hand.
  • the nail regions include nail regions corresponding to different users, and the nail materials include nail materials corresponding to different users;
  • the adding the nail material to the nail area further includes:
  • the nail material corresponding to each of the users is added to the nail area of the corresponding user.
  • the method before entering the special effect shooting mode and acquiring the character image in response to the nail special effect selection instruction, the method further includes:
  • the corresponding relationship between the preset voice information and the nail special effect selection instruction is inquired, and the nail special effect selection instruction is determined.
  • the present disclosure also provides a device for adding nail special effects, including:
  • a character image acquisition unit configured to execute a selection instruction in response to a nail special effect, enter a special effect shooting mode and obtain a character image
  • a nail area determination unit configured to perform determination of a nail area in the character image; the nail area is an unoccluded nail area in the character image;
  • a nail material determining unit configured to execute a selection instruction based on the nail special effect to determine the nail material of the nail region from a plurality of nail materials
  • a nail material adding unit configured to execute adding the nail material to the nail area.
  • the nail region determining unit further configured to execute the nail region determining unit, is further configured to execute determining a hand region image in the person image; based on the hand region image to obtain the nail region semantic information, nail key point information and nail visibility information; determine the nail region in the character image according to the nail region semantic information, the nail key point information and the nail visibility information.
  • the nail region determination unit is further configured to input the hand region image into a nail semantic segmentation model, a nail key point regression model and a nail visibility classification model, respectively;
  • the output results of the semantic segmentation model, the nail key point regression model and the nail visibility classification model determine the nail region semantic information, the nail key point information and the nail visibility information.
  • the nail material adding unit is further configured to obtain a nail direction corresponding to the nail area; and add the nail material to the nail area according to the nail direction.
  • the nail material adding unit is further configured to execute acquiring finger information corresponding to the nail area; and determining the nail corresponding to the finger information from the nail material according to the finger information material; add the nail material corresponding to the finger information to the nail area.
  • the nail material adding unit is further configured to obtain the nail direction corresponding to the nail area; and add the nail material corresponding to the finger information to the nail according to the nail direction. area.
  • the nail regions include nail regions corresponding to different hands, and the nail materials include nail materials corresponding to different hands; the nail material adding unit is further configured to perform adding each of the The nail material corresponding to the hand is added to the corresponding nail area of the hand.
  • the nail region includes nail regions corresponding to different users, and the nail material includes nail materials corresponding to different users; the nail material adding unit is further configured to perform the corresponding operation of each user. The nail material is added to the corresponding user's nail area.
  • the device further includes an instruction determination unit configured to collect the received voice information; query the corresponding relationship between the preset voice information and the nail special effect selection instruction, and determine the nail effect selection instruction .
  • the present disclosure also provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement such as The method for adding nail special effects described in any one of the embodiments of the first aspect.
  • the present disclosure further provides a storage medium, comprising: when instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to perform any one of the embodiments of the first aspect How to add nail effects as described in .
  • the present disclosure also provides a computer program product comprising a computer program stored in a readable storage medium from which at least one processor of an electronic device The computer program is read and executed, so that the electronic device executes the method for adding nail special effects described in any one of the embodiments of the first aspect.
  • the nail special effect selection instruction in response to the nail special effect selection instruction, enter the special effect shooting mode and acquire the character image; then determine the nail area in the character image; the nail area is the unoccluded nail area in the character image; then Based on the nail effect selection instruction, the nail material of the nail area is determined from multiple nail materials; finally, the nail material is added to the nail area; in this way, the unoccluded nail area in the character image is used as the nail area where the nail material needs to be added.
  • the nail material is added to the nail area, which avoids adding the nail material to the non-nail area, which is conducive to the accurate addition of nail effects to the nail area in the image, thereby improving the accuracy of adding nail effects.
  • Fig. 1 is an application environment diagram of a method for adding nail special effects according to an exemplary embodiment.
  • Fig. 2 is a flowchart illustrating a method for adding special effects to a nail according to an exemplary embodiment.
  • FIG. 3 is a schematic diagram of model training and model post-processing of a nail semantic segmentation model according to an exemplary embodiment.
  • Fig. 4 is a flow chart of another method for adding nail special effects according to an exemplary embodiment.
  • Fig. 5 is a flow chart showing yet another method for adding nail special effects according to an exemplary embodiment.
  • Fig. 6 is a block diagram of a device for adding special effects to nails according to an exemplary embodiment.
  • Fig. 7 is an internal structure diagram of an electronic device according to an exemplary embodiment.
  • the current method for adding nail special effects is generally to add the nail material to the nail area determined in the image; however, if the nail area determined in the image is not accurate enough, it will lead to non-nail areas in the image (such as finger Skin) also added nail material, which made it impossible to accurately add nail effects to the nail area in the image.
  • the application environment diagram includes an electronic device 110 , and the electronic device 110 has a function of adding special effects, for example, it may be a smart phone, a tablet computer, a notebook computer, a personal computer, and so on.
  • the electronic device 110 enters a special effect shooting mode and acquires a character image in response to a nail special effect selection instruction; determines a nail area in the character image; the nail area is an unoccluded nail area in the character image; based on the nail effect selection instruction, Determines the nail material for the nail area from multiple nail materials; adds the nail material to the nail area.
  • Fig. 2 is a flow chart of a method for adding nail effects according to an exemplary embodiment. As shown in Fig. 2, the method for adding nail art materials is used in the electronic device as shown in Fig. 1, and includes the following steps:
  • step S210 in response to the nail special effect selection instruction, enter the special effect shooting mode and acquire a person image.
  • the fingernail special effect selection instruction refers to an instruction that triggers the electronic device to enter the special effect shooting mode, which can be triggered by clicking, can also be triggered by touch, and can also be triggered by voice.
  • the special effect shooting mode refers to the mode of adding nail special effects to the nail area in the captured person image.
  • the nail area can refer to the nail area of a person's hand or the nail area of a person's foot, which is not specifically limited here. .
  • a person image refers to an image containing a photographed person; it should be noted that a person image may refer to a static image or a video image; the addition processing of the nail special effects in the static image and the nail special effect in the video image
  • the principle of adding processing is the same, and the present disclosure mainly takes the adding processing of nail special effects in static images as an example for description.
  • a selection entry for a variety of nail material is displayed on the shooting page of the electronic device installed with the image shooting application for the user to select; when the user performs a trigger operation on the selection entry of a certain nail material, the corresponding entry is triggered
  • the fingernail special effect selection instruction after receiving the nail special effect selection instruction, the electronic device responds to the nail special effect selection instruction to enter the special effect shooting mode; when the electronic device enters the special effect shooting mode, it acquires an image including the person being photographed. That is, the image of the person.
  • the shooting page may refer to the shooting page of the short video application.
  • step S220 the nail area in the person image is determined; the nail area is the unoccluded nail area in the person image.
  • the unoccluded nail area in the person image refers to the visible nail area in the person image.
  • the electronic device after acquiring the character image, performs a nail semantic segmentation process on the character image to determine the initial nail area and other areas in the character image; wherein, the initial nail area in the character image is represented by different colors and other areas, for example, use white to represent the initial nail area in the character image, and use black to represent other areas in the character image; perform visibility classification processing on the initial nail area in the character image, and determine the visible initial nail area as The unoccluded nail area in the person image.
  • the key point information of the hand in the image is generally determined by the key point detection algorithm, and then the top of the hand is regarded as the nail according to the key point information of the hand. It can be seen from the comparison that the present disclosure performs the nail semantic segmentation processing on the character image, and then performs the visibility classification processing on the determined initial nail area to obtain the unoccluded nail area in the character image, which avoids the determination by key point information. The accuracy of the nail area is low, thereby improving the accuracy of nail area determination.
  • the nail material of the nail region is determined from the plurality of nail materials.
  • the nail material refers to the material that can add nail special effects to the unobstructed nail area included in the character image; as for the type of the nail material, it can be a solid color, a pattern, or other types, and this disclosure does not specifically Do limit.
  • the electronic device determines a nail material corresponding to the nail special effect selection instruction from a plurality of nail materials, as the nail material corresponding to the nail area.
  • the nail material corresponding to the nail effect selection instruction has been downloaded, obtain the nail material corresponding to the nail effect selection instruction from the local database through the electronic device; if the nail material corresponding to the nail effect selection instruction has not been downloaded, then Send a request to the corresponding server through the electronic device, and receive the nail material corresponding to the nail special effect selection instruction returned by the server according to the request.
  • step S240 the nail material is added to the nail area.
  • the electronic device adjusts the nail material to obtain a target nail material whose size matches the nail area; adds the target nail material to the nail area in the character image correspondingly, and obtains the target nail material added to the nail area. image of people. In this way, by adding the selected nail material to the unoccluded nail area in the character image, it is beneficial to improve the accuracy of adding nail special effects.
  • the nail special effect selection instruction in response to the nail special effect selection instruction, enter the special effect shooting mode and obtain the character image; then determine the nail area in the character image; the nail area is the unoccluded nail area in the character image; then based on the nail special effect Select the instruction to determine the nail material of the nail area from multiple nail materials; finally add the nail material to the nail area; in this way, the unoccluded nail area in the character image is used as the nail area where the nail material needs to be added, and the nail material is added to the nail area.
  • the material is added to the nail area, which avoids adding the nail material to the non-nail area, which is conducive to the accurate addition of nail effects to the nail area in the image, thereby improving the accuracy of adding nail effects.
  • determining the nail area in the person image includes: determining the hand area image in the person image; based on the hand area image, obtaining the nail area semantic information, nail key point information and Nail visibility information; determine the nail area in the character image according to the semantic information of the nail area, the nail key point information and the nail visibility information.
  • obtaining the semantic information of the nail region, the nail key point information and the nail visibility information based on the hand region image includes: inputting the hand region image into the nail semantic segmentation model, the nail key point regression model and the nail visibility respectively.
  • Sex classification model based on the output results of the nail semantic segmentation model, the nail key point regression model and the nail visibility classification model, the nail region semantic information, the nail key point information and the nail visibility information are determined.
  • an electronic device acquires a person image. Since the area occupied by the fingernails of the hand in the person image is small, the person image can be cut to obtain a hand area image with a larger area of the fingernail; for example, , the electronic device recognizes the hand image in the person image through the target detection algorithm; cuts the hand image in the person image to obtain the image including the hand as the hand area image. In this way, the area of the nail in the hand region image will be much larger than that in the person image, which can improve the accuracy of image processing.
  • the electronic device can further preprocess the hand region image, such as denoising, contrast enhancement, resolution enhancement, etc.; in some In the embodiment, the electronic device performs image super-resolution reconstruction processing on the hand region image to obtain a reconstructed image of the hand region image, and the resolution of the reconstructed image is higher than that of the hand region image; wherein, the image super-resolution reconstruction is performed.
  • Processing refers to reconstructing a low-resolution image to obtain a corresponding high-resolution image; the reconstructed image refers to a high-resolution image corresponding to the hand region image.
  • the electronic device inputs the hand region image into the pre-trained nail semantic segmentation model, and performs semantic segmentation processing on the hand region image through the pre-trained nail semantic segmentation model to obtain the semantic segmentation result corresponding to the hand region image.
  • the semantic segmentation The result is the mask map corresponding to the hand area image; wherein, the white area in the mask image represents the initial nail area, and the black area represents other areas; according to the semantic segmentation result corresponding to the hand area image, determine the The initial nail region and other regions, as the semantic information of the nail region.
  • the pre-trained nail semantic segmentation model can be obtained by training in the following ways: the electronic device obtains a sample image containing the fingernail and the actual mask image corresponding to the sample image; input the sample image into the nail semantic segmentation model to be trained, and obtain The predicted mask map corresponding to the sample image; according to the difference between the predicted mask map corresponding to the sample image and the actual mask map, combined with the loss function, the loss value of the nail semantic segmentation model to be trained is obtained; For the loss value of the semantic segmentation model, the model parameters of the nail semantic segmentation model to be trained are adjusted until the loss value is lower than the first preset threshold, and the trained nail semantic segmentation model is used as the pre-trained nail semantic segmentation model.
  • the electronic device inputs the hand area image into the pre-trained nail key point regression model, performs key point regression processing on the hand area image through the pre-trained nail key point regression model, and obtains the key points of the nail in the hand area image. information, as nail key point information.
  • the pre-trained nail key point regression model is trained in the following ways: the electronic device obtains a sample image containing hand nails and the actual nail key point information in the sample image; the sample image is input into the nail key point regression model to be trained. , obtain the predicted nail key point information corresponding to the sample image; according to the difference between the predicted nail key point information corresponding to the sample image and the actual nail key point information, combined with the loss function, the loss value of the nail key point regression model to be trained is obtained ; According to the loss value of the nail key point regression model to be trained, the model parameters of the nail key point regression model to be trained are adjusted until the loss value is lower than the second preset threshold, and the trained nail key point regression model is used as the preset value. Trained nail keypoint regression model.
  • the electronic device inputs the hand area image into the pre-trained nail visibility classification model, and performs visibility classification processing on the hand area image through the pre-trained nail visibility classification model to obtain the nail visibility information of the hand area image; For example, if the nails are visible, you get 1; if the nails are not visible, you get -1.
  • the pre-trained nail visibility classification model can be obtained by training in the following ways: the electronic device obtains a sample image containing hand nails and the actual nail visibility information corresponding to the sample image; input the sample image into the nail visibility classification to be trained The model obtains the predicted nail visibility information corresponding to the sample image; according to the difference between the predicted nail visibility information corresponding to the sample image and the actual nail visibility information, combined with the loss function, the loss of the nail visibility classification model to be trained is obtained. value; according to the loss value of the nail visibility classification model to be trained, adjust the model parameters of the nail visibility classification model to be trained, until the loss value is lower than the third preset threshold, the trained nail visibility classification model is used as Pre-trained nail visibility classification model.
  • the electronic device determines the nail region in the character image according to the semantic information of the nail region, the key point information of the nail, and the visibility information of the nail; for example, the electronic device determines the initial nail region in the hand region image according to the semantic information of the nail region, According to the nail visibility information of the hand area image, from the initial nail area in the hand area image, the unoccluded initial nail area (that is, the initial nail area with the nail visibility information of 1) is screened out as the image in the person image. At the same time, according to the nail key point information of the hand area image, the nail key point information of the nail area in the person image is determined.
  • the hand area image in the person image is first determined; then based on the hand area image, the nail area semantic information, nail key point information and nail visibility information are obtained, and finally the nail area semantic information is comprehensively considered , nail key point information and nail visibility information to determine the nail area in the character image, which is beneficial to improve the accuracy of nail area determination.
  • step S240 adding the nail material to the nail area includes: acquiring the nail direction corresponding to the nail area; and adding the nail material to the nail area according to the nail direction.
  • the nail direction corresponding to the nail area refers to the nail direction corresponding to the nails in the nail area.
  • the electronic device can determine the nail in the nail region in the person image according to the nail key point information Key point information; according to the nail key point information of the nail area in the person image, determine the nail direction corresponding to the nail area in the person image; then, the electronic device moves the nail material to the nail direction corresponding to the nail area in the person image, Add the nail area to the character image to get the character image with the nail material added to the nail area.
  • the key point information of the nail area is represented by four coordinate information, which respectively represent the four vertex information of the key point of the nail area.
  • the key point information of the thumb is [0(168,40), 1(182,35 ), 2 (197, 36), 3 (187, 50)], then according to the two coordinate information of 2 (197, 36), 0 (168, 40), the nail direction of the thumb can be determined;
  • the key point of the index finger The information is [4(53,62), 5(70,52), 6(84,63), 7(69,72)], then according to 6(84,63), 4(53,62) these two Coordinate information, the nail direction of the index finger can be determined;
  • the key point information of the middle finger is [8(38,87), 9(56,78), 10(64,92), 11(51,101)], then according to 10( The two coordinate information of 64, 92) and 8 (38, 87) can determine the nail direction of the middle finger.
  • the nail direction of the ring finger is determined by the
  • the nail direction corresponding to the nail area in the person image is determined, and the nail material is added to the nail area in the person image according to the nail direction corresponding to the nail area, so that the added nail material is more natural Realistic, thus improving the accuracy of adding nail effects.
  • step S240 adding the nail material to the nail area further includes: acquiring finger information corresponding to the nail area; determining the nail material corresponding to the finger information from the finger information according to the finger information; The nail material corresponding to the information is added to the nail area.
  • the finger information corresponding to the nail area refers to the finger category corresponding to the finger in the nail area, such as thumb, index finger, middle finger, ring finger, and little finger.
  • the key point regression processing is performed on the hand region image through the pre-trained nail key point regression model to obtain the key point information of the nail in the hand region image, according to the coordinate sequence number corresponding to the key point information and the pre-
  • the set correspondence between the coordinate serial number of the key point information and the finger information determines the finger information corresponding to the nail area in the person image.
  • the finger information corresponding to the nail area in the person image is 0-3, the finger information corresponding to the nail area is the thumb; if the key point information of the nail in the nail area corresponds to If the coordinate number is 4-7, the finger information corresponding to the nail area is the index finger; if the coordinate number corresponding to the key point information of the nail in the nail area is 8-11, the finger information corresponding to the nail area is the middle finger; The coordinate number corresponding to the key point information of the nail in the area is 12-15, then the finger information corresponding to the nail area is the ring finger; if the coordinate number corresponding to the key point information of the nail in the nail area is 16-19, then the target The finger information corresponding to the nail area is the little finger.
  • the electronic device can also determine the size information of the nail region according to the key point information of the nail in the nail region in the hand region image.
  • the electronic device determines the nail material corresponding to each nail region from the nail materials corresponding to different fingers according to the finger information corresponding to each nail region in the character image; and adds the nail material corresponding to each nail region to the corresponding image of the person.
  • the nail area of is obtained, and the image of the person with the nail material added to the nail area is obtained.
  • the electronic device adds the nail material of the index finger to the nail area corresponding to the index finger in the person image; and adds the nail material of the middle finger to the nail area corresponding to the middle finger in the person image.
  • the finger information corresponding to the nail area is determined, and according to the finger information corresponding to the nail area, the nail material corresponding to the nail area is determined from a plurality of nail materials, and then the nail material corresponding to the nail area is added.
  • the added nail material is more natural and realistic, thus improving the accuracy of adding nail special effects.
  • adding the nail material corresponding to the finger information to the nail area includes: acquiring the nail direction corresponding to the nail area; and adding the nail material corresponding to the finger information to the nail area according to the nail direction.
  • the electronic device determines the nail material corresponding to each nail region from the nail materials corresponding to different fingers according to the finger information corresponding to each nail region in the character image; The nail direction corresponding to each nail area is added to the corresponding nail area in the character image to obtain a character image with nail material added to the nail area.
  • the technical solutions provided by the embodiments of the present disclosure comprehensively consider the nail direction and finger information corresponding to the nail area, so that the nail material added to the nail area in the character image is more natural and realistic, thereby improving the accuracy of adding nail special effects.
  • the nail regions include nail regions corresponding to different hands, and the nail materials include nail materials corresponding to different hands; then, in step S240, adding the nail materials to the nail regions further includes: adding each The nail material corresponding to the hand is added to the corresponding nail area.
  • the nail area may include the nail area of the left hand and the nail area of the right hand of the same person; different hands have different nail materials, for example, the nail material corresponding to the left hand is different from the nail material corresponding to the right hand.
  • the electronic device inputs the hand region image into a pre-trained hand recognition model, performs hand recognition processing on the hand region image through the pre-trained hand recognition model, and obtains hand information in the hand region image ; According to the hand information in the hand area image, determine the hand information corresponding to the nail area in the hand area image. Next, the electronic device determines the nail material corresponding to each nail region from the nail materials corresponding to different hands according to the hand information corresponding to each nail region; and adds the nail material corresponding to each nail region to the corresponding hand in the character image. Nail area, get a character image with nail material added to the nail area.
  • the technical solutions provided by the embodiments of the present disclosure comprehensively consider the nail materials corresponding to different hands, and add the nail materials corresponding to each hand to the nail regions corresponding to different hands in the character image, which is conducive to realizing personalized nail makeup .
  • the nail area includes nail areas corresponding to different users, and the nail materials include nail materials corresponding to different users; then, in the above step S240, adding the nail material to the nail area further includes: adding the nail material corresponding to each user. The nail material is added to the corresponding user's nail area.
  • the electronic device inputs the hand region image into a pre-trained user identification model, and performs user identification processing on the hand region image through the pre-trained user identification model to obtain user information corresponding to the hand in the hand region image ; According to the user information corresponding to the hand in the hand area image, determine the user information corresponding to the nail area in the hand area image. Next, the electronic device determines the nail material corresponding to each nail region from the nail materials corresponding to different users according to the user information corresponding to each nail region; and adds the nail material corresponding to each nail region to the nail region of the corresponding user in the character image, Get the image of the person with the nail material added to the nail area.
  • the technical solutions provided by the embodiments of the present disclosure comprehensively consider the nail materials corresponding to different users, and add the nail materials corresponding to each user to the nail regions corresponding to different users in the character image, which is conducive to realizing personalized nail makeup.
  • step S210 before entering the special effect shooting mode and acquiring the character image in response to the nail special effect selection instruction, the method further includes: collecting the received voice information; querying preset voice information and nail special effects Select the corresponding relationship of the instruction to determine the nail effect selection instruction.
  • the received voice information is the voice information sent by the current user;
  • the corresponding relationship between the preset voice information and the nail effect selection instruction refers to different voice information corresponding to different nail effect selection instructions, such as the voice information of "choose mahjong nail art” , it corresponds to the selection instruction of mahjong nail effects;
  • the voice message "Select plum blossom nail art” corresponds to the selection instruction of plum blossom nail effects.
  • the electronic device monitors the voice information sent by the user in real time. If the voice information sent by the user is monitored, the voice recognition technology is used to obtain the corresponding relationship between the preset voice information and the nail special effect selection instruction. The nail special effect selection instruction corresponding to the voice information is used as the received nail effect selection instruction.
  • the technical solution provided by the embodiments of the present disclosure determines the nail special effect selection instruction through the received voice information, and does not require the user to touch the screen of the electronic device to select the nail material, which simplifies the adding process of the nail material and further improves the adding efficiency of the nail material.
  • Fig. 4 is a flow chart showing a method for adding special effects of nails according to an exemplary embodiment. As shown in Fig. 4, the method for adding special effects of nails is used in the electronic device shown in Fig. 1, and includes the following steps:
  • step S410 in response to the nail special effect selection instruction, enter the special effect shooting mode and acquire a person image.
  • step S420 the hand region image in the person image is determined.
  • step S430 the hand region images are respectively input into the nail semantic segmentation model, the nail key point regression model and the nail visibility classification model; based on the output results of the nail semantic segmentation model, the nail key point regression model and the nail visibility classification model, Determine the nail region semantic information, nail key point information and nail visibility information.
  • step S440 according to the semantic information of the nail region, the nail key point information and the nail visibility information, the nail region in the person image is determined; the nail region is the unoccluded nail region in the person image.
  • step S450 based on the nail effect selection instruction, the nail material of the nail region is determined from the plurality of nail materials.
  • step S460 the nail material is added to the nail area.
  • the semantic information of the nail area, the nail key point information and the nail visibility information are comprehensively considered, and the unoccluded nail area in the character image is used as the nail area where the nail material needs to be added, and the nail material is added to the nail area.
  • the nail area it is avoided to add the nail material to the non-nail area, which is conducive to the accurate addition of nail special effects to the nail area in the image, thereby improving the accuracy of adding nail special effects.
  • the present disclosure further provides an application scenario where the above-mentioned method for adding nail special effects is applied.
  • the application of the method for adding the nail special effect in the application scenario is as follows:
  • the hand region is detected, and the hand region is cropped.
  • the nail material is added to the visible nail area in the image, and the makeup result of the nail area is obtained.
  • steps in the flowcharts of FIGS. 2 and 4 are sequentially displayed in accordance with the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2 and 4 may include multiple steps or multiple stages. These steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. The execution of these steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or phases within the other steps.
  • Fig. 6 is a block diagram of a device for adding special effects to nails according to an exemplary embodiment. 6 , the device includes a character image acquisition unit 610 , a nail region determination unit 620 , a nail material determination unit 630 and a nail material addition unit 640 .
  • the character image acquisition unit 610 is configured to execute a special effect selection instruction in response to the nail, enter a special effect shooting mode and acquire a character image.
  • the nail area determination unit 620 is configured to perform determination of the nail area in the person image; the nail area is the unoccluded nail area in the person image.
  • the nail material determination unit 630 is configured to execute a nail material selection instruction based on a nail effect to determine a nail material of a nail region from a plurality of nail materials.
  • the nail material adding unit 640 is configured to perform adding the nail material to the nail area.
  • the nail region determining unit 620 is further configured to execute the nail region determining unit, and is also configured to execute determining the hand region image in the character image; based on the hand region image, obtain the nail region semantic information , nail key point information and nail visibility information; determine the nail area in the character image according to the nail area semantic information, nail key point information and nail visibility information.
  • the nail region determination unit 620 is further configured to input the hand region image into the nail semantic segmentation model, the nail key point regression model and the nail visibility classification model respectively; The output results of the key point regression model and the nail visibility classification model are used to determine the semantic information of the nail region, the nail key point information and the nail visibility information.
  • the nail material adding unit 640 is further configured to obtain the nail direction corresponding to the nail area; and add the nail material to the nail area according to the nail direction.
  • the nail material adding unit 640 is further configured to obtain finger information corresponding to the nail area; determine the nail material corresponding to the finger information from the finger information according to the finger information; Add to nail area.
  • the nail material adding unit 640 is further configured to obtain the nail direction corresponding to the nail area; and add the nail material corresponding to the finger information to the nail area according to the nail direction.
  • the nail regions include nail regions corresponding to different hands, and the nail materials include nail materials corresponding to different hands; the nail material adding unit 640 is further configured to perform adding the nail materials corresponding to each hand, Add to the corresponding nail area of the hand.
  • the nail regions include nail regions corresponding to different users, and the nail materials include nail materials corresponding to different users; the nail material adding unit 640 is further configured to execute adding the nail materials corresponding to each user to the corresponding nail materials. user's nail area.
  • the device for adding nail special effects further includes an instruction determination unit, configured to execute the collection of the received voice information; query the corresponding relationship between the preset voice information and the nail effect selection instruction, and determine the Nail effect selection command.
  • FIG. 7 is a block diagram of an electronic device 700 for executing the above-mentioned method for adding nail special effects, according to an exemplary embodiment.
  • electronic device 700 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.
  • an electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input/output (I/O) interface 712, a sensor component 714 and communication component 716 .
  • the processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • the processing component 702 can include one or more processors 720 to execute instructions to perform all or some of the steps of the methods described above.
  • processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components.
  • processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.
  • Memory 704 is configured to store various types of data to support operation at electronic device 700 . Examples of such data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, and the like. Memory 704 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read only memory
  • EPROM erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic or Optical Disk Magnetic Disk
  • Power supply component 706 provides power to various components of electronic device 700 .
  • Power supply components 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700 .
  • Multimedia component 708 includes a screen that provides an output interface between electronic device 700 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. A touch sensor can sense not only the boundaries of a touch or swipe action, but also the duration and pressure associated with the touch or swipe action.
  • multimedia component 708 includes a front-facing camera and/or a rear-facing camera. When the electronic device 700 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.
  • Audio component 710 is configured to output and/or input audio signals.
  • audio component 710 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 700 is in operating modes, such as calling mode, recording mode, and voice recognition mode.
  • the received audio signal may be further stored in memory 704 or transmitted via communication component 716 .
  • audio component 710 also includes a speaker for outputting audio signals.
  • the I/O interface 712 provides an interface between the processing component 702 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.
  • Sensor assembly 714 includes one or more sensors for providing status assessment of various aspects of electronic device 700 .
  • the sensor assembly 714 can detect the open/closed state of the electronic device 700, the relative positioning of the components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the electronic device 700 or a component of the electronic device 700. Changes in position, presence or absence of user contact with the electronic device 700 , orientation or acceleration/deceleration of the electronic device 700 , and temperature changes in the electronic device 700 .
  • Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices.
  • Electronic device 700 may access wireless networks based on communication standards, such as WiFi, carrier networks (eg, 2G, 3G, 4G, or 5G), or a combination thereof.
  • the communication component 716 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • electronic device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable A programmed gate array
  • controller microcontroller, microprocessor or other electronic component implementation is used to perform the above method.
  • non-transitory computer-readable storage medium including instructions, such as memory 704 including instructions, executable by the processor 720 of the electronic device 700 to perform the above method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • a computer program product comprising a computer program stored in a readable storage medium from which at least one processor of an electronic device reads and executes The computer program enables the electronic device to execute the method for adding nail special effects described in any one of the embodiments of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Image Processing (AREA)

Abstract

一种指甲特效的添加方法，该方法包括：响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域；基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材；将指甲素材添加到指甲区域。还公开了一种指甲特效的添加装置、电子设备及存储介质。

Description

指甲特效的添加方法及装置
本申请要求于2020年11月5日提交至中国专利局、申请号为202011221931.6的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及特效添加技术领域，尤其涉及一种指甲特效的添加方法、装置、电子设备及存储介质。
背景技术
在当前拍摄图像或者视频的过程中，可以为拍摄到的图像添加各种特效效果，比如将指甲特效添加到图像中的指甲区域上。
相关技术中，目前的指甲特效的添加方法，一般是将指甲素材添加到图像中确定的指甲区域上。
发明内容
本公开提供一种指甲特效的添加方法、装置、电子设备及存储介质。
在一些实施例中，本公开提供一种指甲特效的添加方法，包括：
响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
将所述指甲素材添加到所述指甲区域。
在一示例性实施例中，所述确定所述人物图像中的指甲区域，包括：
确定所述人物图像中的手部区域图像；
基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；
根据所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息，确定所述人物图像中的指甲区域。
在一示例性实施例中，所述基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息，包括：
分别将所述手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；
基于所述指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息。
在一示例性实施例中，所述将所述指甲素材添加到所述指甲区域，包括：
获取所述指甲区域对应的指甲方向；
按照所述指甲方向，将所述指甲素材添加到所述指甲区域。
在一示例性实施例中，所述将所述指甲素材添加到所述指甲区域，还包括：
获取所述指甲区域对应的手指信息；
根据所述手指信息，从所述指甲素材中确定所述手指信息对应的指甲素材；
将所述手指信息对应的指甲素材添加到所述指甲区域。
在一示例性实施例中，所述将所述手指信息对应的指甲素材添加到所述指甲区域，包括：
获取所述指甲区域对应的指甲方向；
按照所述指甲方向，将所述手指信息对应的指甲素材添加到所述指甲区域。
在一示例性实施例中，所述指甲区域包括不同手部对应的指甲区域，所述指甲素材包括不同手部对应的指甲素材；
所述将所述指甲素材添加到所述指甲区域，还包括：
将各个所述手部对应的指甲素材，添加到对应的手部的指甲区域。
在一示例性实施例中，所述指甲区域包括不同用户对应的指甲区域，所述指甲素材包括不同用户对应的指甲素材；
所述将所述指甲素材添加到所述指甲区域，还包括：
将各个所述用户对应的指甲素材，添加到对应的用户的指甲区域。
在一示例性实施例中，其特征在于，在响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像之前，还包括：
采集接收到的语音信息；
查询预设的语音信息与指甲特效选择指令的对应关系，确定所述指甲特效选择指令。
在一些实施例中，本公开还提供一种指甲特效的添加装置，包括：
人物图像获取单元，被配置为执行响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
指甲区域确定单元，被配置为执行确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
指甲素材确定单元，被配置为执行基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
指甲素材添加单元，被配置为执行将所述指甲素材添加到所述指甲区域。
在一示例性实施例中，所述指甲区域确定单元，还被配置为执行所述指甲区域确定单元，还被配置为执行确定所述人物图像中的手部区域图像；基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；根据所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息，确定所述人物图像中的指甲区域。
在一示例性实施例中，所述指甲区域确定单元，还被配置为执行分别将所述手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；基于所述指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息。
在一示例性实施例中，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的指甲方向；按照所述指甲方向，将所述指甲素材添加到所述指甲区域。
在一示例性实施例中，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的手指信息；根据所述手指信息，从所述指甲素材中确定所述手指信息对应的指甲素材；将所述手指信息对应的指甲素材添加到所述指甲区域。
在一示例性实施例中，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的指甲方向；按照所述指甲方向，将所述手指信息对应的指甲素材添加到所述指甲区域。
在一示例性实施例中，所述指甲区域包括不同手部对应的指甲区域，所述指甲素材包括不同手部对应的指甲素材；所述指甲素材添加单元，还被配置为执行将各个所述手部对应的指甲素材，添加到对应的手部的指甲区域。
在一示例性实施例中，所述指甲区域包括不同用户对应的指甲区域，所述指甲素材包括不同用户对应的指甲素材；所述指甲素材添加单元，还被配置为执行将各个所述用户对应的指甲素材，添加到对应的用户的指甲区域。
在一示例性实施例中，所述装置还包括指令确定单元，被配置为执行采集接收到的语音信息；查询预设的语音信息与指甲特效选择指令的对应关系，确定所述指甲特效选择指令。
在一些实施例中，本公开还提供一种电子设备，包括：处理器；用于存储所述处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令，以实现如第一方面的任一项实施例中所述的指甲特效的添加方法。
在一些实施例中，本公开还提供一种存储介质，包括：当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行第一方面的任一项实施例中所述的指甲特效的添加方法。
在一些实施例中，本公开还提供一种计算机程序产品，所述程序产品包括计算机程序，所述计算机程序存储在可读存储介质中，电子设备的至少一个处理器从所述可读存储介质读取并执行所述计算机程序，使得电子设备执行第一方面任一项实施例中所述的指甲特效的添加方法。
根据本公开一些实施例提供的技术方案，响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；接着确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域；然后基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材；最后将指甲素材添加到指甲区域；这样，将人物图像中的未被遮挡的指甲区域作为需要添加指甲素材的指甲区域，并将指甲素材添加到该指甲区域上，避免了将指甲素材添加到非指 甲区域上，有利于对图像中的指甲区域进行准确的指甲特效添加处理，进而提高了指甲特效的添加准确率。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理，并不构成对本公开的不当限定。
图1是根据一示例性实施例示出的一种指甲特效的添加方法的应用环境图。
图2是根据一示例性实施例示出的一种指甲特效的添加方法的流程图。
图3是根据一示例性实施例示出的指甲语义分割模型的模型训练和模型后处理的示意图。
图4是根据一示例性实施例示出的另一种指甲特效的添加方法的流程图。
图5是根据一示例性实施例示出的又一种指甲特效的添加方法的流程图。
图6是根据一示例性实施例示出的一种指甲特效的添加装置的框图。
图7是根据一示例性实施例示出的一种电子设备的内部结构图。
具体实施方式
为了使本领域普通人员更好地理解本公开的技术方案，下面将结合附图，对本公开实施例中的技术方案进行清楚、完整地描述。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
相关技术中，目前的指甲特效的添加方法，一般是将指甲素材添加到图像中确定的指甲区域上；但是，若图像中确定的指甲区域不够准确，会导致图像中的非指甲区域(比如手指皮肤)也添加有指甲素材，造成无法对图像中的指甲区域进行准确地指甲特效添加处理。
本公开所提供的指甲特效的添加方法，可以应用于如图1所示的应用环境中。参照图1，该应用环境图包括电子设备110，电子设备110具有特效添加功能，例如可以是智能手机、平板电脑、笔记本电脑或个人计算机等等。参考图1，电子设备110响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域；基于指甲特效选择指令，从多个指甲素材中确定指甲区 域的指甲素材；将指甲素材添加到指甲区域。
图2是根据一示例性实施例示出的一种指甲特效的添加方法的流程图，如图2所示，美甲素材的添加方法用于如图1所示的电子设备中，包括以下步骤：
在步骤S210中，响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像。
其中，指甲特效选择指令是指触发电子设备进入特效拍摄模式的指令，可以通过点击触发，也可以通过触摸触发，还可以通过语音触发。特效拍摄模式是指为拍摄到的人物图像中的指甲区域添加指甲特效的模式，指甲区域可以是指人的手部指甲区域，也可以是指人的脚部指甲区域，在此不做具体限定。
其中，人物图像是指包含有被拍摄人物的图像；需要说明的是，人物图像可以是指静态图像，也可以是指视频图像；静态图像中的指甲特效的添加处理与视频图像中的指甲特效的添加处理的原理一样，本公开主要以静态图像中的指甲特效的添加处理为例子进行说明。
在一些实施例中，安装有图像拍摄应用程序的电子设备的拍摄页面上显示有多种指甲素材的选择入口，供用户选择；当用户对某个指甲素材的选择入口进行触发操作时，触发对应的指甲特效选择指令；电子设备在接收到该指甲特效选择指令之后，对该指甲特效选择指令进行响应，以进入特效拍摄模式；当电子设备进入特效拍摄模式时，获取包括有被拍摄人物的图像即人物图像。在示例性应用中，拍摄页面可以是指短视频应用程序的拍摄页面。
在步骤S220中，确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域。
其中，人物图像中未被遮挡的指甲区域是指人物图像中的可见指甲区域。
在一些实施例中，在获取人物图像之后，电子设备对人物图像进行指甲语义分割处理，确定出人物图像中的初始指甲区域和其他区域；其中，通过不同的颜色表示人物图像中的初始指甲区域和其他区域，例如，采用白色表示人物图像中的初始指甲区域，采用黑色表示人物图像中的其他区域；对人物图像中的初始指甲区域进行可见性分类处理，确定出可见的初始指甲区域，作为人物图像中未被遮挡的指甲区域。
需要说明的是，相关技术中，在确定指甲区域时，一般是先通过关键点检测算法确定图像中的手部的关键点信息，然后根据手部的关键点信息，将手部的顶端当做指甲区域；经过对比可知，本公开通过对人物图像进行指甲语义分割处理，再对确定出的初始指甲区域进行可见性分类处理，得到人物图像中未被遮挡的指甲区域，避免了通过关键点信息确定的指甲区域的准确性较低的缺陷，从而提高了指甲区域的确定准确率。
在步骤S230中，基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材。其中，指甲素材是指能够为人物图像中包括的未被遮挡的指甲区域添加指甲特效的素材；至于指甲素材的类型，可以是纯色，也可以是图案，也可以是其他类型，具体本公开不做限定。
在一些实施例中，电子设备从多个指甲素材中，确定出指甲特效选择指令对应的指甲素材，作为指甲区域对应的指甲素材。
需要说明的是，若指甲特效选择指令对应的指甲素材已经下载，则通过电子设备从本地数据库获取该指甲特效选择指令对应的指甲素材；若该指甲特效选择指令对应的指甲素材还没下载，则通过电子设备向对应的服务器发送请求，并接收服务器根据请求返回的指甲特效选择指令对应的指甲素材。
在步骤S240中，将指甲素材添加到指甲区域。
在一些实施例中，电子设备将指甲素材进行调整处理，得到大小与指甲区域相匹配的目标指甲素材；将目标指甲素材对应添加到人物图像中的指甲区域，得到指甲区域添加有目标指甲素材的人物图像。这样，通过将选取的指甲素材添加到人物图像中的未被遮挡的指甲区域，有利于提高指甲特效的添加准确率。
上述指甲特效的添加方法中，响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；接着确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域；然后基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材；最后将指甲素材添加到指甲区域；这样，将人物图像中的未被遮挡的指甲区域作为需要添加指甲素材的指甲区域，并将指甲素材添加到该指甲区域上，避免了将指甲素材添加到非指甲区域上，有利于对图像中的指甲区域进行准确的指甲特效添加处理，进而提高了指甲特效的添加准确率。
在一示例性实施例中，在步骤S220中，确定人物图像中的指甲区域，包括：确定人物图像中的手部区域图像；基于手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；根据指甲区域语义信息、指甲关键点信息以及指甲可见性信息，确定人物图像中的指甲区域。
在一些实施例中，基于手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息，包括：分别将手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；基于指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定指甲区域语义信息、指甲关键点信息以及指甲可见性信息。
举例说明，第一，电子设备获取人物图像，由于手部的指甲在人物图像中的占用面积较小，可以对人物图像进行剪切处理，获得指甲所占面积较大的手部区域图像；例如，电子设备通过目标检测算法，识别人物图像中的手部图像；对人物图像中的手部图像进行剪切处理，获得包括手部的图像，作为手部区域图像。这样，指甲在手部区域图像中的面积会远大于在人物图像中的面积，可以提高图像处理的准确度。
需要说明的是，在得到手部区域图像之后，电子设备还可以对手部区域图像做进一步的预处理，比如将手部区域图像进行去噪处理、对比度增强处理、分辨率增强处理等；在一些实施例中，电子设备将手部区域图像进行图像超分辨率重建处理，得到手部区域图像 的重建图像，重建图像的分辨率高于手部区域图像的分辨率；其中，图像超分辨率重建处理是指将低分辨率图像，通过重建处理，得到对应的高分辨率图像；重建图像是指手部区域图像对应的高分辨率图像。
第二，电子设备将手部区域图像输入预先训练的指甲语义分割模型，通过预先训练的指甲语义分割模型对手部区域图像进行语义分割处理，得到手部区域图像对应的语义分割结果，该语义分割结果为手部区域图像对应的掩码图；其中，掩码图中的白色区域代表初始指甲区域，黑色区域代表其他区域；根据手部区域图像对应的语义分割结果，确定手部区域图像中的初始指甲区域和其他区域，作为指甲区域语义信息。
其中，预先训练的指甲语义分割模型可以通过下述方式训练得到：电子设备获取包含手部指甲的样本图像以及样本图像对应的实际掩码图；将样本图像输入待训练的指甲语义分割模型，得到样本图像对应的预测掩码图；根据样本图像对应的预测掩码图以及实际掩码图之间的差值，结合损失函数，得到待训练的指甲语义分割模型的损失值；根据待训练的指甲语义分割模型的损失值，调整待训练的指甲语义分割模型的模型参数，直至损失值低于第一预设阈值时，将训练后的指甲语义分割模型作为预先训练的指甲语义分割模型。
举例说明，参考图3，在指甲语义分割过程中，由于指甲面积非常小(约占整个手部面积的1％)，分割难度大，分割后边缘抖动现象明显；故在模型训练中，采用了多尺度融合的网络，更为丰富的数据增强(如通过颜色扩增，提升了对已经染过颜色后的指甲的分割效果)，更为合理的loss设计。在模型后处理中，通过引导滤波对指甲边缘进行了修正，并对整个网络输出进行了时序平滑处理，对指甲的中心点进行了光流稳定处理，比如KL光流处理。这样，通过这些方法，可以对网络输出进行稳定，从而得到更为稳定的指甲语义分割结果，即得到更为稳定的掩码图。
第三，电子设备将手部区域图像输入预先训练的指甲关键点回归模型，通过预先训练的指甲关键点回归模型对手部区域图像进行关键点回归处理，得到手部区域图像中的指甲的关键点信息，作为指甲关键点信息。
其中，预先训练的指甲关键点回归模型通过下述方式训练得到：电子设备获取包含手部指甲的样本图像以及样本图像中的实际指甲关键点信息；将样本图像输入待训练的指甲关键点回归模型，得到样本图像对应的预测指甲关键点信息；根据样本图像对应的预测指甲关键点信息以及实际指甲关键点信息之间的差值，结合损失函数，得到待训练的指甲关键点回归模型的损失值；根据待训练的指甲关键点回归模型的损失值，调整待训练的指甲关键点回归模型的模型参数，直至损失值低于第二预设阈值时，将训练后的指甲关键点回归模型作为预先训练的指甲关键点回归模型。
第四，电子设备将手部区域图像输入预先训练的指甲可见性分类模型，通过预先训练的指甲可见性分类模型对手部区域图像进行可见性分类处理，得到手部区域图像的指甲可见性信息；比如指甲可见，则得到1；如果指甲不可见，则得到-1。
其中，预先训练的指甲可见性分类模型可以通过下述方式训练得到：电子设备获取包 含手部指甲的样本图像以及样本图像对应的实际指甲可见性信息；将样本图像输入待训练的指甲可见性分类模型，得到样本图像对应的预测指甲可见性信息；根据样本图像对应的预测指甲可见性信息以及实际指甲可见性信息之间的差值，结合损失函数，得到待训练的指甲可见性分类模型的损失值；根据待训练的指甲可见性分类模型的损失值，调整待训练的指甲可见性分类模型的模型参数，直至损失值低于第三预设阈值时，将训练后的指甲可见性分类模型作为预先训练的指甲可见性分类模型。
第五，电子设备根据指甲区域语义信息、指甲关键点信息以及指甲可见性信息，确定人物图像中的指甲区域；例如，电子设备根据指甲区域语义信息，确定手部区域图像中的初始指甲区域，根据手部区域图像的指甲可见性信息，从手部区域图像中的初始指甲区域中，筛选出未被遮挡的初始指甲区域(即指甲可见性信息为1的初始指甲区域)，作为人物图像中的指甲区域；同时，根据手部区域图像的指甲关键点信息，确定人物图像中的指甲区域的指甲关键点信息。
本公开实施例提供的技术方案，先确定人物图像中的手部区域图像；再基于手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息，最后综合考虑指甲区域语义信息、指甲关键点信息以及指甲可见性信息，以确定人物图像中的指甲区域，有利于提高指甲区域的确定准确率。
在一示例性实施例中，在步骤S240中，将指甲素材添加到指甲区域，包括：获取指甲区域对应的指甲方向；按照指甲方向，将指甲素材添加到指甲区域。
其中，指甲区域对应的指甲方向，是指指甲区域中的指甲所对应的指甲方向。
在一些实施例中，在根据指甲区域语义信息、指甲关键点信息以及指甲可见性信息，确定人物图像中的指甲区域之后，电子设备可以根据指甲关键点信息，确定人物图像中的指甲区域的指甲关键点信息；根据人物图像中的指甲区域的指甲关键点信息，确定人物图像中的指甲区域对应的指甲方向；紧接着，电子设备将指甲素材朝着人物图像中的指甲区域对应的指甲方向，添加到人物图像中的指甲区域，得到指甲区域添加有指甲素材的人物图像。
举例说明，指甲区域的关键点信息通过四个坐标信息表示，分别表示指甲区域的关键点的四个顶点信息，比如大拇指的关键点信息为[0(168，40)、1(182，35)、2(197，36)、3(187，50)]，那么根据2(197，36)、0(168，40)这两个坐标信息，可以确定大拇指的指甲方向；食指的关键点信息为[4(53，62)、5(70，52)、6(84，63)、7(69，72)]，那么根据6(84，63)、4(53，62)这两个坐标信息，可以确定食指的指甲方向；中指的关键点信息为[8(38，87)、9(56，78)、10(64，92)、11(51，101)]，那么根据10(64，92)、8(38，87)这两个坐标信息，可以确定中指的指甲方向。另外，针对无名指的指甲方向，通过序号为14和12的坐标信息确定；针对小拇指的指甲方向，通过序号为18和16的坐标信息确定。
本公开实施例提供的技术方案，通过确定人物图像中的指甲区域对应的指甲方向，并按照指甲区域对应的指甲方向，将指甲素材添加到人物图像中的指甲区域，使得添加的指甲素材更加自然逼真，从而提高了指甲特效的添加准确率。
在一示例性实施例中，在步骤S240中，将指甲素材添加到指甲区域，还包括：获取指甲区域对应的手指信息；根据手指信息，从指甲素材中确定手指信息对应的指甲素材；将手指信息对应的指甲素材添加到指甲区域。
其中，指甲区域对应的手指信息，是指指甲区域中的手指对应的手指类别，比如大拇指、食指、中指、无名指、小拇指。
在一些实施例中，在通过预先训练的指甲关键点回归模型对手部区域图像进行关键点回归处理，得到手部区域图像中的指甲的关键点信息之后，根据关键点信息对应的坐标序号以及预设的关键点信息的坐标序号与手指信息的对应关系，确定人物图像中的指甲区域对应的手指信息。
举例说明，若人物图像中的指甲区域中的指甲的关键点信息对应的坐标序号为0-3，则该指甲区域对应的手指信息为大拇指；若指甲区域中的指甲的关键点信息对应的坐标序号为4-7，则该指甲区域对应的手指信息为食指；若指甲区域中的指甲的关键点信息对应的坐标序号为8-11，则该指甲区域对应的手指信息为中指；若指甲区域中的指甲的关键点信息对应的坐标序号为12-15，则该指甲区域对应的手指信息为无名指；若指甲区域中的指甲的关键点信息对应的坐标序号为16-19，则该目标指甲区域对应的手指信息为小拇指。
进一步地，电子设备还可以根据手部区域图像中的指甲区域中的指甲的关键点信息，确定该指甲区域的大小信息。
紧接着，电子设备根据人物图像中的各个指甲区域对应的手指信息，从不同手指对应的指甲素材中，确定各个指甲区域对应的指甲素材；将各个指甲区域对应的指甲素材添加到人物图像中对应的指甲区域，得到指甲区域添加有指甲素材的人物图像。
例如，电子设备将食指的指甲素材，添加到人物图像中的食指对应的指甲区域；将中指的指甲素材，添加到人物图像中的中指对应的指甲区域。
本公开实施例提供的技术方案，通过确定指甲区域对应的手指信息，并根据指甲区域对应的手指信息，从多个指甲素材中确定指甲区域对应的指甲素材，然后将指甲区域对应的指甲素材添加到人物图像中对应的指甲区域，使得添加的指甲素材更加自然逼真，从而提高了指甲特效的添加准确率。
在一示例性实施例中，将手指信息对应的指甲素材添加到指甲区域，包括：获取指甲区域对应的指甲方向；按照指甲方向，将手指信息对应的指甲素材添加到指甲区域。
在一些实施例中，电子设备根据人物图像中各个指甲区域对应的手指信息，从不同手指对应的指甲素材中，确定各个指甲区域对应的指甲素材；分别将各个指甲区域对应的指 甲素材，朝着各个指甲区域对应的指甲方向，添加到人物图像中对应的指甲区域，得到指甲区域添加有指甲素材的人物图像。
本公开实施例提供的技术方案，综合考虑指甲区域对应的指甲方向和手指信息，使得添加到人物图像中的指甲区域的指甲素材更加自然逼真，从而提高了指甲特效的添加准确率。
在一示例性实施例中，指甲区域包括不同手部对应的指甲区域，指甲素材包括不同手部对应的指甲素材；那么，在步骤S240中，将指甲素材添加到指甲区域，还包括：将各个手部对应的指甲素材，添加到对应的指甲区域。
其中，指甲区域可以包括同一个人的左手的指甲区域和右手的指甲区域；不同手部具有不同的指甲素材，比如左手对应的指甲素材跟右手对应的指甲素材不一样。
在一些实施例中，电子设备将手部区域图像输入预先训练的手部识别模型，通过预先训练的手部识别模型对手部区域图像进行手部识别处理，得到手部区域图像中的手部信息；根据手部区域图像中的手部信息，确定手部区域图像中的指甲区域对应的手部信息。接着，电子设备根据各个指甲区域对应的手部信息，从不同手部对应的指甲素材中，确定各个指甲区域对应的指甲素材；将各个指甲区域对应的指甲素材添加到人物图像中对应手部的指甲区域，得到指甲区域添加有指甲素材的人物图像。
本公开实施例提供的技术方案，综合考虑不同手部对应的指甲素材，将各个手部对应的指甲素材，添加到人物图像中不同手部对应的指甲区域，有利于实现指甲的个性化美妆。
在一示例性实施例中，指甲区域包括不同用户对应的指甲区域，指甲素材包括不同用户对应的指甲素材；那么，上述步骤S240，将指甲素材添加到指甲区域，还包括：将各个用户对应的指甲素材，添加到对应的用户的指甲区域。
在一些实施例中，电子设备将手部区域图像输入预先训练的用户识别模型，通过预先训练的用户识别模型对手部区域图像进行用户识别处理，得到手部区域图像中的手部对应的用户信息；根据手部区域图像中的手部对应的用户信息，确定手部区域图像中的指甲区域对应的用户信息。接着，电子设备根据各个指甲区域对应的用户信息，从不同用户对应的指甲素材中，确定各个指甲区域对应的指甲素材；将各个指甲区域对应的指甲素材添加到人物图像中对应用户的指甲区域，得到指甲区域添加有指甲素材的人物图像。
本公开实施例提供的技术方案，综合考虑不同用户对应的指甲素材，将各个用户对应的指甲素材，添加到人物图像中不同用户对应的指甲区域，有利于实现指甲的个性化美妆。
在一示例性实施例中，在步骤S210中，在响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像之前，还包括：采集接收到的语音信息；查询预设的语音信息与指甲特效选择指令的对应关系，确定指甲特效选择指令。
其中，接收到的语音信息为当前用户发出的语音信息；预设的语音信息与指甲特效选择指令的对应关系，是指不同语音信息，对应不同指甲特效选择指令，比如“选择麻将美甲”语音信息，则对应麻将指甲特效选择指令；“选择梅花美甲”语音信息，则对应梅花指甲特效选择指令。
在一些实施例中，电子设备实时对用户发出的语音信息进行监测，若监测到用户发出的语音信息，则通过语音识别技术，并结合预设的语音信息与指甲特效选择指令的对应关系，得到语音信息对应的指甲特效选择指令，作为接收到的指甲特效选择指令。
本公开实施例提供的技术方案，通过接收到的语音信息，确定指甲特效选择指令，无需用户触摸电子设备屏幕选择指甲素材，简化了指甲素材的添加流程，进一步提高了指甲素材的添加效率。
图4是根据一示例性实施例示出的一种指甲特效的添加方法的流程图，如图4所示，指甲特效的添加方法用于如图1所示的电子设备中，包括以下步骤：
在步骤S410中，响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像。
在步骤S420中，确定人物图像中的手部区域图像。
在步骤S430中，分别将手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；基于指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定指甲区域语义信息、指甲关键点信息和指甲可见性信息。
在步骤S440中，根据指甲区域语义信息、指甲关键点信息以及指甲可见性信息，确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域。
在步骤S450中，基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材。
在步骤S460中，将指甲素材添加到指甲区域。
上述指甲特效的添加方法中，综合考虑指甲区域语义信息、指甲关键点信息以及指甲可见性信息，将人物图像中未被遮挡的指甲区域作为需要添加指甲素材的指甲区域，并将指甲素材添加到该指甲区域上，避免了将指甲素材添加到非指甲区域上，有利于对图像中的指甲区域进行准确的指甲特效添加处理，进而提高了指甲特效的添加准确率。
在一示例性实施例中，如图5所示，本公开还提供一种应用场景，该应用场景应用上述的指甲特效的添加方法。在一些实施例中，该指甲特效的添加方法在该应用场景的应用如下：
(1)针对输入图像，进行手部区域检测，并将手部区域剪裁出来。
(2)将剪裁后的手部图像送入网络，该网络采用了同样的Encoder，但是Decoder部分采用了三个分支，分别为指甲语义分割分支，指甲关键点回归分支，指甲可见性分类分支。经过网络输出，得到了指甲区域语义信息、指甲盖关键点信息以及指甲盖是否可见信 息。
(3)根据指甲区域语义信息、指甲盖关键点信息以及指甲盖是否可见信息，确定图像中的可见指甲区域以及可见指甲区域对应的方向信息和手指信息。
(4)根据可见指甲区域对应的方向信息和手指信息，将指甲素材添加到图像中的可见指甲区域，得到指甲区域上妆结果。
上述指甲特效的添加方法中，通过语义分割和关键点的美甲上妆方案，不仅可以得到指甲盖的语义分割信息，同时还可以得到指甲盖的位置信息，方向信息和指甲盖归属于哪个手指的信息，实现更为自然逼真的美甲上妆体验，提升用户体验，使用户自拍场景有更多的可玩性。
应该理解的是，虽然图2、4的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2、4中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
图6是根据一示例性实施例示出的一种指甲特效的添加装置的框图。参照图6，该装置包括人物图像获取单元610，指甲区域确定单元620，指甲素材确定单元630和指甲素材添加单元640。
人物图像获取单元610，被配置为执行响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像。
指甲区域确定单元620，被配置为执行确定人物图像中的指甲区域；指甲区域为人物图像中未被遮挡的指甲区域。
指甲素材确定单元630，被配置为执行基于指甲特效选择指令，从多个指甲素材中确定指甲区域的指甲素材。
指甲素材添加单元640，被配置为执行将指甲素材添加到指甲区域。
在一示例性实施例中，指甲区域确定单元620，还被配置为执行指甲区域确定单元，还被配置为执行确定人物图像中的手部区域图像；基于手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；根据指甲区域语义信息、指甲关键点信息以及指甲可见性信息，确定人物图像中的指甲区域。
在一示例性实施例中，指甲区域确定单元620，还被配置为执行分别将手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；基于指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定指甲区域语义信息、指甲关键点信息以及指甲可见性信息。
在一示例性实施例中，指甲素材添加单元640，还被配置为执行获取指甲区域对应的指甲方向；按照指甲方向，将指甲素材添加到指甲区域。
在一示例性实施例中，指甲素材添加单元640，还被配置为执行获取指甲区域对应的手指信息；根据手指信息，从指甲素材中确定手指信息对应的指甲素材；将手指信息对应的指甲素材添加到指甲区域。
在一示例性实施例中，指甲素材添加单元640，还被配置为执行获取指甲区域对应的指甲方向；按照指甲方向，将手指信息对应的指甲素材添加到指甲区域。
在一示例性实施例中，指甲区域包括不同手部对应的指甲区域，指甲素材包括不同手部对应的指甲素材；指甲素材添加单元640，还被配置为执行将各个手部对应的指甲素材，添加到对应的手部的指甲区域。
在一示例性实施例中，指甲区域包括不同用户对应的指甲区域，指甲素材包括不同用户对应的指甲素材；指甲素材添加单元640，还被配置为执行将各个用户对应的指甲素材，添加到对应的用户的指甲区域。
在一示例性实施例中，本公开提供的指甲特效的添加装置还包括指令确定单元，被配置为执行采集接收到的语音信息；查询预设的语音信息与指甲特效选择指令的对应关系，确定指甲特效选择指令。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
图7是根据一示例性实施例示出的一种用于执行上述指甲特效的添加方法的电子设备700的框图。例如，电子设备700可以是移动电话、计算机、数字广播终端、消息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等。
参照图7，电子设备700可以包括以下一个或多个组件：处理组件702、存储器704、电源组件706、多媒体组件708、音频组件710、输入/输出(I/O)的接口712、传感器组件714以及通信组件716。
处理组件702通常控制电子设备700的整体操作，诸如与显示、电话呼叫、数据通信、相机操作和记录操作相关联的操作。处理组件702可以包括一个或多个处理器720来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件702可以包括一个或多个模块，便于处理组件702和其他组件之间的交互。例如，处理组件702可以包括多媒体模块，以方便多媒体组件708和处理组件702之间的交互。
存储器704被配置为存储各种类型的数据以支持在电子设备700的操作。这些数据的示例包括用于在电子设备700上操作的任何应用程序或方法的指令、联系人数据、电话簿数据、消息、图片、视频等。存储器704可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)、电可擦除可编程只读存储器(EEPROM)、可擦除可编程只读存储器(EPROM)、可编程只读存储器(PROM)、只 读存储器(ROM)、磁存储器、快闪存储器、磁盘或光盘。
电源组件706为电子设备700的各种组件提供电力。电源组件706可以包括电源管理系统，一个或多个电源，及其他与为电子设备700生成、管理和分配电力相关联的组件。
多媒体组件708包括在电子设备700和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件708包括一个前置摄像头和/或后置摄像头。当电子设备700处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件710被配置为输出和/或输入音频信号。例如，音频组件710包括一个麦克风(MIC)，当电子设备700处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器704或经由通信组件716发送。在一些实施例中，音频组件710还包括一个扬声器，用于输出音频信号。
I/O接口712为处理组件702和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件714包括一个或多个传感器，用于为电子设备700提供各个方面的状态评估。例如，传感器组件714可以检测到电子设备700的打开/关闭状态，组件的相对定位，例如组件为电子设备700的显示器和小键盘，传感器组件714还可以检测电子设备700或电子设备700一个组件的位置改变，用户与电子设备700接触的存在或不存在，电子设备700方位或加速/减速和电子设备700的温度变化。传感器组件714可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件714还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件714还可以包括加速度传感器、陀螺仪传感器、磁传感器、压力传感器或温度传感器。
通信组件716被配置为便于电子设备700和其他设备之间有线或无线方式的通信。电子设备700可以接入基于通信标准的无线网络，如WiFi，运营商网络(如2G、3G、4G或5G)，或它们的组合。在一个示例性实施例中，通信组件716经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件716还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备700可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器704，上述指令可由电子设备700的处理器720执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供一种计算机程序产品，该程序产品包括计算机程序，该计算机程序存储在可读存储介质中，电子设备的至少一个处理器从该可读存储介质读取并执行该计算机程序，使得电子设备执行本公开的任一项实施例中所述的指甲特效的添加方法。
本公开所有实施例均可以单独被执行，也可以与其他实施例相结合被执行，均视为本公开要求的保护范围。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (20)

  1. 一种指甲特效的添加方法，应用于电子设备，其特征在于，包括：
    响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
    确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
    基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
    将所述指甲素材添加到所述指甲区域。
  2. 根据权利要求1所述的方法，其特征在于，所述确定所述人物图像中的指甲区域，包括：
    确定所述人物图像中的手部区域图像；
    基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；
    根据所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息，确定所述人物图像中的指甲区域。
  3. 根据权利要求2所述的方法，其特征在于，所述基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息，包括：
    分别将所述手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；
    基于所述指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息。
  4. 根据权利要求1所述的方法，其特征在于，所述将所述指甲素材添加到所述指甲区域，包括：
    获取所述指甲区域对应的指甲方向；
    按照所述指甲方向，将所述指甲素材添加到所述指甲区域。
  5. 根据权利要求1所述的方法，其特征在于，所述将所述指甲素材添加到所述指甲区域，还包括：
    获取所述指甲区域对应的手指信息；
    根据所述手指信息，从所述指甲素材中确定所述手指信息对应的指甲素材；
    将所述手指信息对应的指甲素材添加到所述指甲区域。
  6. 根据权利要求5所述的方法，其特征在于，所述将所述手指信息对应的指甲素材添加到所述指甲区域，包括：
    获取所述指甲区域对应的指甲方向；
    按照所述指甲方向，将所述手指信息对应的指甲素材添加到所述指甲区域。
  7. 根据权利要求1所述的方法，其特征在于，所述指甲区域包括不同手部对应的指甲区域，所述指甲素材包括不同手部对应的指甲素材；
    所述将所述指甲素材添加到所述指甲区域，还包括：
    将各个所述手部对应的指甲素材，添加到对应的手部的指甲区域。
  8. 根据权利要求1所述的方法，其特征在于，所述指甲区域包括不同用户对应的指甲区域，所述指甲素材包括不同用户对应的指甲素材；
    所述将所述指甲素材添加到所述指甲区域，还包括：
    将各个所述用户对应的指甲素材，添加到对应的用户的指甲区域。
  9. 根据权利要求1至8任一项所述的方法，其特征在于，还包括：
    采集接收到的语音信息；
    查询预设的语音信息与指甲特效选择指令的对应关系，确定所述指甲特效选择指令。
  10. 一种指甲特效的添加装置，应用于电子设备，其特征在于，包括：
    人物图像获取单元，被配置为执行响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
    指甲区域确定单元，被配置为执行确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
    指甲素材确定单元，被配置为执行基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
    指甲素材添加单元，被配置为执行将所述指甲素材添加到所述指甲区域。
  11. 根据权利要求10所述的装置，其特征在于，所述指甲区域确定单元，还被配置为执行确定所述人物图像中的手部区域图像；基于所述手部区域图像，得到指甲区域语义信息、指甲关键点信息以及指甲可见性信息；根据所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息，确定所述人物图像中的指甲区域。
  12. 根据权利要求11所述的装置，其特征在于，所述指甲区域确定单元，还被配置为执行分别将所述手部区域图像输入指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型；基于所述指甲语义分割模型、指甲关键点回归模型和指甲可见性分类模型的输出结果，确定所述指甲区域语义信息、所述指甲关键点信息以及所述指甲可见性信息。
  13. 根据权利要求10所述的装置，其特征在于，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的指甲方向；按照所述指甲方向，将所述指甲素材添加到所述指甲区域。
  14. 根据权利要求10所述的装置，其特征在于，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的手指信息；根据所述手指信息，从所述指甲素材中确定所述手指信息对应的指甲素材；将所述手指信息对应的指甲素材添加到所述指甲区域。
  15. 根据权利要求14所述的装置，其特征在于，所述指甲素材添加单元，还被配置为执行获取所述指甲区域对应的指甲方向；按照所述指甲方向，将所述手指信息对应的指甲素材添加到所述指甲区域。
  16. 根据权利要求10所述的装置，其特征在于，所述指甲区域包括不同手部对应的指 甲区域，所述指甲素材包括不同手部对应的指甲素材；所述指甲素材添加单元，还被配置为执行将各个所述手部对应的指甲素材，添加到对应的手部的指甲区域。
  17. 根据权利要求10所述的装置，其特征在于，所述指甲区域包括不同用户对应的指甲区域，所述指甲素材包括不同用户对应的指甲素材；所述指甲素材添加单元，还被配置为执行将各个所述用户对应的指甲素材，添加到对应的用户的指甲区域。
  18. 根据权利要求10至17任一项所述的装置，其特征在于，所述装置还包括指令确定单元，被配置为执行采集接收到的语音信息；查询预设的语音信息与指甲特效选择指令的对应关系，确定所述指甲特效选择指令。
  19. 一种电子设备，其特征在于，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令，以实现以下步骤：
    响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
    确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
    基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
    将所述指甲素材添加到所述指甲区域。
  20. 一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下步骤：
    响应于指甲特效选择指令，进入特效拍摄模式并获取人物图像；
    确定所述人物图像中的指甲区域；所述指甲区域为所述人物图像中未被遮挡的指甲区域；
    基于所述指甲特效选择指令，从多个指甲素材中确定所述指甲区域的指甲素材；
    将所述指甲素材添加到所述指甲区域。
PCT/CN2021/128261 2020-11-05 2021-11-02 指甲特效的添加方法及装置 WO2022095860A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011221931.6A CN112347911A (zh) 2020-11-05 2020-11-05 指甲特效的添加方法、装置、电子设备及存储介质
CN202011221931.6 2020-11-05

Publications (1)

Publication Number Publication Date
WO2022095860A1 true WO2022095860A1 (zh) 2022-05-12

Family

ID=74429369

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/128261 WO2022095860A1 (zh) 2020-11-05 2021-11-02 指甲特效的添加方法及装置

Country Status (2)

Country Link
CN (1) CN112347911A (zh)
WO (1) WO2022095860A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112347911A (zh) * 2020-11-05 2021-02-09 北京达佳互联信息技术有限公司 指甲特效的添加方法、装置、电子设备及存储介质
CN113240777A (zh) * 2021-04-25 2021-08-10 北京达佳互联信息技术有限公司 特效素材处理方法、装置、电子设备及存储介质
CN114880057A (zh) * 2022-04-22 2022-08-09 北京三快在线科技有限公司 图像显示方法、装置、终端、服务器及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170154214A1 (en) * 2015-11-27 2017-06-01 Holition Limited Locating and tracking fingernails in images
CN111047526A (zh) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
CN111091519A (zh) * 2019-12-20 2020-05-01 维沃移动通信有限公司 一种图像处理方法及装置
CN111739028A (zh) * 2020-05-26 2020-10-02 华南理工大学 指甲区域图像获取方法、系统、计算设备及存储介质
CN112347911A (zh) * 2020-11-05 2021-02-09 北京达佳互联信息技术有限公司 指甲特效的添加方法、装置、电子设备及存储介质
CN113284041A (zh) * 2021-05-14 2021-08-20 北京市商汤科技开发有限公司 一种图像处理方法、装置、设备及计算机存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107590807A (zh) * 2017-09-29 2018-01-16 百度在线网络技术（北京）有限公司 用于检测图像质量的方法和装置
CN109376575A (zh) * 2018-08-20 2019-02-22 奇酷互联网络科技（深圳）有限公司 对图像中人体美化的方法、移动终端和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170154214A1 (en) * 2015-11-27 2017-06-01 Holition Limited Locating and tracking fingernails in images
CN111047526A (zh) * 2019-11-22 2020-04-21 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
CN111091519A (zh) * 2019-12-20 2020-05-01 维沃移动通信有限公司 一种图像处理方法及装置
CN111739028A (zh) * 2020-05-26 2020-10-02 华南理工大学 指甲区域图像获取方法、系统、计算设备及存储介质
CN112347911A (zh) * 2020-11-05 2021-02-09 北京达佳互联信息技术有限公司 指甲特效的添加方法、装置、电子设备及存储介质
CN113284041A (zh) * 2021-05-14 2021-08-20 北京市商汤科技开发有限公司 一种图像处理方法、装置、设备及计算机存储介质

Also Published As

Publication number Publication date
CN112347911A (zh) 2021-02-09

Similar Documents

Publication Publication Date Title
CN108965982B (zh) 视频录制方法、装置、电子设备和可读存储介质
WO2022095860A1 (zh) 指甲特效的添加方法及装置
JP6374986B2 (ja) 顔認識方法、装置及び端末
WO2021135601A1 (zh) 辅助拍照方法、装置、终端设备及存储介质
WO2017016149A1 (zh) 联系人信息推荐方法及装置
CN109189986B (zh) 信息推荐方法、装置、电子设备和可读存储介质
WO2017084183A1 (zh) 信息显示方法与装置
WO2022110837A1 (zh) 图像处理方法及装置
US11030733B2 (en) Method, electronic device and storage medium for processing image
CN107368810A (zh) 人脸检测方法及装置
WO2022077970A1 (zh) 特效添加方法及装置
CN111988493B (zh) 交互处理方法、装置、设备及存储介质
WO2022198934A1 (zh) 卡点视频的生成方法及装置
US11216075B2 (en) Method, apparatus and device for triggering shooting mode, and storage medium
CN111984347A (zh) 交互处理方法、装置、设备及存储介质
CN113409342A (zh) 图像风格迁移模型的训练方法、装置及电子设备
CN111526287A (zh) 图像拍摄方法、装置、电子设备、服务器、系统及存储介质
CN112188091B (zh) 人脸信息识别方法、装置、电子设备及存储介质
KR20190111034A (ko) 특징 이미지 획득 방법 및 디바이스, 및 사용자 인증 방법
CN106294596A (zh) 信息搜索的方法及装置
CN111988522B (zh) 拍摄控制方法、装置、电子设备及存储介质
CN113642551A (zh) 指甲关键点检测方法、装置、电子设备及存储介质
US11715234B2 (en) Image acquisition method, image acquisition device, and storage medium
KR102501036B1 (ko) 이미지 촬영 방법, 이미지 촬영 장치 및 저장 매체
CN112784858B (zh) 一种图像数据的处理方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21888557

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21888557

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14/08/2023)