WO2022042195A1 - 护肤辅助方法、设备及存储介质 - Google Patents

护肤辅助方法、设备及存储介质 Download PDF

Info

Publication number
WO2022042195A1
WO2022042195A1 PCT/CN2021/109140 CN2021109140W WO2022042195A1 WO 2022042195 A1 WO2022042195 A1 WO 2022042195A1 CN 2021109140 W CN2021109140 W CN 2021109140W WO 2022042195 A1 WO2022042195 A1 WO 2022042195A1
Authority
WO
WIPO (PCT)
Prior art keywords
skin care
action
duration
images
face
Prior art date
Application number
PCT/CN2021/109140
Other languages
English (en)
French (fr)
Inventor
赵琳
卢曰万
姚钊盈
董辰
胡宏伟
郜文美
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to US18/042,702 priority Critical patent/US20230334641A1/en
Priority to EP21860035.1A priority patent/EP4191468A4/en
Publication of WO2022042195A1 publication Critical patent/WO2022042195A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30088—Skin; Dermal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present application relates to smart homes, and in particular, to an auxiliary method, device and storage medium for skin care.
  • the present application provides an auxiliary method, device and storage medium for skin care, so as to guide the user's skin care.
  • an embodiment of the present application provides a skin care assistance method, which is applied to a first electronic device with a camera.
  • the first electronic device acquires an image through the camera, and the first electronic device acquires an image through the camera.
  • the image contains a hand area and a face area, and the first operation is used to indicate the start of the skin care process;
  • the target image features are extracted from the image, and the target image features are features related to the hand area and the face area;
  • the target image features of output the integrity of the skin care process, wherein the multi-frame images are images generated in time sequence during the skin care process.
  • outputting the integrity of the skin care process according to the target image features corresponding to the multi-frame images may include: determining the skin care actions corresponding to the multi-frame images and each skin care action according to the target image features corresponding to the multi-frame images. Action duration; output the completeness of the skin care process according to the skin care action and the duration of each skin care action.
  • the skin care action and its duration are first determined according to the characteristics of the target image, and then the completeness of the skin care process is output according to the skin care action and the duration of each skin care action to provide a specific implementation method.
  • determining the skin care actions corresponding to the multi-frame images and the duration of each skin-care action according to the target image features corresponding to the multi-frame images may include: according to the target image features corresponding to the multi-frame images and a pre-trained classifier , determine the skin care action corresponding to the multi-frame images; determine the duration of the skin care action.
  • the target image feature is used as the input parameter of the pre-trained classifier, and the output of the classifier is the skin care action corresponding to the input target image feature.
  • the classifier recognizes the next skin care action, it can be understood that the previous skin care action ends, from the first identification of the skin care action to the recognition of the next skin care action, this period of time is the duration of the skin care action.
  • the skin care assistance method may further include: if the duration of the skin care action is greater than the preset duration, prompting the user to perform the next skin care action; If the duration is less than the preset duration, the user is prompted to continue the skin care action.
  • the duration that each skin care action should last is set, that is, the preset duration. If it is less than the preset duration, the skin care action is considered incomplete, and the user is prompted to continue the skin care action; if the duration is longer than the preset duration, the skin care action is considered sufficient, and the user is prompted to perform the next skin care action, so that the user can clarify the follow-up action and further improve the user experience.
  • the duration corresponding to the prompt is a preset value, and after the duration of the prompt is greater than the preset value, the prompt disappears automatically.
  • the prompts and other automated means such as setting the corresponding duration, human participation is reduced and the user experience is further improved.
  • the auxiliary skin care method further includes: detecting whether the skin care actions are standardized; if the skin care actions are not standardized, prompting the standardized actions corresponding to the skin care actions.
  • outputting the completeness of the skin care process according to the skin care action and the duration of each skin care action may include: determining the duration of the skin care process according to the skin care action and the duration and preset duration of each skin care action. Completeness; outputs the completeness of the skincare process.
  • extracting the target image feature from the image may include: if it is determined that the hand is displaced relative to the face between different frame images, extracting the target image feature from the image.
  • determining that the hand is displaced relative to the face between different frame images may include: extracting key point information of the face area and the hand area in the image; determining the hand position between different frame images according to the key point information. There is displacement relative to the face.
  • outputting the completeness of the skin care process may include: displaying the completeness of the skin care process through at least one display type among graphs, illustrations and scores.
  • displaying the completeness of the skin care process through at least one display type among graphs, illustrations and scores.
  • several specific completeness display types are provided.
  • the illustration includes circles corresponding to different skin care actions, wherein the completeness of the circles is used to indicate the completeness of the corresponding skin care actions.
  • the skin care auxiliary method may further include: in response to the second operation acting on the ring, displaying skin care details corresponding to the ring. For example, when the user clicks or double-clicks (ie, the second operation) the ring, the electronic device responds to the click or double-click, and displays the skin care details corresponding to the ring.
  • the integrity of the skin care process is included in the analysis report.
  • outputting the completeness of the skin care process includes: displaying a virtual button for indicating an analysis report; and displaying the content of the analysis report in response to a third operation acting on the virtual button.
  • outputting the integrity of the skin care process includes: sending the integrity of the skin care process to a second electronic device, and the second electronic device displays the integrity.
  • the display limitation of the first electronic device is considered, for example, the display screen of the smart mirror is small.
  • the second electronic device displays the completeness; in addition to the consideration of the display screen area, in addition, the smart mirror body is relatively fixed and inconvenient to carry, while portable devices such as mobile phones can be carried with you, so that users can check the completeness of the skin care process anytime, anywhere.
  • the skin care assistance method further includes: based on the integrity, outputting a strategy for providing skin care guidance to the user.
  • a strategy for providing skin care guidance to the user.
  • outputting the strategy of providing skin care guidance to the user may include: outputting the strategy of providing skin care guidance to the user in the form of pictures and/or videos.
  • the skin care assistance method may further include: preprocessing the image, where the preprocessing includes at least one of cropping, scaling and rotation.
  • extracting the target image feature from the image includes: extracting the target image feature from the preprocessed image.
  • the running time and power consumption of the skin care assistant method are reduced by preprocessing such as cropping, scaling and rotation.
  • an embodiment of the present application provides a skin care auxiliary device, which is applied to a first electronic device having a camera.
  • Skincare aids include:
  • an acquisition module configured to acquire an image through a camera of the first electronic device in response to a first operation acting on the first electronic device, the image includes a hand area and a face area, and the first operation is used to indicate the start of the skin care process;
  • the extraction module is used to extract the target image features in the image, and the target image features are features related to the hand area and the face area;
  • the output module is used for outputting the integrity of the skin care process according to the target image features corresponding to the multi-frame images, and the multi-frame images are images generated in time sequence during the skin care process.
  • the skin care auxiliary device can implement the solution in any of the above-mentioned embodiments through the acquisition module, the extraction module and the output module, and the implementation principle and technical effect thereof are similar, and will not be repeated here.
  • the output module includes:
  • a determining unit configured to determine the skin care actions corresponding to the multi-frame images and the duration of each skin-care action according to the target image features corresponding to the multi-frame images;
  • the output unit is used for outputting the completeness of the skin care process according to the skin care action and the duration of each skin care action.
  • the determining unit is specifically configured to: determine the skin care action corresponding to the multi-frame images according to the target image features corresponding to the multi-frame images and the pre-trained classifier; and determine the duration of the skin care action.
  • the output unit is further configured to: after the determining unit determines the duration of the skin care action, if the duration of the skin care action is greater than the preset duration, prompt the user to perform the next skin care action; If the duration is less than the preset duration, the user is prompted to continue the skin care action.
  • the duration corresponding to the prompt is a preset value, and after the duration of the prompt is greater than the preset value, the prompt disappears automatically.
  • the output unit is further configured to: after the determination unit determines the skin care actions corresponding to the multi-frame images, detect whether the skin care actions are normal; if the skin care actions are not normal, prompt the normal actions corresponding to the skin care actions.
  • the output unit is specifically configured to: determine the completeness of the skin care process according to the skin care action and the duration and preset duration of each skin care action; and output the completeness of the skin care process.
  • the extraction module is specifically configured to: extract the target image feature in the image if it is determined that the hand is displaced relative to the face between different frame images.
  • the extraction module when determining that the hand is displaced relative to the face between different frame images, is specifically configured to: extract the key point information of the face area and the hand area in the image; and determine the different frame images according to the key point information. There is a displacement between the hands relative to the face.
  • the extraction module includes at least one of the following when determining, according to the key point information, that the hand is displaced relative to the face between different frame images:
  • the output module is specifically configured to: display the completeness of the skin care process through at least one display type among charts, diagrams and scores.
  • the diagram includes circles corresponding to different skin care actions, wherein the completeness of the circles is used to indicate the completeness of the corresponding skin care actions.
  • the output module is further configured to: in response to the second operation acting on the ring, display skin care details corresponding to the ring.
  • the integrity of the skincare routine is included in the analysis report.
  • the output module is further configured to: display a virtual button, which is associated with the analysis report; and display the content of the analysis report in response to the third operation acting on the virtual button.
  • the output module is specifically configured to: send the integrity of the skin care process to the second electronic device, and the second electronic device displays the integrity.
  • the output module is further configured to: based on the completeness, output a strategy of skin care guidance for the user.
  • the output module when outputting the strategy of providing skin care guidance to the user, is specifically configured to: output the strategy of providing skin care guidance to the user in the form of pictures and/or videos.
  • the extraction module is further configured to: preprocess the image, where the preprocessing includes at least one of cropping, scaling, and rotation.
  • the extraction module extracts the target image feature from the image, it is specifically: extracting the target image feature from the preprocessed image.
  • the first electronic device is a smart mirror.
  • the completeness may include at least one of whether the skin care action is standard, whether the skin care action is omitted, and whether the skin care time is sufficient.
  • the above-mentioned classifiers may include at least one of the following classifiers:
  • the target image features include at least one of the following features:
  • embodiments of the present application provide an electronic device, including: a camera, a memory, and a processor.
  • the memory stores a computer program executable by the processor; when the processor reads and executes the computer program, the processor is caused to execute the method described in any of the above embodiments.
  • an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program includes at least a piece of code, and at least a piece of code can be executed by a processor to achieve the implementation in any of the above embodiments. the method described.
  • an embodiment of the present application further provides a chip, where the chip includes a programmable logic circuit and an input interface.
  • the input interface is used to obtain the data to be processed; the logic circuit is used to execute the method described in any of the above embodiments on the data to be processed, and the implementation principles and beneficial effects thereof are similar, and will not be repeated here.
  • an embodiment of the present application further provides a computer program product, when the computer program product is run on an electronic device, the electronic device is made to execute the method described in any of the foregoing embodiments, and the implementation principle and The beneficial effects are similar and will not be repeated here.
  • FIG. 1 is an example diagram of an external structure of a cosmetic mirror
  • 2a is a schematic diagram of the application of a cosmetic mirror provided by the embodiment of the application.
  • FIG. 2b is a schematic diagram of another cosmetic mirror application provided by the embodiment of the application.
  • FIG. 3 is an exemplary diagram of a communication system provided by an embodiment of the present application.
  • FIG. 4 is a flowchart of a skin care assistant method provided by an embodiment of the present application.
  • Fig. 5a is an example diagram of facial region key point
  • FIG. 5b is an example diagram of key points in the hand area and key points in the face area provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of a skin care auxiliary method provided by another embodiment of the present application.
  • FIG. 7a is an example diagram of an interface of a skin care process provided by an embodiment of the present application.
  • Fig. 7b is another interface example diagram of the skin care process provided by the embodiment of the application.
  • Fig. 8a is another interface example diagram of the skin care process provided by the embodiment of the application.
  • Fig. 8b is another interface example diagram of the skin care process provided by the embodiment of the application.
  • Fig. 8c is another interface example diagram of the skin care process provided by the embodiment of the present application.
  • FIG. 8d is another example interface diagram of the skin care process provided by the embodiment of the application.
  • FIG. 9a is an example diagram of an interface of the integrity of the skin care process provided by the embodiment of the application.
  • Fig. 9b is another interface example diagram of the integrity of the skin care process provided by the embodiment of the application.
  • FIG. 9c is another example interface diagram of the integrity of the skin care process provided by the embodiment of the present application.
  • 10a is another example interface diagram of the integrity of the skin care process provided by the embodiment of the application.
  • 10b is another example interface diagram of the integrity of the skin care process provided by the embodiment of the application.
  • Figure 10c is another example interface diagram of the integrity of the skin care process provided by the embodiment of the application.
  • 11a is another example interface diagram of the integrity of the skin care process provided by the embodiment of the application.
  • 11b is another example interface diagram of the integrity of the skin care process provided by the embodiment of the application.
  • FIG. 13 is a flowchart of a skin care auxiliary method provided by another embodiment of the application.
  • FIG. 14 is a schematic structural diagram of a skin care auxiliary device provided by an embodiment of the application.
  • FIG. 15 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • FIG. 16 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
  • the smart mirror integrates the mirror function and the information display function. It can interact with the electronic device through the built-in communication module to realize the function of information display.
  • the electronic device may be a mobile phone, a smart scale, a server, and the like.
  • the smart mirror interacts with the smart scale, and the smart scale sends the currently measured weight data to the smart mirror, which is displayed by the smart mirror; Mirror reading news and more to make the most of fragmented time.
  • beauty mirrors are mainly used to help users with daily skin care and makeup.
  • FIG. 1 is an example diagram of an external structure of a beauty mirror.
  • the beauty mirror includes but is not limited to a camera, a mirror, a display screen, a lighting device, and the like.
  • the video stream or image of the user in the process of using the beauty mirror can be obtained through the camera; through the mirror, the user can understand the mirror image of the user in the skin care process; through the display screen, information can be checked or consulted; the lighting device can provide Lighting, so that the user can see the skin condition more closely, etc.
  • the mirror function of the beauty mirror can be switched, for example, it is a mirror under the mirroring function, and it is a display screen under the display function.
  • the display screen is a part of the mirror surface, that is, the cosmetic mirror has both a mirroring function and a display function.
  • the beauty mirror when the user sits/stands in front of the beauty mirror or the user passes by the beauty mirror, the beauty mirror can be automatically activated through technologies such as infrared; or, the user clicks the start button on the beauty mirror to start the beauty mirror.
  • makeup mirror as shown in Figure 2a.
  • the beauty mirror starts when the user presses the start button, turns on the camera on the beauty mirror, and acquires images through the camera; or, in another implementation, the beauty mirror is installed with a beauty application (Appliction, Hereinafter referred to as APP), when the user opens the beauty APP named "skin care", the beauty mirror turns on the camera by default to start acquiring images, as shown in Figure 2b.
  • APP beauty application
  • the embodiments of the present application provide an auxiliary method, device and storage medium for skin care. Its skin care actions in the skin care process, whether the skin care time is complete, etc., assist users in skin care and provide guidance for their skin care.
  • the execution subject may be a smart mirror, or may be an electronic device with a certain computing power, such as a mobile phone, a computer, a server, and a server cluster.
  • FIG. 3 is an example diagram of a communication system provided by an embodiment of the present application.
  • the communication system includes a mobile phone and a smart mirror, and near-field communication is performed between the mobile phone and the smart mirror, so as to implement the technical solutions provided by the following embodiments of the present application.
  • the electronic device is described by taking a mobile phone as an example, but the embodiment of the present application is not limited by this.
  • the smart mirror is used as the executive body. Specifically, after acquiring the video stream or image of the user in the skin care process, the smart mirror analyzes it and executes the skin care auxiliary method.
  • FIG. 4 is a flowchart of an auxiliary method for skin care provided by an embodiment of the present application.
  • the embodiments of the present application provide a skin care assistance method, which is applied to a first electronic device having a camera, such as the above-mentioned smart mirror.
  • the skin care auxiliary method includes:
  • the first electronic device acquires an image through a camera.
  • the image includes the hand area and the face area.
  • the first operation is used to indicate the start of the skin care process.
  • the smart mirror detects the user's first operation, and in response to it, starts to acquire images.
  • the first operation may be the operation of the user clicking the start button on the smart mirror; or, the first operation may be the operation of the user clicking or double-clicking or long-pressing the beauty APP installed on the smart mirror, and so on.
  • the smart mirror detects whether each frame of images includes a hand area and a face area, wherein the video stream includes multiple frames of images.
  • the skin care process is the operation that the user uses the hand area to care for the facial area, including the nursing operation for different parts of the facial area, for example, the application of water, milk and cream for the forehead, face, chin and other areas; And, eye cream smear for the eye area, and so on. Therefore, it is first necessary to detect whether the image contains the hand area and the face area, and screen out the images that do not contain the hand area and the face area.
  • the smart mirror can detect whether the image contains a face region through a face detection algorithm or a face key point detection algorithm, and detect whether the image contains a hand region through a hand detection algorithm or a hand key point detection algorithm.
  • the face key point detection algorithm is used to detect the image, and the key point information of the face area can be obtained, for example, the information of the 68 key points in the face area shown in Figure 5a, including the eyes, mouth, nose, face Contour the key points at these positions, and by extracting the key points, the facial contour can be outlined. Referring to Figure 5b, the key points of the face and hands are shown.
  • the target image features are features related to the hand region and the face region.
  • the target image features contained in the image are further extracted, and the target image features are features related to the hand area and the face area. Since the motion of the hand region relative to the face region is to be analyzed, the features related to the hand region and the face region are extracted here.
  • the target image features may include at least one of the following features:
  • the Euclidean distance can be the preset key points in the face area and the hand area. Euclidean distance between preset keypoints for the region.
  • the embodiment of the present application is not limited, and may be selected according to actual needs or historical experience.
  • the multi-frame images are images generated in chronological order during the skin care process.
  • a skin care process corresponds to a skin care action.
  • information such as skin care actions can be obtained, which can reflect the integrity of the skin care process to a certain extent. For example, if the user forgets to apply eye cream, the completeness of the output skin care process will indicate the omission of this skin care action; or, the completeness of the output skin care process does not include this skin care action, and the user can determine the application of the skin care action according to the completeness of the output. The omission of this skin care action of eye cream.
  • the user should first apply a toner/softening lotion on the face by patting/using a cotton pad/using a beauty egg, etc., and then apply eye cream under the eyes and eyelids with the finger pads. Anti-aging, and finally apply facial essence, lotion, cream, sunscreen and isolation for whitening, among which, sunscreen and isolation are mainly applied during the day. Since the actions of washing face and applying toner/softening lotion are similar to those of applying lotion or cream, they are all massages on the facial area. Therefore, the action of applying water and cream can be defined as skin care action A, and the action of applying eye cream can be defined as skin care action A. It is defined as skin care action B, and the complete skin care process corresponds to: skin care action A-skin care action B-skin care action A.
  • the smart mirror can output the completeness of the skin care process through the display screen, that is, display the completeness of the skin care process, so that the user can understand the skin care process based on the completeness, and then improve or improve it. Improve.
  • the smart mirror when it does not have a display function, it can display the completeness of the skin care process through other electronic devices that communicate with it and have a display function. At this time, the smart mirror sends the output integrity of the skin care process to other electronic devices (ie, the second electronic device), and the other electronic devices display.
  • the completeness of the skin care process is set according to historical big data or actual needs, and its specific meaning is not limited in the embodiments of the present application.
  • the first electronic device acquires an image through a camera, and the image includes a hand area and a face area, and the first operation is used to indicate
  • the target image features are extracted from the image, and the target image features are the features related to the hand area and the face area.
  • Frame images are chronologically generated images during the skincare process.
  • the integrity may include whether the skin care actions are standard, whether the skin care actions are omitted and At least one of whether the skin care time is sufficient.
  • the description thereof is the same as the relevant description of S303 in the foregoing embodiment, and will not be repeated here.
  • whether the skin care action is standard and whether the skin care time is sufficient is similar to the description of whether the skin care action is omitted, and will not be repeated here.
  • outputting the integrity of the skin care process according to the target image features corresponding to the multi-frame images may include: outputting the integrity of the skin care process according to the target image features corresponding to the multi-frame images and the duration of the skin care process.
  • the facial area and skin care techniques applied by the skin care action can be known through the images during the duration. Based on the effect of the skin care action The facial area and skin care techniques can determine whether the skin care action is standard.
  • the above-mentioned outputting the integrity of the skin care process according to the target image features corresponding to the multi-frame images may include: outputting the skin care process according to the target image features corresponding to the multi-frame images, the facial area and the skin care technique acted on by the skin care action. Completion.
  • S303 shown in FIG. 4 will be further described below with reference to FIG. 6 .
  • S303 may include:
  • this step may be specifically: determining the skin care action corresponding to the multi-frame images according to the target image features corresponding to the multi-frame images and the pre-trained classifier; and determining the duration of the skin care action.
  • the classifier is a pre-trained model for classifying skin care actions, which may specifically include at least one of the following classifiers: support vector machine (SVM), random forest, decision tree, neural network Network and ensemble learning (adaboost), etc.
  • SVM support vector machine
  • random forest random forest
  • decision tree decision tree
  • neural network Network neural network Network
  • ensemble learning asdaboost
  • the target image features corresponding to the multi-frame images are used as input parameters for classification, and the skin care actions corresponding to the target image features are obtained through the processing of the classifier, that is, the output of the classifier.
  • the classifier may distinguish between skin care action A, skin care action B, and skin care action C.
  • the descriptions of skin care action A and skin care action B are as described above, and skin care action C indicates that no skin care is performed.
  • the duration of the skin care action can be determined based on at least one frame of image corresponding to the skin care action. Or, use the classifier to output the skin care action for the first time as the start time corresponding to the skin care action, and use the classifier to output the next skin care action for the first time as the end time corresponding to the skin care action and the start time of the next skin care action , and so on, until the end of the skin care process, so as to obtain the duration of each skin care action.
  • a preset duration may be set for each skin care action, and if the duration of the skin care action is greater than its corresponding preset duration, the user is prompted to proceed to the next step.
  • the beauty mirror still does not detect a skin care action after a preset duration T1 after being activated, the user is prompted to perform the first step: cleaning; after the user continues cleaning for a preset duration T2 , prompting the user to perform the second step: anti-aging; after the user's continuous anti-aging duration reaches the preset time T3, the user is prompted to perform the third step: whitening.
  • FIG. 7a if the beauty mirror still does not detect a skin care action after a preset duration T1 after being activated, the user is prompted to perform the first step: cleaning; after the user continues cleaning for a preset duration T2 , prompting the user to perform the second step: anti-aging; after the user's continuous anti-aging duration reaches the preset time T3, the user is prompted to perform the third step: whitening.
  • the user is prompted to perform the skin care action A; after the user continues the skin care action A for the preset duration T2, the user is prompted to perform the skin care action A.
  • the user performs the skin care action B; after the user continues the skin care action B for a preset duration T3, the user is prompted to perform the skin care action A.
  • the user may be prompted to continue the current skin care action through the display screen on the smart mirror.
  • the user is performing the first step: cleaning. Due to the limited time in the early morning, the user performs the first step: cleaning may have a shorter duration, that is, the second step: anti-aging, that is, the user continues cleaning for a duration t2 is less than the preset time T2, at this time, the display screen displays a prompt content similar to "continue cleaning", and when the display time of the prompt content reaches the preset time length T4, it automatically disappears. At this time, (t2+T2) ⁇ T4, the user There are two types of skin care actions in progress:
  • the user is prompted to perform the next skin care action.
  • the user is performing the first step: cleaning
  • the user is performing the first step: the cleaning duration t2 is greater than the preset duration T2
  • the display screen displays the prompt content of "Second step: anti-aging”
  • the user performs the second step: the anti-aging duration t3 is greater than the preset time T3, and at this time, the display screen displays the prompt content of "third step: whitening".
  • the smart mirror can also detect whether the skin care action is standardized, and when the skin care action is not standardized, prompt the user with a standardized action corresponding to the skin care action. For example, as shown in Figure 8c, the user is performing the first step: cleaning. At this time, the smart mirror detects that the skin care action is not standardized, and prompts the user with a prompt similar to "standard cleaning action" through the display screen; when the user clicks this Prompt content, the smart mirror displays the details of the standard cleaning action to the user, such as "massaging the face with fingertips” shown here or “beating the face with hands” not shown in the figure, etc.; and after the user clicks the details, return to "Step 1: Cleaning" is displayed.
  • the user is performing the first step: cleaning.
  • the smart mirror detects that the skin care action is not standardized, and prompts the user with a prompt similar to "standard cleaning action" through the display screen; when the user clicks this As soon as the content is prompted, the smart mirror demonstrates the standardized cleaning action to the user through the display screen; and after the user clicks the display screen again, it returns to display "Step 1: Cleaning".
  • the smart mirror can obtain the completeness of the skincare process according to the skincare action and its corresponding duration and preset duration; then, it outputs the completeness of the skincare process.
  • the preset skincare duration refers to the duration of the entire skincare process, and the preset duration refers to the duration of a skincare action.
  • the two are different concepts, wherein the preset skincare duration is greater than or equal to the preset duration.
  • extracting the target image feature from the image may include: if it is determined that the hand is displaced relative to the face between different frame images, extracting the target image feature. That is to say, only when it is determined that the hand is displaced relative to the face between different frame images, the target image features will be extracted to exclude the situation where the hand has been placed on the face without any movement.
  • the above-mentioned determining that the hand is displaced relative to the face between different frame images may include: extracting key point information of the face area and the hand area in the image; There is displacement relative to the face. Wherein, according to the key point information, it is determined that there is a displacement of the hand relative to the face between different frame images, including at least one of the following:
  • the relative position distance between the hand key point and the face key point changes between different frame images, it is determined that there is a displacement of the hand relative to the face between different frame images.
  • the key points may be specifically a face key point and a hand center point.
  • the displacement of the hand relative to the face means that in different frame images, the relative position distance between the center point of the hand and the key points of the face changes.
  • Figure 5b shows two temporally adjacent images. Among them, the user's mouth is closed in the image on the left side of the arrow, and the user's mouth in the image on the right side of the arrow is slightly open. In this case, only the shape of the mouth is changing, and the relative position of the key point of the face and the center point of the hand The distance change is small, and it can be considered that there is no displacement of the hand relative to the face between the two frames of images.
  • the above embodiment mainly introduces how to determine the integrity of the skin care process, and then describes how to output it.
  • outputting the completeness of the skin care process may include: outputting the completeness of the skin care process through at least one display type among charts, diagrams and scores.
  • Figures 9a and 9b output the completeness of the skin care process in graphical form
  • Figure 9c outputs the completeness of the skin care process in the form of a graph.
  • FIG. 9a in which, different circles represent different skin care actions, and the entire circle is clearly displayed, indicating that the corresponding skin care action is complete and sufficient; if only part of the circle is clearly displayed, it indicates that the corresponding skin care action continues. If the duration does not reach the preset duration, it is incomplete or insufficient; if all the circles are not obvious, it means that the corresponding skin care action is omitted.
  • 9a uses skin care action A and skin care action B as examples for description, but the embodiments of the present application are not limited thereto.
  • a ring can be further divided into at least two parts, each part is used to represent different skin care actions.
  • FIG. 9b illustrates skin care actions with three functions of anti-aging, cleaning and whitening as an example, but the embodiments of the present application are not limited thereto. Among them, anti-aging can be expressed as the skin care action process of applying eye cream.
  • the anti-aging item can be expressed by the length of the ring ;Cleaning can be expressed as the skin care action process of applying water lotion.
  • the user does not use water lotion or uses water lotion for skin care for a short period of time, it will be reflected in the length of the ring.
  • the longer the ring the completeness The higher it is; whitening can be reflected in skin care actions such as facial masks and the length of time.
  • FIG. 9c illustrates skin care action A and skin care action B as examples, wherein the integrity of skin care action A is 80, and the integrity of skin care action B is 97, but the embodiments of the present application are not limited by this. .
  • the score corresponding to the integrity is 0-100, wherein, the higher the score, the higher the integrity. Therefore, in the above example, skin care action A needs to be further refined.
  • the smart mirror displays the whitening details through the display screen (that is, the smart mirror displays the skin care details corresponding to the ring in response to the second operation acting on the ring), As shown in Figure 10a:
  • Integrity 80%, less than the preset duration (1.5 minutes).
  • the smart mirror displays the details of anti-aging through the display screen, as shown in Figure 10b:
  • a gesture indicating "great” is displayed, such as shown in Figure 10c.
  • the smart mirror can send the integrity to other electronic devices that communicate with it, such as the mobile phone shown in Figure 3. 11a, the user clicks or points to the completeness of the graphic form displayed by the smart mirror. In response to this action, the smart mirror sends the details of the corresponding area to the mobile phone.
  • the smart mirror generates an analysis report of completeness.
  • the smart mirror sends the analysis report to the mobile phone, and the mobile phone displays the content of the analysis report, as shown in Figure 11b. Show.
  • the analysis report may be a report corresponding to the entire skin care process, or may be an analysis report corresponding to one of the skin care actions, which is not limited in this application.
  • the above-mentioned skin care assistance method may further include: outputting a strategy of providing skin care guidance to the user based on the completeness. For example, in the form of pictures and/or videos, the strategy of providing skin care guidance to the user is output.
  • the omission of the skin care operation is indicated in the completeness of the skin care process output by the electronic device, and based on the skin care operation output how to correctly apply the eye cream, as shown in Figure 12 Display, use arrows or fingers and other upward and downward pictures to indicate or tell the user that eye cream should be applied on the upper and lower eyelids of the eye area, and use animation to interact with the user to instruct the user to correct and standard skin care actions.
  • the dashed arrows are used to indicate the succession of skin care actions.
  • relevant content of the skin care guidance is included, for example, as shown in FIG. 11a and FIG. 11b .
  • the skin care assistance method may include the following steps:
  • the first electronic device acquires an image through a camera.
  • the user's skin care video is generally obtained through a camera. Due to the need to reduce the power consumption of the electronic device, it is necessary to extract some key image frames from the skin care video in real time for subsequent processing.
  • Image the electronic device may acquire image frames at preset time intervals as key image frames based on a preset algorithm.
  • extracting the key image frame may specifically be: detecting whether the image contains the hand area and the face area, and if the image contains the hand area and the face area, keep it as the key image frame; if the image does not contain the hand area and the face area. The facial area or the facial area is discarded.
  • the preprocessing may include, but is not limited to, at least one of cropping, scaling, and rotation. Since the image size obtained by the smart mirror through the camera is relatively large, in order to reduce the running time and power consumption of the auxiliary skin care method, the image needs to be cropped, zoomed or rotated.
  • S704 determine whether there is a displacement of the hand relative to the face between different frame images.
  • the different frame images may be two frames of images that are adjacent in time, or in order to save the power consumption of the electronic device, the difference between the predetermined time intervals may be analyzed. frame images, the predetermined time interval is greater than the frame interval of the video of the camera. If there is no displacement of the hand relative to the face between different frame images, return to S701.
  • outputting the completeness of the skin care process may include: displaying the completeness of the skin care process.
  • the image is collected by the camera on the smart mirror.
  • Displaying the completeness of the skin care process includes: displaying the completeness of the skin care process on the display screen of the smart mirror, that is, the smart mirror is the executing subject of the skin care auxiliary method.
  • outputting the completeness of the skin care process may include: sending the completeness of the skin care process to the second electronic device, and the second electronic device displays the completeness of the skin care process.
  • any of the foregoing embodiments may be implemented independently, or may be implemented by any combination of at least two of the foregoing embodiments, which is not limited.
  • the operations and steps implemented by the first electronic device may also be implemented by components (eg, chips or circuits) usable in the first electronic device, which are not limited in this embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a skin care auxiliary device according to an embodiment of the present application.
  • the skin care auxiliary device may be a first electronic device, or may be a component of the first electronic device (eg, an integrated circuit, a chip, etc.), or may be other communication Operation of an electronic device.
  • the skin care auxiliary device 70 of this embodiment includes: an acquisition module 71 , an extraction module 72 and an output module 73 .
  • the skin care auxiliary device 70 of this embodiment can implement the solution as in any of the above-mentioned embodiments through the acquisition module 71, the extraction module 72 and the output module 73, and its implementation principle and technical effect are similar, and will not be repeated here.
  • FIG. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 600 in this embodiment may be the first electronic device (or a component usable for the first electronic device) mentioned in the foregoing method embodiments.
  • the electronic device 600 may be used to implement the method corresponding to the first electronic device described in the foregoing method embodiments.
  • the electronic device is a smart mirror.
  • the electronic device 600 includes: at least one processor 601 , at least one network interface 604 or other user interface 603 , memory 605 , and at least one communication bus 602 .
  • the communication bus 602 is used to realize the connection communication between these components.
  • the electronic device 600 optionally includes a user interface 603, including a display (eg, a touch screen, etc.).
  • Memory 605 may include read only memory and random access memory, and provides instructions and data to processor 601 .
  • a portion of memory 605 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • memory 605 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:
  • Operating system 6051 including various system programs for implementing various basic services and processing hardware-based tasks;
  • the application module 6052 includes various application programs, such as the aforementioned beauty APP, media player (Media Player), browser (Browser), etc., for implementing various application services.
  • various application programs such as the aforementioned beauty APP, media player (Media Player), browser (Browser), etc., for implementing various application services.
  • the processor 601 by calling the program or instruction stored in the memory 605, the processor 601 is configured to execute the method described in any of the above embodiments, which is not repeated in this embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
  • the electronic device 20 includes a processor 21 , a memory 22 , a communication module 23 , a camera 24 , a display screen 25 and an audio and video module 26 connected to the processor 21 .
  • the processor 21 may include one or more processing units, for example, the processor 21 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processor
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural-network processing unit neural-network processing unit
  • a memory may also be provided in the processor 21 for storing instructions and data.
  • the memory in the processor 21 is a cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 21 . If the processor 21 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the waiting time of the processor 21 is reduced, thereby improving the efficiency of the system.
  • processor 21 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
  • I2S interface can be used for audio communication.
  • processor 21 may contain multiple sets of I2S buses.
  • the processor 21 can be coupled with the audio and video module 26 through the I2S bus, so as to realize the communication between the processor 21 and the audio and video module 26 .
  • Memory 22 may be used to store computer-executable program code including instructions.
  • the memory 22 may include a program storage area and a data storage area.
  • the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
  • the storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 20 and the like.
  • the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the processor 21 executes various functional applications and data processing of the electronic device 20 by executing the instructions stored in the memory 22 and/or the instructions stored in the memory provided in the processor.
  • the communication module 23 can provide a wireless communication solution including 2G/3G/4G/5G, etc. applied on the electronic device 20 .
  • the communication module 23 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like.
  • the communication module 23 can receive electromagnetic waves through the antenna, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor 21 for demodulation.
  • the communication module 23 can also amplify the signal modulated by the modulation and demodulation processor 21, and then convert it into electromagnetic waves for radiation through the antenna.
  • at least part of the functional modules of the communication module 23 may be provided in the processor 21 .
  • At least part of the functional modules of the communication module 23 may be provided in the same device as at least part of the modules of the processor 21 .
  • the communication module 23 may receive the video shown in this application, and store the received video in the memory 22 .
  • the camera 24 can capture images or videos.
  • the camera 24 can capture the video shown in this application under the control of the processor 21 and store the video in the memory 22 .
  • the display screen 25 is used to display images, videos, and the like.
  • the display screen 25 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode). diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the electronic device 20 may include one or N display screens 25 , where N is a positive integer greater than one.
  • the audio and video module 26 can play the video stored in the memory 22 and display the played video through the display screen 25 .
  • the audio and video module 26 is used to convert digital audio information to analog audio signal output, and also to convert analog audio input to digital audio signal.
  • the audio-video module 26 may also be used to encode and decode audio signals.
  • the audio and video module 26 may be provided in the processor 21 , or some functional modules of the audio and video module 26 may be provided in the processor 21 .
  • the audio-video module 26 may also be used to compress or decompress digital video.
  • Electronic device 20 may support one or more video codecs. In this way, the electronic device 20 can play or record videos in various encoding formats, such as: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • An embodiment of the present application further provides a chip, the chip includes a programmable logic circuit and an input interface, the input interface is used to acquire data to be processed, and the logic circuit is used to perform any one of the above implementations on the data to be processed.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program includes at least one piece of code, and the at least one piece of code can be executed by a processor, and the implementation is as described above.
  • a computer program is stored in the computer-readable storage medium, and the computer program includes at least one piece of code, and the at least one piece of code can be executed by a processor, and the implementation is as described above.
  • the implementation principles and beneficial effects of the methods described in any of the embodiments are similar, and details are not repeated here.
  • the embodiment of the present application also provides a computer program product, when the computer program product runs on an electronic device, the electronic device is made to execute the method described in any of the above embodiments, and the implementation principle and beneficial effects thereof are similar, No further description is given here.
  • All or part of the steps for implementing the above method embodiments may be completed by program instructions related to hardware.
  • the aforementioned program can be stored in a readable memory.
  • the steps including the above method embodiments are executed; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state hard disk, magnetic tape (magnetic tape), floppy disk (floppy disk), optical disc (optical disc) and any combination thereof.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种护肤辅助方法、设备及存储介质，此方法包括：响应于作用于第一电子设备的第一操作，第一电子设备通过摄像头获取图像（S301），图像中包含手部区域与面部区域，第一操作用于指示护肤过程的开始；在图像中提取目标图像特征（S302），目标图像特征为与手部区域和面部区域相关的特征；根据多帧图像对应的目标图像特征，输出护肤过程的完整度（S303），其中多帧图像为护肤过程中按时间顺序生成的图像。通过输出护肤过程的完整度，可以帮助用户了解其在护肤过程中的护肤动作、护肤时间是否完整等，辅助用户护肤，进而为其提供指导。

Description

护肤辅助方法、设备及存储介质
本申请要求于2020年08月24日提交中国专利局、申请号为202010857343.5、申请名称为“护肤辅助方法、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能家居，尤其涉及一种护肤辅助方法、设备及存储介质。
背景技术
随着生活水平的不断提高，人们对护肤和化妆的要求也越来越高。因此，如何进行合理的护肤/化妆，以及找到适合自己的护肤、化妆产品，显得至关重要。
目前，为了帮助用户找到适合自己的化妆产品，通过分析采集的试妆人员的试妆图像，获得试妆人员的模拟的实际化妆过程的动作行为，对试妆图像进行图像处理，进行虚拟着妆，得到试妆人员的虚拟化妆效果，用户可根据虚拟化妆效果选择适合的化妆产品。但是该方案并不能对用户的日常护肤进行指导。
发明内容
本申请提供一种护肤辅助方法、设备及存储介质，以对用户的护肤进行指导。
第一方面，本申请实施例提供一种护肤辅助方法，应用于具有摄像头的第一电子设备，首先，响应于作用于第一电子设备的第一操作，第一电子设备通过摄像头获取图像，该图像包含手部区域与面部区域，第一操作用于指示护肤过程的开始；在图像中提取目标图像特征，目标图像特征为与手部区域和面部区域相关的特征；之后，根据多帧图像对应的目标图像特征，输出护肤过程的完整度，其中多帧图像为护肤过程中按时间顺序生成的图像。通过呈现护肤过程的完整度，可以帮助用户了解其在护肤过程中的护肤动作、护肤时间是否完整等，辅助用户护肤，为其提供指导。
一种可能的实施方式中，根据多帧图像对应的目标图像特征，输出护肤过程的完整度，可以包括：根据多帧图像对应的目标图像特征，确定多帧图像对应的护肤动作及每一护肤动作的持续时长；根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度。
上述过程中，首先根据目标图像特征确定护肤动作及其持续时长，进而根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度，提供一种具体的实现手段。
可选地，根据多帧图像对应的目标图像特征，确定多帧图像对应的护肤动作及每一护肤动作的持续时长，可以包括：根据多帧图像对应的目标图像特征和预先训练好的分类器，确定多帧图像对应的护肤动作；确定护肤动作的持续时长。
上述过程中，将目标图像特征作为预先训练好的分类器的输入参数，分类器的输出即输入的目标图像特征对应的护肤动作。在分类器识别到下一护肤动作时，可以理解，之前护肤动作结束，从初次识别待该护肤动作到识别到下一护肤动作，此段时间即该护肤动作的持续时长。
一种可能的实施方式中，确定护肤动作的持续时长之后，护肤辅助方法还可以包括：若护肤动作的持续时长大于预设时长，则提示用户进行下一护肤动作；或者，若护肤动作的持续时长小于预设时长，则提示用户继续进行护肤动作。
上述过程中，为每一护肤动作设置其应该持续的时长，也就是预设时长。小于该预设时长认为该护肤动作不完整，提示用户继续该护肤动作；大于该预设时长则认为该护肤动作已足够，提示用户进行下一护肤动作，从而可以使得用户明确后续动作，进一步提升用户体验。
一种可能的实施方式中，提示对应的时长为预设值，提示持续的时长大于预设值后，提示自动消失。在护肤过程中，用户关注于使用手部进行护肤，因此，对于提示等通过设置对应的时长等自动化手段，减少人为参与，进一步的提升用户体验。
一种可能的实施方式中，确定多帧图像对应的护肤动作之后，护肤辅助方法还包括：检测护肤动作是否规范；若护肤动作不规范，则提示护肤动作对应的规范动作。
在上述过程中，通过检测护肤动作的规范性，确定是否显示对应的规范动作，以在用户护肤出现问题时，对其进行指导。
一种可能的实施方式中，根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度，可以包括：根据护肤动作及每一护肤动作的持续时长、预设时长，确定护肤过程的完整度；输出护肤过程的完整度。
一种可能的实施方式中，在图像中提取目标图像特征，可以包括：若确定在不同帧图像之间手部相对面部存在位移，则在图像中提取目标图像特征。
上述过程中，在提取目标图像特征之前，首先判断图像中手部相对面部是否存在位移，只有在图像中手部相对面部存在位移时，才会在图像提取目标图像特征；否则，丢弃图像。
一种具体实现中，确定在不同帧图像之间手部相对面部存在位移，可以包括：提取图像中面部区域和手部区域的关键点信息；根据关键点信息，确定不同帧图像之间手部相对面部存在位移。
进一步地，根据关键点信息，确定不同帧图像之间手部相对面部存在位移，包括以下至少一种：
当不同帧图像之间手部关键点与面部关键点的相对位置距离改变，确定不同帧图像之间手部相对面部存在位移；
当不同帧图像之间手部关键点的位置信息改变，确定不同帧图像之间手部相对面部存在位移。
在上述过程中，提供两种具体的确定手部相对面部存在位移的实现方式：一种是通过手部关键点与面部关键点的相对位置距离改变；另一种是通过手部关键点的位置信息改变。
一种可能的实施方式中，输出护肤过程的完整度，可以包括：通过图表、图示和评分中的至少一种显示类型，显示护肤过程的完整度。这里，提供几种具体的完整度的显示类型。
可选地，图示包含不同护肤动作对应的圆环，其中圆环的完整程度用于指示对应护肤动作的完整度。
进一步地，护肤辅助方法还可以包括：响应于作用于圆环的第二操作，显示对应圆环的护肤详情。例如，用户点击或双击(即第二操作)圆环，电子设备对该点击或双击动作做出响应，显示对应圆环的护肤详情。
一种可能的实施方式中，护肤过程的完整度包含于分析报告中。此时，输出护肤过程的完整度，包括：显示虚拟按钮，该虚拟按钮用于指示分析报告；响应于作用于虚拟按钮的第三操作，显示分析报告的内容。
一种可能的实施方式中，输出护肤过程的完整度，包括：发送护肤过程的完整度给第二电子设备，由第二电子设备显示完整度。该实施方式是考虑到第一电子设备的显示限制，例如智能镜的显示屏较小，此时，通过将完整度发送给第二电子设备，例如手机等显示屏相对较大的电子设备，通过第二电子设备显示完整度；除显示屏面积的考量，另外，智能镜体相对固定，不方便携带，而手机等便携设备可以随身携带，这样用户可以随时随地查看护肤过程的完整度。
一种可能的实施方式中，根据多帧图像对应的目标图像特征，输出护肤过程的完整度之后，护肤辅助方法还包括：基于完整度，输出对用户进行护肤指导的策略。通过完整度，用户可以了解其该次护肤过程的详情，例如护肤过程是否存在的问题，进一步基于其中的问题输出相应策略，对用户进行护肤指导。
可选地，输出对用户进行护肤指导的策略，可以包括：通过图片和/或视频的形式，输出对用户进行护肤指导的策略。
一种可能的实施方式中，第一电子设备通过摄像头获取图像之后，护肤辅助方法还可以包括：对图像进行预处理，该预处理包括裁剪、缩放和旋转中的至少一种。相应地，在图像中提取目标图像特征，包括：在预处理后的图像中提取目标图像特征。
上述过程中，通过裁剪、缩放和旋转等预处理，降低护肤辅助方法的运行时间以及功耗。
第二方面，本申请实施例提供一种护肤辅助装置，应用于具有摄像头的第一电子设备。护肤辅助装置包括：
获取模块，用于响应于作用于第一电子设备的第一操作，通过第一电子设备的摄像头获取图像，图像中包含手部区域与面部区域，第一操作用于指示护肤过程的开始；
提取模块，用于在图像中提取目标图像特征，目标图像特征为与手部区域和面部区域相关的特征；
输出模块，用于根据多帧图像对应的目标图像特征，输出护肤过程的完整度，多帧图像为护肤过程中按时间顺序生成的图像。
该护肤辅助装置通过获取模块、提取模块和输出模块可以实现如上述任一实施方式中的方案，其实现原理和技术效果类似，此处不再赘述。
一些实施例中，输出模块，包括：
确定单元，用于根据多帧图像对应的目标图像特征，确定多帧图像对应的护肤动作及每一护肤动作的持续时长；
输出单元，用于根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度。
一些实施例中，确定单元具体用于：根据多帧图像对应的目标图像特征和预先训练好的分类器，确定多帧图像对应的护肤动作；以及，确定护肤动作的持续时长。
一些实施例中，输出单元还用于：在确定单元确定护肤动作的持续时长之后，若护肤动作的持续时长大于预设时长，则提示用户进行下一护肤动作；或者，若护肤动作的持续时长小于预设时长，则提示用户继续进行护肤动作。
一些实施例中，提示对应的时长为预设值，提示持续的时长大于预设值后，提示自动消失。
一些实施例中，输出单元还用于：在确定单元确定多帧图像对应的护肤动作之后，检测护肤动作是否规范；若护肤动作不规范，则提示护肤动作对应的规范动作。
一些实施例中，输出单元具体用于：根据护肤动作及每一护肤动作的持续时长、预设时长，确定护肤过程的完整度；输出护肤过程的完整度。
一些实施例中，提取模块具体用于：若确定在不同帧图像之间手部相对面部存在位移，则在图像中提取目标图像特征。
一些实施例中，提取模块在确定在不同帧图像之间手部相对面部存在位移时，具体用于：提取图像中面部区域和手部区域的关键点信息；根据关键点信息，确定不同帧图像之间手部相对面部存在位移。
一些实施例中，提取模块在根据关键点信息，确定不同帧图像之间手部相对面部存在位移时，包括以下至少一种：
当不同帧图像之间手部关键点与面部关键点的相对位置距离改变，确定不同帧图像之间手部相对面部存在位移；
当不同帧图像之间手部关键点的位置信息改变，确定不同帧图像之间手部相对面部存在位移。
一些实施例中，输出模块具体用于：通过图表、图示和评分中的至少一种显示类型，显示护肤过程的完整度。
一些实施例中，图示包含不同护肤动作对应的圆环，其中圆环的完整程度用于指示对应护肤动作的完整度。
一些实施例中，输出模块还用于：响应于作用于圆环的第二操作，显示对应圆环的护肤详情。
一些实施例中，护肤过程的完整度包含于分析报告中。此时，输出模块还用于：显示虚拟按钮，虚拟按钮关联于分析报告；响应于作用于虚拟按钮的第三操作，显示分析报告的内容。
一些实施例中，输出模块具体用于：发送护肤过程的完整度给第二电子设备，由第二电子设备显示完整度。
一些实施例中，输出模块还用于：基于完整度，输出对用户进行护肤指导的策略。
一些实施例中，输出模块在输出对用户进行护肤指导的策略时，具体用于：通过图片和/或视频的形式，输出对用户进行护肤指导的策略。
一些实施例中，提取模块还用于：对图像进行预处理，预处理包括裁剪、缩放和旋转中的至少一种。相应地，提取模块在图像中提取目标图像特征时，具体为：在预处理后的图像中提取目标图像特征。
在上述任一种可能的实施方式的基础上：
可选地，第一电子设备为智能镜。
其中，完整度可以包括护肤动作是否标准、护肤动作是否遗漏和护肤时间是否充足等中的至少一种。
可选地，上述提及的分类器可以包括以下分类器中的至少一种：
支持向量机、随机森林、决策树、神经网络和集成学习，等等。
可选地，目标图像特征包括以下特征中的至少一种：
手部关键点和面部关键点的欧氏距离；
手部关键点和面部关键点的曼哈顿距离；
手部关键点和面部关键点的相对位移；
光流特征；
等等。
第三方面，本申请实施例提供一种电子设备，包括：摄像头、存储器和处理器。其中，存储器上存储有可供处理器执行的计算机程序；当处理器读取并执行计算机程序时，使得处理器执行如上任一实施例中所述的方法。
第四方面，本申请实施例提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，计算机程序包含至少一段代码，至少一段代码可由处理器执行，实现如上任一实施例中所述的方法。
第五方面，本申请实施例还提供了一种芯片，芯片包括可编程逻辑电路和输入接口。输入接口用于获取待处理的数据；逻辑电路用于对待处理的数据执行上述任一实施例所述的方法，其实现原理以及有益效果类似，此处不再进行赘述。
第六方面，本申请实施例还提供了一种计算机程序产品，当所述计算机程序产品在电子设备上运行时，使得所述电子设备执行上述任一实施例所述的方法，其实现原理以及有益效果类似，此处不再进行赘述。
本申请的这些和其它方面在以下(多个)实施例的描述中会更加简明易懂。
附图说明
图1为美妆镜的一外部结构示例图；
图2a为本申请实施例提供的一美妆镜应用示意图；
图2b为本申请实施例提供的另一美妆镜应用示意图；
图3为本申请实施例提供的一通信系统的示例图；
图4为本申请一实施例提供的护肤辅助方法的流程图；
图5a为面部区域关键点的一示例图；
图5b为本申请实施例提供的手部区域关键点和面部区域关键点的一示例图；
图6为本申请另一实施例提供的护肤辅助方法的流程图；
图7a为本申请实施例提供的护肤过程的一界面示例图；
图7b为本申请实施例提供的护肤过程的另一界面示例图；
图8a为本申请实施例提供的护肤过程的又一界面示例图；
图8b为本申请实施例提供的护肤过程的又一界面示例图；
图8c为本申请实施例提供的护肤过程的又一界面示例图；
图8d为本申请实施例提供的护肤过程的又一界面示例图；
图9a为本申请实施例提供的护肤过程的完整度的一界面示例图；
图9b为本申请实施例提供的护肤过程的完整度的另一界面示例图；
图9c为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图10a为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图10b为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图10c为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图11a为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图11b为本申请实施例提供的护肤过程的完整度的又一界面示例图；
图12为本申请一实施例提供的护肤指导的示例图；
图13为本申请又一实施例提供的护肤辅助方法的流程图；
图14为本申请一实施例提供的护肤辅助装置的结构示意图；
图15为本申请一实施例提供的电子设备的结构示意图；
图16为本申请另一实施例提供的电子设备的结构示意图。
具体实施方式
应当理解，本文中使用的术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“/”表示“或”的关系。
智能镜，集镜面功能与信息显示功能于一体，通过内置的通信模块与电子设备进行交互，可以实现信息显示的功能。所述电子设备可以是手机、智能秤、服务器等。示例性地，智能镜与智能秤进行交互，智能秤将当前测量的体重数据发送至智能镜，由智能镜进行显示；或者，利用智能镜与电子设备的交互，用户可以利用早晚洗漱时间通过智能镜阅读新闻等，以充分利用碎片时间。美妆镜作为智能镜的一种具体实现方式，其主要的功能就是帮助用户进行日常的护肤和化妆。
其中，图1为美妆镜的一外部结构示例图。参考图1，美妆镜包括但不限于摄像头、镜面、显示屏和照明装置等。其中，通过摄像头可以获取用户在使用美妆镜过程中的视频流或图像；通过镜面，用户可以了解其在护肤过程中的镜像；通过显示屏可以进行信息的查收或查阅等；照明装置可以提供光照，以便用户可以更仔细的查看皮肤状况等。可选地，在不同模式下，美妆镜的镜子功能可以切换，例如，在镜像功能下为镜面，在显示功能下为显示屏。或者，显示屏为镜面的一部分，即所述美妆镜同时具备镜像功能和显示功能。
实际应用中，当用户坐/站在美妆镜前或者用户从美妆镜前经过时，美妆镜可以通过红外等技术实现自启动；或者，用户点击美妆镜上的启动按钮，启动美妆镜，如图2a所示。一种实现中，美妆镜从用户按下启动按钮开始，打开美妆镜上的摄像头，通过摄像头获取图像；或者，另一种实现中，美妆镜中安装有美妆应用程序(Appliction，以下简称APP)，用户打开该名称为“护肤”的美妆APP时，美妆镜默认打开摄像头开始获取图像，如图 2b所示。
根据对用户的日常护肤过程调研，在清晨，用户在美妆镜前进行护肤的时长大约10分钟。由于用户清晨赶时间只追求基本步骤，没有时间多步骤的精细护肤，在上妆之前需要确认护肤品被充分吸收。另外，在晚间，用户在美妆镜前护肤的时长大约25分钟。由于用户晚间时间充裕，能够有大量的时间用在护肤过程中，例如确保面膜与面部足够贴合，配合手法确保更好效果。
基于上述调研结果，以及现有试妆方案不能对用户的护肤进行指导的缺陷，本申请实施例提供一种护肤辅助方法、设备及存储介质，通过呈现用户护肤过程的完整度，可以帮助用户了解其在护肤过程中的护肤动作、护肤时间是否完整等，辅助用户护肤，为其护肤提供指导。
需说明的是，对于本申请实施例提供的护肤辅助方案，其执行主体可以是智能镜，也可以是手机、计算机、服务器、服务器集群等具备一定算力的电子设备。
示例地，若以智能镜以外的电子设备作为执行主体，则智能镜将其获取的视频流或图像发送给执行主体，由执行主体对视频流或图像进行分析，执行护肤辅助方法，具体场景图如图3所示例。图3为本申请实施例提供的一通信系统的示例图。如图3所示，该通信系统包括手机和智能镜，手机和智能镜之间进行近场通信，以实施下述各本申请实施例提供的技术方案。该图例中，电子设备以手机为例进行说明，但本申请实施例不以此为限制。
或者，由智能镜作为执行主体。具体地，智能镜在获取用户在护肤过程中的视频流或图像后，对其进行分析，执行护肤辅助方法。
图4为本申请一实施例提供的护肤辅助方法的流程图。本申请实施例提供一种护肤辅助方法，应用于具有摄像头的第一电子设备，例如如上所述的智能镜。如图4所示，该护肤辅助方法包括：
S301、响应于作用于第一电子设备的第一操作，第一电子设备通过摄像头获取图像。
其中，图像中包含手部区域与面部区域。第一操作用于指示护肤过程的开始。参考图2a和图2b，智能镜检测到用户的第一操作，并对其进行响应，开始获取图像。其中，该第一操作可以是用户点击智能镜上启动按钮的操作；或者，该第一操作可以是用户点击或双击或长按智能镜上已安装的美妆APP的操作，等等。
实际应用中，智能镜在获取用户在护肤过程中的视频流和/或周期性获取图像之后，检测各帧图像中是否包含手部区域与面部区域，其中，视频流包含多帧图像。可以理解，护肤过程即用户使用手部区域对面部区域进行护理的操作，包括针对面部区域不同部位的护理操作，例如，针对额部、脸部、下巴等区域进行的水、乳、霜涂抹；以及，针对眼部的眼霜涂抹，等等。因此，首先需要检测图像中是否包含手部区域与面部区域，对不包含手部区域与面部区域的图像进行筛除。
具体地，智能镜可以通过人脸检测算法或人脸关键点检测算法检测图像中是否包含面部区域，通过手部检测算法或手部关键点检测算法检测图像中是否包含手部区域。至于人脸检测算法和手部关键点检测算法的具体描述可参考相关技术，此处不再赘述。其中，采用人脸关键点检测算法对图像进行检测，可以得到面部区域的关键点信息，例如图5a所示的面部区域的68个关键点的信息，包括眼周、嘴周、鼻翼、脸部轮廓这些位置的关键点， 通过提取关键点，可以将面部轮廓勾勒出来。参考图5b，示出面部和手部的关键点。
S302、在图像中提取目标图像特征。
其中，目标图像特征为与手部区域和面部区域相关的特征。
对于包含手部区域与面部区域的图像，进一步提取图像中包含的目标图像特征，该目标图像特征为与手部区域和面部区域相关的特征。由于要分析手部区域相对面部区域的动作，因此这里提取的是与手部区域和面部区域相关的特征。
作为示例，目标图像特征可以包括以下特征中的至少一种：
手部关键点和面部关键点的欧氏距离；
手部关键点和面部关键点的曼哈顿距离；
手部关键点和面部关键点的相对位移；
光流特征；
等等。
关于欧氏距离、曼哈顿距离、相对位移和光流特征的说明，可参考相关技术，本申请实施例不予赘述。
仍以面部区域的68个关键点，以及手部区域的21个关键点(包括位于关节和指尖的关键点)为例，其中，欧氏距离可以是面部区域的预设关键点与手部区域的预设关键点之间的欧氏距离。至于预设关键点的选取，本申请实施例不予限制，具体可根据实际需求或历史经验进行选取。
S303、根据多帧图像对应的目标图像特征，输出护肤过程的完整度。
其中，多帧图像为护肤过程中按时间顺序生成的图像。
示例地，护肤过程对应有护肤动作，基于护肤过程中按时间顺序生成的多帧图像对应的目标图像特征，可以得到护肤动作等信息，这些信息可以在一定程度上反应护肤过程的完整度。例如，若用户忘记涂抹眼霜，输出的护肤过程的完整度中会指示这一护肤动作的遗漏；或者，输出的护肤过程的完整度不包含这一护肤动作，用户通过输出的完整度可以确定涂抹眼霜这一护肤动作的遗漏。例如，通常情况下，用户洗脸后要先通过用手拍/用化妆棉/用美妆蛋等在面部涂抹爽肤水/柔肤水进行清洁，然后用手指指腹在眼部下方、眼皮涂抹眼霜进行抗衰，最后涂抹面部精华、乳液、面霜、防晒和隔离等进行美白，其中，防晒和隔离的涂抹主要在日间。由于洗脸、涂抹爽肤水/柔肤水的动作与涂抹乳液或霜的动作类似，都是对面部区域的按摩，因此，这里可以将涂抹水乳霜的动作定义为护肤动作A，将涂抹眼霜的动作定义为护肤动作B，完整的护肤过程对应为：护肤动作A-护肤动作B-护肤动作A。
在一种具体的实现方式中，智能镜可以通过显示屏输出护肤过程的完整度，也即显示护肤过程的完整度，这样，用户可以基于该完整度了解其护肤过程，进而对其进行完善或改进。
在另一种具体的实现方式中，当智能镜不具备显示功能时，其可以通过与其通信的其他具备显示功能的电子设备显示护肤过程的完整度。此时，智能镜将输出的护肤过程的完整度发送给其他电子设备(即第二电子设备)，由其他电子设备进行显示。
其中，护肤过程的完整度是根据历史大数据或实际需求设置的，对其具体含义，本申请实施例不予限制。
本申请实施例提供的护肤辅助方法，首先，响应于作用于第一电子设备的第一操作，第一电子设备通过摄像头获取图像，图像中包含手部区域与面部区域，第一操作用于指示护肤过程的开始，并在图像中提取目标图像特征，目标图像特征为与手部区域和面部区域相关的特征；之后，根据多帧图像对应的目标图像特征，输出护肤过程的完整度，其中多帧图像为护肤过程中按时间顺序生成的图像。通过护肤过程的完整度的输出，可以帮助用户了解其在护肤过程中的护肤动作、护肤时间是否完整等，辅助用户护肤，为其提供指导。
考虑到用户在美妆镜前进行护肤时，正确的护肤动作可以大大促进肌肤对护肤品的吸收，因此，在上述实施例的基础上，完整度可以包括护肤动作是否标准、护肤动作是否遗漏和护肤时间是否充足中的至少一种。其中，对于护肤动作是否遗漏，其说明与上述实施例中S303的相关描述相同，此处不再赘述。另外，护肤动作是否标准和护肤时间是否充足，与护肤动作是否遗漏的相关描述类似，这里也不再赘述。
示例性地，由于电子设备输出的护肤过程的完整度是根据多帧图像对应的目标图像特征获得的，此多帧图像对应一时长，通过将该时长与总的预设护肤时长进行比较，可以反映护肤时间是否充足。该情况下，上述根据多帧图像对应的目标图像特征，输出护肤过程的完整度，可以包括：根据多帧图像对应的目标图像特征及护肤过程持续的时长，输出护肤过程的完整度。
另外，对于护肤动作是否标准，可以理解，在一护肤动作的持续时间内，该护肤动作所作用的面部区域及护肤手法是可以通过该持续时间内的图像获知的，基于该护肤动作所作用的面部区域及护肤手法，可以确定该护肤动作是否标准。该情况下，上述根据多帧图像对应的目标图像特征，输出护肤过程的完整度，可以包括：根据多帧图像对应的目标图像特征、护肤动作所作用的面部区域及护肤手法，输出护肤过程的完整度。
以下结合图6，对图4所示的S303进行进一步说明。参考图6，S303可以包括：
S3031、根据多帧图像对应的目标图像特征，确定多帧图像对应的护肤动作及每一护肤动作的持续时长。
一些实施例中，该步骤可以具体为：根据多帧图像对应的目标图像特征和预先训练好的分类器，确定多帧图像对应的护肤动作；确定护肤动作的持续时长。
其中，分类器是预先训练好的用于对护肤动作进行分类的模型，其具体可以包括以下分类器中的至少一种：支持向量机(support vector machine，SVM)、随机森林、决策树、神经网络和集成学习(adaboost)等。
具体地，将多帧图像对应的目标图像特征作为分类的输入参数，经分类器的处理，得到目标图像特征对应的护肤动作，也就是分类器的输出。作为一种示例，分类器可以区分护肤动作A、护肤动作B和护肤动作C。其中，护肤动作A和护肤动作B的相关描述如前所述，护肤动作C表示未进行护肤。
在确定护肤动作之后，基于该护肤动作对应的至少一帧图像，可以确定该护肤动作的持续时长。或者，将分类器第一次输出护肤动作作为该护肤动作对应的起始时刻，将分类器第一次输出下一护肤动作作为该护肤动作对应的结束时刻以及该下一护肤动作的起始时刻，以此类推，直至护肤过程的结束，从而得到每一护肤动作的持续时长。
具体应用中，可为每个护肤动作设置一预设时长，若护肤动作的持续时长大于其对应的预设时长，则提示用户进行下一步。例如，参考图7a，若美妆镜在启动后，在经历预设 时长T1后仍没有检测到护肤动作，则提示用户进行第一步：清洁；在用户持续清洁的时长达到预设时长T2后，提示用户进行第二步：抗衰；在用户持续抗衰的时长达到预设时长T3后，提示用户进行第三步：美白。或者，参考图7b，若美妆镜在启动后，在经历时长T1后仍没有检测到护肤动作，则提示用户进行护肤动作A；在用户持续护肤动作A的时长达到预设时长T2后，提示用户进行护肤动作B；在用户持续护肤动作B的时长达到预设时长T3后，提示用户进行护肤动作A。
需说明的是，以上仅给出两种提示示例，但本申请不以此为限，还可以通过其他提示方式通知用户当前应进行的护肤动作。
进一步地，在护肤动作的持续时长小于预设时长时，可通过智能镜上的显示屏提示用户继续当前护肤动作。例如，参考图8a，用户正在进行第一步：清洁，由于清晨时间有限，用户进行第一步：清洁的持续时长可能较短，即开始第二步：抗衰，即用户持续清洁的时长t2小于预设时长T2，此时，显示屏显示类似“继续清洁”的提示内容，并在提示内容的显示时长达到预设时长T4后，自动消失，此时，(t2+T2)<T4，用户正在进行的护肤动作有两种情况：
①回到第一步：清洁；
②继续第二步：抗衰。
或者，若护肤动作的持续时长大于预设时长，则提示用户进行下一护肤动作。例如，参考图8b，用户正在进行第一步：清洁，用户进行第一步：清洁的持续时长t2大于预设时长T2，此时，显示屏显示“第二步：抗衰”的提示内容；用户进行第二步：抗衰的持续时长t3大于预设时长T3，此时，显示屏显示“第三步：美白”的提示内容。
一些实施例中，智能镜还可以检测护肤动作是否规范，并在护肤动作不规范时，提示该护肤操作对应的规范动作给用户。例如，如图8c所示，用户正在进行第一步：清洁，此时，智能镜检测到护肤动作不规范，通过显示屏提示类似“规范清洁动作”的提示内容给用户；当用户点击这一提示内容，智能镜显示规范清洁动作的详情给用户，例如这里所示的“指腹按摩脸部”或图中未示出的“手部拍打脸部”等；并在用户点击详情后，返回显示“第一步：清洁”。又例如，如图8d所示，用户正在进行第一步：清洁，此时，智能镜检测到护肤动作不规范，通过显示屏提示类似“规范清洁动作”的提示内容给用户；当用户点击这一提示内容，智能镜通过显示屏演示规范清洁动作给用户；并在用户再次点击显示屏后，返回显示“第一步：清洁”。
S3032、根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度。
智能镜根据护肤动作及其对应的时长、预设时长，可以得到护肤过程的完整度；然后，输出护肤过程的完整度。
需明确，预设护肤时长是指整个护肤过程持续的时长，预设时长是指一护肤动作持续的时长，两个为不同的概念，其中，预设护肤时长大于或等于预设时长。
在上述基础上，进一步地，在图像中提取目标图像特征，可以包括：若确定在不同帧图像之间手部相对面部存在位移，则提取目标图像特征。也就是说，只有在确定不同帧图像之间手部相对面部存在位移时，才会提取目标图像特征，以排除手部一直放到脸上没有进行任何动作的情况。
一些实施例中，上述确定在不同帧图像之间手部相对面部存在位移，可以包括：提取图像中面部区域和手部区域的关键点信息；根据关键点信息，确定不同帧图像之间手部相对面部存在位移。其中，根据关键点信息，确定不同帧图像之间手部相对面部存在位移，包括以下至少一种：
一种情况，当不同帧图像之间手部关键点与面部关键点的相对位置距离改变，确定不同帧图像之间手部相对面部存在位移。示例地，关键点可以具体为面部关键点与手部中心点。手部相对面部存在位移也就是在不同帧图像中，手部中心点与面部关键点的相对位置距离发生改变。进一步地，不同帧图像中，手部中心点与面部关键点的相对位置距离的改变值大于预设改变值时，确定不同帧图像之间手部相对面部存在位移；或者，不同帧图像中，手部中心点与面部关键点的相对位置距离的改变值小于预设改变值时，确定不同帧图像之间手部相对面部不存在位移。仍参考图5b，图5b示出了两帧时间相邻的图像。其中，箭头左边图像中用户的嘴部是闭合的，箭头右边图像中用户的嘴部是微张的，该情况下，仅是嘴部形状在改变，其中面部关键点与手部中心点相对位置距离改变很小，可以认为这两帧图像之间手部相对面部不存在位移。
另一种情况，当不同帧图像之间手部关键点的位置信息改变，确定不同帧图像之间手部相对面部存在位移。
以上实施例主要介绍了如何确定护肤过程的完整度，接下来介绍如何对其进行输出。
一种具体实现中，输出护肤过程的完整度，可以包括：通过图表、图示和评分中的至少一种显示类型，输出护肤过程的完整度。示例地，图9a和图9b以图示的形式输出护肤过程的完整度；图9c以图表的形式输出护肤过程的完整度。
参考图9a，其中，不同的圆环代表不同的护肤动作，整个圆环完全明显呈现，表示对应的护肤动作是完整的、充分的；若圆环仅部分明显呈现，表示对应的护肤动作的持续时长没有达到预设时长，是不完整或不充分的；若圆环全部不明显，表示对应的护肤动作遗漏。其中，图9a是以护肤动作A和护肤动作B为例进行说明的，但本申请实施例不以此为限制。例如，还可以将一个圆环进一步分为至少两个部分，每一部分分别用于表示不同的护肤动作。
参考图9b，其中，不同的圆环代表不同功能的护肤动作，整个圆环完全明显呈现，表示对应功能的护肤动作是完整的、充分的；若圆环仅部分明显呈现，表示对应功能的护肤动作的持续时长没有达到预设时长，是不完整或不充分的；若圆环全部不明显，表示对应功能的护肤动作遗漏。示例地，图9b是以抗衰、清洁和美白三种功能的护肤动作为例进行说明的，但本申请实施例不以此为限制。其中，抗衰可以表示为涂抹眼霜的护肤动作过程，例如，在整个护肤过程中，用户并未使用眼霜以及使用眼霜过程时间较短，那么在抗衰这一项上可通过环形的长度来表现；清洁可以表示为涂抹水乳的护肤动作过程，当用户未使用水乳或者使用水乳进行护肤的时长较短，则在该环形的长度上有一定的体现，环形越长，则说明完整度越高；美白可以体现在做面膜等护肤动作及时长上。
参考图9c，其中，100表示对应的护肤动作是完整的、充分的；大于0，且小于100的数值表示对应的护肤动作的持续时长没有达到预设时长，是不完整或不充分的；0表示对应的护肤动作遗漏。示例地，图9c是以护肤动作A和护肤动作B为例进行说明的，其中，护肤动作A的完整度为80，护肤动作B的完整度为97，但本申请实施例不以此为限 制。
另外，通过评分的显示类型，输出护肤过程的完整度，示例如下：
护肤动作A：80分；
护肤动作B：100分。
其中，完整度对应的评分为0～100，其中，评分越高说明完整度越高。因此，在上述示例中，护肤动作A需进一步完善。
此外，还可以在用户指向或点击完整度时，显示对应区域的详情。以图9b为例，当用户点击“美白”对应的圆环时，智能镜通过显示屏显示美白详情(即智能镜响应于作用于圆环的第二操作，显示对应圆环的护肤详情)，如图10a所示：
美白详情：
时长：1分钟
动作：规范
完整度：80％，小于预设时长(1.5分钟)。
或者，当用户点击“抗衰”对应的圆环时，智能镜通过显示屏显示抗衰详情，如图10b所示：
抗衰详情：
时长：1分钟
动作：规范
完整度：100％
可选地，在完整度为100％时，显示表示“棒极了”的手势，例如图10c所示。
考虑到智能镜的显示屏限制，智能镜可以将完整度发送给与其进行通信的其他电子设备，例如图3所示的手机。示例地，参考图11a，用户点击或指向智能镜显示的图示形式的完整度，响应于这一动作，智能镜发送对应区域的详情给手机，这里以用户点击美白对应的圆环为例，发送美白详情给手机，由手机显示美白详情。或者，智能镜生成完整度的分析报告，在用户指向或点击标识有类似“分析报告”字样的虚拟按钮或区域时，智能镜发送分析报告给手机，由手机显示分析报告内容，如图11b所示。另需说明的是，分析报告可以是整个护肤过程对应的报告，也可以是其中某一护肤动作对应的分析报告，对此本申请不进行限制。
进一步地，在输出护肤过程的完整度之后，上述护肤辅助方法还可以包括：基于完整度，输出对用户进行护肤指导的策略。示例地，通过图片和/或视频的形式，输出对用户进行护肤指导的策略。
例如，当用户在清晨护肤过程中，忘记涂抹眼霜，这样在电子设备输出的护肤过程的完整度中指示该护肤操作的遗漏，并基于该护肤操作输出如何正确进行眼霜的涂抹，如图12所示，通过箭头或者手指等向上向下的图片指示或告诉用户应该在眼睛部位的上眼脸和下眼睑涂抹眼霜等，以及使用动画与用户进行互动指示用户正确和标准护肤动作。其中，虚线箭头用于表示护肤动作的承接。
或者，在分析报告或护肤动作对应的详情中，包含护肤指导的相关内容，示例地，如图11a和图11b所示。
该实施例通过输出护肤过程的完整度，可视化呈现对用户护肤过程的评价，并进一步 基于完整度给出护肤指导的策略，能够提醒用户在护肤过程中遇到的问题并提供护肤建议，进一步帮助用户促进肌肤对护肤产品的吸收。
另外，考虑到电子设备的功耗问题，参考图13，该护肤辅助方法可以包括以下步骤：
S701、响应于作用于第一电子设备的第一操作，第一电子设备通过摄像头获取图像。
示例性地，实际应用中，用户在智能镜前进行护肤时，一般通过摄像头获取用户的护肤视频，由于需要降低电子设备的功耗，需要实时从护肤视频中提取部分关键图像帧作为后续要处理的图像。或者，电子设备可以基于预设算法，获取预设时间间隔的图像帧为关键图像帧。可选地，提取关键图像帧，可以具体为：检测图像中是否包含手部区域与面部区域，若图像中包含手部区域与面部区域，则保留，作为关键图像帧；若图像中不包含手部区域或面部区域，则丢弃。
S702、对图像进行预处理。
其中，预处理可以包括但不限于裁剪、缩放和旋转等中的至少一种。由于智能镜通过摄像头获取的图像尺寸比较大，为了降低护肤辅助方法的运行时间以及功耗，需要对图像进行裁切或缩放或旋转等处理。
S703、提取图像中面部区域和手部区域的关键点信息。
S704、根据关键点信息，确定不同帧图像之间手部相对面部是否存在位移。
若不同帧图像之间手部相对面部存在位移，则执行S705；所述不同帧图像可以是在时间上相邻的两帧图像，也可以为了节约电子设备的功耗，分析预定时间间隔的不同帧图像，该预定时间间隔大于摄像头的视频的帧间隔。若不同帧图像之间手部相对面部不存在位移，则返回S701。
S705、在图像中提取目标图像特征。
S706、根据多帧图像对应的目标图像特征，确定多帧图像对应的护肤动作及每一护肤动作的持续时长。
S707、根据护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度。
可选地，输出护肤过程的完整度，可以包括：显示护肤过程的完整度。其中，图像是由智能镜上的摄像头采集得到的。显示护肤过程的完整度，包括：在智能镜的显示屏上，显示护肤过程的完整度，即智能镜作为护肤辅助方法的执行主体。
或者，输出护肤过程的完整度，可以包括：发送护肤过程的完整度给第二电子设备，由第二电子设备显示护肤过程的完整度。
以上提供两种输出方案，分别由智能镜或第二电子设备呈现护肤过程的完整度。
需要说明的是，上述任一实施例可以单独实施，也可以是上述各实施例中至少两个任意结合来实施，对此不做限定。
可以理解的是，上述各个实施例中，由第一电子设备实现的操作和步骤也可以由可用于第一电子设备的部件(例如芯片或者电路)实现，本申请实施例对此不作限定。
图14为本申请一实施例提供的护肤辅助装置的结构示意图。该护肤辅助装置可以是第一电子设备，也可以是第一电子设备的部件(例如，集成电路，芯片等等)，或者可以是其他通信模块，用于实现上述任一实施例中对应于第一电子设备的操作。如图14所示，本实施例的护肤辅助装置70包括：获取模块71、提取模块72和输出模块73。本实施例 的护肤辅助装置70通过获取模块71、提取模块72和输出模块73可以实现如上述任一实施例中的方案，其实现原理和技术效果类似，此处不再赘述。
图15为本申请一实施例提供的电子设备的结构示意图。如图15所示，本实施例所述的电子设备600可以是前述方法实施例中提到的第一电子设备(或者可用于第一电子设备的部件)。电子设备600可用于实现上述方法实施例中描述的对应于第一电子设备的方法，具体参见上述方法实施例中的说明。示例地，该电子设备为智能镜。
电子设备600包括：至少一个处理器601，至少一个网络接口604或者其他用户接口603，存储器605，至少一个通信总线602。通信总线602用于实现这些组件之间的连接通信。该电子设备600可选的包含用户接口603，包括显示器(例如，触摸屏等)。
存储器605可以包括只读存储器和随机存取存储器，并向处理器601提供指令和数据。存储器605的一部分还可以包括非易失性随机存取存储器(NVRAM)。
在一些实施方式中，存储器605存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集：
操作系统6051，包含各种系统程序，用于实现各种基础业务以及处理基于硬件的任务；
应用程序模块6052，包含各种应用程序，例如如前所述的美妆APP、媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。
在本申请实施例中，通过调用存储器605存储的程序或指令，处理器601用于执行如上任一实施例中所述的方法，本申请实施例在此不再赘述。
可见，采用上述方案后，可以帮助用户了解其在护肤过程中的护肤动作、护肤时间是否完整等，辅助用户护肤，进而为其提供指导。
图16为本申请另一实施例提供的电子设备的结构示意图。请参见图16，电子设备20包括处理器21，以及与处理器21连接的存储器22、通信模块23、摄像头24、显示屏25和音视频模块26。
处理器21可以包括一个或多个处理单元，例如：处理器21可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
在一些实施例中，处理器21中还可以设置存储器，用于存储指令和数据。例如，处理器21中的存储器为高速缓冲存储器。该存储器可以保存处理器21刚用过或循环使用的指令或数据。如果处理器21需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器21的等待时间，因而提高了系统的效率。
在一些实施例中，处理器21可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口， 用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。I2S接口可以用于音频通信。在一些实施例中，处理器21可以包含多组I2S总线。处理器21可以通过I2S总线与音视频模块26耦合，实现处理器21与音视频模块26之间的通信。
存储器22可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。存储器22可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备20使用过程中所创建的数据(比如音频数据，电话本等)等。此外，存储器22可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器21通过运行存储在存储器22的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备20的各种功能应用以及数据处理。
通信模块23可以提供应用在电子设备20上的包括2G/3G/4G/5G等无线通信的解决方案。通信模块23可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。通信模块23可以由天线接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器21进行解调。通信模块23还可以对经调制解调处理器21调制后的信号放大，经天线转为电磁波辐射出去。在一些实施例中，通信模块23的至少部分功能模块可以被设置于处理器21中。在一些实施例中，通信模块23的至少部分功能模块可以与处理器21的至少部分模块被设置在同一个器件中。例如，通信模块23可以接收本申请所示的视频，并将接收到的视频存储至存储器22。
摄像头24可以进行图像或者视频的拍摄。例如，摄像头24可以在处理器21的控制下拍摄本申请所示的视频，并将视频存储至存储器22。
显示屏25用于显示图像，视频等。显示屏25包括显示面板。显示面板可以采用液晶显示器(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备20可以包括1个或N个显示屏25，N为大于1的正整数。
音视频模块26可以播放存储器22中存储的视频，并通过显示屏25显示播放的视频。例如，音视频模块26用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音视频模块26还可以用于对音频信号编码和解码。在一些实施例中，音视频模块26可以设置于处理器21中，或将音视频模块26的部分功能模块设置于处理器21中。音视频模块26还可以用于对数字视频压缩或解压缩。电子设备20可以支持一种或多种视频编解码器。这样，电子设备20可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
本申请实施例还提供了一种芯片，所述芯片包括可编程逻辑电路和输入接口，所述输入接口用于获取待处理的数据，所述逻辑电路用于对待处理的数据执行上述任一实施例所述的方法，其实现原理以及有益效果类似，此处不再进行赘述。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由处理器执行，实现如上述任一实施例所述的方法，其实现原理以及有益效果类似，此处不再进行赘述。
本申请实施例还提供了一种计算机程序产品，当所述计算机程序产品在电子设备上运行时，使得所述电子设备执行上述任一实施例所述的方法，其实现原理以及有益效果类似，此处不再进行赘述。
实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一可读取存储器中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储器(存储介质)包括：只读存储器(read-only memory，ROM)、RAM、快闪存储器、硬盘、固态硬盘、磁带(magnetic tape)、软盘(floppy disk)、光盘(optical disc)及其任意组合。
本申请实施例是参照根据本申请实施例的方法、设备、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理单元以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理单元执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请实施例的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (25)

  1. 一种护肤辅助方法，应用于具有摄像头的第一电子设备，其特征在于，所述方法包括：
    响应于作用于所述第一电子设备的第一操作，所述第一电子设备通过摄像头获取图像，所述图像中包含手部区域与面部区域，所述第一操作用于指示护肤过程的开始；
    在所述图像中提取目标图像特征，所述目标图像特征为与所述手部区域和面部区域相关的特征；
    根据多帧所述图像对应的目标图像特征，输出护肤过程的完整度，所述多帧图像为护肤过程中按时间顺序生成的图像。
  2. 根据权利要求1所述的方法，其特征在于，所述完整度包括护肤动作是否标准、护肤动作是否遗漏和护肤时间是否充足中的至少一种。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据多帧所述图像对应的目标图像特征，输出护肤过程的完整度，包括：
    根据多帧所述图像对应的目标图像特征，确定多帧所述图像对应的护肤动作及每一护肤动作的持续时长；
    根据所述护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度。
  4. 根据权利要求3所述的方法，其特征在于，所述根据多帧所述图像对应的目标图像特征，确定多帧所述图像对应的护肤动作及每一护肤动作的持续时长，包括：
    根据多帧所述图像对应的目标图像特征和预先训练好的分类器，确定多帧所述图像对应的护肤动作；
    确定所述护肤动作的持续时长。
  5. 根据权利要求4所述的方法，其特征在于，所述确定所述护肤动作的持续时长之后，所述方法还包括：
    若所述护肤动作的持续时长大于预设时长，则提示用户进行下一护肤动作；
    或者，若所述护肤动作的持续时长小于预设时长，则提示用户继续进行所述护肤动作。
  6. 根据权利要求5所述的方法，其特征在于，所述提示对应的时长为预设值，所述提示持续的时长大于所述预设值后，所述提示自动消失。
  7. 根据权利要求4至6中任一项所述的方法，其特征在于，所述确定多帧所述图像对应的护肤动作之后，所述方法还包括：
    检测所述护肤动作是否规范；
    若所述护肤动作不规范，则提示所述护肤动作对应的规范动作。
  8. 根据权利要求4至7中任一项所述的方法，其特征在于，所述分类器包括以下分类器中的至少一种：
    支持向量机、随机森林、决策树、神经网络和集成学习。
  9. 根据权利要求3至8中任一项所述的方法，其特征在于，所述根据所述护肤动作及每一护肤动作的持续时长，输出护肤过程的完整度，包括：
    根据所述护肤动作及每一护肤动作的持续时长、预设时长，确定护肤过程的完整度；
    输出护肤过程的完整度。
  10. 根据权利要求3至9中任一项所述的方法，其特征在于，所述目标图像特征包括以下特征中的至少一种：
    手部关键点和面部关键点的欧氏距离；
    手部关键点和面部关键点的曼哈顿距离；
    手部关键点和面部关键点的相对位移；
    光流特征。
  11. 根据权利要求1至10中任一项所述的方法，其特征在于，所述在所述图像中提取目标图像特征，包括：
    若确定在不同帧图像之间手部相对面部存在位移，则在所述图像中提取目标图像特征。
  12. 根据权利要求11所述的方法，其特征在于，所述确定在不同帧图像之间手部相对面部存在位移，包括：
    提取所述图像中面部区域和手部区域的关键点信息；
    根据所述关键点信息，确定不同帧图像之间手部相对面部存在位移。
  13. 根据权利要求12所述的方法，其特征在于，所述根据所述关键点信息，确定不同帧图像之间手部相对面部存在位移，包括以下至少一种：
    当不同帧图像之间手部关键点与面部关键点的相对位置距离改变，确定不同帧图像之间手部相对面部存在位移；
    当不同帧图像之间手部关键点的位置信息改变，确定不同帧图像之间手部相对面部存在位移。
  14. 根据权利要求1至13中任一项所述的方法，其特征在于，所述输出护肤过程的完整度，包括：
    通过图表、图示和评分中的至少一种显示类型，显示护肤过程的完整度。
  15. 根据权利要求14所述的方法，其特征在于，所述图示包含不同护肤动作对应的圆环，其中圆环的完整程度用于指示对应护肤动作的完整度。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    响应于作用于所述圆环的第二操作，显示对应圆环的护肤详情。
  17. 根据权利要求1至16中任一项所述的方法，其特征在于，所述护肤过程的完整度包含于分析报告中，所述输出护肤过程的完整度，包括：
    显示虚拟按钮，所述虚拟按钮用于指示所述分析报告；
    响应于作用于所述虚拟按钮的第三操作，显示所述分析报告的内容。
  18. 根据权利要求1至17中任一项所述的方法，其特征在于，所述输出护肤过程的完整度，包括：
    发送所述护肤过程的完整度给第二电子设备，由所述第二电子设备显示所述完整度。
  19. 根据权利要求1至18中任一项所述的方法，其特征在于，所述根据多帧所述图像对应的目标图像特征，输出护肤过程的完整度之后，所述方法还包括：
    基于所述完整度，输出对用户进行护肤指导的策略。
  20. 根据权利要求19所述的方法，其特征在于，所述输出对用户进行护肤指导的策略，包括：
    通过图片和/或视频的形式，输出对用户进行护肤指导的策略。
  21. 根据权利要求1至20中任一项所述的方法，其特征在于，所述第一电子设备通过摄像头获取图像之后，所述方法还包括：
    对所述图像进行预处理，所述预处理包括裁剪、缩放和旋转中的至少一种；
    所述在所述图像中提取目标图像特征，包括：在预处理后的图像中提取目标图像特征。
  22. 根据权利要求1至21中任一项所述的方法，其特征在于，所述第一电子设备为智能镜。
  23. 一种电子设备，其特征在于，包括：存储器和处理器；其中，所述存储器上存储有可供所述处理器执行的计算机程序；
    当所述处理器读取并执行所述计算机程序时，使得所述处理器执行如权利要求1至22中任一项所述的方法。
  24. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由处理器执行，实现如权利要求1至22中任一项所述的方法。
  25. 一种程序产品，其特征在于，当所述程序产品在电子设备上运行时，使得所述电子设备执行如权利要求1至22中任一项所述的方法。
PCT/CN2021/109140 2020-08-24 2021-07-29 护肤辅助方法、设备及存储介质 WO2022042195A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/042,702 US20230334641A1 (en) 2020-08-24 2021-07-29 Skin Care Auxiliary Method, Device, and Storage Medium
EP21860035.1A EP4191468A4 (en) 2020-08-24 2021-07-29 SKIN CARE AID METHOD, DEVICE AND STORAGE MEDIUM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010857343.5A CN114187649A (zh) 2020-08-24 2020-08-24 护肤辅助方法、设备及存储介质
CN202010857343.5 2020-08-24

Publications (1)

Publication Number Publication Date
WO2022042195A1 true WO2022042195A1 (zh) 2022-03-03

Family

ID=80354517

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/109140 WO2022042195A1 (zh) 2020-08-24 2021-07-29 护肤辅助方法、设备及存储介质

Country Status (4)

Country Link
US (1) US20230334641A1 (zh)
EP (1) EP4191468A4 (zh)
CN (1) CN114187649A (zh)
WO (1) WO2022042195A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108154121A (zh) * 2017-12-25 2018-06-12 深圳市美丽控电子商务有限公司 基于智能镜的化妆辅助方法、智能镜及存储介质
CN108171143A (zh) * 2017-12-25 2018-06-15 深圳市美丽控电子商务有限公司 美妆方法、智能镜及存储介质
US20180260983A1 (en) * 2015-12-28 2018-09-13 Panasonic Intellectual Property Management Co., Lt Makeup simulation assistance apparatus, makeup simulation assistance method, and non-transitory computer-readable recording medium storing makeup simulation assistance program
EP3462284A1 (en) * 2017-06-12 2019-04-03 Midea Group Co., Ltd. Control method, controller, intelligent mirror and computer readable storage medium
CN111539882A (zh) * 2020-04-17 2020-08-14 华为技术有限公司 辅助化妆的交互方法、终端、计算机存储介质
CN111553220A (zh) * 2020-04-21 2020-08-18 海信集团有限公司 一种智能设备及数据处理方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160142742A (ko) * 2015-06-03 2016-12-13 삼성전자주식회사 메이크업 거울을 제공하는 디바이스 및 방법
US10885322B2 (en) * 2019-01-31 2021-01-05 Huawei Technologies Co., Ltd. Hand-over-face input sensing for interaction with a device having a built-in camera

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180260983A1 (en) * 2015-12-28 2018-09-13 Panasonic Intellectual Property Management Co., Lt Makeup simulation assistance apparatus, makeup simulation assistance method, and non-transitory computer-readable recording medium storing makeup simulation assistance program
EP3462284A1 (en) * 2017-06-12 2019-04-03 Midea Group Co., Ltd. Control method, controller, intelligent mirror and computer readable storage medium
CN108154121A (zh) * 2017-12-25 2018-06-12 深圳市美丽控电子商务有限公司 基于智能镜的化妆辅助方法、智能镜及存储介质
CN108171143A (zh) * 2017-12-25 2018-06-15 深圳市美丽控电子商务有限公司 美妆方法、智能镜及存储介质
CN111539882A (zh) * 2020-04-17 2020-08-14 华为技术有限公司 辅助化妆的交互方法、终端、计算机存储介质
CN111553220A (zh) * 2020-04-21 2020-08-18 海信集团有限公司 一种智能设备及数据处理方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4191468A4

Also Published As

Publication number Publication date
CN114187649A (zh) 2022-03-15
EP4191468A4 (en) 2023-12-20
EP4191468A1 (en) 2023-06-07
US20230334641A1 (en) 2023-10-19

Similar Documents

Publication Publication Date Title
CN114502061B (zh) 使用深度学习的基于图像的自动皮肤诊断
US20230333635A1 (en) Systems, methods, apparatuses and devices for detecting facial expression and for tracking movement and location in at least one of a virtual and augmented reality system
US20230215208A1 (en) Interaction Method for Electronic Device for Skin Detection, and Electronic Device
US20200245873A1 (en) Detecting respiratory tract infection based on changes in coughing sounds
CN109475294B (zh) 用于治疗精神障碍的移动和可穿戴视频捕捉和反馈平台
Al-Rahayfeh et al. Eye tracking and head movement detection: A state-of-art survey
US10485426B2 (en) Device having health detecting function, display apparatus, system and health detecting method
CN105787442B (zh) 一种基于视觉交互面向视障人群的穿戴式辅助系统及其使用方法
WO2016008354A1 (en) System and method for display enhancement
CN111898407B (zh) 一种基于人脸动作识别的人机交互操作系统
CN108161933A (zh) 交互模式选择方法、系统及接待机器人
Le et al. Eye blink detection for smart glasses
CN112016367A (zh) 一种情绪识别系统、方法及电子设备
CN108153169A (zh) 导览模式切换方法、系统和导览机器人
US20220084529A1 (en) Method and apparatus for awakening wearable device
US12094243B2 (en) Method and apparatus for discreet person identification on pocket-size offline mobile platform with augmented reality feedback with real-time training capability for usage by universal users
Zhang et al. Speechin: A smart necklace for silent speech recognition
US20240212388A1 (en) Wearable devices to determine facial outputs using acoustic sensing
Bharath Controlling mouse and virtual keyboard using eye-tracking by computer vision
CN110297720A (zh) 通知装置、通知方法及存储通知程序的介质
Chew et al. Low-cost eye gesture communication system for people with motor disabilities
WO2022042195A1 (zh) 护肤辅助方法、设备及存储介质
Vasanthan et al. Facial expression based computer cursor control system for assisting physically disabled person
ATASOY et al. Real-Time motorized electrical hospital bed control with eye-gaze tracking
KR20230154380A (ko) 행동 및 발화 패턴 기반 감성 인식 결과에 의해 사용자의 감성 상태에 적합한 헬스케어 서비스를 제공하는 시스템 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21860035

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021860035

Country of ref document: EP

Effective date: 20230228

NENP Non-entry into the national phase

Ref country code: DE