FR3115907A1 - Interface and analysis techniques for a personalized skin care regimen based on skin diagnosis - Google Patents

Abstract

Interface and Analysis Techniques for a Personalized Skin Care Regimen Based on a Skin Diagnosis A computer system generates a computer-assisted personal care program based at least in part on a digital model of a user's face. In one embodiment, a computer system generates the model based on one or more images or scans of a user's face, and generates a computer-assisted personal care program based on the model, environmental data, and user data. In one embodiment, a program generated based on the model includes multiple steps to be performed using a computer-controlled skin care device, and user prompts for performing the steps configured to be presented in a user interface. Abstract: FIGURE 1

Description

Interface and analysis techniques for a personalized skin care regimen based on skin diagnosis

SUMMARY

This summary is provided to present a selection of concepts in a simplified form described in more detail below in the Detailed Description. This summary is not intended to identify key features of claimed subject matter, nor is it intended to be used as an aid in determining the setting of claimed subject matter.

This disclosure relates to the generation of personalized personal care regimens based on skin diagnosis. In described embodiments, a computer system generates computer-assisted personal care programs for intelligent personal care devices based on digital models of the user's skin.

In one aspect, a computer system performs operations including obtaining a digital model of the face of a living human subject; obtaining user data associated with the living human subject; and generating a computer-assisted skin care program based at least in part on the digital model of the live human subject's face, the environmental data, and the user data. In one embodiment, the computer-based skin care program includes one or more operational parameters (eg, brushing speed or intensity, cleaning duration) for a computer-based skin care device. In another embodiment, the computer-assisted skincare program includes a product recommendation (eg, a recommended device accessory, skin cleanser, eye cream, sunscreen, make-up, etc.) presented by a user interface. In another embodiment, the computer-assisted skincare program includes a self-care recommendation (eg, hydration recommendation, UV protection recommendation, stress reduction recommendation, sleep recommendation, etc. .) presented by a user interface.

In one embodiment, the digital model is based on one or more digital scans of the face, and includes a plurality of facial features to be targeted in a skin care program, such as one or more blemishes, areas of hyperpigmentation, texture, or wrinkles, or a combination thereof.

In one embodiment, the computer system obtains environmental data (eg, temperature data, humidity data, pollution data, or UV radiation data, or a combination thereof) associated with the environment of the living human subject, and the computer-aided skin care program is further based on the environmental data. Environmental data is obtained from a remote computer system or from one or more environmental sensors, or a combination thereof. In some embodiments, the environmental sensors are embedded in a skin care device or in a smartphone or other computing device, in communication with a skin care device.

In some embodiments, a user interface includes one or more user interface elements for providing feedback on or modifying the computer-assisted skin care program, or other data on which the program is based, such as user preferences (eg, target age, device preferences). In such an embodiment, operations performed by the computer system further include obtaining user feedback (e.g., rating or level of satisfaction, adjusted age target, or skin care goal , etc.) relating to the user interface computer-assisted skin care program, and/or modification of the computer-assisted skin care program.

Computer-implemented methods, computer-based data, computer devices, and computer systems are disclosed.

The foregoing aspects and many of the advantages of the present invention flowing therefrom will be better understood by reference to the following detailed description, made in conjunction with the accompanying drawings, in which:

There is a schematic diagram that provides a high level overview of a system that includes a skin care device according to aspects of the present disclosure;

There is a block diagram that illustrates an exemplary embodiment of a client computing device according to aspects of the present disclosure;

FIGURE 3 is a block diagram illustrating an exemplary embodiment of a skin care device according to aspects of the present disclosure;

There is a flow chart that illustrates an exemplary embodiment of a method for generating a computer-assisted skin care program based at least in part on a digital model of the living human subject's face according to aspects of the this disclosure;

There is a screen shot diagram of a screen illustrating a skin care program selection screen, according to a disclosed embodiment;

There is a screen capture diagram illustrating a face scan analysis screen, according to one embodiment described;

There is another screenshot diagram illustrating a face scan analysis screen, according to the same embodiment described as the ;

There is a screen capture diagram illustrating a questionnaire screen for obtaining user data, according to a described embodiment;

There is a screenshot diagram illustrating a questionnaire screen for obtaining user data, according to the same embodiment described as the ;

There is a screen capture diagram illustrating a questionnaire screen for obtaining user data, according to the same embodiment described as FIGS. 7 and 8;

There is a screenshot diagram illustrating a results screen in which a skin care plan was generated based on a face scan and other inputs, according to a disclosed embodiment;

There is a screen shot diagram illustrating a skin care program tutorial screen, according to a disclosed embodiment;

There is a screenshot diagram illustrating a user preferences screen, according to a described embodiment;

There is a screen shot diagram illustrating a skincare program edit screen, according to a disclosed embodiment;

There is a screen shot diagram illustrating a skin care program editing screen, according to the same disclosed embodiment as the ; And

There is a block diagram that illustrates aspects of an example computing device suitable for use with embodiments of the present disclosure.

DETAILED DESCRIPTION

Although a wide variety of personal care products and personal care devices are available today, discovering the best way to use these products or devices effectively often relies on trial and error. Although consumers may use Internet search engines to find relevant information about the capabilities of these devices, they are often unable to configure or use these devices optimally based on this information. In addition, as these devices become more complex and include Internet-connected and more programmable functions, the act of configuring and using these devices also becomes more complex, especially for ordinary consumers. This disempowers consumers to benefit from all the ways to configure and use their devices.

Embodiments described herein provide technical solutions to one or more of the technical problems described above, or other technical problems.

In embodiments disclosed herein, a computer system generates a computer-assisted skin care program based at least in part on a digital model of a user's face. In one embodiment, a computer system obtains a digital scan of a user's face along with other inputs, such as environmental data associated with the user's environment, and user data, associated with the user. The system uses this data to generate a computer-assisted skin care program. In one embodiment, the computer system includes one or more computer servers that obtains the digital scan of a client device, such as a smartphone with integrated digital camera, and generates a digital model of the client's face. digital scanning based user.

In one embodiment, the computer-assisted skin care program includes user prompts, recommendations, or other outputs presented to a user via video, audio, or other output by a client computing device or by a human-machine interface integrated in a skin care device. In one embodiment, the generated program includes steps for using the skin care device tailored to the user's face, the user's environment, and other user information. such as user age and preferences, and skin care goals. The generated program can be a new program or a modified version of a previously existing program. For example, if a facial scan reveals a new defect that was not present when an existing program was developed, the generated program can be adapted to provide an additional step to address the defect. As another example, an existing care program may be adjusted to take into account new environmental data (e.g. reduced humidity, increased temperature, increased pollution, etc.). The generated program can be based on criteria such as characteristics of a skincare device (eg available settings or accessories), previous results, user preferences or behaviors, time of day, month/season current year, day of the week, or similar.

A wide variety of operational parameters for skin care devices can be uniquely defined in assisted or scheduled care programs. For example, adjustable parameters for skin brushes or electric massagers may include adjustable power settings, timers, speed settings (e.g., speed of rotation or oscillation of a brush head ), haptic feedback parameters, or other parameters or combinations of parameters. These parameters can be automatically set and/or adjusted over time in a computer-assisted skin care program.

In one embodiment, the smart skin care device provides cues or prompts to users during a care program, such as audio cues, haptic cues, or visual cues, which can be used to guide a user through different phases of the care program, such as moving a skin care device to different skin areas, changing speeds, changing cleaning modes, and the like. In these embodiments, the smart skin care device may include speakers, displays, lights, or haptic electronics, such as touch sensors or vibrating actuators. The assisted care program functionality of the smart skincare device can be activated in various ways, such as pressing a physical button (e.g. an on/off switch or a dedicated button to activate a care program ), by voice command or in any other way.

In one embodiment, an intelligent skin care device further includes wireless communication circuitry, such as a wireless transceiver for Wi-Fi communication. In this embodiment, the An intelligent skin care device can access a voice interface service over a wireless communication circuit from a remote computer system. In another embodiment, the smart personal skin care device further includes a Bluetooth or near field communication (NFC) circuit, either in place of a Wi-Fi communication circuit or in more of it. In this embodiment, the smart skin care device can communicate with an external client computing device, such as a smartphone or tablet computer, which can, in turn, access a voice interface service by its communication circuit from a remote computer system. In another embodiment, the smart skin care device includes circuitry for communicating with a smart speaker, such as an Amazon Echo or Google Home device, which may include its own voice interface service or communicate with a remote computer system to access voice interface services.

In one embodiment, user data may be obtained from user profiles, which may be used to store system preferences, social media account or contact information, product/service provider preferences or ratings, location information, responses to questionnaires, information from connected devices (e.g., smart personal care devices, such as hair dryers, electric skin brushes, or the like), or other information.

In one embodiment, an intelligent skin care device includes sensors to detect characteristics of the physical environment (eg, humidity, temperature, ambient light levels, etc.) of the device during a skin care routine . Alternatively, the system may obtain data from remote sensors which may not be included in the skin care device itself. The information generated by these sensors can be used by the skin care device (or computer systems that may be in communication with the device) so that it can guide the user through the program or make adjustments for better results. .

Information obtained from devices may be used to guide personalized product or care plan selections, or for other purposes. This information may include device identification information, configuration information, usage information (e.g., number of times a device is used, duration of use, time of day, etc.), on sensors (e.g. measurements of environmental conditions, such as temperature, humidity, pollution levels, UV radiation levels, etc., or skin types, such as pH levels), position information, combinations of such information , or other information. Sensor information can be provided by Internet of Things (IoT) devices connected or already configured to receive sensors. These devices may include weather stations, mobile computing devices, and the like.

FIGURE 1 is a block diagram illustrating a system in which various aspects of the present disclosure may be implemented. As shown on the , system 100 includes a skin care device 102 (in the illustrated example, a skin brush or electric massager) with a wireless communication circuit. The skin care device 102 may send and/or receive information (e.g., usage data, device identification/configuration data, environmental data, or the like) to and/or from a system remote computing device 110, either directly or through one or more intermediate devices, such as a client computing device 104, as described below.

In the illustrated arrangement shown in the , a client computing device 104 connects to the remote computing system 110, which can generate personalized content for a user, such as computer-assisted skin care routines, product recommendations, or settings or parameters to operate a personal care device. In examples described herein, the personal care device is a skin care device 102, as described in more detail below. In one embodiment, a personalized care program for a skin cleansing device includes a defined pattern of skin area to be cleaned during the program, durations for each skin area, and brush head speeds or cleaning modes. cleaning for each skin area. In one embodiment, custom programs or device settings may be downloaded to client computing device 104 for later transmission to skin care device 102. Skin care devices 102 may also include or communicate with environmental sensors and/or other computing devices. Illustrated components of remote computer systems 110 and skin care device 102 are described below.

Client computing device 104 may be used by a consumer, skin care professional, or other entities to interact with other components of system 100, such as remote computing systems 110 or skin care device 102 In one embodiment, client computing device 104 is a mobile computing device such as a smartphone or tablet computer. However, any other suitable type of computing device capable of communicating over the network and having a user interface may be used, including but not limited to a desktop computer, laptop computer, smart speaker, or watch. smart device (or combinations of these devices).

Illustrated components and functionality of remote computer system 110 will now be described. The remote computing system includes one or more computing servers that implement the illustrated components, for example, in a cloud computing device. As illustrated on the , computer system 110 includes care plan engine 112, product data store 120, and user profile data store 122. Care plan engine 112 generates assisted care plan information, which can then be transmitted, for example, to the client computing device 104 and/or the skin care device 102. The assisted care plan information may include, for example, care plan instructions for programming or configuring the skincare 102 in a special way, product recommendations, tutorials, or other information.

In one embodiment, care plan engine 112 generates assisted care plan information, based on receipt of information from product data store 120 and user information from user profile data store 122 , the skin care device, the client computing device 104, or a combination thereof or any other source or combination of sources. Treatment program engine 112 may, for example, receive a request for an updated treatment program from skin care device 102 or client computing device 104, obtain information from product data store 120 (for settings and configurations available for skin care device 102, available accessories, etc.), user profile data store 122 (e.g., user responses to questions they have, data device usage, preferred care programs, location, age, products used, etc., or client computing device 104 (e.g., information describing the user's current location, satisfaction with previous programs (shown, e.g., by means of a numerical rating system or a number of stars), current stress level and mood, etc.) and use this information to carry out further treatment. care program engine 112 can use the information obtained, for example, to generate a new program or update a previously defined program.

Care plan engine 112 may employ artificial intelligence or machine learning techniques (e.g., model matching, feature matching and extraction, classification, artificial neural networks, deep learning architecture, genetic algorithms , or similar). In one embodiment, machine learning techniques are used to analyze data, such as a digital scan of a user's face, environmental data, and user data, and create or revise treatment plans, recommendations of products or other content based on this data. For example, to generate a personalized skincare plan, skincare plan engine 112 may analyze digital scans to measure or map wrinkles, pigmentation, skin texture, etc., of the user's skin. In such a scenario, the care plan engine 112 can use this information to recommend, generate, or modify a particular care plan that suits the particular characteristics of the user's skin.

In one embodiment, location information obtained by a skin care device or client computing device may be used to search and obtain other information from other devices or computing systems that may be relevant to a treatment program. For example, a client computing device 104 may provide location information to a remote computing system 110, which may, in turn, obtain current environmental data (e.g., weather information, pollution information, UV radiation information, etc. .) for the respective position. The remote computer system 110 can then use the environmental data to generate or modify a care plan. In an illustrated scenario, the remote computer system 110 uses the location information to determine if the user is in a hot, polluted city, and generates or modifies a care plan to account for that environment.

In one embodiment, information is stored, transmitted, and shared in system 100 securely and reliably. For example, secure connections are established between client computing devices and skin care devices, between client computing devices and server systems, or any other set of communication devices. Users can be securely reminded of care program events, environmental alerts (e.g., temperature, humidity, pollution, or UV radiation warnings), product expirations, and the like through instant notifications to a client application, email, instant messaging, or any other communication channel.

There is a block diagram illustrating an exemplary embodiment of a client computing device 104 according to various aspects of the present disclosure. There represents a non-limiting example of characteristics and configurations of client computing devices; many other features and configurations are possible within the scope of this disclosure.

In the example shown on the , client computing device 104 includes camera 250 and client application 260. Client application 260 includes user interface 276, which may include interactive functionality, such as data collection or questionnaire elements, tools for or edit user preferences, skincare editing tools, tutorials, virtual “try on” functionality to virtually try on different products or cosmetics, or other items. Visual elements of user interface 276 are presented on a display 240, such as a touch screen display. Personalized content, such as computer-assisted skin care programs, may be obtained by client computing device 104 (eg, from remote computing system 110) and presented by user interface 276. user interface illustration are described below, with reference to FIGURES 5-14.

In one embodiment, client application 260 also includes an image capture/scan module 270, which is configured to capture and process digital images (e.g., color images, depth images, etc.) or scans on which the digital models described herein are based. In one embodiment, the digital images or scans are transmitted to the remote computer system 110 or other external computer system where the digital skin models are generated. According to another solution, the digital models are generated in the client computing device 104, or in any other place. In one embodiment, the digital models include 3D topology, and texture information that can be used to reproduce an accurate representation of the user's facial structure and general appearance, as well as to perform skin diagnostics (e.g. to detect blemishes, areas of hyperpigmentation, visible pores, etc.). In one embodiment, user interface 276 includes interface elements to help accurately capture the digital images or scans upon which these digital skin models are based.

In one embodiment, a communication module 278 of client application 260 is used to prepare information for transmission to other devices or systems, such as remote computer system 110 or skin care device 102, or to receive and interpret information from it. This information may include captured digital images, scans, or videos, skin care device settings, personalized care programs, user preferences, user IDs, device IDs, or the like.

Other characteristics of client computing devices are not shown on the for ease of illustration. An illustrative description of computing devices is provided below, with reference to the .

FIGURE 3 is a block diagram illustrating components included in an exemplary embodiment of a skin care device, according to various aspects of the present disclosure. There shows a non-limiting example of features and configurations of a skin care device; many other features and configurations are possible within the scope of this disclosure.

In the example shown on the , the skin care device 102 includes a treatment application unit 302 configured to apply treatment to a user, a power source 304, a human-machine interface device 306, a processor 310, an interface network 312, and computer media 314. A non-limiting example of a skin care device 102 is an electric facial cleansing brush, with a brush head that oscillates, rotates, or otherwise moves, so as to perform a cleaning action. Other non-limiting examples of skin care devices 102 include a massage device or a phototherapy device that can be used to expose the skin to light (e.g., at a particular wavelength and/or for a particular length of time) to achieve therapeutic benefits.

In one embodiment, the processing application unit 302 includes one or more devices that collectively apply processing to a user. For example, if skin care device 102 is an electric skin brush, treatment application unit 302 may include a drive motor, an armature coupled to the drive motor that accepts a removable brush, and the brush head itself. As another example, if skin care device 102 is a portable phototherapy device, treatment application unit 302 may include one or more light-emitting diodes (LEDs) or other light-emitting devices that emit light. light at wavelengths and intensity suitable for phototherapy applications.

In one embodiment, the power source 304 is a rechargeable battery which powers the processing application unit 302 for its operation. Power source 304 may also provide power to operate the other components of skin care device 102. In other embodiments, instead of a battery, the skin care device skin 102 can be coupled to an external power source, such as an electrical outlet.

The human-machine interface (HMI) 306 can include any type of device capable of receiving user input or generating output for presentation to a user. In one embodiment, HMI 306 includes a speaker 308 to allow care device 102 to present audio content (e.g., care program signals in the form of speech, tones, etc., recorded or synthesized), and/or a microphone receives audio input, such as voice commands from the user. In addition to voice signals, speaker 308 may present another output to accompany the care program, such as music, sounds of a relaxing nature, or the like. Other non-limiting examples of possible 306 HMI components include a push button switch, rocker switch, capacitive switch, rotary switch, slide switch, and touch screen.

Processor 310 is configured to execute computer-executable instructions stored on computer medium 314. In one embodiment, processor 310 is configured to receive and transmit signals to and/or from other components of the skin care 102 via a communication bus or other circuit. Network interface 312 is configured to transmit and receive signals to and from client computing device 104 (or other computing devices) on behalf of the processor. Network interface 312 may implement any suitable communication technology, including, but not limited to, short-range wireless technologies, such as Bluetooth, infrared, near-field communication, and Wi-Fi. ; long-range wireless technologies, such as WiMAX, 2G, 3G, 4G, LTE, and 5G; and wired technologies, such as USB, FireWire, and Ethernet. Computer medium 314 is any type of computer medium on which computer-executable instructions can be stored, including, but not limited to, flash memory, ROM, EPROM, EEPROM, and FPGA. The computing medium 314 and the processor 310 may be combined in a single device, such as an ASIC, or the computing medium 314 may include a cache, register, or other component of the processor 310.

In the illustrated embodiment, computer medium 314 stores computer-executable instructions which, in response to execution by one or more processors 310, cause skin care device 102 to implement an assist engine. treatment program 316 and a treatment control engine 318. Treatment control engine 318 controls one or more aspects of skin care device 102 in a treatment program, as described above. In one embodiment, the care plan is generated and/or modified by the care plan engine 112, as described above. In one embodiment, treatment control engine 318 adjusts settings or configurations for skin care device 102, which may be generated and/or modified by treatment program engine 312, as described below. above. In one embodiment, the treatment control engine 318 controls basic functions, such as turning the treatment application unit on or off, brush head speed control , or the like. In one embodiment, treatment control engine 318 detects input from HMI 306, and activates treatment application unit 302 or modifies a function of skin care device 102 in response to the input. 'entrance. Treatment control engine 318 may then detect subsequent input from HMI 306 and disable treatment application unit 302 or make other adjustments to the function of skin care device 102 in response, or allow processing application unit 302 to run for a predetermined amount of time before automatically disabling processing application unit 302.

In one embodiment, care plan assistance engine 316 controls the presentation of care plan cues, such as causing speaker 308 to produce audio cues, so as to guide the actions of a user during a care program. In one embodiment, care plan assistance engine 316 collects data, such as data describing care plan execution, environmental data, or other data. In one embodiment, data collected by care plan assistance engine 316 is transmitted over network interface 312 to remote computing system 110 (e.g., by client computing device 104 or directly) for storage (by example, in product data store 120 or profile data store 122) or for further processing (e.g., to update a computer-assisted care program, adjust skin care device settings, etc.). In one embodiment, the collected data is used along with the facial scan data to make these updates or adjustments.

In one embodiment, processor(s) 310 and computing medium 314 provide circuitry that may be collectively referred to as a skin care unit of skin care device 102. In one embodiment , the skin care unit is operatively coupled with the HMI 306 and the treatment application unit 302, and includes circuitry for controlling the treatment application unit 302. In one embodiment , the skin care unit includes circuitry for presenting voice cues, visual cues, or other cues by the HMI in a computer-assisted care program to guide the operator in the use of a skin care device. Alternatively, the skin care device 102 may include a different circuit, or the circuit may be implemented in some other way.

The devices shown in FIGURES 1-3 or other devices used in depicted embodiments may communicate with each other through a network (not shown), which may include any suitable communication technology, including, but without limitation, wired technologies such as DSL, Ethernet, fiber optic, USB, and Firewire; wireless technologies such as WiFi, WiMAX, 3G, 4G, LTE, 5G, and Bluetooth; and the Internet. In general, the communication between the components of the systems on the or other computing devices may occur directly or through intermediate devices.

Many other arrangements disclosed and described with reference to FIGURES 1-3 are possible. For example, functionality described as being implemented in multiple components may instead be combined into a single component, or functionality described as being implemented in a single component may be implemented in multiple illustrated components, or in other components not shown in FIGURES 1-3. As another example, devices in FIGURES 1-3 shown as including particular components may instead include more components, fewer components, or different components without departing from the scope of described embodiments.

In system components shown in the or devices shown in FIGURES 2 and 3, or by components of these systems and devices operating in combination, several technical advantages are obtained. For example, the ability to automatically generate or modify treatment plans based on digital scans of a user's face in combination with additional data, such as user data and environmental data, overcomes the technical limitations of previous technologies that depended the user's ability to configure his own devices. As another example, system 100 allows certain aspects of the process to be performed independently by skin care devices or client computing devices, while moving other processing is the responsibility of remote computing system 110 (which may relatively reliable and powerful computing system), thereby improving performance and preserving battery life to provide the functionality provided by skin care devices or client computing devices.

In general, the word "engine", as used herein, refers to logic embedded in hardware or software instructions written in a programming language, such as C, C++, COBOL, JAVA™, PHP, Perl, HTML , CSS, JavaScript, VBScript, ASPX, Microsoft .NET™, and/or similar. An engine can be compiled into executable programs or written in interpreted programming languages. Software engines can be called by other engines or by themselves. Generally, the motors described here relate to logic modules which can be merged with other motors or divided into sub-motors. Engines can be stored in any type of computer media or computer storage device, and be stored on and run by one or more general-purpose computers, creating a specialized computer configured to provide the engine or functionality thereof.

As can be understood by those skilled in the art, a "data store", as described herein, can be any suitable device configured to store data accessible by a computing device. An example of a data store is a fast and reliable database management system (DBMS) that runs on one or more computing devices, and is accessible over a high-speed network. Another example of a data store is a key value store. However, any other suitable storage technique and/or device capable of quickly and reliably providing the stored data, in response to requests, may be used, and the computing device may be accessed locally instead of be on a network, or may be provided as a "cloud" service. A data store may also include data stored in an organized fashion on computer media, as described below. Those skilled in the art will recognize that separate datastores described herein may be combined into a single datastore, and/or a single datastore described herein may be separated into multiple datastores, without departing from the scope of this disclosure.

There is a flowchart illustrating an exemplary embodiment of a method of generating a computer-assisted skin care program based at least in part on a digital model of the living human subject's face. As illustrated, method 400 is implemented by a computer system. Method 400 may be implemented by a server computer system including features of remote computer system 110, by a client computer device, by skin care device 102, or by any other computer device or system.

From a start block, the method 400 proceeds to block 402, where the computer system obtains a digital model (e.g., based on one or more images or digital scans) of a living human subject. In one embodiment, the computer system includes a server computer that obtains one or more digital images or scans from a client device, such as a smartphone with an integrated digital camera. In such an embodiment, these images or scans are captured by the client device and uploaded to the server computer, which generates the digital model. In one embodiment, the digital model includes a highly accurate model of facial features, including eye and lip edges, iris size and position, skin features including pimples, texture, and wrinkles, and the like. It is understood that the skin characteristics described herein are exemplary only, and that other characteristics or combinations of these characteristics are also desirable and are within the scope of this disclosure. In one embodiment, source images are captured and digital models are generated using Modiface software available from Modiface, Inc.

Method 400 proceeds to an optional step 404, in which the computer system obtains environmental data associated with the environment of the living human subject. In one embodiment, the computing system obtains environmental data from environmental sensors at the user's location (eg, by sensors embedded in a client computing device or a skin care device used by the user) . Alternatively, the computer system obtains environmental data from another device or computer system. In one embodiment, the computer system determines the approximate location of the user (eg, as reported by the user or a location-providing computing device) and requests environmental data relating to that location from a computer system intended to provide meteorological or environmental data.

The method proceeds to step 406, in which the computer system obtains user data associated with the living subject. In one embodiment, user data may be obtained from user profiles stored in a user profile data store 122. Alternatively, user data may be obtained directly from a client computing device 104, a skin care device 102, or from any other source. In one embodiment, User Data includes system or device preferences, device usage information, contact or social media account information, product/service provider preferences or ratings, position, responses to questionnaires, or a combination thereof, or other information. In one embodiment, the user data includes an age target, which a user can set to adjust a skincare regimen to a particular target age. For example, a 50-year-old user can select an age target of 45, which causes a moderate skincare plan to be generated, to achieve a moderate difference in appearance. Alternatively, the 50-year-old user selects an age target of 40, which results in a more intensive skin care regimen being generated.

The method proceeds to step 408, in which the computer system generates a computer-assisted skin care program based at least in part on the digital model of the living human subject's face, environmental data, and user data. . In one embodiment, the skin care program engine 112 of the remote computer system 110 obtains these inputs and generates the computer-assisted skin care program. Alternatively, the program is generated based on different combinations of inputs. For example, if environmental data is not obtained or not available, the computer system generates a computer-assisted skin care program based on the digital model of the face and the user data.

In one embodiment, certain aspects of the computer-assisted skin care program, such as user prompts, are configured to be presented in a user interface. In one embodiment, the computer-assisted skin care program automatically includes activation of one or more voice cues or other prompts stored in the memory of the skin care device. According to another solution, the program is presented by a display or any other modality. The computer-assisted skin care program may be controlled, for example, by the care program assist engine 316 of the skin care device 102.

In one embodiment, a user interface (UI) includes one or more user interface elements for providing or modifying user data, capturing images or facial scans, or otherwise obtaining or modifying data on which programs computer-assisted skin care program may be based on, and/or to provide feedback on or modify a previously generated computer-assisted skin care program. In such an embodiment, the operations performed by the computer system further include obtaining user feedback (for example, a satisfaction rating, an adjusted age target, or a care goal for skin, etc.) relating to the UI computer-assisted skin care program and/or modification of the computer-assisted skin care program.

Aspects of an UI illustration for collecting information and providing user feedback and control for generated computer-assisted skin care programs will now be described with reference to FIGURES 5. -14. The UI shown with reference to FIGURES 5-14 is exemplary only, and it is understood that many other UI features, as well as other care program generation techniques for computer-aided skin can be developed according to the present disclosure.

There shows a screenshot of an illustration of the skin care program selection screen 500, according to a disclosed embodiment. Screen 500 includes options to select a predefined care plan or, if the user desires, to generate a custom care plan. Screen 500 includes an illustration of UI elements in the form of a button 502 for initiating a process for capturing one or more face images or scans, a button 504 for initiating a process for obtaining environmental data, and a button 506 for obtaining user data, in the form of a questionnaire or a “skin quiz”. In one embodiment, after activating button 502, a user is directed to take a self-portrait or a series of such images, which is analyzed by face scanning software to generate a digital model. of the user's face that includes skin features. In one embodiment, the features include the edges of the eyes and lips, skin features including blemishes, areas of hyperpigmentation, texture, or wrinkles, or other features or combinations thereof , these. The home screen includes an additional button 508 which can be used to add a predefined skin care program.

FIGURES 6A and 6B show screenshots of an illustration of face scan analysis screens that may be presented to a user after generation of a face scan, according to one disclosed embodiment. As mentioned above, the model of the user's face can include detected facial features, such as lip and eye rims, as well as skin features, such as wrinkles, texture, and blemishes. On the , screen 600 includes a self-portrait image of the user with graphic overlays depicting areas of interest or concern on a user's face, including detected areas of wrinkles, sag, and visible pores. These overlays are developed based on the results of the face scan. The example shown on screen 600 also includes skin ratings, including an overall condition rating for the user's face, as well as sub-ratings relating to visible pores, wrinkles, sagging, and pigmentation, and labeled graphic overlays corresponding to detected problem areas. The screen includes a scrollable tab 610 which allows a user to scroll the screen to view personal skin care recommendations. On the , screen 602 shows tab 610 in the unrolled position, revealing additional feedback relating to a skin condition. Additionally, the region in which this problem is most prominent is displayed in graphic overlay 612, which is generated based on the facial scan.

FIGURES 7-9 show screen shots of an illustration of questionnaire screens 700, 800, and 900, according to one described embodiment. The screens shown include UI elements that are used to collect user data on topics such as skin type, skin care goals, and facial areas, which may be of particular concern to the user. Other UI elements can be used to collect information on similar or different topics.

There shows a screenshot of an illustration of a results screen in which a skin care plan has been generated based on a face scan and other inputs, according to a disclosed embodiment. On results screen 1000, the user is informed by UI element 1010 that a schedule has been generated based on the face scan, environmental data, and user data. For example, the program may include steps to reduce visible pores identified in the facial scan, and use an appropriate cleanser for hot, humid weather identified in the environmental data, and a longer cleanse time for areas of distress. attention identified by the user. The user has the choice, thanks to the button 1020, to add this new program to his skin care device. In one embodiment, activation of button 1020 causes program data and instructions to execute the new program to be transmitted by client computing device 104 to skin care device 102 via Bluetooth. Alternatively, the data and program instructions corresponding to the generated skin care program may be transmitted, stored, or otherwise implemented.

There shows a screenshot of an illustration of a skin care program tutorial screen 1100, according to a disclosed embodiment. On the tutorial screen 1100, the UI provides graphic animations which are superimposed on a self-portrait image and which show, for example, usage pathways for cleaning steps, such as the cleaning step forehead depicted on screen 1100. In one embodiment, proper placement of the graphic animation relative to the self-portrait image is guided by the template previously generated based on the facial scan.

There shows a screenshot of an illustration of a user preference screen 1200, according to a described embodiment. Screen 1200 includes an example of how the UI can be designed to not only set user preferences, such as reminders to follow a schedule, but also provide opportunities to update generated schedules. previously with new face scans or other data. The 1200 display includes a reminder for a "selfie check every 7 days". This can be used to remind a user to save a new face scan every 7 days (or any other desired interval) and then update a previously generated schedule if needed. In addition to the features shown in the , other user preferences (e.g. device settings, sound or voice settings, haptic feedback settings, visual UI settings, etc.) can also be set using similar or different UI elements .

In one embodiment, the computer-assisted care plan includes a plurality of steps, and the step of generating (eg, creating or updating) the computer-assisted care plan includes selecting from available options for one or more steps based on a combination of data including a digital scan of the user's face and other user data (e.g., user preferences, device usage patterns, questionnaire/survey responses , etc.), as well as environmental data. In one embodiment, these selections are further based on selection criteria, such as skin care device characteristics (e.g., available brush heads, available brush head speeds), previous results, time of day, day of the week, day of the year, or other selection criteria, or a combination thereof.

There shows a screen shot of an illustration of an edit skincare program screen 1300, according to one disclosed embodiment. Screen 1300 includes other examples of how the UI can be designed to update previously generated programs. Screen 1300 includes UI elements showing current steps in a program (neck and chin cleaning steps), and a button element 1310 configured to allow a user to add a step to the program, as well as a button element 1320 configured to allow a user to view and/or edit a serum or cream for use in the program. In one embodiment, pressing button 1310 brings up another program edit screen 1400, as shown in . In the screen illustration 1400, the UI includes sliders 1410, 1420, which allow the user to select parameters (e.g., intensity and duration) to add a step to a program, as well as a button element 1430, which allows a user to select a desired position (eg, chin, cheek, etc.) to add the step.

In one embodiment, the parameters to be added to the steps depend on previously collected data. In one scenario illustration, a user has previously indicated that they have sensitive skin, and scanning the user's face indicates a problem area that may benefit from a longer treatment time. In the context of the , this scenario involves a default value for the new step set to a minimum intensity level ("1"), and a maximum value ("3") of slider 1420 is reduced to avoid intensity levels that are not not suitable for sensitive skin. Also, slider 1410 is set to a default duration (eg, 20 sec.) that is appropriate for an identified problem area in the face scan.

FIGURE 15 is a block diagram illustrating aspects of an example computing device 1500 suitable for use with embodiments of the present disclosure. While FIGURE 15 is described with reference to a computing device implemented as a device on a network, the description below is applicable to servers, personal computers, mobile phones, smartphones, tablets, computing devices devices, and other devices, which may be used to implement portions of embodiments of this disclosure. Additionally, those skilled in the art and others will recognize that computing device 1500 can be any of any number of devices currently available or yet to be developed.

In its most basic configuration, computing device 1500 includes at least one processor 1502 and system memory 1504 connected by a communication bus 1506. Depending on the exact configuration and type of device, system memory 1504 may be volatile or nonvolatile, such as read only memory (“ROM”), random access memory (“RAM”), EEPROM, flash memory, or similar memory technology. Those skilled in the art and others will recognize that system memory 1504 generally stores data and/or program modules that are immediately accessible to processor 1502 and/or currently operated on it. In this regard, processor 1502 can serve as the computing center of computing device 1500 by supporting the execution of instructions.

As further illustrated in FIGURE 15, computing device 1500 may include a network interface 1510 comprising one or more components for communicating with other devices on a network. Embodiments of the present disclosure can access basic services that use network interface 1510 to perform communications using common network protocols. The 1510 network interface may also include a wireless network interface configured to communicate by one or more wireless communication protocols, such as WiFi, 2G, 3G, LTE, 5G, WiMAX, Bluetooth, and or the like.

In the exemplary embodiment depicted in FIGURE 15, computing device 1500 also includes storage media 1508. However, services may be accessed using a computing device that does not include means for storing data on a local storage medium. Therefore, storage medium 1508 shown in FIGURE 15 is shown with a dashed line to indicate that storage medium 1508 is optional. In either case, storage medium 1508 may be volatile or non-volatile, removable or non-removable, implemented using any technology capable of storing information, such as, but not limited to, hard drive, solid state drive, CD ROM, DVD, or other disk storage, magnetic cassettes, magnetic tape, magnetic disk storage unit, and/or the like.

As used herein, the term "computer media" includes volatile and non-volatile media and removable and non-removable media embodied in any process or technology, capable of storing information, such as human-readable instructions. computer, data structures, program modules, or other data. In this regard, system memory 1504 and storage medium 1508 depicted in FIGURE 15 are simple examples of computer media. In one embodiment, the computer media are used to store data for use by programs.

Suitable commercially available implementations of computing devices are known including processor 1502, system memory 1504, communication bus 1506, storage medium 1508, and network interface 1510. For ease of illustration, and since this is not important to understanding the claimed subject matter, FIGURE 15 does not show some of the components typical of many computing devices. In this regard, computing device 1500 may include input devices, such as a keyboard, numeric keypad, mouse, microphone, touch input device, touch screen, tablet, and/or the like. . These input devices may be coupled to computing device 1500 by wired or wireless connections including RF, infrared, serial, parallel, Bluetooth, USB, or other suitable connection protocols using physical or wireless connections. Similarly, computing device 1500 may also include output devices, such as a display, speakers, printer, etc. Since these devices are well known in the art, they are not further illustrated or described herein.

Extensions and a other solutions

Many other solutions for the systems and devices described herein are possible. For example, individual modules or subsystems can be separated into additional modules or subsystems, or combined into fewer modules or subsystems. As another example, modules or subsystems may be omitted or supplemented with other modules or subsystems. As another example, functions indicated as being performed by a particular device, module, or subsystem may instead be performed by one or more other devices, modules, or subsystems. Although certain examples in this disclosure include descriptions of devices comprising specific hardware components in specific arrangements, the techniques and tools described herein can be modified to accommodate different hardware components, combinations, or arrangements. Additionally, while some examples in this disclosure include descriptions of specific usage scenarios, the techniques and tools described here can be modified to accommodate different usage scenarios. Functionality described as being implemented in software may instead be implemented in hardware, or vice versa.

Many other solutions for the techniques described here are possible. For example, processing steps in the different techniques can be separated into additional steps or combined into fewer steps. As another example, processing steps in the different techniques may be omitted or supplemented by other techniques or processing steps. As another example, processing steps described as occurring in a particular order may instead occur in a different order. As another example, processing steps described as being performed in a series of steps may instead be managed in parallel, by multiple software modules or processes simultaneously managing one or more of the illustrated processing steps.

Many other solutions for the user interfaces described here are possible. In practice, the user interfaces described here may be implemented as separate user interfaces or as different states of the same user interface, and the different states may be presented in response to different events, e.g., user input events . User interfaces can be customized for different devices, input and output capabilities, and the like. For example, user interfaces may be presented differently depending on display size, display orientation, whether the device is a mobile device, etc. The information and elements of the user interfaces shown in the user interfaces can be modified, supplemented, or replaced by other elements in different possible implementations. For example, different combinations of graphical user interface elements including text boxes, sliders, drop-down menus, radio buttons, soft buttons, etc., or any other user interface elements, including hardware elements , such as buttons, switches, scroll wheels, microphones, cameras, etc., can be used to accept user input in various forms. As another example, the user interface elements used in a particular implementation or configuration may depend on the particular input and/or output capabilities of a device (e.g., touch screen). Information and user interface elements can be presented in different temporal, logical and spatial arrangements in different possible implementations. For example, information or user interface elements described as being presented simultaneously on a single page or screen may also be presented at different times, on different pages or different screens, etc. As another example, certain information or user interface elements may be presented conditionally based on previous input, user preferences, or the like.

Although embodiments have been illustrated and described, it will be appreciated that various modifications may be made herein without departing from the spirit and scope of the invention.

Claims (9)

Non-transitory computer medium on which instructions are stored, configured to, once the instructions have been executed by one or more computer device(s) of a computer system, cause the computer system to execute operations comprising:
obtaining a digital model of the face of a living human subject;
obtaining user data associated with the living human subject; And
generating a computer-assisted personal care program based at least in part on the digital model of the living human subject's face and the user data;
wherein obtaining the digital model of the face includes obtaining a plurality of skin characteristics selected from the following: one or more blemishes, areas of hyperpigmentation, texture, or wrinkles, or a combination of said elements. A computer medium according to claim 1, wherein the computer-assisted personal care program includes one or more operational parameters for a computer-controlled personal care device. Computer medium according to claim 1, the operations further comprising:
obtaining environmental data associated with the environment of the living human subject, wherein the computer-assisted personal care program is further based on the environmental data
wherein obtaining the environmental data includes obtaining temperature data, humidity data, pollution data, or UV radiation data, or a combination thereof. A computer-implemented method comprising:
obtaining, by a computing device, a digital model of the face of a living human subject;
obtaining, by the computing device, user data associated with the living human subject; And
generating, by the computing device, a computer-assisted skin care program based at least in part on the digital model of the living human subject's face, environmental data, and the user data. A computer-implemented method according to claim 4, wherein generating the computer-assisted skin care program includes generating one or more operational parameters for a computer-driven skin care device. A computer-implemented method according to claim 4, wherein the digital model of the face includes a plurality of skin features, and wherein the plurality of skin features includes at least one of the following: one or more blemishes, areas of hyperpigmentation, texture, or wrinkles, or a combination thereof. A computer-implemented method according to claim 4, further comprising obtaining environmental data associated with the environment of the living human subject, wherein the computer-assisted skin care program is further based on environmental data. A computer-implemented method according to claim 7, wherein obtaining environmental data includes obtaining temperature data, humidity data, pollution data, or UV radiation data, or a combination of these. A computer system comprising a processor and a non-transitory computer medium on which instructions are stored, configured to, once the instructions have been executed by one or more computer device(s) of a computer system, cause the computer system to execute operations including:
obtaining a digital model of the face of a living human subject;
obtaining user data associated with the living human subject;
obtaining environmental data associated with the environment of the living human subject;
generating a computer-assisted skin care program based at least in part on the digital facial model, user data, and environmental data of the living human subject.