CN110135930B - Method, system and storage medium for virtually applying makeup effect and promoting product - Google Patents

Method, system and storage medium for virtually applying makeup effect and promoting product Download PDF

Info

Publication number
CN110135930B
CN110135930B CN201811313304.8A CN201811313304A CN110135930B CN 110135930 B CN110135930 B CN 110135930B CN 201811313304 A CN201811313304 A CN 201811313304A CN 110135930 B CN110135930 B CN 110135930B
Authority
CN
China
Prior art keywords
user
makeup
target images
profile data
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811313304.8A
Other languages
Chinese (zh)
Other versions
CN110135930A (en
Inventor
林宜玮
吴鸿
林盈吟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of CN110135930A publication Critical patent/CN110135930A/en
Application granted granted Critical
Publication of CN110135930B publication Critical patent/CN110135930B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0251Targeted advertisements
    • G06Q30/0255Targeted advertisements based on user history
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0251Targeted advertisements
    • G06Q30/0269Targeted advertisements based on user profile or attribute
    • G06Q30/0271Personalized advertisement
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0631Item recommendations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • G06Q30/0643Graphical representation of items or shoppers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Strategic Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

Disclosed are methods, systems, and storage media for virtually applying cosmetic effects and promoting products, wherein a computing device obtains and verifies a user credential provided by a user of a client device. The computing device obtains an album containing a plurality of digital images and accesses a file data associated with the user credentials. According to the file data, a group of target images is extracted from the plurality of digital images, and each target image in the group depicts a face area of the user. The computing device retrieves at least one recommended product for each target image according to the archive data. The computing device edits each target image according to the retrieved at least one recommended product. The computing device generates a user interface displayed on the client device, the user interface displays the one or more edited target images, and the user interface further displays the at least one recommended product retrieved in association with the one or more edited target images.

Description

Method, system and storage medium for virtually applying makeup effect and promoting product
Technical Field
The present invention relates to media editing, and more particularly, to a method, system, and non-transitory computer-readable storage medium for representing a simulated makeup appearance of a cosmetic product in an album and promoting the cosmetic product.
Background
With the widespread use of smart phones, tablets, and other display devices, people can access digital content at any time, and applications in smart phones and other portable display devices are becoming popular for viewing media content. Although Targeted Advertising is commonly practiced on the internet, there is still a need for improved platforms that can promote products to the public.
Disclosure of Invention
In one embodiment, a computing device obtains and verifies user credentials, the user credentials provided by a user of a client device. The computing device obtains an album containing a plurality of digital images and accesses a file data associated with the user credentials. According to the archive data, a group of target images is extracted from the plurality of digital images, wherein each target image in the group depicts a face area of the user. The computing device retrieves at least one recommended product for each target image according to the archive data, and edits each target image according to the retrieved at least one recommended product. The computing device generates a user interface that is displayable on the client device, the user interface displaying the one or more edited target images, the user interface further displaying the at least one recommended product retrieved in association with the one or more edited target images.
Preferably, the step of editing each target image comprises: and representing the virtual makeup of a makeup product corresponding to the at least one retrieved recommended product on the face area of the user.
Preferably, the step of accessing profile data associated with the user credentials comprises accessing a user profile data and a cosmetic preference profile data, the user profile comprising personal data associated with the user; the makeup preference profile data specifies one or more makeup effects selected by the user, wherein each makeup preference profile data is associated with a reference image that is specified in the makeup preference profile data in which the one or more makeup effects are depicted, wherein the reference image associated with the makeup preference profile data is uploaded to the cloud computing device.
Preferably, the one or more makeup effects depicted in the reference image are applied by the user in advance.
Preferably, the step of retrieving at least one recommended product for each target image according to the profile data comprises: for each target image: accessing an Exchangeable Image File Format (EXIF) and obtaining a position data for each target Image; determining a facial skin color of a facial region of a user; determining an age of the user according to the user profile; determining makeup preference according to the makeup preference profile data; and retrieving at least one recommended product for each target image based on at least one of location data, facial skin color, age of the user, and cosmetic preferences.
Preferably, the step of retrieving at least one recommended product for each target image based on at least one of location data, facial skin color, age of the user, and cosmetic preferences comprises: assigning a weight for each location data, facial skin color, age of the user, and makeup preference; at least one recommended product is retrieved for each target image based on a weighted combination of the location data, facial skin color, age of the user, and cosmetic preferences.
Preferably, the step of retrieving at least one recommended product for each target image according to the profile data comprises: identifying another user according to the overlapped file data; and retrieving at least one recommended product for each target image according to at least one recommended product selected by another user in advance and applied.
In yet another embodiment, a system includes a memory storing a plurality of instructions and a processor coupled to the memory, the processor configured with the plurality of instructions including at least: a user credential is obtained and verified, the user credential being provided by a user of a client device. The processor is further configured to obtain an album containing a plurality of digital images and access a file data associated with the user credentials. The processor is further configured to extract a group of target images from the plurality of digital images according to the file data, wherein each target image in the group depicts a facial area of the user. The processor is further configured to retrieve at least one recommended product for each target image based on the profile data. The processor is further configured to edit each target image according to the retrieved at least one recommended product. The processor is further configured to generate a user interface displayed on the client device, the user interface displaying the one or more edited target images, the user interface further displaying the at least one recommended product retrieved in association with the one or more edited target images.
Preferably, the processor edits each target image by expressing a virtual makeup of a makeup product corresponding to the retrieved at least one recommended product to the face area of the user.
Preferably, the processor instructions for accessing profile data associated with the user credential are provided by accessing a user profile and cosmetic preference profile data; the user profile includes personal data associated with the user, and makeup preference profile data detailing one or more makeup effects selected by the user, wherein each makeup preference profile data is associated with a reference image detailing the one or more makeup effects depicted in the makeup preference profile data, wherein the reference image associated with the makeup preference profile data is uploaded to the cloud computing device.
Preferably, the one or more makeup effects depicted in the reference image are applied by the user in advance.
Preferably, the instructions for the processor to retrieve at least one recommended product for each target image based on the profile data comprises the steps of: for each target image: accessing an exchangeable image file format and obtaining a position data for each target image; determining a facial skin color of a facial region of a user; determining an age of the user according to the user profile; determining makeup preference according to the makeup preference profile data; and retrieving at least one recommended product for each target image based on at least one of location data, facial skin color, age of the user, and cosmetic preferences.
Preferably, the instructions for the processor to retrieve at least one recommended product for each target image based on at least one of location data, facial skin tone, age of the user, and cosmetic preferences include the steps of: assigning weights for location data, facial skin color, age of the user, and cosmetic preferences; and retrieving at least one recommended product for each target image based on a weighted combination of the location data, facial skin color, age of the user, and cosmetic preferences.
Preferably, the instructions for the processor to retrieve at least one recommended product for each target image according to the profile data includes the steps of: identifying another user according to the overlapped file data; and retrieving at least one recommended product for each target image according to at least one recommended product selected by another user in advance and applied.
In another embodiment, a non-transitory computer-readable storage medium stores instructions for execution by a computing device having a processor, the instructions when executed by the processor obtaining and verifying user credentials, the user credentials provided by a user of a client device. The processor is configured to obtain an album containing a plurality of digital images. The processor is configured to access a profile data associated with the user credential. The processor is configured to extract a group of target images from the plurality of digital images according to the archive data, wherein each target image in the group depicts a facial area of the user. The processor is configured to retrieve at least one recommended product for each target image based on the profile data. The processor is configured to edit each target image according to the retrieved at least one recommended product. The processor is configured to generate a user interface displayed on the client device, the user interface displaying one or more edited target images, the user interface further displaying at least one recommended product retrieved in association with the one or more edited target images.
Preferably, the processor edits each target image by expressing a virtual makeup of a makeup product corresponding to the retrieved at least one recommended product to the face area of the user.
Preferably, the processor accesses profile data associated with the user credential by accessing: a user profile and makeup preference profile data, the user profile including personal data associated with the user; the makeup preference profile data specifies one or more makeup effects selected by the user, wherein each makeup preference profile data is associated with a reference image that is specified in the makeup preference profile data in which the one or more makeup effects are depicted, wherein the reference image associated with the makeup preference profile data is uploaded to the cloud computing device.
Preferably, the one or more makeup effects depicted in the reference image are applied by the user in advance.
Preferably, the processor retrieving at least one recommended product for each target image according to the profile data comprises the following steps: for each target image: accessing an exchangeable image file format and obtaining a position data for each target image; determining a facial skin color of a facial region of a user; determining an age of the user according to the user profile; and determining makeup preferences according to the makeup preference profile data; and retrieving at least one recommended product for each target image based on at least one of location data, facial skin color, age of the user, and cosmetic preferences.
Preferably, the processor retrieving at least one recommended product for each target image based on at least one of location data, facial skin tone, age of the user, and cosmetic preferences comprises performing the steps of: assigning weights for location data, facial skin color, age of the user, and cosmetic preferences; and retrieving at least one recommended product for each target image based on a weighted combination of the location data, facial skin color, age of the user, and cosmetic preferences. For a better understanding of the features and technical content of the present invention, reference should be made to the following detailed description of the invention and accompanying drawings, which are provided for purposes of illustration and description only and are not intended to limit the invention.
Drawings
For a better understanding of the nature and technical content of the present invention, reference should be made to the following drawings which are provided to illustrate, by way of example, certain embodiments of the invention and to accompanying descriptive matter, in which various components are illustrated and described in non-limiting examples. In addition, the parts designated by reference numerals in the drawings are commonly used for various viewing angles.
Fig. 1 is a block diagram of virtual application makeup effects and promotion products of a cloud computing device according to various embodiments of the present invention.
Fig. 2 is a schematic diagram of the cloud computing device in fig. 1 according to various embodiments of the present invention.
Fig. 3 is a top-level flow chart illustrating a process of executing a portion of the functions of the cloud computing device of fig. 1 in virtual application of makeup effects and promotion of products according to various embodiments of the present invention.
Fig. 4 is a diagram illustrating a cloud computing device of fig. 1 extracting a group of target images from an album to promote a product according to various embodiments of the invention.
Fig. 5 is a schematic diagram illustrating the cloud computing device of fig. 1 providing a user interface on a display of a client device to promote a product according to various embodiments of the present invention.
Detailed Description
Although targeted advertising is widely implemented on the internet, there is still a need for an improved platform to deliver products to the public. The following embodiments are directed to an improved platform for virtually applying cosmetic products to digital images of a cloud album and recommending cosmetic products to a user. Hereinafter, a system for implementing a makeup virtual application to promote a product will be described in detail, and then the operation of components within the system will be described. Fig. 1 is a block diagram of a cloud computing device 102 implementing virtual makeup application and product promotion. The cloud computing device 102 may include a server computer, a cloud infrastructure, or any system with cloud computing capabilities. In addition, the cloud computing device 102 may employ multiple computing devices that may be arranged, for example, in one or more Server banks (Server banks), electronic Computer banks (Computer banks), or other arrangements, which may be located in a single location or distributed across different geographic locations.
The cloud computing device 102 is coupled to a network 120, and the network 120 may be Internet (Internet), intranet (intranet), extranet (extranet), Wide Area network (Wans), Local Area network (Lans), Wired network (Wired network), Wireless network (Wireless network), or other suitable network, and may also be coupled to connect two or more Networks. Client devices 122a, 122b, 122c are coupled in communication with cloud computing device 102 via network 120 to upload and access digital images 124 in albums 118 stored in a database 116 maintained by cloud computing device 102. As known to those skilled in the art, the digital image 124 can be encoded in any of the following file formats: JPEG (Joint Photographic Experts group), TIFF (tagged Image File format), PNG (Portable Network graphics), GIF (graphics Interchange format), BMP (bitmap), or other types of digital formats, but not limited thereto.
Alternatively, digital image 124 may be transcoded from a still image in a video into the following file format: MPEG-1(Motion Picture Experts Group-1), MPEG-2, MPEG-4, H.264, 3GPP (third Generation part front project), 3GPP-2, SD-Video (Standard-Definition Video), HD-Video (High-Definition Video), DVD (Digital Video Disc) multimedia, VCD (Video Compact Disc) multimedia, HD-DVD (High-Definition Digital Video Disc) multimedia, DTV/HDTV (Digital Video/High-Definition Digital Video) multimedia, AVI Media (Audio Video inter), DV (Digital Video), Windows (Windows) QT, WMV (Video) Audio, Video Media Group, flash (Video) multimedia, Video Media Group, Video (MPEG) Media Group, Video (MPEG-3, HD-Video) Video (High-Definition Video) multimedia, Video (Audio Video) multimedia, Video (Audio Media Group, Video (WM) Media Group, Windows (Windows) Media Group, Windows (Windows III), (Windows) Media Group, Windows (Windows) Media Group III), (Windows (MPEG) Media Group III), (Windows Group III), (WMV) and Windows Group III), (WM) Media Group III), (WM) Media Group III, Windows (MPEG Media Group III), (WM) Media Group III, Audio Group III), but is not limited thereto.
A virtual makeup applicator 104 executes on a processor in the cloud computing device 102 and includes a user credential server 106, a target photograph identifier 108, a product recommendation server 110, and an image editor 112. The user credential server 106 is configured to obtain and verify user credentials transmitted from a user of a client device 122 in order to access an album 118 previously uploaded by the user. Upon receiving the user credentials of the user, the user credentials server 106 accesses the profile data 126 in the database 116 to verify the user credentials.
If the user's credentials are verified by the user credential server 106, the target photo identifier 108 is configured to obtain an album 118 containing a plurality of digital images 124, and the profile data 126 accessed by the user credential server 106 may include general information about the user, such as user account information (e.g., login and password data), user age, location of the user, friends/acquaintances who are granted access to the user's album 118, and the like. The profile data 126 may also include a user's makeup preference profile data detailing the user's makeup results, preferences of the cosmetic product that may generate various facial features, data of the cosmetic product that was purchased in the past, and the like.
The target photo identifier 108 generates a group of target images from the plurality of digital images 124 according to the file data, and the individual can recognize that the content of the digital images 124 in the album 118 is wide and can include, for example, landscape photos or photos including other individuals. In some embodiments, the target images of the group include a digital image 124 associated with the profile data and specifically depicting the facial area of the user.
The product recommendation server 110 is configured to retrieve at least one recommended product for each target image according to the profile data. For example, based on the age of the user in the profile data and the particular cosmetic products that have been purchased in the past in the cosmetic preference profile data, the product recommendation server 110 may retrieve and recommend a lipstick for a particular brand to the user.
The image editor 112 is configured to edit each target image according to the retrieved recommended product. For example, the image editor 112 may apply a lipstick simulation of a particular brand to some or all of the target images based on the attributes of the target images. Further, the image editor 112 may apply the lipstick of the specific brand in an outdoor setting simulation to the target image depicting the user, and the image editor 112 may also apply the eye shadow of the specific brand in an indoor setting simulation to the target image depicting the user. In this regard, the image editor 112 may be configured to edit the target images based on the archival data and the properties of each target image. The image editor 112 is further configured to generate a user interface for display on the client device 122, where the user interface displays one or more edited target images. The user interface further displays the retrieved recommended products (e.g., lipstick) associated with the one or more edited target images.
Fig. 2 is a block diagram of the cloud-side computing device 102 in fig. 1. The cloud computing device 102 may be embodied as any of a variety of wired and/or wireless computing devices, such as a server electronic computing device or any system with computing capabilities. As previously described, the cloud computing device 102 may employ a plurality of computing devices that may be arranged, for example, as one or more server banks, electronic calculator banks, or other arrangements. The computing device may be located in a single location or dispersed across different geographic locations.
As shown in fig. 2, the cloud computing device 102 includes a memory 214, a processing device 202, a plurality of Input/Output interfaces (I/O interfaces) 204, a network Interface 206, a display 208, a peripheral Interface 211, and a mass storage 226, and each of the components in the computing device 102 is connected via a Local Data Bus (Local Data Bus) 210.
The Processing device 202 may include any custom made or commercially available processor, a Central Processing Unit (CPU) or a coprocessor among several computing processors associated with the cloud computing device 102, a semiconductor microprocessor (in the form of a microchip), a Macroprocessor (microprocessor), one or more Application Specific Integrated Circuits (Asics), a plurality of suitably configured digital logic gates, and other common electronic configurations comprising a plurality of discrete components for coordinating the overall operation of the computing system, both independently and in various combinations.
The Memory 214 may include either a Volatile Memory component (Volatile Memory Element) or a non-Volatile Memory component (Nonvolatile Memory Elements). For example, the volatile Memory component includes a Random Access Memory (RAM), such as a Dynamic Random Access Memory (DRAM) or a Static Random Access Memory (SRAM). The non-volatile Memory component may be a Read-Only Memory (ROM), hard disk, magnetic tape, or Compact Disc Read-Only Memory (cd-ROM). The memory 214 generally includes a Native operating System 216, one or more Native applications (Native Application), Emulation systems (Emulation System), or Emulated applications (Emulated Application) for any kind of operating System and/or Emulated hardware platform or Emulated operating System. For example, the aforementioned applications (i.e., native applications or simulation applications) may comprise specific software, i.e., comprise some or all of the components of computing device 102 in FIG. 1. In such embodiments, the components are stored in the memory 214 and executed by the processing apparatus 202, and thus, the processing apparatus 202 may perform the operations/functions of the disclosed features. The components in the memory 214 are well known to those skilled in the art, and therefore some of the components in the memory 214 are not described in detail for the sake of brevity. In some embodiments, the cloud computing device 102 may be implemented in hardware and/or software.
The input/output interface 204 provides a plurality of interfaces to input or output information. For example, when the cloud computing device 102 comprises a personal computer, the aforementioned components may be connected to one or more input/output interfaces 204, such as a keyboard and a mouse, as shown in FIG. 2. The display 208 may include a computer monitor, a plasma screen of a personal computer, a liquid crystal display of a hand held device, a touch screen, or other display device.
In the present disclosure, a non-transitory computer readable storage medium stores a program for use by or in connection with an instruction execution system, apparatus, or device. More specifically, specific examples of the computer-readable storage medium may include, but are not limited to, a Portable computer diskette, a random access Memory, a Read-Only Memory, an Erasable Programmable Read-Only Memory (EPROM, EEPROM, or Flash Memory), and a Portable Compact Disc Read-Only Memory (CDROM).
Referring to fig. 3, fig. 3 is a flowchart 300 of virtual application of makeup effects and product promotion in various embodiments of the cloud-side computing device 102 of fig. 1 according to the present disclosure. The flow diagram 300 in fig. 3 is merely an example of different types of functional arrangements that may be used to implement the operation of the various components of the cloud computing device 102. In other words, the flowchart 300 of fig. 3 may be considered to describe one or more embodiments of a method of carrying out the cloud computing device 102.
Although flowchart 300 of fig. 3 discloses a particular order of execution of the steps, the order of the steps is merely to aid in understanding the present invention and the actual order of operation may differ from that described. For example, the order of execution of the blocks may be adjusted, reversed, or the blocks may be combined. Also, several blocks shown in FIG. 3 in sequential order may be performed simultaneously or partially simultaneously. And such modifications and alterations are still within the scope of the present disclosure.
In block 310, cloud computing device 102 of fig. 1 may obtain and verify user credentials provided by a user from a client device 122. In block 320, cloud computing device 102 obtains an album containing a plurality of digital images. In block 330, the cloud computing device 102 accesses a file data 126 (FIG. 1) regarding the user credentials. In some embodiments, accessing profile data regarding the user credentials includes accessing a user profile that includes personal data of the user and accessing a cosmetic preference profile of one or more cosmetic effects selected by the user.
In some embodiments, each makeup preference profile is associated with an image in which one or more makeup effects are depicted in the makeup preference profile data, and the one or more makeup effects may be applied by the user in advance. In this regard, the image may be used as a reference image to indicate one or more makeup effects preferred by the user. In some embodiments, cloud computing device 102 is configured to analyze the reference image and identify cosmetic effects previously applied by the user to identify preferred cosmetic effects of the user. The cloud computing device 102 may then utilize the information to provide the user with recommended products. In some embodiments, images relating to makeup preference profile data are uploaded to cloud computing device 102. At block 340, the cloud computing device 102 extracts a group of target images from the plurality of digital images according to the archive data. In some embodiments, the target image of the group includes a facial region of the user depicted with the client device 122.
In block 350, the cloud computing device 102 retrieves at least one recommended product for each target image according to the user profile. In some embodiments, cloud computing device 102 retrieves at least one recommended product for each target image based on profile data 126 in the following steps. The cloud computing device 102 accesses an Exchangeable Image File Format (EXIF) of each target Image and obtains a location data for each target Image. The cloud computing device 102 determines a facial skin tone of the user's facial region. Cloud computing device 102 determines an age of the user based on the user profile. Cloud computing device 102 determines makeup preferences based on the makeup preference profile data.
The cloud computing device 102 retrieves at least one recommended product based on at least one of the location data, the facial skin color, the user's age, cosmetic preferences, and combinations thereof. In some embodiments, the step of retrieving at least one recommended product involves assigning weights to the location data, the facial skin color, the user's age, and the makeup preference, and retrieving at least one recommended product for each target image based on a weighted combination of the location data, the facial skin color, the user's age, and the makeup preference.
In some embodiments, the cloud computing device 102 retrieves at least one recommended product for each target image according to the overlapped profile data by: according to the overlapped file data, another user is identified, and then at least one recommended product is searched for each target image according to at least one recommended product selected by the other user in advance and applied. Further, referring back to FIG. 1, for example, the user credential server 106 can determine that there is a similarity between a current user profile and a second user profile. For example, the user credential server 106 may examine the user profile data and determine that the current user and the second user are of the same age and/or reside in the same city (e.g., location). The user credential server 106 may examine the second user's makeup preference profile data and identify cosmetic products of prior interest to the second user. Based on the information, the product recommendation server 110 may display the same cosmetic product and recommend it to the current user.
In block 360, the cloud computing device 102 edits each target image according to the retrieved at least one recommended product. In some embodiments, editing each target image comprises: and representing the virtual makeup formed corresponding to the at least one recommended product retrieved on the face area of the user.
In block 370, the cloud computing device 102 generates a user interface on the client device to display one or more edited target images. The user interface further displays at least one recommended product retrieved and associated with the one or more edited target images. Finally, the flow chart of fig. 3 ends.
Having described the basic architecture of a system for creating virtual makeup and promoting products, please refer to the illustrations of various embodiments that include various features. Fig. 4 is a diagram illustrating various embodiments of the cloud-side computing device 102 of fig. 1 extracting a group of target images from an album to promote a product. Target photo recognizer 108 in cloud computing device 102 accesses an album 118 of the user associated with the user credentials submitted with the user of client device 122.
In one embodiment, the target photo recognizer 108 performs face recognition and generates a group of target images 402 from the digital images 124 of the album 118. As previously mentioned, the content of the digital image 124 in the album 118 is very extensive, and may be, for example, landscape photos or photos including other individuals, etc. As shown in FIG. 4, the target photo recognizer 108 extracts the digital image 124 specifically depicting the user's facial area associated with the user's credentials. In some embodiments, the target photo recognizer 108 may be further configured to decorate the target image 402 depicting the user and other individuals, the cropping target image 402 only remaining covering the user, resulting in a cropped photo 404.
FIG. 5 is a diagram illustrating an example of a user interface 502 provided by a display of the client 122 of FIG. 1 to promote a product according to various embodiments of the invention. The client 122 may be embodied as a smart phone, a tablet computing device, or the like. As shown in FIG. 5, the user interface 502 includes a preview window 504 depicting the user's facial area. The image editor 112 (fig. 1) renders one or more cosmetic effects in the virtual application based on the cosmetic products retrieved by the product recommendation server 110 (fig. 1).
As shown in the exemplary embodiment, user interface 502 displays information 506 applied to a cosmetic product (e.g., lipstick) for a face region of a user. The information 506 displayed to the user includes, but is not limited to: product descriptions (e.g., brand name, product image), product specifications (e.g., color, packaging quantity, texture), purchase information (e.g., Uniform Resource Locator (URL) of a web retailer selling a product), and so forth.
The disclosure is only a preferred embodiment of the invention, and is not intended to limit the scope of the claims, so that all technical equivalents and modifications using the contents of the specification and drawings are included in the scope of the claims.

Claims (18)

1. A method implemented in a cloud computing device, comprising:
obtaining and verifying a user credential, the user credential provided by a user of a client device;
obtaining an album containing a plurality of digital images;
accessing profile data associated with the user credentials, wherein the profile data includes at least one of age, location, or selected brand;
extracting a group of target images from the digital images according to the archive data, wherein each target image in the group depicts a face area of the user;
retrieving at least one recommended product for each of the target images according to the archive data, wherein retrieving at least one recommended product for each of the target images according to the archive data comprises:
identifying another user according to the overlapped file data; and
retrieving at least one recommended product for each of the target images according to at least one recommended product selected and applied by the other user in advance;
editing each target image according to the at least one retrieved recommended product; and
generating a user interface displayed on the client device, the user interface displaying one or more edited target images, the user interface further displaying at least one of the recommended products retrieved in association with one or more edited target images.
2. The method of claim 1, wherein the step of editing each of the target images comprises:
and representing the virtual makeup of a makeup product corresponding to the at least one retrieved recommended product on the face area of the user.
3. The method of claim 1, wherein accessing the profile data associated with the user credentials comprises accessing:
a user profile data, said user profile including personal data associated with said user; and
a makeup preference profile data detailing one or more makeup effects selected by the user, wherein each of the makeup preference profile data is associated with a reference image detailing one or more makeup effects depicted in the makeup preference profile data, wherein the reference image associated with the makeup preference profile data is uploaded to the cloud computing device.
4. The method of claim 3, wherein one or more of the cosmetic effects depicted in the reference image were previously applied by the user.
5. The method of claim 3, wherein retrieving at least one recommended product for each of the target images based on the profile data comprises:
for each of the target images, performing the following steps:
accessing an exchangeable image file format and obtaining a position data for each of the target images;
determining a facial skin color of the facial region of the user;
determining an age of the user according to the user profile; and
determining makeup preference according to the makeup preference profile data; and
retrieving at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin color, the age of the user, and the makeup preferences.
6. The method of claim 5, wherein retrieving at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin tone, the age of the user, and the cosmetic preferences comprises:
assigning a weight to each of the location data, the facial skin color, the age of the user, and the makeup preferences; and
retrieving at least one of the recommended products for each of the target images based on a weighted combination of the location data, the facial skin color, the age of the user, and the cosmetic preferences.
7. A system implemented on a cloud computing device, comprising:
a memory, said memory storing a plurality of instructions; and
a processor coupled to the memory and configured with a plurality of the instructions, the plurality of instructions comprising:
obtaining and verifying a user credential, the user credential provided by a user of a client device;
obtaining an album containing a plurality of digital images;
accessing profile data associated with the user credentials, wherein the profile data includes at least one of age, location, or selected brand;
extracting a group of target images from the digital images according to the archive data, wherein each target image in the group depicts a face area of the user;
retrieving at least one recommended product for each of the target images according to the profile data, wherein the instructions for the processor to retrieve at least one recommended product for each of the target images according to the profile data comprises the steps of:
identifying another user according to the overlapped file data; and
according to at least one of the recommended products previously selected and applied by the other user,
retrieving at least one of the recommended products for each of the target images;
editing each target image according to the at least one retrieved recommended product; and
generating a user interface displayed on the client device, the user interface displaying one or more edited target images, the user interface further displaying at least one of the recommended products retrieved in association with one or more edited target images.
8. The system of claim 7, wherein the processor edits each of the target images by representing a virtual makeup of a makeup product corresponding to the retrieved at least one of the recommended products on the face area of the user.
9. The system of claim 7, wherein the instructions for the processor to access the profile data associated with the user credential are accessed by:
a user profile, the user profile including personal data related to the user; and
a makeup preference profile data detailing one or more makeup effects selected by the user, wherein each of the makeup preference profile data is associated with a reference image detailing one or more makeup effects depicted in the makeup preference profile data, wherein the reference image associated with the makeup preference profile data is uploaded to the cloud computing device.
10. The system of claim 9, wherein one or more of the cosmetic effects depicted in the reference image were previously applied by the user.
11. The system of claim 9, wherein the instructions for the processor to retrieve at least one recommended product for each of the target images based on the profile data comprises the steps of:
for each of the target images, by:
accessing an exchangeable image file format and obtaining a position data for each of the target images;
determining a facial skin color of the facial region of the user;
determining an age of the user according to the user profile; and
determining makeup preference according to the makeup preference profile data; and
retrieving at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin color, the age of the user, and the makeup preferences.
12. The system of claim 11, wherein the instructions for the processor to retrieve at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin color, the age of the user, and the makeup preferences include the steps of:
assigning weights to the location data, the facial skin color, the age of the user, and the makeup preferences; and
retrieving at least one of the recommended products for each of the target images based on a weighted combination of the location data, the facial skin color, the age of the user, and the cosmetic preferences.
13. A non-transitory computer readable storage medium storing instructions for execution by a computing device having a processor, wherein when the instructions are executed by the processor, the computing device performs at least the following:
obtaining and verifying a user credential, the user credential provided by a user of a client device;
obtaining an album containing a plurality of digital images;
accessing profile data associated with the user credentials, wherein the profile data includes at least one of age, location, or selected brand;
extracting a group of target images from the digital images according to the archive data, wherein each target image in the group depicts a face area of the user;
retrieving at least one recommended product for each of the target images according to the profile data, wherein the instructions for the processor to retrieve at least one recommended product for each of the target images according to the profile data comprises the steps of:
identifying another user according to the overlapped file data; and
retrieving at least one recommended product for each of the target images according to at least one recommended product selected and applied by the other user in advance;
editing each target image according to the at least one retrieved recommended product; and
generating a user interface displayed on the client device, the user interface displaying one or more edited target images, the user interface further displaying at least one of the recommended products retrieved in association with one or more edited target images.
14. The non-transitory computer readable storage medium of claim 13, wherein the processor edits each of the target images by representing a virtual makeup of a cosmetic product corresponding to the retrieved at least one of the recommended products on the facial area of the user.
15. The non-transitory computer readable storage medium of claim 13, wherein the processor accesses the profile data associated with the user credential by accessing:
a user profile, the user profile including personal data related to the user; and
a makeup preference profile data detailing one or more makeup effects selected by the user, wherein each of the makeup preference profile data is associated with a reference image detailing one or more makeup effects depicted in the makeup preference profile data, wherein the reference image associated with the makeup preference profile data is uploaded to a cloud computing device.
16. The non-transitory computer-readable storage medium of claim 15, wherein one or more of the cosmetic effects depicted in the reference image were previously applied by the user.
17. The non-transitory computer readable storage medium of claim 15, wherein the processor retrieves at least one recommended product for each of the target images according to the profile data comprises:
for each of the target images, by:
accessing an exchangeable image file format and obtaining a position data for each of the target images;
determining a facial skin color of the facial region of the user;
determining an age of the user according to the user profile; and
determining makeup preference according to the makeup preference profile data; and
retrieving at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin color, the age of the user, and the makeup preferences.
18. The non-transitory computer readable storage medium of claim 17, wherein the processor retrieves at least one of the recommended products for each of the target images based on at least one of the location data, the facial skin tones, the age of the user, and the cosmetic preferences, comprising:
assigning weights to the location data, the facial skin color, the age of the user, and the makeup preferences; and
retrieving at least one of the recommended products for each of the target images based on a weighted combination of the location data, the facial skin color, the age of the user, and the cosmetic preferences.
CN201811313304.8A 2018-02-02 2018-11-06 Method, system and storage medium for virtually applying makeup effect and promoting product Active CN110135930B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862625400P 2018-02-02 2018-02-02
US62/625,400 2018-02-02

Publications (2)

Publication Number Publication Date
CN110135930A CN110135930A (en) 2019-08-16
CN110135930B true CN110135930B (en) 2022-02-11

Family

ID=67568334

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811313304.8A Active CN110135930B (en) 2018-02-02 2018-11-06 Method, system and storage medium for virtually applying makeup effect and promoting product

Country Status (1)

Country Link
CN (1) CN110135930B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113392671A (en) * 2020-02-26 2021-09-14 上海依图信息技术有限公司 Commodity retrieval method and device based on customer actions and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668451A (en) * 2007-03-08 2010-03-10 惠普开发有限公司 Method and system for recommending a product based upon skin color estimated from an image
KR20140077322A (en) * 2012-12-14 2014-06-24 한국전자통신연구원 Method for recommending cosmetic products and apparatus using the method
CN104873197A (en) * 2014-02-28 2015-09-02 花王株式会社 Skin evaluation method, skin care product or cosmetic recommendation method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012181688A (en) * 2011-03-01 2012-09-20 Sony Corp Information processing device, information processing method, information processing system, and program
US9760935B2 (en) * 2014-05-20 2017-09-12 Modiface Inc. Method, system and computer program product for generating recommendations for products and treatments

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668451A (en) * 2007-03-08 2010-03-10 惠普开发有限公司 Method and system for recommending a product based upon skin color estimated from an image
KR20140077322A (en) * 2012-12-14 2014-06-24 한국전자통신연구원 Method for recommending cosmetic products and apparatus using the method
CN104873197A (en) * 2014-02-28 2015-09-02 花王株式会社 Skin evaluation method, skin care product or cosmetic recommendation method

Also Published As

Publication number Publication date
CN110135930A (en) 2019-08-16

Similar Documents

Publication Publication Date Title
EP3522117B1 (en) Systems and methods for virtual application of cosmetic effects to photo albums and product promotion
US10867416B2 (en) Harmonizing composite images using deep learning
US20190166980A1 (en) Systems and Methods for Identification and Virtual Application of Cosmetic Products
US20210264139A1 (en) Creating videos with facial expressions
WO2021008166A1 (en) Method and apparatus for virtual fitting
KR101348521B1 (en) Personalizing a video
US8692940B2 (en) Method for producing a blended video sequence
US9058757B2 (en) Systems and methods for image or video personalization with selectable effects
US20140369627A1 (en) Systems and Methods for Image Editing
US8135724B2 (en) Digital media recasting
EP3690825B1 (en) Systems and methods for virtual application of makeup effects based on lighting conditions and surface properties of makeup effects
US11682067B2 (en) Systems and methods for virtual application of cosmetic products based on facial identification and corresponding makeup information
US11582519B1 (en) Person replacement utilizing deferred neural rendering
US11581020B1 (en) Facial synchronization utilizing deferred neural rendering
CN111784431B (en) Video generation method, device, terminal and storage medium
CN110135930B (en) Method, system and storage medium for virtually applying makeup effect and promoting product
US20210015242A1 (en) Systems and methods for recommendation of makeup effects based on makeup trends and facial analysis
KR101947553B1 (en) Apparatus and Method for video edit based on object
KR102553332B1 (en) Method and apparatus for editing content on a live broadcasting platform
US20180330167A1 (en) Personalized Augmented Reality
CN110135929A (en) It is rendered in the system, method and storage media of virtual makeup application program
CN110136272B (en) System and method for virtually applying makeup effects to remote users
CN110930173B (en) System and method for execution on computing device
US20240144585A1 (en) Systems and methods for adjusting lighting intensity of a face chart
WO2018018076A1 (en) Creating videos with facial expressions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant