Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/90—Determination of colour characteristics
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
  • G06F3/04842—Selection of displayed objects or displayed text elements
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
  • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T11/00—2D [Two Dimensional] image generation
  • G06T11/001—Texturing; Colouring; Generation of texture or colour
• • A—HUMAN NECESSITIES
  • A45—HAND OR TRAVELLING ARTICLES
  • A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
  • A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
  • A45D44/005—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms for selecting or displaying personal cosmetic colours or hairstyle
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30196—Human being; Person
  • G06T2207/30201—Face

Definitions

• a user’s lip tone can be one from a spectrum of possible lip tones, making it a challenge to predict the resulting hue of an applied lipstick Choosing the desired lipstick hue can be a difficult decision for a user, which is further compounded when the resulting hue of the lipstick is shifted after application to the user’s lip.
• the user’s lip tone and the type of lipstick are contributing factors to an accurate prediction. This can be additionally skewed based on other aspects of the user’ s regimen, including applying other treatments to the lips before and after the lipstick is applied, for example with a lip balm, a lip gloss, or a lip liner.
• an applied pressure of the lipstick onto the user’s lips can affect the resulting coverage and hue. Accordingly, better methods and systems for predicting resulting hue of applied lip cosmetics are desired.
• the present disclosure relates to a method of predicting applied lipstick color, including: extracting reference hue information from one or more digital images of a plurality of lip-tone cards, each lip-tone card including a plurality of lip-tone swatches, a white swatch, and a black swatch; extracting applied hue information from one or more digital images of a plurality of lip- tone cards coated with lipstick; and generating a predicted applied lipstick color based on the extracted applied hue information and one or more inputs indicative of a lipstick selection.
• the present disclosure additionally relates to a method of predicting a resulting hue, tint, tone, or shade of a lipstick product on a user includes generating one or more instances on a graphical user interface indicative of a predicted applied lipstick hue, tint, tone, or shade based on receiving user lip-hue information and a lipstick product selection; and generating one or more instances on a graphical user interface of a lipstick product predicted to result in a hue, tint, tone, or shade, on the user, based on the received user lip-hue information and a desired coated lip-hue.
• Fig. 1A shows a graph of a reflectance measurement, according to an exemplary embodiment of the present disclosure.
• Fig. IB shows a liptone card and a first application card, according to an exemplary embodiment of the present disclosure.
• Fig. 2A shows additional lipstick applied to additional liptone cards, according to an embodiment of the present disclosure.
• Fig. 2B shows a graph relating chroma and lightness, according to an embodiment of the present disclosure.
• Figs. 3-9 show an overview of the features of an embodiment for determining the liptone of a user.
• Fig. 1A shows a graph of a reflectance measurement, according to an exemplary embodiment of the present disclosure.
• the graph relates a reflectance of a material on a y-axis as a function of a wavelength detected on an x-axis.
• Any peak(s) exhibited by a trace in the graph describe hues predominantly reflected by the material.
• the peak correlates to the color of the material.
• a peak at -400 nm describes a blue material.
• a peak at -700 nm describes a red material.
• constructive interference can be seen in the reflectance measurement if peak intensity increases for an additional layer of material added (wherein the material transmits light), as well as for a secondary peak exhibited at a multiple of the peak wavelength.
• a secondary peak for 400 nm blue light emerges at 800 nm.
• the reflectance of the material is described as specular or diffuse based on a smoothness of a surface of the material. Notably, these optical properties impact the appearance and design of lipsticks, and ultimately, their application by a user.
• Fig. IB shows a liptone card 105 and a first application card 105a, according to an exemplary embodiment of the present disclosure.
• the liptone card 105 includes a black swatch 110, a white swatch 115, and at least one liptone swatch 120.
• the at least one liptone swatch 120 is an area on the liptone card 105 that represents a liptone of the user. These can vary and be selected based on liptones measured from the users or predicted via emulation software and reproduced on the liptone card 105.
• the at least one liptone swatch 120 is printed on the liptone card 105.
• the black swatch 110 and the white swatch 115 are areas on the liptone card 105 that represent a black and a white hue, respectively.
• the black swatch 110 and white swatch 115 provide a reference point for which the at least one liptone swatch 120 is compared.
• the black swatch 110 has an RGB value that represents the hue black, for example (0, 0, 0)
• the white swatch 115 has an RGB value that represents the hue white, for example (255, 255, 255).
• An uncoated reflectance spectrum is determined for each of the at least one lipstone swatch 120.
• an optical spectrometer configured to scan through the visible light range can be used.
• an optical image is obtained for each of the at least one liptone swatch 120.
• a camera including an optical sensor is used to capture the images.
• a first lipstick 125 is applied to the black swatch 110, the white swatch 115, and each of the at least one liptone swatch 120 to produce the first application card 105a including a first coated black swatch 110a, a first coated white swatch 115a, and a first at least one coated liptone swatch 120a.
• a hue of the first lipstick 125 is substantially fuchsia.
• the liptone card 105 includes six at least one liptone swatches 120 that progress from lighter near the white swatch 115 to darker near the black swatch 110.
• the six first at least one coated liptone swatches 120a coated with the first lipstick 125a progress similarly from lighter near the first coated white swatch 115a to darker near the first coated black swatch 110a. Since the first coated white swatch 115a and the first coated black swatch 110a are also coated by the first lipstick 125a, the first coated white swatch 115a can exhibit a lightest tint for the first lipstick 125a and the first coated black swatch 110a can exhibit a darkest shade for the first lipstick 125a.
• the first at least one coated liptone swatches 120a can exhibit varying shifts in hue due to the contribution of the underlying liptone hue.
• a coated reflectance is determined for the first coated black swatch 110a, the first coated white swatch 115a, and each of the first at least one coated liptone swatch 120a.
• the optical spectrometer is used. Images are also obtained, for example using the optical camera.
• the shift in the reflectance spectra from the uncoated to the coated reflectance is determined for each of the swatches.
• the change in the reflectance spectra from the white swatch 115 to the first coated white swatch 115a describes a shift in hue without any additional contributions from the user’s liptone, while also describing a maximum lightness and a maximum chroma.
• the change in the reflectance spectra from the black swatch 110 to the first coated black swatch 110a describes a shift in hue without any additional contributions from the user’s liptone, while also describing a minimum lightness and a minimum chroma.
• the changes in the reflectance spectra from each of the at least one liptone swatches 120 to each of the corresponding first at least one coated liptone swatches 120a can be compared to both the change in the reflectance spectra from the white swatch 115 to the first coated white swatch 115a and from the black swatch 110 to the first coated black swatch 110a to determine the shift in hue due to contributions from the user’s liptone.
• chroma describes a chromatic strength of an object’s hue, or the perceived amount of difference from a grey of the same lightness, while lightness describes a perceived brightness of the object’s hue as compared to a perfectly white object.
• a library of images of the first application card 105a including a wide range of the first at least one coated liptone swatch 120a is generated for the first lipstick 125a and stored in a memory storage.
• the hue, chroma, and lightness are also determined for each of the first at least one coated liptone swatch 120a in the library and respective spectra for each are stored in the memory storage.
• Fig. 2A shows additional lipstick applied to additional liptone cards 105, according to an embodiment of the present disclosure.
• a hue of the second lipstick 125b is, for example, substantially scarlet
• a hue of the third lipstick 125c is, for example, substantially orange.
• the second lipstick 125b is applied to the black swatch 110, the white swatch 115, and each of the at least one liptone swatch 120 to produce the second application card 105b including a second coated black swatch 110b, a second coated white swatch 115b, and a second at least one coated liptone swatch 120b.
• the third lipstick 125b is applied to the black swatch 110, the white swatch 115, and each of the at least one liptone swatch 120 to produce the third application card 105c including a third coated black swatch 110c, a third coated white swatch 115c, and a third at least one coated liptone swatch 120c.
• the second application card 105b including a wide range of the second at least one coated liptone swatch 120b is generated for the second lipstick 125b
• the third application card 105c including a wide range of the third at least one coated liptone swatch 120c is generated for the third lipstick 125c, and all are added to the library.
• the corresponding reflectance spectra, chroma, and lightness are also determined and stored in the memory storage.
• Fig. 2B shows a graph relating chroma and lightness, according to an embodiment of the present disclosure.
• a trace is generated describing the chroma (y-axis) as a function of the lightness (x-axis) for each of the swatches for the first lipstick 125a, second lipstick 125b, and third lipstick 125c. While only six liptones are plotted in the graph of Fig. 2B for each lipstick, it may be appreciated that as more of the at least one liptone swatch 120 are coated and measured for each lipstick, the data in each of the traces becomes more granular.
• the library can be utilized to predict the hue of a lipstick applied on the user’s lips.
• the lipstick is the first lipstick 125a and the user’s liptone matches one of the at least one liptone swatches 120.
• the hue of the user’s liptone is within a predetermined deviation threshold of the hue of one of the at least one liptone swatches 120.
• the corresponding reflectance measurement of the first coated at least one liptone swatch 120a for the first lipstick 125a describes the hue of the applied first lipstick 125a on the at least one liptone swatch 120 that most closely matches the user’s liptone
• the user’s liptone does not match one of the at least one liptone swatches 120 in the library. That is, the hue of the user’s liptone is outside a predetermined deviation threshold of the hue of one of the at least one liptone swatches 120.
• interpolation between two of the at least one liptone swatches 120 most closely matching the user’s liptone is performed to yield an interpolated liptone swatch that more closely matches the user’s liptone.
• An interpolation between the two corresponding reflectance measurements for the two of the at least one liptone swatches 120 (most closely matching the user’s liptone) is also performed to yield an interpolated hue for the applied first lipstick 125a.
• targeted data in the library can be interpolated, for example chroma and lightness. Due to the diversity in users, collecting data on myriad liptones to expand the library can improve the accuracy of predicting the hue, chroma, and lightness of a lipstick after application to the user’s lips.
• the library can be sorted by a single liptone to match the liptone of the user.
• all of the lipsticks that have been applied to the at least one liptone swatch 120 that matches the user’s liptone are sorted Subsequently, the user selects a desired appearance (hue) for the applied lipstick from the library and a determination of the lipstick to apply can be performed [0023]
• the library does not include the matching liptone of the user That is, the hue of the user’s liptone is outside a predetermined deviation threshold of the hue of one of the at least one liptone swatches 120.
• interpolation between two of the at least one liptone swatches 120 most closely matching the user’s liptone is performed to yield the interpolated liptone swatch.
• the user’s liptone most closely matches a first at least one liptone swatch 1201 and a second at least one liptone swatch 1201.
• Interpolation between the two corresponding reflectance measurements for all of the lipsticks applied to the first at least one liptone swatch 1201 and the second at least one liptone swatch 1201 is performed to predict the resulting hue of all the applied lipsticks on the interpolated liptone swatch.
• interpolation of the reflectance measurements for a first of the first at least one coated liptone swatch 120al and a second of the first at least one coated liptone swatch 120a2 is performed to predict the hue of the first lipstick 125a coated on the interpolated liptone swatch.
• interpolation of the reflectance measurements for a first of the second at least one coated liptone swatch 120bl and a second of the second at least one coated liptone swatch 120b2 is performed to predict the hue of the second lipstick 125b coated on the interpolated liptone swatch.
• the same interpolation is performed on the third lipstick 125c and all other lipsticks in the library.
• All of the predicted coated hues of the lipsticks on the interpolated liptone swatch can be filtered to yield predicted coated hues within the predetermined deviation threshold of the user’s desired appearance (hue) for the applied lipstick, and a recommendation is made for the lipsticks corresponding to the predicted coated hues within the predetermined deviation threshold.
• a lipstick yielding the exact user’s desired appearance may not exist, the user can be recommended a plurality of options that closely match the desired appearance (hue). For example, the user may select one of the recommended lipsticks and a second recommendation can be made for a product to lighten the tint or darken the shade of the recommended lipstick.
• a recommendation is made for two or more lipsticks to mix coatings in order to achieve the desired appearance (hue).
• the user selects a desired appearance (hue) for the applied lipstick from the library but it does not match any entry in the library.
• the user’s liptone matches one of the at least one liptone swatches 120
• An interpolation is performed between all the lipsticks coated on the matching at least one liptone swatches 120 in order to determine at least two lipsticks that match the desired appearance (hue) when coated on the matching user’s liptone.
• a first recommendation is made for a lipstick that yields a slightly darker shade and a second recommendation is made for a lipstick that yields a slightly lighter tint when coated on the at least one liptone swatch 120 that matches the user’s liptone.
• the liptone card 105 includes additional swatches that vary the hue, chroma, and lightness of the coated lipsticks.
• the liptone card 105 includes additional at least one liptone swatches 120 having lip balm applied thereon before coating with the lipsticks.
• the liptone card 105 includes additional at least one liptone swatches 120 for coating with the lipsticks followed by an additional coating of a second material, such as lip gloss, lip cream, a moisturizer, or any other top coat.
• the liptone card 105 includes additional at least one liptone swatches 120 for coating with different types of lipsticks, for example a matte lipstick, a satin lipstick, a lip tint, a metallic lipstick, etc.
• the liptone card 105 includes additional at least one liptone swatches 120 for coating with a lipstick at varied application pressures, such as light, normal, and firm pressures.
• myriad appearances for myriad lipsticks of the same hue can be catalogued and used to improve a user’s lipstick selection accuracy and experience.
• the liptone card 105 includes artificial lips, for example instead of the at least one liptone swatch 120 being printed on the liptone card 105
• Each of the artificial lips is pigmented according to diverse user liptones, and advantageously, are textured to resemble the texture of real human lips, which affect the resulting appearance of applied lipsticks.
• the artificial lips are molded collagen grown from skin keratinocytes grown on a scaffold, such as an acellular, nonimmunogenic dermal equivalent.
• the aforementioned myriad coatings are coated onto the artificial lips.
• lip balm is applied first, followed by the satin lipstick, followed by the lip gloss.
• a reflectance measurement and optical image can be obtained for each of the steps as each coating is applied and stored in the memory as part of the library.
• the reflectance measured from the artificial lips provides a more diffuse reflectance and realistic application environment as compared to a flat swatch
• Figs. 3-8 illustrate additional details of features for (1) providing the guidance to lipstick formulation on coverage (2) evaluating the current lipstick application protocol and (3) creating models to predict lipstick application on different lip tones from the lipstick bulk tone.
• Embodiments of the present disclosure may also be as set forth in the following parentheticals.
• a method of predicting applied lipstick color comprising: extracting reference hue information from one or more digital images of a plurality of lip-tone cards, each lip-tone card including a plurality of lip-tone swatches, a white swatch, and a black swatch; extracting applied hue information from one or more digital images of a plurality of lip-tone cards coated with lipstick; and generating a predicted applied lipstick color based on the extracted applied hue information and one or more inputs indicative of a lipstick selection.
• a method of predicting a resulting hue, tint, tone, or shade of a lipstick product on a user comprising: generating one or more instances on a graphical user interface indicative of a predicted applied lipstick hue, tint, tone, or shade based on receiving user lip-hue information and a lipstick product selection; and generating one or more instances on a graphical user interface of a lipstick product predicted to result in a hue, tint, tone, or shade, on the user, based on the received user lip-hue information and a desired coated lip-hue.
• a method of predicting applied lipstick color comprising: receiving a desired lip appearance from a user, the desired lip appearance having a target hue; determining a hue of lips of the user; determining, based on application of a lipstick applied to a plurality of liptone swatches on a liptone card, a final hue of the applied lipstick color, the plurality of liptone swatches having a hue within a predetermined deviation range of the hue of the user’s lips; determining the lipstick correlating to the final hue that most closely matches the target hue of the desired lip appearance; and transmitting, to the user, information of the lipstick.
• step of determining the plurality of liptone swatches having a hue within a predetermined deviation range of the hue of the user’s lips further comprises: receiving the plurality of liptone cards, each including the plurality of liptone swatches, a white swatch, and a black swatch; determining a hue of each of the plurality of liptone swatches, the white swatch, and the black swatch; and determining a subset of the plurality of liptone swatches having the hue within a predetermined threshold of the hue of the user’s lips.
• step of determining the lipstick correlating to the final hue that most closely matches the target hue of the desired lip appearance further comprises: coating the subset of the plurality of liptone swatches, the white swatch, and the black swatch with the lipstick; determining the final hue of each of the coated subset of the plurality of liptone swatches; and determining the lipstick coated on the subset of the plurality of liptone swatches having the final hue that most closely matches the target hue of the desired lip appearance.
• a method of predicting applied lipstick color comprising: receiving a lipstick selection from a user, the lipstick having a hue; determining a hue of lips of the user; determining, via a liptone card having the lipstick selection applied thereon, a final hue of the user’s lips having the lipstick selection applied; and transmitting to the user the final hue of the user’s lips.
• step of determining the final hue of the user’s lips having the lipstick selection applied further comprises: receiving the plurality of liptone cards, each including the plurality of liptone swatches, a white swatch, and a black swatch; determining a hue of each of the plurality of liptone swatches, the white swatch, and the black swatch; and determining a matching liptone swatch having a matching hue within a predetermined threshold of the hue of the user’s lips; and applying the lipstick selection to the matching liptone swatch.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Human Computer Interaction (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Spectrometry And Color Measurement (AREA)
• Cosmetics (AREA)
• Measuring And Recording Apparatus For Diagnosis (AREA)

Applications Claiming Priority (5)

Publications (1)

Family

ID=74661532

Family Applications (1)

Country Status (5)

Families Citing this family (1)

Family Cites Families (3)

• 2021
  • 2021-01-29 KR KR1020227023225A patent/KR20220108159A/ko unknown
  • 2021-01-29 JP JP2022544354A patent/JP7530982B2/ja active Active
  • 2021-01-29 EP EP21706149.8A patent/EP4097682A1/de active Pending
  • 2021-01-29 CN CN202180011828.5A patent/CN115039131A/zh active Pending
  • 2021-01-29 WO PCT/US2021/015803 patent/WO2021155217A1/en unknown

Also Published As

Similar Documents

Legal Events