EP4162447A1 - Auf mundpflege basierende digitale bildgebungssysteme und verfahren zur bestimmung des wahrgenommenen attraktivität eines gesichtsbildteils - Google Patents
Auf mundpflege basierende digitale bildgebungssysteme und verfahren zur bestimmung des wahrgenommenen attraktivität eines gesichtsbildteilsInfo
- Publication number
- EP4162447A1 EP4162447A1 EP20938571.5A EP20938571A EP4162447A1 EP 4162447 A1 EP4162447 A1 EP 4162447A1 EP 20938571 A EP20938571 A EP 20938571A EP 4162447 A1 EP4162447 A1 EP 4162447A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image portion
- facial image
- attractiveness
- digital image
- facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/179—Human faces, e.g. facial parts, sketches or expressions metadata assisted face recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30036—Dental; Teeth
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Definitions
- the present invention relates to oral care based digital imaging systems and methods for processing information associated with image data such as a digital image, a video defined by a sequence of digital images (also known as frames) .
- image data such as a digital image, a video defined by a sequence of digital images (also known as frames) .
- the present invention relates to a system and a method for determining perceived attractiveness of a facial image portion of at least one person depicted in a digital image.
- Attractiveness plays a central role in human preoccupation with self-image as seen in the proliferation of bodily practices aimed at constantly improving the body and its influence on social relationships.
- Visual cues can strongly influence the attractiveness of a person in the perception of one self or by a population of people.
- One visual cue is facial appearance of a person and concepts used to describe the facial appearance can influence whether a person is perceived to be attractive relative to another person or a population of people.
- attractiveness is highly subjective. Consumers also seek to improve their attractiveness through the use of a variety of consumer products including but limited to oral care products, dental treatments, skin care products, or the like. However, it is difficult to improve the attractiveness without prior knowledge as to what is impacting attractiveness.
- US Patent No. 6,571,003B1 describes an apparatus and method for displaying information associated with a plurality of skin defects and in particular for determining and displaying the location of one or more analysis areas and defect areas associated with a digital image of human skin and for determining the severity of these defects as well as displaying an improvement and/or worsening to the defect areas.
- US Patent No. 8,073,212 describes methods and products for analyzing gingival tissues.
- US Patent No. 10,405,754 describes standardized oral health assessment and scoring using digital images.
- the present invention relates to a computer-implemented method for determining perceived attractiveness of a facial image portion of at least one person depicted in a digital image, the method comprising the steps of:
- FIG. 1 is a diagram illustrating an exemplary system for determining perceived attractiveness of a facial image portion over a network according to the present invention
- FIG. 2 is a diagram illustrating an exemplary functional block diagram associated with detecting a facial image portion according to the present invention
- FIG. 3 is a diagram illustrating an exemplary functional block diagram associated with generating an image description according to the present invention
- FIG. 4 is a diagram illustrating an exemplary functional block diagram associated with presenting an image description according to the present invention
- FIG. 5 is a flow chart illustrating a method for determining perceived attractiveness of a facial image portion according to the present invention
- FIG. 6A is a screen shot illustrating an exemplary graphical user interface presenting a plurality of image descriptions to a user for visualizing perceived attractiveness of a smile in the digital image according to the present invention
- FIG. 6B is the digital image of FIG. 6A comprising an image description presented in the form of a heat map over the facial image portion depicted in the digital image according to the present invention
- FIG. 7 is a schematic drawing illustrating the sof tware architecture of a system according to present invention using an exemplary convolutional neural network (CNN) used for filtering the detected facial image portion and generating the image descriptor for determining perceived attractiveness of a facial image portion according to the present invention
- CNN convolutional neural network
- FIGS. 8A and 8B are conceptual illustrations of exemplary filter visualizations in a CNN of FIG. 10, showing features of interest depicted in one or more filtered feature maps according to the present invention
- FIGS. 9A to 9C are a series of process flow diagrams illustrating a method of obtaining a digital image of a facial image portion according to the present invention.
- FIG. 10 is a flow chart illustrating a method of obtaining digital data image of a detected facial image portion according to the present invention.
- FIGS. 11A, 11B, and 11C are a series of process flow diagrams illustrating a method of generating an image description for a facial image portion according to the present invention
- FIG. 12A illustrate a digital image illustrating an exemplary presentation of an image description for a facial image portion to a user according to the present invention
- FIG. 12B is a detailed view of the facial image portion depicted in the screen shot of FIG. 12A;
- FIG. 13A illustrate a digital image illustrating a variation of an exemplary presentation of an image description for a facial image portion to a user according to the present invention
- FIG. 13B is a detailed view of the facial image portion depicted in the screen shot of FIG. 13A;
- FIG. 14 is a flow chart illustrating a method of providing a product recommendation to improve perceived attractiveness of a facial image portion according to the present invention
- FIGS. 15A to 15D are screen shots, each illustrating an exemplary user interface for determining perceived attractiveness of a facial image portion according to the present invention
- FIG. 15E is a screen shot, illustrating an exemplary user interface for displaying details of a product recommendation for treating facial features defining a facial image portion so as to improve perceived attractiveness of the facial image portion;
- FIG. 16 is a flow chart illustrating a method of demonstrating efficacy of a customized oral care regimen in improving perceived attractiveness of one or more oral features of at least one person depicted in a digital image according to the present invention
- FIGS. 17A and 17B is a series of process flow diagrams illustrating the method of FIG. 16;
- FIG. 18 is a flow chart illustrating a method of demonstrating efficacy of an oral care product according to the present invention.
- FIG. 19A is a digital image illustrating at least a portion of a facial image portion of a person at the start of a predetermined time period prior to treatment with a product recommendation, wherein perceived attractiveness of the facial image portion of the person is determined according to a method according to the present invention
- FIG. 19B is a digital image illustrating the at least a portion of a facial image portion of the person after 1 week of use of the product recommendation;
- FIG. 19C is a digital image illustrating the at least a portion of the facial image portion of the person after 2 weeks of use of the product recommendation;
- FIG. 20 is a flow chart illustrating a method of tracking improvement in the perceived attractiveness of the facial image portion of one or more oral features of at least one person depicted in a digital image according to the present invention.
- FIG. 21 is a screen shot illustrating an exemplary presentation of an image description for a facial image portion of a plurality of persons.
- the present invention relates to a method, apparatus and system for determining perceived attractiveness of a facial image portion in a digital image, and a graphical user interface for visualizing perceived attractiveness.
- a facial image portion is of a person, and may comprise one or more facial features, a facial expression, or combinations thereof.
- Facial features may include nose, mouth, eyes, facial skin, teeth, gum.
- Facial expression may be a smile.
- the perceived attractiveness of a facial image portion provides a benefit as it is multi-faceted, i.e. a perceived attractiveness provides both visual facial features which appear healthy (hereinafter “healthy-looking facial features” ) and visual facial features which appear to have problems or that appear to be less healthy than the healthy-look facial features.
- perceived attractiveness is impacted by positive attributes and negative attributes present in a facial image portion depicted in a digital image.
- Positive attributes may comprise whiteness of teeth, pinkness of gums, smoothness of teeth surfaces or positive appearances of the teeth or gums.
- Negative attributes may comprise teeth stains, gum redness, swollen gums or the like.
- Perceived attractiveness means a quality of a facial image portion of a person depicted in a digital image as perceived by a population of people (hereinafter “population” ) that appeals to the population.
- the population may include professionals, industry experts, consumers or combinations thereof.
- Perceived attractiveness may include but is not limited to, an affinity or a liking for a person having a facial image portion depicted in the digital image, attractiveness of a facial image portion in the context of a person having the facial image portion includes an attribute of the facial image portion that the person is motivated to do something to improve attractiveness of the facial image portion.
- “Person” as used herein means a human being depicted in a digital image.
- “Facial image portion” as used herein means any concept, digital image, or image digital portion based on detection of one of a face of a person depicted or more faces of people, including but not limited to one or more facial features, one or more oral features, a facial expression, or combinations thereof, for example, as determined or detected by the pixel data or otherwise pixels of one or more corresponding digital image (s) .
- “Facial feature” as used herein is an element of a face, and may include but is not limited teeth, gum, nose, mouth, eyes, facial skin, including such features as determined or detected by the pixel data or otherwise pixels of one or more corresponding digital image (s) .
- “Facial expression” as used herein is one or more motions or positions of the muscles beneath the skin of the face, and may include but is not limited to a smile.
- “Smile” as used herein is made up of teeth and/or gums but does not include the lips of the mouth, including, for example, as determined or detected by the pixel data or otherwise pixels of one or more corresponding digital image (s) .
- Oral feature as used herein is an element of the mouth, and may include but is not limited to oral cavity soft tissue, gums, teeth, including, for example, as determined or detected by the pixel data or otherwise pixels of one or more corresponding digital image (s) .
- “Attractiveness Score ( ⁇ ) ” as used herein means a probability value indicative of how appealing a facial image portion of a person depicted in a digital image is to a population of people (hereinafter “population” ) based on positive and negative attributes of the facial image portion (e.g. teeth).
- the probability value may be determined by a model constructed by a machine learning system trained by a training dataset, wherein the training dataset comprises (i) a plurality of simulated images of a facial image portion (e.g., teeth) comprising positive (white areas) and negative (stained areas) attributes; and (ii) an associated class definition (e.g. facial staining) based on positive and negative attributes.
- the probability value may be a numerical value indicative of a perceived attractiveness of a facial image portion depicted in a digital image calculated by the system herein (an attractiveness model is described hereinafter as an example of a machine learning system) , based on the positive and negative attributes of the facial image portion in the digital image.
- An attractiveness model may be based on training data obtained from the raw consumer choice databy estimating the part-worth utilities for the eight attributes’ main effects and limited interaction terms via hierarchical bayes (HB) estimation.
- the Attractiveness Score for any particular training image could then be calculated from the sum of the part-worth utilities across the chosen attribute levels.
- Attribute as used herein means a measurable property of the facial image portion.
- Cosmetic dental attribute as used herein means all cosmetic dental attributes that provide an oral health effect on an area of the oral cavity or impact appearance and/or feel thereof. Some non-limiting examples of a cosmetic dental attribute may include gum inflammation/redness, gum firmness, gum bleeding, gum sensitivity, yellowness, lightness, front surface staining, interproximal (IP) staining in between adjacent teeth, marginal staining (around the gum line) , opacity, shine.
- IP interproximal
- Convolutional neural network is a type of feed-forward artificial neural network where the individual neurons are tiled in such a way that they respond to overlapping regions in the visual field.
- Oral care product refers to a product that includes an oral care active and regulates and/or improves a cosmetic dental attribute condition.
- An oral care product may include but is not limited to, toothpaste, mouth rinse, dental floss, whitening strips, or the like.
- Digital image refers to a digital image formed by pixels in an imaging system including but not limited to standard RGB, or the like and under images obtained under different lighting conditions and/or modes.
- Non-limiting examples of a digital image include color images (RGB) , monochrome images, video, multispectral image, hyperspectral image or the like.
- Non-limiting light conditions include white light, blue light, UV light, IR light, light in a specific wavelength, such as for example light source emitting lights from 100 to 1000 nm, from 300 to 700 nm, from 400 to 700 nm or different combinations of the upper and lower limits described above or combinations of any integer in the ranges listed above.
- a digital image may be a single photograph or a single frame in a series of frames defining a video.
- Image obtaining device refers to a device configured for obtaining images, including but not limited to a digital camera, a photo scanner, a computer readable storage medium capable of storing digital images, and any electronic device including picture taking capabilities.
- “User” as used herein refers to a person who uses at least the features provided herein, including, for example, a device user, a product user, a system user, and the like.
- Module as used herein can be associated with software, hardware, or any combination thereof. In some implementations, one or more functions, tasks, and/or operations of modules can be carried out or performed by software routines, software processes, hardware, and/or any combination thereof.
- Heat map refers to a graphical representation of image data comprised in a digital image in which portions of the facial image portion depicted in the digital image are visually highlighted to identify targets of analysis to be presented in the image description. For example, if the target of analysis is a negative attribute of the facial image portion, an area of the facial image portion which comprises the negative attribute will be visualized.
- Treating refers to providing a product recommendation, customized instructions, use of a recommended product for improving perceived attractiveness of a facial image portion of a subject depicted in a digital image.
- the subject is a person.
- the system described is a system 10 for determining perceived attractiveness of a smile 521 of a person depicted in a digital image 51.
- the apparatus 14 described is an apparatus 14 for determining perceived attractiveness of a smile 521 of a person, and a system for providing a product recommendation to improve perceived attractiveness of a smile 521 of a person depicted in a digital image is also described.
- positive and negative attributes of a smile 521 relate to cosmetic dental attributes as described hereinbefore, i.e. all cosmetic dental attributes that provide an oral health effect on an area of the oral cavity or impact appearance and/or feel thereof.
- the apparatus and the method may be configured for use in a variety of applications to determine perceived attractiveness of other facial image portions, wherein the facial image portion is one or more facial features including but not limited to the nose, skin, lips, eyes, combinations thereof.
- FIG. 1 is a schematic diagram illustrating a system 10 for determining perceived attractiveness of a facial image portion 52 of a person depicted in a digital image 51 according to the present invention.
- the system 10 is a cloud-based system configured for use anywhere, such as for example, through a portable electronic device 12 comprising an image obtaining device 18 and a display (not shown) .
- the portable electronic device 12 may be connected to an apparatus 14 for generating for display on a display, a graphical user interface for visualizing perceived attractiveness of a facial image portion through a network 100.
- the system 10 may be configured as a stand-alone system. It is further contemplated that the portable electronic device 12 may be a touch sensitive display.
- the system 10 may include a network 100, which may be embodied as a wide area network (such as a mobile telephone network, a public switched telephone network, a satellite network, the internet, etc. ) , a local area network (such as wireless-fidelity, Wi-Max, ZigBee TM , Bluetooth TM , etc. ) , and/or other forms of networking capabilities. Coupled to the network 100 are a portable electronic device 12, and an apparatus 14 for generating for display on a display, a graphical user interface 30 (see FIG. 2A) for visualizing perceived attractiveness. The apparatus 14 is remotely located and connected to the portable electronic device 12 through the network 100.
- a wide area network such as a mobile telephone network, a public switched telephone network, a satellite network, the internet, etc.
- a local area network such as wireless-fidelity, Wi-Max, ZigBee TM , Bluetooth TM , etc.
- the apparatus 14 is remotely located and connected to the portable electronic device 12 through the network 100.
- the network 100 may be used to acquire digital images from the portable electronic device 12 and transmitting the digital images to the apparatus 14 to be used in the method 200 according to the present invention described hereinafter with respect to FIG. 7.
- An input device 12a may be coupled to or integral with the portable electronic device 12 for receiving a user input for initiating the processor 14b.
- the portable electronic device 12 may comprise an output device 12b for presenting the image description 53 for the facial image portion 52 depicted in the digital image 51.
- the input device 12a may include but is not limited to a mouse, a touch screen display, or the like.
- the output device 12b may include but is not limited to a touch screen display, a non-touch screen display, a printer, audio output devices such as for example, speakers.
- the portable electronic device 12 may be a mobile telephone, a tablet, a laptop, a personal digital assistant and/or other computing device configured for capturing, storing, and/or transferring a digital image such as a digital photograph. Accordingly, the portable electronic device 12 may include an image obtaining device 18 such as a camera integral with the device 12 for obtaining images and an output device 12b for displaying the images. The portable electronic device 12 may also be configured for communicating with other computing devices via the network 100.
- the apparatus 14 may include a non-transitory computer readable storage medium 14a (hereinafter “storage medium” ) , which stores image obtaining logic 144a, image analysis logic 144b and graphic user interface (hereinafter “GUI” ) logic 144c.
- storage medium stores image obtaining logic 144a, image analysis logic 144b and graphic user interface (hereinafter “GUI” ) logic 144c.
- the storage medium 14a may comprise random access memory (such as SRAM, DRAM, etc. ) , read only memory (ROM) , registers, and/or other forms of computing storage hardware.
- the image obtaining logic 144a, image analysis logic 144b and the GUI logic 144c define computer executable instructions.
- a processor 14b is coupled to the storage medium 14a, wherein the processor 14b is configured to, based on the computer executable instructions, for implementing a method 200 for determining perceived attractiveness of a facial image portion of a person or persons in depicted in a digital image 51 according to the present invention as described herein after with respect to FIGS. 2 to 4 and the flowchart of FIG. 5.
- FIG. 2 is a diagram illustrating an exemplary functional block diagram of a facial image portion pre-processing module 40 containing image obtaining logic 144a for obtaining a digital image 51 comprising a facial image portion 52 according to the present invention.
- the pre-processing module 40 may comprise a first pre-processing sub-module 40A for detecting the facial image portion 52 and a second pre-processing sub-module 40B for detecting one or more features which define the facial image portion 52.
- FIG. 3 is a diagram illustrating an exemplary functional block diagram of an attractiveness model module 42 containing image analysis logic 144b for analyzing positive and negative attributes of the facial image portion 52 of the person depicted in the digital image 51, generating an Attractiveness Score 57 and an image description 53 according to the present invention.
- the attractiveness model module 42 may comprise a first attractiveness sub-module 42A for generating an Attractiveness Score 57 indicative of a perceived attractiveness of the facial image portion 52, and a second attractiveness sub-module 42B for generating an image description 53.
- FIG. 4 is a diagram illustrating an exemplary functional block diagram of a visualization module 44 containing the GUI logic 144c for presenting an image description 53 according to the present invention.
- the visualization module 44 may comprise a first visualization sub-module 44A for presenting the Attractiveness Score 57, asecond visualization sub-module 44B for presenting the image description 53 as a heat map, and a third visualization sub-module 44C for presenting the image description 53 as alternative text 531.
- the facial image portion pre-processing module 40, the attractiveness model module 42 or the visualization module 44 may be implemented, in part or in whole, as software, hardware, or any combination thereof.
- the attractiveness model module 42 may be implemented, in part or in whole, as software running on one or more computing devices or computing systems, such as on a server computing system or a client computing system.
- the attractiveness model module 42 or at least a part thereof can be implemented as or within a mobile application (e.g. APP) , a program or an applet, or the like, running on a client computing system such as the portable electronic device 12 of FIG. 1.
- the computing system may be in communication with a content server configured to store an obtained digital image or a plurality of obtained digital images.
- Each of the modules 40, 42, 44 can be implemented using one or more computing devices or systems that include one or more servers, such as network servers or cloud servers.
- the processor 14b is configured to, based on the computer executable instructions, for implementing a method 200 for determining perceived attractiveness of a facial image portion of a person or persons in depicted in a digital image 51 according to the present invention as described herein after with respect to the flowchart of FIG. 5.
- steps 202, 204, 206, 208, 210, 212, 214 of the method 200 according to the present invention is described hereinafter with reference to FIG. 5 as individual processes for performing each step.
- Each process may also be described as a sub-routine, i.e. a sequence of program instructions that performs a corresponding step according to the method 200 according to the present invention.
- the processor 14b When the processor 14b is initiated, the processor 14b causes a first digital image 51 of at least a portion of a face of the subject to be obtained, e.g. via image obtaining logic 144a in step 202.
- the first digital image 51 may be a teeth image.
- the facial image portion 52 is a smile 521 defined by a combination of teeth and gum as shown in FIG. 6B, and the smile comprises positive and negative attributes.
- the processor 14b analyzes the facial image portion 52 using a learning machine trained to evaluate the features of interest.
- step 206 an Attractiveness Score 57 is generated for the facial image portion 52.
- the method 200 may comprise further generating an image description 53 comprising the facial image portion 52 in step 208 based on the Attractiveness Score 57, and presenting the image description 53 to a user for determining perceived attractiveness of the facial image portion 52 in step 210.
- presenting the image description 53 may comprise one of: displaying the image description 53 in the digital image 51 as alternative text, displaying the image description 53 in the digital image 51 as a heat map, providing the image description 53 for audible presentation to the user, and combinations thereof.
- an Attractiveness Score 57 of a facial image portion depicted in a digital image provided by an user (consumer) By generating an Attractiveness Score 57 of a facial image portion depicted in a digital image provided by an user (consumer) , further generating an image description 53 based on the Attractiveness Score and presenting the image description 53 to the consumer, users and/or consumers can obtain information related to the facial image portion 52 which impact perceived attractiveness of the facial image portion 52. It will be appreciated that the method 200 may also be adapted for application in image processing of other facial image portions such as for example, facial skin.
- the present invention also relates to a human machine user interface (hereinafter “user interface” ) for determining perceived attractiveness of a facial image portion 52 in a digital image 51.
- the user interface may be a graphical user interface on a portable electronic apparatus including a touch screen display/display with an input device and an image obtaining device 18.
- the user interface may comprise a first area of the touch screen display displaying a first digital image of at least a portion of a face of the subject comprising a facial image portion obtained from the image obtaining device 18 and a second digital image interposed on the first digital image, the second digital image having the at least a portion of a face of the subject, the displayed facial image portion and the displayed image description for the displayed facial image portion.
- the user interface may further comprise a second area of the touch screen display different from the first area, the second area displaying a selectable icon for receiving a user input, wherein an image of at least one product recommendation item to improve perceived attractiveness of the facial image portion is displayed on the touch screen display if the user activates the selectable icon.
- the method 200 for determining perceived attractiveness may be applied in various different applications including but limited to providing a product recommendation, providing personalized product use instructions to consumers, visualization of product efficacy, and to monitor progress in improvement in perceived attractiveness of a facial image portion after use of a recommended product.
- exemplary applications described hereinafter relate to oral features as a specific example of a facial image portion and such oral features include teeth, gum, and combinations thereof, it will be appreciated the method may be adapted for other facial features.
- FIG. 6A is a screen shot illustrating an exemplary graphical user interface 30 presenting an image description 53 of a facial image portion 52 of a person in the digital image 51 to a user for determining perceived attractiveness of the facial image portion 52 according to the present invention.
- the digital image 51 may comprise a facial image portion 52 which the processor 14b has been programmed to determine perceived attractiveness for and there the facial image portion 52 is detected by the processor 14b (hereinafter “detected facial image portion 52” ) by the pre-processing module 40.
- the facial image portion 52 may include one or oral features, one or more facial expressions, or combinations thereof. Oral features may include mouth, teeth, gum or any feature in the oral cavity. Facial expressions may include a smile.
- the image description 53 may comprise alternative text 531 displayed in the graphical user interface 30, a heat map 532 displayed on the digital image 51 that identifies at least one area (hereinafter “identified area” ) in the facial image portion 52 comprising the negative attributes of the facial image portion 52, or a combination of the alternate text 531 and the heat map 532.
- the alternative text 531 includes a description that indicates the impact of the identified area in the facial image portion 52 on the perceived attractiveness of the facial image portion 52.
- the heat map 532 may display parts of the teeth with different defects which require different corresponding oral care treatments.
- the heat map 532 may include one or more region of interest highlighted in the teeth image associated with the person depicted in the digital image 51.
- the selectable input screen object 54 may comprise a text label comprising a description of the feature of the selectable input screen object 54.
- the selectable input screen object 54 may comprise a text label describing directions for processing a request for additional information about the facial image portion 52, for example, the text label may comprise a description related to proceeding to a different user interface directed to a method for providing a product recommendation for improving perceived attractiveness.
- the whiteness of the identified areas on the teeth may be improved, and accordingly the description may be related to understanding a solution for improving whiteness of the identified areas on the teeth, and consequently improve a perceived attractiveness of the detected facial image portion 52.
- FIG. 6B is the digital image 51 of FIG. 6A with the heat map 532.
- the detected facial image portion 52 is a smile 521 defined by a combination of oral features, teeth and gum.
- the smile 521 comprises positive and negative attributes as described hereinafter. Specifically, at least a portion of the smile 521 is defined by a first oral feature 521A and a second oral feature 521B.
- the first oral feature 521A may be a first tooth and the second oral feature 521 B may be a second tooth located in a different part of an area of the facial image portion 52.
- the first oral feature 521A comprises a highlighted region of interest 533 of the heat map 532 highlighted in the teeth image thereby indicative of negative oral attribute (yellowness) .
- the second oral feature 521B does not comprise a highlighted region of interest of the heat map 532 highlighted in the teeth image thereby indicative of positive oral attribute (whiteness) .
- FIG. 7 is a schematic drawing illustrating an exemplary system architecture 80 configured for implementing the method 200 based on a convolutional neural network (CNN) model.
- FIGS. 9A and 9B are conceptual illustrations of exemplary filter visualizations in the CNN model of FIG. 8, showing features of interest depicted in one or more filtered feature maps according to the present invention.
- the CNN model is described as an example of a machine learning algorithm, specifically a deep learning algorithm, for implementing methods and systems according to the present invention.
- Deep learning algorithms are concerned with building much larger and more complex neural networks and, as described hereinafter, the present invention is directed to analysis by a model trained by very large datasets of labelled analog data, such as digital images. Therefore, other deep learning algorithms which may be used to implement methods according to the present invention may include, but is not limited to, Recurrent Neural Networks (RNNs) , Long Short-Term Memory Networks (LSTMs) , Stacked Auto-Encoders, Deep Boltzmann Machine (DBM) , Deep BeliefNetworks (DBN) .
- RNNs Recurrent Neural Networks
- LSTMs Long Short-Term Memory Networks
- DBM Deep Boltzmann Machine
- DBN Deep BeliefNetworks
- FIG. 7 illustrates the system architecture 80 of the CNN model, the CNN components which make up the CNN model, and exchanges between each CNN component for performing the method 200 according to the present invention.
- CNN models are able to extract the hierarchies of visual features through stack-admissible neural layers equipped by receptive fields implementing convolutional kernels recognizing primary visual features to complex visual features of the image’s components.
- each layer of the CNN model extracts rich information representing the original stimulus.
- the system architecture 80 of the CNN model comprises CNN components which are operably connected through CNN exchanges arranged to generate the Attractiveness Score 57, the details of the CNN components and its respective function are described in Table 1 below.
- each CNN exchange connecting each of the above CNN components is described in Table 2 below, and the sequence of the analyzing step 204 and generating step 206 is according to the direction ofCNN exchanges as shown in FIG. 7.
- analyzing may comprise filtering the digital image 51 in a first exchange 90 to obtain one or more filtered feature maps comprising features of interest associated with the facial image portion 52 and the features of interest are analysed.
- FIG. 8A shows a first filtered feature map X with positive attributes
- FIG. 8B shows a second filtered feature map Y with negative attributes.
- Positive attributes may comprise whiteness of teeth, pinkness of gums, smoothness of teeth surfaces or positive appearances of the teeth or gums.
- Negative attributes may comprise teeth stains, gum redness, swollen gums or the like.
- the Attractiveness Score 57 may be generated based on a first set of characteristics associated with a first feature of interest in a first filtered feature map (Layer Y) and a second set of characteristics associated with a second feature of interest in a second filtered feature map (Layer Y) .
- the first feature of interest may comprise a first plurality of oral features including gum and teeth located in an upper part of the oral cavity.
- the second feature of interest may comprise a second plurality of oral features including gum and teeth located in a lower part of the oral cavity.
- the method may further comprise generating an abnormality output 85 indicative of the second feature of interest comprising negative attributes which negatively impact a condition of the first feature of interest.
- FIGS. 9A, 9B and 9C is a series of process flow diagrams illustrating how the digital image 51 is obtained.
- FIG. 10 is a flow chart of a process 300 of obtaining the digital image 51 corresponding to the step 202.
- FIG. 9A An input image 50a of a face of a person is illustrated in FIG. 9A.
- the input image 50a may be taken by a user, for example, using the camera 18 of the portable electronic device 12.
- the input image 50a may also be further processed by machine learning and computer vision techniques to detect the person’s face and/or facial image portion automatically.
- the method 300 may comprise a face detection module that employs a Dlib face detection library to detect a face depicted in the input image 50a and draw a first detector box 55 bounding the detected face in the input image 50a.
- An example of how to apply a Dlib face detection library for finding facial landmarks for detecting faces in digital images may be found at the following published reference namely, D.E. King. Dlib-ml: A machine learning toolkit. J. Mach. Learning Research, 10: 1755–1758, 2009.
- FIG. 9B illustrates a step 302 of cropping the input image 50a using the detector box 55 of FIG. 9A to obtain an edited image 50b which comprises at least a portion of the face comprising the smile of the person as an example of a facial image portion 52 according to the present invention.
- the second pre-processing sub-module 40B may be a feature detection module configured to detect facial features such as teeth and/or gum which define the facial image portion 52 (smile) and draw a second detector box bounding the facial image portion 52.
- the edited image 50b may be further cropped using the second detector box to obtain a digital image 51 in step 306 and as shown in FIG. 9C.
- FIG. 11A illustrates a digital image 51 comprising a facial image portion 52 depicted in the digital image 51 before analysis.
- the digital image 51 may be cropped according to the method 300 of obtaining a digital image.
- FIG. 11B illustrates a second digital image 51b comprising an image description 53 presented visually as a heat map 532 overlaying the digital image 50.
- Displaying the image description 53 in the digital image 51 as a heat map 532 may comprise generating the heat map 532, wherein generating the heat map comprises overlaying a layer 120B on at least a portion of the digital image 52 comprising the facial image portion, wherein the layer 120B is a pixel map that identifies the at least one area comprising at least one of said analyzed negative attributes.
- the heat map 532 visualizes the positive attributes as a second layer 120A and visualizes the negative attributes as the layer 120B in the at least one area in the facial image portion 52 depicted in the digital image 51.
- FIG. 11C illustrates a third digital image 51c comprising the facial image portion 52 having the heat map 532 with the layer 120B only overlaid on the facial image portion 52 to depict negative attributes present in the facial image portion 52 only.
- the heat map 532 may be configured to have the layer 120A only overlaid on the facial image portion 52 to depict positive attributes present in the facial image portion 52 as shown in the third digital image 51c.
- the image description 53 may be presented as a color region 130 on the teeth to indicate the areas for improving perceived attractiveness of the smile (facial image portion 52) of the person.
- the image description may be presented in the form of colored boundary lines 131 that frame regions on the teeth to indicate the areas for improving perceived attractiveness of the smile of the person.
- FIG. 14 is a flow chart illustrating a method 400 for providing a product recommendation for improving perceived attractiveness of a facial image portion 52 depicted in a digital image 51.
- FIGS. 15A to 15E are screen shots, each illustrating an exemplary user interface cooperating with each other for providing a product recommendation according to the present invention.
- FIGS. 15A to 15E are described as a series of user interfaces which are provided in a sequential manner in response to a preceding user interface, it will be appreciated that the user interfaces of FIGS. 15A to 15E may be programmed in multiple ways to define an overall user interface for providing a product recommendation according to methods according to the present invention as described hereinbefore.
- all the user interfaces of FIGS. 15A to 15E define an exemplary user interface for providing a product recommendation for improving perceived attractiveness according to the present invention.
- the method comprises transmitting a digital image of at least one person in step 402, wherein the digital image comprises a facial image portion of the at least one person, wherein the facial image portion has both positive and negative attributes.
- step 404 an image description is received wherein the image description identifies at least one area in the facial image portion comprising at least one of the negative attributes analysed using the method 200.
- the image description is presented in step 406.
- step 408 a product recommendation for improving perceived attractiveness of the at least one of the analyzed positive and/or negative attributes is presented to a user.
- FIG. 15A is a screen shot of a user interface 160A for transmitting an input image 50a of a face of a person to the apparatus 14 of FIG. 1 according to step 402.
- the user interface 160A may display a first text object 161 wherein the first text object 161 may comprise any terms and/or phrases that can be used to describe information about a method for determining perceived attractiveness of a facial image portion of a person according to the present invention.
- the first text object 161 may include text depicted in the user interface 160A that relates to a method of comparing perceived attractiveness of a facial image portion 52 of a person relative to a population of people or a rhetorical question, i.e. “Do you have a winning smile? ” .
- the first text object 161 may include an advertising device used in advertisements for engaging consumers, e.g. a question related to the facial image portion 52.
- the input image 50a may be captured by a user and transmitting the input image 50a to the apparatus 14, for example, using a mobile phone as a selfie image through a selectable icon 162 displayed on the user interface 160.
- FIG. 15B is a screen shot of a user interface 160B that displays a second text object 163 to the user that indicates a status of the method 400.
- a user interface 160C displays alternative text 531 indicative of an Attractiveness Score 57 that is obtained in step 404.
- the image description 53 may further comprise alternative text 531a displayed in the user interface 160D wherein the alternative text 531a is associated with information about the Attractiveness Score.
- the alternate text 531a may be an Attractiveness Aggregate, such as for example an aggregate value calculated according to a mathematical formula based on the Attractiveness Score.
- the user interface 160C further displays a selectable input icon 164 for sending a request to present the image description 53 in the form of a heat map 532 in step 406 as shown in FIG. 15D.
- a user interface 160D is similar to the user interface 30 of FIG. 6A and shown to illustrate a sequence of the series of user interfaces that form part of the method described hereinbefore.
- the facial image portion 52 that is being determined is the smile of a person depicted in the digital image 51 and accordingly the product recommendation shown in a user interface 170 of FIG. 15E is an oral care product 172 for improving perceived attractiveness of the smile.
- the image description 53 may comprise alternative text 531 related to oral care information described hereinafter:
- the regions of interest are corners in the mouth which are hard to access with the toothbrush during brushing (hereinafter “brushing dead corners” )
- the present invention also relates to a method of demonstrating efficacy of a customized oral care regimen to a user, and this method may be used by dental professionals for performing remote oral care consultation for users in need of treatment but who are not able to travel to the dental clinics at which the dental professionals are located.
- FIG. 16 is a flow chart illustrating a method 500 of demonstrating efficacy of a customized oral care regimen in improving perceived attractiveness of one or more oral features of at least one person depicted in a digital image according to the present invention.
- the customized oral care regimen may comprise providing brushing instructions and/or providing an oral care product recommendation to be used in conjunction with the brushing instructions.
- the method 500 may be used to analyze weekly images of one or more oral features (e.g. teeth and/or gums) to visualize stain areas on the teeth.
- users may receive a personalized oral care consultation with product usage instructions at the supervised brushing and a picture of their teeth analyzed according to the method.
- Use of the method 500 may comprise several key benefits:
- the first digital image 60 shows areas to pay attention to-stained areas are labeled in bright pink (as shown in FIG. 17A) .
- the method 500 may comprise the steps of:
- FIG. 17A is a first digital image 60 of oral features of a subject determined for perceived attractiveness at Day 0, i.e. the start of a predetermined time period.
- FIG. 17B is a second digital image of oral features of the subject determined for perceived attractiveness at the end of the predetermined time period.
- the predetermined time period may be two weeks of use of a consumer product, such as an oral care product.
- the image descriptions in the first and second digital images identify at least one area in the oral features depicted in the digital images which comprise the analyzed negative attributes. Accordingly, to determine if there is a reduction in negative attributes of the oral features, the image descriptions may be analyzed for the pixel count to demonstrate if there has been a reduction in negative attributes. As shown in FIG.
- the image description for the oral care features in the first digital image 60 has 12714 pixels while the image description for the oral care features in the second digital image 61 of FIG. 17B has 7894 pixels.
- the reduction in the number of pixels correspond to a reduction in negative attributes of the oral care features.
- the negative attributes may be teeth stains.
- the facial image portion in the first digital image 60 may be treated with a reference product for comparative analysis between products.
- FIG. 18 is a flow chart illustrating a method 700 of demonstrating efficacy of a consumer product or an oral care regimen over a product usage time period.
- the method 700 may comprise the steps of:
- FIG. 19A is a first digital image 511 comprising at least a portion of a facial image portion 52 of a person.
- the digital image 511 is analyzed according to a method 200 according to the present invention at the start of a predetermined time period prior to performing the customized oral care instructions.
- the digital image 511 further comprises an image description that identifies a plurality of areas 533 in the facial image portion 52 comprising at least one analyzed negative attribute. Specifically, the image description is presented as a heat map 533.
- An Attractiveness Score 534 corresponding to a numerical value (for example 27) , a first numerical value 535 corresponding to a number of identified areas (for example 11 identified areas) , and a second numerical value 536 corresponding to a total number of pixels defining the identified areas (for example, 9742 pixels) are presented in the digital image 511 to provide information related to the perceived attractiveness of the facial image portion 52 to the user.
- FIG. 19B is a second digital image 512 illustrating the at least a portion of a facial image portion of the person after 1 week of use of the product recommendation item.
- FIG. 19C is a third digital image 513 illustrating the at least a portion of the facial image portion of the person after 2 weeks of use of the product recommendation item. Asummary of the improvement in the Attractiveness Score in FIG. 19B, FIG. 19C relative to FIG. 19A is described in Table 3 below.
- the reduction in the number of identified areas corresponding to negative attributes of the oral features demonstrate that use of the product recommendation reduces the negative attributes thereby improving the Attractiveness Score, and consequently a perceived attractiveness of the facial image portion.
- the method may comprise a step of repeating the determining step and comparing in step (iv) over a predetermined period of time.
- the predetermined period of time may be one week, preferably two weeks, more preferably three weeks.
- a technical effect is that it enables tracking of an improvement in the perceived attractiveness of the facial image portion over the predetermined period of time thereby allowing users to monitor progress and product usage accordingly.
- the perceived attractiveness of the facial image portion may include one or more oral features of at least one person depicted in a digital image.
- the one or more oral features may include but is not limited to, teeth, and the perceived attractiveness is teeth whitening.
- FIG. 20 is a flow chart illustrating a method 800 of tracking improvement in perceived attractiveness of the facial image portion of one or more oral features of at least one person depicted in a digital image according to the present invention.
- the oral feature may be teeth and the method 800 may be used to keep recording and tracking of a digital image of a person’s teeth and an analysis result so that the method could repeat the analysis based on the timing to show improvement/progress of their teeth attributes (e.g. teeth whitening) and/or attractiveness during a period of time.
- teeth attributes e.g. teeth whitening
- This function could be used to support product advertising claims including but not limited to “21 days teeth whitening challenge” , “whiten your teeth in a predetermined number of days” , “whitening longevity ( ⁇ ) ” , “long lasting (24h) whitening” , or any teeth whitening related attributes.
- the product may be an oral care product including but limited to toothpaste, white strip, mouth rinse or any form suitable for applying an oral care treatment.
- teeth attractiveness is described as a desired attribute related to perceived attractiveness in the method 800, it will be appreciated that the method 800 can be applied to other attributes including but not limited to healthy gums, teeth shine, or any other consumer relevant descriptions that may be used for the image description relative to oral feature attributes as described hereinafter in Table 5.
- the method 800 may comprise the following steps of:
- - obtaining 802 a digital image of at least a portion of a face of the subject to be obtained, e.g. via image obtaining logic 144a.
- the digital image 51 may be a teeth image.
- the time code may include but is not limited to date, time, user information or the like.
- - selecting 810 a time period or preset the time period based on a product advertising claim.
- step 812 Generating 812 an image description based on the analysis result in step 806.
- the image description may be presented in step 812.
- the analysis result in step 806 may comprise an attractiveness score, at least one area of the one or more oral features that consumers still need to improve, or other data generated by the Attractiveness model described hereinbefore.
- generating the image description in step 812 may comprise generating an image summary and an analysis result data summary from the analysis results from a database.
- the database may be stored on a server coupled to the system.
- the method 800 may comprise further presenting in step 814 a product recommendation including but not limited to, continued usage of a product (currently used by the consumer) for a predetermined number of days, adding a new product to the consumer’s oral care regimen for a better result, or any suitable treatment for improving teeth attractiveness.
- FIG. 21 is a screen shot of a digital image 51 of a plurality of persons wherein an image description 53 is presented for each detected facial image portion 52 for each of the plurality of persons.
- the detected facial image portion 52 is a smile 521 and an Attractiveness Score 57 is presented visually in the digital image 51.
- the Attractiveness Score 57 may include but is not limited to a numerical value indicative of perceived attractiveness of the smile 521 relative to a population of people.
- the Attractiveness Score 57 may be a smile score for the smile 521.
- the CNN model described hereinbefore may be trained and evaluated by a dataset of simulated teeth images.
- a training dataset of simulated teeth images ( “Simulated Images Dataset” ) may be built as described hereinafter for defining the Attractiveness Score.
- the training dataset design criteria may be based on eight different teeth attributes as described in Table 4 below, and different score levels ranging from 0%to 100%are assigned to each set of images belong to the same teeth attribute.
- Level1 Level2 Level3 Level4 Level5 1 Lightness 20% 40% 60% 80% 100% 2 Yellowness 20% 30% 40% 50% 60% 3 Opacity 20% 40% 60% 80% 100% 4 Facial Staining 20% 40% 60% 80% 100% 5 Shine 20% 40% 60% 80% 100% 6 Interproximal (IP) Staining 20% 40% 60% 80% 100% 7 Marginal Staining 20% 40% 60% 80% 100% 8 Gum Redness 20% 40% 60% 80% 100%
- each simulated image corresponding to a different score level There may be a set of simulated images for facial staining, each simulated image corresponding to a different score level.
- the preparation of the simulated images is based on an assumption that a simulated image corresponding to a lower score has a predetermined area of front teeth surfaces having facial staining (negative attribute) and a larger area of white front teeth surfaces (positive attribute) and will be deemed to be more attractive relative to another image having the same predetermined area of front teeth surfaces but corresponding to a higher score.
- the predetermined area of the facial staining is the same from low to high scores, but the color intensity of the facial staining in different images increases from low to high scores.
- a set of three different images may be shown side by side to consumers represent combinations of all eight attributes.
- the particular level of the eight attributes was determined by a balanced, designed, discrete choice (conjoint) randomization. So, within each choice set, up to all eight attributes’ levels differed among the three images according to the randomization. This was to determine what they really perceived as most attractive.
- the three images may be made of any combination of the set of attributes below, including facial staining, with a given level of each attribute represented in each set of teeth.
- An attractiveness model based on the training data may be obtained from the raw consumer choice data by estimating the part-worth utilities for the eight attributes’ main effects and limited interaction terms via hierarchical bayes (HB) estimation.
- the Attractiveness Score for any particular training image could then be calculated from the sum of the part-worth utilities across the chosen attribute levels.
- the Simulated Images Dataset can be modified in the same way based on knowing which skin attributes to define, e.g. pigmentation, or other skin attributes, and that can be built into the Attractiveness model and analysed accordingly. For example, if the facial image portion is skin, a Simulated Images Dataset may be generated by modifying skin images based on the dataset design criteria described hereinbefore for teeth and then applied to the Attractiveness model for determining attractiveness of skin.
- Simulated Images Dataset An advantage of the Simulated Images Dataset is that it is easy to define a level of measure for attributes that are relevant to the consumer and thereby gain a better and controllable measure of the attributes that is driving their perception of attractiveness.
- Use of simulated images provides an advantage of using data that is consumer relevant to generate the score, therefore the score is consumer relevant and is not a random result generated from random stock facial images.
- a predetermined population size of real people images may be gathered to build a training dataset based on the predetermined population of real people, and a discrete choice model may be used to estimate the attractiveness of the facial image portion.
- a process for building a training dataset may comprise the following steps:
- the training data set can be created for any system that can be broken down into bodily attributes and their levels.
- Discrete choice models may be used to describe the attributes.
- the discrete choice model is conjoint statistics which may be used to describe combinations of fixed (controlled) attributes.
- the discrete choice model may be MaxDiffanalysis which may be used to describe collections of non-fixed (uncontrolled) attribute images (e.g. a large set of clinical images) that have known scores for the attribute levels identified (e.g. clinical grading for staining, yellowness or any desired oral feature attribute) .
- the term “attractiveness” may have multiple words used for the image description that is displayed in the step (e) presenting 210 the image description 53 to a user according to the present invention.
- Table 5 below is a non-exhaustive list of consumer relevant descriptions that may be used for the image description is described below relative to the relevant facial image portion, specifically, oral feature attributes.
- An oral oral care based digital imaging computer-implemented method for determining perceived attractiveness of a facial image portion (52) of at least one person depicted in a digital image (51) comprising the steps of:
- the Attractiveness Score is generated as a probability value indicative of how appealing a facial image portion of a person depicted in a digital image is to a population of people based on positive and negative attributes of the facial image portion, preferably the probability value is determined by a model constructed by a machine learning system trained by a training dataset, wherein the training dataset comprises i) a plurality of simulated images of a facial image portion comprising positive and negative attributes; and (ii) an associated class definition based on positive and negative attributes.
- the facial image portion (52) is selected from the group consisting of: facial skin, one or more oral features, one or more facial expressions, and combinations thereof.
- facial image portion (52) comprises one or more oral features selected from the group consisting of: oral cavity soft tissue, gum, teeth, and combinations thereof.
- the first oral feature comprises a first set of characteristics indicative of positive cosmetic dental attributes for the facial image portion (52) , each positive cosmetic dental attribute is assigned a positive value indicative that the first oral feature is healthy; wherein the second oral feature comprises a second set of characteristics indicative of negative cosmetic dental attributes for the facial image portion (52) , wherein the first oral feature and the second oral feature are located in different parts of the at least one area in the facial image portion (52) .
- analyzing in step (b) comprises filtering the facial image portion (52) to obtain one or more filtered feature maps comprising a first feature of interest and a second feature of interest, each of the first and second features of interest is associated with the facial image portion (52) ; wherein the first feature of interest comprises a first set of characteristics indicative of positive attributes for the facial image portion (52) , and the second feature of interest comprises a second set of characteristics indicative of negative attributes for the facial image portion (52) , wherein the first feature of interest and the second feature of interest are located in different parts of the at least one area in the facial image portion.
- presenting the image description (53) comprises one of: displaying the image description (53) in the digital image (51) as alternative text (531) , displaying the image description (53) in the digital image (51) as a heat map (532) , providing the image description (53) for audible presentation to the user, and combinations thereof.
- displaying the image description (53) in the digital image (51) as a heat map (532) comprises generating the heat map (532) , wherein generating the heat map comprises overlaying a layer on at least a portion of the digital image comprising the facial image portion, wherein the layer is a pixel map that identifies the at least one area comprising at least one of said analyzed positive and/or negative attributes.
- a method (400) for presenting a product recommendation for improving perceived attractiveness of a facial image portion comprising:
- a digital image (51) of at least one person wherein the digital image comprises a facial image portion of the at least one person, wherein the facial image portion has both positive and negative attributes;
- a method of demonstrating efficacy of a customized oral care regimen in improving perceived attractiveness of one or more oral features of at least one person depicted in a digital image comprising:
- a method for demonstrating efficacy of a consumer product in improving perceived attractiveness of a facial image portion of at least one person depicted in a digital image comprising:
- step (iii) and step (iv) over a period of time for tracking improvement in the perceived attractiveness of the facial image portion; wherein the period of time is from one day to three days, preferably from three days to seven days, more preferably from seven days to fourteen days.
- a mobile application capable of being compiled to run on a client computing system for obtaining a digital image comprising at least one oral feature of at least one person, wherein the digital image includes a facial image portion of the at least one person, wherein said computing system is in communication with a content server configured to store the obtained digital image;
- an image processing device (14) in communication with the mobile application through a network (100) ; wherein said image processing device (14) comprises a processor (14b) configured to, based on computer-executable instructions stored in a memory (14a) to analyze the facial image portion, generate an Attractiveness Score indicative of the perceived attractiveness of the facial image portion based on the analyzed facial image portion in the obtained digital image; and further generating an image description that identifies at least one area in said facial image portion indicative of the Attractiveness Score;
- a display generating unit for generating a display to display the image description indicative of the Attractiveness Score.
- An oral care based digital imaging method for providing information to graphical user interfaces for improving perceived attractiveness of a facial image portion of at least one person depicted in a digital image, the oral care based digital imaging method comprising:
- a graphical user interface (30) on a portable electronic apparatus including a touch screen display or a display with an input device and an image obtaining device for obtaining a digital image comprising at least one oral feature of at least one person, wherein the digital image includes a facial image portion of the at least one person;
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Geometry (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Library & Information Science (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/094342 WO2021243640A1 (en) | 2020-06-04 | 2020-06-04 | Oral care based digital imaging systems and methods for determining perceived attractiveness of facial image portion |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4162447A1 true EP4162447A1 (de) | 2023-04-12 |
Family
ID=78831558
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20938571.5A Pending EP4162447A1 (de) | 2020-06-04 | 2020-06-04 | Auf mundpflege basierende digitale bildgebungssysteme und verfahren zur bestimmung des wahrgenommenen attraktivität eines gesichtsbildteils |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4162447A1 (de) |
CN (1) | CN115668279A (de) |
WO (1) | WO2021243640A1 (de) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023133368A1 (en) * | 2022-01-04 | 2023-07-13 | The Procter & Gamble Company | Oral care based digital imaging systems and methods for evaluating an oral care product |
WO2024075109A1 (en) * | 2022-10-05 | 2024-04-11 | Facetrom Limited | Attractiveness determination system and method |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6571003B1 (en) | 1999-06-14 | 2003-05-27 | The Procter & Gamble Company | Skin imaging and analysis systems and methods |
RU2008152155A (ru) | 2006-07-25 | 2010-08-27 | Дзе Проктер Энд Гэмбл Компани (US) | Способы и продукты для анализа десневых тканей |
US8885873B2 (en) * | 2011-09-09 | 2014-11-11 | Francis R. Palmer Iii Md Inc. | Systems and methods for using curvatures to analyze facial and body features |
US9478043B2 (en) * | 2014-01-29 | 2016-10-25 | Abdullaibrahim Abdulwaheed | Measuring teeth whiteness system and method |
US10405754B2 (en) | 2015-12-01 | 2019-09-10 | University Of South Florida | Standardized oral health assessment and scoring using digital imaging |
CN108701323B (zh) * | 2016-03-21 | 2023-11-10 | 宝洁公司 | 用于提供定制的产品推荐的系统和方法 |
US10547780B2 (en) * | 2018-05-14 | 2020-01-28 | Abdul Abdulwaheed | Body part color measurement detection and method |
CN110472489B (zh) * | 2019-07-05 | 2023-05-05 | 五邑大学 | 一种人脸美丽等级预测方法、装置及存储介质 |
CN111091040B (zh) * | 2019-10-15 | 2023-04-07 | 西北大学 | 基于全局轮廓和脸型结构分类的人脸吸引力数据处理方法 |
CN111191137A (zh) * | 2019-12-31 | 2020-05-22 | 广州皓醒湾科技有限公司 | 基于牙齿颜色确定刷牙推荐方案的方法及装置 |
-
2020
- 2020-06-04 EP EP20938571.5A patent/EP4162447A1/de active Pending
- 2020-06-04 CN CN202080101672.5A patent/CN115668279A/zh active Pending
- 2020-06-04 WO PCT/CN2020/094342 patent/WO2021243640A1/en unknown
Also Published As
Publication number | Publication date |
---|---|
WO2021243640A1 (en) | 2021-12-09 |
CN115668279A (zh) | 2023-01-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11055762B2 (en) | Systems and methods for providing customized product recommendations | |
US11416988B2 (en) | Apparatus and method for visualizing visually imperceivable cosmetic skin attributes | |
CN114502061A (zh) | 使用深度学习的基于图像的自动皮肤诊断 | |
JP6271708B2 (ja) | システム及びユーザ肌画像生成装置 | |
CA2692667C (en) | Method and apparatus for realistic simulation of wrinkle aging and de-aging | |
US11605243B2 (en) | Apparatus and method for determining cosmetic skin attributes | |
US11151421B2 (en) | Method for treating a surface | |
CN108694736B (zh) | 图像处理方法、装置、服务器及计算机存储介质 | |
WO2021243640A1 (en) | Oral care based digital imaging systems and methods for determining perceived attractiveness of facial image portion | |
Elhelw et al. | A gaze-based study for investigating the perception of visual realism in simulated scenes | |
US11978207B2 (en) | Oral care based digital imaging systems and methods for determining perceived attractiveness of a facial image portion | |
US20240104738A1 (en) | Oral care based digital imaging systems and methods for analyzing attributes of a facial image portion | |
Lu et al. | Volume composition and evaluation using eye-tracking data | |
Hecher et al. | A comparative perceptual study of soft-shadow algorithms | |
JP7520679B2 (ja) | いきいきとした顔の度合い推定方法、いきいきとした顔の度合い推定装置、及びいきいきとした顔の度合い推定プログラム | |
JP2009294958A (ja) | 顔画像の合成方法 | |
Taubert et al. | Shape-invariant encoding of dynamic primate facial expressions in human perception | |
JP2023087699A (ja) | 幸福実感の表れた顔の度合い推定方法、幸福実感の表れた顔の度合い推定装置、及び幸福実感の表れた顔の度合い推定プログラム | |
JP2023038871A (ja) | 特徴抽出方法及び特徴抽出システム | |
FR3135556A1 (fr) | Détection et visualisation de signes cutanés au moyen d’une carte thermique | |
KR20240009440A (ko) | 컴퓨터-기반 신체 부위 분석 방법들 및 시스템들 | |
Li | Analysis of Extended Multitask Learning Model on Facial Pain Evaluation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20221122 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230429 |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) |