EP4687658A1 - Automated skin condition evaluation - Google Patents
Automated skin condition evaluationInfo
- Publication number
- EP4687658A1 EP4687658A1 EP24781676.2A EP24781676A EP4687658A1 EP 4687658 A1 EP4687658 A1 EP 4687658A1 EP 24781676 A EP24781676 A EP 24781676A EP 4687658 A1 EP4687658 A1 EP 4687658A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- region
- image data
- interest
- control region
- skin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2576/00—Medical imaging apparatus involving image processing or analysis
- A61B2576/02—Medical imaging apparatus involving image processing or analysis specially adapted for a particular organ or body part
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
- A61B5/1075—Measuring physical dimensions, e.g. size of the entire body or parts thereof for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/442—Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/449—Nail evaluation, e.g. for nail disorder diagnosis
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30088—Skin; Dermal
Definitions
- Certain conditions that affect human skin and/or nails may increase the thickness of skin and/or nails.
- skin or nail psoriasis may cause hyperkeratosis (excessive thickness of skin tissue), as well as pitting or other visible features on skin or nails.
- hyperkeratosis excessive thickness of skin tissue
- pitting or other visible features on skin or nails may be difficult.
- treatment may be developed and administered to address these and other conditions, objectively evaluating the effect of the treatment on an afflicted region may be challenging.
- aspects of the present disclosure relate to automated skin condition evaluation.
- a region of interest within image data of a user is identified that is afflicted by a skin or nail condition.
- a control region that is not afflicted by the condition may similarly be identified, such that a set of chromophore concentrations is generated for the control region.
- an estimated thickness is generated for the region of interest.
- the amount of chromophores at the control region are at least approximately the same as the amount of chromophores at the region of interest, such that a difference in coloration between the control region and the region of interest is predominantly the result of a difference in thickness (since chromophore concentration would be expected to decrease with increasing skin thickness if the amount of chromophores are held constant).
- some skin conditions may alter the roughness or texture of the skin without significantly affecting the skin’s associated thickness, such that surface roughness may additionally or alternatively be determined for the region of interest.
- a skin/nail condition severity metric may be generated for the region of interest. An indication of such thickness, surface roughness, and/or condition severity metrics may ultimately be provided to a user, thereby enabling the objective determination as to whether a skin/nail condition is present. Additionally, or alternatively, it may thus be possible to objectively track the progression of a skin/nail condition over time.
- Figure 1 illustrates an overview of an example system with which automated skin condition evaluation may be performed according to aspects described herein.
- Figure 2A illustrates example image data of a hand of a user that may be evaluated according to the disclosed automated skin condition evaluation techniques.
- Figure 2B illustrates an example control region that is used for automated skin condition evaluation according to aspects described herein.
- Figure 2C illustrates an example region of interest that is evaluated based on the control region of Figure 2B according to aspects described herein.
- Figure 3 A illustrates an overview of an example method for generating an estimated thickness for a region of interest according to aspects described herein.
- Figure 3B illustrates an overview of another example method for generating an estimated thickness for a region of interest according to aspects described herein.
- Figure 4A illustrates an overview of an example method for determining a condition severity based on an estimated thickness and an estimated roughness according to aspects described herein.
- Figure 4B illustrates an overview of an example method for determining whether a skin condition is present according to aspects described herein.
- Figure 5 illustrates an example of a suitable operating environment in which one or more aspects of the present application may be implemented.
- Certain conditions that affect human skin and/or nails may increase the thickness of skin and/or nails. As another example, such conditions may cause pitting or other visible features on skin or nails. While treatment may be developed and administered to address these and other conditions, it may be difficult to objectively evaluate the severity /progression of such conditions, as well as the effect of treatment. For example, an individual’s skin and/or nails may be manually scored (e.g., by the individual or by a proctor), which may inadvertently introduce inconsistencies across individuals, or even across observations for the same individual, due to the subjectivity of manual scoring.
- image data is captured of a user (e g., including the user’s hands, feet, nails, arms, legs, or any of a variety of other skin and/or nail regions).
- a region of interest is identified (e.g., manually or automatically) within the image data, where the region of interest is afflicted by a skin or nail condition.
- a region that is not afflicted by the skin or nail condition is similarly identified within the image data (which is referred to herein as a “control region”).
- a set of chromophore concentrations is generated for the control region based on the image data for the control region.
- a model that generates the set of chromophore concentrations based on the image data for the control region uses a predetermined or otherwise assumed tissue (e.g., skin/nail) thickness for the control region.
- An estimated tissue thickness is then generated for the region of interest based on the set of chromophore concentrations generated for the control region.
- This estimate of tissue thickness for the control region may be generated by applying the aforementioned model in reverse.
- the chromophore concentrations of the control region are applicable to the region of interest, such that a difference in coloration between the control region and the region of interest is predominantly the result of a difference in thickness (as the chromophore concentration for the region of interest would be expected to decrease with increasing skin thickness, assuming the amount of chromophores is substantially similar).
- the resulting estimated thickness for the region of interest may thus at least be a relative measurement (e.g., relative to the control region), which may be used to identify a difference in thickness between the control region and the region of interest, as well as changes in thickness over time, among other examples.
- an indication as to an estimated thickness for a region of interest may be provided as a relative measurement as compared to a control region.
- a surface roughness is additionally or alternatively determined for a region of interest.
- some skin conditions e.g., prurigo nodularis
- Other conditions may alter both tissue thickness and roughness.
- evaluating surface roughness in addition to or as an alternative to thickness may enable the evaluation of a wider range of skin/nail conditions.
- a skin/nail condition severity metric may be generated.
- the condition severity metric is generated based on a combination of the thickness and the surface roughness for a region of interest. The combination may be generated based on a thickness weighting and a roughness weighting, such that the condition severity metric is a weighted average of the generated thickness and roughness metrics.
- the condition severity metric is generated using a machine learning model that is trained (e.g., using annotated training data and/or reinforcement learning) to generate a condition severity metric for a set of inputs that includes a determined thickness and/or surface roughness for a region of interest.
- the condition severity metric may be generated based on multiple thickness and/or surface roughness determinations (e g., for multiple regions of interest and/or for measurements for the same region of interest at multiple times).
- Generated thickness, surface roughness, and/or condition severity metrics may be used for any of a variety of subsequent processing according to aspects described herein. For instance, an indication of one or more such metrics may be provided to a user, thereby enabling the user to determine whether a skin/nail condition is present and/or to track the progression of the condition over time. As another example, such metrics may be generated for a population of users based on image data for each user that was acquired over time, such that the efficacy of a skin/nail treatment may be objectively quantified and evaluated across the population of users.
- aspects of the present disclosure may be performed using image data that was recently captured (e.g., contemporaneously with the disclosed processing) and/or may be performed retrospectively (e.g., based on image data that was previously captured), among other examples.
- a first set of thickness, surface roughness, and/or condition severity metrics may be generated (e.g., contemporaneously with when the image data was captured) and a second set of metrics may be generated for the image data at a later point in time, as may be the case when different or improved models are available with which to process the image data and generate one or more associated metrics.
- a set of chromophore concentrations may be generated for a control region and/or a region of interest according to any of a variety of techniques. It will be appreciated that any of a variety of wavelength bands may be used. For instance, spectral reconstruction or spectral super-resolution may be used to reconstruct any number of wavelength bands (e.g., between 400nm to 700nm at lOnm intervals from RGB image data that contains three spectral bands). Additional examples of such aspects are described in the following paper, which is hereby incorporated by reference in its entirety: Kaya, B., Can, Y. B., & Timofte, R. (2018). Towards Spectral Estimation from a Single RGB Image in the Wild, https://doi.org/10.48550/arxiv.1812.00805.
- Intensities in the respective wavelength bands may correspond to the respective concentrations of various chromophores (e.g., melanin, oxygenated hemoglobin, deoxyhemoglobin, and bilirubin), such that the processed wavelength bands are further processed (e.g., based on an absorption coefficient for each chromophore, which may be associated with a tissue thickness) to generate a set of chromophore concentrations accordingly.
- various chromophores e.g., melanin, oxygenated hemoglobin, deoxyhemoglobin, and bilirubin
- a set of wavelength band intensities may be multiplied by a color transformation matrix N ⁇ , thereby converting the set of wavelength band intensities into another color system or other set of wavelengths.
- Said other color system or other set of wavelengths may have any number m of dimensions.
- the resulting set of wavelength band intensities is referred to herein as a spectral vector.
- the resulting spectral vector is multiplied by an estimation matrix N 2 , thereby generating a vector of chromophore concentrations (also referred to herein as a set of chromophore concentrations) that corresponds to a chromophore space.
- the estimation matrix is determined using a computer model of human skin (e.g., comprising the stratum comeum, epidermis, and dermis).
- the computer model may be programmed with absorption coefficients corresponding to various chromophores, as may be obtained from literature.
- N 2 may be determined (and/or validated) using experimental data, among other examples.
- a method for determining whether skin in a region of interest is likely to exhibit abnormal thickness is provided.
- a set of wavelength band intensities represented by the vector C e.g., according to the RGB color system, IR 3
- a color transformation matrix e.g., a color transformation matrix for the control region is transformed according to a color transformation matrix, thereby yielding a spectral vector X for the control region in a spectral space (e.g., , having m dimensions that each correspond to a wavelength band).
- Methods for determining a suitable estimation matrix N2 are provided in the aforementioned Nishidate publication. While examples are described with respect to various matrices (which may thus imply linear transformations), it will be appreciated that such aspects are illustrative and, in other examples, nonlinear functions /(C) and g(X) are also possible.
- /(C) and g(X) may each be approximately invertible.
- N 2 is a square invertible matrix
- This expected set of wavelength band intensities C2 represents the wavelengths that we expect to observe from the region of interest if the skin tissue thickness at the region of interest is the same as that of the control region.
- a set of wavelength band intensities R for a region of interest may thus be compared according to the example equation below:
- is a distance metric (e g., the Euclidean distance between the two vectors R and C) and A > 1 is a user-defined tuning value. If the above equation is true for wavelength band intensities C of the control region and R of the region of interest, it may be determined that the region of interest exhibits an abnormal skin thickness.
- indicates the amount of error that may be introduced by digital processing of wavelength band intensity data. Such errors may result from the rounding that inevitably results from digitizing analog data wavelength data, and the compounding of such rounding errors through mathematical matrix operations. Such errors are unavoidably introduced by performing any digital mathematical operation on analog wavelength data.
- may be due to errors introduced by said digital processing rather than any true difference in the wavelength band intensities C and R.
- can no longer be explained by digital processing error and is therefore indicative of actual differences in coloration between the region of interest and the control region. If the above equation is true, then an estimated thickness ( ) of tissue at the region of interest as compared to the control region may be quantified according to the example equation below:
- any differences in coloration between the region of interest and the control region are assumed to be due to thicker skin at the region of interest compared to the control region. Therefore, the thickness t computed by the equation above is an estimate of how much thicker skin at the region of interest is compared to the skin at the control region.
- may be used to evaluate a distance between Y and Z, such that if the distance exceeds a tuning value X > 0, it is determined that the region of interest exhibits an abnormal skin thickness.
- an estimated thickness (7) of tissue at the region of interest as compared to the control region may be quantified according to the example equation below:
- any of the aforementioned estimates of thickness t may be further converted by some function h(/) that converts detected differences in coloration between the region of interest and the control region to units of length, such as millimeters or microns.
- the function h(7) may be experimentally determined and may, in some examples, comprise multiplying the thickness t by an experimentally determined scaling constant.
- a surface roughness may be determined for a control region and/or a region of interest according to any of a variety of techniques.
- the image data may be processed to determine a set of excerpts (e.g., cropped segments of the image having a predetermined width and/or according to a predetermined spacing, wherein such cropped segments may partially overlap or not overlap).
- the image data is converted to greyscale to reduce the impact of color variations as compared to variability in brightness/darkness.
- Each of the determined excerpts is processed to evaluate variations in pixel values along the excerpt.
- an excerpt that exhibits a higher degree of variability may thus be determined to have more surface roughness than an excerpt that exhibits a lower degree of variability, such that a surface roughness metric may be generated accordingly.
- Additional examples of such aspects are described by the following paper, which is hereby incorporated by reference in its entirety: Jafari, A., Fazayeli, A., & Zarezadeh, M. (2014). Estimation of orange skin thickness based on visual texture coarseness. Biosystems Engineering, 117, 73-82. https://d0i.0rg/l 0.1016/j .biosystemseng.2013.08.010.
- FIG. 1 illustrates an overview of an example system 100 with which automated skin condition evaluation may be performed according to aspects described herein.
- system 100 comprises data processing platform 102, computing device 104, and network 106.
- data processing platform 102 and computing device 104 communicate via network 106.
- network 106 may comprise a local area network, a wireless network, or the Internet, or any combination thereof, among other examples.
- data processing platform 102 includes image preprocessor 108, region identifier 110, chromophore processor 112, thickness evaluator 114, roughness determiner 115, and condition determination engine 116.
- image preprocessor 108 obtains image data from computing device 104, which is preprocessed according to aspects described herein. It will be appreciated that image preprocessor 108 may obtain image data from any of a variety of additional or alternative sources, as may be the case when previously stored image data is processed. In examples, image preprocessor 108 generates image data corresponding to one or more wavelength bands (e.g., a red, green, and blue channel or corresponding to any of a variety of wavelength ranges). As another example, image preprocessor 108 generates a greyscale representation of the image data, as may be used by roughness determiner 115.
- wavelength bands e.g., a red, green, and blue channel or corresponding to any of a variety of wavelength ranges.
- image preprocessor 108 generates a greyscale representation of the image data, as may be used by roughness determiner 115.
- image preprocessor 108 adjusts the brightness and/or contrast of the image data to account for the lighting of the environment in which the image data was obtained (e.g., by image capture device 118).
- image preprocessor 108 generates a spectral vector for a given set of wavelength band intensities, for example based on a color transformation matrix Ni according to aspects described herein. It will thus be appreciated that any of a variety of preprocessing operations may be performed by image preprocessor 108.
- Region identifier 110 processes the image data to identify a region of interest and/or a control region.
- region identifier 110 identifies a region based on a user indication received from computing device 104.
- image processing application 120 may enable a user to identify one or more regions of interest and/or control regions.
- region identifier 110 automatically identifies a control region that includes skin and/or nails that are “normal” (e.g., unafflicted by a condition for which the processing is being performed).
- the control region is determined based on a region of interest (e.g., as may be automatically or manually determined), as may be the case when a control region is identified at a similar part of the user’s body as the region of interest.
- a region of skin or a nail that is relatively smooth may be identified as the control region.
- image processing application 120 may perform such aspects to extract identified regions and thus reduce the amount of data that is transmitted from computing device 104 to data processing platform 102. It will also be appreciated the order in which image preprocessor 108 and region identifier 1 10 operates on image data received from the computing device 104 may be interchangeable, according to different embodiments.
- the region identifier 110 may act on image data received from the computing device 104 first, and then the image preprocessor 108 may act on image data output from the region identifier 110.
- the image preprocessor 108 may act on image data received from the computing device 104 first, and then the region identifier 110 may act on image data output from the image preprocessor 108.
- Data processing platform 102 is further illustrated as comprising chromophore processor 112, which processes image data for a control region (e.g., as was identified by region identifier 110 and/or preprocessed by image preprocessor 108) to generate a set of chromophore concentrations according to aspects described herein. As noted above, any of a variety of techniques may be used to generate the set of chromophore concentrations.
- mathematical transformations / e.g., a color transformation matrix, to the extent not already applied by image preprocessor 108 and N 2 (e.g., an estimation matrix) may be used to evaluate functions /(C) and g(X) described above, thereby generating a set of chromophores based on a set of wavelength bands for given image data (e.g., corresponding to control region and a region of interest).
- Example chromophores include, but are not limited to, melanin, oxygenated hemoglobin, deoxyhemoglobin, and bilirubin. It will be appreciated that fewer, additional, or alternative chromophores may be used in other examples.
- Thickness evaluator 114 processes a set of chromophore concentrations to generate an estimated thickness corresponding to a region of interest (e.g., as may have been identified by region identifier 1 10).
- the set of chromophore concentrations may correspond to a control region and may have been generated by chromophore processor 112.
- one or more evaluations according to the example equations described above may be performed. For instance, thickness evaluator 114 evaluates a distance between a set of wavelength band intensities for a control region C and a region of interest R is compared to a distance between C and an expected set of wav elength band intensities C 2 according to tuning value (e.g.,
- thickness evaluator 114 performs processing according to the above-described example equations
- a set of chromophores may be determined for a given region of interest, as may be the case when a data store includes known or expected chromophore concentrations for a given demographic and/or skin/nail region, among other examples.
- a control region need not be used in some examples.
- a set of wavelength band intensities Co that is derived from expected chromophore concentrations for a given demographic and/or skin/nail region (e.g., as experimentally determined or observed from a population of other users) may be substituted in place of the set of wavelength band intensities C from a control region in the aforementioned calculations and equations.
- thickness evaluator 114 processes a pixel of the image data corresponding to the region of interest based on the set of chromophore concentrations to generate a corresponding estimate of thickness t or h(z), as previously explained.
- multiple pixels within the region of interest are processed to generate an estimate of thickness for each pixel, thereby forming a thickness map that indicates a thickness gradient for tissue within the region of interest.
- a single estimated thickness (e.g., relative to the control region) may be generated for multiple pixels within the region of interest, or even for the entire region of interest as a whole.
- Roughness determiner 115 processes a region of interest to generate a surface roughness metric associated therewith.
- a region of interest e.g., as may have been generated by image preprocessor 108
- image preprocessor 108 is processed to generate one or more strips or excerpts therein, such that brightness/darkness variations within a strip or excerpt may be used to generate the surface roughness metric accordingly (e.g., where higher variability is indicative of increased surface roughness).
- Data processing platform 102 further comprises condition determination engine 116, which processes an estimated thickness (or a thickness map) generated by thickness evaluator 114 and/or a surface roughness generated by roughness determiner 115 to generate a condition severity metric.
- the severity metric may comprise a weighted average of the estimated thickness and the surface roughness.
- a machine learning model is used to generate a classification based on the estimated thickness and/or surface roughness accordingly.
- condition determination engine 116 evaluates a thickness map and/or a surface roughness corresponding to a region of interest to determine a condition and a severity associated therewith.
- Data processing platform 102 may provide an indication of a determined thickness, surface roughness, and/or condition severity metric to computing device 104.
- generated metrics and/or associated image data are stored in a data store for subsequent processing. For instance, historical metrics may be maintained for one or more individuals, thereby enabling longitudinal analysis of the progression of an individual’s skin/nail condition. It will therefore be appreciated that the data processing techniques described herein may be used in any of a variety of contexts.
- computing device 104 comprises image capture device 118 and image processing application 120.
- computing device 104 is a mobile computing device, a tablet computing device, or a laptop computing device.
- Image capture device 118 is usable to capture image data, including, but not limited to, an image and/or one or more frames of video.
- image processing application 120 obtains image data from image capture device 118, which is processed (e.g., by data processing platform 102) according to aspects described herein.
- a user operates image processing application 120 to capture at least a portion of an individual’s body using image capture device 118.
- the individual is the user (e.g., as may be the case if the user is using image processing application 120 for selfdiagnosis).
- another individual may operate computing device 104 (e.g., as may be the case in a clinical trial).
- the captured image data is provided to data processing platform 102, such that one or more generated metrics (e.g., as may be generated by thickness evaluator 114, roughness determiner 115, and/or condition determination engine 116) may be received in response.
- data processing platform 102 provides an indication as to an identified skin condition and an associated severity metric. The indication may be presented to the user of computing device 104 by image processing application 120 accordingly.
- computing device 104 is illustrated as comprising image capture device 118, it will be appreciated that, in other examples, image capture device 118 may be another device, such as a camera, which communicates with computing device 104 and/or data processing platform 102 (e.g., via a wired connection, a wireless connection, and/or network 106). Additionally, it will be appreciated that while system 100 is illustrated as comprising one data processing platform 102 and one computing device 104, any number of such elements may be used in other examples. For example, image data from multiple devices may be processed by a single data processing platform or, as another example, different computing devices may each have an associated data processing platform.
- computing device 104 may implement aspects associated with data processing platform 102, such that image data obtained by image capture device 118 is processed by image processing application 120 in addition to or as an alternative to processing performed by data processing platform 102.
- image processing application 120 may offer improved user privacy and may reduce the amount of data that is transferred to data processing platform 102, among other examples.
- Figure 2A illustrates example image data 200 of a hand of a user that may be evaluated according to the disclosed automated skin condition evaluation techniques.
- image data 200 may be captured by image capture device 118 as a result of a user operating image processing application 120 of computing device 104 in Figure 1. While image data 200 depicts the hand of an individual, it will be appreciated that any of a variety of other regions of a user’s body may be evaluated in other examples.
- Image data 200 may be processed according to aspects described herein to identify control region 202 and region of interest 204, as illustrated by Figures 2B and 2C.
- regions 202 and 204 may be automatically identified by a region identifier, such as region identifier 110 discussed above with respect to Figure 1.
- user input may be received (e.g., by an image processing application, such as image processing application 120) that includes an indication of a boundary corresponding to control region 202 and/or a boundary corresponding to region of interest 204.
- user input may indicate a general region of image data 200, such that the general region is processed by a region identifier to identify a more specific control region and/or region of interest therein.
- the nail in control region 202 of Figure 2B is processed according to aspects described herein to generate a set of chromophore concentrations, with which a thickness of the nail in region of interest 204 in Figure 2C is generated.
- a thickness map is generated, which may indicate different (e g., increased) thicknesses corresponding to regions 206 illustrated in Figure 2C accordingly.
- Figure 3A illustrates an overview of an example method 300 for generating an estimated thickness for a region of interest according to aspects described herein.
- aspects of method 300 are performed by a data processing platform, such as data processing platform 102 in Figure 1.
- Method 300 begins at operation 302, where RGB image data of a dermatological region is obtained.
- the image data is obtained from an image capture device of a computing device, such as image capture device 118 of computing device 104 in Figure 1.
- the image data is obtained from a data store (e.g., of a data processing platform). It will therefore be appreciated that image data may be obtained from any of a variety of sources.
- the obtained image data is decomposed into discrete wavelength bands (e.g., between 400nm and 700nm, as illustrated). Aspects of operation 304 may be performed by an image preprocessor, such as image preprocessor 108 in Figure 1.
- the discrete wavelength bands include a spectral vector that correspond to a spectral space, as may be generated based on a color transformation matrix (e.g., according to function /(X) and color transformation matrix Ni described above).
- a color transformation matrix e.g., according to function /(X) and color transformation matrix Ni described above.
- Method 300 progresses to operation 308, where image data corresponding to the extracted control region(s) is processed to generate a set of chromophore concentrations.
- a model including a set of coefficients corresponding to a predetermined thickness may be used to generate the set of chromophore concentrations in some examples.
- Such aspects may be performed by a chromophore processor (e.g., chromophore processor 112 in Figure 1) in some examples.
- a chromophore processor e.g., chromophore processor 112 in Figure 1
- an estimation matrix N2 is used to transform a spectral vector and thus generate the set of chromophores (e.g., according to function g(X) described above).
- one or more pixels in the region(s) of interest that were extracted at operation 306 are processed at operation 310 to generate a corresponding estimated thickness (e.g., the estimated thickness t or h(/), as may be generated by a thickness evaluator, such as thickness evaluator 114 in Figure 1).
- a thickness evaluator such as thickness evaluator 114 in Figure 1.
- the set of chromophore concentrations that was generated at operation 308 may be used to determine the estimated thickness, such that the amount of chromophores is assumed to be substantially similar while the thickness is permitted to vary, as the chromophore concentration may decrease with increasing skin thickness (thereby solving the inverse problem as was processed at operation 308).
- one or more comparisons are performed at operation 308 (e.g., based on a color transformation matrix and/or an estimation matrix) to determine whether the region of interest exhibits an abnormal skin thickness, as were discussed above with respect to thickness evaluator 1 14 and the associated example equations.
- a thickness map is generated based on the estimated thicknesses (e.g., t or h(/)) that were generated at operation 310. For instance, multiple pixels or other subparts of a region of interest may be processed to generate a corresponding estimated thickness, such that the thickness map generated at operation 312 includes each of the thickness estimates.
- operation 314 Flow progresses to operation 314, where gradient information is extracted from the thickness map and used to determine a condition for the region of interest accordingly.
- aspects of operation 314 are performed by a condition determination engine, such as condition determination engine 116 in Figure 1.
- operation 314 may determine an associated condition and/or a condition severity metric, among other examples.
- operation 314 comprises providing an indication of the determined condition and/or severity (e.g., to a computing device, such as computing device 104).
- operation 314 comprises storing the generated metrics and/or associated data in a data store for subsequent processing.
- Method 300 terminates at operation 314.
- Figure 3B illustrates an overview of another example method 350 for generating an estimated thickness for a region of interest according to aspects described herein.
- aspects of method 350 are performed by a data processing platform, such as data processing platform 102 in Figure 1.
- Method 350 begins at operation 352, where image data is obtained.
- image data may be obtained from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 in Figure 1), among other examples.
- the image data comprises at least a part of a user’s body, such as one or more regions that are afflicted by a skin/nail condition.
- aspects of operation 352 may be similar to those discussed above with respect to operation 302 of method 300 in Figure 3A and are therefore not necessarily redescribed in detail.
- the image data may be decomposed into its constituent red, green, and blue channels.
- image data corresponding to one or more wavelength bands is generated.
- Aspects of operation 354 may be performed by an image preprocessor, such as image preprocessor 108 in Figure 1.
- Aspects of operation 354 may be similar to those discussed above with respect to operation 304 of method 300 in Figure 3A and are therefore not necessarily redescribed in detail.
- Operation 354 is illustrated using a dashed box to indicate that, in some examples, operation 354 may be omitted.
- the image data obtained at operation 352 may not need to be preprocessed, as may be the case when the image data is captured in controlled or consistent conditions or image data corresponding to one or more wavelength bands is already provided, among other examples.
- a control region and a region of interest are determined. Aspects of operation 356 may be similar to those discussed above with respect to operation 306 of method 300 in Figure 3A and are therefore not necessarily redescribed in detail. For instance, the regions may be automatically identified or may be determined based on a received user indication (e.g., corresponding to user input at an application, such as image processing application 120), among other examples. In some instances, aspects of operation 356 are performed by a region identifier, such as region identifier 110 in Figure 1.
- operation 358 Flow progresses to operation 358, where a set of chromophores are generated for the control region that was determined at operation 356.
- Aspects of operation 358 may be similar to those discussed above with respect to operation 308 of method 300 in Figure 3A and are therefore not necessarily redescribed in detail.
- aspects of operation 358 may be performed by a chromophore processor, such as chromophore processor 112 in Figure 1.
- Operation 358 may comprise determining a set of chromophore concentrations that contribute to or otherwise explain a coloration depicted within image data corresponding to the control region according to any of a variety of techniques as described above.
- an estimated thickness is generated for image data corresponding to a region of interest based on the set of chromophore concentrations that was generated at operation 358. Aspects of operation 360 may be similar to those discussed above with respect to operation 310 and/or 312 of method 300 in Figure 3A and are therefore not necessarily redescribed in detail.
- multiple thickness estimates are generated (e.g., for multiple pixels or other subparts within the region of interest).
- a thickness map including the thickness estimates may be generated accordingly, thereby indicating a thickness gradient within the region of interest.
- the resulting estimate includes an indication as to whether the region of interest exhibits an abnormal skin thickness, which may thus be a binary indication.
- an indication of the estimated thickness(es) for the region of interest is provided.
- the indication may be provided to a computing device, such as computing device 104 in Figure 1.
- an indication is provided as to whether the region of interest exhibits an abnormal skin thickness.
- the indication is stored in a data store for subsequent evaluation, as may be the case when the progression of a corresponding condition is analyzed over time.
- the indication may be provided for subsequent processing to generate a condition severity metric according to aspects described herein, as may be generated by a condition determination engine (e g., condition determination engine 116 in Figure 1) performing aspects of method 400 in Figure 4A, which is discussed in greater detail below.
- Method 350 terminates at operation 362.
- the indication is provided for subsequent processing to determine whether a skin condition is present according to aspects described herein, as may be generated by performing aspects of method 450 in Figure 4B.
- Figure 4A illustrates an overview of an example method 400 for determining a condition severity based on an estimated thickness and an estimated roughness according to aspects described herein.
- aspects of method 400 are performed by a data processing platform, such as data processing platform 102 in Figure 1.
- Method 400 begins at operation 402, where image data is obtained.
- image data may be obtained from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 in Figure 1), among other examples.
- the image data comprises at least a part of a user’s body, such as one or more regions that are afflicted by a skin/nail condition.
- aspects of operation 402 may be similar to those discussed above with respect to operations 302 and 352 of methods 300 and 350, respectively, and are therefore not necessarily redescribed in detail.
- an estimated thickness is generated for a region of interest of the image data that was obtained at operation 402.
- Operation 404 may comprise performing aspects of method 300 and/or 350 discussed above with respect to Figures 3A and 3B, respectively.
- Aspects of operation 406 may be performed by a roughness determiner, such as roughness determiner 115 in Figure 1.
- a roughness determiner such as roughness determiner 115 in Figure 1.
- any of a variety of techniques may be used to generate the surface roughness metric.
- a greyscale representation of the region of interest e.g., as may have been generated by an image preprocessor, such as image preprocessor 108 is processed at operation 406 to generate one or more strips therein, such that brightness/darkness variations therein may be used to generate the surface roughness metric accordingly.
- Operation 406 is illustrated using a dashed box to indicate that, in some examples, operation 406 may be omitted.
- a condition severity metric may be generated based on an estimated thickness and an estimated surface roughness (e.g., in instances where operations 404 and 406 are performed) or based on an estimated thickness (e.g., in instances where operation 406 is omitted), among other examples.
- a condition severity corresponding to the region of interest is determined based on the estimated thickness that was generated at operation 404 and, optionally, the estimated surface roughness that was generated at operation 406.
- Aspects operation 408 may be performed by a condition determination engine, such as condition determination engine 116 in Figure 1. Aspects of operation 408 may be similar to those discussed above with respect to operations 314 of Figure 3A and are therefore not necessarily redescribed below in detail.
- the severity metric may comprise a weighted average of the estimated thickness and the surface roughness.
- a machine learning model is used to generate a classification based on the estimated thickness and/or surface roughness accordingly.
- any of a variety of techniques may be used to generate a condition severity metric based on an estimated thickness and/or a surface roughness according to aspects described herein.
- Figure 4B illustrates an overview of an example method 450 for determining whether a skin condition is present according to aspects described herein.
- aspects of method 400 are performed by a data processing platform, such as data processing platform 102 in Figure 1.
- Method 450 begins at operation 452, where image data is obtained.
- image data may be obtained from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 in Figure 1), among other examples.
- the image data comprises at least a part of a user’s body, such as one or more regions that are afflicted by a skin/nail condition.
- aspects of operation 452 may be similar to those discussed above with respect to operations 302, 352, and/or 402 of methods 300, 350, and 400, respectively, and are therefore not necessarily redescribed in detail.
- Aspects of operation 406 may be performed by a roughness determiner, such as roughness determiner 115 in Figure 1.
- a roughness determiner such as roughness determiner 115 in Figure 1.
- any of a variety of techniques may be used to generate the surface roughness metric.
- a greyscale representation of the region of interest e.g., as may have been generated by an image preprocessor, such as image preprocessor 108 is processed at operation 406 to generate one or more strips therein, such that brightness/darkness variations therein may be used to generate the surface roughness metric accordingly.
- determination 458 it is determined whether the skin thickness is abnormal. For example, the determination may comprise determining whether processing that was performed at operation 454 yielded a true evaluation processing result. If it is determined that the skin thickness is not abnormal, flow branches “NO” to determination 460, where it is determined whether the skin texture is abnormal. If it is determined that the skin texture is not abnormal, flow branches “NO” to operation 462, where an indication is provided (e.g., to a computing device, such as computing device 104 in Figure 1) that a skin condition is likely not present. It will be appreciated that any of a variety of additional or alternative operations may be performed, for example to store the indication in association with the image data, thereby enabling historical or retrospective analysis of an individual’s skin. Method 450 terminates at operation 462.
- method 450 may yield an indeterminate result in instances where the skin thickness is not abnormal (e.g., as a result of branching “NO” at determination 458) and the skin exhibits an abnormal texture (e.g., branching “YES” at determination 460). Similar to operation 462, any of a variety of additional or alternative processing may be performed at operation 464. Method 450 terminates at operation 464.
- an indication e.g., to a computing device, such as computing device 104 in Figure 1.
- Method 450 terminates at operation 466.
- Figure 5 illustrates an example of a suitable operating environment 500 in which one or more of the present embodiments may be implemented.
- This is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality.
- Other well-known computing systems, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics such as smart phones, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
- operating environment 500 typically may include at least one processing unit 502 and memory 504.
- memory 504 storing, among other things, APIs, programs, etc. and/or other components or instructions to implement or perform the system and methods disclosed herein, etc.
- memory 504 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two.
- This most basic configuration is illustrated in Figure 5 by dashed line 506.
- environment 500 may also include storage devices (removable, 508, and/or nonremovable, 510) including, but not limited to, magnetic or optical disks or tape.
- environment 500 may also have input device(s) 514 such as a keyboard, mouse, pen, voice input, etc. and/or output device(s) 516 such as a display, speakers, printer, etc. Also included in the environment may be one or more communication connections, 512, such as LAN, WAN, point to point, etc.
- input device(s) 514 such as a keyboard, mouse, pen, voice input, etc.
- output device(s) 516 such as a display, speakers, printer, etc.
- Also included in the environment may be one or more communication connections, 512, such as LAN, WAN, point to point, etc.
- Operating environment 500 may include at least some form of computer readable media.
- the computer readable media may be any available media that can be accessed by processing unit 502 or other devices comprising the operating environment.
- the computer readable media may include computer storage media and communication media.
- the computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data.
- the computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium, which can be used to store the desired information.
- the computer storage media may not include communication media.
- the communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- modulated data signal may mean a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
- the communication media may include a wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
- the operating environment 500 may be a single computer operating in a networked environment using logical connections to one or more remote computers.
- the remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above as well as others not so mentioned.
- the logical connections may include any method supported by available communications media.
- Such networking environments are commonplace in offices, enterprise- wide computer networks, intranets and the Internet.
- program modules may be stored in the system memory 504. While executing on the processing unit 502, program modules (e.g., applications, Input/Output (I/O) management, and other utilities) may perform processes including, but not limited to, one or more of the stages of the operational methods described herein such as the methods illustrated in Figures 2, 3A-3B, or 4A-4B, for example.
- program modules e.g., applications, Input/Output (I/O) management, and other utilities
- I/O Input/Output
- examples of the invention may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors.
- examples of the invention may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in Figure 5 may be integrated onto a single integrated circuit.
- SOC system-on-a-chip
- Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or "burned") onto the chip substrate as a single integrated circuit.
- the functionality described herein may be operated via application-specific logic integrated with other components of the operating environment 500 on the single integrated circuit (chip).
- Examples of the present disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies.
- examples of the invention may be practiced within a general purpose computer or in any other circuits or systems.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Geometry (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202363493468P | 2023-03-31 | 2023-03-31 | |
| PCT/US2024/021294 WO2024206212A1 (en) | 2023-03-31 | 2024-03-25 | Automated skin condition evaluation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4687658A1 true EP4687658A1 (en) | 2026-02-11 |
Family
ID=92907396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP24781676.2A Pending EP4687658A1 (en) | 2023-03-31 | 2024-03-25 | Automated skin condition evaluation |
Country Status (5)
| Country | Link |
|---|---|
| EP (1) | EP4687658A1 (en) |
| JP (1) | JP2026507886A (en) |
| CN (1) | CN120916691A (en) |
| AU (1) | AU2024248521A1 (en) |
| WO (1) | WO2024206212A1 (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0217570D0 (en) * | 2002-07-30 | 2002-09-11 | Univ Birmingham | Method and apparatus for quantifying material or object properties |
| ATE498355T1 (en) * | 2007-09-20 | 2011-03-15 | Biocompatibles Uk Ltd | DEVICE AND METHOD FOR MEASURING COLLAGEN THICKNESS |
| US10575789B2 (en) * | 2014-12-05 | 2020-03-03 | Ricoh Co., Ltd. | Random forest based erythema grading for psoriasis |
-
2024
- 2024-03-25 AU AU2024248521A patent/AU2024248521A1/en active Pending
- 2024-03-25 JP JP2025551980A patent/JP2026507886A/en active Pending
- 2024-03-25 WO PCT/US2024/021294 patent/WO2024206212A1/en not_active Ceased
- 2024-03-25 EP EP24781676.2A patent/EP4687658A1/en active Pending
- 2024-03-25 CN CN202480022552.4A patent/CN120916691A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| AU2024248521A1 (en) | 2025-10-09 |
| WO2024206212A1 (en) | 2024-10-03 |
| JP2026507886A (en) | 2026-03-06 |
| CN120916691A (en) | 2025-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102839023B1 (en) | Device and method for visualizing cosmetic skin properties | |
| US11605243B2 (en) | Apparatus and method for determining cosmetic skin attributes | |
| US8498460B2 (en) | Reflectance imaging and analysis for evaluating tissue pigmentation | |
| US10325146B2 (en) | Hierarchical differential image filters for skin analysis | |
| US20140016832A1 (en) | Method and an apparatus for determining vein patterns from a colour image | |
| US10282868B2 (en) | Method and system for generating accurate graphical chromophore maps | |
| Acha et al. | Segmentation and classification of burn images by color and texture information | |
| Jung et al. | Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution | |
| Kong et al. | Gender classification based on spatio-frequency feature fusion of OCT fingerprint images in the IoT environment | |
| Bidokh et al. | Enhancing Wireless Capsule Endoscopy images from intense illumination specular reflections using the homomorphic filter | |
| EP4687658A1 (en) | Automated skin condition evaluation | |
| Jia et al. | Visualizing veins from color images under varying illuminations for medical applications | |
| JP6183030B2 (en) | Skin pigment concentration measurement method | |
| JP7725725B2 (en) | Method and apparatus for analyzing images of a subject's body hair and skin | |
| KR102450422B1 (en) | Method and apparatus for acquiring skin analysis information using multi-wavelength | |
| JP6566395B2 (en) | Endoscope image processing apparatus, endoscope image processing method, and endoscope image processing program | |
| Vyas et al. | Hyperspectral signature analysis of skin parameters | |
| Prasanna et al. | A Two-Stage Deep Learning Approach with DoubleU-Net for Enhanced Early Skin Cancer Detection in Smart Healthcare. | |
| Obukhova et al. | Endoscopic Image Processing Method for Virtual Chromoendoscopy with Controlled Color Transform and Local Contrast Accuracy | |
| US20260051059A1 (en) | Artificial intelligence(ai)-based systems and methods for generating and evaluating reconstructed multi-spectral images depicting skin | |
| Smorawa et al. | Comparison of global methods for the image contrast enhancement based on biometric images | |
| US20250087016A1 (en) | Adaptable Camera-based Contactless SpO2 Detection | |
| US20240398097A1 (en) | Method and system for determining cosmetic skin attributes based on disorder value | |
| Khan et al. | Learning based contrast enhancement evaluation using cartoon texture decomposition | |
| JP2024098259A (en) | Organ evaluation system and organ evaluation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250916 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| P01 | Opt-out of the competence of the unified patent court (upc) registered |
Free format text: CASE NUMBER: UPC_APP_0005690_4687658/2026 Effective date: 20260216 |