US20210345942A1 - Method and Device for Determining Nature or Extent of Skin Disorder - Google Patents
Method and Device for Determining Nature or Extent of Skin Disorder Download PDFInfo
- Publication number
- US20210345942A1 US20210345942A1 US17/282,644 US201917282644A US2021345942A1 US 20210345942 A1 US20210345942 A1 US 20210345942A1 US 201917282644 A US201917282644 A US 201917282644A US 2021345942 A1 US2021345942 A1 US 2021345942A1
- Authority
- US
- United States
- Prior art keywords
- skin
- subject
- affected
- surface model
- disorder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000017520 skin disease Diseases 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000013507 mapping Methods 0.000 claims abstract description 10
- 230000003595 spectral effect Effects 0.000 claims description 23
- 238000010801 machine learning Methods 0.000 claims description 5
- 238000004040 coloring Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 description 67
- 238000003384 imaging method Methods 0.000 description 15
- 201000010099 disease Diseases 0.000 description 14
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 14
- XUMBMVFBXHLACL-UHFFFAOYSA-N Melanin Chemical compound O=C1C(=O)C(C2=CNC3=C(C(C(=O)C4=C32)=O)C)=C2C4=CNC2=C1C XUMBMVFBXHLACL-UHFFFAOYSA-N 0.000 description 10
- 208000010668 atopic eczema Diseases 0.000 description 10
- 230000036074 healthy skin Effects 0.000 description 10
- 201000004624 Dermatitis Diseases 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 9
- 238000011002 quantification Methods 0.000 description 9
- 238000011282 treatment Methods 0.000 description 7
- 208000006981 Skin Abnormalities Diseases 0.000 description 6
- 230000008859 change Effects 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 238000002835 absorbance Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 5
- 206010047642 Vitiligo Diseases 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 238000013517 stratification Methods 0.000 description 4
- 230000000699 topical effect Effects 0.000 description 4
- 201000004681 Psoriasis Diseases 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 238000002560 therapeutic procedure Methods 0.000 description 3
- 206010004950 Birth mark Diseases 0.000 description 2
- 206010010356 Congenital anomaly Diseases 0.000 description 2
- 208000030148 Inflammatory linear verrucous epidermal nevus Diseases 0.000 description 2
- 230000003466 anti-cipated effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000034373 developmental growth involved in morphogenesis Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000024924 glomerular filtration Effects 0.000 description 2
- 230000012010 growth Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 230000000630 rising effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000036555 skin type Effects 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 230000002861 ventricular Effects 0.000 description 2
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 102000008186 Collagen Human genes 0.000 description 1
- 108010035532 Collagen Proteins 0.000 description 1
- 206010012438 Dermatitis atopic Diseases 0.000 description 1
- 206010020751 Hypersensitivity Diseases 0.000 description 1
- 208000000453 Skin Neoplasms Diseases 0.000 description 1
- 206010040829 Skin discolouration Diseases 0.000 description 1
- 206010042496 Sunburn Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 208000026935 allergic disease Diseases 0.000 description 1
- 230000007815 allergy Effects 0.000 description 1
- 201000008937 atopic dermatitis Diseases 0.000 description 1
- 238000001815 biotherapy Methods 0.000 description 1
- 229920001436 collagen Polymers 0.000 description 1
- 239000006071 cream Substances 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 230000005584 early death Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005182 global health Effects 0.000 description 1
- 230000007773 growth pattern Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 208000017169 kidney disease Diseases 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000037311 normal skin Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 210000004789 organ system Anatomy 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000002271 resection Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 201000000849 skin cancer Diseases 0.000 description 1
- 231100000257 skin discolouration Toxicity 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000003319 supportive effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000009121 systemic therapy Methods 0.000 description 1
- 238000011277 treatment modality Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0075—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/1032—Determining colour for diagnostic purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
- A61B5/1075—Measuring physical dimensions, e.g. size of the entire body or parts thereof for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/444—Evaluating skin marks, e.g. mole, nevi, tumour, scar
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/445—Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G06K9/3233—
-
- G06K9/6267—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2576/00—Medical imaging apparatus involving image processing or analysis
- A61B2576/02—Medical imaging apparatus involving image processing or analysis specially adapted for a particular organ or body part
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
- A61B5/015—By temperature mapping of body part
-
- G06K2209/05—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30088—Skin; Dermal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
Definitions
- the present disclosure relates to methods and devices for quantifying the nature and/or extent of a skin disorder.
- a key factor in clinical management of any disease is accurate and objective measurement of severity. This does not currently exist for skin disease. The reason for this is that it has been technically difficult. Most skin diseases are “multi-focal”, affecting various parts of the body. In addition, the skin is the only organ system that cannot be measured by radiological methods such as X-ray or ultrasound.
- Imaging systems for skin such as mole mapping are unable to measure the total extent/severity of a skin disease. They are limited in that they are only imaging systems for monitoring change of individual areas, and are not able to provide a total skin measurement of extent and severity across the entirety of the body.
- a computer-implemented method for quantifying the nature and/or extent of a skin disorder comprises: receiving one or more images of skin of a subject; determining a surface model of the skin of the subject; mapping the one or more images of the skin of the subject onto the surface model to form an image-mapped surface model of the skin of the subject; and determining a value for the nature and/or extent of a skin disorder based on the image-mapped surface model of the skin of the subject.
- the implementations described herein provide an accurate and objective quantification of the nature and/or extent of a skin disorder.
- the skin disorder may be any form of skin condition that can be detected via changes in the spectral properties of the affected regions.
- mapping images onto a surface model the severity of the skin disorder can be determined even where the skin disorder is multi-modal or wraps around body parts. This can be achieved by receiving images taken of the subject from a variety of angles around the subject (in the round). This allows the images to be mapped onto the model effectively to allow effective quantification of the nature and/or extent of the skin disorder.
- the nature and/or extent can be used to determine the overall severity of the skin disorder.
- the severity can be quantified based on the extent (e.g. the surface area) affected, or the acuteness of the affected area(s) (e.g. the severity per unit area or the intensity/acuteness of the condition within any affected areas).
- the severity may be measured based on the spectral properties of the affected areas.
- Images of skin need not be taken by the system at the time of quantification. Instead, the images could be taken of the subject in advance and accessed from a database of images (e.g. stored locally in memory or accessed via a network).
- a database of images e.g. stored locally in memory or accessed via a network.
- the method further comprises identifying one or more regions of skin that are affected by a skin disorder based on the image-mapped surface model of the skin, and the value for the nature and/or extent of the skin disorder is determined based on the one or more regions of skin that are affected by the skin disorder. Accordingly, the method may be able to classify areas of the model as affected by the skin disorder.
- the method further comprises receiving a set of characteristics of skin that is affected by the skin disorder, wherein the identification of the one or more regions of skin that are affected by a skin disorder is based on the set of characteristics.
- the characteristics could be a difference in colour and/or shading of skin relative to a reference area of skin.
- the reference area may be an area not affected by the skin disorder (e.g. healthy skin) or an area of reduced severity of the skin disorder.
- the characteristics could be received by accessing characteristics stored by the computer system or receiving characteristics input from an external system or by a user.
- the characteristics could be received through analysis of a section of the image mapped surface model (or of one of the images) that has been identified as being affected by a skin condition and/or analysis of a section of the image mapped surface model (or of one of the image) that has been identified as showing the reference area.
- the one or more regions of skin that are affected by a skin disorder are identified using a machine learning classifier.
- the machine learning classifier may be pre-trained based on pre-classified training data.
- determining the value for the nature and/or extent of the skin disorder comprises one or more of: determining the surface area of the one or more regions of skin that are affected by the skin disorder; determining the proportion of the overall surface area of skin that is affected by the skin disorder; determining one or more of an average discolouration, an average shading, or an average colouring of the one or more regions of skin that are affected by the skin disorder; and determining, within the one or more regions of skin that are affected by the skin disorder, the strength of one or more spectral signatures associated one or more corresponding molecules that characterise the skin disorder.
- the nature, extent and/or severity may quantified based on the area of skin affected and/or the spectral properties of the affected area(s).
- the colour and/or shading may be as compared to the colour and/or shading of skin not affected by the skin disorder (e.g. to obtain discolouration or a change in shading).
- the severity value may be proportional to the difference between the colour and/or shading of the area affected by the skin disorder and the colour and/or shading of the area not affected by the skin disorder.
- the spectral signature may be detected and measured in order to identify the skin disorder and determine the value for the nature, extent and/or severity of the skin disorder.
- a spectral signature may be a particular frequency or set of frequencies associated with the one or more molecules. Where the spectral signature includes a particular set of frequencies, the signature may also define a particular range of ratios of intensity/strength for the relevant frequencies. The strength of the spectral signature may be determined based on the intensity (e.g. the average intensity) of the signature within the one or more regions of skin that are affected by the skin disorder.
- the method further comprises identifying one or more regions of skin not affected by the skin disorder. This allows the characteristics of the affected regions to be compared to the unaffected regions (e.g. healthy skin) to more accurately measure the severity of the skin condition.
- the unaffected regions e.g. healthy skin
- the method further comprises receiving a further set of characteristics, the further set of characteristics being of skin that is not affected by the skin disorder, and the identification of the one or more regions of skin not affected by the skin condition is based on the further set of characteristics.
- the further set of characteristics may be received in a similar manner to the characteristics for skin affected by a skin disorder (e.g. stored, input, or determined from an area of the image-mapped surface model identified by the user to not be affected by the skin disorder).
- a skin disorder e.g. stored, input, or determined from an area of the image-mapped surface model identified by the user to not be affected by the skin disorder.
- the method comprises receiving position data indicating the shape of the skin of the patient and wherein the surface model is determined based on the position data.
- the position data could be range data, e.g. range data obtained via active range measurements (e.g. from a 3D camera).
- the position data need not be measured as part of the method, but could instead be accessed from a database of measurements (e.g. stored locally or accessed via a network).
- the surface model is determined based on the one or more images. These images may be taken from various positions around the subject. The boundaries of the object may be determined using object recognition methods. The determination of the surface model based on the images could be in addition to determining the model based on position data (i.e. in combination with), or could be without the use of position data (i.e. from the images alone).
- the method further comprises correcting colour and/or shading in the image mapped surface model to account for surface geometry, lighting or camera angles.
- a system for quantifying the severity of a skin disorder comprising a processor configured to implement any method described herein.
- Images of skin need not be taken by the system, but could instead be accessed from a database of images (e.g. locally stored or via a network).
- a non-transitory computer readable medium comprising computer executable instructions that, when executed by a computer, cause the computer to implement any method described herein.
- FIG. 1 shows a system for determining the nature and/or extent of a skin disease according to an embodiment of the invention
- FIG. 2 shows a method for quantifying the nature and/or extent of a skin disease according to an embodiment
- FIG. 3 shows a method of calibrating a system according to the embodiments described herein;
- FIG. 4 shows a computing system for determining the nature and/or extent of a skin disorder according to an embodiment
- FIG. 5 shows classified images and a classified surface model produced according to an embodiment for a single subject.
- the embodiments described herein provide a total skin disease measurement system. This provides a revolutionary step beyond existing imaging/monitoring systems for skin.
- the embodiments described herein obtain accurate and objective numeric measurements of total surface area affected by any skin condition (skin disease, or any other form of skin abnormality), and its characteristics, including for extensive and multifocal diseases.
- this system can be implemented using images and measurements taken with off-the-shelf products.
- the methods can be applied either on-line or off-line to re-analyse and compare historical patient image data with current images. Accordingly, previously taken 2D and 3D images may be analysed off-line to determine the extent of the skin condition.
- Multifocal diseases are diseases that affect two or more locations on the body. Given that the skin is the largest organ of the body, and has a large surface area, these multiple locations may be spaced apart from each other, on different body parts, or located on opposite sides of the body. Furthermore, given the geometry of the body, skin diseases can be difficult to measure as they can wrap around one or more sections of the body. This makes it difficult to image and quantify skin diseases, and in particular, multi-focal skin diseases.
- the methods described herein are applicable to measuring a variety of skin diseases including: a) Psoriasis, b) congenital melanocytic naevi (CMN), c) vitiligo, d) eczema, and e) Inflammatory Linear Verrucous Epidermal Nevus (ILVEN).
- CPN congenital melanocytic naevi
- ILVEN Inflammatory Linear Verrucous Epidermal Nevus
- the implementations described herein overcome this problem by classifying areas affected by a skin condition based on the difference in colour and texture relative to one or more areas of healthy skin on the subject. Furthermore, the implementations make use of one or more cameras that provide improved colour characterisation relative to the naked eye.
- Geometric and radiometric calibration of the imaging system allow different wavelengths and colour channels to be selected in order to maximise the optical signal of a given disease vs healthy skin. For instance, the intensity of red on black skin is a useful quantifier of the severity of eczema.
- the embodiments described herein introduce the Skin Sizer. This provides a radical innovation in skin disease management. Anticipated outcomes will vary with the skin condition. In general, the anticipated benefits include:
- the embodiments described herein are able to provide the same order of magnitude impact on skin disease management as the measurement of left ventricular ejection fraction and glomerular filtration rate in their relevant medical fields.
- FIG. 1 shows a system for determining the nature and/or extent of a skin disease according to an embodiment of the invention.
- the system comprises two cameras 10 , 12 for recording the colour of the skin of a subject 2 (e.g. a patient) and a range camera 14 for recording range measurements so that a 3D surface model of the subject can be generated.
- a subject 2 e.g. a patient
- a range camera 14 for recording range measurements so that a 3D surface model of the subject can be generated.
- the cameras 10 , 12 can be called two dimensional (2D) cameras as they output 2D images of the colour of the skin of the subject 2 .
- Each camera 10 , 12 outputs data indicating the colour detected by the camera at each pixel.
- Each of the 2D cameras 10 , 12 may be any generally available digital camera, such as a DSLR or a monochrome camera equipped with one or more coloured filters tuned to the skin tone and disease being investigated.
- the imaging geometry and spectral response on the imaging system can be quantified in order to deliver accurate, precise and reliable mapping between information captured at each pixel and the total skin surface of the patient
- the images taken by the cameras may be lit using electronic flashes or controlled photographic studio lighting.
- a colour chart can also be located within the field of view of each camera to allow the spectral properties of the environment to be characterised. This provides a consistent calibration and colour correction for improving the accuracy of the images.
- the colour chart is located adjacent to the subject 2 , in the plane of sharp focus.
- the range camera 14 can be called a three dimensional (3D) camera as it outputs range data of the skin of the subject 2 that can be used to determine a surface model of the skin.
- the range camera 14 outputs data in the form of a 2D image indicating the distance between the sensor and the subject at each pixel.
- the range camera 14 may be any generally available range camera, one such example is the ASUSTM XTION 3D range camera.
- the cameras 10 , 12 and range camera 14 output their respective images to a computing system 20 .
- the computing system 20 is configured to combine the images to produce photogrammetric colour corrected images of the skin of the subject 2 for determining the nature and/or extent of a skin disease for the patient.
- the measurements by the cameras 10 , 12 and range camera 14 are taken simultaneously to ensure that the measurements can be combined to produce an accurate 3D representation of the skin independent of patient body motion.
- the relative positions and orientations of the cameras 10 , 12 and the range camera 14 are known through a photogrammetric calibration procedure.
- the cameras 10 , 12 and range camera 14 are mounted on a rigid, but mobile structure. This ensures that, after the system has been calibrated to determine the relative positions of the sensors, the images from the cameras 10 , 12 may be combined with the range measurements by the computing system 20 to obtain a 3D surface model of the skin of the subject.
- the field of view of the two cameras 10 , 12 overlaps the field of view of the active camera 14 across the entirety of an imaging region that contains the subject 2 .
- the field of view of each camera 10 , 12 overlaps the field of view of the other by around 50%.
- the field of view of the range camera 14 overlaps the fields of view of both of the cameras 10 , 12 .
- the cameras 10 , 12 and range camera 14 are capable of capturing image sequences or videos.
- the cameras 10 , 12 and range camera may take a stream of measurements synchronised in time such that the required measurements can be obtained by turning the subject 2 in the field of view of the cameras whilst the image sequences are being recorded. Time-synchronised measurements can then be output for analysis by the computing system 20 . These time-synchronised measurements may be all of the measurements taken over time or a subset (a sample) of the measurements taken over time.
- the present embodiment employs two cameras 10 , 12 and a single range camera 14 arranged vertically (e.g. each located along a single vertical axis) to ensure that there is sufficient coverage across the entire height of the subject 2 at the required resolution for accurate measurement of skin diseases.
- alternative embodiments can employ differing numbers of cameras based to make use of varying properties of differing cameras and range cameras.
- a single camera and a single range camera may be utilised, provided that the field of view of the two cameras overlaps to form an imaging volume. This may be achieved with coverage sufficient to cover the height of the subject 2 if a camera with a sufficiently high resolution is utilised at a sufficient distance from the subject 2 (or with an appropriate wide-angle lens).
- a single camera may be utilised with only partial imaging coverage across the height of the subject. This would then require multiple sets of images to be taken at differing heights in order to fully map the skin of the subject 2 .
- FIG. 1 utilises RGB cameras configured to detect light in the optical wavelength range.
- Alternative embodiments may make use of cameras operating over differing wavelengths, for instance, ultraviolet or infrared cameras.
- black and white cameras may be used. This can allow different spectral wavelengths to be analysed when quantifying the extent of skin conditions having different spectral properties.
- FIG. 1 shows a computing system 20 connected directly to the cameras 10 , 12 and range camera 14 , the measurements of the subject do not need to be taken at the same time and at the same location as the analysis.
- alternative embodiments apply the methods described herein to historical image data stored in a database. It is therefore possible to analysis pre-existing images to obtain a severity measurement for the skin condition.
- FIG. 2 shows a method for quantifying the nature and/or extent of a skin disease according to an embodiment. This method may be performed in the computing device 20 of FIG. 1 .
- the system Prior to measurement of a subject, the system must first be calibrated (not shown). Baseline calibration measurements can be taken to ensure that the colour images necessary for colour classification are geometrically registered with the range data from the 3D camera. The calibration method shall be discussed in more detail later.
- Colour images and range data are received 102 .
- Data obtained from all three cameras can be stored in memory of the computing system 20 , along with patient meta-data and photogrammetric bundle adjustment logs for assessing precision and accuracy.
- the data may be stored in a database and the receiving step 102 may instead comprise accessing the data stored in the database.
- a data fusion method then registers colour images and 3D range data to pixel map the high-resolution image data onto a dense triangle mesh.
- a surface model of the skin is determined from the range data 104 . This can be achieved by combining range measurements taken from various angles around the subject. This produces a 3D model of the surface of the skin of the subject.
- the surface model may be divided into a polygon mesh, such as a triangle mesh.
- the colour images are projected onto the 3D surface model 106 . This may be achieved by projecting the colour data from each pixel back onto its corresponding polygon within the polygon mesh, or by projecting the polygons into the images and extracting the 2D pixel data within each polygon.
- the image-mapped surface model is then analysed to classify the skin into one or more regions affected by a skin condition and one or more regions not affected by a skin condition 108.
- This may be achieved via a machine-learning classifier that is trained to detect one or more skin conditions.
- the user may select one or more skin conditions for classification, or the system may automatically detect the type of skin conditions based on the properties of the affected area(s).
- the spectral absorbance of skin indicates the make up of that area of skin, which may then be disrupted by illness.
- the characteristics of healthy skin vary between subjects. For instance, melanin has a known spectral absorbance and variations in melanin are the main determinant in the variation in healthy skin colour between races.
- the characteristics of healthy skin on a specific patient can be used to determine whether any region of skin is affected by a skin condition. For instance, an area of reduced melanin (relative to the healthy skin of the subject) can indicate the presence of vitiligo (in which melanin is destroyed).
- haemoglobin also has a known spectral absorbance and a change in this absorbance can indicate a skin abnormality, for instance, via an increase in the redness of the skin.
- Collagen is a further component that has a characteristic spectral absorbance and that varies in quantity in various skin conditions.
- the embodiments described herein make use of these known effects to classify areas of skin affected by a skin condition and to quantify the severity of the skin condition.
- the severity may be measured based on the extent of skin that is affected (the area of skin that is affected).
- the severity can also be measured based on the intensity (or acuteness) of the skin condition in the affected area(s). This can be indicated by the difference in spectral properties relative to healthy skin (or any other reference area of skin).
- the severity of a skin condition (such as eczema) can be indicated by the amount of increased haemoglobin (increased redness) indicated in the affected regions.
- a reduction in melanin can be indicative of an increased severity of vitiligo and can be indicated by a change in the spectral properties (e.g. red, yellow, brown and/or black colours) of the affected areas.
- the difference in spectral properties may be measured relative to healthy skin or a different type of reference region.
- the reference region may be an area affected by other skin conditions but not affected by the skin condition being quantified.
- the reference region may be a region of previously known abnormal skin in order to quantify the severity of new skin regions. Furthermore, the reference region may be a region of reduced severity compared to the region being quantified.
- the method determines the nature and/or extent of the skin disease based on the classified model 108 . This may be determined by determining the amount of skin that is affected by the skin condition. For instance, the percentage of the overall surface area that is affected by the skin condition may be calculated. Alternatively, or in addition, the surface area of the affected area, such as in m 2 , may be determined.
- the colour of the affected area(s) can be analysed to determine the severity of the skin condition in the affected areas.
- skin discolouration is indicative of a skin condition.
- the severity of the condition is indicated by the amount of discolouration.
- the average skin colour across the affected area(s) may be determined and compared to the average colour of the unaffected area(s). The average discolouration of the affected areas can then be used to quantify the severity of the condition.
- Colour reflectance and surface normal data information can provide an accurate map of skin surface colour which allows the identification of diseased skin and calculation of total area of diseased skin as a percentage of total skin.
- the method of FIG. 2 utilises a classifier for identifying regions that are affected by a skin condition and regions that are not affected by a skin condition.
- This classifier can be trained via machine learning in order to identify these regions.
- This training can involve classifying a set of labelled training data. The training data may be already classified. This may be input manually by a user in order to identify the affected regions. The classifier can then be trained on the training data, for instance, via reinforcement learning.
- the classifier may be trained to identify a specific abnormality, any form of abnormality, or to classify affected areas based on the abnormality so that the type of abnormality is identified.
- the system receives an input from the user identifying one or more regions of abnormal skin (i.e. regions affected by the skin condition) as well as one or more reference regions of skin (e.g. regions not affected by the skin condition). This allows the system to take into account variations in skin colour when classifying the skin of the subject.
- the pre-trained classifier uses the spectral properties of the reference and abnormal regions to divide the surface model into abnormal regions and reference regions.
- the method may utilise a colour correction step.
- the system may correct the colouring of the model. This may be based on the range data and/or the geometry of the model. For instance, the image content may be corrected based on surface geometry to account for oblique angles between camera and skin and fall off in illumination. This can help to make the colouring of the model more accurate and therefore increase the accuracy of the classification and quantification of the skin abnormality.
- FIG. 3 shows a method of calibrating a system according to the embodiments described herein. This method may be applied to the embodiment of FIG. 1 .
- the camera(s) 201 and range camera 202 are calibrated.
- the camera(s) are calibrated by assessing their spectral response to given wavelengths of light, for example from a mono-chromator or a set of narrow band filters with known transmission characteristics. Information from the colour chart recorded under the same lighting conditions as the patient can then be used to correct the colour of at each pixel in the image. Comparison between registered pixels from each of the cameras improves the capability of the system to make reliable measurements. This allows the system to account for any changes in the properties of the environmental lighting.
- the range camera is calibrated by taking range measurements of an object of known size. This allows the system to determine the scale of the range measurements.
- Photogrammetric markers are placed within the field of view of the range camera and camera(s) 204 . Then, the position and size of the markers are measured with the range camera 206 . These are detailed in a range image output from the range camera 206 . Without moving the markers, an image of the markers is taken using each camera 208 . The camera image(s) and range measurements may be taken simultaneously.
- the camera and range image are then compared to obtain a relationship between the pixels in the images and the range measurements.
- the coordinates of the markers within each image are determined. Equally, the coordinates of the markers within the range image are determined.
- the coordinates from the range data are interpolated at subpixel marker image coordinates.
- the coordinates are then used as control points for photogrammetric processing using a least squares bundle adjustment solution to simultaneously obtain best estimates of the imaging geometry of each camera and the geometric relationship between each camera and the photogrammetric markers.
- the markers may be of a known size and shape. Accordingly, the markers can be used to calibrate the cameras and range sensor (steps 201 and 202 ) at the same time as registering the cameras with the range sensors (steps 206 - 210 ).
- the geometric calibration between the camera and range images allows colour data from the images to be accurately registered with the range data.
- the photogrammetric calibration parameters and their variance covariance matrices output from the bundle adjustment describe both the internal and external camera imaging geometry and its uncertainty. This functional and stochastic relationship allows the image data and the colour measurements to be mapped onto surface models calculated from the range data.
- the system may be configured to check its calibration and registration data using control points.
- control points with a known, planar shape are located within the imaging volume.
- the system is configured to perform intercomparison between the best fit XYZ control points and those estimated from the bundle adjustment, checking the planarity of the XYZ control points used in the solution to ensure that the calibration is accurate.
- the system may be configured to analyse image measurement residuals and estimated parameters and compare them to equivalent residuals/parameters taken from historical system usage as well as alternative resection (with the cameras at an alternative angle to the subject) to ensure consistency.
- the system may be configured to determine the XYZ coordinates of features located within an overlapping area contained in multiple images and compare these to each other to ensure consistency.
- the position and orientation of each camera may be subsequently determined based on the images and compared to the positions and orientations determined during calibration to ensure consistency.
- embodiments discussed herein discuss the use of range measurements to generate a model of the surface of the skin of a subject
- alternative embodiments may generate a model based solely on image data from a network of images made with 2D cameras. This can be achieved by analysing images of the subject taken at a variety of angles in order to generate a surface model of the subject. The colour data from the images may then be mapped onto the surface model to allow any skin abnormalities to be analysed as per steps 108 and 110 of the method of FIG. 2 .
- the use of active range measurements provides a more efficient and effective method for classifying and quantifying skin abnormalities because it is not reliant on finding areas of common detail in each image.
- alternative embodiments may classify the skin of a subject based on black and white images.
- skin abnormalities may be indicated by a change in shading.
- FIG. 4 shows a computing system 300 for determining the nature and/or extent of a skin disorder according to an embodiment.
- the computing system 300 may be the computing system 20 of FIG. 1 .
- the computing system may implement the methods of FIGS. 3 and 4 .
- the computing system 300 may be any general purpose computing system provided that it is loaded with executable code for performing the methods described herein.
- the computing system comprises an interface 310 for receiving one or more images of a subject.
- the interface may also receive range data of the subject.
- the computing system 300 further comprises a controller 320 configured to perform the processing steps discussed herein.
- the controller 320 can be, by way of example without limitation, a digital processor, a computer microchip, electronic circuitry, or any other form of computing processor.
- the computing system 300 also includes memory for storing calibration data to allow the controller to generate a surface model of the skin of the subject and to map the colour data onto the surface model.
- the memory 330 also stores classification data to allow the controller to classify areas of the textured model into one or more regions affected by a skin condition and one or more regions not affected by the skin condition.
- the memory 330 is non-volatile memory.
- the memory 330 stores executable computer-readable instructions which, when executed by the controller 320 , cause the controller to implement the methods discussed herein.
- the controller 320 further comprises a calibration module 321 .
- the calibration module 321 is configured to calibrate the system based on measurements of known shape and measurements of known colour.
- the controller 320 further comprises a surface modelling module 322 configured to determine a surface model of the skin of the subject from measurement data of the location of the skin of the subject.
- the measurement data may comprise position measurements of the skin, for instance, range data.
- the measurement data may comprise image data. Accordingly, a surface model of the subject may be determined solely from image data, solely from position data, or from a combination of both image and position data.
- the controller 320 further comprises an image mapping module 323 configured to map colour data from the images onto the surface model to form an image-mapped surface model of the skin of the subject.
- the controller 320 further comprises a colour correction module 324 for correcting the mapped colour data to take account of position, lighting, surface geometry and specularity.
- the controller 320 further comprises a classification module 325 for classifying various regions of the image-mapped surface model as either affected by a skin condition or not affected by a skin condition.
- the controller 320 further comprises a quantification module 326 for quantifying the nature and/or extent of the skin condition. This may be based on surface area affected by the skin condition and/or the average discolouration of the areas affected by the skin condition.
- a system according to the embodiment of FIG. 1 has been tested in 10 patients recruited from the Paediatric Dermatology department in Great Ormond Street Hospital, under standard lighting conditions, in the medical illustration department (professional photographic set up), and data analysed in the Civil, Environmental and Geomatic Engineering (CEGE) department of UCL. This was done under full approval of the London Bloomsbury Research Ethics Committee.
- FIG. 5 shows classified images and a classified surface model produced according to an embodiment for a single subject.
- FIG. 5 a shows two camera images of the subject.
- FIG. 5 b shows a 3D reconstruction of the subject based on combined ranging and image information.
- FIG. 5 c shows a shaded view of the 3D reconstruction demonstrating the separation from the reference background. This separation allows the calculation of the surface area affected by the disease as a percentage of the total body surface area. In this case, the percentage affected is 26%.
- the areas identified as affected by a skin condition are shown in blue, normal skin is shown in red.
- the present methodology is able to provide quantified severity measurements of skin diseases. This is distinguished from imaging systems, such as dermoscopy (hand held magnification aid) or mole mapping (photographic 2D imaging system for monitoring of changes over time) for skin cancer surveillance, which are unable to quantify the extent and severity of multifocal skin diseases being examined.
- imaging systems such as dermoscopy (hand held magnification aid) or mole mapping (photographic 2D imaging system for monitoring of changes over time) for skin cancer surveillance, which are unable to quantify the extent and severity of multifocal skin diseases being examined.
- the embodiments descried herein provide a framework for significantly more accurate and objective measurements than existing skin disease severity scoring systems, even when compared to well-validated methods such as the Patient Oriented Eczema Measure (POEM), or the consensus classification system for congenital melanocytic naevi.
- POEM Patient Oriented Eczema Measure
- the systems and methods described herein are capable of measuring and quantifying a wide variety of skin conditions, including multifocal skin conditions.
- Alternative methods are unable to provide a holistic quantification of the extent or severity of multifocal skin conditions across the whole body as they are only able to consider isolated regions of the body.
- the systems and methods described herein are able to accurately quantify and track the progression of skin diseases, even in children.
- Children are difficult to monitor as they are likely to grow between measurements and, in addition, their body proportions change (for example, the head grows much less than the torso—a baby's head is much bigger as a proportion of its length than an adult's—also boys torso's grow longitudinally much more than girls', etc.).
- This can present challenges when quantifying the severity or extent of a skin condition which is present during childhood.
- the embodiments described herein are able to quantify the extent of the skin condition based on the percentage of the overall surface are of the subject that is affected. This therefore accounts for changes in body size so that users can determine whether the skin condition is spreading or receding.
- Implementations of the subject matter and the operations described in this specification can be realized in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be realized using one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
- an artificially generated propagated signal e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
- a computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Dermatology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Signal Processing (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Physiology (AREA)
- Quality & Reliability (AREA)
- Psychiatry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Dentistry (AREA)
- Geometry (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
Abstract
Description
- The present disclosure relates to methods and devices for quantifying the nature and/or extent of a skin disorder.
- Skin diseases constitute the 10th greatest cause of disability-adjusted life years in the UK, defined as the number of years lost due to ill-health, disability or early death (Newton, J. N. et al. Changes in health in England, with analysis by English regions and areas of deprivation, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet 386, 2257-74 (2015)). This results in heavy socioeconomic costs, with a total cost to the NHS in England for 2010/2011 of £2,135,000,000 (excluding the substantial costs of burn injuries), and $75 billion of direct costs in the USA in 2013 (Lim, H. W. et al. The burden of skin disease in the United States. J Am Acad Dermatol 76, 958-972 e2 (2017)). In addition, these costs are rising steeply, due to new treatment modalities and aging populations. Optimisation of management of skin disease is therefore a high priority.
- A key factor in clinical management of any disease is accurate and objective measurement of severity. This does not currently exist for skin disease. The reason for this is that it has been technically difficult. Most skin diseases are “multi-focal”, affecting various parts of the body. In addition, the skin is the only organ system that cannot be measured by radiological methods such as X-ray or ultrasound.
- Existing measurement methods for skin are semi-quantitative, inaccurate, and operator-dependent (Schmitt, J., Langan, S., Williams, H. C. & European Dermato-Epidemiology, N. What are the best outcome measurements for atopic eczema? A systematic review. J Allergy Clin Immunol 120, 1389-98 (2007)). These measurement methods may use a tape measure or the rule of nines (burns, vitiligo), visual estimation of affected area (psoriasis, birthmarks), or severity scores such as Patient Oriented Eczema Measure (eczema). These are inherently subjective and inaccurate. In addition, these methods are not generally used outside specialist centres.
- Existing imaging systems for skin such as mole mapping are unable to measure the total extent/severity of a skin disease. They are limited in that they are only imaging systems for monitoring change of individual areas, and are not able to provide a total skin measurement of extent and severity across the entirety of the body.
- The lack of accurate and objective measurement for whole body skin disease has a serious impact on patients, as well as doctors and hospitals. There is no accurate baseline measurement of severity, and therefore no consistent stratification of patients for treatment. Patients are usually treated sequentially by a treatment ladder when treatments fail, with no accurate numeric documentation of response or lack of response. Not only does this likely result in less effective treatment, but also likely incurs associated socioeconomic costs both to the patient and the NHS, in the form of increased number of appointments in secondary and tertiary care, and loss of school and work time.
- Furthermore, without accurate measurement of extent or severity of skin disease there are no clear guidelines on the amounts of topical therapy to prescribe. This can result in doctors prescribing an incorrect dosage. Underestimating the required dosage can result in a reduced effectiveness of the treatment. Overestimating the required dosage can result in substantial wastage. Prescriptions for topical skin therapies are a major cost, and an inability to measure the amount required by a patient will inevitably lead to inaccurate prescription. Globally this problem is exacerbated even further, with a total lack of skin disease severity measurement in most parts of the world.
- There is therefore a need for a means for accurately quantifying the nature and severity of skin diseases.
- According to a first aspect there is provided a computer-implemented method for quantifying the nature and/or extent of a skin disorder. The method comprises: receiving one or more images of skin of a subject; determining a surface model of the skin of the subject; mapping the one or more images of the skin of the subject onto the surface model to form an image-mapped surface model of the skin of the subject; and determining a value for the nature and/or extent of a skin disorder based on the image-mapped surface model of the skin of the subject.
- The implementations described herein provide an accurate and objective quantification of the nature and/or extent of a skin disorder. The skin disorder may be any form of skin condition that can be detected via changes in the spectral properties of the affected regions. By mapping images onto a surface model the severity of the skin disorder can be determined even where the skin disorder is multi-modal or wraps around body parts. This can be achieved by receiving images taken of the subject from a variety of angles around the subject (in the round). This allows the images to be mapped onto the model effectively to allow effective quantification of the nature and/or extent of the skin disorder.
- The nature and/or extent can be used to determine the overall severity of the skin disorder. The severity can be quantified based on the extent (e.g. the surface area) affected, or the acuteness of the affected area(s) (e.g. the severity per unit area or the intensity/acuteness of the condition within any affected areas). The severity may be measured based on the spectral properties of the affected areas.
- Images of skin need not be taken by the system at the time of quantification. Instead, the images could be taken of the subject in advance and accessed from a database of images (e.g. stored locally in memory or accessed via a network).
- According to a further implementation the method further comprises identifying one or more regions of skin that are affected by a skin disorder based on the image-mapped surface model of the skin, and the value for the nature and/or extent of the skin disorder is determined based on the one or more regions of skin that are affected by the skin disorder. Accordingly, the method may be able to classify areas of the model as affected by the skin disorder.
- According to a further implementation the method further comprises receiving a set of characteristics of skin that is affected by the skin disorder, wherein the identification of the one or more regions of skin that are affected by a skin disorder is based on the set of characteristics.
- The characteristics could be a difference in colour and/or shading of skin relative to a reference area of skin. The reference area may be an area not affected by the skin disorder (e.g. healthy skin) or an area of reduced severity of the skin disorder. The characteristics could be received by accessing characteristics stored by the computer system or receiving characteristics input from an external system or by a user.
- Alternatively or in addition, the characteristics could be received through analysis of a section of the image mapped surface model (or of one of the images) that has been identified as being affected by a skin condition and/or analysis of a section of the image mapped surface model (or of one of the image) that has been identified as showing the reference area.
- According to a further implementation the one or more regions of skin that are affected by a skin disorder are identified using a machine learning classifier. The machine learning classifier may be pre-trained based on pre-classified training data.
- According to a further implementation determining the value for the nature and/or extent of the skin disorder comprises one or more of: determining the surface area of the one or more regions of skin that are affected by the skin disorder; determining the proportion of the overall surface area of skin that is affected by the skin disorder; determining one or more of an average discolouration, an average shading, or an average colouring of the one or more regions of skin that are affected by the skin disorder; and determining, within the one or more regions of skin that are affected by the skin disorder, the strength of one or more spectral signatures associated one or more corresponding molecules that characterise the skin disorder.
- Accordingly, the nature, extent and/or severity may quantified based on the area of skin affected and/or the spectral properties of the affected area(s). The colour and/or shading may be as compared to the colour and/or shading of skin not affected by the skin disorder (e.g. to obtain discolouration or a change in shading). The severity value may be proportional to the difference between the colour and/or shading of the area affected by the skin disorder and the colour and/or shading of the area not affected by the skin disorder.
- Furthermore, where a particular spectral signature is known to be associated with one or more molecules that are indicative of the skin disorder, the spectral signature may be detected and measured in order to identify the skin disorder and determine the value for the nature, extent and/or severity of the skin disorder. A spectral signature may be a particular frequency or set of frequencies associated with the one or more molecules. Where the spectral signature includes a particular set of frequencies, the signature may also define a particular range of ratios of intensity/strength for the relevant frequencies. The strength of the spectral signature may be determined based on the intensity (e.g. the average intensity) of the signature within the one or more regions of skin that are affected by the skin disorder.
- According to a further implementation the method further comprises identifying one or more regions of skin not affected by the skin disorder. This allows the characteristics of the affected regions to be compared to the unaffected regions (e.g. healthy skin) to more accurately measure the severity of the skin condition.
- According to a further implementation, the method further comprises receiving a further set of characteristics, the further set of characteristics being of skin that is not affected by the skin disorder, and the identification of the one or more regions of skin not affected by the skin condition is based on the further set of characteristics.
- The further set of characteristics may be received in a similar manner to the characteristics for skin affected by a skin disorder (e.g. stored, input, or determined from an area of the image-mapped surface model identified by the user to not be affected by the skin disorder).
- According to a further implementation the method comprises receiving position data indicating the shape of the skin of the patient and wherein the surface model is determined based on the position data. This allows the surface model (and therefore the severity value) to be more accurately and more efficiently determined. The position data could be range data, e.g. range data obtained via active range measurements (e.g. from a 3D camera). The position data need not be measured as part of the method, but could instead be accessed from a database of measurements (e.g. stored locally or accessed via a network).
- According to a further implementation the surface model is determined based on the one or more images. These images may be taken from various positions around the subject. The boundaries of the object may be determined using object recognition methods. The determination of the surface model based on the images could be in addition to determining the model based on position data (i.e. in combination with), or could be without the use of position data (i.e. from the images alone).
- According to a further implementation the method further comprises correcting colour and/or shading in the image mapped surface model to account for surface geometry, lighting or camera angles.
- According to a further aspect there is provided a system for quantifying the severity of a skin disorder, the system comprising a processor configured to implement any method described herein.
- Images of skin need not be taken by the system, but could instead be accessed from a database of images (e.g. locally stored or via a network).
- According to a further implementation there is provided a non-transitory computer readable medium comprising computer executable instructions that, when executed by a computer, cause the computer to implement any method described herein.
- Arrangements of the present invention will be understood and appreciated more fully from the following detailed description, made by way of example only and taken in conjunction with drawings in which:
-
FIG. 1 shows a system for determining the nature and/or extent of a skin disease according to an embodiment of the invention; -
FIG. 2 shows a method for quantifying the nature and/or extent of a skin disease according to an embodiment -
FIG. 3 shows a method of calibrating a system according to the embodiments described herein; -
FIG. 4 shows a computing system for determining the nature and/or extent of a skin disorder according to an embodiment; and -
FIG. 5 shows classified images and a classified surface model produced according to an embodiment for a single subject. - The embodiments described herein provide a total skin disease measurement system. This provides a revolutionary step beyond existing imaging/monitoring systems for skin. The embodiments described herein obtain accurate and objective numeric measurements of total surface area affected by any skin condition (skin disease, or any other form of skin abnormality), and its characteristics, including for extensive and multifocal diseases. Importantly, this system can be implemented using images and measurements taken with off-the-shelf products. The methods can be applied either on-line or off-line to re-analyse and compare historical patient image data with current images. Accordingly, previously taken 2D and 3D images may be analysed off-line to determine the extent of the skin condition.
- Improvement in the management of skin diseases is a key priority for health services. There is a steeply increasing budget for topical and systemic therapies within the UK and US over the last decade. Similarly, this is a priority for patients, as skin diseases such as eczema have become increasingly common, now affecting one third of children. Associated patient costs are in quality of life for the patient and families/carers, loss of school and work time, and high costs of travel to secondary and tertiary care centres.
- Multifocal diseases are diseases that affect two or more locations on the body. Given that the skin is the largest organ of the body, and has a large surface area, these multiple locations may be spaced apart from each other, on different body parts, or located on opposite sides of the body. Furthermore, given the geometry of the body, skin diseases can be difficult to measure as they can wrap around one or more sections of the body. This makes it difficult to image and quantify skin diseases, and in particular, multi-focal skin diseases.
- Current methods are unable to accurately and objectively measure these types of diseases. The methods described herein are applicable to measuring a variety of skin diseases including: a) Psoriasis, b) congenital melanocytic naevi (CMN), c) vitiligo, d) eczema, and e) Inflammatory Linear Verrucous Epidermal Nevus (ILVEN).
- Furthermore, it can be difficult to quantify the extent or severity of skin conditions accurately across different subjects as the appearance skin conditions depends on the skin type of the individual (e.g. skin colour). For instance, it is often difficult to detect some skin diseases on darker skin by eye, particularly in fine detail. In addition, as skin type can vary significantly between subjects, this makes it difficult to provide an objective measure of skin conditions across different subjects. The implementations described herein overcome this problem by classifying areas affected by a skin condition based on the difference in colour and texture relative to one or more areas of healthy skin on the subject. Furthermore, the implementations make use of one or more cameras that provide improved colour characterisation relative to the naked eye.
- Geometric and radiometric calibration of the imaging system allow different wavelengths and colour channels to be selected in order to maximise the optical signal of a given disease vs healthy skin. For instance, the intensity of red on black skin is a useful quantifier of the severity of eczema.
- Given the large surface area and multi-focal nature of many skin diseases, they are impossible currently to quantify as a total affected surface area, with respect to total body surface area. In addition, the severity and nature is difficult to quantify accurately. Most diseases do not have measurement system for severity, whereas common diseases such as eczema have multiple competing severity systems. In addition, the nature and severity of skin diseases at different ages can be difficult to characterise due to the large variation in body size and shape between subjects, and in particular due to differential growth of different body parts at different ages. Furthermore, the large variety of skin colours means that accurate identification and characterisation of affected areas can be difficult.
- The embodiments described herein introduce the Skin Sizer. This provides a radical innovation in skin disease management. Anticipated outcomes will vary with the skin condition. In general, the anticipated benefits include:
- 1) Reduction in time to instigation of appropriate patient treatment through accurate and objective patient stratification, leading to decreased secondary/tertiary care outpatient appointments and GP appointments, and to decreased drug costs;
- 2) Reduction in number of unnecessary investigations through better patient stratification;
- 3) Reduction in clinician time during appointments, with measurement performed before seeing the doctor (3-5 minutes) and eliminating the need for existing classification and severity scores (5-10 minutes);
- 4) Reduced prescription wastage of topical therapies by accurate and objective measurement of surface area of affected skin at each visit (and prescribing of the correct volumes of creams);
- 5) Increased quality of life of patients and families due to all of the above;
- 6) Reduced socioeconomic burden on the UK economy from loss of school and work attendance, and costs of travel to secondary/tertiary centres;
- There is overwhelming evidence from the literature from other diseases that accurate, objective disease severity measurement improves patient care and ultimately reduces costs. Some key severity measurements in recent medical history include left ventricular ejection fraction in cardiovascular disease, and glomerular filtration rate in renal disease, both of which rapidly became key severity measures for stratification of patient outcome and disease management.
- As the first real measurement system for multifocal skin disease, the embodiments described herein are able to provide the same order of magnitude impact on skin disease management as the measurement of left ventricular ejection fraction and glomerular filtration rate in their relevant medical fields.
-
FIG. 1 shows a system for determining the nature and/or extent of a skin disease according to an embodiment of the invention. The system comprises twocameras range camera 14 for recording range measurements so that a 3D surface model of the subject can be generated. - The
cameras subject 2. Eachcamera 2D cameras - To improve the accuracy of the colour readings and applicability to a wide range of imaging situations, the images taken by the cameras may be lit using electronic flashes or controlled photographic studio lighting. A colour chart can also be located within the field of view of each camera to allow the spectral properties of the environment to be characterised. This provides a consistent calibration and colour correction for improving the accuracy of the images. Preferably the colour chart is located adjacent to the
subject 2, in the plane of sharp focus. - The
range camera 14 can be called a three dimensional (3D) camera as it outputs range data of the skin of the subject 2 that can be used to determine a surface model of the skin. Therange camera 14 outputs data in the form of a 2D image indicating the distance between the sensor and the subject at each pixel. Therange camera 14 may be any generally available range camera, one such example is the ASUS™ XTION 3D range camera. - The
cameras range camera 14 output their respective images to acomputing system 20. Thecomputing system 20 is configured to combine the images to produce photogrammetric colour corrected images of the skin of the subject 2 for determining the nature and/or extent of a skin disease for the patient. The measurements by thecameras range camera 14 are taken simultaneously to ensure that the measurements can be combined to produce an accurate 3D representation of the skin independent of patient body motion. - The relative positions and orientations of the
cameras range camera 14 are known through a photogrammetric calibration procedure. Thecameras range camera 14 are mounted on a rigid, but mobile structure. This ensures that, after the system has been calibrated to determine the relative positions of the sensors, the images from thecameras computing system 20 to obtain a 3D surface model of the skin of the subject. - The field of view of the two
cameras active camera 14 across the entirety of an imaging region that contains thesubject 2. In the present embodiment, the field of view of eachcamera range camera 14 overlaps the fields of view of both of thecameras - In order to get full coverage of the skin of the subject 2, it may be necessary to take measurements from multiple viewpoints relative to the
subject 2. This can be achieved by turning the subject relative to thecameras cameras range cameras 14 may be arranged around the subject to get full 360° coverage. This can ensure that measurements covering all angles can be taken simultaneously, to improve the accuracy of the 3D model once the measurements have been combined. - In one embodiment, the
cameras range camera 14 are capable of capturing image sequences or videos. In this embodiment, thecameras computing system 20. These time-synchronised measurements may be all of the measurements taken over time or a subset (a sample) of the measurements taken over time. - The present embodiment employs two
cameras single range camera 14 arranged vertically (e.g. each located along a single vertical axis) to ensure that there is sufficient coverage across the entire height of the subject 2 at the required resolution for accurate measurement of skin diseases. Nevertheless, alternative embodiments can employ differing numbers of cameras based to make use of varying properties of differing cameras and range cameras. - For instance, a single camera and a single range camera may be utilised, provided that the field of view of the two cameras overlaps to form an imaging volume. This may be achieved with coverage sufficient to cover the height of the subject 2 if a camera with a sufficiently high resolution is utilised at a sufficient distance from the subject 2 (or with an appropriate wide-angle lens). Alternatively, a single camera may be utilised with only partial imaging coverage across the height of the subject. This would then require multiple sets of images to be taken at differing heights in order to fully map the skin of the
subject 2. - The embodiment of
FIG. 1 utilises RGB cameras configured to detect light in the optical wavelength range. Alternative embodiments may make use of cameras operating over differing wavelengths, for instance, ultraviolet or infrared cameras. Alternatively, black and white cameras may be used. This can allow different spectral wavelengths to be analysed when quantifying the extent of skin conditions having different spectral properties. - Whilst the embodiment of
FIG. 1 shows acomputing system 20 connected directly to thecameras range camera 14, the measurements of the subject do not need to be taken at the same time and at the same location as the analysis. In fact, alternative embodiments apply the methods described herein to historical image data stored in a database. It is therefore possible to analysis pre-existing images to obtain a severity measurement for the skin condition. -
FIG. 2 shows a method for quantifying the nature and/or extent of a skin disease according to an embodiment. This method may be performed in thecomputing device 20 ofFIG. 1 . - Prior to measurement of a subject, the system must first be calibrated (not shown). Baseline calibration measurements can be taken to ensure that the colour images necessary for colour classification are geometrically registered with the range data from the 3D camera. The calibration method shall be discussed in more detail later.
- Colour images and range data are received 102. Data obtained from all three cameras (the 2D cameras and the 3D camera) can be stored in memory of the
computing system 20, along with patient meta-data and photogrammetric bundle adjustment logs for assessing precision and accuracy. Alternatively, the data may be stored in a database and the receivingstep 102 may instead comprise accessing the data stored in the database. - A data fusion method then registers colour images and 3D range data to pixel map the high-resolution image data onto a dense triangle mesh. To achieve this, a surface model of the skin is determined from the
range data 104. This can be achieved by combining range measurements taken from various angles around the subject. This produces a 3D model of the surface of the skin of the subject. The surface model may be divided into a polygon mesh, such as a triangle mesh. Then, the colour images are projected onto the3D surface model 106. This may be achieved by projecting the colour data from each pixel back onto its corresponding polygon within the polygon mesh, or by projecting the polygons into the images and extracting the 2D pixel data within each polygon. In either case decisions are made utilising range camera surface normals compared to each colour camera, individual camera magnification of the local skin surface, local image sharpness, the presence of occlusions, local skin surface secularity, shadows and non-skin content such as clothes and hair. The result is an image-mapped surface representation of the body. - The image-mapped surface model is then analysed to classify the skin into one or more regions affected by a skin condition and one or more regions not affected by a
skin condition 108. This may be achieved via a machine-learning classifier that is trained to detect one or more skin conditions. The user may select one or more skin conditions for classification, or the system may automatically detect the type of skin conditions based on the properties of the affected area(s). - The spectral absorbance of skin indicates the make up of that area of skin, which may then be disrupted by illness. The characteristics of healthy skin vary between subjects. For instance, melanin has a known spectral absorbance and variations in melanin are the main determinant in the variation in healthy skin colour between races.
- Nevertheless, the characteristics of healthy skin on a specific patient can be used to determine whether any region of skin is affected by a skin condition. For instance, an area of reduced melanin (relative to the healthy skin of the subject) can indicate the presence of vitiligo (in which melanin is destroyed). In addition, haemoglobin also has a known spectral absorbance and a change in this absorbance can indicate a skin abnormality, for instance, via an increase in the redness of the skin. Collagen is a further component that has a characteristic spectral absorbance and that varies in quantity in various skin conditions.
- The embodiments described herein make use of these known effects to classify areas of skin affected by a skin condition and to quantify the severity of the skin condition. The severity may be measured based on the extent of skin that is affected (the area of skin that is affected). The severity can also be measured based on the intensity (or acuteness) of the skin condition in the affected area(s). This can be indicated by the difference in spectral properties relative to healthy skin (or any other reference area of skin). For instance, the severity of a skin condition (such as eczema) can be indicated by the amount of increased haemoglobin (increased redness) indicated in the affected regions. Equally, a reduction in melanin can be indicative of an increased severity of vitiligo and can be indicated by a change in the spectral properties (e.g. red, yellow, brown and/or black colours) of the affected areas.
- The difference in spectral properties may be measured relative to healthy skin or a different type of reference region. For instance, the reference region may be an area affected by other skin conditions but not affected by the skin condition being quantified.
- Equally, the reference region may be a region of previously known abnormal skin in order to quantify the severity of new skin regions. Furthermore, the reference region may be a region of reduced severity compared to the region being quantified.
- The method then determines the nature and/or extent of the skin disease based on the
classified model 108. This may be determined by determining the amount of skin that is affected by the skin condition. For instance, the percentage of the overall surface area that is affected by the skin condition may be calculated. Alternatively, or in addition, the surface area of the affected area, such as in m2, may be determined. - Alternatively or in addition, the colour of the affected area(s) can be analysed to determine the severity of the skin condition in the affected areas. As discussed, skin discolouration is indicative of a skin condition. The severity of the condition is indicated by the amount of discolouration. The average skin colour across the affected area(s) may be determined and compared to the average colour of the unaffected area(s). The average discolouration of the affected areas can then be used to quantify the severity of the condition.
- The methods described herein provide an accurate, rapid and objective calculation of the nature, extent or severity of multifocal skin disease. Colour reflectance and surface normal data information can provide an accurate map of skin surface colour which allows the identification of diseased skin and calculation of total area of diseased skin as a percentage of total skin.
- The method of
FIG. 2 utilises a classifier for identifying regions that are affected by a skin condition and regions that are not affected by a skin condition. This classifier can be trained via machine learning in order to identify these regions. This training can involve classifying a set of labelled training data. The training data may be already classified. This may be input manually by a user in order to identify the affected regions. The classifier can then be trained on the training data, for instance, via reinforcement learning. The classifier may be trained to identify a specific abnormality, any form of abnormality, or to classify affected areas based on the abnormality so that the type of abnormality is identified. - To identify the affected region(s), the system receives an input from the user identifying one or more regions of abnormal skin (i.e. regions affected by the skin condition) as well as one or more reference regions of skin (e.g. regions not affected by the skin condition). This allows the system to take into account variations in skin colour when classifying the skin of the subject. The pre-trained classifier uses the spectral properties of the reference and abnormal regions to divide the surface model into abnormal regions and reference regions.
- Prior to classification, the method may utilise a colour correction step. After the images are projected onto the surface model, the system may correct the colouring of the model. This may be based on the range data and/or the geometry of the model. For instance, the image content may be corrected based on surface geometry to account for oblique angles between camera and skin and fall off in illumination. This can help to make the colouring of the model more accurate and therefore increase the accuracy of the classification and quantification of the skin abnormality.
-
FIG. 3 shows a method of calibrating a system according to the embodiments described herein. This method may be applied to the embodiment ofFIG. 1 . - Firstly, the camera(s) 201 and
range camera 202 are calibrated. The camera(s) are calibrated by assessing their spectral response to given wavelengths of light, for example from a mono-chromator or a set of narrow band filters with known transmission characteristics. Information from the colour chart recorded under the same lighting conditions as the patient can then be used to correct the colour of at each pixel in the image. Comparison between registered pixels from each of the cameras improves the capability of the system to make reliable measurements. This allows the system to account for any changes in the properties of the environmental lighting. The range camera is calibrated by taking range measurements of an object of known size. This allows the system to determine the scale of the range measurements. - Photogrammetric markers are placed within the field of view of the range camera and camera(s) 204. Then, the position and size of the markers are measured with the
range camera 206. These are detailed in a range image output from therange camera 206. Without moving the markers, an image of the markers is taken using eachcamera 208. The camera image(s) and range measurements may be taken simultaneously. - The camera and range image are then compared to obtain a relationship between the pixels in the images and the range measurements. The coordinates of the markers within each image are determined. Equally, the coordinates of the markers within the range image are determined. The coordinates from the range data are interpolated at subpixel marker image coordinates. The coordinates are then used as control points for photogrammetric processing using a least squares bundle adjustment solution to simultaneously obtain best estimates of the imaging geometry of each camera and the geometric relationship between each camera and the photogrammetric markers.
- The markers may be of a known size and shape. Accordingly, the markers can be used to calibrate the cameras and range sensor (
steps 201 and 202) at the same time as registering the cameras with the range sensors (steps 206-210). - The geometric calibration between the camera and range images allows colour data from the images to be accurately registered with the range data. The photogrammetric calibration parameters and their variance covariance matrices output from the bundle adjustment describe both the internal and external camera imaging geometry and its uncertainty. This functional and stochastic relationship allows the image data and the colour measurements to be mapped onto surface models calculated from the range data.
- To ensure accuracy of the system, the system may be configured to check its calibration and registration data using control points. To achieve this, control points with a known, planar shape are located within the imaging volume. The system is configured to perform intercomparison between the best fit XYZ control points and those estimated from the bundle adjustment, checking the planarity of the XYZ control points used in the solution to ensure that the calibration is accurate. Furthermore, the system may be configured to analyse image measurement residuals and estimated parameters and compare them to equivalent residuals/parameters taken from historical system usage as well as alternative resection (with the cameras at an alternative angle to the subject) to ensure consistency. Furthermore, the system may be configured to determine the XYZ coordinates of features located within an overlapping area contained in multiple images and compare these to each other to ensure consistency. Furthermore, the position and orientation of each camera may be subsequently determined based on the images and compared to the positions and orientations determined during calibration to ensure consistency.
- Whilst the embodiments described herein discuss the quantification of skin diseases, these methods may equally be applied to the quantification of any form of skin disorder or skin condition that is identifiable via discolouration, such as sunburn.
- In addition, whilst the embodiments discussed herein discuss the use of range measurements to generate a model of the surface of the skin of a subject, alternative embodiments may generate a model based solely on image data from a network of images made with 2D cameras. This can be achieved by analysing images of the subject taken at a variety of angles in order to generate a surface model of the subject. The colour data from the images may then be mapped onto the surface model to allow any skin abnormalities to be analysed as per
steps FIG. 2 . By utilising only camera data, the system can be implemented more cost effectively. Nevertheless, the use of active range measurements provides a more efficient and effective method for classifying and quantifying skin abnormalities because it is not reliant on finding areas of common detail in each image. - Furthermore, whilst the embodiments described herein utilise range and camera measurements from a fixed position, it is possible to take these measurements from a mobile device, such as a smart phone or tablet computer. This can be combined with the use of continuous measurements (e.g. video) so that the model can be accurately assembled from a large number of measurements taken quickly from various locations around the subject.
- In addition, whilst the embodiments described herein utilise colour images, alternative embodiments may classify the skin of a subject based on black and white images. In this case, skin abnormalities may be indicated by a change in shading.
- Furthermore, whilst the embodiments described herein may use of a range camera to provide range data indicating the position of the surface of the skin, alternative methods of mapping the surface of the skin may be utilised.
-
FIG. 4 shows acomputing system 300 for determining the nature and/or extent of a skin disorder according to an embodiment. Thecomputing system 300 may be thecomputing system 20 ofFIG. 1 . The computing system may implement the methods ofFIGS. 3 and 4 . Thecomputing system 300 may be any general purpose computing system provided that it is loaded with executable code for performing the methods described herein. - The computing system comprises an
interface 310 for receiving one or more images of a subject. The interface may also receive range data of the subject. Thecomputing system 300 further comprises acontroller 320 configured to perform the processing steps discussed herein. - The
controller 320 can be, by way of example without limitation, a digital processor, a computer microchip, electronic circuitry, or any other form of computing processor. - The
computing system 300 also includes memory for storing calibration data to allow the controller to generate a surface model of the skin of the subject and to map the colour data onto the surface model. Thememory 330 also stores classification data to allow the controller to classify areas of the textured model into one or more regions affected by a skin condition and one or more regions not affected by the skin condition. Thememory 330 is non-volatile memory. Thememory 330 stores executable computer-readable instructions which, when executed by thecontroller 320, cause the controller to implement the methods discussed herein. - The
controller 320 further comprises acalibration module 321. Thecalibration module 321 is configured to calibrate the system based on measurements of known shape and measurements of known colour. - The
controller 320 further comprises asurface modelling module 322 configured to determine a surface model of the skin of the subject from measurement data of the location of the skin of the subject. The measurement data may comprise position measurements of the skin, for instance, range data. Alternatively, or in addition, the measurement data may comprise image data. Accordingly, a surface model of the subject may be determined solely from image data, solely from position data, or from a combination of both image and position data. - The
controller 320 further comprises animage mapping module 323 configured to map colour data from the images onto the surface model to form an image-mapped surface model of the skin of the subject. - The
controller 320 further comprises acolour correction module 324 for correcting the mapped colour data to take account of position, lighting, surface geometry and specularity. - The
controller 320 further comprises aclassification module 325 for classifying various regions of the image-mapped surface model as either affected by a skin condition or not affected by a skin condition. - The
controller 320 further comprises aquantification module 326 for quantifying the nature and/or extent of the skin condition. This may be based on surface area affected by the skin condition and/or the average discolouration of the areas affected by the skin condition. - A system according to the embodiment of
FIG. 1 has been tested in 10 patients recruited from the Paediatric Dermatology department in Great Ormond Street Hospital, under standard lighting conditions, in the medical illustration department (professional photographic set up), and data analysed in the Civil, Environmental and Geomatic Engineering (CEGE) department of UCL. This was done under full approval of the London Bloomsbury Research Ethics Committee. -
FIG. 5 shows classified images and a classified surface model produced according to an embodiment for a single subject.FIG. 5a shows two camera images of the subject.FIG. 5b shows a 3D reconstruction of the subject based on combined ranging and image information.FIG. 5c shows a shaded view of the 3D reconstruction demonstrating the separation from the reference background. This separation allows the calculation of the surface area affected by the disease as a percentage of the total body surface area. In this case, the percentage affected is 26%. In each of the figures, the areas identified as affected by a skin condition are shown in blue, normal skin is shown in red. - NHS and global health care costs for skin disease are currently rising steeply, with the arrival of biological therapies for common diseases such as psoriasis and recently eczema. The methods presently available are unable to produce the advantages associated with the embodiments described herein.
- Firstly, the present methodology is able to provide quantified severity measurements of skin diseases. This is distinguished from imaging systems, such as dermoscopy (hand held magnification aid) or mole mapping (photographic 2D imaging system for monitoring of changes over time) for skin cancer surveillance, which are unable to quantify the extent and severity of multifocal skin diseases being examined.
- Secondly, the embodiments descried herein provide a framework for significantly more accurate and objective measurements than existing skin disease severity scoring systems, even when compared to well-validated methods such as the Patient Oriented Eczema Measure (POEM), or the consensus classification system for congenital melanocytic naevi.
- Thirdly, the systems and methods described herein are capable of measuring and quantifying a wide variety of skin conditions, including multifocal skin conditions. Alternative methods are unable to provide a holistic quantification of the extent or severity of multifocal skin conditions across the whole body as they are only able to consider isolated regions of the body.
- In addition, the systems and methods described herein are able to accurately quantify and track the progression of skin diseases, even in children. Children are difficult to monitor as they are likely to grow between measurements and, in addition, their body proportions change (for example, the head grows much less than the torso—a baby's head is much bigger as a proportion of its length than an adult's—also boys torso's grow longitudinally much more than girls', etc.). This can present challenges when quantifying the severity or extent of a skin condition which is present during childhood.
- For instance, current methods extrapolate the size of an affected area of skin up to a Projected Adult Size (an estimated size in adulthood). The differential growth patterns of various body parts makes this particularly difficult when affected areas of skin cross between various body parts (e.g. from the head down to the torso). This can lead to inaccuracies in the assessment of affected surface area.
- The embodiments described herein are able to quantify the extent of the skin condition based on the percentage of the overall surface are of the subject that is affected. This therefore accounts for changes in body size so that users can determine whether the skin condition is spreading or receding.
- Further embodiments make use of a growth projection mechanism for providing a more accurate projected adult measurement for standardisation of quantification. This makes use of known growth patterns for males and females (including difference in growth speed for different body parts) to transform the model of the subject into a predicted adult model. A projected adult measurement then may be made based on the predicted adult model (e.g. based on the surface area affected in the predicted adult model).
- Furthermore, the methods described herein are easy to use and therefore can be applied easily by a wider variety of users. This is in contrast to a number of the current severity measures that require a high level of training to implement.
- Narrative market research from members of the British Society of Paediatric Dermatology has demonstrated overwhelming support for an accurate measurement system for skin conditions.
- In parallel, patient surveys and patient group engagement prioritised accurate skin measurement system research, and the need for improved personalised treatment based on individual measurements. Two surveys were designed specifically for this project, in collaboration with the PPI co-I on this project. Results were highly supportive of this topic and research questions as Research Priorities. For example, in answer to the question “Do you think it would be useful if there was a camera that could take a picture and measure accurately how extensive and how severe your child's skin disease is?” 95% of respondents ticked yes. In addition, in answer to the question “Do you think developing a new and accurate measurement system should be a priority for birthmark research?” 85% of responses were positive. To a question of prioritising the features of the Skin Sizer system, the desire for accuracy of measurement scored at 90%.
- Implementations of the subject matter and the operations described in this specification can be realized in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be realized using one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).
- While certain arrangements have been described, the arrangements have been presented by way of example only, and are not intended to limit the scope of protection. The inventive concepts described herein may be implemented in a variety of other forms. In addition, various omissions, substitutions and changes to the specific implementations described herein may be made without departing from the scope of protection defined in the following claims.
Claims (11)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1816083.8 | 2018-10-02 | ||
GB1816083.8A GB2577700B (en) | 2018-10-02 | 2018-10-02 | Method and device for determining nature or extent of skin disorder |
PCT/GB2019/052753 WO2020070477A1 (en) | 2018-10-02 | 2019-09-30 | Method and device for determining nature or extent of skin disorder |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210345942A1 true US20210345942A1 (en) | 2021-11-11 |
Family
ID=68343160
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/282,644 Pending US20210345942A1 (en) | 2018-10-02 | 2019-09-30 | Method and Device for Determining Nature or Extent of Skin Disorder |
Country Status (4)
Country | Link |
---|---|
US (1) | US20210345942A1 (en) |
EP (1) | EP3860437A1 (en) |
GB (1) | GB2577700B (en) |
WO (1) | WO2020070477A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220257173A1 (en) * | 2021-02-17 | 2022-08-18 | Optum Technology, Inc. | Extended-reality skin-condition-development prediction and visualization |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102019130369A1 (en) * | 2019-11-11 | 2021-05-12 | Gottfried Wilhelm Leibniz Universität Hannover | Camera system for prescribing a detailed image of a patient's skin and online dermatoscope |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110301441A1 (en) * | 2007-01-05 | 2011-12-08 | Myskin, Inc. | Analytic methods of tissue evaluation |
US20150206022A1 (en) * | 2014-01-22 | 2015-07-23 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for classifying a skin infection |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
IL124616A0 (en) * | 1998-05-24 | 1998-12-06 | Romedix Ltd | Apparatus and method for measurement and temporal comparison of skin surface images |
US7546156B2 (en) * | 2003-05-09 | 2009-06-09 | University Of Rochester Medical Center | Method of indexing biological imaging data using a three-dimensional body representation |
US20120078088A1 (en) * | 2010-09-28 | 2012-03-29 | Point of Contact, LLC. | Medical image projection and tracking system |
MY174941A (en) * | 2011-03-25 | 2020-05-27 | Institute Of Tech Petronas Sdn Bhd | Methodology and apparatus for lesion area measurement of skin pigmentation disorders using digital imaging |
US20140378810A1 (en) * | 2013-04-18 | 2014-12-25 | Digimarc Corporation | Physiologic data acquisition and analysis |
US9717417B2 (en) * | 2014-10-29 | 2017-08-01 | Spectral Md, Inc. | Reflective mode multi-spectral time-resolved optical imaging methods and apparatuses for tissue classification |
US11304604B2 (en) * | 2014-10-29 | 2022-04-19 | Spectral Md, Inc. | Reflective mode multi-spectral time-resolved optical imaging methods and apparatuses for tissue classification |
US10362984B2 (en) * | 2015-01-27 | 2019-07-30 | Healthy.Io Ltd | Measuring and monitoring skin feature colors, form and size |
US10413232B2 (en) * | 2016-02-10 | 2019-09-17 | Balter Medical, As. | Optical transfer diagnosis for detection and monitoring of tissue disorders |
-
2018
- 2018-10-02 GB GB1816083.8A patent/GB2577700B/en active Active
-
2019
- 2019-09-30 US US17/282,644 patent/US20210345942A1/en active Pending
- 2019-09-30 EP EP19794197.4A patent/EP3860437A1/en active Pending
- 2019-09-30 WO PCT/GB2019/052753 patent/WO2020070477A1/en unknown
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110301441A1 (en) * | 2007-01-05 | 2011-12-08 | Myskin, Inc. | Analytic methods of tissue evaluation |
US20150206022A1 (en) * | 2014-01-22 | 2015-07-23 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for classifying a skin infection |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220257173A1 (en) * | 2021-02-17 | 2022-08-18 | Optum Technology, Inc. | Extended-reality skin-condition-development prediction and visualization |
Also Published As
Publication number | Publication date |
---|---|
WO2020070477A1 (en) | 2020-04-09 |
GB2577700B (en) | 2021-12-08 |
GB2577700A (en) | 2020-04-08 |
EP3860437A1 (en) | 2021-08-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11452455B2 (en) | Skin reflectance and oiliness measurement | |
US20200229695A1 (en) | Calculating Conjunctival Redness | |
US7519210B2 (en) | Method of assessing localized shape and temperature of the human body | |
EP2544583B1 (en) | System, method and article for normalization and enhancement of tissue images | |
Rodriguez et al. | Automated grading system for evaluation of ocular redness associated with dry eye | |
Chang et al. | Multimodal sensor system for pressure ulcer wound assessment and care | |
US10872408B2 (en) | Method and system for imaging and analysis of anatomical features | |
JP2017504370A (en) | Method and system for wound assessment and management | |
US20210345942A1 (en) | Method and Device for Determining Nature or Extent of Skin Disorder | |
CN108151670A (en) | A kind of back three dimensional symmetry degree evaluating apparatus and method | |
Pineda et al. | Robust automated reading of the skin prick test via 3D imaging and parametric surface fitting | |
Choi et al. | Characteristics of subjective recognition and computer‐aided image analysis of facial erythematous skin diseases: a cornerstone of automated diagnosis | |
Wu et al. | Automatic foot scanning and measurement based on multiple RGB-depth cameras | |
US9962089B2 (en) | Methodology and apparatus for objective assessment and rating of psoriasis lesion thickness using digital imaging | |
Jdid et al. | Validation of digital photographic reference scales for evaluating facial aging signs | |
Hillmer et al. | Evaluation of facial vitiligo severity with a mixed clinical and artificial intelligence approach | |
Cassidy et al. | Artificial intelligence for automated detection of diabetic foot ulcers: A real-world proof-of-concept clinical evaluation | |
Alagha et al. | Objective grading facial paralysis severity using a dynamic 3D stereo photogrammetry imaging system | |
Fadzil et al. | Independent component analysis for assessing therapeutic response in vitiligo skin disorder | |
Udrea et al. | Real-time acquisition of quality verified nonstandardized color images for skin lesions risk assessment—A preliminary study | |
Mutsvangwa et al. | Precision assessment of stereo-photogrammetrically derived facial landmarks in infants | |
Nishino et al. | Detection and visualization of intracutaneous allergic type‐specific elements using long‐wavelength near‐infrared hyperspectral imaging | |
US20230284903A1 (en) | System and method for acquisition and quantification of images with ocular staining | |
KR20150117074A (en) | Apparatus and Method for skin condition diagnosis | |
Noor Azhar et al. | Improving the reliability of smartphone-based urine colorimetry using a colour card calibration method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KINSLER, VERONICA, UNITED KINGDOM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:UCL BUSINESS PLC;REEL/FRAME:056942/0040 Effective date: 20180611 Owner name: UCL BUSINESS PLC, UNITED KINGDOM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KINSLER, VERONICA;ROBSON, STUART;BOEHM, JAN;AND OTHERS;REEL/FRAME:056941/0938 Effective date: 20180316 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |