NL2022603B1

NL2022603B1 - Skin tone measurement

Info

Publication number: NL2022603B1
Application number: NL2022603A
Authority: NL
Inventors: Kryuchkova Larisa
Original assignee: Uvisio B V
Priority date: 2019-02-20
Filing date: 2019-02-20
Publication date: 2020-08-31
Also published as: WO2020171695A1; EP3927233A1

Abstract

A reference device for aiding the measurement of a skin tone of a skin area of a skin, comprising: a surface; an opening in the surface providing an observation window on the skin area when the reference device is placed upon the skin; reference markers arranged to the opening and/or the surface, such that an image taken for the measurement shows the skin area and the reference markers in substantially the same light conditions and/or under the same angle.

Description

SKIN TONE MEASUREMENT

FIELD OF THE INVENTION The invention relates to a method in the field of skin tone measurement, a reference device for aiding the measurement of a skin tone and a system for measuring a skin tone.

BACKGROUND OF THE INVENTION The exposure of the skin to sunlight over longer periods of time is a well- known cause of skin diseases, such as skin cancer. It is known that the duration of the exposure and the intensity of the exposure to sunlight influences the change of developing a skin disease. Another factor influencing the chance of developing a skin disease is the vulnerability of the skin for such diseases.

A factor in the vulnerability of the skin for such diseases is the melanin content of the skin. The melanin content of the skin may be measured by measuring the skin tone. The paler the skin tone, the lower the MED or minimal erythemal dose. The MED relates directly to sun burn sensitivity or even the chance of developing skin diseases.

Skin tone or skin colour may be classified. An example of the classification of skin is given in S. Del Bino and F. Bernerd, British Journal of Dermatology. The classification ranges in this example from very light, light, intermediate, tan, brown to dark.

Measuring the skin tone is typically done with the use of a specialized electronic device. A specialized electronic device is typically complex and not readily available. Also, when a person would like to measure his or her skin tone over time, it is necessary to carry the specialized electronic device around, which is considered a burden.

In the field of cosmetics, skin tone may be determined for selecting cosmetics, typically the colour of the cosmetics. A solution in the field of cosmetics is to take an image with a smartphone. The image shows a face of a person. The image further shows a colour reference card, held by the same person. And it may even be that the smartphone is held by this person. A disadvantage of this measurement is that itis inaccurate.

SUMMARY OF THE INVENTION An object of the invention is to overcome one or more of the disadvantages mentioned above. Further, it is an object of the invention to provide a simple skin tone measurement, which is accurate enough for use in a calculation for determining the chance of developing a skin disease and/or for determining the sun burn sensitivity, such as the time a person may be exposed to sunlight with a particular intensity.

According to a first aspect of the invention, a Reference device for aiding the measurement of a skin tone of a skin area of a skin, comprising: - a surface; - an opening in the surface providing an observation window on the skin area when the reference device is placed upon the skin; - reference markers arranged to the opening and/or the surface, such that an image taken for the measurement shows the skin area and the reference markers in substantially the same light conditions and/or under the same angle.

The area of the skin selected for the measurement of the skin tone should preferably be flat. The area of the skin selected for the measurement of the skin tone should preferable be exposed to sunlight, such as direct sunlight. Typically, the area of the skin selected for measurement of the skin tone is in the face or on an extremity of a body, such as arms and legs. In specific cases, the area of the skin selected for measurement of the skin tone may be on the torso.

As the reference device has an opening placed upon the skin during measurement, the area of the skin selected for the measurement is well controlled. Typically, the selection comprises a relative flat area of the skin. As the area is relatively flat, the light conditions are relatively the same over the area. The skin itself will most likely not cause any shadow effects as it is relatively flat. Further, any light impinging onto the skin will typically cause the same effects, such as absorption and/or reflection of light, over the complete area. Hence, the measurement accuracy will be increased as the measurement may be performed by averaging over a substantial part of the area.

Furthermore, the reference device comprises the reference markers. The reference markers provide a reference within the image. The reference may be for a position, such as positioning the opening and thus the skin area relative to the position of the reference markers. The reference may be for an orientation, such as positioning the opening under a certain angle and thus the skin area relative to the orientation ofthe reference markers. Preferably, the reference is for position and orientation. The reference markers are typically markers easily detected by image processing algorithms. Thus, the reference markers may provide a relatively simple means for locating the opening and the skin area.

Alternatively, the reference may be for a colour. The colour is typically a predefined colour, such that the image processing algorithm may relate or calibrate the pixel values in the image to the predefined colour. By calibrating each image, the influence of light conditions on the actual skin tone measurement may be minimized.

Alternatively, the reference markers provide a combination of a reference for position and colour, as for example described above. The combination provides also a combination of the mentioned advantages. The combination further simplifies the image processing.

It is an insight of the inventor that the skin tone measurement may be improved by arranging the reference markers to the opening and/or the surface. The result is that the reference markers and the skin area are arranged to each other during skin tone measurement. As the reference markers and the skin area are typically flat, the light impinging on both may undergo the same effect, especially if the angle of both have the same or substantially the same angle. If the reference markers and the skin area undergo the same effect, equal colours will appear equal or substantially equal in the image. Hence, providing substantially the same angle to the skin area and the reference markers improves the accuracy of the skin tone measurement.

A further effect of the reference markers and the skin area being arranged to each other during skin tone measurement, is that the skin area and the reference markers are in close proximity to each other. Light conditions may vary over the image, but if placed in close proximity to each other, the variation in light conditions is typically minimized, which results in that the equal colours in the skin area and on the reference markers will appear equal or substantially equal in the image. Hence, providing the skin area and the reference markers in close proximity improves the accuracy of the skin tone measurement.

Close proximity in the context of this invention may be within 10 cm, preferably within 5 cm, more preferably 3 cm, most preferably 2 cm. Furthermore, close proximity may be within half of the image width or height, preferably a quarter, more preferably 3/16-th, most preferably 1/8-th.

In the context of this invention the image is typically taken by a smartphone.

The smartphone is typically held about 10 cm from the skin area and the reference markers while taking the image.

The image is typically taken close enough for having the skin area and the reference markers each stretching out over a plurality of pixels in the image.

This allows the image processing algorithm to take a plurality of pixels for averaging over the plurality of pixels for increasing the accuracy of the measurement.

The distance is typically far enough for having the skin area and the reference markers to be in focus for the smartphone.

An in-focus image allows the image processing algorithm to sharply detect the pixels belonging to the skin area and/or the reference markers.

Hence, the reference device according to the invention enables a simplified yet accurate skin tone measurement.

In an embodiment of the invention, the reference markers are arranged around the opening.

As the reference markers are arranged around the opening, the distance between the observed skin through the opening and the reference markers is minimized.

This minimizing of the distance provides that the light conditions and/or the angle are the same or substantially the same.

In an embodiment of the invention, the reference markers comprise colour reference markers for colour reference for the measurement of the skin type.

As the skin tone is measured, the reference markers are advantageously provided with a colour close to or within the range of skin colours for improved calibrating the colours in the image.

In an embodiment of the invention, the colour reference markers comprise a surface having a skin colour.

Light conditions and angle of a surface, such as the skin surface or reference markers, influence the perceived colour in an image taken from this surface.

Providing a colour to the reference marker allows measuring the skin tone with the aid of a colour reference, allowing for a measurement with improved accuracy.

In a further embodiment of the invention, the colour reference markers form pairs having the same colour and are arranged at opposite sides of the opening.

As the light conditions may change over the area in view of the image, the change of the light condition is typically changing from one end to the other end of the image.

Arranging colour reference markers having the same colour at opposite sides of the opening allows the calibration algorithm processing the image to correct for this gradually varying light condition over the image area.

Typically, the average colour of two colour reference markers is taken and applied as reference to the skin area inbetween the reference markers. In a further refinement, the colour reference markers allow for a linear interpolation correcting the light condition of a skin area depending on the distance to the colour reference marker.

In a further embodiment of the invention, multiple colour reference markers 5 having the same colour are arranged around the opening, preferably at equal distances between the colour reference markers.

In a further embodiment of the invention, the colour reference markers form a colour range. The colour range is typically the colour range of the human skin. Thus, the colour range typically comprise skin colours ranging from black, brown, dark, yellow, pale, to white. The two colour calibration makers differing the most are typically just beyond, at or close to the extremes of skin tone to be measured. Furthermore, the colours of the colour reference markers are typically spread evenly over the colour range. As the colour reference markers form the colour range, this provides the advantage of improved calibration of the image colours, such that the skine tone may be measured with more accuracy.

In a further embodiment of the invention, the reference markers comprise position reference markers, wherein the opening and the colour reference markers are defined relative to the position and/or orientation of the position reference markers. Colours in the image may change under light conditions. Further, typically an image processing algorithm may detect a reference in the image with relative ease if the contrast is high. On the other hand, a reference marker having a high contrast is typically not having a colour suitable as colour reference. Hence, this embodiment advantageously comprises position reference markers with high contrast for relatively easy detection of the position reference markers in the image. Typically, the position reference markers comprise the colours white and black and/or define sharp lines or surfaces wherein the surfaces preferably have straight or circular edges. Further, the position reference markers advantageously define the position of the opening and/or colour reference markers. Thus, as soon as the position reference markers are found in the image, with relative ease, predefined areas in the image may be selected for analysis without further algorithms searching for the colour reference markers and/or opening.

In an embodiment of the invention, the position reference markers are arranged relative to the colour reference markers such that at least four of the centres of the markers substantially form an ellipse. The image may typically be distorted by that the reference markers and the skin surface are under approximately the sameangle in the image. At least three, but preferably four position reference markers are positioned such that the centres of these position reference markers may be connected to each other by a line having the shape of an ellipse. As the shape of the ellipse is determined by the angle of the surfaces in the image as well as the direction of this angle, the image may be easily corrected for this angle. Or the position of the opening and the other reference markers, typically colour reference markers, may be easily calculated from the measured angle and direction of the angle.

In a further embodiment of the invention, the position reference markers are arranged relative to the opening such that the centre of the opening is substantially the centroid of the position reference markers. This advantageously predefines the opening showing the skin to be measured.

In an embodiment of the invention, each reference marker comprises: - a circumference line having a colour; and - an enclosed reference surface having a colour different from the colour of the circumference line. The circumference line may advantageously be used to locate the reference marker within the image. Furthermore, the circumference line may advantageously be used to locate the reference surface within the image.

In an embodiment of the invention, at least a part of the reference markers is on the surface and/or wherein the surface is translucent and at least a part of the reference markers is in or behind the surface. The manufacturing process is simplified by placing the reference markers on top of the surface. Placing the reference markers within a translucent surface, provides a more robust reference device against scratching and other abrasive incidents.

In an embodiment of the invention, the reference device comprises a raised edge enclosing the opening and the reference markers. The measurement taking the opening and reference markers in the image into account may be disturbed by direct light and shadow effects caused by this direct light and objects in the light path. A raised edge may prevent direct light from the outside to influence the measurement. In a further embodiment, the raised edge has a matt colour, such as a matt black colour, onthe inside of the raised edge, such that the influence of reflections of the light on the measurement are minimized.

In a particular embodiment of the invention, the opening providing an observation window is covered with a translucent material. The essence is that the skin tone of the skin area shown in the observation window can be measured. If the opening is without translucent material in the opening, having the benefit of causing noaberrations in the measurement, or with translucent material, having the benefit of shielding the skin, is within the scope of the claims.

According to another aspect of the invention, a system for measuring a skin tone of a skin area of a skin, comprising: - a reference device according to any of the preceding embodiments; - a mobile device having a camera for taking an image of the surface and the skin area when the reference device is placed upon the skin; and - a processor for providing a skin tone based on the image.

According to another aspect of the invention, an image of the surface of a reference device according to any of the embodiments, wherein the reference device is placed upon a skin for aiding the measurement of a skin tone of the skin.

According to another aspect of the invention, a method for measuring a skin tone of a skin area of a skin, comprising the steps of: - receiving an image (200) showing the skin area and colour reference markers (151) each having a reference colour; - locating of the colour reference markers in the image, wherein the skin area is arranged relative to the colour reference markers; - locating of the skin area in the image based on the colour reference markers; - providing the image in an image format having a luminance component and at least one colour component; - calibrating a reference scale based on the combination of the luminance component and the at least one colour component of the reference colours; and - calculating the skin tone based on the combination of the luminance component and the at least one colour component of the skin area and the reference scale.

The method comprises an image showing the skin area and colour reference markers each having a reference colour. The colour reference markers are preferably part of a reference device as detailed in the text.

The method steps of locating skin area and providing may be interchanged or done in parallel. The method steps of locating colour reference markers and providing may be interchanged or done in parallel. Further, the method step of locating skin area may be done in parallel or after the calibrating step. Further, the method step of calculating takes as input a located skin area within the image and the image in animage format having a luminance component and at least one colour component independent of the order and/or parallelism of the preceding method steps.

The method identifies the combination of the luminance component and the at least one colour component for the skin area as crucial to the calculation of the skin tone. To improve the accuracy, the calculation uses a calibrated reference scale. As the calculating step is based on the combination, the calibrating step also uses the same combination to further improve the accuracy of the calculated skin tone.

The image received has colour reference markers arranged to the skin area. In the locating step the colour reference markers are located. The output of the locating step is used by the calibrating step to calibrate the reference scale. And the output of the locating step also defines the location of the skin area. Thus, the step of locating the skin area is based on the location of the colour reference markers without the need to process the image. The absence of processing the image for locating the skin area has the effect of simplifying the method step and thus the method as a whole, while maintaining or even improving the accuracy of the measured skin tone. Furthermore, the position of the skin area in the image is more accurately defined, allowing not only for a simplified search of the skin area, but also to define the area to be measured more accurately.

The method is preferably applied for non-medical uses, more preferably for cosmetic uses. The method may also be implemented in software run on a processor. The software may be used for medical or non-medical uses. The non-medical use may comprise cosmetic uses.

In an embodiment of the invention, the colour reference markers are arranged around and/or in close proximity to the skin area. As lighting conditions may vary over the image, placing the colour reference markers around and/or in close proximity of the skin area from which the skin tone is to be calculated provides the advantage of decreasing the change in light condition between the colour reference markers and the skin area. The decreased change results in an increased accuracy and/or reliability of the calculated skin tone.

In an embodiment of the invention, the colour reference markers are arranged substantially parallel and in close proximity, substantially flush, with the skin area. If the colour reference markers are arranged substantially flush with the skin area, the angle of the skin area is substantially the same as the colour reference markers in the image. As a result, the reliability and/or accuracy of the skin tone calculated is increased.

In an embodiment of the invention, the received image is in an sRGB format and/or the image format having a luminance component and at least one colour component adheres to the CIELab colour space. The advantage of receiving the image in SRGB format is that the most common cameras provide images in this format. Thus, allowing the method to receive images in sRGB format allows a simple link with most common cameras. Further, the inventor is of the opinion that the CIELab colour space is particularly advantageous for determining the skin tone. Hence, a method converting from sRGB format to CIELab colour space combines these advantages.

In an embodiment of the invention, the providing step comprises the steps of: - applying an inverse gamma correction to the received image for providing a gamma corrected image; - applying on the gamma corrected image a conversion for providing a XYZ image having a XYZ colour space; - applying on the XYZ image a conversion for providing the image adhering to the CIELab colour space. This embodiment provides the advantage of a transformation path between the sRGB format to the CIELab colour space.

In an embodiment of the invention, the at least one colour component is blue-yellow colour component in the CIELab colour space. The blue-yellow colour component provides the advantage of being a good indicator for the skin tone.

In an embodiment of the invention, the calibrating step and/or calculating step is not based on the green-red colour component in the CIELab colour space. Ignoring the green-red colour component provides the advantage of ignoring a bad indicator for the skin tone.

In an embodiment of the invention, the reference colours of the colour reference markers form a continuous colour scale. Having a continuous scale provides the advantage that the skin area may be matched with a particular position on this scale having the same skin tone. The position on the scale may then directly relate back to the skin tone of the skin area. In a further embodiment, the continuous colour scale may be positioned around the skin area, especially wherein the colour scale is circular with the skin area in the middle of the circle. The skin tone may then be matched with a particular angle from the centre of the circle.

In an embodiment of the invention, the reference colours of the colour reference markers provide colour samples. The colour samples cover multiple pixels inthe image. The multiple pixels in the image may advantageously be averaged to come to a more stable reference colour.

In an embodiment of the invention, the calculating step comprises the step of interpolating between and/or extrapolating from two colour. Interpolating between colours provides typically a means for increasing the accuracy of a skin tone measurement with a high degree of certainty. Extrapolating setting out from two colours may advantageously provide the means for increasing the accuracy of the skin tone measurement, while not all extreme skin tones are represented inside the colour range formed by the reference colours. Furthermore, the reference colours may also be concentrated at a particular sub-range for providing more accuracy to the skin tone measurement for skin tones of interest, such as vulnerable to sun burns and/or skin diseases.

In an embodiment of the invention, the calibrating step comprises the step of segmenting the reference scale in skin tone segments and the calculating step comprises the step of determining that the skin area falls within a certain skin tone segment. This embodiment provides the advantage of simplifying the method for determining the skin tone by matching the skin tone with a segment of the reference scale. The segment may be part of a continuous colour scale or part of a segmented colour scale having only discrete colour steps.

In an embodiment of the invention, wherein the image shows two position reference markers, and wherein the step of locating of the colour reference markers comprises the steps of: - locating the two position reference markers; - locating two centres of respective position reference markers, wherein the reference markers are arranged in a ring around the skin area; - determining an origin halfway between the two centres; - determining a line segment perpendicular to the line joining the two centres and extending from the origin; - determining a third point halfway the intersection between the line segment and the ring; and - allocating a segment of the colour reference marker as colour reference based on the line segment, the origin, a predefined angle between the line segment and a distance from the origin based on a predefined distance. Using the knowledge of the arrangement of the reference markers provides the advantage of a simplified vision algorithm. Due to this simplification, the vision algorithm may be run on a smartphone.

Or due to this simplification, a cropped image may be transported from the vision algorithm locating the reference markers and the skin area and the vision algorithm doing the colour calibration and calculating the skin tone. The cropped image provides the advantage of having a substantially smaller data size. In an embodiment of the invention, the method comprises the steps of: - recognizing an inner ellipse in the image; and/or - recognizing an outer ellipse in the image; wherein the step of locating two position reference markers is based on the location of the inner ellipse and/or the location of the outer ellipse. The vision algorithm may advantageously recognize the ellipse, such that the position reference markers may be found in a more limited area of the image increasing the processing speed of the image.

In an embodiment of the invention, the method comprises the steps of: - calculating a centre ellipse through the two centres and the third point; and - correcting the distance between the origin and the segment with the knowledge of the shape of the centre ellipse, the inner ellipse and/or the outer ellipse. The centre ellipse typically joins all centre points of all reference markers. Furthermore, the centre ellipse defines the shape of the colour reference marker maximum distance and the colour reference marker minimum distance. Furthermore, the colour reference marker maximum distance and the colour reference marker minimum distance may advantageously be defined at some relative distance between the centre ellipse and the outer ellipse and the inner ellipse respectively.

In an embodiment of the invention, the method comprises the steps of: - calculating the relative angle of the skin area or reference markers in the image based on the shape of the inner ellipse, the outer ellipse and/or the centre ellipse; and - rejecting the image if the relative skin angle exceeds a predefined limit angle. The reference markers and skin area may be seen as part of a reference plane.

A camera taking the image has a viewing axis which is typically going through the centre of the image sensor and the lenses of the camera. The relative angle is between this reference plane and the viewing axis. The relative angle is ideally 90 degrees or the viewing axis is perpendicular to the reference plane. The inner ellipse, the outer ellipse and the centre ellipse are circles if the relative angle is 90 degrees.

Any deviation from the 90 degrees shows the inner ellipse, the outer ellipse and thecentre ellipse as ellipses in the image. The shape, more specific the flattening of the ellipse, relates to the relative angle. The relative angle for an image suitable for processing is typically within the range of 50-90 degrees, preferably 60-90 degrees, more preferably 70-90 degrees, most preferable 80-90 degrees. An image taken under a relative angle outside the range may be rejected. Images taken with a relative angle within the range allow advantageously for correct or at least more accurate calculating the skin tone.

According to another aspect of the invention, a computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform any of the methods of embodiments. According to another aspect of the invention, a device for processing an image for measuring a skin tone of a skin area of a skin, comprising: - a receiver for receiving the image showing the skin area and colour reference markers each having a reference colour; and - a processor for performing the steps of: - locating of the colour reference markers in the image, wherein the skin area is arranged relative to the colour reference markers; - locating of the skin area in the image based on the colour reference markers; - providing the image in an image format having a luminance component and at least one colour component; - calibrating a reference scale based on the combination of the luminance component and the at least one colour component of the reference colours; and - calculating the skin tone based on the combination of the luminance component and the at least one colour component of the skin area and the reference scale. According to another aspect of the invention, a system for measuring a skin tone of a skin area of a skin, comprising: - a mobile device having a camera for taking an image; and - a device for processing an image for measuring a skin tone of a skin area of a skin.

According to another aspect of the invention, an image for a method for measuring a skin tone of a skin area of a skin according to any of the embodiments, wherein the reference markers are arranged to the skin area.

In an embodiment of the invention, the reference markers are arranged around and/or in close proximity to the skin area, providing the advantage of increased reliability and/or accuracy as specified in the text.

According to another aspect of the invention, a method for predicting a skin reaction to UV exposure, comprising the steps of: - receiving a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion; - training a neural network based on the training dataset, wherein the training comprises the steps of: - segmenting the lesion from the rest of the image; - extracting lesion parameters from the segmented lesion; and - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network; wherein the method further comprises the steps of - receiving an image showing a skin area; - analysing the skin area with the use of the trained neural network; and - predicting the skin reaction after UV exposure based on the analyses.

The method according to the current invention allows the training of a neural network with a specific training set for predicting the skin reaction to UV exposure.

Typically, the predicting step also comprises an advising step providing an advice how long a person from whom the skin is analysed may be exposed to sunshine of a particular sun power with or without the application of sun cream.

Typically, the method takes into account the amount of sunshine local to where the person is or will be.

Typically, the method takes into account the geographical location of the person.

Typically, the method takes into account the local weather.

The typing of the lesion typically comprises colour typing of the lesion and/or shape typing of the lesion as specified throughout this text.

In an embodiment of the invention, wherein the typing of the skin reaction comprises a label from the group of no reaction, sun burned reaction, photo dermatoses reaction. In a further embodiment, the typing may comprise a value specifying a severity of a particular skin reaction. In a further embodiment of the invention, wherein the photo dermatoses reaction typing comprises a sub-typing in a phototoxic reaction and a photoallergic reaction. In an embodiment of the invention, wherein the lesion parameters comprise one or more of the group of: - colour variegation; - border typing; - surface typing; and - shape. Colour variegation is the appearance of a skin area with different coloured zones. The border of a lesion may be sharp, such as having a high contrast or may have a gradient, such as blending of the border of the lesion with the surrounding skin area, irregularity of the border, waviness of the border or any other variation in between. The surface type of the lesion may be rough, ribbed, pocked or flat. The surface typing may be based on one or more of the group of: - greatest diameter; - area of the lesion; - aspect ratio; - thinness ratio; - circularity index; and - compactness shape factor.

The shape may be round, sharp angled or any other variation. In a further embodiment of the invention, wherein the colour variegation is based on one or more of the group of minimum, maximum, average, standard deviation of the skin tone of the lesion.

In an embodiment of the invention, wherein the training record comprises an unexposed image showing the skin area. Providing the neural network with a before and after exposure image may advantageously enhance the training such that the training may be done with less training records or the neural network may be trained to provide a more accurate prediction or both.

In a further embodiment, the unexposed image is taken from a different area normally not exposed to sunlight, such as the inner arm. The unexposed image maybe used to determine the individual melanin content in skin and the minimum erythermal dose of skin. Innate pigmentation is best described in the L* versus b* correlation.

In an embodiment of the invention, wherein the segmenting step and/or the analysing step determines freckles, birthmarks and/or moles on the skin, wherein preferably these parts of the skin are excluded from the predicting step. Colouring due to UV exposure is typically different from surrounding skin. The neural network may be trained to excluded such parts of the skin, such that during the analyses these parts are also excluded such that the prediction may be more accurate. In particular images the lesion may comprise the whole skin area, for example if a large part of the skin is red after sunshine exposure.

In an embodiment of the invention, wherein the image shows reference markers and wherein the reference markers are arranged to the skin area, providing the advantages as specified throughout the text. In an embodiment of the invention, wherein the reference markers are arranged around and/or in close proximity to the skin area, providing the advantages as specified throughout the text. In an embodiment of the invention, wherein the image shows the surface of a reference device, providing the advantages as specified throughout the text.

According to another aspect of the invention, a method for predicting a skin reaction to UV exposure, comprising the steps of: - receiving an image showing a skin area; - analysing the skin area with the use of a trained neural network; and - predicting the skin reaction after UV exposure based on the analyses; wherein the neural network was trained with a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion.

In an embodiment of the invention, wherein the image shows reference markers and wherein the reference markers are arranged to the skin area. In a further embodiment of the invention, wherein the reference markers are arranged around and/or in close proximity to the skin area. In an embodiment of the invention, wherein the image shows the surface of a reference device according to any of the embodiments specified throughout the text.

In an embodiment of the invention, wherein training the neural network comprises the steps of: - segmenting the lesion from the rest of the image; and - extracting lesion parameters from the segmented lesion; - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network.

According to another aspect of the invention, an artificial intelligence system for predicting a skin reaction after UV exposure, wherein the artificial intelligence system comprises a computer system and a computer program product which, when running on the computer system executes the steps of: - receiving a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion; - training a neural network based on the training dataset, wherein the training comprises the steps of: - segmenting the lesion from the rest of the image; - extracting lesion parameters from the segmented lesion; and - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network; wherein the method further comprises the steps of - receiving an image showing a skin area; - analysing the skin area; and - predicting the skin reaction after UV exposure based on the analyses.

According to another aspect of the invention, an artificial intelligence system for predicting a skin reaction after UV exposure, wherein the artificial intelligence system comprises a computer system and a computer program product which, when running on the computer system executes the steps of: - receiving an image showing a skin area; - analysing the skin area based on a trained neural network; and - predicting the skin reaction to UV exposure based on the analyses; wherein the neural network was trained with a training dataset comprising a plurality of training records, wherein each training record comprises:

- an exposed image showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion; According to another aspect of the invention, a computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform the method specified in an embodiment according to the invention.

BRIEF DESCRIPTION OF THE DRAWINGS The invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which: Figure 1 schematically shows a top view of an embodiment of a reference device 100; Figure 2 schematically shows a cross-section of the reference device 100 in figure 1; Figure 3 schematically shows an embodiment of a surface 110 according to the current invention; Figure 4 schematically shows a position reference marker 160 according to the current invention; Figure 5 schematically shows a still image 200 of a surface 110 according the current invention; Figure 6 schematically shows the optional step of recognizing the surface; Figure 7 schematically shows the step of locating the position reference markers 160; Figure 8 schematically shows the step of locating the centres of the reference markers; Figure 9 schematically shows the step of allocating a segment of the colour reference marker as colour reference; Figure 10 shows a model for measuring skin tone; Figure 11 shows the test results of a skin measurement; and Figure 12 schematically shows an embodiment of a computer programproduct, computer readable medium and/or non-transitory computer readable storage medium according to the invention. The figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMERALSowe eee ee Eeen

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS The following figures may detail different embodiments. Embodiments can be combined to reach an enhanced or improved technical effect. These combined embodiments may be mentioned explicitly throughout the text, may be hint upon in the text or may be implicit.

Figure 1 schematically shows a top view of an embodiment of a reference device 100. The reference device comprises a reference element 103. The reference device may further comprise a closing element 101 and an elongated element 102 connecting the closing element to the reference element. The elongated element may be flexible and/or stretchable.

The shown embodiment of the reference device is a bracelet. The closing element goes around a wrist and then under the reference element, where it is attached to the reference element. The reference device may then be used as a regular bracelet. In an embodiment the reference element is provided with a cover. In an embodiment of the closing element, the closing element is couplable to the side of the reference element providing the advantage of reducing the distance between the skin and the reference element.

In an alternative embodiment, the reference device is a clip for on a smartphone. The reference device thereby may predefine the distance between the lens of the camera of the smartphone. This provides the advantage that the image taken by the smartphone has the reference markers in focus. Furthermore, as skin is typically placed in contact with the reference element, the skin is also in focus of this camera.

In an alternative embodiment, the reference device only comprises the reference element. In an alternative embodiment, the reference device comprises thereference element. The reference device may be a separate element from the camera, such a smartphone, taking the image of the reference device positioned on the skin for measuring the skin tone of the skin.

The reference element 103 further comprises a surface 110 having an opening 120, which is used as observation window 121 to observe the skin through the opening when taking an image of the surface and the skin. The reference element may further comprise a raised edge 130. The dashed line II defines the position of the cross-section shown in Figure 2 with the viewing direction along the arrows.

Figure 2 schematically shows a cross-section of the reference device 100 in figure 1. The reference device comprises a reference element 103. The reference element comprises the surface 110 showing the reference markers. The surface typically has a circular outer border and has typically a diameter in the range of 30-15 mm, preferably 27-17 mm, more preferably 24-18 mm.

The surface has an opening 120 for observing skin to be measured. The opening is typically is circular and has typically a diameter in the range of 5-15 mm, preferably 6-12 mm, more preferably 7-10 mm.

The layer carrying the surface is typically very thin for aligning the skin and the surface, such that both are in focus. Preferably the layer is such thin that the skin is flush, substantially flush or almost flush with the surface.

Further, the reference element may comprise a raised edge. The raised edge provides the advantage that light illuminating the surface and opening from the side is prevented or at least minimized. The raised edge has typically a ring shape and has a height in the range of 30-15mm, preferably 27-18 mm, more preferably 20-24 mm. Preferably the inside of the ring has a matt colour, such as matt black, for minimizing the amount of reflection of light, e.g. stray light, coming from the side.

The reference element may comprise closing counter elements 132. The closing counter elements may be magnets, wherein the closing element 101 comprises metal or another magnet for cooperating with the magnets of the closing counter elements for forming coupling for allowing the reference device to be used as clip or bracelet. The magnets are typically positioned about 15 mm apart and centred around the opening 120.

Figure 3 schematically shows an embodiment of a surface 110 according to the current invention. The surface comprises a plurality of reference markers 150. The reference markers may be sub-divided in position reference markers 160 and colour reference markers 151. In this particular embodiment, the surface comprises 2 positionreference markers and eight colour reference markers. The position reference markers may define the relative position of the opening to the position reference markers. The position reference markers may define the relative position of the colour reference markers to the position reference markers.

The surface may further comprise an outer ellipse marking the outer edge of the reference markers. The surface may further comprise an inner ellipse marking the area between the reference markers and the opening. The outer and inner ellipse are typically circles. As the image taken of the reference device for determining the skin tone is typically taken under an angle almost perpendicular to the surface, a circle will typically show up in the image as an ellipse, hence the naming.

The colour reference markers are typically arranged close to the opening, such that light variations are minimal between the opening and the colour reference marker. Furthermore, colour reference markers on opposite sides of the opening may have the same colour for minimizing the influence of a light intensity gradient over the surface area on the measurement.

Figure 4 schematically shows a position reference marker 160 according to the current invention. The position reference marker may comprise four rectangles arranged in four quadrants. The position reference marker comprises two white position reference markers 165 arranged in quadrants diagonal of each other and having a white colour. The position reference marker comprises a black position reference marker 166 and a grey position reference marker 187 arranged in quadrants along the other diagonal and having a black and grey colour respectively. The contrast of the white position reference markers to the black and grey reference markers is high. Preferably the grey colour is selected close to the black and far from white, such that segmentation in the vision algorithm is relatively simple. Furthermore, the colour distance between black and grey is such that the vision algorithm may distinguish between the darker and lighter rectangle with relative ease and high certainty. Furthermore, the use of the grey and black coloured reference marker surfaces is to advantageously allow not only a position detection, but also an orientation detection of the algorithm processing the image comprising the position reference marker.

Figure 5 schematically shows a still image 200 of a surface 110 according the current invention. The still image shows the surface in the middle of the image. The still image may further comprise a rectangular overlay. The rectangular overlay provides guidance to a user holding the camera, such as a smartphone, for getting the surface comprising the reference markers in substantially the correct position.

The still image may further comprise a circular overlay.

This circular overlay may provide the same guidance as the rectangular overlay.

Furthermore, this circular overlay allows the user holding the camera to arrange the overlay over the reference markers, thus between the outer and inner ellipse.

As long as the circular overlay is within these ellipses, the reference markers are generally in the right position.

Further, as long as the circular overlay is within these ellipses, the reference device is held under an angle good enough for the measurement.

If the angle of the reference device is too oblique, the circular overlay will not fit between the inner and outer ellipse.

Hence, having a surface with reference markers having a general circular symmetry and a camera, e.g. smartphone, having a viewer with a circular symmetrical overlay for cooperating with the circular symmetrical surface provides the advantage of arranging the distance between camera and the reference markers and/or arranging the angle between the camera and the reference markers.

The rectangular overlay has typically a width and a height of respectively half the width and half the height of the still image.

The circular overlay is typically centred to the centre of the still image and has a diameter smaller than the lesser of half the height and half the width of the still image.

Figures 6-9 schematically show a surface according to the current invention.

The surface comprises a plurality of reference markers 150. The figures show the typical steps in sequential order of a vision algorithm processing a still image.

The vision algorithm may be part of the measurement of the skin tone.

Figure 6 schematically shows the optional step of recognizing the surface.

The surface comprises an inner ellipse 145 and an outer ellipse 146. The ellipses typically have a high contrast with the surrounding.

Hence, the ellipses may advantageously be detected by the vision algorithm after the image is binarized.

As the ellipses are recognized, the amount of image processed in the vision algorithm may be greatly reduced, enhancing the speed of processing the image.

Figure 7 schematically shows the step of locating the position reference markers 160. As the position reference markers have a high contrast and a particular size, the white position reference markers, the black position reference marker and the grey position reference marker may be easily detected after binarizing and segmenting the image.

The search may be limited, and thus enhanced in processing speed, with the knowledge of the location of the ellipses in step 6. After the white position reference markers, the black position reference marker and the grey position reference marker are found a position reference markercentre 161 is determined. The origin O is defined as the circular symmetrical point of the surface, is located halfway between the two position reference markers. Further, a line v is determined. The line segment v is perpendicular to the line through the position reference marker centres and extends from the origin O.

The vision algorithm then determines an inner ellipse edge 146 and an outer ellipse edge 141 of the inner ellipse and the outer ellipse with the line segment v respectively. A centre ellipse point 148 is halfway between the inner and outer ellipse edges.

Figure 8 schematically shows the optional step of locating the centres of the reference markers 150. The vision algorithm determines the centre ellipse 147. The centre ellipse joins the known points of the two position reference marker centres 161 and the centre ellipse point 148 and is thereby fixed.

Further, the vision algorithm comprises the step of locating the colour reference marker centres 152 based on a predefined marker angle © relative to line segment v around the origin O. The origin O is the centroid of the position reference markers. The origin may also be the centroid of the colour reference markers.

Figure 9 schematically shows the step of allocating a segment of the colour reference marker as colour reference. Based on the line segment v and the origin O, a colour reference marker maximum positive deviation 155, a colour reference marker maximum negative deviation 156, a colour reference marker maximum distance 157, a colour reference marker minimum distance 158.

The segment of the colour reference marker may be used as colour reference in the rest of the vision algorithm. Furthermore, based on the information of this part of the vision algorithm, the opening showing the skin to be measured is located. Based on the colours of the segment of the colour reference markers measured, the vision algorithm may enhance the binarizing and/or segmenting of the skin area observed through the opening.

Figure 10 shows a model for measuring skin tone. The source of the figure is an article titled “Variations in skin colour and the biological consequences of UV radiation exposure” authored by S. Del Bino and F. Bernerd, published in the British Journal of Dermatology. 2013 Oct; 169 Supplement 3:33-40.

The article proposes to use the Lab colour space or CIELab colour space. On the horizontal axis is the yellow chrome or blue-yellow line. On the vertical axis is the luminance. The article proposes further to apply the following formula for differentiating between skin tone or skin colours.

ITA9 = 180. atan =>) 7 b The ITA is the abbreviation for individual typology angle. An ITA angle of over 55 corresponds to the lightest skin with little melanin content and highest sun sensitivity and an ITA angle under -30 corresponds the maximum melanin content and lowest sun sensitivity.

Figure 11 shows the test results of a skin measurement. The population consists of 25 persons. For each person skin of the left and right cheek and the inner arm is measured. The population has different skin types. Compared to figure 10, the horizontal and vertical axis are swapped. The banana shaped contour in figure 10 is a perfect match with figure 11, thereby validating the proposed model.

The minimal erythema dose or minimum erythema dose may be abbreviated with MED. The MED is the shortest exposure to ultraviolet radiation that produces reddening of the skin within 1 to 6 hours and disappears in 24 hours. The minimal erythemal dose is used to calculate the duration of therapeutic exposure to ultraviolet light. For treatment using a “hot” ultraviolet lamp (UV-A or UV-B), the dose is calculated at a distance of 30 inches. The minimal erythemal dose for “cold” ultraviolet (UV-C) is standardized at 30 to 38 seconds at a distance of 1 inch.

Burning, edema, and peeling occur at doses at or above the second degree erythemal dose. Minimal erythemal dose Smallest dose that produces erythema within 1 to 6 hr First degree erythemal Erythema lasts for 1 to 3 days. Some scaling of the Second degree erythemal | Erythema with associated edema, peeling, and Third degree erythemal Severe erythema and burning with associated dose (E3) blistering, peeling, and edema. E3 is approximately 10 times the MED.

Skin tanning or reddening are typical reactions of the skin to sunlight or UV exposure. Basically, one can distinguish between the erythema, the immediate and persistent pigment darkening, and the delayed neomelanization.

Tanning and reddening (erythema) in the L*a*b* colour space can be determined by changes in |*, a* and b* parameters. The erythema converges towards (L~45b~17 a~ 45).

The I*, a* and b* parameters can be graphically represented to analyse the erythema, the immediate and persistent pigment darkening over time and the delayed neomelanization.

Photodermatoses are caused by atypical reactions to the sunlight or UV exposure. The cardinal symptom is severe skin lesions that develop a few hours to several days after sun exposure. Skin tone assessment inside and outside of the lesion is used to determine the presence and the border of the lesion of the skin. For this the image of the region of the skin (region of interest) is typically taken before the UV exposure and after UV exposure. The LAB values of the images before and after exposure are compared and regions with no or little deviations are subtracted, yielding the lesion image. After this, the lesion image is being classified to different types of skin lesions induced by UV light using the database of images and with the help of supervised machine learning.

Figure 12 schematically shows an embodiment of a computer program product, computer readable medium and/or non-transitory computer readable storage medium 1000 having a writable part 1010 including a computer program 1020, the computer program including instructions for causing a processor system to perform a method according to the invention.

It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, an object code, a code intermediate source and an object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub- routines will be apparent to the skilled person. The sub-routines may be stored together in one executable file to form a self-contained program. Such an executablefile may comprise computer-executable instructions, for example, processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub- routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computer-executable instructions corresponding to each processing stage of at least one of the methods set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically.

The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a data storage, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or used in the performance of, the relevant method.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or stages other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures arerecited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Examples, embodiments or optional features, whether indicated as non- limiting or not, are not to be understood as limiting the invention as claimed.

EMBODIMENTS |

1. Reference device (100) for aiding the measurement of a skin tone of a skin area of a skin, comprising: - a surface (110); - an opening (120) in the surface providing an observation window (121) on the skin area when the reference device is placed upon the skin; - reference markers (150) arranged to the opening and/or the surface, such that an image (200) taken for the measurement shows the skin area and the reference markers in substantially the same light conditions and/or under the same angle.

2. Reference device, according to the preceding embodiment, wherein the reference markers are arranged around the opening.

3. Reference device, according to any of the preceding embodiments, wherein the reference markers comprise colour reference markers (151) for colour reference for the measurement of the skin type.

4. Reference device, according to the preceding embodiment, wherein the colour reference markers comprise a surface having a skin colour.

5. Reference device, according to any of the preceding embodiments 3-4 depending on embodiment 2, wherein the colour reference markers form pairs having the same colour and are arranged at opposite sides of the opening.

6. Reference device, according to any of the preceding embodiments 3-5, wherein the colour reference markers form a colour range.

7. Reference device, according to the preceding embodiment, wherein the colour reference markers are evenly distributed over the colour range.

8. Reference device, according to any of the preceding embodiments 3-7, wherein the reference markers comprise position reference markers (160), wherein the opening and the colour reference markers are defined relative to the position and/or orientation of the position reference markers.

9. Reference device, according to the preceding embodiment, wherein the position reference markers have at least partly a high contrast.

10. Reference device, according to any of the preceding embodiments 8-9, wherein the position reference markers are arranged relative to the colour reference markers such that at least four of the centres of the markers substantially form an ellipse.

11. Reference device, according to any of the preceding embodiments 8-10, wherein the position reference markers are arranged relative to the opening such that the centre of the opening is substantially the centroid of the position reference markers.

12. Reference device, according to any of the preceding embodiments, wherein each reference marker comprises: - a circumference line having a colour; and - an enclosed reference surface having a colour different from the colour of the circumference line.

13. Reference device, according to any of the preceding embodiments, wherein at least a part of the reference markers is on the surface and/or wherein the surface is translucent and at least a part of the reference markers is in or behind the surface.

14. Reference device, according to any of the preceding embodiments, comprising a raised edge (130) enclosing the opening and the reference markers.

15. System for measuring a skin tone of a skin area of a skin, comprising: - a reference device (100) according to any of the preceding embodiments; - a mobile device having a camera for taking an image of the surface and the skin area when the reference device is placed upon the skin; and - a processor for providing a skin tone based on the image.

16. Image (200) of the surface of a reference device according to any of the embodiments 1-14, wherein the reference device is placed upon a skin for aiding the measurement of a skin tone of the skin.

EMBODIMENTS II

1. Method for measuring a skin tone of a skin area of a skin, comprising the steps of: - receiving an image (200) showing the skin area and colour reference markers (151) each having a reference colour; - locating of the colour reference markers in the image, wherein the skin area is arranged relative to the colour reference markers; - locating of the skin area in the image based on the colour reference markers; - providing the image in an image format having a luminance component and at least one colour component; - calibrating a reference scale based on the combination of the luminance component and the at least one colour component of the reference colours; and - calculating the skin tone based on the combination of the luminance component and the at least one colour component of the skin area and the reference scale.

2. Method according to the preceding embodiment, wherein the colour reference markers are arranged around and/or in close proximity to the skin area.

3. Method according to any of the preceding embodiments, wherein the colour reference markers are arranged substantially parallel and in close proximity, substantially flush, with the skin area.

4. Method according to any of the preceding embodiments, wherein the received image is in an sRGB format and/or the image format having a luminance component and at least one colour component adheres to the CIELab colour space.

5. Method according to the preceding embodiment, wherein the providing step comprises the steps of: - applying an inverse gamma correction to the received image for providing a gamma corrected image; - applying on the gamma corrected image a conversion for providing a XYZ image having a XYZ colour space;

- applying on the XYZ image a conversion for providing the image adhering to the CIELab colour space.

6. Method according to any of the embodiments 4-5, wherein the at least one colour component is blue-yellow colour component in the CIELab colour space.

7. Method according to the preceding embodiment, wherein the calibrating step and/or calculating step is not based on the green-red colour component in the CIELab colour space.

8. Method according to any of the preceding embodiments, wherein the reference colours of the colour reference markers form a continuous colour scale.

9. Method according to any of the embodiments 1-7, wherein the reference colours of the colour reference markers provide colour samples.

10. Method according to any of the preceding embodiments, wherein the calculating step comprises the step of interpolating between and/or extrapolating from two colour.

11. Method according to any of the embodiments 1-9, wherein the calibrating step comprises the step of segmenting the reference scale in skin tone segments and the calculating step comprises the step of determining that the skin area falls within a certain skin tone segment.

12. Method according to any of the preceding embodiments, wherein the image (200) shows two position reference markers (160), and wherein the step of locating of the colour reference markers comprises the steps of: - locating the two position reference markers; - locating two centres (161) of respective position reference markers, wherein the reference markers are arranged in a ring around the skin area; - determining an origin (O} halfway between the two centres; - determining a line segment (v) perpendicular to the line joining the two centres and extending from the origin; - determining a third point (148) halfway the intersection between the line segment and the ring; and

- allocating a segment (155, 156, 157, 158) of the colour reference marker as colour reference based on the line segment, the origin, a predefined angle (®+6, ©-06) between the line segment and a distance from the origin based on a predefined distance.

13. Method according to the preceding embodiment, comprising the steps of: - recognizing an inner ellipse (145) in the image; and/or - recognizing an outer ellipse (140) in the image; wherein the step of locating the two position reference markers is based on the location of the inner ellipse and/or the location of the outer ellipse.

14. Method according to any of the embodiments 12-13, comprising the steps of: - calculating a centre ellipse through the two centres and the third point; and - correcting the distance between the origin and the segment with the knowledge of the shape of the centre ellipse, the inner ellipse and/or the outer ellipse.

15. Method according to any of the embodiments 13-14, comprising the steps of: - calculating the relative angle of the skin area or reference markers in the image based on the shape of the inner ellipse, the outer ellipse and/or the centre ellipse; and - rejecting the image if the relative skin angle exceeds a predefined limit angle.

16. Computer program product (1000) comprising a computer readable medium (1010) having computer readable code (1020) embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform any of the methods of embodiments 1-15.

17. Device for processing an image (200) for measuring a skin tone of a skin area of a skin, comprising: - a receiver for receiving the image showing the skin area and colour reference markers each having a reference colour; and - a processor for performing the steps of: - locating of the colour reference markers in the image, wherein the skin area is arranged relative to the colour reference markers;

- locating of the skin area in the image based on the colour reference markers; - providing the image in an image format having a luminance component and at least one colour component; - calibrating a reference scale based on the combination of the luminance component and the at least one colour component of the reference colours; and - calculating the skin tone based on the combination of the luminance component and the at least one colour component of the skin area and the reference scale.

18. System for measuring a skin tone of a skin area of a skin, comprising: - a mobile device having a camera for taking an image (200); and - a device according to embodiment 17.

19. Image for a method for measuring a skin tone of a skin area of a skin according to any of the embodiments 1-15, wherein the reference markers are arranged to the skin area.

20. Image according to the preceding embodiment, wherein the reference markers are arranged around and/or in close proximity to the skin area.

EMBODIMENTS III

1. Method for predicting a skin reaction to UV exposure, comprising the steps of: - receiving a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image (200) showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion; - training a neural network based on the training dataset, wherein the training comprises the steps of: - segmenting the lesion from the rest of the image; - extracting lesion parameters from the segmented lesion; and - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network; wherein the method further comprises the steps of - receiving an image (200) showing a skin area; - analysing the skin area with the use of the trained neural network; and - predicting the skin reaction after UV exposure based on the analyses.

2. Method according to the preceding embodiment, wherein the typing of the skin reaction comprises a label from the group of no reaction, sun burned reaction, photo dermatoses reaction.

3. Method according to the preceding embodiment, wherein the photo dermatoses reaction typing comprises a sub-typing in a phototoxic reaction and a photoallergic reaction.

4. Method according to any of the preceding embodiments, wherein the lesion parameters comprise one or more of the group of: - colour variegation; - border typing; - surface typing; and - shape.

5. Method according to the preceding embodiment, wherein the colour variegation is based on one or more of the group of minimum, maximum, average, standard deviation of the skin tone of the lesion. 6 Method according to any of the preceding embodiments 4-5, wherein the border typing is based on one or more of the group of: - border irregularity; - border gradient; and - waviness shape factor.

7. Method according to any of the preceding embodiments 4-6, wherein the surface typing is based on one or more of the group of: - greatest diameter; - area of the lesion; - aspect ratio; - thinness ratio; - circularity index; and - compactness shape factor.

8. Method according to any of the preceding embodiments, wherein the training record comprises an unexposed image (200) showing the skin area.

9. Method according to any of the preceding embodiments, wherein the segmenting step and/or the analysing step determines freckles, birthmarks and/or moles on the skin, wherein preferably these parts of the skin are excluded from the predicting step.

10. Method according to any of the preceding embodiments, wherein the image shows reference markers (150) and wherein the reference markers are arranged to the skin area.

11. Method according to the preceding embodiment, wherein the reference markers are arranged around and/or in close proximity to the skin area.

12. Method according to any of the preceding embodiments 10-11, wherein the image shows the surface of a reference device (100).

13. Method for predicting a skin reaction to UV exposure, comprising the steps of: - receiving an image (200) showing a skin area; - analysing the skin area with the use of a trained neural network; and - predicting the skin reaction after UV exposure based on the analyses; wherein the neural network was trained with a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image (200) showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion.

14. Method according to the preceding embodiment, wherein the image shows reference markers (150) and wherein the reference markers are arranged to the skin area.

15. Method according to the preceding embodiment, wherein the reference markers are arranged around and/or in close proximity to the skin area.

16. Method according to any of the preceding embodiments 13-15, wherein the image shows the surface (110) of a reference device (100).

17. Method according to any of the preceding embodiments 13-16, wherein training the neural network comprises the steps of: - segmenting the lesion from the rest of the image; and - extracting lesion parameters from the segmented lesion; - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network.

18. Artificial intelligence system for predicting a skin reaction after UV exposure, wherein the artificial intelligence system comprises a computer system and a computer program product which, when running on the computer system executes the steps of: - receiving a training dataset comprising a plurality of training records, whereineach training record comprises: - an exposed image (200) showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion; - training a neural network based on the training dataset, wherein the training comprises the steps of: - segmenting the lesion from the rest of the image; - extracting lesion parameters from the segmented lesion; and - providing each training record together with the associated extracted lesion parameters to the neural network for training the neural network; wherein the method further comprises the steps of - receiving an image (200) showing a skin area; - analysing the skin area; and - predicting the skin reaction after UV exposure based on the analyses.

19. Artificial intelligence system for predicting a skin reaction after UV exposure, wherein the artificial intelligence system comprises a computer system and a computer program product which, when running on the computer system executes the steps of: - receiving an image (200) showing a skin area; - analysing the skin area based on a trained neural network; and - predicting the skin reaction to UV exposure based on the analyses; wherein the neural network was trained with a training dataset comprising a plurality of training records, wherein each training record comprises: - an exposed image (200) showing a skin area comprising a lesion after UV exposure; and - concomitant information comprising at least the amount of UV exposure of the skin before the image was taken and a typing of the lesion;

20. Computer program product (1000) comprising a computer readable medium (1010) having computer readable code (1020) embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform any of the methods of embodiments 1-17.

Claims

Conclusions

A reference device (100) for assisting in measuring a skin tone of a skin area of a skin, comprising: - a surface (110); - an opening (120) in the surface for providing an observation window (121) on the skin area when the reference device is placed on the skin; - reference marks (150) arranged at the aperture and / or the surface so that an image (200) taken for measurement shows the skin area and marks in substantially the same light conditions and / or at the same angle.

A reference device according to the preceding claim, wherein the reference marks are arranged around the opening.

A reference device according to any one of the preceding claims, wherein the reference marks comprise color reference marks (151) for color reference for measuring skin type.

Reference device according to any of the preceding claims, wherein the color reference marks comprise a surface with a skin color.

A reference device according to any one of the preceding claims 3-4 when dependent on claim 2, wherein the color reference marks form pairs of the same color and are arranged on opposite sides of the opening.

Reference device, according to any of the preceding claims 3-5, wherein the color reference marks form a range of colors.

Reference device according to the preceding claim, wherein the color reference marks are evenly distributed over the color range.

A reference device, according to any of the preceding claims 3-7, wherein the reference marks include position marks (160), the aperture and the color reference marks being defined relative to the position and / or orientation of the position reference marks.

Reference device according to the preceding claim, wherein the position reference marks have at least partially high contrast.

A reference device, according to any of the preceding claims 8-9, wherein the position reference marks are arranged relative to the color reference marks so that at least four of the centers of the marks essentially form an ellipse.

A reference device, according to any of the preceding claims 8-10, wherein the position reference marks are arranged relative to the opening so that the center of the opening is substantially the center of gravity of the position reference marks.

Reference device, according to any one of the preceding claims, wherein each reference mark comprises: - an outline with a color; and - an enclosed reference surface has a color that is different from the color of the outline line.

A reference device as claimed in any preceding claim, wherein at least some of the reference marks are on the surface and / or wherein the surface is transparent and at least some of the reference marks are in or below the surface.

A reference device, according to any preceding claim, comprising a raised rim (130) enclosing the opening and the reference marks.

System for measuring a skin tone of a skin area of a skin, comprising: - a reference device (100) according to any one of the preceding claims; - a mobile device with a camera for taking images of the surface and the skin area when the reference device is placed on the skin; and

- a processing unit for providing a skin tone based on the image.

An image of the surface of a reference device according to any one of claims 1 to 14, wherein the reference device is disposed on a skin to aid in the measurement of a skin tone of the skin.