WO2015097239A1

WO2015097239A1 - Method of analysing facial images for detecting a flush effect

Info

Publication number: WO2015097239A1
Application number: PCT/EP2014/079227
Authority: WO
Inventors: Laurent Petit
Original assignee: Galderma Research & Development
Priority date: 2013-12-23
Filing date: 2014-12-23
Publication date: 2015-07-02
Also published as: FR3015741A1; US10269113B2; EP3087551B1; EP3087551A1; US20170076444A1; FR3015741B1

Abstract

Method of analysing images of the face of a human being comprising the following steps: - the acquisition of at least one image of the face is carried out so that the nose, the eyes and the cheeks are aligned on pre-existing markers, - the components of the image in at least one zone of the face are determined, - the components are compared with stored values, and - the presence of a flush effect is determined if the components are greater than the stored values.

Description

Method for analyzing facial images for detecting a flush effect

The technical field of the invention is skin analysis, and more particularly the analysis of the evolution of the colorimetric parameters of the skin.

The subject of the invention is a method for analyzing images of the face of a human being, comprising the following steps:

acquiring at least one image of the face so that the nose, the eyes and the cheeks are aligned with pre-existing markers,

the components of the image are determined in at least one area of the face,

the components are compared to stored values, and the presence of a flush effect is determined if the components are greater than the memorized values.

The areas of the face may be cheeks.

The components can be the color components L * (clarity), a * (red / green axis), b * (yellow / blue axis).

The stored values can be the values of the colorimetric components determined at an earlier date for the same face.

The presence of a flush effect can be determined if the color component associated with the red color is greater than the stored value associated with the red color.

Images can be acquired via a smart phone or smartphone.

Furthermore, the verticality of image acquisition can be provided by an internal sensor of a smart phone, for example a three-axis gyroscope.

The acquisition can be performed while the subject is holding a colorimetric chart through the mouth, in order to correct the colorimetric components. At least one parameter chosen from temperature, atmospheric pressure, ambient luminosity and hygrometry can be acquired in order to correct the colorimetric components of an effect of the environment on the face.

Advantageously, the values of the determined colorimetric components can be configured for self-evaluation of skin diseases such as melasma, nevi, rosacea, erythema, reddening, acne, psoriasis, dermatitis, actinic keratosis, rash and seborrheic dermatitis.

Other objects, features and advantages of the invention will become apparent on reading the following description, given solely by way of nonlimiting example and with reference to the appended drawings in which:

FIGS. 1a and 1b illustrate examples of photographs acquired with two different acquisition systems, FIGS. 2a, 2b, 2c, 3a and 3b illustrate the operation of an acquisition software,

FIGS. 4a, 4b and 4c illustrate colorimetric patterns, FIGS. 5a and 5b illustrate two views of a colorimetric chart support,

FIG. 6 illustrates the zones analyzed in a photograph, FIGS. 7, 8 and 9 respectively illustrate the evolution of colorimetric parameters L *, a * and b * as a function of time, over 20 days,

FIGS. 10, 11 and 12 illustrate the evolution of each colorimetric parameter for the average of the two cheeks of an individual, per day,

FIGS. 13 to 15 illustrate the evolution of the set of colorimetric parameters independently for each cheek,

Figures 16 to 18 illustrate the overall colorimetric evolution of the face (right cheek and left cheek combined), Figures 19a, 19b, 20a and 20b illustrate the evolution of the skin color under the effect of tanning, Figures 21, 22 and 23 illustrate the color evolution of each cheek,

Figures 24, 25 and 26 illustrate the overall colorimetric evolution,

FIGS. 28a and 28b illustrate photos taken with a very high level of ambient light,

Figure 29 illustrates the evolution of the hygrometry rate over time,

FIG. 30 illustrates the evolution of the temperature over time,

FIG. 3 illustrates the evolution of the atmospheric pressure over time,

Figure 32 illustrates the differences in ambient brightness observed on the acquisitions of another person, Figure 33 illustrates the rate of hygrometry as a function of time on the acquisitions of another person,

Figure 34 illustrates the temperature versus time on the acquisitions of another person,

Figure 35 illustrates the evolution of the atmospheric pressure over time on the acquisitions of another person, Figure 36 illustrates the evolution of the ambient luminosity, Figure 37 illustrates photographs acquired with an ambient brightness exceeding 4000. lux,

FIGS. 38a, 38b, 39a and 39b illustrate the influence of the vertical inclination during the acquisition of a photograph,

Figures 40a and 40b illustrate two remote and connected objectives;

FIGS. 41 a and 41 b illustrate two phases of data acquisition for the test of a data acquisition software application.

FIGS. 42a and 42b are two photographs of a suj and taken during the phases of FIGS. 41a and 41b, respectively in the morning and during a flush effect, and

RECTIFIED SHEET (RULE 91) ISA / EP FIG. 43 illustrates the evolution of the number of flushes during the day recorded by the acquisition software according to the invention.

Firstly, a technical protocol will be described allowing the analysis of the flush effect or brutal redness on a face. The aim is to quantify the change in the color of the face, which becomes redder during this flush effect, the aim being to conduct a clinical study to show the reduction of these effects with the taking of a specific active ingredient. Advantageously, this study can be used for a self-evaluation of skin diseases such as melasma, nevus, rosacea, erythema, reddening, acne, psoriasis, dermatitis, actinic keratosis, rash and seborrheic dermatitis.

A first study allowed to define a most optimal portable system for the acquisition of photos in portrait mode. It was decided to use two separate camera systems, namely a first and a second camera respectively marketed by Samsung under the trademarks:

- Samsung Galaxy S4 ®

- Samsung Galaxy S4 Mini ®

The aim is to characterize the two chosen acquisition systems and to validate the acquisition sensitivity according to the illumination conditions and the environment of the shooting.

After a description of the specifications of the two systems and the acquisition software subsequently designated by the term "SmartCam" developed specifically for this application, the acquisition protocol and the methods of analysis will be detailed.

The results, the problems encountered and the solutions and recommendations envisaged will then be detailed.

A practical use of the embedded software application in the camera systems will finally be described with reference to FIGS. 41a, 41b, 42a, 42b and 43 in the context of treatment of a flush with Brimonidine. To make all the acquisitions, as indicated above, two smartphones or smartphone, in French, namely the phone marketed under the Samsung Galaxy S4 brand, as well as a Samsung Galaxy S4 Mini phone that has a smaller footprint but performance a little weaker than the Samsung Galaxy S4. With the second S4 mini phone, the resolution of the camera is slightly lower and all sensors for humidity, temperature, hygrometry are not present.

The features of the first device (Samsung Galaxy S4) are:

Dimensions 136.6 x 69, 8 x 7.9 mm

Weight 130 g

Front-End Camera Resolution 2, 1M Pixel (1920x1080)

Dorsal camera resolution 13 M Pixels

Android operating system

Phone Yes

GPS Yes

Proximity Sensors, Ambient Light,

Thermometer, Hygrometer, Barometer, Accelerometer, Three Axis Gyroscope, Magnetometer

The features of the second device (Samsung® Galaxy S4 Mini) are:

Dimensions 124.6 x 61, 3 x 8, 9 mm

Weight 107 g

Front-End Camera Resolution 1, 9 M Pixels (1392x 1392)

Resolution dorsal camera 8 M Pixels

Android operating system

Phone Yes

GPS Yes

Proximity Sensors, Accelerometer, Three Axis Gyroscope, Magnetometer In order to compare the two systems, a series of photos were taken under the same lighting conditions and at a few seconds intervals with the Samsung Galaxy S4 and the Samsung Galaxy S4 Mini. Figures la and lb show examples of photos acquired with both systems, the figure being acquired with the Samsung Galaxy S4 Mini (1393x1393) and Figure lb being acquired with the Samsung Galaxy S4 (1080x1920).

We then observe that the resolution of the Galaxy S4 Mini is square (1393x1393 pixels) and requires to position a little further from the lens compared to the Galaxy S4 (1080x1920 pixels).

Positioning farther from the lens results in a reduction of optical distortions (commonly referred to by those skilled in the art under the acronym "fish-eye effects") which are slightly visible with the Samsung Galaxy S4 . The analysis of photos being focused on non-morphological analyzes, this has no negative effects. By cons, we note that in the same acquisition environment, the Galaxy S4 takes pictures brighter and that the gloss is also more clearly visible on the skin. The images are also more contrasted with the Galaxy S4 compared to those acquired with the Samsung Galaxy S4 Mini.

And the details are also more visible with the Galaxy S4 than on the Galaxy S4 Mini.

In addition, a data acquisition software, embedded in the phones allows in particular:

- to define, at the beginning of the study, two horizontal axes on the photo in "live" mode (real time, in French language) allowing to align the eyes and the mouth of the subject, to allow the fast repositioning of the subject to using these two horizontal axes and a vertical axis,

to define a vertical inclination indicator making it possible to limit the variability of image acquisition angle, to store the images acquired in high definition (maximum resolution of the camera) in JPEG format, and to store the data relating to each image taking as the date and time but also the information of the various sensors: ambient brightness, atmospheric pressure, temperature, hygrometry and vertical inclination.

When launching the software, the "live" mode is displayed directly with the horizontal axes and the vertical axis (figure 2a). These axes were defined in the software administration area at the beginning of the study.

The user positions himself in front of the camera and optimally adjusts his face to align with the marks.

Once the subject is well positioned, it presses on the screen to take the picture. A new window appears with the photo acquired in full resolution and the user can choose to keep it or cancel it (Figure 2b).

The software application can also display a tilt indicator on the interface (in the middle right) of the software in "live" mode to force the correct positioning of the device vertically (green zone of the indicator tilt) at the time of imaging (Figure 2c).

When the photo is validated, it is saved in the "Pictures / SmartCam" folder of the camera and the related data is stored in the database.

All of this data is available to the user (Figure 3a). We then find the list of all the photos acquired with date, time, ambient brightness and on the devices allowing it: temperature, humidity and atmospheric pressure.

The user can also view photos in full screen with a dedicated interface (Figure 3b) with the ability to zoom in on the photo to see the details.

In the administration part of the software, it is possible to export all the data stored in the database to generate a formatted text file for easy import into Microsoft Excel ®.

In order to quantify the illumination drifts and to correct them according to the different acquisition environments, three specific patterns (figures 4a, 4b and 4c) are used to implement a feasibility study:

- a digital test chart 30 40x 12 mm color patches with 4x4 mm patches (Figure 4a),

- a uniform gray target (Figure 4b), and

a test pattern with 30 Munsell patches in 40x 12 mm format with 4x4 mm patches (Figure 4c).

A specific CAD support (FIGS. 5a and 5b) is also used to allow the user to position the sight in front of his mouth in a reproducible manner.

The data acquisition protocol must allow mainly:

validate the acquisition of the "SmartCam" acquisition software,

to validate the quality (resolution, color rendering) of the resulting images,

to study the performance of the different colorimetric patterns, and

to analyze the variability of acquisition according to the environment.

The acquisition system should be placed as vertically as possible in front of the face.

The face must cover the entire field of acquisition to maintain a good resolution.

During the first image capture, the user must define the repositioning guides by moving red and green sliders on the eyes and mouth.

Repositioning guides should not be modified during the study. It is nevertheless possible to move the axes (Parameter menu at the bottom right) at the very beginning of the study if they are not correctly positioned.

Shooting is carried out according to the following procedure:

position the sight in your mouth by holding it by the groove with the teeth

on the "SmartCam" acquisition software home screen display: "position yourself as best as possible to be centered in relation to the vertical axis and relative to the horizontal guides of the mouth and eyes", click anywhere on the image to take the picture a window appearing with the acquired photo, validate if it is correct (no blur, good positioning) or cancel to start again.

The acquired photos are not erasable from the "SmartCam" software.

The acquired photos can be consulted by clicking on the button at the bottom left of the home screen. The list of all the acquisitions appears and it is possible to click on an element to see the photo in complete resolution on the screen.

It is necessary to make sure that the sight is present on each photo (otherwise it will be unusable).

At least these steps should be performed at least for 10 consecutive days. The following images should be taken under normal conditions (no particular effort before) and if possible at fixed times and in the same environment:

3 acquisitions in a row with the target in the morning (up) 3 acquisitions in the afternoon with the test pattern at noon (if possible in the same environment as the morning) 3 acquisitions in the evening with the test pattern at night (bedtime). Other images can be taken:

during the day and in different places outside and inside always with the test pattern and always 3 times in a row to analyze the reproducibility of the system, and in special conditions (after the sport, after a time of exposure to the sun, ...)

Following all the acquisitions made, the following photos are available:

5 1 photos of a first individual taken with a Galaxy S4 Mini distributed as follows:

o 17 pictures with 3 repetitions ie 5 1 pictures with the digital chart;

168 photos of a second individual with a Galaxy S4 distributed as follows:

o 28 photos with 3 repetitions ie 84 photos with the digital target, and

o 28 photos with 3 repetitions ie 84 photos with the gray test pattern;

450 photos taken with a Galaxy S4 distributed as follows:

o 50 photos with 3 repetitions or 150 photos with the digital target,

o 50 photos with 3 repetitions or 150 photos with the gray chart, and

o 50 photos with 3 repetitions or 150 photos with the Munsell sight.

A total of 669 photos to analyze.

In order to define which pattern is the most efficient, the relative differences of the 3 components L *, a * and b * are analyzed on a set of 50 acquisitions with 3 repetitions each time. The photos were acquired with the first device, a Samsung Galaxy S4.

Each photo was corrimated by colorimetric registration algorithms with adjustment of the convergence constraints given the variations between each shooting condition which are very important.

For the 30-patch test, multi-patch calibration algorithms are used while for the gray test, a "white point compensation" technique is used. The results of the photoradiometric recalibration are compared with each test pattern on the L *, a * and b * components of a defined area on each cheek. The standard deviation of the mean values of the 2 cheeks (Table 1) is calculated on the 3 repetitions after colorimetric registration.

Table 1: Mean relative standard deviation of the 2 cheeks with 3 repetitions of

50 acquisitions for each type of target

It can be seen that the relative differences (Table 1) between 3 repetitions after corrorometric correction are of the same order of magnitude regardless of the pattern used. The variations observed seem less important with the digital chart 30 patches. Indeed, it has a slightly glossy coating which can optimize the color correction.

We will study below the performance of the colorimetric correction (Table 2) on all 30 patches of the digital chart on the 150 photographs acquired (50 photos with 3 repetitions) with this pattern.

Table 2: Mean Variations of the 30 Digital Sample Patches Before and After Color Correction on 150 Photos

There is a significant decrease in all L *, a * and b * colorimetric parameters.

Nearly 50% of the average colorimetric variability is optimized for all shades. All the photos analyzed in the rest of this study will be those that are recalculated compared to the digital chart 30 patches. It will now be described the reproducibility of the acquisition with the corrimetric registration. Ten successive acquisitions of views are made with a digital colorimetric chart. Each acquisition is made in the same environment but the pattern is removed and repositioned between each acquisition.

The results presented in the following table 3 therefore take into account the repositioning variability of the test pattern.

Table 3: Average variations on the colorimetric parameters of the cheeks on 10 photos taken successively in the same

environment.

It can be seen that the variabilities are substantially identical for the right cheek and the left cheek. The variability is greater on the clarity component L * than on the other components a * and b *.

The results show that the accuracy of the measurement will be of the order of +/- 1.5 units in L * and of the order of +/- 1 units in a * and +/- 0.5 units in b * for acquisitions made in the same environment and under the same lighting conditions.

The variabilities on each L *, a * and b * color component were also analyzed for all repetitions regardless of the acquisition environment for each suj and (Tables 4a, 4b and 4c).

Table 4a: Variations on the L * colorimetric parameter of the cheeks (right and left) over 3 repetitions on all the acquired photos.

Table 4b: Variations on the colorimetric parameter a * of the cheeks (right and left) over 3 repetitions on all the acquired photos.

Table 4c: Variations on the colorimetric parameter b * of the cheeks (right and left) over 3 repetitions on all the acquired photos.

Note that overall average variabilities are relatively good for each component. Nevertheless, relatively large maximum variabilities are observed, especially on the L * component, which are mainly due to the variations of illumination at each acquisition condition.

We will now analyze the colorimetric evo lution on the cheeks of each of the 3 subjects (Individuals 1, 2 and 3) over time.

First, we will analyze the global time effect over 20 days.

This analysis was performed on the photos taken by the individual 1 for 20 days with the second Galaxy S4 mini camera with a total of 5 1 photos.

The colorimetric data are extracted on both cheeks after colorimetric registration with the 30 patches digital test pattern.

The analysis zones were manually defined on each photo as shown in Figure 6.

In order to analyze the time effect, we calculate the average of the data acquired every day. This allows in particular to optimize the signal to noise ratio. Figures 7, 8 and 9 show the evolution of the parameters L *, a * and b * respectively as a function of time over 20 days.

We then notice strong gaps between the right cheeks and left in clarity (L *) especially on the 12th day. This is mainly due to drifting lights or shadows worn on the face when taking pictures.

The parameter a * (green / red axis) changes in the same way for the 2 cheeks. The parameter b * (blue / yellow) has, meanwhile, strong variability.

If we study the general evolution of each parameter, we see a very different evolution from the 18th day with a significant decrease in the parameters L * and a * and a significant increase in the parameter b *.

The same trends can be found in the graphs in Figures 10, 11 and 12, which show the evolution of each colorimetric parameter for the average of the two cheeks per day.

We will now analyze the time effect over 17 days.

This analysis was performed on the photos taken by the second individual for 17 days with the first Galaxy S4 device with a total of 84 photos.

The colorimetric data are extracted on both cheeks after colorimetric registration with the 30-patches digital test pattern.

The analysis zones were manually defined on each photo as for the previous analysis.

Figures 13 to 15 show the evolution of the set of colorimetric parameters independently for each cheek. Data was averaged for each day.

As illustrated in Figure 13, we see that the first day, the 2 cheeks are very differently exposed with very different brightness values (20 units of deviation). For the other analysis times, the right cheek is always more exposed than the left cheek but in much smaller proportions (5 to 10 units of difference). The parameter a * (red / green axis), shown in Figure 14, appears to be identical between the 2 cheeks in the course of time except at the first beat.

As for the parameter a *, the parameter b * (yellow / blue axis), illustrated in FIG. 15, is relatively identical regardless of the side of the analyzed face except for the first time where the left / right differences are very important.

We will now study the overall colorimetric evolution of the face (right cheek and left cheek combined), illustrated in Figures 16 to 18.

If we consider that the first time corresponding to a problem of acquisition (related to inhomogeneous lighting on the 2 cheeks), we observe a quasi stability of the different colorimetric parameters L *, a * and b * over time with differences of only a few units.

These results show that the skin of the second individual did not evolve colorimetrically during the 17 days of analysis.

We will now analyze the tanning effect on the measurements.

This analysis was performed on the photos taken by the second individual for 14 days with the first Galaxy S4 device with a total of 150 photos.

Analysis areas have been manually set on each photo.

The evolution of the color of the skin is visible in the photos acquired as shown in Figures 19a, 19b, 20a and 20b.

At first, we will focus on the colorimetric evolution on the 2 cheeks in the same light environment. The photos were acquired in front of a fluorescent tube ceiling light in a closed room at the same time every day for 12 days.

The orientation of the face with respect to the luminous ceiling light has not been controlled in order to study the measurement variabilities as a function of the position of the face with respect to the illumination. Three photos were acquired each day and the following graphs show the results obtained during 12 days of analysis.

Through Figures 2 1, 22 and 23, we see that the left cheek is generally more overexposed than the right cheek but the changes are similar for the 3 parameters.

The overall evolution of the skin on the 2 cheeks at the same time and for all the lighting conditions and all the shots acquired per day will now be examined. Each day, the average of all the data relating to the acquired photos is averaged in the morning, afternoon and evening in different luminous environments. The following graphs then show the evolution of the L *, a * and b * colorimetric parameters for each day.

In FIGS. 24, 25 and 26, it can be seen that the brightness parameter L * decreases by more than 10 units in 12 days by averaging all the measurements made each day, ie a minimum of 9 photos per day under 3 conditions. different lighting each time. This shows that the skin has darkened and thus has an evolution of tanning over time.

The most important changes between the 2nd and the 3rd day and between the 13th and the 14th day correspond to strong sun exposure these days.

In addition, the redness parameter (a *) increases by more than 4 units over the 12 days of analysis, which shows a reddening of the skin. The b * parameter remains almost constant throughout the study.

As previously mentioned, the "SmartCam" data acquisition software can record data for the different sensors for each acquisition. We will now study the influence of the data of these different sensors on the acquisition of the images and the results obtained previously, and first analyze the different sensors over a period of 12 days

The graph in Figure 27 shows the data acquired by the third individual for 12 days with a the first device, the Samsung Galaxy S4 phone. Figures 28a and 28b show photos with a very high level of ambient light, respectively 2094 lux and 912 lux.

We will now study the other data as a function of time.

Figure 29 illustrates the evolution of the hygrometry rate over time. The hygrometry rate is very variable depending on the location of the image. In water rooms (bathroom for example) the humidity can be very high. No correlation between the ambient moisture content and the gloss (L *) in the acquired photos has been identified.

Figure 30 illustrates the evolution of temperature over time. Ambient temperature can be an important factor for the sensation of the heat effect during the flush effect. This data could be used in addition to the heat felt. On the other hand, in the framework of a feasibility study, no correlation between the temperature and the evolution of the colorimetric parameters was observed, the temperature remaining relatively stable during the study (between 23 ° and 32 °) VS).

Figure 31 illustrates the evolution of atmospheric pressure over time. The atmospheric pressure shows in particular a change of location at the beginning of the study but does not allow a correlation with the measured colorimetric values.

The different sensors will now be analyzed over a period of 15 days

The data was acquired by the second individual for 15 days with the first device, a Samsung Galaxy S4.

The differences in ambient brightness observed on the acquisitions of photographs, illustrated in FIG. 32, are not a source of drift during the acquisition. The variations of 260 lux maximum remain very small.

The hygrometry rate, shown in Figure 33, varies with time with an overall average of 54%. Any correlation between these values and the evolution of the colorimetric parameters has not been identified.

The temperature, illustrated in FIG. 34, is also relatively stable and could present data of interest for the clinical study. It does not influence, in this study, significantly on the color of the skin of the cheeks.

Again, the atmospheric pressure, shown in Figure 35, varies slightly but is not an indicator that can be correlated with skin color.

The data was acquired by the first individual for 20 days with the second device, a Samsung Galaxy S4 Mini. This unit has only a gyroscope and a brightness sensor.

We will focus here only on the data relating to the ambient brightness as shown in the graph of Figure 36.

It should be noted that some photos were acquired with an ambient luminosity exceeding 4000 lux. Figure 37 below presents a picture of these acquisitions.

The inclination of the device will now be studied.

At each acquisition, information about the inclination of the device is stored in the database associated with the "SmartCam" acquisition software.

After analyzing all the inclination data of the apparatus at each acquisition, shown in the following table 5, it can be seen that the median value is relatively constant between the subjects. However, the acquisition angles can vary relatively significantly with a maximum deviation of 30 °.

Table 5: Angles of inclination of shots. These variations of the angle of inclination can modify the zone of analysis on the cheeks but also the geometry of illumination with respect to the axis of shooting as shown in FIGS. 38a, 38b, 39a and 39b. The photograph of figure 38a is taken with a vertical inclination of the apparatus at 73 ° while that of figure 38b is acquired with a vertical inclination of 105 °. The photograph of Fig. 39a is taken with a vertical inclination of the apparatus at 66 ° while that of Fig. 39b is acquired with a vertical inclination of 96 °.

In order to better control the orientation of the shooting, it will be necessary to activate the tilt indicator (figure 2c) via the administration part of the software allowing to constrain the angle of acquisition, between 85 ° and 95 ° for example, at the time of the image taking to limit the variability related to the angle of acquisition.

In view of the foregoing, both type acquisition systems provide good quality images for overall colorimetric analysis. The resolution of the images is satisfactory with a rectangular field for the S4 and square for the S4 Mini which limits the resolution if one wants to have the complete face in the field.

We will therefore favor the first device that also has a larger screen for real-time viewing at the time of acquisition.

Note, however, a relatively large effect of JPEG compression on the resulting photos during high magnifications. These systems do not give access to the raw data (RAW format) of the photos at the sensor output and impose the JPEG format with low compression.

Regarding the software application, it is relatively intuitive and has all the desired features.

Advantageously, a control of the ambient luminosity by means of a brightness sensor integrated in a smart phone to avoid overexposure of a particular area on the face or lighting conditions too far from the conditions

RECTIFIED SHEET (RULE 91) ISA / EP initials is set up. A message will appear on the screen to warn users if the brightness condition is unsatisfactory.

However, it will have to evolve to integrate other functionalities related to the clinical study.

Several optimizations have already been envisaged:

addition of an ambient light control to prevent overexposure of a particular area on the face or lighting conditions too far from the initial conditions,

adding a control of the acquired image to validate the sharpness and thus limit the blur of moving,

addition of a validation of the colorimetric coherence between different areas of the face (2 cheeks and forehead) to try to limit at best the drifts of illumination,

adding one or more reference areas on the face to T0 to try to limit the drifts from one acquisition to another, and

setting up a white screen with maximum brightness when shooting to try to get a relatively bright light from the camera.

In addition, the 3 test patterns are suitable for the step of photoradiometric registration of the images. However the numerical chart seems the most appropriate because it allows to adjust not only the color temperature, as with the gray chart, but also to optimize certain specific tints.

The Munsell pattern, although very diffuse, does not perform better than the pattern made in digital printing given the large variations in lighting.

The focus of sight made for this study seems to be suitable for a good repositioning in front of the subject. Its dimensions can be optimized if necessary.

The main problem for colorimetric registration is that the color variations on the test pattern do not take into account all the variations on the face because of the too important differences in lighting geometry. This is why we must try to constrain the best light environment when taking pictures. The ideal would be to have very uniform uniform lighting.

We will now proceed to the analysis of the colorimetric data.

The colorimetric chart allows to optimize by almost 50% the color drifts between each shot.

After image registration, the repeatability of the system is of the order of 1.5 units in L * and about 1 unit in a * and b * under uniform illumination conditions.

In non-uniform lighting conditions in the facial area, the variation in geometry from one image to another may result in gaps of 7 units in L * and b * and 4 units in a * maximum . These variations are all the greater if one changes location and type of lighting.

However, it is possible by averaging all the data to obtain interesting results on the colorimetric evolution of the skin as the effect of tanning over time.

As part of the analysis of the flush effect, it will be necessary to multiply acquisitions to optimize the signal-to-noise ratio.

Several solutions are to explore:

use the magnetometer to force the user to change orientation and thus change the lighting geometry, - use specific colorimetric parameters such as the dH (difference of chromatic tone) which does not take into account the clarity L *,

use shadow-like invariant parameters such as normalized RGB space or L1L2L3 specific spaces, use the parameter a * which seems less influenced by the luminous environment and which takes into account the redness, and

use reference areas on the skin such as the chin or forehead to perform a contrast calculation between "healthy" skin and "flush-effect" skin.

In addition, the colorimetric data can be advantageously analyzed for a self-evaluation of skin diseases such as melasma, nevus, rosacea, erythema, redness, acne, psoriasis, dermatitis, actinic keratosis, rash and seborrheic dermatitis.

It is also possible to use various types of sensors provided in the cameras.

The temperature, pressure and humidity sensors present in the first telephone make it possible to extract interesting information but do not necessarily help to stabilize the image acquisition or to control the acquisition environment.

On the other hand, the brightness and tilt sensors (gyroscope), which are also present in the second device, can help to constrain the image taking.

Thus, the various sensors of the cameras are used to acquire parameters, especially on the shooting conditions. The "SmartCam" acquisition software application is then programmed to correct the colorimetric components of the effect of the parameters measured on the face.

In other words, it will be possible to integrate a real time preview of these sensors at the time of the image taking to constrain the user to correctly position his device or to change location because the lighting atmosphere is much too much important (greater than 500 lux for example).

In order to optimize image capture in very high resolutions, other lenses such as those marketed under the Sony ® brand can be used. These connected lenses connect to Wi-Fi on an Android smartphone or an iPhone® and allow you to take excellent shots. Currently, the JPG format is the only supported, the RAW will arrive later.

The QX10 lens is made up of an 18.2-megapixel Exmor R CMOS sensor, plus a 3.6 x Cari Zeiss® zoom lens. Optics with an aperture of 3.5 to 3.9 with a microphone and shutter button. The QX100 features a 20.2-megapixel CMOS sensor, 3.6x f / 1.8 optical zoom, a microphone, a dedicated shutter button, and two rings for manual adjustments.

The lenses fit on the smartphone and a specific Sony® application can take snapshots. The smartphone is used to control the photo lens. It will be necessary to know if it is possible to control these objectives with an external application. Their price is between 250 and 450 US dollars.

In terms of the various features of these Sony lenses, we can find various modes of fixation, Wi-Fi and NFC connections, Android and iOS compatibility, direct display of images on the screen of the connected terminal, simultaneous backup on the objective and the connected smartphone, the manual controls from the smartphone which allows to drive the objectives directly from the connected terminal, the recording of videos in Full HD 1080p.

The Sony QX10, shown in Figure 40a, is built around a 1 / 2.3-inch Exmor CMOS photo sensor with an 18-megapixel resolution, and its stabilized zoom lens covers the focal range of 25-250mm. apertures of f / 3.3-5.9 For the image processing, it benefits from the BIONZ processor.The sensitivity ranges from 100 to 12800 ISO depending on the shooting modes. 6.24 x 6.18 x 3.33 cm for a weight of 90 grams.

The Sony® QX100, shown in Figure 40b, is built around a 1 "CMOS Exmor photo sensor with a definition of 20 megapixels and 3.6x stabilized (Carl Zeiss®) optics covering the focal range. -100 mm Its openings are at f / 1.8- 4.9. The QX 100 also benefits from the BIONZ image processing processor. Its sensitivity ranges from 160 to 25600 ISO. Its dimensions are 6.25 x 6.25 x 5, 55 cm for 179 grams.

Finally, reference will be made to FIGS. 41a, 41b, 42a, 42b and 43 for a practical use of the software application embedded in the camera systems in the context of treatment of a flush with the drug. brimonidine.

In order to test the software application embedded in the phones, a clinical trial for a Brimonidine flush treatment was performed, during which shots were taken using the "SmartCam" acquisition software.

This Ha phase study, generally designed to evaluate assay needs, has two periods. To account for variations between individuals, this study was designed to improve statistical power in the first period and a study plan was planned in the second period. Advantageously, it is a randomized, double-blind, randomized, placebo-controlled study.

The first period is performed in a treatment center and consists of three sessions. It lasts a week.

The second period lasts four weeks and is conducted by a suj and at home. It serves to validate the blush pattern and identify problems or difficulties.

24 subjects with ETR (Erythematotelangiectatic rosacea) and 12 subjects with PPR (Papulopustular rosacea) participated in this study.

The complete study lasts approximately 13 weeks. There is a subjective participation of 5 weeks and a screening period of up to 4 weeks for each subject.

The first period of the study based on the flush pattern (redness) is performed in a treatment center. This period aims at making an objective evaluation of the intra- individual variability on each subject.

In this first week of the study, we set up tests on half-faces of each suj and with two treatments different: Vehicle (placebo equivalent, non-active) and Brimonidine 0.5% (product to be tested, active).

As illustrated in FIG. 41a, the vehicle is applied to the two half-faces of the subject in the first day. Two days later, treatment with Brimonidine 0.5% is applied to one of the two half-faces of the subject. On the fifth day, all vehicles are replaced by Brimonidine 0.5%.

The second period of the study is conducted at home by each subject via a smart phone. This is a subjective assessment based on trials in two parallel groups (Figure 41b).

The second period lasts 4 weeks. A first group of subjects implements a Brimonidine-based treatment on the complete face and the placebo is applied for a second group of subjects during the first two weeks. In the next two weeks, both groups of subjects exchange their products to stay.

During the second period of the study, each subject uses the first smart phone and the "SmartCam" data acquisition software.

As previously stated, the software application allows the fast repositioning of the subject using both horizontal axes and a vertical axis and the image acquisition vertical control using the smart phone's gyro sensor.

The acquisition methods are identical to those presented above. The numerical pattern is adopted in this period of the study. Each subject must take a first reference photo in the morning (at sunrise) and an additional photo during each flush effect during the day.

Figures 42a and 42b show an example of a subject in the eleventh day of the second period. The photo visible in Figure 42a is taken in the morning at 8:20 and considered as reference. During a flush effect at 15:59, an additional photo is taken (Figure 42b). The cheeks (left and right) of the subject in Figure 42b are significantly more red than those shown in the morning picture (Figure 42a).

The results of the study during the second period on the change of the colorimetric parameter a * since the morning are shown in Table 6. The abbreviations PP and ITT in Table 6 correspond respectively to the analyzes "Per Protocol" and "Intention- To-Treat "known to those skilled in the art.

It can be seen that the means and medians of average change since the morning are always positive. In other words, the cheeks during flush effects are more red than in the morning. The fact that the 95% confidence interval is between [0.47, 2.07] confirms the ability of the software application to detect flushes.

Change in average Brimonidin Placebo Total since morning 0.5%

Two Number 14 16 30

first

weeks /

PP

Mean ± σ 1.15 ± 3.24 1.34 ± 2.10 1.25 ± 2.64

Median 0.92 1.04 0.93

(Min, Max) (-4.3, 10.2) (-1.4, 5.9) (-4.3, 10.2)

Two Number 15 16 31

first

weeks /

ITT

Mean ± σ 1.44 ± 3.32 1.34 ± 2.10 1.39 ± 2.71

Median 0.94 1.04 0.94

(Min, Max) (-4.3, 10.2) (-1.4, 5.9) (-4.3, 10.2)

Two Number 15 14 29

latest

weeks /

PP

Mean ± σ 1.47 ± 2.91 1.2 ± 2.51 1.34 ± 2.68 Median 1.98 1.58 1.67

(Min, Max) (-4.1, 5.5) (-2.9, 5.3) (-4.1, 5.5)

Two Number 15 14 29

latest

weeks /

ITT

Mean ± σ 1.47 ± 2.91 1.2 ± 2.51 1.34 ± 2.68

Median 1.98 1.58 1.67

(Min, Max) (-4.1, 5.5) (-2.9, 5.3) (-4.1, 5.5)

Total / PP Number 29 30 59

Mean ± σ 1.32 ± 3.02 1.34 ± 2.10 1.25 ± 2.64

Median 1.49 1.26 1.31

(Min, Max) (-4.3, 10.2) (-2.9, 5.9) (-4.3, 10.2)

Total / ITT Number 30 30 60

Mean ± σ 1.46 ± 3.074 1.27 ± 2.26 1.37 ± 2.67

Median 1.58 1.26 1.40

(Min, Max) (-4.3, 10.2) (-2.9, 5.9) (-4.3, 10.2)

Table 6: Change in average means since morning.

In addition, the software application also makes it possible to study the distribution of the flushes during the day thanks to the time data (date and time) stored, relative to each taking of images.

As illustrated in Figure 43, we can observe the sum of the number of flushes distributed throughout the day. There are more flushes appeared in the afternoon (12h-19h) than in the rest of the time of the day. The application of a treatment makes it possible to reduce the appearance of flushes in this period of the day. These important data can then be analyzed to end a treatment.

With the ability to detect flushes and record their distributions during the day, the application makes it possible to perform self-assessments of skin diseases such as melasma, nevus, rosacea, erythema, reddening acne, psoriasis, dermatitis, actinic keratosis, rash and seborrheic dermatitis.

Claims

1. A method of analyzing facial images of a human comprising the steps of:

acquiring at least one image of the face so that the nose, the eyes and the cheeks are aligned with pre - existing markers,

the components of the image are determined in at least one area of the face,

2. The method of claim 1, wherein the areas of the face are the cheeks.

3. A method according to any one of the preceding claims, wherein the components are the L *, a *, b * colorimetric components.

4. A method as claimed in any one of the preceding claims, wherein the stored values are the values of the colorimetric components determined at an earlier date for the same face.

5. Method according to any one of the preceding claims, wherein the presence of a flush effect is determined if the color component associated with the red color is greater than the stored value associated with the red color.

6. Method according to any one of the preceding claims, wherein the acquisition of images is carried out via a smart phone.

7. A method according to any one of the preceding claims, wherein the verticality of the image acquisition is provided via an internal sensor of a smart phone.

8. Method according to any one of the preceding claims, in which the acquisition is carried out while the suj and holds a colorimetric chart through the mouth, to correct the colorimetric components.

9. A method according to any one of the preceding claims, wherein at least one parameter selected from temperature, atmospheric pressure, ambient luminosity and hygrometry is acquired in order to correct the colorimetric components of an effect of the environment on the face.

The method according to any of the preceding claims, wherein the values of the determined colorimetric components are configured for a self-evaluation of skin diseases such as melasma, nevus, rosacea, erythema, reddening. , acne, psoriasis, dermatitis, actinic keratosis, rash and seborrheic dermatitis.