EP2160717A2

EP2160717A2 - Method and device for acquiring and processing images for detecting evolutive lesions

Info

Publication number: EP2160717A2
Application number: EP08805866A
Authority: EP
Inventors: Laurent Petit
Original assignee: Galderma Research and Development SNC
Current assignee: Galderma Research and Development SNC
Priority date: 2007-05-29
Filing date: 2008-05-28
Publication date: 2010-03-10
Also published as: US20100284582A1; WO2008152297A2; FR2916883B1; WO2008152297A3; FR2916883A1; CA2687596A1

Abstract

The invention relates to a lesion detection method that comprises: generating successive images of the surface to be analysed; developing at least one variation profile based on the time of a parameter of the images generated; and comparing at least one developed profile with a lesion detection threshold value.

Description

Method and device for acquisition and image processing for the detection of progressive lesions

The invention relates to the field of image processing and, in particular, to the field of dermatological image processing.

More particularly, the invention relates to the acquisition and processing of images for the detection of progressive lesions.

A particularly interesting application of the invention thus relates to the detection of acne lesions of the skin by image processing.

The appearance and evolution of a dermatological pathology, such as acne, can be monitored by image processing. But this requires, in this case, to implement successive recording shots obtained at different times of an organ to monitor, in this case the skin, and compare the data thus obtained to detect the appearance and the development of new lesions or on the contrary, their disappearance.

In this regard, reference may be made to EP-A-927405 and FR-A-2 830 961 in which the detection of an acne lesion is carried out by comparing two by two images taken successively in time in order to detect and locate areas of differences between images.

In particular, in EP-A-0 927 405, it is intended to calculate a deformation applied to a first image to make it correspond to a second image subsequently formed, this deformation then serving as a basis for the detection of lesions.

In view of the above, the object of the invention is to overcome the disadvantages associated with detection techniques according to the state of the art and, in particular, to propose a method and a device for detecting progressive lesions that do not require to implement image comparisons. The object of the invention is therefore, according to a first aspect, a method for acquiring and processing images for the detection of progressive lesions.

This method comprises the steps of: - forming successive images of a surface to be analyzed; developing at least one variation profile as a function of time of a parameter of the images formed; and comparing at least one developed profile with a threshold value for detecting lesions. According to another characteristic of the method according to the invention, a profile of the variation of the intensity of the image is developed for different color components of the images.

It is thus possible to develop a variation profile of the intensity of the image for at least one color component selected from a red component, a blue component, and a green component.

A variation profile of the value of a color component ratio of the images can be further developed, such as, for example, a profile of variation of the ratio of the intensity of the red component to the blue component.

According to another characteristic of the method according to the invention, during the formation of the images, successive shots of said surface are taken in different lighting modes so that at each instant of shooting a set of images obtained according to successive lighting modes.

The method may further include storing the images formed in an image database and viewing the images by selecting the images and displaying the selected images on a display screen. During the step of displaying an image, it is also possible to delimit a region of interest in the image and to insert into the image during viewing a corresponding area of an image formed according to another lighting mode and extracted from the image database.

The method may further include a step of processing the images formed by geometrically matching the images.

The object of the invention is also, according to a second aspect, an acquisition and image processing device for the detection of progressive lesions, comprising suitable imaging means for the formation of successive images of a surface. to analyze and means of image processing.

According to a general characteristic of this device, the processing means comprise calculation means adapted to elaborate at least one variation profile as a function of time of a parameter of the images formed and means for comparing at least one profile developed with a threshold value for detecting lesions.

For example, the parameter includes at least one parameter selected from image intensity for a red component, image intensity for a blue component, image intensity for a green component, and a component ratio. color pictures.

According to yet another characteristic of the device according to the invention, the latter comprises suitable lighting means, together with the imaging means, for the formation of images according to different lighting modes, a base of images for storing formed images, a visualization screen for viewing images extracted from the image database, and a human - machine interface for delimiting a region of interest in an image being viewed, the means for processing comprising means for inserting into said image a corresponding region of an image formed in a different illumination mode and extracted from the image base. Other objects, features and advantages of the invention will become apparent on reading the following description, given solely by way of nonlimiting example, and with reference to the appended drawings, in which: FIG. 1 is a block diagram illustrating the general architecture of an image acquisition and processing device according to the invention;

FIG. 2 is a block diagram showing the structure of the central unit of the device of FIG. 1; FIGS. 3 and 4 illustrate the method of resetting the images;

FIGS. 5 to 9 show the human-machine interface of the device of FIG. 1 allowing the adjustment of display parameters and the choice of a region of interest; FIG. 10 shows the procedure for superimposing an area extracted from another image in the region of interest;

- Figures 1 1 and 12 illustrate the procedure of automatic detection of lesions; and FIG. 13 illustrates a flowchart illustrating the operation of the image acquisition and processing method according to the invention.

Referring to Figure 1, there is shown the general architecture of an image acquisition and processing device according to the invention, designated by the general reference numeral 10.

In the exemplary embodiment shown, this device is intended to follow the evolution over time of acne lesions by taking successive shots according to predetermined periods of time of the skin of a patient, and to realize the archiving the images formed, their visualization and comparison.

It will be noted, however, that such a device is intended to monitor the evolution over time of progressive lesions, such as acne, psoriasis, rosacea, pigment disorders, onychomycosis, actinic keratosis, cutaneous cancers.

Such a device can thus advantageously be used by practitioners to determine the effectiveness of a treatment or, for example, to implement clinical trials in order to likewise assess the effectiveness of a new treatment. product.

It should be noted, however, that the invention is not limited to use in the field of dermatology and may also apply mutatis mutandis to any other field in which it is necessary to carry out a comparative analysis of successive images of an organ or, in general, a surface to be examined.

It will also be noted that it is not beyond the scope of the invention when one follows the evolution in time of progressive lesions from a periodic acquisition of data of other kinds, from an archiving of this data and further processing of this data.

As shown in FIG. 1, in the illustrated embodiment in which the data is image data, the device 10 essentially comprises a camera 12 placed on a fixed support 13 and a lighting device. 14 connected to a central unit 15 comprising a set of hardware and software means for controlling the operation of the camera 12 and the lighting device 14 in order to take pictures of the skin of a camera. patient P according to various lighting modes and this, in a successive manner and control the subsequent exploitation of the results.

Indeed, in the embodiment envisaged in which the device 10 is intended to enable a practitioner or a research laboratory to determine the effectiveness of a treatment, the patient P undergoes examination sessions, for example one every day, for a period of time which may be of the order of one month and, at each visit, photographs are taken according to various modes of illumination respectively serving to assess various characteristics of the lesions or to acquire data relating to parameters of the patient's skin.

For example, photographs are taken under illumination under natural light, in parallel polarized light, and in cross-polarized light.

In fact, light polarized in parallel makes it possible to easily appreciate the reliefs of the lesions while crossed polarized light makes it possible to facilitate the counting of the inflammatory lesions by improving their visualization. The modes of shooting can also be performed by UVA irradiation or irradiation, near infra-red, by using an infrared thermography, or at different wavelengths (multispectral images). It is also possible to perform an arithmetic combination of these images thus formed. Other types of illumination may also be used, or the images formed may be combined with complementary data obtained by means of appropriate measuring means.

Thus, in a nonlimiting manner, it would further be possible to combine the image data with data obtained by means of various measuring devices, for example by means of an evaporimeter to determine the insensible loss of water of the skin, at the using a sebumeter to determine the level of sebum in the skin, or by means of a pH meter to determine, for example, changes in the skin due to an irritating treatment,. The image data may also be associated with microcirculation or desquamation information using appropriate measuring instruments, or with hydration using, for example, a corneometer.

The lighting device 14 incorporates various lighting means for emitting the radiation chosen, for example, as indicated above, in a normal light, parallel or perpendicular polarized light. But in other modes of In this embodiment, the lighting device 14 may furthermore incorporate, if desired, a source of UVA radiation, a source of radiation emitting in the near infrared range or in the infrared range or at different wavelengths. to form multispectral images or to make arithmetic combinations of such images.

As can be seen in FIG. 1, the central unit 15 is associated with an image database 16, or generally a database, in which all the images taken at each visit are stored and organized according to the data bases. various lighting modes associated with complementary data delivered by the measuring devices. It is also associated with a human-machine interface 17 constituted, for example, by a keyboard, a mouse, or any other means suitable for the intended use and comprising a display screen 18 for viewing the images formed.

As can be seen, the device 10 can communicate by wired or wireless link with a remote user station 19 or a network of such stations making it possible, for example, to retrieve, consult, compare and remotely exploit the images stored in the database 16.

Finally, in order to make the shooting conditions substantially reproducible, the device 10 is completed by a support 20 placed at a distance and at a fixed height relative to the camera 12 to allow accurate positioning of the camera. the area of the body of the patient P relative to the latter.

The support 20 may advantageously be supplemented by complementary means making it possible to position and precisely maintain the selected body area, for example in the form of a chin strap or bearing surfaces for the patient's head so that, at each visit, the patient's face is positioned precisely in relation to the camera. However, in order to improve the performance of the device and to make the images comparable with each other by exactly matching the body parts from one examination to another, the central unit pre-processes the images formed by geometric registration of the images. images.

Depending on the case, this registration may be rigid, that is to say that it does not change the forms, or non-rigid, or affine, and thus modify the forms according to a certain number of degrees of freedom.

As will be described in detail later, this registration is performed with respect to a reference image, that is to say, on the one hand, with respect to an image formed during a reference examination and , on the other hand, with respect to a reference image. For example, this reference image may be constituted by an image taken according to a predetermined acquisition mode, for example taken under natural light.

After having performed this pretreatment, the images, previously organized, are stored in the image database 16, so that they can later be consulted and compared.

To do this, referring to Figure 2, the CPU 15 comprises a set of hardware and software modules for image processing, organization and operation.

It thus comprises, in the embodiment envisaged, a first module 21 for managing images or data, making it possible to group patients suffering from the same pathology or to create a clinical study relating, for example, to a treatment of which the performance must be assessed, or an existing study selected.

This module 21 makes it possible to define and organize, in the database 16, a memory zone assigned an identifier and grouping a certain number of patients, a set of visits, specific modes of shooting, zones of the body photographed, even regions of interest in the stored images and parameters to be monitored, derived from the measuring devices.

For example, during the creation of a study by the module 21, a determination is made of a reference shooting mode on which the other images will subsequently be recalibrated.

The first management module 21 is associated with a second image management module 22 which makes it possible to import images into the device 10 and to link them to a previously created study, to a patient, to a visit, to a region of interest and a mode of shooting.

The central unit 15 is also provided with a module 23 for resetting the images.

This registration module 23 comprises a first stage 23a ensuring the registration of all the images formed during the different visits on a reference visit and a second floor

23-b ensuring the registration of the images of each visit to a reference image taken according to a predetermined mode of shooting, in this case in natural light.

Referring to FIGS. 3 and 4, the registration of the images implemented by the central unit 15 is based on a comparison of an image I to be recalibrated with respect to a reference image I _re f.

In other words, it is a question of specifying a set of reference areas Z _re f whose number and surface are parameterizable and of comparing each of the zones Z _re f with the reference image I _re f for example by scanning each reference area on the reference image.

In practice, this comparison is to develop a similarity criterion, for example a correlation coefficient of the reference zones Z _ref to the reference image and consists in searching in the reference image, the area Z _'ref the most similar of each reference zone Z _re f of the image to be recalibrated I. As can be seen in FIG. 4, this calculation makes it possible to elaborate a vector field V each illustrating the deformation to be applied to a reference zone so as to correspond to a similar zone of the reference image. From this vector field, the image registration module performs a calculation of the transformation to be applied to the image I to obtain an exact matching of a body area from one examination to another or, in general, from one image to another.

It is, in other words, to look for the affine or free transformation that best represents the vector field and to apply this transformation to the whole image.

As the skin is an elastic material, it has been found that a non-rigid, ie non-affine, registration allows a better registration of the images after regularization of the vector field, which makes it possible to impose constraints. to transformation and not allow any type of transformation.

It is further proposed to the user a representation of the transformation performed to validate or not the registration of an image and thus avoid a subsequent comparison of images whose changes are too important.

For example, to do this, we superimpose a grid image or, in general, a fictitious mesh, to a recalibrated image, and we apply the same transformation to this mesh as that suffered during the registration of the images. It is thus possible to easily appreciate the level of deformation applied to the image.

After performing the registration, the CPU 15 may optionally correct bias in the image by correcting the intensity of the imaged image so that its intensity is similar to the reference image. After having performed this pretreatment, the central processing unit 15 stores the images in the image database 16, associated, if necessary, as indicated above, with data complementary. For this purpose, it uses a module 24 for generating a set of corrected images, in particular to be able to export the images so that they can be used in processing software of other types. The central unit 15 further comprises a dynamic display module of the set of imaged images, designated by the general reference numeral 25.

This module 25 is directly parameterizable by the human-machine interface 17 in conjunction with the screen 18 and comprises all the hardware and software means for navigating within the image database 16 to display the set of recalibrated images of adjust the viewing parameters, such as zoom, brightness, contrast, the shooting mode displayed, delimit regions of interest or, as will be described in detail later, to incorporate into a region defined in an image being viewed, a corresponding region extracted from another image, for example an image taken according to another mode of shooting.

Referring to FIGS. 5 to 9, for this purpose, the central unit 15 causes the display, on the screen 18, of a certain number of windows or, in general, of an interface proposing to the user a number of tools to allow such dynamic visualization of the images.

Firstly, with reference to FIG. 5, a first window F1 makes it possible to display all the visits previously made and to select one of the visits in order to extract the corresponding images from the image database.

A second window F2 (FIG. 6) makes it possible to choose, for each image, an acquisition mode and complementary images relating, for example, to other areas of the photographed face. For example, a first icon II makes it possible to select the zone of the face to identify, for example the right cheek, the left cheek, the forehead, the chin, while a second icon 12 makes it possible to select the exposure mode, eg natural light, parallel or cross polarized light, ....

In addition, a control window F3 (FIG. 7) makes it possible to display, in a global image, an image portion under examination and to move rapidly in the image.

The central unit 15 can also provide a control window F4 for adjusting the zoom level, brightness and contrast of the image displayed (Figure 8) or a window F5 to select a slideshow scroll mode according to which the images of the different visits or a visit framing a selected visit are presented on the screen with an adjustable scrolling speed (FIG. 9).

Referring to FIGS. 2 and 10, the processing unit 15 further includes an image processing module 26 which cooperates with the display module 25 to jointly offer the user a tool for selecting a region of interest R in an image being viewed, to select another image, for example an image taken in another mode of shooting, to import a zone Z of the selected image corresponding to the region of interest R and d to integrate in image I the zone Z extracted from the selected image.

Thus, for example, after having selected a region of interest R and another mode of taking pictures, the central unit 15 and, in particular, the processing module 26, extracted from the image corresponding to the selection, the zone Z corresponding to the region of interest and inserts it into the image in order to be able to dynamically dispose of another mode of shooting in a selected portion of an image being viewed.

Of course, any other data extracted from the database, or only some of this data, can also be incorporated in the region of interest R in place of or in addition to the imported zone Z, for example any type of data obtained. by the different devices for measuring a parameter of the skin, such as pH data, insensitive water loss, sebumetry, hydration data such as skinchip or corneometry, microcirculation, desquamation, color or the elasticity of the skin.

Finally, while also referring to FIGS. 11 and 12, the central unit 15 is provided with an automatic lesion detection module 27 ensuring, for example, a comparison of the data associated with each pixel with a detection threshold value. of lesions. Indeed, with reference to FIG. 11, which relates to healthy skin, and on which is represented the evolution of the intensity i of a portion of the image as a function of time t, for the red color ( curve C l), for the green color (curve C2), for the blue color (curve C3) and for the red / blue ratio (C4), we see that, in a healthy region, the intensity profile oscillates around an average value corresponding to the color of the skin.

On the contrary, as shown in FIG. 12, which corresponds to a skin presenting acne lesions, and on which the curves C 1, C '2, C' 3 and C '4 respectively correspond to the curves C 1, C 2, C 3 and C 4 of Figure 1 1, in an injured region, the intensity profile as a function of time has a perfectly identifiable peak when it is present on the skin, that is to say that the skin becomes darker or lighter, or more red, depending on the type of lesion. It is then possible to automatically detect and qualify the appearance of a lesion by comparing the intensity profiles with a threshold value. For example, as shown, the variation profile of the red / blue signal ratio can be compared with an intensity threshold value corresponding to a value "2". Thus, as is apparent from FIGS. 11 and 12, the automatic lesion detection module 27 proceeds to extract, for each image, zone by zone, the values of the parameters monitored, and elaborates thus, for all the images formed successively in time, and for each parameter, a profile of variation of the parameter as a function of time.

As indicated above, the monitored parameter can be constituted by any type of parameter related to the images, and in particular a colorimetric parameter, that is to say, in particular, the intensity of the red, green and blue components. and the component ratio, for example the ratio of the intensity of the red component to the blue component. The module 27 thus collects all the values of the parameters monitored over a configurable period of time and produces curves illustrating the evolution of these parameters to present them to the user. As illustrated in Figures 1 1 and 12, it is therefore possible, for example, to have the evolution of the values of the components of red, green and blue and the ratio of these components.

For each of the monitored zones, the detection module 27 performs a calculation of the difference of the value of the parameters with a corresponding lesion detection threshold value. Of course, this calculation is carried out after selection, by the user, of one or more parameters, depending on the nature of the lesion to be detected and, if appropriate, after entering a threshold value or of several respective threshold values by the user. Indeed, the threshold value, which can be stored in memory in the central unit 15 or entered manually, is parameterizable and depends on the monitored parameter.

As indicated above, the appearance of a lesion results in a variation in the damaged area of the color components. In the example illustrated in FIG. 12, the lesion causes a relatively large decrease in the blue and green components, compared with the modification of the red component, which is This translates into a locally important increase in the ratio of red and blue components during the onset of lesion.

It is thus possible here to detect the appearance of the lesion from the variation of the ratio of the red and blue components, by comparison with a detection threshold value, for example set at "2".

Of course, another threshold value is used when the detection of a lesion is made from another parameter. The detection of a lesion is carried out by the module 27, zone by zone. Of course, the dimensions of the monitored zones constitute a parameterizable value which depends on the size of the lesions to be detected.

Finally, the main phases of the image acquisition and processing method according to the invention will be described with reference to FIG. 13 for the detection of the evolution over time of acne lesions carried out, in the example considered in FIG. from image data formed according to respective illumination modes.

During a first step 30, the central unit 15 proceeds to the successive acquisition of a set of images taken successively in time during various visits of a patient and, for each visit, according to various modes of Shooting.

Subsequently or beforehand, the CPU 15 implements the study and management management modules 21 and 22 to create a study or assign the images formed to a previously entered study.

In the following step 32, the images are re-aligned according to the procedure previously indicated, by implementing the image registration modules 23-a and 23-b in order to, on the one hand, recalibrate the images. images on a reference visit and, on the other hand, to record, at each visit, an image on a reference image taken according to a selected shooting mode. After resetting, a set of staggered images (step 33) is then produced, which are then stored in the image database 16.

As indicated above, the image data can be supplemented by data provided by other types of sensors in order to supplement the available information.

In the next phase 34, at the request of a user, the images stored in the image database 16, supplemented if necessary by additional data or a part of such data, can be viewed. To do this, the central unit 15 offers the user a number of interfaces making it possible to select display parameters, to select one or more regions of interest, and to navigate from one image to another at the same time. within the region of interest, to choose various areas of a face, ...

Claims

1. A method for acquiring and processing images for the detection of progressive lesions, characterized in that it comprises the steps of: - forming successive images of a surface to be analyzed;

developing at least one variation profile as a function of time of a parameter of the images formed; and

- Comparison of at least one developed profile with a threshold value of detection of lesions.

2. Method according to claim 1, characterized in that a profile of variation of the intensity of the image is produced for different color components of the image.

3. Method according to claim 2, characterized in that a profile of variation of the intensity of the image is developed for at least one color component selected from a component of red, a component of blue, and a component of green.

4. Method according to one of claims 2 and 3, characterized in that one furthermore develops a profile of variation of the value of a ratio of color components of the images.

5. Method according to claim 4, characterized in that one develops a profile of variation of the ratio between the intensity of the red component and the blue component.

6. Method according to any one of claims 1 to 5, characterized in that during the formation of images, it takes successive shots of said surface in different lighting modes, so that at each moment of shooting, a set of images obtained in respective lighting modes is formed.

7. Method according to any one of claims 1 to 6, characterized in that it further comprises steps of storing the images formed in a base of images (16) and of viewing the images by selecting the images and displaying selected images on a display screen, and in that during the step of When viewing an image, a region of interest is delimited in the image and a corresponding zone (Z) of an image formed in a different and extracted illumination mode is inserted into the image being viewed. of the image database.

8. Method according to any one of claims 1 to 7, characterized in that it further comprises a step of processing the images formed by geometric matching of the images.

9. A device for acquiring and processing images, for the detection of progressive lesions, comprising camera means (12) suitable for forming successive images of a surface to be analyzed and means for processing images. images (15), characterized in that the image processing means (15) comprise calculation means adapted to produce at least one variation profile as a function of time of a parameter of the images formed and the comparison means of 'm mo ins a developed profile with a threshold value of detection of lesions.

10. Method according to claim 9, characterized in that the parameter comprises at least one parameter chosen from the intensity of the images for a component of red, the intensity of the image for a component of blue, the intensity of the images. for a component of green, and a ratio of color components of the images.

1 1. Device according to one of claims 9 and 10, characterized in that it comprises lighting means (14) adapted, together with the means of shooting (12), for the formation of images according to different lighting modes, an image database (16) for storing the formed images, a display screen for viewing images extracted from the image database and a man - machine interface (17) adapted for delimiting a region of interest in an image being viewed, the processing means including means for inserting into said image a corresponding area of an image formed in a different illumination mode and extracted from the image database.