ES2924934A1 - Smart dermatoscope and procedure for classification of dermatological lesions (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

Intelligent dermatoscope and procedure for the classification of dermatological lesions. The dermatoscope is made up of a portable device, in which a casing with an ergonomic manual grip control (4) participates, a casing that integrates a high-resolution digital camera (7), with optical and digital zoom, an auxiliary light adjustable, a microprocessor, a touch screen (8), a rechargeable battery (9), an on and off button, and a Wi-Fi, 4G and Bluetooth wireless communications module for remote management of the images taken, management in which a procedure is used in which lesions are first segmented, then the descriptors are extracted, the data is increased and implemented in a convolutional neural network model, to finally add a "bootstrap" machine learning meta-algorithm. bagging". (Machine-translation by Google Translate, not legally binding)

Description

DESCRIPTION

Smart dermatoscope and procedure for classification of dermatological lesions

TECHNICAL SECTOR

The present invention refers to an intelligent dermatoscope, specially designed to allow a patient's skin to be analyzed in situ by taking images through the device itself and to discriminate between cancerous and non-cancerous lesions, malignant diseases and benign diseases based on the processing of the images taken and from a procedure that is also the object of the invention.

The invention is therefore situated in the field of medical diagnostic instruments, and more specifically in the field of dermatology.

BACKGROUND OF THE INVENTION

Until 2012, the use of pre-trained convolutional neural networks for image classification was quite limited, given that the error rate of the best classical algorithms was around 26%.

In that same year, a convolutional neural network called Alexnet managed to break all records and reduced the error rate to 16%. As The Economist newspaper commented on neural networks, "All of a sudden, people started paying attention, not just within the AI community, but across the tech industry."

Already in 2015 the performance of humans was exceeded (around 5% error rate) and with the networks used for this project, the error rate is already less than 1%.

Another advance that has made it possible to achieve the results obtained is the transfer learning technique . This technique allows the knowledge acquired in one task to be used in another related but different task. Thanks to transfer learning, tasks for which we do not have much data due to the cost of collecting it are they can be greatly improved by reusing the knowledge gained in other tasks for which we have much more data (in our case other image classification problems that are not related to skin lesions).

The reality is that these methods had not been applied to date for the analysis and diagnosis of possible skin lesions, such as cancerous and non-cancerous lesions, malignant diseases and benign diseases.

EXPLANATION OF THE INVENTION

The intelligent dermatoscope proposed by the invention fills the previously described technical gap.

For this, and more specifically, the dermatoscope of the invention is constituted from a portable device, in which a casing with an ergonomic manual grip control participates, casing that defines a resistant frame for the electronic components that participate in the device, which integrates a high-resolution digital camera, with optical and digital zoom, as well as an adjustable auxiliary light.

The camera will incorporate interchangeable protection means that protect it from possible bumps and scratches.

The camera will be associated with a microprocessor, which in turn is connected to a touch screen, with a high-capacity battery, an on/off button, and a wireless communications module for remote management of the images taken.

Regarding the procedure that is carried out by means of the device of the invention, five stages are defined in it:

1. Segmentation of lesions.

2. Extraction of descriptors.

3. Data augmentation.

4. Model of convolutional neural networks.

5. Bootstrap/bagging aggregation or packaging (machine learning meta-algorithm.

Regarding the segmentation process, a k-means clustering algorithm is used to separate the lesion from the rest of the skin.

This stage is important because it allows extracting essential descriptors to improve the classification results.

In a second stage and once the lesion is separated from the rest of the skin, lesion descriptors are extracted based on:

• Asymmetries (when the halves of the lesion are different)

• Edge irregularities.

• Color (uniformity of pigmentation, different colors)

These numerical descriptors are integrated with the convolutional neural network.

In a third stage, the data is increased; it is a technique aimed at increasing the diversity of the training data set (in our case images) by applying realistic random transformations. These techniques are necessary since the same lesion can be photographed from different angles, with different light intensity, different skin colors... An attempt will be made to multiply each image randomly, simulating the different conditions in which the same lesion can be photographed. find.

The next phase consists of integrating the data obtained in a convolutional neural network.

With the pre-trained models, we use the fine-tuning technique, which consists of “transferring” the learning of a network from one domain (Imagenet image classification) to another (skin lesion classification).

Finally, a "bootstrap" is added, that is, a machine learning meta-algorithm designed to improve the stability, accuracy, and reduce variance of machine learning algorithms.

The method thus described allows to discriminate between cancerous, non-cancerous lesions, malignant diseases and benign diseases depending on the configuration mode.

DESCRIPTION OF THE DRAWINGS

To complement the description that is going to be made below and in order to help a better understanding of the characteristics of the invention, according to a preferred example of its practical embodiment, a set of plans is attached as an integral part of said description. where, by way of illustration and not limitation, the following has been represented:

Figure 1.- Shows a front perspective view of an intelligent dermatoscope made according to the object of the present invention.

Figure 2.- Shows a rear perspective view of the device of the previous figure.

Figure 3.- Shows a perspective and exploded view of the device of figures 1 and 2.

Figure 4.- Shows a perspective view of the assembly of figure 3 partially assembled.

Figure 5.- Shows, finally, an opposite perspective view of the assembly of figure 4.

PREFERRED EMBODIMENT OF THE INVENTION

In view of the figures outlined, it can be seen how the dermatoscope of the invention is constituted from a portable device (1), in which a chassis participates central (3) to which two extreme half-shells (2-2') can be attached, all of them made of nylon with carbon fiber, light and very resistant, which guarantees perfect operation throughout its entire useful life cycle .

The assembly forms a frame or casing in which an ergonomic and non-slip handle (4) is defined, to facilitate use with one hand, finished off with a loop (5) adjustable to the wrist to minimize the risk of accidental falls.

The frame is complemented by TPe Shore 70A corner reinforcements (6), to protect the integrity of the shell in the event of an impact.

The device includes a digital camera (7) of high resolution, preferably 108 MP, with optical and digital zoom, equipped with dual native ISO, which gives it greater dynamic range, more vivid details and professional noise control.

The camera is complemented by an auxiliary light adjustable in 5 brightness levels, with the option to choose between polarized and non-polarized light, and an auto-off function, including easily interchangeable protectors.

Said digital camera (7) will be associated with a microprocessor, which in turn is connected to a touch screen (8), AMOLED and Qualcomm® Snapdragon™ 732G processor with 8nm processing technology with advanced CPU and GPU architecture, Enhanced data computing and image processing capabilities, these electronic components being powered by a 10400 mAh battery (9), with charging status indicator LEDs and USB C fast charging port.

The device will be activated and deactivated through the corresponding on and off button, having a Wi-Fi, 4G and Bluetooth wireless communications module.

Based on this structuring, the device will allow high quality and sharp images to be taken of different areas of the skin to be analyzed, images that will be managed according to the following procedure:

In a first phase, a process of image segmentation will be carried out, in order to separate the lesion from the rest of the skin. For this, a k-means clustering algorithm is used.

This stage is important because it allows extracting essential descriptors to improve the classification results.

In a second stage and once the lesion is separated from the rest of the skin, lesion descriptors are extracted based on:

• Asymmetries (when the halves of the lesion are different)

• Edge irregularities.

• Color (uniformity of pigmentation, different colors)

These numerical descriptors are integrated with the convolutional neural network.

In a third stage, the data is increased. It consists of a technique aimed at increasing the diversity of the training data set (in our case images) by applying realistic random transformations. These techniques are necessary since the same lesion can be photographed from different angles, with different light intensity, different skin colors, etc. An attempt will be made to multiply each image randomly, simulating the different conditions in which the same lesion can be found with the following random transformations:

• Rotations.

• Brightness and contrast settings.

• Different zooms.

• Optical distortions.

• Variation of sharpness.

• Variation in the compression quality of the images.

• Injection of different types of noise.

Thanks to the increase in data, thousands of synthetic variations can be created from each training image, serving both as a regularization technique and as a way to give stability to our algorithm.

In a fourth phase, the processed images are integrated into a convolutional neural network.

Convolutional neural networks are distinguished from other neural networks by their vastly superior performance with images, speech, or audio. We will use pre-trained models with the Imagenet image dataset. This dataset consists of around 15 million images with 1000 different classes and is widely used in research on machine vision algorithms.

With the pre-trained models, we use the fine-tuning technique, which consists of “transferring” the learning of a network from one domain (Imagenet image classification) to another (skin lesion classification). For this, the pre-trained network will be used as a starting point that will be re-trained with our dataset of skin lesion images (with a modification of the final part of the model adapting it to our objective and adding the extracted descriptors).

For the pre-trained models, we use the convolutional neural networks called “EfficientNetV2”. These networks are characterized by their efficiency in terms of speed of training, inference and size. The network has 20,331,360 trainable parameters to which must be added the parameters intended for the extracted descriptors.

For the training and manipulation of these networks, we will use Google's Tensorflow neural network library and its repository of pre-trained models Tensorflow-Hub.

The training process is modified in different ways to suit the very particular qualities of our data.

• We must be aware that the images are tagged by human beings, therefore it is taken into account that some of them may have erroneous labels. To make the model more robust and that the performance is not affected by the fact that there are some erroneous images, two techniques are applied:

^o “Label smoothing”: It is a regularization technique intended to add uncertainty to the labels to avoid overfitting.

^o Noisy labels ( "noisy labels"). It is a set of techniques aimed at estimating the number of "erroneous" labels and adapting to them.

^• There are certain types of lesion (especially some melanomas) that are much more difficult to distinguish. To deal with this, we apply a relaxed “boosting”, in which through cross-validation we assign more weight to the data in which a simplified model performs worse.

^• The "learning rate" of the learning algorithm is modified with an exponential decay and adapts to the new amount of data.

^• Due to the nature of the app, we have a large number of untagged images. From there we choose randomly among the images in which the model gives the most confidence and we introduce them during learning (never in more than one epoch) and adding uncertainty to the label. This section is really important, since tagging images is very expensive, it takes the time of highly qualified personnel. Thanks to this we can use untagged images instead of discarding them.

Finally, the method of the invention provides for the addition of an automatic learning meta-algorithm or "bootstrap", designed to improve the stability, accuracy and reduce the variance of automatic learning algorithms.

We use this technique thanks to the efficiency of 'EfficientNetV2' networks. We train 5 networks with randomly chosen subsets of training images and average their results (we assemble the 5 networks). This technique allows us to improve the stability and performance of the algorithm extensively.

Results:

Finally, to measure the performance of the algorithm, the set of images (41,200) has been separated into a training set and a test set. The results have been following:

• Model malignant/benign diseases.

To train the model, 35,637 training images and 6,289 test images have been used with the following results:

• Of a total of 1,580 images with malignant lesions, 1,454 were correct.

• Of a total of 4,709 images with benign lesions, 4,550 were correct.

• Accuracy: 90.1%,

• Completeness (“recall”): 92%

In short, the total percentage of correctly classified images is 95.5%.

• Model malignant diseases.

To train the model, 8,199 training images and 1,447 test images have been used. Below are the estimated results when using all the images (including the test ones):

• Class 0: Melanoma (840 images):

^o Accuracy: 98.4%

^o Completeness (“recall”): 98.2%

• Class 1: Basal Cell Carcinoma (509 images):

^o Accuracy: 96.5%

^o Completeness ("recall"): 97.6%

• Class 2: Squamous Cell Carcinoma (98 images):

^o Accuracy: 94.7%

^o Completeness ("recall"): 90.8%

The result is that the total percentage of correctly classified images is 97.5%.

• Model benign diseases.

To train the model, 26775 training images and 4710 test images have been used with the following results:

• Class 0: Nevus (4182 images):

^o Accuracy: 99.2%

^o Completeness ("recall"): 98.7%

• Class 1: Seborrheic Keratosis/Lentigo (451 images):

^o Accuracy: 89.3.5%

^o Completeness (“I relapsed!”): 92.5%

Class 2: Others (77 images):

^o Accuracy: 94.7%

^o Completeness (“recall”): 90.8%

The total percentage of correctly classified images is 98%.

Claims (12)

1a.- Intelligent dermatoscope, characterized in that it is constituted from a portable device, in which a casing with an ergonomic manual grip control (4) participates, casing that integrates a high resolution digital camera (7), with zoom optical and digital, an adjustable auxiliary light, a microprocessor, a touch screen (8), a rechargeable battery (9), an on/off button, and a Wi-Fi, 4G and Bluetooth wireless communications module for remote management of the pictures taken. 2a.- Intelligent dermatoscope, according to claim 1a, characterized in that the casing is constituted from a central chassis (3) to which two end semi-casings (2-2') are attachable, elements obtained in nylon with carbon fiber . 3a.- Intelligent dermatoscope, according to claim 1a, characterized in that the casing includes reinforcements (6) for anti-impact protection. 4a.- Intelligent dermatoscope, according to claim 1a, characterized in that the high resolution digital camera (7) is 108 MP, with digital optical zoom and equipped with dual native ISO. 5a.- Intelligent dermatoscope, according to claim 1a, characterized in that the auxiliary light of the camera includes 5 levels of brightness regulation, selection means between polarized and non-polarized light, and an auto-off function. 6a.- Intelligent dermatoscope, according to claim 1a, characterized in that the touch screen (8) is of the AMOLED type and Qualcomm® Snapdragon™ 732G processor with 8nm processing technology and advanced CPU and GPU architecture, improved data computing and image processing capabilities. 7a.- Intelligent dermatoscope, according to claim 1a, characterized in that the battery (9) is 10400 mAh, with charge status indicator LEDs and USB C fast charging port. 8a.- Procedure for the classification of dermatological lesions, which starting from the dermatological images obtained from the device of claims 1 to 7, comprising the following operating phases: a) Image segmentation, separating the lesion from the rest of the skin using a k-means clustering algorithm. b) Extraction of injury descriptors based on: ^o Asymmetries (when the halves of the lesion are different) ^o Edge irregularities. ^o Color (uniformity of pigmentation, different colors) c) Data augmentation by applying realistic random transformations such as: ^or Rotations. ^o Brightness and contrast adjustments. ^o Different zooms. ^or optical distortions. ^o Sharpness variation. ^o Variation in the compression quality of the images. ^o Injection of different types of noise. d) Integration of the processed images in a convolutional neural network using pre-trained models. e) Addition of a machine learning meta-algorithm or "bootstrap", designed to improve the stability, accuracy and reduce the variance of machine learning algorithms. 9a.- Procedure for the classification of dermatological lesions, according to claim 8a, characterized in that, in phase d) the fine-tuning technique is applied with the pre-trained models, transferring the learning of a domain network (the classification of the images) to another (the classification of skin lesions), adding the descriptors extracted. 10a.- Procedure for the classification of dermatological lesions, according to claims 8a and 9a, characterized in that, in phase d) for the pre-trained models, convolutional neural networks called "EfficientNetVZ" are used. 11a.- Procedure for the classification of dermatological lesions, according to claims 8a to 10a, characterized in that, in phase d) regularization techniques are used to add uncertainty to the labels to avoid overfitting, techniques aimed at estimating the amount of “wrong” labels and to adapt to them and cross-validation techniques in which more weight is assigned to the data in which a simplified model performs worse. 12a.- Procedure for the classification of dermatological lesions, according to claim 8a, characterized in that, in phase e) five networks are trained with randomly chosen subsets of training images and their results are averaged, we assemble the five networks.