WO2019190358A1

WO2019190358A1 - Method of automated analysis of digital fluorographic images

Info

Publication number: WO2019190358A1
Application number: PCT/RU2019/050035
Authority: WO
Inventors: Николай Григорьевич АНДРИАНОВ; Виктор Иванович КЛАССЕН; Антон Владимирович МАЛЬЦЕВ; Артем Альбертович САФИН
Original assignee: Общество с ограниченной ответственностью "ФтизисБиоМед"
Priority date: 2018-03-27
Filing date: 2019-03-27
Publication date: 2019-10-03
Also published as: RU2684181C1

Abstract

The claimed technical solution relates to the field of digital image processing in medicine and is intended for the automated analysis of fluorographic images of the chest of patients as regards changes or pathologies present in the lung area. With the aid of a client module, at least one initial digital fluorographic image is uploaded to a set of convolutional neural networks. With the aid of the set of neural networks, one or more of the uploaded digital fluorographic images are sequentially processed to identify areas of interest according to the response of each neural network. The resultant processing during the neural network response is transmitted to a module for analyzing and combining the convolutional network output, which processes and applies the output from the set of neural networks to the initial digital image to identify pathologies if they are present.

Description

METHOD FOR AUTOMATED ANALYSIS OF DIGITAL FLUOROGRAPHIC IMAGES

FIELD OF TECHNOLOGY

[1] The claimed technical solution relates to the field of digital imaging in medicine and is intended for the automated analysis of fluorographic images of the chest of patients for changes or pathologies in the lung area. The claimed solution can be used in mobile complexes to accelerate the processing of images, in clinics for pre-processing images, ranking by importance for reading images by a doctor.

BACKGROUND

[2] A well-known automated system for the diagnosis of medical images using deep convolutional neural networks (US patent 9,589,374 B1, 03/07/2017). This invention discloses methods for applying deep convolutional neural networks (SNA) to the analysis of medical images for real-time diagnosis. The above invention uses the analysis of CT and MRI images, which are processed using two convolutional neural networks and other software modules, to obtain a response with the likelihood of areas of interest in the patient’s images, which are necessary for further analysis by the attending physician.

[3] In the article “Automatic Liver and Tumor Segmentation of CT and MRI Volumes Using Cascaded Fully Convolutional Neural Networks” (Patrick Ferdinand Christ et al. Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI). 20.02 .2017), approaches to the automated analysis of CT and MRI images for detecting liver pathologies are considered, using the U-NET type convolutional neural network. [4] It is also known to use an ensemble of three SNA for the analysis of medical images for the availability of relevant information, depending on the type of training of the SNA. In the aggregate use of an ensemble of three SNAs, this approach allows one to obtain more accurate data with minimization of recognition errors (Kostin K.A. Master's thesis "Adaptive classifier of pathologies for computer diagnosis of diseases using convolutional neural networks for medical images and video data. 05/30/2017) .

[5] This solution is, in its technical essence, the closest analogue. The main disadvantage of this solution is the adjustment of the SNA that does not imply the separation of responses by weighted coefficients with their subsequent re-weighing in the layers of each SNA and the separation of the training sample by the type of pathologies during the ensemble training, which leads to a rather high degree of errors during recognition of structural changes in fluorographic images. Moreover, this solution, as such, is not used for the analysis of fluorographic images.

SUMMARY OF THE INVENTION

[6] The technical problem of the claimed solution to be solved is to minimize the error of false positives of the SNA ensemble and, accordingly, increase the recognition accuracy of areas of interest when analyzing graphic information, due to the new principle of teaching the SNA ensemble and their subsequent work based on the training.

[7] The technical result coincides with the technical problem being solved.

[8] Thanks to the automated system, the analysis time of fluorographic images is significantly reduced, while the accuracy of detection of pathologies is set at a high level and the influence of the human factor is reduced. DESCRIPTION OF DRAWINGS

[9] FIG. 1 illustrates an automated fluorographic image analysis system.

[10] FIG. 2 illustrates an example of an input image.

[11] FIG. 3 illustrates a processed image using the inventive system.

DETAILED DESCRIPTION OF THE INVENTION

[12] A key feature of the technical solution is the training method and structure of the convolutional neural network. To achieve a technical result, apply:

1. A specially marked base of fluorographic images for training with classification of each area.

2. Using an ensemble of three convolutional neural networks of the U-NET type with different settings and organization of input data for training.

3. Reweighting classes according to their importance in the sample.

4. The combination of output images to increase the training base.

[13] The first network is configured to work only with hazardous areas (pathologies of the first kind), the second and third networks are configured with all areas, but with different thresholds and architecture.

[14] The regions are divided into two groups: hazardous and non-hazardous. All areas in the groups are classified by type of pathology. The following classification was used:

Pathologies of the 1st kind (dangerous)

I. Infiltration (focus) - over 1.5 cm.

II. Cavity

III. Pneumothorax

IV. Hydrothorax

V. Hearth

VI. Pathological changes in the roots of the lungs VII. Fluid level

Viii. Foci

Pathologies of the 2nd kind (non-hazardous)

I. Interstitial changes in the pulmonary parenchyma

II. Cirrhosis

III. Fibrothorax

IV. Pleural changes

V. Calcifications / Calcifications

VI. Diaphragmatic hernia

VII. Bone changes

Viii. Chains of metal seams

IX. Foreign bodies

X. A plot of high transparency (not a cavity)

Xi. Atelectasis

XII. Changes in the mediastinal organs

[15] In FIG. 1 shows a view of a system for automated analysis of fluorographic images.

pos. 1 - input image (digital X-ray photograph) pos. 2 - client module for remote image analysis (work is possible without it)

pos. 3 - image loading module

pos. 4 - convolutional neural network N ° 1

pos. 5 - convolutional neural network N ° 2

pos. 6 - convolutional neural network N ° 3

pos. 7 - module for analysis and integration of the results of convolutional networks, output

pos. 8 - processed image.

[16] The system for the automated analysis of fluorographic images is software. For implementation, TensorFlow and Keras machine learning libraries are used. BY can work on any modern computer with a graphics processor from Nvidia or on Jetson TX2 mobile platforms.

[17] Automated analysis of fluorographic images is performed using three convolutional neural networks of the U-NET type. A special database of images has been prepared for training. The images show areas with changes in the structure of the lungs, which indicate the presence of tuberculosis or other pathologies, both hazardous to human health and non-hazardous. Regions are divided into two groups: hazardous and non-hazardous. All areas in the groups are classified by type of pathology. The following classification was used:

Pathologies of the 1st kind (dangerous)

I. Infiltration (focus) - over 1.5 cm.

II. Cavity

III. Pneumothorax

IV. Hydrothorax

V. Hearth

VI. Pathological changes in the roots of the lungs

VII. Fluid level

Viii. Foci

Pathologies of the 2nd kind (non-hazardous)

I. Interstitial changes in the pulmonary parenchyma

II. Cirrhosis

III. Fibrothorax

IV. Pleural changes

V. Calcifications / Calcifications

VI. Diaphragmatic hernia

VII. Bone changes

Viii. Chains of metal seams

IX. Foreign bodies

X. A plot of high transparency (not a cavity) XL Atelectasis

XII. Changes in the mediastinal organs.

[18] In FIG. 2 shows an example of an input image. To increase the base, transformations were applied to the input images. Segmentation maps are formed from the marked areas and fed to the training input along with the source images.

[19] Cards with hazardous areas are only sent to the entrance of the first network, cards with all areas to the entrance of the second and third, but they have different thresholds and architecture. Architecture: Unet (8 folding layers, 8 expanding layers, 32 start filters, each layer x1.5 filters on a convolution, x1.5 filters on a roll-out. Input - 1 channel, output - 1 channel). The selection of threshold values is carried out experimentally from the picture at the output.

[20] During the training process, the weights in the layers are reweighed for better convergence, based on the reliability of what doctors note (data from doctors / comparison of different markings). There is a choice of more or less reliable classes. The input image is fed into the trained network, and the output is three segmentation cards with weights in each pixel of the entire image, but the cards are active only in those places where pathologies are highlighted.

[21] At other points, the probability of changes is close to zero. The map data is processed, areas with increased response are distinguished, which characterize pathologies of different types in different layers, their area and weight, the results are compared with thresholds that are experimentally selected and set for each network.

[22] Processing the response of a neural network occurs without a neural network, "manually". The overall response energetics, maximum response are searched for and the area of exceeding threshold values is estimated. All thresholds and algorithms are selected empirically. [23] Downloading the image (digital X-ray photograph) for processing takes place using a special software module 3. The image is processed sequentially by each convolutional neural network, each of which, based on the experience gained during the training, gives a judgment, highlighting it if there is a suspicious area. The next program module 7 collects the results of all three networks, combines them and superimposes them on the original image. The output module image 8 displays the processed image with a selected pathological area, if any. The response from the networks is displayed in different colors and different intensities depending on the magnitude of the response.

[24] FIG. 3 illustrates an example of the operation of an image processing system. Image analysis can be performed both locally and remotely. Client module 2 is used for remote access. Any suitable computer device (personal computer, laptop, tablet, etc.) can be used as module 2.

[25] This method is only possible when accessing the Internet. In this case, the client part of program 2 remotely connects to the server where the automated system for the analysis of fluorographic images is deployed and implements the transmission of the image for analysis, as well as the reception and output of the processing result.

Claims

CLAIM

A method for the automated analysis of digital fluorographic images using an automated system for the analysis of fluorographic images for the detection of pathologies, comprising stages in which:

- using the client module, at least one source digital digital X-ray image is loaded into the ensemble of convolutional neural networks, the ensemble of convolutional neural networks containing three neural networks, in which one of the networks is configured to process and identify dangerous pathologies of the first kind, and the remaining networks to process and identify dangerous pathologies of the 1st kind and non-dangerous pathologies of the 2nd kind, and each network is trained using threshold values unique to this network;

using the said ensemble of neural networks, sequential processing of the loaded one or more digital fluorographic images is performed to identify areas of interest depending on the response of each neural network;

- the received processing during the response of the neural networks is transmitted to the analysis and integration module of the convolutional network, which processes and superimposes the results of the ensemble of neural networks on the original downloaded digital image to identify pathologies if any.