RU2734575C1 - Method and system for identifying new growths on x-ray images - Google Patents

Abstract

FIELD: data processing.

SUBSTANCE: invention can be used for automated analysis of X-ray images. Summary of the invention is that at least one X-ray image is obtained; processing said image, during which its primary normalization and cleaning from noise; image brightness map is distributed over RGB color channels, wherein in the red channel, normalization is performed, contrast is increased and the most contrast contours are detected on the image; in the blue channel, the contrast of the processed image is increased by modifying its histogram; in a green color channel its initial information is stored; generating an image based on processing of RGB channels; performing processing of the obtained image using a machine learning model, during which classification of image areas is performed by selecting reference points and sequential processing of said points using a floating window of a given pixel dimension; calculating a vector for each said reference point, containing numerical parameters for criteria considered when BI-RADS classification.

EFFECT: high accuracy of identifying new growths on X-ray images.

7 cl, 6 dwg

Description

The present invention relates to the field of computer technology in relation to the field of medicine, in particular to a method for automated analysis of X-ray images.

Today, with the development of technologies in the field of machine learning, such solutions are increasingly used in the medical field, in particular to help specialists in data analysis and clinical decision making.

The following technologies can be considered as examples of solutions for the analysis of medical images using artificial neural networks (ANNs).

BOTKIN AI cloud-based medical image analysis platform (https://botkin.ai). Third Opinion Medical Image Recognition Platform (https://3opinion.ai/ru). Fluorogram analysis service PhthisisBioMed (http://ftizisbiomed.ru/). QuantX (www.qlarityimaging.com) is a breast cancer recognition system. GOOGLE Lymph Node Assistant (LYNA) is an AI-based system for recognizing breast cancer.

From patent application US 20020186875 A1, 12.12.2002, a system for analyzing medical images, in particular, radiological images, is known. This decision is based on the analysis of image areas to determine the type of tissue and the possible presence of one or another pathology. Tissue classification is carried out by analyzing pixels in the studied window on the image and comparing the image with a given tissue pattern.

The disadvantage of the known solutions is the lack of accuracy and speed of machine learning algorithms for image classification, which is due to the following:

• Step-by-step analysis of the floating window image:

If the step is too small, then the analysis of the image of an ANN of a modern architecture with the number of parameters exceeding 20 million takes a long time (tens of minutes), even taking into account the use of modern tensor computers. Operational work requires access to a high-performance distributed computer.

If the step is too large, then the probability of skipping a window containing information sufficient for the correct classification of the ANN increases and, as a result, skipping a malignant neoplasm or other type of pathology that is critical for the patient's clinical picture.

SSD / YOLO-type neural network training. Neural networks of this type are built on top of the existing architecture (ResNet, Inception, VGG16). They require complex markup and long training, but due to the complexity of finding an optimal solution by a neural network for simultaneous segmentation and classification of the input image, the accuracy of such architectures in many cases does not exceed 70%, which is unacceptable for the problem of detecting symptoms of a dangerous disease.

The present development is aimed at solving a technical problem inherent in the known solutions from the prior art, in particular, by creating a new effective method for identifying neoplasms in X-ray images.

The technical result consists in increasing the accuracy of identification of neoplasms in X-ray images by augmentation of input images and their subsequent processing using trained ANNs.

The technical result is achieved by the fact that a computer-implemented method for identifying neoplasms on X-ray images is performed using at least one processor and containing the stages at which:

- get at least one X-ray image;

- carry out the processing of the above-mentioned image, during which its primary normalization and cleaning from noise is performed;

- perform the distribution of the image brightness map by RGB color channels, while

in the red channel, normalization, contrast enhancement and selection of the most contrasting contours in the image are carried out;

in the blue channel, the contrast of the processed image is increased by modifying its histogram;

its original information is saved in the green color channel;

- form an image based on processing RGB channels;

- performing the processing of the obtained image using a machine learning model, during which the classification of image areas is performed by selecting the reference points and sequential processing of these points using a floating window of a given pixel dimension;

- a vector is calculated for each said reference point containing numerical parameters for the criteria taken into account in the BI-RADS classification.

In one of the particular examples of the implementation of the method, the machine learning model is a single convolutional artificial neural network (ANN).

In another particular example of the implementation of the method, the accuracy of the ANN is additionally evaluated using a 4-channel ANN trained to take into account the characteristic vectors of the investigated image areas.

In another particular example of the implementation of the method, for each reference point, the fractal dimension is calculated using the algorithm for finding boundaries and contrasting areas.

In another particular example of the implementation of the method, the dimension of the floating window changes based on the dimension of the checked image, and its proportions of height to width are selected according to a given table.

In another particular example of the implementation of the method, noise removal is carried out using Gaussian blur filtering or combining the exclusion of external pixels with the subsequent closure of the contours.

In another particular example of the implementation of the method, the primary normalization is carried out using the MINMAX algorithm or centering by zero.

The claimed invention is also realized by a system for identifying neoplasms on X-ray images, comprising at least one processor, at least one memory means storing machine-readable instructions, which, when executed by the processor, perform the above method.

FIG. 1 shows the process of performing the claimed method.

FIG. 2 shows an example of an augmented image.

FIG. 3 shows a diagram of the image classification algorithm.

FIG. 4 shows an example of images for training ANN.

FIG. 5 shows an example of images with ANN markup.

FIG. 6 shows a general view of the computing system.

FIG. 1 shows a general scheme for implementing the claimed method (100) for identifying neoplasms on X-ray images. The claimed method (100) is implemented using a software and hardware complex (HSC), which can be any type of computer system, for example, a server. X-ray DICOM images (10), for example, mammography data, enter the PAK. At step (101), preprocessing of the resulting images is performed, in particular, their normalization, processing with an anti-noise filter. Normalization can be done using the MINMAX function (https://en.wikipedia.org/wiki/Minimax) or zero centering (https://www.quora.com/What-is-zero-centering-data -preprocessing-technique).

Filtering with Gaussian blur or combining the exclusion of external pixels with the subsequent closure of the contours can be used to filter the image from noise. Erosion and closing are well-known morphological algorithms (see, for example, https://docs.opencv.org/trunk/d9/d61/tutorial_py_morphological_ops.html).

Further, at step (102), the image brightness map (10) is distributed over three chromaticity channels (RGB channels) for subsequent augmentation, in particular, sequential augmentation of X-ray images (10) presented in the color mode of the brightness map (grayscale map or black and white picture). Augmentation is performed in order to accentuate and visualize the components of a black-and-white X-ray image (brightness maps) containing useful information.

Color channels are processed as follows. For the red color channel, several stages of normalization, contrast enhancement and highlighting of the most contrasting contours are performed. In this case, automatic or manual adjustment of the filter sensitivity is possible. The green color channel preserves the original image information (no transformations or processing occurs), which is necessary to prevent the loss of useful information as a result of applying filters to other channels. In the blue color channel, the contrast of image areas is increased by modifying its histogram in several stages, which makes it possible to create a gradient "highlight" of the brightest areas of the image regardless of their size.

In step (103) obtained in step (102), the luminance maps of each of the color channels are combined into a final color image (Fig. 2).

The use of augmentation allows you to solve the following tasks:

• Selection of relatively small or low-contrast areas of the image on the brightness map to simplify the reading of information by a human operator;

• Distribution of differentiated graphical descriptive information about contrast boundaries over three different chromaticity channels allows (as shown by a number of tests performed), to accelerate the training procedure for convolutional modules of artificial neural networks (accelerates the search for a solution by an optimizer algorithm in comparison with feeding the same brightness maps to three input channels ).

In step (104), the image obtained in step (103) is processed. FIG. 3 shows a general algorithm for image processing (10) using an ANN. Processing is carried out with the help of the location of the control points - "anchors" of the floating window of a given pixel dimension. The dimension of the window is determined by the choice of the number of checkpoints horizontally and vertically in the image (10), for example, 8x8 or 16x16 pixels. A customizable set of windows is used, the setting of which is flexible and implies a set of window sizes (length and width) in units proportional to the size of the image (for example, percent), sequentially applied and evaluated by the classifier from each "anchor". The proportions of height to width of the floating window can also be matched to the specified table.

In step (105), the classification of the image regions (10) for the reference points is performed. The classification of each such area of the image is carried out sequentially (parallelization of the algorithm is possible when accessing the cluster of calculators). At the same time, during training, the ANN is not trained to segment the image (10), which increases the final accuracy of the classifier (due to the use of a combined convolutional system, for example, YOLO (see https://appsilon.com/object-detection-yolo-algorithm/ )). The classification is carried out in accordance with the BI-RADS scale (Breast Imaging-Reporting and Data System, https://ru.wikipedia.org/wiki/BI-RADS). BI-RADS is a standardized scale for evaluating the results of mammography, ultrasound and MRI according to the degree of risk of breast cancer. For each section of the reference point (anchor), a feature vector is calculated, which makes it possible to classify a section of the image for belonging to a particular form of neoplasm.

The calculated vector at step (105) is a "harmony vector" of the image areas with preservation of information content. After the selected areas are classified by the main machine learning model, the accuracy of the 4-channel ANN complex is additionally evaluated, taking into account the characteristic vectors of the studied image areas.

The fractal dimension is calculated on image areas processed by the algorithm for searching and highlighting boundaries and contrasting areas (an example of such an algorithm is the Sobel / Laplace transform) with the condition of preserving areas containing information, binarization thresholds are determined recursively or cyclically, taking into account the condition for maintaining the number of image areas containing information, a site containing information is considered any area of uneven brightness for which the condition of maintaining the specified proportion of light to dark pixels is met.

The effectiveness of the claimed method (100) is based on the fact that the physical effect of the tumor on the surrounding tissues increases the complexity of their structure (non-standard distribution of tissue density) and similar changes are revealed when calculating the fractal complexity

FIG. 4 shows an example of input data that was used to train the ANN used in the work of the method (100). The input data of the complex are pixel arrays of the DICOM format. The number of floating point learning parameters supplied to the ANN input: ~ 59 million, single precision. Dimension of input data in pixels: 299 x 299 x 3. Model architecture: Convolutional ANN with Inception modules.

The sensitivity and specificity of the model for a threshold of 0.5 (softmax yield class) in clinical trials was 90% and 43%, respectively, for a threshold of 0.85 - 90% and 78%. Sensitive and specificity for laboratory tests, respectively: 96% and 87%.

The classification accuracy of the basic model used in method (100) based on training with procedural augmentation was ~ 99.8 -100%. FIG. 5 shows an example of the final classification of image areas. The convergence time of one instance of the model for a database of 3000 images on a configuration of two GPU / TPU Tesla V100 is 5 hours.

FIG. 6 shows a general view of a computing system (200) suitable for performing the method (100). In general, the system (200) contains components such as: one or more processors (201), at least one random access memory (202), persistent data storage (203), input / output interfaces (204), I / In (205), networking tools (206).

The processor (201) performs the basic computational operations necessary for the functioning of the system (200) or the functionality of one or more of its components. The processor (201) executes the necessary computer-readable instructions contained in the main memory (202).

Memory (202), as a rule, is made in the form of RAM and contains the necessary software logic that provides the required functionality. The data storage medium (203) can be performed in the form of HDD, SSD disks, raid array, network storage, flash memory, optical information storage devices (CD, DVD, MD, Blue-Ray disks), etc. The means (203) allows for long-term storage of various types of information, for example, the history of processing requests (logs), user IDs, camera data, images, etc.

Interfaces (204) are standard means for connecting and working with the system (200) or other computing devices. The interfaces (204) may represent, for example, USB, RS232, RJ45, LPT, COM, HDMI, PS / 2, Lightning, FireWire, etc. The choice of interfaces (204) depends on the specific implementation of the system (200), which can be a personal computer, mainframe, server cluster, thin client, smartphone, laptop, etc., as well as connected third-party devices.

As means of I / O data (205) can be used: keyboard, joystick, display (touch screen), projector, touchpad, mouse, trackball, light pen, speakers, microphone, etc.

Networking means (206) are selected from a device that provides network reception and transmission of data, for example, Ethernet card, WLAN / Wi-Fi module, Bluetooth module, BLE module, NFC module, IrDa, RFID module, GSM modem, etc. The tool (206) provides the organization of data exchange via a wired or wireless data transmission channel, for example, WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM.

The system components (200) are usually interfaced via a common data bus.

In the present application materials, the preferred disclosure of the implementation of the claimed technical solution was presented, which should not be used as limiting other, particular embodiments of its implementation, which do not go beyond the scope of the claimed scope of legal protection and are obvious to specialists in the relevant field of technology.

Claims (8)

1. A computer-implemented method for identifying neoplasms in X-ray images, characterized in that it is performed using at least one processor, and containing the stages, which: obtain at least one X-ray image; carry out the processing of the above-mentioned image, during which its primary normalization and cleaning from noise is performed; the distribution of the brightness map of the image according to the RGB chromaticity channels is performed, while in the red channel, normalization, increase in contrast and the selection of the most contrasting contours in the image are carried out; in the blue channel, the contrast of the processed image is increased by modifying its histogram; its original information is saved in the green color channel; form an image based on processing RGB channels; performing the processing of the obtained image using a machine learning model, during which the classification of image areas is performed by selecting the reference points and sequential processing of these points using a floating window of a given pixel dimension; calculating a vector for each said reference point containing numerical parameters for the criteria taken into account in the BI-RADS classification. 2. The method according to claim 1, characterized in that the machine learning model is an artificial neural network (ANN). 3. The method according to claim 2, characterized in that the accuracy of the ANN is additionally evaluated using a 4-channel ANN trained to take into account the characteristic vectors of the investigated image areas. 4. The method according to claim 1, characterized in that for each reference point, the fractal dimension is calculated using an algorithm for finding boundaries and contrasting areas. 5. The method according to claim 1, characterized in that the dimension of the floating window changes based on the dimension of the checked image, and its proportions of height to width are selected according to a given table. 6. The method according to claim 1, characterized in that the noise removal is carried out by filtering with Gaussian blur or by combining the exclusion of external pixels with subsequent closure of the contours. 7. The method according to claim 1, characterized in that the primary normalization is performed using the MINMAX function or centering on zero. 8. A system for identifying neoplasms in X-ray images, characterized in that it contains at least one processor, at least one memory means storing machine-readable instructions, which, when executed by the processor, perform the method according to any one of claims. 1-7.

Images