RU2813480C1 - Method of processing magnetic resonance imaging images for generating training data - Google Patents

Abstract

FIELD: physics; image processing.

SUBSTANCE: invention can be used for magnetic resonance imaging (MRI) processing. Magnetic resonance imaging (MRI) for generating training data is implemented by a processing device comprising a processor, memory and machine-readable instructions which cause the processor to perform the following steps: receiving magnetic resonance imaging (MRI) images of patients; processing of the obtained images, at which: determining the brightness value of each pixel; calculating the brightness histogram along the Y axis, for which: calculating the brightness of each line by adding the brightness of the pixels of each line, determining the average brightness value for each line; calculating the root-mean-square deviation of brightness of all pixels of a line from the average for the same line and calculating for each line the product of the sum of brightness by the root-mean-square deviation of brightness; calculating a brightness histogram along the X axis, for which: calculating the brightness of each column by adding the brightness of the pixels of each column, determining the average brightness value for each column; calculating the root-mean-square deviation of the brightness of all pixels of the column from the average for the same column and calculating for each column the product of the sum of the brightness by the root-mean-square deviation of brightness; setting threshold values of brightness; excluding on the Y axis lines, the brightness value for which is below the first threshold value, and on the X axis columns, the brightness value for which is below the second threshold value; normalizing the obtained images; outputting the final image to the input of the neural network for training; storing the obtained neural network model on a processing device.

EFFECT: high accuracy of determining anomalies in MRI images.

14 cl, 2 dwg

Description

Field of technology

The present invention relates to magnetic resonance imaging (MRI) image processing using neural networks to detect abnormalities in the brain.

State of the art

Medical imaging is becoming increasingly important in modern medicine. In some cases, image processing, such as image segmentation and/or image enhancement, is performed on medical images to help doctors make diagnoses. It is therefore desirable to provide systems and a method for implementing more accurate and more efficient image segmentation and/or image enhancement.

Learning-based approaches aim to create a predictive model based on available training data. Once the prediction model has been trained, it can be applied to any new MRI image of the same type. Specifically, a supervised learning algorithm is used that uses a known set of input data and known answers or conclusions to that data, and then trains the model to generate reasonable predictions to respond to the new data.

The image obtained during MRI has a different picture shape. The scanning area includes a significant amount of surrounding tissue that is not related to the subject of study. These extra areas create additional noise when training neural networks, leading to overtraining and decreased accuracy. This problem is especially common when working with medical data, since patient data is always limited. Various image preprocessing methods are used to highlight the area of interest in the picture.

Thus, an image processing system is known in the prior art (US 11475569 B2), which uses a trained neural network that processes an image of the patient's chest to obtain an object area and a background area (image segmentation). To train a neural network, a binary image of the object area and the background area is created by adding the pixel brightness along the Y axis. The area whose pixel brightness does not satisfy the threshold value is removed.

To deliver an accurate diagnosis, you should select the smallest possible area without loss of information. An MRI image is a layered image, each slice of which has its own thickness. Therefore, the use of a static (unchanging) threshold value allows us to filter out only a small amount of unnecessary information, which leads to noisy training data and, as a consequence, to inaccurate operation of the neural network.

Therefore, the main challenge is to prepare more informative training data to detect anomalies and make more accurate diagnoses.

To solve this problem, the claimed solution uses a new approach to preparing training data, which uses dynamic threshold values on each layer of the MRI image, which changes depending on the brightness values of the pixels along the X and Y axes.

The technical result achieved by the claimed solution is to increase the accuracy of detecting anomalies in MRI images.

Disclosure of the Invention

The stated goal, the technical result required and obtained when using the invention is achieved by the fact that the method of processing magnetic resonance imaging (MRI) images to generate training data is implemented by a processing device containing a processor, memory and machine-readable instructions that induce the processor to perform the following steps:

receiving magnetic resonance imaging (MRI) images of patients;

processing of the received images, in which:

determine the brightness value of each pixel;

calculate the brightness histogram along the Y axis, for which: calculate the brightness of each line by adding the brightness of the pixels of each line, determine the average brightness value for each line; the standard deviation of the brightness of all pixels in a row from the average for the same line is calculated, and for each line the product of the sum of brightnesses and the standard deviation of brightness is calculated;

calculate the brightness histogram along the X axis, for which: calculate the brightness of each column by adding the brightness of the pixels of each column, determine the average brightness value for each column; the standard deviation of the brightness of all pixels in a column from the average for the same column is calculated, and for each column the product of the sum of the brightnesses and the standard deviation of the brightness is calculated;

set brightness thresholds;

exclude along the Y axis rows whose brightness value is below the first threshold value, and along the X axis columns whose brightness value is below the second threshold value;

carry out normalization of the obtained images;

the final image is supplied to the input of the neural network for training;

the resulting neural network model is stored on the processing device.

In this case, the first threshold value is set to 10% of the maximum brightness histogram value for the Y axis.

In this case, the second threshold value is set to 5% of the maximum value of the brightness histogram for the X axis.

In addition, image normalization includes:

setting the minimum brightness value as the 10% percentile of all patient MRI images;

setting the maximum brightness value as the 99% percentile of all patient MRI images;

subtracting the average brightness value for all patients from all brightness values;

dividing the resulting brightness value by the average variance over the entire data set.

In this case, the final image, which is fed to the input of the neural network, has a dimension of N*224*224.

Moreover, the first and second threshold values are calculated for each layer of MRI images.

The stated goal, the technical result required and obtained by using the invention is achieved by the fact that the method for diagnosing anomalies in magnetic resonance imaging (MRI) images of a patient using a neural network contains the following steps:

scanning a patient with a medical imaging device;

sending the resulting MRI image to the processing device;

MRI image processing by a neural network stored in the processing device and trained in the method according to point 1;

highlighting areas identified as anomalous in the image;

sending an image with selected areas to the client terminal and/or storage device.

In this case, images are taken in real time.

Also imaged are MRI images of the brain.

Moreover, the anomaly is a tumor.

In addition, the selection of areas is color-coded, identifying the most dangerous areas.

The goal, the technical result required and obtained when using the invention is achieved by the fact that a system for diagnosing anomalies in magnetic resonance imaging (MRI) images using a neural network, containing: a medical imaging device, a processing device, a storage device, interconnected via a network, one or several client terminals, in which:

the medical device sends the magnetic resonance imaging (MRI) image of the patients to the processing device;

The processing device is configured to:

receiving the original magnetic resonance imaging (MRI) image of the patient;

processing the original MRI image with a neural network trained using the method according to point 1;

highlighting areas identified as anomalous;

sending an image with selected areas to the client terminal and/or storage device.

In this case, images are taken in real time.

Also, the client terminal is the terminal of a medical professional or patient.

Thus, the technical result is achieved by processing MRI images using a neural network trained as follows:

for each of the set of MRI images used as training data:

determine the brightness value of each pixel in the image;

calculate the brightness histogram along the Y axis, for which: calculate the brightness of each line by adding the brightness of the pixels of each line, determine the average brightness value for each line; the standard deviation of the brightness of all pixels in a row from the average for the same line is calculated, and for each line the product of the sum of brightnesses and the standard deviation of brightness is calculated;

calculate the brightness histogram along the X axis, for which: calculate the brightness of each column by adding the brightness of the pixels of each column, determine the average brightness value for each column; the standard deviation of the brightness of all pixels in a column from the average for the same column is calculated, and for each column the product of the sum of the brightnesses and the standard deviation of the brightness is calculated;

brightness thresholds are set for each image layer;

exclude along the Y axis rows whose brightness value is below the first threshold value, and along the X axis columns whose brightness value is below the second threshold value;

carry out normalization of the obtained images;

and the final image is fed to the input of the neural network for training.

Brief description of drawings

The essence of the invention is illustrated by drawings.

In fig. Figure 1 shows a diagram of the neural network architecture for the task of classifying a multilayer MRI image, where 1) are images of MRI layers; 2) - conditional image of the architecture of the convolutional part of the neural network; 3) - operation of selecting the maximum values of the output vector among all MRI layers; 4) - classifier for the required number of classes; 5) - the total weights of the layers of the neural network.

In fig. 2 shows a graph of the loss function on training data during the learning process, where 1) - preprocessed; 2) - original size.

Carrying out the invention

The claimed invention discloses a system for diagnosing anomalies in magnetic resonance imaging (MRI) images, which contains a medical imaging device, a processing device, a data storage device, and one or more client terminals connected through a network.

A medical imaging device is an imaging device that is used to scan an object located within its detection area and generate a plurality of scan data (eg, digital signals) used to generate one or more images related to the object. In the present disclosure, the term “object” may include a specific part, an organ of a patient. For example, the object may include a brain.

The processing device may process data and/or information received from the medical imaging device and/or storage device. For example, a processing device may generate one or more medical images (eg, brain images) by processing scan data (eg, digital signals) from a medical imaging device. In some embodiments, the processing device may be a computer or a server or group of servers. A group of servers can be centralized or distributed. In some embodiments, the processing device may be local or remote.

The network may include any suitable network that can facilitate the exchange of information and/or data for the anomaly diagnostic system. In some embodiments, the network may be and/or include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN), a wide area network (WAN), etc.), a wired network (e.g., Ethernet network), wireless network (e.g., 802.11 network, Wi-Fi network, etc.), cellular network (e.g., Long Term Evolution (LTE) network), frame relay network, virtual private network (“VPN”), satellite network, telephone network, wireless local area network (WLAN).

The client terminal includes a mobile device, a tablet computer, a laptop computer, etc., or any combination thereof.

The storage device may be connected to a network for communication with one or more components of the anomaly diagnostic system (eg, medical imaging device, processing device, client terminal 13, etc.). In some embodiments, a storage device may be directly connected to or interface with one or more system components, or be part of a processing device, or located on one or more servers.

The claimed diagnostic system uses a deep learning approach to classify brain MRI findings as “probably normal” or “probably abnormal.”

When solving an MRI classification problem, the class labels used for training are “weak”. “Weak” means that for all MRI layers of which there may be more than several hundred, we have only one healthy/disease detected label. That is, some of the layer images have signs of disease, but for the most part they are absent. Therefore, it is preferable to use a neural network architecture in which all the layers' images are fed simultaneously FIG. 1.

In fig. Figure 1 shows a diagram of the neural network architecture for the task of classifying a multilayer MRI image, where 1) are images of MRI layers; 2) - conditional image of the architecture of the convolutional part of the neural network; 3) - operation of selecting the maximum values of the output vector among all MRI layers; 4) - classifier for the required number of classes; 5) - the total weights of the layers of the neural network.

The dimensionality of such input data greatly exceeds the number of available MRI samples for each data class. For example, in the open dataset [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification", arXiv:2107.02314, 2021] the dimension ranges from 40×256×256 to 176×256×256. In connection what causes the problem of noise, which does not allow the neural network to learn and detect anomalies qualitatively.In Fig. 2 it can be seen that the total error for all raw data fluctuates around a certain value (curve 2), while it should decrease with iterations (curve 1 for preprocessed data).

The neural network used in this solution was trained on a large heterogeneous data set covering a wide range of pathological (abnormal) conditions, including neoplasms, hemorrhages, heart attacks and others.

To train the neural network, historical patient data is used, in particular, MRI images of the brain with diagnosed diagnoses, collected over a certain period of time and stored on a storage device.

An MRI image is a layered image, each slice of which has its own thickness.

For each layer of the MRI image there is a histogram of pixel brightness along the X and Y axes.

The algorithm for calculating brightness along the X axis is as follows:

add the brightness values of the pixels of each column, thereby obtaining the brightness of each column;

determine the average brightness value for each column;

calculating the standard deviation of the brightness value for each column;

calculate the product of the sum of the brightness values and the standard deviation for each column.

The algorithm for calculating brightness along the Y axis is as follows:

add the brightness values of the pixels of each row, thereby obtaining the brightness of each row;

determine the average brightness value for each line;

calculating the standard deviation of the brightness value for each line;

calculate the product of the sum of the brightness values and the standard deviation for each line.

Next, for each layer, a threshold value is set along the X axis and a threshold value along the Y axis, that is, the threshold value changes dynamically from layer to layer. Each threshold value depends on the maximum value of the X and Y luminance histograms on each layer. So the X-axis threshold (first threshold) is set to 5% of the maximum value of the X-axis brightness histograms, and the Y-axis threshold (second threshold value) is set to 10% of the maximum value of the Y-axis brightness histograms.

The percentages indicated are selected empirically in order to maintain a certain balance. On the one hand, to cut out the smallest possible area of the MRI, but at the same time, so that it includes all the signs of the required pathology (anomaly).

Available open data on brain MRI were used in the selection process. The selection mechanism includes the following steps:

- random subsamples with different numbers of MRIs are created,

- optimal thresholds are calculated for each subsample, allowing the neural network to be trained to maximum accuracy,

- the mathematical expectation of the distribution of thresholds of subsamples from the available set of MRI images is taken as an estimate of the threshold.

- the value is rounded in increments of 5, since the estimate is quite rough.

After that, rows and columns whose brightness values are below the set threshold are excluded from the image. That is, uninformative areas are excluded in which the X-axis excludes columns whose brightness value is below the second threshold value, and the Y-axis excludes rows whose brightness value is lower than the first threshold value.

To improve the stability of training, the brightness values are normalized in order to group them around the zero value (preprocessing algorithm). Set the minimum brightness value as the 10% percentile of all MRI images of the patient and the maximum brightness value as the 99% percentile of all MRI images of the patient. Then the average brightness value for all patients is subtracted from all values, and the resulting brightness value is divided by the average variance over the entire data set.

The above percentages are due to the following. Signs of diseases on MRI are contrasting areas that differ in shape and brightness from normal tissue. In this case, the research conditions are selected specifically so that the necessary pathologies can be seen in sufficient contrast. Because of this, the brightness values of the pixels are in a very wide range, which depends, among other things, on the density of the tissues of a particular person. Dynamic range based on quantile calculation allows you to normalize all values in the range [-1,1] and at the same time filter out outliers on both sides, as opposed to normalizing using minimum and maximum brightness. The upper value of 99% cuts out too bright hard tissues of the skull and possible random bright objects in the image, but does not affect the highlighted contrast areas of pathologies. The lower value of 10% was selected based on the amount of noise observed in the images.

After the preprocessing algorithm, the final data array supplied to the input of the neural network has a dimension of N*224*224, where N is the number of MRI layers. The use of dimension 224 is due to the fact that in this case, processing such data requires smaller memory, and the necessary visibility of pathology signs is preserved.

The trained neural network is stored on the processing device for subsequent diagnosis of anomalies in the patient’s MRI images.

The operation of the diagnostic system is as follows.

The processing device is connected via a network to a medical imaging device, which creates an MRI image of the brain in real time by scanning an organ of interest, such as the head.

By sending a signal to the medical device, the MRI image is automatically transferred to the processing device.

The processing device on which the trained neural network is located processes the MRI image, for which, on each layer, areas are cut off for which the pixel brightness value along the X and Y axes is below the established threshold values.

As a result of processing, the image is classified as “probably normal” or “probably abnormal” while simultaneously highlighting abnormal areas.

After which, the processing result is stored on a storage device for subsequent use by a medical professional and/or sent in real time to the doctor’s client device.

In one embodiment, two neural networks may be used. The first neural network determines the gray level for each pixel and excludes regions for which the value is less than a threshold (background region).

After which the second neural network, the learning algorithm of which is described above, processes the remaining area (brain area), classifying the image and identifying areas of anomalies.

As follows from the description of the claimed invention, the proposed method for processing magnetic resonance imaging (MRI) images to generate training data ensures the achievement of the stated technical result, namely, increases the accuracy of determining anomalies in MRI images.

The analysis shows that all the general and specific features of the group of inventions are essential, since each of them is necessary for the industrial implementation of the claimed group of inventions, and together the features make it possible to achieve the declared technical result.

Thus, in the sections “State of the art” and “Disclosure of the invention” it was shown that all general and specific features of the proposed solution are essential and together satisfy the patentability criterion of “novelty”. In the section “Implementation of the invention” it was shown that the claimed group of inventions is technically feasible, allows solving the stated inventive problems and confidently achieving the required technical result when using it (the invention), which indicates the industrial applicability of the proposed method and device. Based on the above, we believe that the claimed method for processing magnetic resonance imaging (MRI) images to generate training data, as well as a method for diagnosing anomalies in magnetic resonance imaging (MRI) images of a patient using a neural network and a system for diagnosing anomalies in images magnetic resonance imaging (MRI) using a neural network, satisfy all the requirements for patentability for inventions.

Claims (39)

1. A method of processing magnetic resonance imaging (MRI) images to generate training data, implemented by a processing device comprising a processor, memory, and machine-readable instructions causing the processor to perform the following steps: receiving magnetic resonance imaging (MRI) images of patients; processing of the received images, in which: determine the brightness value of each pixel; calculate the brightness histogram along the Y axis, for which: calculate the brightness of each line by adding the brightness of the pixels of each line, determine the average brightness value for each line; the standard deviation of the brightness of all pixels in a row from the average for the same line is calculated, and for each line the product of the sum of brightnesses and the standard deviation of brightness is calculated; calculate the brightness histogram along the X axis, for which: calculate the brightness of each column by adding the brightness of the pixels of each column, determine the average brightness value for each column; the standard deviation of the brightness of all pixels in a column from the average for the same column is calculated, and for each column the product of the sum of the brightnesses and the standard deviation of the brightness is calculated; set brightness thresholds; exclude along the Y axis rows whose brightness value is below the first threshold value, and along the X axis columns whose brightness value is below the second threshold value; carry out normalization of the obtained images; the final image is supplied to the input of the neural network for training; the resulting neural network model is stored on the processing device. 2. The method according to claim 1, characterized in that the first threshold value is set to 10% of the maximum value of the brightness histogram for the Y axis. 3. The method according to claim 1, characterized in that the second threshold value is set to 5% of the maximum value of the brightness histogram for the X axis. 4. The method according to claim 1, characterized in that image normalization includes: setting the minimum brightness value as the 10% percentile of all patient MRI images; setting the maximum brightness value as the 99% percentile of all patient MRI images; subtracting the average brightness value for all patients from all brightness values; dividing the resulting brightness value by the average variance over the entire data set. 5. The method according to claim 1, characterized in that the final image, which is fed to the input of the neural network, has a dimension of N*224*224. 6. The method according to claim 1, characterized in that the first and second threshold values are calculated for each layer of the MRI images. 7. A method for diagnosing anomalies in magnetic resonance imaging (MRI) images of a patient using a neural network, comprising the steps: scanning the patient with a medical imaging device; sending the resulting MRI image to a processing device; processing the MRI image with a neural network stored in the processing device and trained by the method according to claim 1; highlighting areas identified as abnormal in the image; sending an image with selected areas to the client terminal and/or storage device. 8. The method according to claim 6, characterized in that the images are received in real time. 9. The method according to claim 6, characterized in that the images are MRI images of the brain. 10. The method according to claim 6, characterized in that the anomaly is a tumor. 11. The method according to claim 6, characterized in that the selection of areas is color-coded, identifying the most dangerous areas. 12. A system for diagnosing anomalies in magnetic resonance imaging (MRI) images using a neural network, containing: a medical imaging device, a processing device, a storage device, one or more client terminals interconnected via a network, in which: the medical device sends the magnetic resonance imaging (MRI) image of the patients to the processing device; The processing device is configured to: receiving the original magnetic resonance imaging (MRI) image of the patient; processing the original MRI image with a neural network trained by the method according to claim 1; highlighting areas identified as abnormal; sending an image with selected areas to the client terminal and/or storage device. 13. The system according to claim 12, characterized in that images are received in real time. 14. The system according to claim 12, characterized in that the client terminal is a medical professional or patient terminal.