CN111815558A

CN111815558A - Medical image processing system, method and computer storage medium

Info

Publication number: CN111815558A
Application number: CN202010500328.5A
Authority: CN
Inventors: 吕旭阳; 廖术
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2020-06-04
Filing date: 2020-06-04
Publication date: 2020-10-23

Abstract

The invention discloses a medical image processing system, a method and a computer storage medium, wherein the system comprises: the system comprises a first terminal, a cloud server and a second terminal, wherein the first terminal is used for acquiring a medical image from medical imaging equipment and sending the medical image to the cloud server; a cloud server for receiving the medical image, determining a first quality level of the medical image based on a quality assessment model; when the first quality grade is matched with the preset quality grade, reconstructing the medical image based on the image reconstruction model to obtain a reconstructed medical image; returning the reconstructed medical image to the first terminal; the quality evaluation model is determined by machine learning training based on the first sample medical image and the corresponding quality grade label; the image reconstruction model is determined by machine learning training based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image. The invention reduces the reconstruction cost of the medical image and improves the reconstruction efficiency and the quality of the reconstructed image.

Description

Medical image processing system, method and computer storage medium

Technical Field

The present invention relates to the field of medical image processing technologies, and in particular, to a medical image processing system, method, and computer storage medium.

Background

Magnetic Resonance (MR) imaging techniques are not only ideal in terms of tissue contrast, spatial resolution, etc., but also have good imaging effects on soft tissues, and can provide more abundant image information, thus being widely applied. However, the imaging speed of the magnetic resonance imaging technology is slow, and the imaging time is too long, so that some patients can involuntarily move in the imaging process to generate motion artifacts, thereby affecting the imaging quality.

In order to obtain high-quality MR images in a short time, in the related art, a device manufacturer may provide a hardware device, i.e., a reconstruction workstation, for the MR imaging device, and reconstruct the MR images through the reconstruction workstation to obtain reconstructed MR images with better quality, but this way of reconstructing MR images not only has a slow reconstruction speed, but also needs to rely on expensive hardware devices, resulting in high reconstruction cost of MR images.

Disclosure of Invention

In order to solve the problems of the prior art, embodiments of the present invention provide a medical image processing method, system and computer storage medium. The technical scheme is as follows:

in one aspect, a medical image processing system is provided, the system comprising:

the system comprises a first terminal, a cloud server and a second terminal, wherein the first terminal is used for acquiring a medical image from medical imaging equipment and sending the medical image to the cloud server;

the cloud server is used for receiving the medical image and determining a first quality level of the medical image based on a quality evaluation model; when the first quality grade is matched with a preset quality grade, reconstructing the medical image based on an image reconstruction model to obtain a reconstructed medical image; returning the reconstructed medical image to the first terminal;

wherein the quality evaluation model is determined by machine learning training based on the first sample medical image and the corresponding quality grade label; the image reconstruction model is determined by machine learning training based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image.

Optionally, the returning, by the cloud server, the reconstructed medical image to the first terminal includes:

the cloud server carries out diagnosis analysis on the reconstructed medical image based on a diagnosis analysis model to obtain a diagnosis analysis result corresponding to the medical image;

the cloud server determines a target historical treatment scheme matched with the diagnosis analysis result in a historical treatment scheme library, wherein the historical treatment scheme library stores the corresponding relation between the historical treatment scheme and the historical diagnosis analysis result;

the cloud server establishing a first mapping relationship between the reconstructed medical image, the diagnostic analysis results and the target historical treatment plan;

the cloud server returns the first mapping relation to the first terminal;

wherein the diagnostic analysis model is determined by machine learning based on the second sample medical image and corresponding diagnostic label information; the diagnostic label information includes a disease type and/or a lesion type.

Optionally, the diagnostic analysis model includes a plurality of diagnostic analysis models corresponding to different diagnostic analysis tasks;

the cloud server performing diagnostic analysis on the reconstructed medical image based on a diagnostic analysis model comprises:

the cloud server determines a current diagnosis and analysis task according to the body part corresponding to the reconstructed medical image;

the cloud server determines a target diagnosis analysis model matched with the current diagnosis analysis task in the diagnosis analysis models;

the cloud server performs diagnostic analysis on the reconstructed medical image based on the target diagnostic analysis model.

Optionally, the cloud server is further configured to:

when the first quality grade is not matched with the preset quality grade, performing diagnostic analysis on the medical image based on the diagnostic analysis model to obtain a diagnostic analysis result corresponding to the medical image;

determining a target historical treatment scheme matched with the diagnosis and analysis result in a historical treatment scheme library, wherein the historical treatment scheme library stores the corresponding relation between the historical treatment scheme and the historical diagnosis and analysis result;

establishing a second mapping relationship between the medical image, the diagnostic analysis results, and the target historical treatment plan;

and returning the second mapping relation to the first terminal.

Optionally, the cloud server is further configured to: training the quality assessment model based on a first sample medical image and a quality grade label corresponding to the first sample medical image; the cloud server training the quality assessment model comprises:

acquiring a first sample medical image set; the first sample medical image in the first sample medical image set is a medical image obtained by sampling based on the same undersampling rate; each first sample medical image is labeled with a quality grade label;

determining the first sample medical image with the quality grade label matched with the preset quality grade as a positive sample medical image to obtain a positive sample image set;

determining the first sample medical image with the quality grade label not matched with the preset quality grade as a negative sample medical image to obtain a negative sample image set;

training a preset first neural network model based on the positive sample image set and the negative sample image set, and adjusting model parameters of the preset first neural network model in the training process until the prediction quality grade output by the preset first neural network model is matched with the input sample image;

and taking the first neural network model corresponding to the current model parameter as the quality evaluation model.

Optionally, the cloud server is further configured to: training the image reconstruction model based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image; the training of the image reconstruction model by the cloud server comprises:

acquiring a fully sampled medical image and an under-sampled medical image corresponding to the fully sampled medical image; the fully sampled medical images comprise fully sampled medical images corresponding to a plurality of coil channels of the medical imaging device;

taking the fully sampled medical image and the corresponding undersampled medical image as a training data pair;

based on the training data pair, performing image reconstruction training by using a preset second neural network model, and adjusting the model parameters of the preset second neural network model in the training process until a predicted image output by the preset second neural network model is matched with a fully sampled medical image in an input training data pair;

and taking the second neural network model corresponding to the current model parameters as the image reconstruction model.

Optionally, the first terminal is further configured to: responding to an adjustment instruction of a user for a diagnosis and analysis result in the mapping relation, and generating an adjusted mapping relation; sending the adjusted mapping relation to the cloud server;

the cloud server is further configured to: and training and updating the diagnostic analysis model by using the adjusted mapping relation according to a preset time interval.

Optionally, the first terminal is further configured to: acquiring a current treatment scheme, and sending the current treatment scheme to the cloud server;

the cloud server is further configured to: receiving the current treatment plan; establishing a corresponding relation between the current treatment scheme and the diagnosis and analysis result; and updating the corresponding relation between the current treatment scheme and the diagnosis and analysis result to the historical treatment scheme library.

In another aspect, a medical image processing method is provided, which is applied to a cloud server, and the method includes:

receiving a medical image sent by a first terminal, wherein the medical image is acquired by the first terminal from a medical imaging device;

determining a first quality level of the medical image based on a quality assessment model;

when the first quality grade is matched with a preset quality grade, reconstructing the medical image based on an image reconstruction model to obtain a reconstructed medical image;

returning the reconstructed medical image to the first terminal;

wherein the quality evaluation model is determined by machine learning training based on a first sample medical image and a quality grade label corresponding to the first sample medical image; the image reconstruction model is determined by machine learning training based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image.

Optionally, the method further includes: training the quality assessment model based on a first sample medical image and a quality grade label corresponding to the first sample medical image;

the training the quality assessment model comprises:

Optionally, the method further includes: training the image reconstruction model based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image;

the training the image reconstruction model comprises:

In another aspect, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, which is loaded and executed by a processor to implement the medical image processing method as described above.

The medical image processing system in the embodiment of the invention comprises a first terminal and a cloud server, wherein the first terminal is used for acquiring a medical image from a medical imaging device and sending the medical image to the cloud server, the cloud server is used for receiving the medical image and determining a first quality level of the medical image based on a quality evaluation model, when the first quality grade is matched with the preset quality grade, the medical image is reconstructed based on the image reconstruction model to obtain a reconstructed medical image, and the reconstructed medical image is returned to the first terminal, wherein the quality assessment model is determined by machine learning training based on the first medical image and the corresponding quality level label, the image reconstruction model is determined by machine learning training based on the fully sampled medical image and the undersampled medical image corresponding to the fully sampled medical image. Therefore, the system does not need to rely on an expensive reconstruction workstation for the reconstruction of the medical image, and the cloud server is combined with the machine learning model obtained by pre-training on the cloud server to complete the reconstruction of the medical image, so that the reconstruction cost of the medical image is reduced, and the reconstruction efficiency and the quality of the reconstructed image are greatly improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a medical image processing system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of another medical image processing system according to an embodiment of the present invention;

fig. 3 is a schematic process diagram of a cloud server training quality assessment model according to an embodiment of the present invention;

fig. 4 is a schematic process diagram of training an image reconstruction model by a cloud server according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of another medical image processing system according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of another medical image processing system according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of another medical image processing system according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of another medical image processing system according to an embodiment of the present invention;

FIG. 9 is a flow chart of a medical image processing method according to an embodiment of the present invention;

FIG. 10 is a flow chart illustrating another medical image processing method according to an embodiment of the present invention;

fig. 11 is a block diagram of a hardware structure of a server according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Referring to fig. 1, a schematic structural diagram of a medical image processing system according to an embodiment of the invention is shown. As shown in fig. 1, the medical image processing system 100 may include a first terminal 110 and a cloud server 120.

The first terminal 110 and the cloud server 120 may be in connected communication through a network, which may be a wired network or a wireless network. In the embodiment of the present invention, in order to effectively increase the transmission speed of medical image data and increase the efficiency of image reconstruction, a network connection may be established between the first terminal 110 and the cloud server 120 based on a fifth-generation mobile communication technology. The fifth generation mobile communication technology has the characteristics of high data rate, low delay, energy conservation, large-scale equipment connection and the like, the data transmission rate of the fifth generation mobile communication technology is far higher than that of the prior cellular network, the maximum data transmission rate can reach 10Gbit/s, and the network delay is lower than 1 millisecond.

Specifically, the first terminal 110 may be configured to acquire a medical image from a medical imaging device and send the medical image to the cloud server.

The first terminal 110 may be a hardware device having various operating systems, such as a desktop computer, a tablet computer, and a notebook computer. The first terminal 110 may have the image processing client 101 running therein, and the user may log in the cloud server 120 through the image processing client 101 in the first terminal 110, in a specific implementation, the image processing client 101 may be a web client.

The medical imaging device may be, but is not limited to, the Magnetic Resonance (MR) imaging device shown in fig. 1, and accordingly the medical image may be, but is not limited to, the Magnetic Resonance (MR) image shown in fig. 1. The first terminal 110 may establish an intranet connection with the medical imaging device through the local area network, after the medical imaging device obtains a medical image after completing scanning, the medical image may be sent to the first terminal 110 based on the established intranet connection, and accordingly, the first terminal 110 receives the medical image, and then the first terminal 110 may log in the cloud server 120 through the image processing client 101 thereon, and send the medical image to the cloud server 120, and perform image reconstruction processing on the medical image through the cloud server 120.

The cloud server 120 may be a single server or a server cluster of multiple servers. The cloud server 120 may be configured to receive the medical image sent by the first terminal 110, and determine a first quality level of the medical image based on the quality evaluation model; when the first quality level is matched with the preset quality level, the medical image is reconstructed based on the image reconstruction model to obtain a reconstructed medical image, and the reconstructed medical image is returned to the first terminal 110, so that the medical image is reconstructed by using the cloud resources. The quality evaluation model is determined by machine learning training based on a first sample medical image and a corresponding quality grade label, and the image reconstruction model is determined by machine learning training based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image.

In a specific implementation, as shown in fig. 2, the cloud server 120 may include a web server 121 and a medical image processing server 122, which are in direct communication with the web client, where the web server 121 is configured to receive the medical image sent by the first terminal 110, and forward the medical image to the medical image processing server 122, and the medical image processing server 122 performs subsequent medical image processing.

In this embodiment of the present specification, the preset quality level refers to a quality level corresponding to a medical image that needs to be reconstructed, that is, when a quality level of a certain medical image meets the preset quality level, the medical image is the medical image that needs to be reconstructed. In a specific implementation, the preset quality level can be determined by a doctor according to the artifact degree and the tissue structure definition degree of the medical image.

In a possible embodiment, the representation of the preset quality level may be a generally described quality level, for example, the preset quality level is a low quality level, that is, the medical image with the low quality level is the medical image to be reconstructed.

In another possible embodiment, the representation form of the preset quality level may also be a specific quality level value, for example, the quality level 1, that is, only the medical image with the quality level 1 is the medical image to be reconstructed; of course, the preset quality level may also be a specific quality level numerical range, such as quality levels 1 to 3, that is, the medical images with quality levels of 1 or 2 or 3 are all the medical images that need to be reconstructed.

In the embodiment of the invention, the cloud server 120 pre-trains and stores the quality evaluation model and the image reconstruction model based on the machine learning algorithm, and can directly call the trained model for processing when receiving the medical image.

Based on this, in the embodiment of the present invention, the cloud server 120 is further configured to train the quality assessment model based on the first sample medical image and the quality level label corresponding to the first sample medical image. The specific process of the cloud server 120 training the quality assessment model is described below.

As shown in fig. 3, the cloud server 120 acquires a first sample medical image set, where the first sample medical images in the first sample medical image set are medical images sampled based on the same undersampling rate, and each first sample medical image is labeled with a corresponding quality level label.

Specifically, the quality level label labeled on the first medical image needs to refer to the representation form of the preset quality level label. For example, when the representation of the preset quality level is a low quality level, the labeled quality level label may include a high quality level and a low quality level; when the expression form of the preset quality grade is a specific quality grade numerical value or a quality grade numerical value range, the labeled quality grade label is the specific quality grade numerical value. In practice, the user may select the specific representation of the quality grade according to the requirement. In a specific implementation, the user may label the quality level label of the first sample medical image according to the signal-to-noise ratio, the degree of the artifact and the degree of the tissue structure clarity.

For example, when the expression form of the preset quality level is a low quality level, the user may label the medical image with a signal-to-noise ratio lower than a preset signal-to-noise ratio threshold, an obvious artifact, and an unclear tissue structure as the low quality level, label the medical image with a signal-to-noise ratio higher than the preset signal-to-noise ratio threshold, and substantially no artifact, and a clearer tissue structure as the high quality level.

It should be noted that, in the embodiment of the present invention, the first medical image in the first medical image set used for training the quality assessment model is a medical image sampled based on the same undersampling rate. The obtained medical images are adopted as the training data of the quality evaluation model at the same undersampling rate, so that the influence of poor quality images on the quality evaluation model evaluation result caused by undersampling operation can be eliminated, and the quality evaluation model can screen out the poor quality images caused by patient movement or improper operation of technicians in the imaging process, such as medical images with extremely low signal-to-noise ratio, obvious artifacts and unclear tissue structures.

That is to say, in the embodiment of the present invention, the difference of the undersampling rates is not used as a basis for determining the quality level of the medical image by the quality assessment model, because different undersampling rates generally cause the difference of the medical image in the imaging quality, so that the quality assessment model obtained by training can be based on the influence of factors other than the undersampling rate on the image quality level, for example, the influence of factors such as artifacts and signal-to-noise ratio can be considered.

After the first sample medical image set is acquired, the cloud server 120 divides the first sample medical image into positive and negative samples. It should be noted that, in the embodiment of the present invention, the positive and negative samples are divided from the perspective of image reconstruction, that is, the sample requiring image reconstruction is determined to be a positive sample, and the sample not requiring image reconstruction is determined to be a negative sample.

Specifically, the cloud server 120 determines the first sample medical image with the quality level label matched with the preset quality level as a positive sample medical image, so as to obtain a positive sample image set; and determining the first sample medical image with the quality grade label not matched with the preset quality grade label as a negative sample medical image to obtain a negative sample image set. The quality grade label is matched with a preset quality grade, which indicates that the corresponding first sample medical image meets the requirement of reconstruction, and the first sample medical image is used as a positive sample medical image; and the quality grade label is not matched with the preset quality grade, which indicates that the corresponding first sample medical image does not meet the requirement of reconstruction and is used as the negative sample medical image.

After the positive and negative samples of the first sample medical image set are divided, training data can be obtained, and the cloud server 120 performs specific training of the quality evaluation model by using the training data. Specifically, the cloud server 120 trains a preset first neural network model based on the positive sample image set and the negative sample image set, and adjusts the model parameters of the preset first neural network model during the training until the predicted quality level output by the preset first neural network model matches with the input sample image.

The cloud server 120 uses the first neural network model corresponding to the current model parameter (i.e., the model parameter when the predicted quality level output by the preset first neural network model matches with the input sample image) as the quality evaluation model of the embodiment of the present invention.

In practical applications, the cloud server 120 needs to first construct the preset first neural network model, where the preset first neural network model may be an image classification model based on a deep convolutional neural network, for example, the image classification model may be, but is not limited to, vgnet, ResNet, and the like. In one possible embodiment, the cloud server 120 takes the positive sample images in the positive sample image set and the negative sample images in the negative sample image set as input of an image classification model, the image classification model outputs the classification results, i.e., the predicted quality levels, corresponding to the positive sample images and the negative sample images, obtains a cross entropy loss function by combining the quality level labels of the corresponding input sample images, and updates the model parameters of the image classification model through the cross entropy loss function. The cross entropy (cross entropy) is mainly used for measuring difference information between two probability distributions, the difference between the predicted quality level and the labeled quality level label of the image classification model can be measured by taking the cross entropy as a loss function, and the cross entropy loss function is used for performing back propagation in the image classification model so as to update model parameters in the image classification model.

Specifically, when the model parameters in the image classification model are updated through the cross entropy loss function, the model parameters in the image classification model can be updated through the cross entropy loss function when it is determined that the image classification model is not converged according to the cross entropy loss function. The convergence of the image classification model may mean that a difference between a predicted quality level output by the image classification model and a quality level label of the corresponding input sample image is smaller than a predetermined threshold, or a change rate of the difference between the predicted quality level and the quality level label of the input sample image approaches a certain lower value (e.g., approaches 0). In one possible implementation, after a certain round of iterative training is completed, a cross entropy loss function is calculated through the predicted quality level output by the round of iterative training and the quality level label of the input sample image, and if the calculated loss function is small or the difference between the calculated loss function and the cross entropy loss function of the previous round of iterative training approaches to 0, the image classification model is considered to be converged.

Based on the foregoing description, the cloud server 120 in the embodiment of the present invention is further configured to train an image reconstruction model based on the fully sampled medical image and the undersampled medical image corresponding to the fully sampled medical image. The specific process of the cloud server 120 training the image reconstruction model is described below.

As shown in fig. 4, the cloud server 120 acquires a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image. Wherein the fully sampled medical images comprise fully sampled medical images corresponding to a plurality of coil channels of the medical imaging device. Due to the fact that the sensitivities of the multiple coils are different, when the fully sampled medical images in the training data comprise fully sampled medical images of multiple coil channels, compared with the fully sampled medical images of only a single coil channel, the reconstruction effect of the image reconstruction model obtained through training can be improved by means of the sensitivities of the multiple coils. Taking a fully sampled medical image as a fully sampled MR image as an example, in the MR scanning process, filling K space data one by one along the phase coding direction to obtain K space fully sampled data, fully collecting the K space fully sampled data and performing Fourier inverse transformation to obtain a fully sampled MR image; and acquiring the K space full-sampling data at a certain sampling rate, and performing Fourier inverse transformation to obtain an undersampled MR image corresponding to the full-sampling MR image.

The cloud server 120 takes the fully sampled medical image and the corresponding undersampled medical image as a training data pair; and based on the training data pair, performing image reconstruction training by using a preset second neural network model, and adjusting the model parameters of the preset second neural network model in the training process until a predicted image output by the preset second neural network model is matched with the fully sampled medical image in the input training data pair.

The cloud server 120 takes the second neural network model corresponding to the current model parameters (i.e., the model parameters when the predicted image output by the preset second neural network model is matched with the fully sampled medical image in the input training data pair) as the image reconstruction model in the embodiment of the present invention.

In practical applications, the cloud server 120 needs to first construct the preset second neural network model, which may be an image segmentation model based on deep learning, for example, the image segmentation model may be, but is not limited to, FCN, UNet + +, and the like. The FCN (full Convolutional network model) can classify an image at a pixel level, receive an input image of any size, perform up-sampling on the last Convolutional layer by deconvolution, restore the last Convolutional layer to the same size as the input image, generate a prediction for each pixel, and simultaneously retain spatial information in the original input image, and finally perform pixel-by-pixel classification in the up-sampled feature map, that is, calculate the loss of softmax classification pixel by pixel, which is equivalent to that each pixel corresponds to a training sample. UNet and UNet + + are similar image segmentation networks obtained by improvement on the basis of FCN, can realize multi-scale and are more suitable for processing of super-large medical images.

In order to improve the representation power of the image reconstruction model and obtain a reconstructed image with higher quality, in a possible implementation manner, the preset second neural network model may include a cascaded full convolution neural network model FCN and a recurrent neural network model RNN, and a set of training data pairs is trained for multiple recurrent iterations in a parameter sharing manner in the RNN, so that a recursive iteration stage of an iterative reconstruction algorithm may be effectively simulated, and each recurrent iteration adds a supplement to details compared with the previous iteration.

During specific training, the cloud server 120 inputs a training data pair consisting of a fully sampled medical image and a corresponding undersampled medical image into a preset second neural network model, the preset second neural network model learns a mapping relation from the undersampled medical image to the fully sampled medical image, a predicted image obtained based on the undersampled medical image is compared with the corresponding fully sampled medical image to obtain a calculation loss, and when the calculation loss is less than a preset target loss, the output predicted image can be considered to be matched with the fully sampled medical image in the input training data pair. The preset target loss can be set according to the actual condition; when the calculation loss exceeds the preset target loss, parameters (namely model parameters) in the mapping relation can be updated by adopting an error back propagation algorithm, and iterative training is continued on the basis of the updated model parameters.

In practical applications, in order to improve the reconstruction quality of the medical image, the cloud server 120 may further automatically update the trained quality assessment model and the trained image reconstruction model.

In a possible embodiment, the cloud server 120 and the first terminal 110 are in communication based on an internal network connection of a hospital, and the cloud server 120 may obtain corresponding sample image data from a medical image database connected to the internal network to perform update training on the quality assessment model and the image reconstruction model, so as to continuously optimize the prediction effect of the model.

In another possible embodiment, the cloud server 120 may be connected and communicated with a network outside the internal network of the hospital (i.e., an external network), and the cloud server 120 may acquire corresponding sample image data from a medical image database of the external network according to a set update time to perform update training on the quality assessment model and the image reconstruction model. The set update time can be determined according to actual needs, and may be, for example, 9:00 pm every week, and the like.

According to the embodiment of the invention, the quality evaluation model and the image reconstruction model are deployed at the cloud server, the first quality grade of the medical image is determined based on the quality evaluation model after the medical image sent by the first terminal is obtained, and when the first quality grade is matched with the preset quality grade, the image reconstruction model is called to reconstruct the medical image to obtain the reconstructed medical image with higher quality and return the reconstructed medical image to the first terminal, so that the reconstruction efficiency of the medical image and the quality of the reconstructed image are improved, the expensive hardware equipment of a reconstruction workstation is not required to be relied on, and the image reconstruction cost is reduced.

In another possible embodiment, in order to reduce the time for direct film reading by the doctor, the cloud server 120 may also train and store a diagnosis analysis model in advance based on the second sample medical image and corresponding diagnosis label information, where the diagnosis label information may include a disease type and/or a lesion type, so that the cloud server 120 may include the following steps as shown in fig. 5 when returning the reconstructed medical image to the first terminal:

the cloud server 120 performs diagnostic analysis on the reconstructed medical image based on the diagnostic analysis model to obtain a diagnostic analysis result corresponding to the medical image.

The cloud server 120 determines a target historical treatment plan in the historical treatment plan library that matches the diagnostic analysis results. The historical treatment scheme library stores the corresponding relation between the historical treatment scheme and the historical diagnosis and analysis result. In one possible embodiment, in order to improve the processing efficiency of the cloud server on the medical image and return the processing result as soon as possible, only one recent historical treatment plan may be stored in the historical treatment plan library for the same diagnosis analysis result. In another possible embodiment, for recommending more comprehensive treatment plans to the doctor, a preset number of historical treatment plans within a preset historical time period may be stored for the same diagnosis analysis result in the historical treatment plan library, for example, each diagnosis analysis result corresponds to 5 historical treatment plans within the last 1 month.

The cloud server 120 establishes a first mapping relationship between the reconstructed medical image, the diagnostic analysis results, and the target historical treatment plan; and returning the first mapping relation to the first terminal.

The first terminal 110 may display the reconstructed medical image, the corresponding diagnosis and analysis result, and the target historical treatment scheme to the doctor, and the target historical treatment scheme may be used as a reference for the doctor to determine the treatment scheme according to the diagnosis and analysis result, so that the time for the doctor to directly read the film is greatly reduced, and the diagnosis and treatment scheme determination efficiency of the doctor is improved. In addition, the quality of the reconstructed medical image is improved, so that the diagnostic analysis model can be more accurate in determining the diagnostic analysis result of the reconstructed medical image.

In practical applications, different diagnostic and analytical tasks are often involved, and the different diagnostic and analytical tasks are generally associated with body parts, for example, in the case of ventricles, the corresponding diagnostic and analytical tasks are to determine the type of disease (such as heart failure type, etc.); in the case of lung, the corresponding diagnostic and analytical task is to determine the lesion type of lung (e.g. pulmonary tuberculosis, emphysema, etc.). Because the diagnostic and analytical tasks are different, the network structures of the related diagnostic and analytical models are different, for example, when the diagnostic and analytical task is to determine the disease type, the diagnostic and analytical model is a segmented network structure, and when the diagnostic and analytical task is to determine the lesion type, the diagnostic and analytical model is a classified network structure.

In order to improve the applicability of the medical image processing system in the embodiment of the present invention, the cloud server 120 may train a plurality of diagnostic analysis models for different diagnostic analysis tasks, and establish a correspondence between the diagnostic analysis tasks and the diagnostic analysis models, so that the cloud server 120 may include, when performing diagnostic analysis on the reconstructed medical image based on the diagnostic analysis models:

the cloud server 120 determines the current diagnosis and analysis task according to the body part corresponding to the reconstructed medical image. For example, a first diagnostic analysis model is trained for determining a disease type for a diagnostic analysis task and a second diagnostic analysis model is trained for determining a lesion type for the diagnostic analysis task. If the body part corresponding to the reconstructed medical image is a ventricle, the cloud server 120 may determine that the current diagnostic and analysis task is to determine a disease type of the ventricle; if the body part corresponding to the reconstructed medical image is a lung, the cloud server 120 may determine that the current diagnostic analysis task is to determine a lesion type of the lung.

The cloud server 120 determines a target diagnostic analysis model of the diagnostic analysis models that matches the current diagnostic analysis task. In the above example, the cloud server 120 may determine the first diagnostic analysis model as the target diagnostic analysis model for the current diagnostic analysis task to determine the disease type of the ventricle; for the current diagnostic analysis task to determine the lesion type of the lung, the second diagnostic analysis model may be determined as the target diagnostic analysis model.

The cloud server 120 performs diagnostic analysis on the reconstructed medical image based on the target diagnostic analysis model.

Taking the example that the body part corresponding to the reconstructed MR image is a ventricle, the cloud server 120 inputs the reconstructed MR image into a first diagnostic analysis model, which divides the reconstructed MR image into MR division data of left and right ventricles and myocardium, obtains parameters such as volume and ejection fraction, and outputs a heart failure type according to the ejection fraction analysis, where the heart failure type can be used as a current diagnostic analysis result. The cloud server 120 searches a target historical treatment plan matched with the diagnosis result from the historical treatment plan library according to the heart failure type, establishes a first mapping relation among the reconstructed MR image, the current diagnosis analysis result and the target historical treatment plan, and returns the first mapping relation to the first terminal 110, so that the first terminal 110 can present the current diagnosis result and the recommended treatment plan (i.e., the target historical treatment plan) to a doctor.

Based thereon, the cloud server 120 is further configured to train a diagnostic analysis model based on the second sample medical image and the corresponding diagnostic label information. The diagnosis label information corresponding to the second sample medical image can be labeled by a doctor according to the corresponding disease type and/or focus type of the body part corresponding to the second sample medical image.

In practical applications, the cloud server 120 may construct a preset third neural network model according to a diagnostic analysis task corresponding to the diagnostic analysis model. For example, for the diagnosis and analysis task to determine the disease type, the preset third neural network model may be a segmented network structure based on deep learning, such as but not limited to a full convolution neural network FCN; for the diagnostic analysis task to determine the lesion type, the preset third neural network model may be a classification network structure based on a deep convolutional neural network, such as but not limited to VGGNet, ResNet. For training of the segmentation network model, the training data thereof may include the third sample medical image, the corresponding diagnostic label information, and the segmentation gold criteria of the third sample medical image corresponding to the muscle body part, for example, the ventricular segmentation gold criteria includes left ventricle segmentation, right ventricle segmentation, and myocardium segmentation data. For training of the classification network model, the training data thereof includes third sample medical images and corresponding diagnostic label information.

In another possible embodiment, as shown in fig. 6, the cloud server 120 is further configured to perform a diagnostic analysis on the medical image based on a pre-trained diagnostic analysis model when the first quality level does not match the preset quality level, so as to obtain a diagnostic analysis result corresponding to the medical image. Specifically, when the first quality level of the medical image output by the quality evaluation model is not matched with the preset quality level, the quality of the medical image is better, image reconstruction may not be performed, and at this time, diagnosis and analysis may be directly performed based on the medical image.

The cloud server 120 determines a target historical treatment scheme matched with the diagnosis and analysis result in a historical treatment scheme library, wherein the historical treatment scheme library stores the corresponding relation between the historical treatment scheme and the historical diagnosis and analysis result;

the cloud server 120 establishes a second mapping relationship between the medical image, the diagnostic analysis result and the target historical treatment plan; and returning the second mapping relation to the first terminal.

Therefore, in the embodiment of the invention, the image reconstruction is performed only when the evaluated quality grade is matched with the preset quality grade, namely the medical image quality is not high, the intelligent diagnosis analysis is performed based on the reconstructed medical image, the image reconstruction is not performed when the evaluated quality grade is not matched with the preset quality grade, namely the medical image quality is good, the intelligent diagnosis analysis is directly performed based on the medical image, and the accuracy of the diagnosis analysis result is improved while the diagnosis analysis efficiency of the cloud server is improved.

In one possible embodiment, in order to improve the accuracy of the diagnosis result of the diagnosis analysis model, as shown in fig. 7, the first terminal 110 is further configured to generate an adjusted mapping relationship in response to an adjustment instruction of the diagnosis analysis result in the mapping relationship (including the aforementioned first mapping relationship and second mapping relationship) by the user; and sending the adjusted mapping relationship to the cloud server 120.

Correspondingly, the cloud server 120 is further configured to train and update the diagnostic analysis model by using the adjusted mapping relationship according to a preset time interval. The preset time interval may be set according to actual needs, for example, the preset time interval may be set to 3 days or 1 week, and the like, which is not specifically limited in the present invention.

In the embodiment of the present invention, the first terminal 110 returns the mapping relationship modified by the doctor to the cloud server 120, and the cloud server 120 trains and updates the corresponding diagnostic analysis model according to the diagnostic analysis result after being corrected in the mapping relationship, so as to improve the diagnostic analysis result of the subsequent diagnostic analysis model.

In one possible embodiment, as shown in fig. 8, the first terminal 110 is further configured to obtain a current treatment plan, and send the current treatment plan to the cloud server 120. Wherein the current treatment plan may be a new treatment plan generated by the first terminal 110 according to the doctor's modification of the recommended historical treatment plan.

Correspondingly, the cloud server 120 is further configured to receive the current treatment plan; establishing a corresponding relation between the current treatment scheme and the diagnosis and analysis result; and updating the corresponding relation between the current treatment scheme and the diagnosis and analysis result to the historical treatment scheme library, so that the effectiveness of the recommended historical treatment scheme can be improved.

An embodiment of the present invention further provides a medical image processing method, which may be applied to a cloud server of the medical image processing system according to the embodiment of the present invention, and as shown in fig. 9, the method may include:

s901, receiving a medical image sent by a first terminal, wherein the medical image is acquired by the first terminal from a medical imaging device.

S903, determining a first quality grade of the medical image based on the quality evaluation model.

S905, when the first quality grade is matched with a preset quality grade, reconstructing the medical image based on an image reconstruction model to obtain a reconstructed medical image.

And S907, returning the reconstructed medical image to the first terminal.

In one possible embodiment, as shown in fig. 10, the method may further include:

s1010, training the quality evaluation model based on the first sample medical image and the quality grade label corresponding to the first sample medical image.

Specifically, the step S1010 may include the following steps:

(1) acquiring a first sample medical image set; the first sample medical image in the first sample medical image set is a medical image obtained by sampling based on the same undersampling rate; each first sample medical image is labeled with a quality grade label;

(2) determining the first sample medical image with the quality grade label matched with the preset quality grade as a positive sample medical image to obtain a positive sample image set;

(3) determining the first sample medical image with the quality grade label not matched with the preset quality grade as a negative sample medical image to obtain a negative sample image set;

(4) training a preset first neural network model based on the positive sample image set and the negative sample image set, and adjusting model parameters of the preset first neural network model in the training process until the prediction quality grade output by the preset first neural network model is matched with the input sample image;

(5) and taking the first neural network model corresponding to the current model parameter as the quality evaluation model.

With continued reference to fig. 10, in one possible embodiment, the method may further comprise: s1020, training the image reconstruction model based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image.

Specifically, the step S1020 may include the following steps:

(1) acquiring a fully sampled medical image and an under-sampled medical image corresponding to the fully sampled medical image; the fully sampled medical images comprise fully sampled medical images corresponding to a plurality of coil channels of the medical imaging device;

(2) taking the fully sampled medical image and the corresponding undersampled medical image as a training data pair;

(3) based on the training data pair, performing image reconstruction training by using a preset second neural network model, and adjusting the model parameters of the preset second neural network model in the training process until a predicted image output by the preset second neural network model is matched with a fully sampled medical image in an input training data pair;

(4) and taking the second neural network model corresponding to the current model parameters as the image reconstruction model.

The medical image processing method provided by the embodiment of the invention is completed by combining the cloud server with the machine learning model obtained by pre-training on the cloud server, and image reconstruction is carried out when the evaluated quality level is matched with the preset quality level, namely the medical image quality is not high, so that the reconstruction cost of the medical image is reduced, and the reconstruction efficiency and the quality of the reconstructed image are greatly improved.

Since the medical image processing method provided by the embodiment of the present invention corresponds to the functions of the cloud server in the medical image processing systems provided by the foregoing several embodiments, the descriptions of the relevant parts regarding the functions of the cloud server in the foregoing medical image processing system also apply to the medical image processing method provided by the present embodiment, and the detailed description thereof is omitted in the present embodiment.

An embodiment of the present invention provides a server, where the server includes a processor and a memory, where the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement an image processing function of a cloud server provided in the above embodiment.

The memory may be used to store software programs and modules, and the processor may execute various functional applications and medical image processing by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.

Fig. 11 is a block diagram of a hardware structure of a server according to an embodiment of the present invention, as shown in fig. 11, the server 1100 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1110 (the processors 1110 may include but are not limited to Processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory 1130 for storing data, and one or more storage media 1120 (e.g., one or more mass storage devices) for storing applications 1123 or data 1122. The memory 1130 and the storage medium 1120 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 1120 may include one or more modules, each of which may include a series of instruction operations for a server. Still further, the central processor 1110 may be configured to communicate with the storage medium 1120, and execute a series of instruction operations in the storage medium 1120 on the server 1100. The server 1100 may also include one or more power supplies 1160, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1140, and/or one or more operating systems 1121, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The input output interface 1140 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 1100. In one example, i/o Interface 1140 includes a Network adapter (NIC) that may be coupled to other Network devices via a base station to communicate with the internet. In one example, the input/output interface 1140 can be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

It will be understood by those skilled in the art that the structure shown in fig. 11 is only an illustration and is not intended to limit the structure of the electronic device. For example, server 1100 may also include more or fewer components than shown in FIG. 11, or have a different configuration than shown in FIG. 11.

Embodiments of the present invention also provide a computer-readable storage medium, which may be disposed in a server to store at least one instruction or at least one program for implementing the medical image processing method, where the at least one instruction or the at least one program is loaded and executed by the processor to implement the medical image processing method provided by the foregoing method embodiments.

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A medical image processing system, characterized in that the system comprises:

2. The medical image processing system according to claim 1, wherein the cloud server returning the reconstructed medical image to the first terminal includes:

the cloud server returns the first mapping relation to the first terminal;

3. The medical image processing system of claim 2, wherein the diagnostic analysis model includes a plurality of diagnostic analysis models corresponding to different diagnostic analysis tasks;

4. The medical image processing system of claim 2, wherein the cloud server is further configured to:

and returning the second mapping relation to the first terminal.

5. The medical image processing system of claim 1, wherein the cloud server is further configured to: training the quality assessment model based on a first sample medical image and a quality grade label corresponding to the first sample medical image; the cloud server training the quality assessment model comprises:

6. The medical image processing system of claim 1, wherein the cloud server is further configured to: training the image reconstruction model based on a fully sampled medical image and an undersampled medical image corresponding to the fully sampled medical image; the training of the image reconstruction model by the cloud server comprises:

7. The medical image processing system of claim 4, wherein the first terminal is further configured to: responding to an adjustment instruction of a user for a diagnosis and analysis result in the mapping relation, and generating an adjusted mapping relation; sending the adjusted mapping relation to the cloud server;

8. The medical image processing system of claim 4, wherein the first terminal is further configured to: acquiring a current treatment scheme, and sending the current treatment scheme to the cloud server;

9. A medical image processing method is applied to a cloud server, and comprises the following steps:

returning the reconstructed medical image to the first terminal;

10. A computer readable storage medium having stored therein at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the medical image processing method according to claim 9.