CN115458161A

CN115458161A - Breast cancer progression analysis method, device, apparatus, and medium

Info

Publication number: CN115458161A
Application number: CN202211281659.XA
Authority: CN
Inventors: 许文仪; 刘长东; 邵涛; 周子捷
Original assignee: Lianren Healthcare Big Data Technology Co Ltd
Current assignee: Lianren Healthcare Big Data Technology Co Ltd
Priority date: 2022-10-19
Filing date: 2022-10-19
Publication date: 2022-12-09

Abstract

The embodiment of the invention discloses a breast cancer progression analysis method, a breast cancer progression analysis device, breast cancer progression analysis equipment and a breast cancer progression analysis medium, wherein the breast cancer progression analysis method comprises the following steps: acquiring a clinical examination report of a target object, and a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations; respectively extracting feature vectors in a breast cancer medical image, an image analysis report and a clinical examination report; and splicing the characteristic vectors, and inputting the spliced target characteristic vectors into a breast cancer progress analysis model trained in advance to obtain a target analysis result. The embodiment of the invention solves the problem of breast cancer focus positioning difference, can realize the intellectualization and automation of breast cancer progress analysis, and improves the accuracy and efficiency of breast cancer progress analysis.

Description

Breast cancer progression analysis method, device, apparatus, and medium

Technical Field

The embodiment of the invention relates to the technical field of medical image processing, in particular to a breast cancer progress analysis method, a breast cancer progress analysis device, breast cancer progress analysis equipment and a breast cancer progress analysis medium.

Background

Most of the current treatment methods for breast cancer adopt new adjuvant therapy and need to analyze the breast cancer progression. The traditional method is to analyze breast cancer progression through surgery and pathological examination results to determine whether the treated breast cancer is alleviated.

The existing breast cancer progression analysis is that a doctor analyzes a medical image of a patient before radiotherapy and a medical image of the patient after radiotherapy. Since the medical image before radiotherapy and the medical image after radiotherapy are not imaged at the same time, the shooting technicians may be different, and doctors with different experiences may have a difference in positioning the breast cancer focus, which consumes a lot of time and results are not accurate enough.

Disclosure of Invention

The embodiment of the invention provides a breast cancer progress analysis method, a breast cancer progress analysis device, a breast cancer progress analysis equipment and a breast cancer progress analysis medium, solves the problem of breast cancer focus positioning difference, can realize intellectualization and automation of breast cancer progress analysis, and improves the accuracy and efficiency of breast cancer progress analysis.

In a first aspect, an embodiment of the present invention provides a breast cancer progression analysis method, including:

acquiring a clinical examination report of a target object, a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations;

respectively extracting feature vectors in a breast cancer medical image, an image analysis report and a clinical examination report;

and splicing the characteristic vectors, and inputting the spliced target characteristic vectors into a breast cancer progress analysis model trained in advance to obtain a target analysis result.

In a second aspect, an embodiment of the present invention further provides a breast cancer progression analysis apparatus, including:

the analysis data acquisition module is used for acquiring a clinical examination report of a target object, and a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations;

the analysis feature extraction module is used for respectively extracting feature vectors in the breast cancer medical image, the image analysis report and the clinical examination report;

and the analysis result determining module is used for splicing the characteristic vectors and inputting the spliced target characteristic vectors into a pre-trained breast cancer progress analysis model to obtain a target analysis result.

In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:

one or more processors;

a memory for storing one or more programs;

when the one or more programs are executed by the one or more processors, the one or more processors implement the breast cancer progression analysis method of any of the embodiments.

In a fourth aspect, embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the breast cancer progression analysis method provided in any embodiment of the present invention.

According to the technical scheme of the embodiment of the invention, a clinical examination report of a target object, a medical image of breast cancer to be analyzed in two consecutive imaging examinations and an image analysis report are obtained; respectively extracting feature vectors in a breast cancer medical image, an image analysis report and a clinical examination report; and splicing the characteristic vectors, and inputting the spliced target characteristic vectors into a breast cancer progress analysis model trained in advance to obtain a target analysis result. The technical scheme of the embodiment of the invention solves the problem of breast cancer focus positioning difference, can realize the intellectualization and automation of breast cancer progress analysis, and improves the accuracy and efficiency of breast cancer progress analysis.

Drawings

FIG. 1 is a flow chart of a method for analyzing breast cancer progression according to an embodiment of the present invention;

FIG. 2 is a flow chart of a breast cancer progression analysis method provided by an embodiment of the present invention;

fig. 3 is a block diagram of a breast cancer progression analysis apparatus according to an embodiment of the present invention;

fig. 4 is a block diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be further noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings, not all of them.

It should be noted that the terms "first," "second," and "third," etc. in the description and claims of the invention and the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a flowchart of a breast cancer progression analysis method according to an embodiment of the present invention, and this embodiment is applicable in a scenario of breast cancer progression analysis based on medical images, and in particular, this embodiment is more applicable in a scenario of breast cancer progression analysis based on MRI (Magnetic Resonance Imaging) images. The method may be performed by a breast cancer progression analysis apparatus, which may be implemented in software and/or hardware, integrated in a computer device having an application development function.

As shown in fig. 1, the breast cancer progression analysis method of the present embodiment includes the following steps:

s110, acquiring a clinical examination report of the target object, and a medical image and an image analysis report of the breast cancer to be analyzed in two consecutive imaging examinations.

Wherein, the target object refers to a breast cancer patient to be subjected to breast cancer progression analysis.

Two consecutive imaging examinations refer to two imaging examinations of the target subject at different times, and may be, for example, two imaging examinations of the target subject before and after treatment.

The clinical examination report refers to a clinical examination report of the target subject during a previous imaging examination period, and may be, for example, a clinical examination report of the target subject before treatment.

Optionally, the clinical test report may include the physical examination result, the breast cancer tumor marker examination result, the immunohistochemical examination result, the tissue biopsy result, the information of the drug used for treatment, and the like.

The physical examination is used for primary screening of breast cancer, and whether the primary patient has breast abnormality signs, such as breast lump, breast skin change, nipple discharge and the like, and lymph node abnormality is judged.

The common detection indexes in the breast cancer tumor marker detection comprise detection indexes which can provide a supplement basis for the accurate diagnosis of breast cancer, such as serum cancer antigen 15-3 (CA 15-3), serum carcinoembryonic antigen (CEA), serum cancer antigen 125 (CA 125) and the like, and the postoperative recurrence and metastasis conditions can be monitored.

Wherein, the common detection indexes in the immunohistochemical detection are Ki-67, HER-2, ER, PR and the like, and the molecular type of the breast cancer can be confirmed, thereby providing basis for later treatment. The positive explanation of ER and PR is hormone-dependent breast cancer, while the positive explanation of Ki-67 and HER-2 represents that the tumor is highly invasive and easy to relapse and transfer.

The tissue biopsy is used for a suspected breast cancer patient, a tumor can be punctured under the guidance of ultrasound, and a small amount of tumor tissue is taken out for pathological examination.

The information on the medication for treatment is information on the medication used for the treatment of breast cancer on the target object, and specifically includes information on the name of the medication for treatment, the number of days taken, the dose, the specification of the medication, and the like.

The medical images of breast cancer to be analyzed may be images obtained by breast molybdenum target radiographic examination, ultrasound images obtained by breast ultrasound, and MRI images obtained by scanning with a magnetic resonance apparatus.

Image analysis reports include medical history, comparison to past exams, scanning techniques, fibroglandular composition of the breast and parenchymal background enhancement and any associated image findings. The lesion morphological characteristics and the dynamic enhancement curve characteristics are described, morphological analysis is taken as the primary judgment basis, and the type of the dynamic enhancement curve is required to be judged for the patients with morphological characteristic judgment difficulty. The morphological characteristics need to be comprehensively analyzed for the signal characteristics on the T1WI and the T2WI before enhancement and the performance after enhancement. However, the lesion morphology description is based on the enhanced morphology, and the lesions are classified into punctate reinforcement, lump reinforcement and non-lump reinforcement.

It is understood that the doctor obtains the medical image and the image analysis report of the breast cancer of the imaging examination after the target object is treated according to the clinical examination report of the target object, the medical image and the image analysis report of the breast cancer of the previous imaging examination, and the treatment of the target object, and performs further breast cancer progression analysis according to the medical image and the image analysis report of the breast cancer to be analyzed in the two imaging examinations of the target object after a certain period of treatment, which may be, for example, one month, half a year, and the like.

And S120, respectively extracting feature vectors in the breast cancer medical image, the image analysis report and the clinical examination report.

Firstly, preset image omics data of breast cancer medical images in two consecutive imaging examinations are extracted, and the preset image omics data are encoded to obtain a first feature vector.

The preset imaging omics data are imaging omics feature data obtained by performing feature extraction on breast cancer medical images in two successive imaging examinations.

Specifically, the first feature vector is obtained by extracting features of the omics data of the medical images twice in sequence and inputting the features into a deep learning model for processing a time sequence problem to be encoded.

The characteristics of the image omics data can comprise shape characteristics, first-order characteristics, gray level matrix characteristics, gray level region size matrix characteristics, gray level travel matrix characteristics, neighborhood gray level difference matrix characteristics, gray level related characteristics, wavelet characteristics and the like.

Specifically, the shape feature includes features such as Mesh Volume, volume, surface Area to Volume ratio, sphericity, maximum 3D diameter, maximum2D diameter (Slice), maximum2D diameter (Column), maximum2D diameter (Row), maximum Axis Length, minimum Axis Length, length (shortest Axis Length), elongation, and Flatness.

First-order features include Energy, total Energy, entropy, minimum, 10th percentile, 90th percentile, maximum, mean, median, intersourcerange, range, mean Absolute Deviation, robust Mean Absolute Deviation, romean square Absolute Deviation, sroot Mean square, skawness, kurtosis, variance, and Uniformity.

The gray co-occurrence matrix characteristics include Autocorrelation, joint Average, cluster progress, cluster Shade, cluster trend, contrast, correlation, difference Average, difference Entropy, difference Variance, joint Energy, joint Entropy, information Measure of Correlation 1 (IMC 1, correlation information Measure 1), information Measure of Correlation2 (IMC 2, correlation information Measure 2), inverse Difference Motion (IDM), maximum Correlation Coefficient (MCC), inverse Difference Motion Normalized (IDMN), inverse Difference (ID, inverse), inverse Difference Normalized (IDN, normalized Inverse), inverse Variance value (Inverse), maximum Probability, sum Average, sum Entropy, and Sum square.

The Gray Area Size matrix characteristics include Small Area Emphasis (SAE, small Area Emphasis), large Area Emphasis (LAE, large Area Emphasis), gray Level Non-Uniformity (GLN, gray Non-Uniformity), gray Level Non-Uniformity (GLNN, normalized Gray Non-Uniformity), size-Zone Non-Uniformity (SZN, region Size Non-Uniformity), size-Zone Non-Uniformity (SZNN, normalized region Size Non-Uniformity), and Zone Percentage (ZP, region Percentage), gray Level Variance (GLV, gray Variance), zone Variance (ZV, region Variance), zone entry (ZE, region Entropy), low Gray Level Zone phases (LGLZE, low Gray Level Zone phases (HGLZE, high Gray Level Zone Emphasis), small Area Low Gray Level phases (SALGLE, small Area Low Gray Level Emphasis), small Area High Gray Level phases (SAHGLE, small Area High Gray Level Emphasis), large Area Low Gray Level phases (LALGLE, large Area Low Gray Level Emphasis), and Large Area High Gray Level phases (LAHGLE, large Area High Gray Level) are among other features of Emphasis.

The Gray scale Run matrix features include Short Run empty (SRE, short Run Emphasis), long Run empty (LRE, long Run Emphasis), gray Level Non-Uniformity (GLN, gray Non-Uniformity), gray Level Non-Uniformity (GLNN, normalized Gray Non-Uniformity), run Level Non-Uniformity (RLN, normalized stroke Non-Uniformity), run duration (rlrp, stroke Percentage), gray Level Variance (GLV, gray Variance), run Variance (RV, stroke Variance), run entry (lrre, stroke Non-Uniformity), low Gray Level lgn empty (lge, low Gray Run Emphasis), high Gray Level empty (hgh, high Gray Level (hgh, high Gray Run), high Gray Level (hgh, high Gray Run, low stroke), and Low stroke (hgh, high stroke).

The neighborhood gray level difference matrix characteristics include characteristics such as Coarseness, contrast, busyness, complexity, and Strength.

The Gray-Level correlation matrix features include Small-dependent Emphasis (SDE), large-dependent Emphasis (LDE), gray Level Non-Uniformity (GLN), dependent Non-Uniformity (DN, dependent Non-Uniformity), dependent Non-Uniformity (DNN, normalized dependent Non-Uniformity), gray Level Variance (GLV, gray Variance), dependent Variance (DV, dependent Variance), and dependent intensity (DE, dependent Entropy), low Gray Level Emphasis (LGLE, low Gray Level Emphasis), high Gray Level Emphasis (HGLE, high Gray Level Emphasis), lgdependent Gray Level Emphasis (LGLE), small-dependent Gray Level Emphasis (sdle), high Gray Level Emphasis (HGLE, high Gray Level Emphasis), and Low Gray Level Emphasis (HGLE, low Gray Level Emphasis).

The wavelet features are the intensity and texture features of medical images of breast cancer obtained by wavelet decomposition calculation and concentrated in different frequency ranges within the tumor volume.

And then, inputting image analysis reports in two consecutive imaging examinations into a preset coding model to obtain a second feature vector.

The second feature vector is obtained by processing the text content of the image analysis report, extracting information in the image analysis report by using a clause model, inputting the information into a preset coding model and coding the information.

Further, character recognition and preprocessing are carried out on the contents of the image analysis reports twice in sequence, and the preprocessed character contents are input into a preset coding model to obtain a second feature vector, wherein the preset coding model is a pre-training language characterization model.

Among them, the pre-trained language Representation model (BERT) is a typical Bidirectional coding model, BERT still uses the transform algorithm, which is based on the Multi-Head attention mechanism, and BERT stacks multiple transform models and pre-trains the Bidirectional depth Representation by adjusting the Bidirectional transforms in all layers, and the pre-trained BERT model can be fine-tuned through an additional output layer, so the applicability is wider, and more repetitive model training work is not needed.

Basic steps of the BERT model: firstly, randomly covering or replacing any character or word in a sentence in the content of the character, then enabling a model to predict the covered or replaced part through the understanding of the context in the pre-training process, and then only predicting the content of the covered part in the process of doing so; then selecting continuous context sentences from the reported contents, and identifying the continuity of the sentences by using a Transformer model to realize a BERT model for performing bidirectional prediction through context; and loading a pre-trained BERT model, and continuing training on the text content of the image analysis report.

The Transformer model can realize parallel computation and is a structure of an encoding module and a decoding module. Wherein, the coding component is composed of a multilayer coder, the decoding component is also composed of a Decoder (Decoder) with the same layer number, and each coder is composed of two sublayers: a Self-Attention layer (Self-Attention layer) and an FFN (Position-wise Feed Forward Network). The input to the encoder will first flow into the Self-Attention layer. The encoder can use the information of other words in the input sentence when encoding the specific word, and then the output of the Self-Attention layer flows into a feedforward network; the decoder also has a Self-Attention layer and a FNN, but there is also an Attention layer between them for the decoder to focus on relevant parts of the input sentence.

And finally, inputting the clinical examination report into a preset coding model to obtain a third feature vector.

The third feature vector is obtained by processing the text content of the clinical examination report by using a clause model for the clinical examination report, extracting information in the clinical examination report, inputting the information into a preset coding model and coding the information.

Specifically, the content of the clinical examination report is subjected to character recognition and preprocessing, and the preprocessed character content is input into a preset coding model to obtain a third feature vector, wherein the preset coding model is a pre-training language characterization model.

It can be understood that the pre-training language representation model is adopted to extract the text data features of the contents of the clinical examination report of the patient, and the image features and the text features are further spliced, so that the fusion of the character features and the image features is realized, and the breast cancer progress analysis is facilitated.

And S130, splicing the feature vectors, and inputting the spliced target feature vectors into a breast cancer progress analysis model trained in advance to obtain a target analysis result.

It is to be understood that "first feature vector", "second feature vector", and "third feature vector", etc., are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.

Specifically, the first feature vector, the second feature vector and the third feature vector are spliced and fused, the splicing sequence is not required, the spliced vectors are target feature vectors, and the feature vectors can be summed up for comprehensive analysis by splicing the feature vectors, so that feature extraction is facilitated; and then inputting the target characteristic vector into a breast cancer progress analysis model trained in advance to obtain a target analysis result.

It can be understood that the breast cancer progression analysis model needs to be trained first, and then the trained breast cancer progression analysis model needs to be used for analysis, so as to obtain a target analysis result.

The target analysis result is used to describe a breast cancer progression analysis result, which may be, for example, a breast cancer progression analysis result of non-remission, partial remission, complete remission, or the like.

Wherein, the breast cancer progression analysis model comprises a full connection layer, a Dropout layer and a normalization and result output layer.

The full connected layer (FCN) is a layer in which each node is connected to all nodes in the previous layer to integrate the extracted features. The parameters of a fully connected layer are also typically the most due to its fully connected nature. In a CNN (Convolutional Neural network) structure, after passing through a plurality of Convolutional layers and pooling layers, 1 or more than 1 fully-connected layer is connected, each neuron in the fully-connected layer is fully connected with all neurons in the previous layer. The output value of the last fully connected layer is transmitted to an output, which can be a softmax layer, and is classified by softmax logistic regression.

The Dropout layer is used for reducing overfitting of the neural network, discarding a node of a hidden layer in the network, and once a certain node is selected as the discarded node, setting the output of the node to be 0 in the inference process of the neural network; for the process of feedback adjustment of the weight of each node in the network, the weight and bias of the node do not participate in updating. That is, in a certain iteration, some nodes in the network do not participate in the training.

The normalized exponential function, or Softmax function, is a gradient log normalization of a finite term discrete probability distribution. The Softmax function is widely applied to various probability-based multi-classification problem methods including multi-term logistic regression, multi-term linear discriminant analysis, a naive bayes classifier, an artificial neural network and the like, is mainly used for multi-classification problems, and can map data records in a database to one of given classes, so that the Softmax function can be applied to data prediction.

Specifically, feature vectors in the breast cancer medical image, the image analysis report and the clinical examination report are spliced, the spliced feature vectors are input into a breast cancer progress analysis model which is trained in advance, and classification results are output through a full connection layer, a Dropout layer and a normalization and result output layer, so that breast cancer progress analysis is completed.

According to the technical scheme of the embodiment of the invention, a clinical examination report of a target object, a medical image of breast cancer to be analyzed in two consecutive imaging examinations and an image analysis report are obtained; respectively extracting feature vectors in a breast cancer medical image, an image analysis report and a clinical examination report; the feature vectors are spliced, and the target feature vectors obtained through splicing are input into a breast cancer progress analysis model trained in advance to obtain a target analysis result, so that the problem of breast cancer focus positioning difference is solved, the intellectualization and automation of breast cancer progress analysis can be realized, and the accuracy and efficiency of breast cancer progress analysis are improved.

Fig. 2 is a flowchart of a breast cancer progression analysis method according to an embodiment of the present invention, which belongs to the same inventive concept as the breast cancer progression analysis method according to the foregoing embodiment, and further describes the extraction of feature vectors based on the foregoing embodiment. The method may be performed by a breast cancer progression analysis apparatus, which may be implemented in software and/or hardware, integrated in a computer device having an application development function.

As shown in fig. 2, the breast cancer progression analysis method includes the following steps:

s210, acquiring a clinical examination report of the target object, and acquiring a medical image and an image analysis report of the breast cancer to be analyzed in two consecutive imaging examinations.

Specifically, an MRI image to be analyzed, an image analysis report, and a clinical examination report in two consecutive imaging examinations of a target object are acquired.

S220, extracting the image omics data of the magnetic resonance dynamic enhanced scanning imaging and the image omics data of the apparent diffusion coefficient image of the magnetic resonance from the breast cancer medical images in the two times of imaging examination.

Wherein the mammary gland magnetic resonance dynamic enhancement scan is performed by injecting magnetic resonance paramagnetic contrast agent pre-T ₁ W ₁ The gradient echo sequence is continuously and rapidly acquired in multiple Time phases, the Dynamic process of Contrast agent entering and discharging tissues is recorded in real Time, and a Dynamic Enhancement Time Curve (DCE-TIC) and corresponding parameter values of a focus are obtained and are used for magnetic resonance examination for observing the microvascular biological characteristics of breast tumor tissues.

An Apparent Diffusion Coefficient (ADC) image of magnetic resonance is a distribution map of ADC values calculated by using DWI (diffusion weighted imaging). ADC is one of the main factors influencing DWI, and the signal of DWI and ADC have negative exponential relation, namely the value of ADC is increased and the signal of DWI is decreased. In vivo, the DWI signal is sensitive to physiological activities such as heart beat, respiration, perfusion, limb movement, etc. in addition to the ADC, and the measured ADC does not merely reflect the dispersion of water molecules. DWI has become more and more widely used in disease diagnosis in the breast, liver, etc., can provide tissue contrast different from conventional MRI images, and can provide potential information on the survival and development of brain tissue. It is very sensitive to the identification of acute cerebral infarction and other brain acute lesions, and can provide information on lesions such as tumor, infection, trauma and demyelination.

Further, the extracting of the imaging data of the mri dynamic enhanced scan imaging and the imaging data of the mri apparent diffusion coefficient image from the breast cancer medical image in the two subsequent imaging examinations respectively comprises:

firstly, carrying out image segmentation on a breast cancer medical image in a magnetic resonance dynamic enhanced scanning mode in the prior imaging examination to obtain a target focus region image in the magnetic resonance dynamic enhanced scanning mode, and inputting the target focus region image in the magnetic resonance dynamic enhanced scanning mode and a breast cancer medical image in the two successive magnetic resonance dynamic enhanced scanning modes into a first preset feature extraction algorithm to obtain image omics data of magnetic resonance dynamic enhanced scanning imaging in the breast cancer medical image in the two successive imaging examinations.

It can be understood that the image segmentation is required to be performed on the breast cancer medical image in the DCE mode in the previous imaging examination to obtain an image of a target lesion region in the DCE mode, then the image of the target lesion region in the DCE mode and the breast cancer medical image in the DCE mode twice are respectively superimposed to obtain a DCE mode superimposed image in the imaging examination twice, and the DCE mode superimposed image in the imaging examination twice is input into the first preset feature extraction algorithm to obtain the image omics data of the breast cancer medical image in the DCE mode including the target lesion region in the breast cancer medical images in the previous and subsequent examinations.

The first preset feature extraction algorithm is Pradiomics, is an open-source Python package, can specify the image type for extracting features, and supports two-dimensional and three-dimensional feature extraction. First, various filters may be selected to process the original image, and feature extraction may be performed using the original image and the image after passing through the various filters.

Processing the original image under the default condition of the image after wavelet filtering, laplacian of a Gaussian filter and the like; then, specifying features to be extracted; the feature extractor may also be configured, such as image normalization and image resampling.

Then, image segmentation is carried out on the image of the magnetic resonance diffusion weighted imaging modality in the prior imaging examination to obtain a target lesion region image under the magnetic resonance diffusion weighted imaging modality, and the target lesion region image under the magnetic resonance diffusion weighted imaging modality and the breast cancer medical image of the two successive magnetic resonance diffusion weighted imaging modalities are input into a second preset feature extraction algorithm to obtain image omics data of the magnetic resonance apparent diffusion coefficient image extracted from the breast cancer medical image of the two successive imaging examinations.

It can be understood that the image segmentation is required to be performed on the breast cancer medical image in the DWI modality in the previous imaging examination to obtain an image of a target lesion region in the DWI modality, then the image of the target lesion region in the DWI modality and the ADC image in the DWI modality in the two consecutive imaging examinations are input into the second preset feature extraction algorithm, the image of the target lesion region in the DWI modality is superimposed with the ADC image in the magnetic resonance in the two consecutive imaging examinations to obtain an ADC superimposed image of the target lesion region in the DWI modality in the two consecutive imaging examinations, and the omics data of the ADC superimposed image in the two consecutive imaging examinations are extracted to obtain the image omics data of the breast cancer medical image in the two consecutive imaging examinations including the ADC image of the target lesion region.

And S230, inputting the imaging data of the magnetic resonance dynamic enhanced scanning imaging and the imaging data of the apparent diffusion coefficient image of the magnetic resonance in the breast cancer medical image in the previous imaging examination into a preset long-short term memory network to obtain an initial feature vector.

The preset Long-Short Term Memory Network (LSTM) is a time circulation Neural Network, contains LSTM blocks or other Neural networks of a kind, is specially designed for solving the Long-Term dependence problem of a general circulation Neural Network (RNN), is suitable for processing and predicting important events with very Long interval and delay in a time sequence, and can be used as a complex nonlinear unit for constructing a larger deep Neural Network. All RNNs have a chain form of repeating neural network modules.

The preset LSTM network is the result of structural optimization based on the original LSTM, which includes: a bidirectional recurrent neural network and a deep recurrent neural network. The main structure of the bidirectional cyclic neural network is composed of two unidirectional cyclic neural networks. At each moment, the input is simultaneously provided for the two cyclic neural networks with opposite directions, and the output is jointly determined by the two unidirectional cyclic neural networks; the deep layer circulation neural network is used for enhancing the expression capability of the model, the structure of the circulation body is copied for multiple times at each moment, the parameters in the circulation body of each layer are consistent, and the parameters in different layers can be different.

It can be understood that the image data and the image features of the DCE modal image are extracted according to the DCE modal image of the mri dynamic enhanced scan breast cancer in the previous imaging examination, the omics data and the image features of the ADC image are obtained according to the DWI modal image of the mri diffusion weighted imaging in the previous imaging examination, and the image features are simultaneously input into the preset LSTM for time sequence feature processing to obtain the initial feature vector.

S240, the initial characteristic vector, the image omics data of the magnetic resonance dynamic enhanced scanning imaging and the image omics data of the magnetic resonance apparent diffusion coefficient image in the breast cancer medical image of the last imaging examination are simultaneously input into a preset long short-term memory network to obtain a first characteristic vector.

It can be understood that the DCE modal image of the mri dynamic enhanced scan breast cancer obtained in the subsequent imaging examination extracts the image data of the DCE modal image and the image features thereof, and the DWI modal image of the mri image in the subsequent imaging examination extracts the iconic data of the ADC image and the image features thereof.

Specifically, image data and image features of a DCE modal image are extracted according to a DCE modal image of magnetic resonance dynamic enhanced scanning breast cancer in the prior imaging examination, the image omics data and the image features of an ADC image are extracted according to a DWI modal image of magnetic resonance diffusion weighted imaging in the prior imaging examination, the image features are simultaneously input into a preset LSTM, and time sequence feature processing is carried out to obtain an initial feature vector; acquiring DCE modal images of magnetic resonance dynamic enhanced scanning breast cancer in the last imaging examination, extracting image data and image characteristics of the DCE modal images, and extracting the imaging omics data and image characteristics of ADC images according to DWI modal images of magnetic resonance diffusion weighted imaging in the last imaging examination; and then simultaneously inputting the image features into a preset LSTM, and performing time sequence feature processing to obtain a first feature vector.

And S250, inputting the image analysis reports in the two consecutive imaging examinations into a preset coding model to obtain a second feature vector.

And S260, inputting the clinical examination report into a preset coding model to obtain a third feature vector.

And S270, splicing the feature vectors, and inputting the spliced target feature vectors into a breast cancer progress analysis model trained in advance to obtain a target analysis result.

According to the technical scheme of the embodiment of the invention, the clinical examination report of the target object, the medical image of the breast cancer to be analyzed in the two successive imaging examinations and the image analysis report are obtained, the preset imaging omics data of the medical image of the breast cancer in the two successive imaging examinations are extracted, the preset imaging omics data are coded to obtain the first characteristic vector, and the characteristic vectors of the image analysis report and the clinical examination report are extracted to obtain the second characteristic vector and the third characteristic vector, so that the problem of positioning difference of the breast cancer focus is solved, the intellectualization and automation of the breast cancer progress analysis can be realized, and the accuracy and efficiency of the breast cancer progress analysis are improved.

Fig. 3 is a block diagram showing a configuration of an apparatus for analyzing breast cancer progression according to an embodiment of the present invention, which is applicable to a case of analyzing breast cancer progression based on MRI images. The device can be realized by software and/or hardware, and is integrated in a computer device with application development function.

As shown in fig. 3, the breast cancer progression analysis apparatus includes: an analysis data acquisition module 310, an analysis feature extraction module 320, and an analysis result determination module 330.

The analysis data acquisition module 310 is configured to acquire a clinical examination report of a target object, and a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations; an analysis feature extraction module 320, configured to extract feature vectors in the breast cancer medical image, the image analysis report, and the clinical examination report, respectively; and the analysis result determining module 330 is configured to splice the feature vectors, and input the spliced target feature vectors into a pre-trained breast cancer progress analysis model to obtain a target analysis result.

According to the technical scheme of the embodiment of the invention, a clinical examination report of a target object, a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations are obtained; respectively extracting feature vectors in a breast cancer medical image, an image analysis report and a clinical examination report; the feature vectors are spliced, and the target feature vectors obtained through splicing are input into a breast cancer progress analysis model trained in advance to obtain a target analysis result, so that the problem of breast cancer focus positioning difference is solved, the intellectualization and automation of breast cancer progress analysis can be realized, and the accuracy and efficiency of breast cancer progress analysis are improved.

Optionally, the analysis feature extraction module 320 is further configured to:

extracting preset imaging omics data of medical images of breast cancer in two successive imaging examinations, and coding the preset imaging omics data to obtain a first feature vector;

inputting image analysis reports in two successive image examination into a preset coding model to obtain a second feature vector;

and inputting the clinical examination report into a preset coding model to obtain a third feature vector.

respectively extracting the imaging data of the magnetic resonance dynamic enhanced scanning imaging and the imaging data of the apparent diffusion coefficient image of the magnetic resonance from the breast cancer medical image in the two subsequent imaging examinations;

inputting the imaging data of the magnetic resonance dynamic enhanced scanning imaging and the imaging data of the apparent diffusion coefficient image of the magnetic resonance in the breast cancer medical image in the previous imaging examination into a preset long-short term memory network to obtain an initial characteristic vector;

and simultaneously inputting the initial characteristic vector, the magnetic resonance dynamic enhanced scanning image group data in the breast cancer medical image of the last imaging examination and the image group data of the apparent diffusion coefficient image of the magnetic resonance into a preset long-short term memory network to obtain a first characteristic vector.

carrying out image segmentation on a breast cancer medical image in a magnetic resonance dynamic enhancement scanning mode in the prior imaging examination to obtain a target lesion region image in the magnetic resonance dynamic enhancement scanning mode, and inputting the target lesion region image in the magnetic resonance dynamic enhancement scanning mode and a breast cancer medical image in the sequential magnetic resonance dynamic enhancement scanning mode into a first preset feature extraction algorithm to obtain image data of magnetic resonance dynamic enhancement scanning imaging extracted from the breast cancer medical image in the sequential imaging examination;

and the target lesion region image under the magnetic resonance diffusion weighted imaging modality and the apparent diffusion coefficient images of the two successive magnetic resonance diffusion weighted imaging modalities are input into a second preset feature extraction algorithm to obtain the image omics data of the magnetic resonance apparent diffusion coefficient images extracted from the breast cancer medical images in the two successive imaging modalities.

Optionally, the analysis feature extraction module 320 is further configured to: and performing character recognition and preprocessing on the contents of the image analysis reports twice in sequence, and inputting the preprocessed character contents into a preset coding model to obtain a second feature vector, wherein the preset coding model is a pre-training language characterization model.

and performing character recognition and preprocessing on the content of the clinical examination report, and inputting the preprocessed character content into a preset coding model to obtain a third feature vector, wherein the preset coding model is a pre-training language characterization model.

The breast cancer progression analysis device provided by the embodiment of the invention can execute the breast cancer progression analysis method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Fig. 4 is a block diagram of a computer device according to an embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 4 is only an example and should not impose any limitations on the functionality or scope of use of embodiments of the present invention. The computer device 12 may be any terminal device having computing capabilities and may be configured in a breast cancer progression analysis device.

As shown in FIG. 4, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) through network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, implementing a breast cancer progression analysis method provided by an embodiment of the present invention, the method including:

acquiring a clinical examination report of a target object, and a medical image and an image analysis report of breast cancer to be analyzed in two consecutive imaging examinations;

The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements a breast cancer progression analysis method as provided by any of the embodiments of the present invention, the method comprising:

Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It will be understood by those skilled in the art that the modules or steps of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and optionally they may be implemented by program code executable by a computing device, such that it may be stored in a memory device and executed by a computing device, or it may be separately fabricated into various integrated circuit modules, or it may be fabricated by fabricating a plurality of modules or steps thereof into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It is to be noted that the foregoing description is only exemplary of the invention and that the principles of the technology may be employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments illustrated herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in some detail by the above embodiments, the invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the invention, and the scope of the invention is determined by the scope of the appended claims.

Claims

1. A method of analyzing breast cancer progression, comprising:

respectively extracting feature vectors in the breast cancer medical image, the image analysis report and the clinical examination report;

2. The method of claim 1, wherein the extracting feature vectors in the breast cancer medical image, image analysis report and clinical examination report respectively, further comprises:

extracting preset image omics data of the breast cancer medical image in two consecutive imaging examinations, and coding the preset image omics data to obtain a first feature vector;

inputting the image analysis reports in two consecutive imaging examinations into a preset coding model to obtain a second feature vector;

and inputting the clinical examination report into the preset coding model to obtain a third feature vector.

3. The method of claim 2, wherein said extracting the pre-defined imagemics data for the medical image of breast cancer in two consecutive imaging examinations and encoding the pre-defined imagemics data to obtain a first feature vector comprises:

extracting the imaging omics data of the magnetic resonance dynamic enhanced scanning imaging and the imaging omics data of the apparent diffusion coefficient image of the magnetic resonance from the breast cancer medical image in the two subsequent imaging examinations respectively;

and simultaneously inputting the initial characteristic vector, the imaging data of the magnetic resonance dynamic enhanced scanning imaging and the imaging data of the apparent diffusion coefficient image of the magnetic resonance in the breast cancer medical image of the last imaging examination into the preset long-short term memory network to obtain the first characteristic vector.

4. The method according to claim 3, wherein the extracting of the magnetic resonance dynamic enhanced scan imaging and the magnetic resonance apparent diffusion coefficient image from the breast cancer medical images in the two subsequent imaging examinations respectively comprises:

performing image segmentation on a breast cancer medical image in a magnetic resonance dynamic enhanced scanning mode in the prior imaging examination to obtain a target lesion region image in the magnetic resonance dynamic enhanced scanning mode, and inputting the target lesion region image in the magnetic resonance dynamic enhanced scanning mode and a breast cancer medical image in the sequential magnetic resonance dynamic enhanced scanning mode into a first preset feature extraction algorithm to obtain image omics data of magnetic resonance dynamic enhanced scanning imaging extracted from the breast cancer medical image in the sequential imaging examination;

the method comprises the steps of carrying out image segmentation on a breast cancer medical image in a magnetic resonance diffusion weighted imaging modality in the prior imaging examination to obtain a target lesion area image in the magnetic resonance diffusion weighted imaging modality, and inputting the target lesion area image in the magnetic resonance diffusion weighted imaging modality and an apparent diffusion coefficient image in the two successive magnetic resonance diffusion weighted imaging modalities into a second preset feature extraction algorithm to obtain image omics data of the magnetic resonance apparent diffusion coefficient image extracted from the breast cancer medical image in the two successive imaging examinations.

5. The method of claim 2, wherein the inputting the image analysis report in two consecutive imaging examinations into a predetermined coding model to obtain a second feature vector comprises:

and performing character recognition and preprocessing on the contents of the image analysis reports twice in sequence, and inputting the preprocessed character contents into a preset coding model to obtain a second feature vector, wherein the preset coding model is a pre-training language characterization model.

6. The method of claim 2, wherein inputting the clinical test report into the predetermined coding model to obtain a third feature vector comprises:

7. The method of any one of claims 1-6, wherein the breast cancer progression analysis model comprises a fully connected layer, a Dropout layer, a normalization and results output layer.

8. An apparatus for analyzing breast cancer progression, comprising:

9. A computer device, characterized in that the computer device comprises:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the breast cancer progression analysis method of any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method for breast cancer progression analysis according to any one of claims 1 to 7.