CN117671463A - Multi-mode medical data quality calibration method - Google Patents
Multi-mode medical data quality calibration method Download PDFInfo
- Publication number
- CN117671463A CN117671463A CN202311679486.1A CN202311679486A CN117671463A CN 117671463 A CN117671463 A CN 117671463A CN 202311679486 A CN202311679486 A CN 202311679486A CN 117671463 A CN117671463 A CN 117671463A
- Authority
- CN
- China
- Prior art keywords
- image
- deformation
- model
- motion
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 88
- 230000033001 locomotion Effects 0.000 claims abstract description 230
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000002059 diagnostic imaging Methods 0.000 claims abstract description 26
- 238000003384 imaging method Methods 0.000 claims abstract description 15
- 230000007774 longterm Effects 0.000 claims abstract description 11
- 230000001965 increasing effect Effects 0.000 claims abstract description 10
- 230000006872 improvement Effects 0.000 claims abstract description 7
- 230000035772 mutation Effects 0.000 claims abstract description 4
- 238000009826 distribution Methods 0.000 claims description 55
- 230000008859 change Effects 0.000 claims description 54
- 238000004458 analytical method Methods 0.000 claims description 37
- 238000001514 detection method Methods 0.000 claims description 32
- 230000002159 abnormal effect Effects 0.000 claims description 31
- 230000005856 abnormality Effects 0.000 claims description 31
- 238000006243 chemical reaction Methods 0.000 claims description 30
- 230000003287 optical effect Effects 0.000 claims description 28
- 238000001914 filtration Methods 0.000 claims description 24
- 238000011156 evaluation Methods 0.000 claims description 22
- 238000005457 optimization Methods 0.000 claims description 21
- 238000010606 normalization Methods 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 19
- 230000000007 visual effect Effects 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 17
- 238000013527 convolutional neural network Methods 0.000 claims description 15
- 239000000203 mixture Substances 0.000 claims description 14
- 238000005516 engineering process Methods 0.000 claims description 13
- 230000000737 periodic effect Effects 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 10
- 230000007170 pathology Effects 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000012512 characterization method Methods 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 7
- 230000004614 tumor growth Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 claims description 6
- 210000000056 organ Anatomy 0.000 claims description 6
- 230000035790 physiological processes and functions Effects 0.000 claims description 6
- 206010028980 Neoplasm Diseases 0.000 claims description 5
- 238000003708 edge detection Methods 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 5
- 230000004048 modification Effects 0.000 claims description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 4
- 230000000295 complement effect Effects 0.000 claims description 4
- 238000003066 decision tree Methods 0.000 claims description 4
- 230000005489 elastic deformation Effects 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000005286 illumination Methods 0.000 claims description 4
- 230000007787 long-term memory Effects 0.000 claims description 4
- 238000010827 pathological analysis Methods 0.000 claims description 4
- 239000013589 supplement Substances 0.000 claims description 4
- 238000012706 support-vector machine Methods 0.000 claims description 4
- 238000012098 association analyses Methods 0.000 claims description 3
- 230000006399 behavior Effects 0.000 claims description 3
- 238000010191 image analysis Methods 0.000 claims description 3
- 230000003902 lesion Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims description 3
- 230000001105 regulatory effect Effects 0.000 claims description 3
- 238000005096 rolling process Methods 0.000 claims description 3
- 238000011160 research Methods 0.000 abstract description 9
- 238000003745 diagnosis Methods 0.000 abstract description 4
- 201000010099 disease Diseases 0.000 abstract description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 3
- 238000002595 magnetic resonance imaging Methods 0.000 description 19
- 238000013184 cardiac magnetic resonance imaging Methods 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 4
- 210000004556 brain Anatomy 0.000 description 2
- 230000000747 cardiac effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003702 image correction Methods 0.000 description 2
- 231100000915 pathological change Toxicity 0.000 description 2
- 230000036285 pathological change Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000000611 regression analysis Methods 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000029058 respiratory gaseous exchange Effects 0.000 description 2
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000004064 dysfunction Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004217 heart function Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000013140 knowledge distillation Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002600 positron emission tomography Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 210000000115 thoracic cavity Anatomy 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Landscapes
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The application provides a multi-mode medical data quality calibration method, which comprises the steps of obtaining a multi-mode image sequence, comparing image sequence parameters, eliminating equipment differences, extracting image quality characteristics, comparing, judging equipment differences, combining metadata, the image quality characteristics and the image characteristics to form a comprehensive characteristic set, increasing data diversity through means such as image rotation, scaling and overturning, and judging that if structural characteristic mutation exceeds a preset threshold value, image processing is correspondingly carried out. The method and the device remarkably improve the quality and consistency of medical imaging data, effectively cope with the difference between different imaging devices, and provide more accurate and reliable image data for clinicians and researchers. The method can distinguish and correct the image deformation caused by the device difference or physiological motion, and ensure that the image quality is not affected by the factors. In addition, the method can also identify and enhance key feature details in the image, and improve comparability and usability of the image data in long-term research and multi-center clinical trials. By such improvements, the use of medical image data in disease diagnosis, treatment planning and medical research has become more accurate and efficient, providing better quality medical services to patients.
Description
Technical Field
The invention relates to the technical field of information, in particular to a multi-mode medical data quality calibration method.
Background
In modern medical imaging scientific research applications, magnetic resonance imaging, positron emission tomography and other multi-modality medical images play a critical role in disease diagnosis, treatment planning and research. However, these advanced imaging techniques face a number of challenges in scientific applications, particularly problems associated with physiological motion and device variability. In medical imaging procedures such as MRI and PET, small movements of the patient, such as breathing and heartbeat, can cause movement distortion during the imaging procedure. Such deformations may affect the quality and accuracy of the image and thus negatively impact diagnosis and treatment planning. For example, in cardiac or thoracic imaging, patient respiration and heartbeat can cause image blurring, making it difficult to identify critical anatomy. In addition, such physiological motion can also increase the time and complexity of image acquisition, placing additional burden on the patient and medical team. Different medical imaging devices, such as MRI and PET scanners from different manufacturers, vary in their specifications and performance. These differences may be in terms of spatial resolution, signal-to-noise ratio, contrast, etc. Thus, images acquired by different devices may differ in level of detail and sharpness even under the same imaging conditions. This difference between the devices presents challenges to clinicians and researchers in interpreting and comparing data. Especially in long-term scientific research, the equipment difference may cause data inconsistency, which affects the final scientific research result. In scientific practice, determining whether image distortion is caused by physiological motion or device differences is a complex and difficult task. Both of these factors may lead to degradation of image quality, but they require different correction methods. Motion distortion typically requires real-time monitoring and correction, while device differences require post-processing techniques to normalize and calibrate the images of different devices. Conventional calibration methods often fail to effectively distinguish between these two factors, resulting in limited calibration effectiveness. In order to solve the above problems, a new calibration technique that takes into consideration the characteristics of physiological motion and device differences is urgently needed. So as to promote the accuracy of scientific research.
Disclosure of Invention
The invention provides a multi-mode medical data quality calibration method, which mainly comprises the following steps:
collecting a multi-device medical multi-mode medical image sequence containing metadata, judging parameter differences of the metadata, extracting characteristics of the image based on a large model Vision Transformer, and identifying and correcting the differences of multi-mode data among different devices; extracting general features of different medical equipment images according to the multi-mode medical image sequence, comparing the similarity of the general features of the different medical equipment images, transferring the general features to the medical equipment images of a specified type when the similarity is higher than a preset threshold value, performing data cleaning and enhancement, performing parameter normalization before pathological analysis, and acquiring and uploading standardized images to a cloud platform, wherein the general features comprise textures, shapes and color distribution of the images; according to the time sequence, identifying the pixel motion and structural change of the standardized image, constructing a physiological motion model for motion abnormality identification, and constructing an equipment difference model for structural deformation detection; matching the detected deformation with the physiological motion model and the equipment difference model, and judging the coincidence degree of the deformation and the physiological motion model; quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image; according to the characteristic of the image deformation incapable of judging the category, simultaneously using the physiological motion model and the equipment difference model to identify and correct the image deformation incapable of judging the category, and performing iterative optimization and effect evaluation on the processed image; performing physiological periodic analysis on the processed image sequence, extracting and evaluating dynamic characteristics, performing data adjustment and first mode conversion, and obtaining an enhanced medical image sequence, wherein the dynamic characteristics comprise pixel brightness, pixel color intensity, local movement and local shape change; according to the enhanced medical image sequence, detecting pixel motion, identifying and correcting abnormal motion and deformation conditions, performing image semantic analysis and label prediction, analyzing sequence continuity, and obtaining medical imaging data; and according to the medical imaging data and the image metadata, performing equipment difference identification and calibration by adopting visual feature extraction, and performing second mode conversion and standardization.
In some embodiments, the collecting the multi-device medical multi-modality medical image sequence containing metadata, determining the parameter differences of the metadata, extracting the features of the image itself based on the large model of Vision Transformer, and identifying and correcting the differences of the multi-modality data between different devices includes:
acquiring a multi-mode image sequence through different medical imaging devices, wherein the multi-mode image sequence comprises metadata of each sequence, the metadata comprises a device type and imaging parameters, and the imaging parameters comprise resolution and contrast; comparing metadata parameters of different image sequences, and if the parameter difference in the metadata exceeds a parameter preset threshold, confirming that the difference between devices exists; according to medical images acquired by multiple devices, a Vision transducer-based model is applied, and quality features of the images are extracted, wherein the quality features comprise resolution, contrast and noise level; comparing the quality characteristic representations of the extracted images, and if the quality characteristic difference of the extracted images exceeds a quality preset threshold, determining that data difference exists between different devices; under the condition that the device difference cannot be clearly distinguished, combining the metadata and the image quality characteristics to form a comprehensive characteristic set; when the size of the image data set is lower than the preset number, the data diversity is increased through the technical means of image rotation, scaling and overturning; extracting structural features of the image, wherein the structural features comprise length, width proportion and edge features; if the structural features of the images have mutation in the time sequence, judging that potential deformation exists; and performing image processing on the image area with the deformation degree exceeding the deformation preset threshold, wherein the image processing comprises clipping the deformation area and interpolating the complement structure.
In some embodiments, the steps of extracting common features of different medical device images according to the multi-modal medical image sequence, comparing similarities of the common features of the different medical device images, and migrating the common features to a medical device image of a specified type when the similarities are higher than a preset threshold, performing data cleaning and enhancement, performing parameter normalization before pathological analysis, and obtaining and uploading standardized images to a cloud platform, wherein the common features include textures, shapes and color distribution of the images, include:
according to a multi-mode image sequence obtained from different medical imaging devices, model training is carried out by using a convolutional neural network algorithm, general features are extracted, and general feature similarity of images of different devices is compared, wherein the general features comprise textures, shapes and color distribution of the images; if the similarity is higher than a similarity preset threshold, directly migrating the general feature into the medical image of the appointed type; if the general features are not suitable for the medical images of the appointed type, acquiring multi-equipment images and labels thereof, extracting equipment invariant features by using a shared encoder, and predicting the labels by combining with an equipment exclusive decoder; continuously detecting and analyzing abnormal conditions predicted by the image labels, if the proportion of the abnormal samples is larger than a preset threshold value of the proportion of the abnormal samples, judging that interference difference parameters exist, performing data cleaning or enhancement processing, reducing differences among data distribution by adopting normalization processing, and generating new samples to supplement a data set; performing parameter normalization on images acquired by different devices, ensuring that all image data have consistent data quality and feature distribution before pathology analysis is performed, and using the normalized image data for a main task of pathology prediction; the median filtering algorithm is applied, so that the data precision is reduced, key characteristics are reserved, redundant parameter differences are eliminated, and standardized and good-quality image characterization is obtained; and uploading the standardized image to a cloud platform to realize sharing and cooperation among a plurality of devices.
In some embodiments, the identifying the pixel motion and the structural change of the standardized image according to the time sequence, constructing a physiological motion model for motion abnormality identification, and constructing a device difference model for structural deformation detection includes:
extracting optical flow and key point descriptors of a time sequence on a standardized image; analyzing the motion of pixel points in an image sequence by an optical flow method, and estimating the motion mode of an image block; if the optical flow vector is discontinuous or suddenly changed in the sequence, judging that the motion is abnormal; according to the standardized medical image sequence, a physiological motion model is constructed by applying long-term motion constraint of a model learning time sequence based on a transducer; comparing the current image motion with the model constraint, judging whether the current image motion accords with the model constraint, and if not, judging that the current image motion is deformed; creating a human motion knowledge graph containing skeletal node and joint connection information; mapping a new sample to the human motion knowledge graph, comparing topological structure changes, and detecting structural changes; according to the human motion knowledge graph, building a device difference model by using characteristic representations of nodes and edges in the graph rolling network learning graph, analyzing the representation of a new sample on the graph, and comparing the differences between the characteristics of the nodes and edges and the known normal structure mode, wherein the characteristics of the nodes and edges comprise the positions, the sizes and the shapes of the nodes; if the characteristics of the new sample and the known distribution have obvious drift, the deformation is caused by the difference of the parameters of the equipment; if the new sample detects motion and structural abnormality at the same time, carrying out association analysis to judge an abnormality source; if the motion abnormality is associated with the time series model, determining a physiological change; if the structural abnormality is associated with the equipment difference model, judging that the parameter is deviated; according to the standardized image sequence data, long-term motion mode and structural change are identified by utilizing long-term memory network learning time-space sequence characteristics, association of two factors including time and topology is fused, accuracy of deformation detection and abnormal source analysis is improved, and the time-space sequence characteristics comprise motion mode and structural change.
In some embodiments, the matching the detected deformation with the physiological motion model and the device difference model, and determining the degree of coincidence between the deformation and the physiological motion model includes:
calculating the Jaccard distance between the deformation and the standard physiological motion mode, wherein the smaller the distance is, the more the deformation accords with the physiological motion mode; calculating Euclidean distance between the deformation and the topological structure difference of the designated equipment map, wherein the smaller the distance is, the more likely the deformation is from the equipment difference; the output of the physiological motion model and the device difference model is processed by using a normalization processing method, the dimension influence is eliminated, and the output of the physiological motion model and the device difference model is ensured to be in the same numerical range; model training is carried out by using a decision tree algorithm according to the matching measurement of the deformation characteristics, the physiological motion model and the equipment difference model, a potential source prediction model of the deformation is constructed, the potential source of the deformation is judged, and the matching measurement is Jaccard distance or Euclidean distance; analyzing the linear correlation of the deformation with the output of the physiological motion model and the device difference model by using the Pearson correlation coefficient; if the linear correlation between the deformation characteristic and the motion model is greater than the linear correlation between the deformation characteristic and the equipment difference model, judging that the deformation is caused by physiological motion; if the linear correlation between the deformation characteristic and the equipment difference model is greater than the linear correlation between the deformation characteristic and the motion model, judging that the deformation is caused by the difference of the equipment parameters; according to the characteristics extracted from the multi-mode medical image, including optical flow, key point descriptors and structural changes in the image sequence, model training is carried out by adopting a support vector machine algorithm, and the similarity between the detected deformation and the learned deformation mode is determined; judging the deformation mode category of the detected deformation according to the similarity between the detected deformation and the mode; the detection result with insufficient confidence is marked as a category which cannot be judged; taking the judgment of the physiological motion model and the equipment difference model about deformation as a new sample, training data, and improving the judgment capability of the model; summarizing the detection result through repeated iterative learning, judging the attribution of deformation, and further comprising: and quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image.
In some embodiments, the quantifying nonlinear deformation evaluation criteria from deformation features in the multimodal medical image comprises:
acquiring medical image data of different modalities containing complex deformation features, wherein the complex deformation comprises deformation caused by tumor growth or organ movement; for different modes containing complex deformation characteristicsPerforming preliminary analysis on the medical image in a state, and determining a general mode of deformation characteristics, wherein the deformation characteristics comprise intensity change and shape distortion; based on the result of the preliminary analysis, formula D is set nl (x, y) =α·Δi (x, y) +β·Δg (x, y), where Δi (x, y) represents a change in pixel intensity, Δg (x, y) represents a change in local geometry, and α and β are coefficients that regulate the importance of both; obtaining delta I (x, y) by calculating pixel intensity differences between adjacent frames, determining delta G (x, y) by evaluating shape changes of the local area; according to the historical data, determining optimal values of coefficients alpha and beta of images of different image modes, wherein the image modes comprise CT and MRI, so as to reflect the characteristics of different imaging technologies; applying a defined formula D on a sequence of medical images of different modalities nl (x, y) generating a non-linear deformation score for each pixel or region, identifying non-linear deformation regions in the image; analyzing the distribution of nonlinear deformation scores, and identifying the area with the highest deformation degree in the image; testing and evaluating the validity of the formula by using the image data of the actual clinical case, and ensuring the applicability of the formula in multi-modal medical image analysis; and adjusting parameters in the formula according to the test and evaluation results.
In some embodiments, the identifying and correcting the image deformation of the unable to judge class by using the physiological motion model and the device difference model simultaneously according to the characteristic of the image deformation of the unable to judge class, and performing iterative optimization and effect evaluation on the processed image, including: aiming at complex image deformation which cannot be clearly distinguished from attribution, detecting by using the physiological motion model and the equipment difference model; acquiring the output of the physiological motion model and the device difference model, fusing the two outputs by using an attention mechanism, calculating the association weight of the output of each model, and evaluating the interpretation degree of the two models on the current deformation; if the weight of one model is higher than that of the other model, judging that the interpretation importance of the prediction result of the model with high weight on deformation is higher than that of the other model; if the weight difference value of the physiological motion model and the equipment difference model is smaller than a weight preset threshold value, the two model prediction results are equally important, and a detection result is obtained through joint decision; if the physiological motion model predicts that physiological motion abnormality exists, correcting deformation by using a data interpolation and motion regularization algorithm; if the equipment parameter deviation exists in the equipment difference model prediction, adopting an image enhancement and mode conversion method to adjust the image; for complex image deformation which cannot be clearly attributed, simultaneously applying a parameter adjustment method of physiological motion abnormality and equipment parameter deviation to perform joint optimization processing on the image; detecting again after treatment, extracting deformation characteristics, comparing with the deformation characteristics before optimization, and evaluating correction effects; after multiple iterations, selecting a model with processing precision and reasoning time being larger than preset threshold values, and applying the model to the deformation condition which cannot be clearly attributed to, so that the high efficiency of detection and optimization is ensured; further comprises: estimating and quantifying deformation characteristics between images according to the intensity and geometric variation of the time series medical images;
The method for estimating and quantifying deformation characteristics among images according to the intensity and geometric variation of the time series medical images specifically comprises the following steps: acquiring multi-modal medical images from different points in time showing potential deformation characteristics, determining a pattern of variation in the images, the pattern of variation comprising variations in pixel intensity and geometry; for image I 1 And I 2 Using the formula for each pixel position (x, y)Calculating absolute values of intensity differences of corresponding pixel points of the two images, and calculating average values of the intensity differences of all pixels to obtain an overall intensity difference metric delta I (I 1 ,I 2 ) N is the total number of pixels in the image; for each image, extracting boundaries of key structural elements by using a Canny algorithm; for each structure element, the formula Hausdorff (S 1 ,S 2 )＝max{sup s1 ∈S1inf s2 ∈S2d(s 1 ,s 2 ),sup s2 ∈S2inf s1 ∈S1d(s 1 ,s 2 ) Calculating Hausdorff distance between boundaries, wherein S 1 And S is 2 Is composed of boundary points of structural elements extracted from two images by Canny algorithmIs set of (d), d (s 1 ,s 2 ) Is the point s 1 Sum s 2 Euclidean distance between them; using the formulaCalculating the degree of geometry change ΔG (I) 1 ,I 2 )，S 1k And S is 2k Respectively represent image I 1 And I 2 The boundary of corresponding structural elements in (a), wherein the structural elements comprise organs and tumors, and M is the number of the structural elements; definition formula F ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Delta t quantization of two time points t 1 And t 2 Medical image I in between 1 And I 2 Wherein Δt=t 2 -t 1 Wherein I 1 And I 2 Images at two different time points respectively, Δt being the time difference; applying a formula according to images in different modes and different time sequences, and adjusting values of gamma and delta according to the reaction condition of the formula to the change; applying formula F to image data of actual clinical cases ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Δt, evaluate its behavior in actual situations, and adjust formulas and algorithms according to the evaluation result.
In some embodiments, the performing physiological periodic analysis on the processed image sequence, extracting and evaluating dynamic features, performing data adjustment and first modality conversion, and obtaining an enhanced medical image sequence, where the dynamic features include pixel brightness, pixel color intensity, local movement, and local shape change, includes:
performing physiological periodic analysis on the processed image sequence, acquiring time domain statistical characteristics under different scales by using wavelet transformation, and acquiring main frequency components of the image sequence by using fast Fourier transformation; if the main frequency component is missing, the image sequence is indicated to not reflect the complete physiological process; judging whether the dynamic characteristics of the image sequence accord with standard normal distribution or not by utilizing a Gaussian mixture model according to the dynamic characteristics extracted from the image sequence, wherein the dynamic characteristics extracted from the image sequence comprise time variation of pixel intensity and local motion characteristics; if the dynamic characteristics of the image sequence accord with single normal distribution, the image data are consistent, and no obvious abnormality exists; if the data accords with a plurality of normal distributions, different types of change modes are indicated; if the analysis result of the Gaussian mixture model shows that the dynamic characteristics do not accord with the standard distribution or have abnormality, the distribution shape is adjusted by using a data enhancement method, and the regularization characteristics are regulated by using the data enhancement method, wherein the data enhancement method comprises statistic attribute modification, affine transformation, elastic deformation, random noise addition and modal conversion; for samples with insufficient characteristics, generating diversified data by using a variable self-encoder according to the data with complete main frequency components; converting the image mode from CT to MRI, and increasing dynamic characterization of the physiological state under different modes; carrying out time domain and frequency domain feature extraction again on the processed image sequence, and judging whether periodic signal components are added; using a structural similarity index and a peak signal-to-noise ratio to evaluate the consistency of the image quality before and after processing, and ensuring that the generated data reach the standard in authenticity and effectiveness; further comprises: performing customized enhancement and quality improvement treatment according to the image region characteristics;
The customized enhancement and quality improvement processing is performed according to the image area characteristics, and specifically includes: analyzing the image by using a Canny edge detection algorithm, and identifying areas with contrast lower than a preset threshold; in the area with the contrast lower than the preset threshold value, sharpening enhancement and contrast enhancement are carried out by using a local histogram equalization technology; identifying a fuzzy or noise area in the image by using a Gaussian mixture model, improving the definition of the image by applying a Wiener filtering algorithm to the fuzzy area, removing noise in the image by applying a median filtering algorithm to the noise area, and enhancing and recovering details in the fuzzy area by using a super-resolution technology; for the region with the loss of details or the quality lower than a preset threshold value in the image, reconstructing by adopting a bicubic interpolation technology; applying a Wiener filtering algorithm and a median filtering algorithm to the reconstruction region, eliminating noise possibly introduced, and keeping the natural appearance and consistency of the image; the method of histogram equalization and color correction is combined to unify and optimize illumination, contrast and color balance of the whole image sequence; the image processing model is customized and optimized based on the characteristics of the CT and MRI images produced by the different devices.
In some embodiments, the detecting pixel motion, identifying and correcting abnormal motion and deformation according to the enhanced medical image sequence, performing image semantic analysis and label prediction, analyzing sequence continuity, and obtaining medical imaging data includes:
preprocessing the enhanced image sequence, including denoising and contrast adjustment; detecting pixel position change in each frame by using a light flow method, and determining pixel motion between two continuous frames; extracting a motion mode from the pixel motion, and identifying abnormal motion in an image sequence, including burst movement or deformation; identifying new or residual deformation conditions according to the optical flow result, and determining a movement area which is inconsistent with the expected physiological movement mode; applying a histogram equalization technique and a fourier transform to the identified deformed portion to correct or minimize the deformed portion; if the image sequence interruption is detected, connecting faults by using a cubic spline interpolation method to generate a smooth motion track; selecting effective features and eliminating features irrelevant to lesions according to the image semantic tags; according to the preprocessed and enhanced medical image sequence, predicting the label of each image by using a convolutional neural network, and judging the label consistency of the whole sequence; analyzing the difference between the labels by a regression method, and determining the consistency of the sequences; generating space-time characteristics by an optical flow method, acquiring motion codes by using discrete cosine transform, and judging the continuity of the codes; counting the proportion of invalid frames in the sequence, measuring the proportion of available frames, and if the proportion of the invalid frames is higher than a proportion preset threshold value, performing a new round of data enhancement; setting a key performance index threshold, wherein the key performance index comprises an intersection ratio, a peak signal-to-noise ratio and a processing frame rate; and continuously iterating the optimization processing until all the set performance indexes are met.
In some embodiments, the performing device difference recognition and calibration with visual feature extraction based on the medical imaging data and image metadata, performing a second modality conversion and normalization, comprises:
according to different types of medical imaging data and image metadata, a convolutional neural network is adopted to extract visual features in an image and the visual features are associated with equipment types, wherein the image metadata comprises manufacturer, model and scanning settings, and the visual features comprise textures, edges, contrast and shape of the image; comparing the device difference recognition results based on the metadata and the convolutional neural network model, and guiding the subsequent image calibration direction by using the comparison result; performing calibration conversion on the images judged to be from other devices; calculating the signal-to-noise ratio of the converted image, comparing the signal-to-noise ratio with the data of the target equipment, and evaluating the quality of parameter calibration; model training is carried out by adopting a generated countermeasure network according to images of different modes, key characteristics and distribution differences among different modes are learned, data distribution and characteristic layer conversion are carried out by using the generated countermeasure network model, and the image modes comprise CT and MRI; and on the standardized images of the equipment and the mode, the physiological motion model and the equipment difference model are repeatedly used for detection, so that self-optimized image processing circulation is realized, and standardized parameter images with signal to noise ratio larger than a preset threshold value of the signal to noise ratio and clear characteristics are obtained.
The technical scheme provided by the embodiment of the invention can have the following beneficial effects:
the invention discloses a multi-mode medical data quality calibration method, which remarkably improves the quality and consistency of medical imaging data, effectively aims at the difference between different imaging devices, and provides more accurate and reliable image data for clinicians and researchers. The method can distinguish and correct the image deformation caused by the device difference or physiological motion, and ensure that the image quality is not affected by the factors. In addition, the method can also identify and enhance key feature details in the image, and improve comparability and usability of the image data in long-term research and multi-center clinical trials. By such improvements, the use of medical image data in disease diagnosis, treatment planning and medical research has become more accurate and efficient, providing better quality medical services to patients.
Drawings
FIG. 1 is a flow chart of a multi-modal medical data quality calibration method of the present invention.
Fig. 2 is a schematic diagram of a multi-modal medical data quality calibration method according to the present invention.
FIG. 3 is a schematic diagram of a multi-modal medical data quality calibration method according to the present invention.
Detailed Description
For a further understanding of the present invention, the present invention will be described in detail with reference to the drawings and examples. The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the invention are shown in the drawings.
Referring to fig. 1-3, a method for calibrating quality of multi-modal medical data according to the present embodiment may specifically include:
step S101, collecting a multi-device medical multi-mode medical image sequence containing metadata, judging parameter differences of the metadata, extracting features of the image itself based on a large model of Vision Transformer, and identifying and correcting the differences of the multi-mode data among different devices.
Acquiring a multi-mode image sequence through different medical imaging devices, wherein the multi-mode image sequence comprises metadata of each sequence, the metadata comprises a device type and imaging parameters, and the imaging parameters comprise resolution and contrast; comparing metadata parameters of different image sequences, and if the parameter difference in the metadata exceeds a parameter preset threshold, confirming that the difference between devices exists; according to medical images acquired by multiple devices, a Vision transducer-based model is applied, and quality features of the images are extracted, wherein the quality features comprise resolution, contrast and noise level; comparing the quality characteristic representations of the extracted images, and if the quality characteristic difference of the extracted images exceeds a quality preset threshold, determining that data difference exists between different devices; under the condition that the device difference cannot be clearly distinguished, combining the metadata and the image quality characteristics to form a comprehensive characteristic set; when the size of the image data set is lower than the preset number, the data diversity is increased through the technical means of image rotation, scaling and overturning; extracting structural features of the image, wherein the structural features comprise length, width proportion and edge features; if the structural features of the images have mutation in the time sequence, judging that potential deformation exists; and performing image processing on the image area with the deformation degree exceeding the deformation preset threshold, wherein the image processing comprises clipping the deformation area and interpolating the complement structure.
For example, there are two medical imaging devices, device a and device B, which are used to acquire a sequence of brain MRI images. For device a, its resolution is 256x256 pixels, contrast is 10, and noise level is 5. Whereas for device B, the resolution is 512x512 pixels, the contrast is 15, and the noise level is 8. Thus, there is a large difference in resolution, contrast, noise level, and the like between the device a and the device B. In order to compare the metadata parameters of different image sequences, a threshold of 5 is set. From this threshold, it was found that device a and device B exceeded the preset threshold in terms of resolution and contrast, confirming that there was an inter-device difference. Next, quality features of the image were extracted using a vision transducer-based model, the image resolution of device a was 256×256 pixels, the contrast was 10, the noise level was 5, and the image resolution of device B was 512×512 pixels, the contrast was 15, and the noise level was 8. Comparing the characteristics, setting a threshold to be 3, finding that the quality characteristic difference of the equipment A and the equipment B in the aspects of resolution and contrast exceeds a preset threshold according to the threshold, and determining that the data difference exists between different equipment. Under the condition that the device difference cannot be clearly distinguished, combining the metadata, the image quality characteristics and the image self characteristics to form a comprehensive characteristic set; under the condition of insufficient data or poor quality, the data diversity is increased by the technical means of image rotation, scaling and overturning; in addition, structural features of the image, including length, width scale and edge features, can also be extracted. There is a set of brain MRI image sequences, where one image is 200mm long, 100mm wide, and 2:1 length to width ratio, with sharp edge features. If an image suddenly appears in the time series, the length is 180mm, the width is 120mm, the ratio of the length to the width is 3:2, the edge feature becomes blurred, and the potential image deformation can be judged. For the image area with the deformation degree exceeding the preset threshold, image processing such as clipping the deformation area and interpolating the complement structure can be performed to repair the problem caused by deformation.
Step S102, extracting general features of different medical equipment images according to the multi-mode medical image sequence, comparing the similarity of the general features of the different medical equipment images, transferring the general features to the medical equipment images of a specified type when the similarity is higher than a preset threshold, performing data cleaning and enhancement, performing parameter normalization before pathological analysis, and acquiring and uploading a standardized image to a cloud platform, wherein the general features comprise textures, shapes and color distribution of the images.
According to a multi-mode image sequence obtained from different medical imaging devices, model training is carried out by using a convolutional neural network algorithm, general features are extracted, and general feature similarity of images of different devices is compared, wherein the general features comprise textures, shapes and color distribution of the images; if the similarity is higher than a similarity preset threshold, directly migrating the general feature into the medical image of the appointed type; if the general features are not suitable for the medical images of the appointed type, acquiring multi-equipment images and labels thereof, extracting equipment invariant features by using a shared encoder, and predicting the labels by combining with an equipment exclusive decoder; continuously detecting and analyzing abnormal conditions predicted by the image labels, if the proportion of the abnormal samples is larger than a preset threshold value of the proportion of the abnormal samples, judging that interference difference parameters exist, performing data cleaning or enhancement processing, reducing differences among data distribution by adopting normalization processing, and generating new samples to supplement a data set; performing parameter normalization on images acquired by different devices, ensuring that all image data have consistent data quality and feature distribution before pathology analysis is performed, and using the normalized image data for a main task of pathology prediction; the median filtering algorithm is applied, so that the data precision is reduced, key characteristics are reserved, redundant parameter differences are eliminated, and standardized and good-quality image characterization is obtained; and uploading the standardized image to a cloud platform to realize sharing and cooperation among a plurality of devices.
For example, there are three different medical imaging devices A, B and C, which are to be compared for their feature similarity of acquired image sequences. First, a number of multimodal image sequences are acquired from devices A, B and C and texture, shape and color distribution features of the images are extracted using a pre-trained convolutional neural network model. These features are represented as vectors, such as image texture feature vector [2,8,5], shape feature vector [6,4,9], color distribution feature vector [1,3,7]. The feature similarity of the different device images is then compared, and if there is one image from device a, the feature vector is [2,8,5]. The similarity between this feature vector and the image feature vectors of devices B and C is calculated, for example, the similarity to device B is 0.6 and the similarity to device C is 0.3. If the similarity is above a preset threshold of 0.55, the features may be migrated directly to the medical image of the specified type, and if the similarity is below the preset threshold, further processing is required. If the feature does not fit in the target domain, more multi-device images and their labels need to be acquired. The shared encoder is then used to extract the device invariant features and the device specific decoder is used to predict the respective tags, such as training the shared encoder and the device a decoder with the device a image and the tag, training the shared encoder and the device B decoder with the device B image and the tag, and so on. In the continuous detection and analysis of image tag predictions, the proportion of abnormal samples can be calculated, which is 20% in the image predictions of device a. If the abnormal sample proportion is greater than the preset threshold value by 10%, the differential parameter with interference can be judged. Data cleansing or enhancement processes may be performed, such as deleting abnormal samples or generating new samples to supplement the data set. In order to reduce the data distribution differences between different devices, normalization processes, such as parameter normalization performed on images acquired by different devices, may be used to ensure that all image data has a consistent data quality and feature distribution. The normalized image data is then used for the primary task of pathology prediction. To deploy the optimized model on a different device, knowledge distillation techniques may be used to extract knowledge about the pathology from a large base model for pathology analysis and migrate to a small efficient model. And then, the optimized model is published to different devices to realize sharing and collaboration. And finally, applying a median filtering algorithm to reduce the data precision, reserving key characteristics, eliminating redundant parameter differences and obtaining standardized and quality image characterization. And uploading the standardized image to a cloud platform to realize sharing and cooperation among a plurality of devices.
And step S103, recognizing the pixel motion and the structural change of the standardized image according to the time sequence, constructing a physiological motion model for motion abnormality recognition, and constructing a device difference model for structural deformation detection.
Extracting optical flow and key point descriptors of a time sequence on a standardized image; analyzing the motion of pixel points in an image sequence by an optical flow method, and estimating the motion mode of an image block; if the optical flow vector is discontinuous or suddenly changed in the sequence, judging that the motion is abnormal; according to the standardized medical image sequence, a physiological motion model is constructed by applying long-term motion constraint of a model learning time sequence based on a transducer; comparing the current image motion with the model constraint, judging whether the current image motion accords with the model constraint, and if not, judging that the current image motion is deformed; creating a human motion knowledge graph containing skeletal node and joint connection information; mapping a new sample to the human motion knowledge graph, comparing topological structure changes, and detecting structural changes; according to the human motion knowledge graph, building a device difference model by using characteristic representations of nodes and edges in the graph rolling network learning graph, analyzing the representation of a new sample on the graph, and comparing the differences between the characteristics of the nodes and edges and the known normal structure mode, wherein the characteristics of the nodes and edges comprise the positions, the sizes and the shapes of the nodes; if the characteristics of the new sample and the known distribution have obvious drift, the deformation is caused by the difference of the parameters of the equipment; if the new sample detects motion and structural abnormality at the same time, carrying out association analysis to judge an abnormality source; if the motion abnormality is associated with the time series model, determining a physiological change; if the structural abnormality is associated with the equipment difference model, judging that the parameter is deviated; according to the standardized image sequence data, long-term motion mode and structural change are identified by utilizing long-term memory network learning time-space sequence characteristics, association of two factors including time and topology is fused, accuracy of deformation detection and abnormal source analysis is improved, and the time-space sequence characteristics comprise motion mode and structural change.
For example, on a normalized image, there is a sequence of images of size 100x100 pixels, which contains 10 images. An optical flow vector in each image is extracted by using an optical flow method, the motion direction and the speed of a pixel point are represented, one pixel point (p 1) is selected as a key point in the first image, and the optical flow vector is calculated as (2, 1). In the next image, the position of the pixel is found to be changed, and the optical flow vector is calculated as (3, -2). This means that the pixel point is shifted by 1 pixel unit to the right and 3 pixel units downward in the time series. By analysis of the optical flow vectors, the motion pattern of the image block can be estimated, and if in the first 5 images, it is observed that the optical flow vectors remain in the horizontal direction all the time and are suddenly moved up by 10 pixel units in the 6 th image, it can be determined that the motion is abnormal. On the other hand, long-term motion constraints of the time series can be learned using a transducer-based model. If the motion constraint learned by the model is that the motion direction of the image block should be consistent in time series, and no abrupt change should occur. For the current image, the optical flow vector extracted by the optical flow method can be used for comparing with the model constraint, and whether the motion of the current image accords with the model constraint or not can be judged. If the optical flow vector is discontinuous or abrupt in the sequence, it may be determined as distorted. In addition, a human motion knowledge graph containing skeletal node and articulation information may be created. If there is a known human motion knowledge graph, it includes each skeletal node of human body and the association relationship between them. The image of the new sample can be mapped into a human motion knowledge graph, and the change of the topological structure can be compared to detect the structural change. According to the human motion knowledge graph, by using the graph convolution network to learn the characteristic representation of the nodes and edges in the graph, whether the new sample is matched with the characteristic distribution in the domain can be judged. If the characteristics of the new sample drift significantly from the known distribution, it may be indicated that the deformation is due to a difference in the parameters of the device. Thus, by learning spatiotemporal sequence features using long-term memory networks, long-term motion patterns and structural changes can be identified using normalized image sequence data. By combining the association of two factors of time and topology, the accuracy of deformation detection and anomaly source analysis can be improved, and the time-space sequence characteristics comprise a motion mode and structural change.
And step S104, matching the detected deformation with the physiological motion model and the equipment difference model, and judging the coincidence degree of the deformation and the physiological motion model and the equipment difference model.
Calculating the Jaccard distance between the deformation and the standard physiological motion mode, wherein the smaller the distance is, the more the deformation accords with the physiological motion mode; calculating Euclidean distance between the deformation and the topological structure difference of the designated equipment map, wherein the smaller the distance is, the more likely the deformation is from the equipment difference; the output of the physiological motion model and the device difference model is processed by using a normalization processing method, the dimension influence is eliminated, and the output of the physiological motion model and the device difference model is ensured to be in the same numerical range; model training is carried out by using a decision tree algorithm according to the matching measurement of the deformation characteristics, the physiological motion model and the equipment difference model, a potential source prediction model of the deformation is constructed, the potential source of the deformation is judged, and the matching measurement is Jaccard distance or Euclidean distance; analyzing the linear correlation of the deformation with the output of the physiological motion model and the device difference model by using the Pearson correlation coefficient; if the linear correlation between the deformation characteristic and the motion model is greater than the linear correlation between the deformation characteristic and the equipment difference model, judging that the deformation is caused by physiological motion; if the linear correlation between the deformation characteristic and the equipment difference model is greater than the linear correlation between the deformation characteristic and the motion model, judging that the deformation is caused by the difference of the equipment parameters; according to the characteristics extracted from the multi-mode medical image, including optical flow, key point descriptors and structural changes in the image sequence, model training is carried out by adopting a support vector machine algorithm, and the similarity between the detected deformation and the learned deformation mode is determined; judging the deformation mode category of the detected deformation according to the similarity between the detected deformation and the mode; the detection result with insufficient confidence is marked as a category which cannot be judged; taking the judgment of the physiological motion model and the equipment difference model about deformation as a new sample, training data, and improving the judgment capability of the model; summarizing the detection result through repeated iterative learning, judging the attribution of deformation, and further comprising: and quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image.
For example, the Jaccard distance of the deformation to the standard physiological movement pattern is calculated, and the resulting Jaccard distance is 2. This means that the overlap between the deformation and the physiological motion pattern is 20% of the total, indicating that the deformation relatively corresponds to the physiological motion pattern. Next, the euclidean distance of the distortion to the specified device map topology difference is calculated, and the resulting euclidean distance is 5. This suggests that the degree of difference between the deformation and the device difference is small, possibly resulting from the similarity of the devices. In order to eliminate the dimension influence, the output of the physiological motion model and the device difference model is normalized. If the output range of the physiological motion model is 0-100, the output range of the device difference model is 0-10, and the output of the device difference model are mapped into the range of 0-1 through normalization processing. Next, model training is performed by using a decision tree algorithm, and a potential source prediction model of the deformation is constructed. And training according to the matching metrics of the deformed characteristics, the physiological motion model and the equipment difference model by taking the Jaccard distance and the Euclidean distance as the matching metrics. Through repeated iterative learning, the discrimination capability of the model can be improved, and the attribution of deformation can be finally judged. In addition, pearson correlation coefficients can be used to analyze the linear correlation between deformation and the output of the physiological motion model, the device difference model. If the linear correlation coefficient between the deformation characteristic and the motion model is 8 and the linear correlation coefficient between the deformation characteristic and the device difference model is 4, the deformation is more likely to be caused by physiological motion according to the comparison of the linear correlation coefficients. In addition, features extracted from the multimodal medical image may also be trained using support vector machine algorithms to determine the similarity between newly detected deformations and deformation patterns that have been learned. Based on the similarity between the new deformation and the pattern, it may be determined that the new deformation is most likely to belong to the known deformation pattern class. Finally, through repeated iterative learning and summarizing of the detection results, the attribution of deformation can be judged, and the discrimination capability of the model is improved. If the detection result with insufficient confidence coefficient exists, the detection result is marked as a category which cannot be judged.
Step S104, quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image.
Acquiring medical image data of different modalities containing complex deformation features, wherein the complex deformation comprises deformation caused by tumor growth or organ movement; preliminary analysis is carried out on medical images of different modes containing complex deformation characteristics, and the general modes of the deformation characteristics are determined, wherein the deformation characteristics comprise intensity change and shape distortion; based on the result of the preliminary analysis, formula D is set nl (x, y) =α·Δi (x, y) +β·Δg (x, y), where Δi (x, y) represents a change in pixel intensity, Δg (x, y) represents a change in local geometry, and α and β are coefficients that regulate the importance of both; obtaining delta I (x, y) by calculating pixel intensity differences between adjacent frames, determining delta G (x, y) by evaluating shape changes of the local area; according to the historical data, determining optimal values of coefficients alpha and beta of images of different image modes, wherein the image modes comprise CT and MRI, so as to reflect the characteristics of different imaging technologies; applying a defined formula D on a sequence of medical images of different modalities nl (x, y) generating a non-linear deformation score for each pixel or region, identifying non-linear deformation regions in the image; analyzing the distribution of nonlinear deformation scores, and identifying the area with the highest deformation degree in the image; testing and evaluating the validity of the formula by using the image data of the actual clinical case, and ensuring the applicability of the formula in multi-modal medical image analysis; and adjusting parameters in the formula according to the test and evaluation results.
For example, there is a need to study the performance of non-linear deformation features caused by tumor growth in CT and MRI images. Tumor images containing complex deformation features were collectedSequences, including CT and MRI modalities. These images are first subjected to a preliminary analysis to determine the general pattern of deformation features. In the preliminary analysis, it was observed that nonlinear deformation caused by tumor growth was mainly manifested by a change in strength and a distortion in shape. From these observations, equation D is set nl (x, y) =α·Δi (x, y) +β·Δg (x, y) for quantifying the degree of nonlinear deformation of any point (x, y) in the image. Where Δi (x, y) represents the change in pixel intensity, Δg (x, y) represents the change in local geometry, and α and β are coefficients that regulate the importance of both. In order to determine the optimal values of the coefficients alpha and beta of the images of different image modes, historical data are used for experiments and analysis to obtain alpha=7, beta=3 of a CT mode and alpha=5, beta=5 of an MRI mode; next, the defined formula D will be applied nl (x, y) onto the sequence of medical images of different modalities, a nonlinear deformation score is generated for each pixel or region. Δi (x, y) is derived by calculating the pixel intensity difference between adjacent frames, and Δg (x, y) is determined by evaluating the shape change of the local area. The distribution of the non-linear deformation scores will then be analyzed to identify the regions of the image with the highest degree of deformation. Through further investigation of these regions, the non-linear deformation characteristics caused by tumor growth can be understood in depth. A CT image sequence is selected that contains complex deformation features caused by tumor growth. Will apply equation D nl (x, y) and generate a non-linear deformation score for each pixel or region. By comparison with the manual labeling of clinical experts, equation D can be evaluated nl Accuracy and reliability of (x, y). According to the test and evaluation results, parameters in the formula can be adjusted, and the accuracy and reliability of the identification and quantification of the nonlinear deformation characteristics are improved. Thus, by applying the defined formula D on medical images of different modalities nl (x, y) non-linear deformation features in the image can be identified and quantified.
And step S106, according to the characteristic of the image deformation incapable of judging the category, simultaneously using the physiological motion model and the equipment difference model to identify and correct the image deformation incapable of judging the category, and performing iterative optimization and effect evaluation on the processed image.
Aiming at complex image deformation which cannot be clearly distinguished from attribution, detecting by using the physiological motion model and the equipment difference model; acquiring the output of the physiological motion model and the device difference model, fusing the two outputs by using an attention mechanism, calculating the association weight of the output of each model, and evaluating the interpretation degree of the two models on the current deformation; if the weight of one model is higher than that of the other model, judging that the interpretation importance of the prediction result of the model with high weight on deformation is higher than that of the other model; if the weight difference value of the physiological motion model and the equipment difference model is smaller than a weight preset threshold value, the two model prediction results are equally important, and a detection result is obtained through joint decision; if the physiological motion model predicts that physiological motion abnormality exists, correcting deformation by using a data interpolation and motion regularization algorithm; if the equipment parameter deviation exists in the equipment difference model prediction, adopting an image enhancement and mode conversion method to adjust the image; for complex image deformation which cannot be clearly attributed, simultaneously applying a parameter adjustment method of physiological motion abnormality and equipment parameter deviation to perform joint optimization processing on the image; detecting again after treatment, extracting deformation characteristics, comparing with the deformation characteristics before optimization, and evaluating correction effects; after multiple iterations, a model with processing precision and reasoning time being larger than preset threshold values is selected and applied to the deformation condition which cannot be clearly attributed, so that the high efficiency of detection and optimization is ensured.
For example, the physiological motion model and the device difference model respectively output a correlation index ranging from 0 to 1, which indicates the interpretation degree of the current deformation, and a preset threshold value of 5 is set, which indicates that when the weight difference of the two models exceeds 5, the interpretation importance of the prediction result of one model on the deformation is considered to be greater than that of the other model. If the correlation index output by the physiological motion model is 7 and the correlation index output by the equipment difference model is 6, the weight difference value of the correlation index and the equipment difference model is 1 and is smaller than a preset threshold value, and the prediction results of the two models are equal in importance. And obtaining a detection result through joint decision. If the correlation index output by the physiological motion model is 8 and the correlation index output by the equipment difference model is 4, the weight difference value of the correlation index and the equipment difference model is 4 and is larger than a preset threshold value, and the explanation importance of the prediction result of the physiological motion model on deformation is larger than that of the equipment difference model. The physiological motion model will be chosen as the main interpretation model and applied to all ambiguous deformations. In dealing with physiological motion anomalies, data interpolation and motion regularization algorithms may be used to correct for distortion. For an image with abnormal physiological motion, normal data points can be generated around abnormal points by an interpolation method, and then the whole motion track is smoothed by using a motion regularization algorithm, so that deformation is corrected. When the parameter deviation of the equipment is processed, the image can be adjusted by adopting the methods of image enhancement and mode conversion. For the image deformation caused by the deviation of one device parameter, the image can be enhanced, the definition and the contrast of the image can be improved, or the image can be converted into another mode, so that more accurate deformation characteristics can be obtained. For complex deformation which cannot be clearly attributed, a parameter adjustment method of physiological motion abnormality and equipment parameter deviation can be simultaneously applied to perform joint optimization processing on the image. And then detecting again, extracting deformation characteristics, comparing with the result before optimization, and evaluating the correction effect. Through multiple iterations, a model with processing precision and reasoning time larger than preset thresholds can be selected and applied to all ambiguous deformation conditions so as to ensure the high efficiency of detection and optimization.
And evaluating and quantifying deformation characteristics among the images according to the intensity and geometric change of the time series medical images.
Acquiring multi-modal medical images from different points in time showing potential deformation characteristics, determining a pattern of variation in the images, the pattern of variation comprising variations in pixel intensity and geometry; for image I 1 And I 2 Using the formula for each pixel position (x, y)Calculating absolute values of intensity differences of corresponding pixel points of the two images, and calculating average values of the intensity differences of all pixels to obtain an overall intensity difference metric delta I (I 1 ,I 2 ) N is the total number of pixels in the image; for each image, use is made ofExtracting boundaries of key structural elements by a Canny algorithm; for each structure element, the formula Hausdorff (S 1 ,S 2 )＝max{sup s1 ∈S1inf s2 ∈S2d(s 1 ,s 2 ),sup s2 ∈S2inf s1 ∈S1d(s 1 ,s 2 ) Calculating Hausdorff distance between boundaries, wherein S 1 And S is 2 Is a set of boundary points of structural elements extracted from two images by the Canny algorithm, d (s 1 ,s 2 ) Is the point s 1 Sum s 2 Euclidean distance between them; using the formulaCalculating the degree of geometry change ΔG (I) 1 ,I 2 )，S 1k And S is 2k Respectively represent image I 1 And I 2 The boundary of corresponding structural elements in (a), wherein the structural elements comprise organs and tumors, and M is the number of the structural elements; definition formula F ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Delta t quantization of two time points t 1 And t 2 Medical image I in between 1 And I 2 Wherein Δt=t 2 -t 1 Wherein I 1 And I 2 Images at two different time points respectively, Δt being the time difference; applying a formula according to images in different modes and different time sequences, and adjusting values of gamma and delta according to the reaction condition of the formula to the change; applying formula F to image data of actual clinical cases ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δΔG(I 1 ,I 2 ) Δt, evaluate its behavior in actual situations, and adjust formulas and algorithms according to the evaluation result.
For example, there are two different points in time t 1 And t 2 Captured cardiac MRI image I 1 And I 2 These two time points may be several months apart and it is desirable to evaluate the shape and size changes of the heart over this period of time. Acquisition of I 1 And I 2 And (5) primarily observing the two images, and identifying the pixel intensity and geometric shape change of the heart region. Applying the formulaThe pixel intensity difference is calculated. An absolute value of the intensity difference is calculated at each pixel location in the image. These difference values are averaged to obtain an overall intensity difference metric Δi. Using the Canny algorithm at I 1 And I 2 The boundary of the heart is extracted. For the boundary of the heart, the Hausdorff distance Hausdorff is calculated in two images (S 1k ,S 2k ). Use formula +.>The degree of geometry change is calculated. Define and apply equation F ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Δt, and adjusting the parameters γ and δ according to the characteristics of the cardiac MRI image. The evaluation formula is expressed in the actual clinical case by applying formula F ts (I 1 ,I 2 Δt), the shape and size change of the heart region is evaluated. The formulas are further adjusted and optimized based on clinical feedback.
Step S107, performing physiological periodic analysis on the processed image sequence, extracting and evaluating dynamic characteristics, performing data adjustment and first mode conversion, and obtaining an enhanced medical image sequence, wherein the dynamic characteristics comprise pixel brightness, pixel color intensity, local movement and local shape change.
Performing physiological periodic analysis on the processed image sequence, acquiring time domain statistical characteristics under different scales by using wavelet transformation, and acquiring main frequency components of the image sequence by using fast Fourier transformation; if the main frequency component is missing, the image sequence is indicated to not reflect the complete physiological process; judging whether the dynamic characteristics of the image sequence accord with standard normal distribution or not by utilizing a Gaussian mixture model according to the dynamic characteristics extracted from the image sequence, wherein the dynamic characteristics extracted from the image sequence comprise time variation of pixel intensity and local motion characteristics; if the dynamic characteristics of the image sequence accord with single normal distribution, the image data are consistent, and no obvious abnormality exists; if the data accords with a plurality of normal distributions, different types of change modes are indicated; if the analysis result of the Gaussian mixture model shows that the dynamic characteristics do not accord with the standard distribution or have abnormality, the distribution shape is adjusted by using a data enhancement method, and the regularization characteristics are regulated by using the data enhancement method, wherein the data enhancement method comprises statistic attribute modification, affine transformation, elastic deformation, random noise addition and modal conversion; for samples with insufficient characteristics, generating diversified data by using a variable self-encoder according to the data with complete main frequency components; converting the image mode from CT to MRI, and increasing dynamic characterization of the physiological state under different modes; carrying out time domain and frequency domain feature extraction again on the processed image sequence, and judging whether periodic signal components are added; and (3) evaluating the consistency of the image quality before and after processing by using the structural similarity index and the peak signal-to-noise ratio, and ensuring that the generated data reach the standard in authenticity and effectiveness.
For example, there is a set of cardiac MRI images that capture a series of heart cycles that need to be analyzed to assess the regularity of the heart beat and the functional state of the heart. The cardiac MRI image sequence is preprocessed, including image denoising and alignment. And acquiring time domain statistical characteristics of the image sequence under different scales by using wavelet transformation, and analyzing the periodical change of the heartbeat. The primary frequency component of the image sequence is extracted by applying a fast fourier transform, and the primary frequency of the heartbeat is identified. The change of pixel intensity with time and local motion characteristics are extracted from the image sequence, and whether the characteristics accord with standard normal distribution is analyzed by using a Gaussian mixture model. If the data accords with the single normal distribution, the heart function is stable and no obvious abnormality exists. If the data conforms to a plurality of normal distributions, it may be indicative of irregular beating or dysfunction of the heart. If the Gaussian mixture model analysis shows that the data does not accord with the standard distribution or has abnormality, a data enhancement method is used for adjusting the distribution shape, and the data enhancement method comprises statistical attribute modification, affine transformation, elastic deformation, random noise addition and modal conversion. For samples with insufficient features, more diversified data is generated using a variational self-encoder. Consider converting an image modality from CT to MRI, enhancing the dynamic characterization of cardiac states. And re-extracting time domain and frequency domain characteristics of the processed image sequence, and checking whether the periodic signal component is added. And the consistency of the images before and after processing is evaluated by using the structural similarity index and the peak signal-to-noise ratio, so that the authenticity and the effectiveness of the data are ensured.
And carrying out customized enhancement and quality improvement treatment according to the image area characteristics.
Analyzing the image by using a Canny edge detection algorithm, and identifying areas with contrast lower than a preset threshold; in the area with the contrast lower than the preset threshold value, sharpening enhancement and contrast enhancement are carried out by using a local histogram equalization technology; identifying a fuzzy or noise area in the image by using a Gaussian mixture model, improving the definition of the image by applying a Wiener filtering algorithm to the fuzzy area, removing noise in the image by applying a median filtering algorithm to the noise area, and enhancing and recovering details in the fuzzy area by using a super-resolution technology; for the region with the loss of details or the quality lower than a preset threshold value in the image, reconstructing by adopting a bicubic interpolation technology; applying a Wiener filtering algorithm and a median filtering algorithm to the reconstruction region, eliminating noise possibly introduced, and keeping the natural appearance and consistency of the image; the method of histogram equalization and color correction is combined to unify and optimize illumination, contrast and color balance of the whole image sequence; the image processing model is customized and optimized based on the characteristics of the CT and MRI images produced by the different devices.
For example, there is a CT image in which areas of low contrast are concentrated in the tumor portion. The Canny edge detection algorithm can be used to identify these areas of low contrast. First, a contrast threshold 5 is set. Then, a Canny edge detection algorithm is applied to the image to obtain edge information in the image. Next, a region with a contrast lower than the threshold is found from the edge information. If the contrast ratio of 100 pixels in a certain region of the image is lower than 5. Local histogram equalization techniques may be used to enhance the contrast of these regions. For this region, a histogram of pixel values may be calculated and the pixel values reassigned according to a histogram equalization algorithm to enhance contrast. After histogram equalization, the average contrast of this region is increased to 7. In a certain area in the image, the Gaussian mixture model considers that blurring exists, the definition of the area can be improved by applying a Wiener filtering algorithm, and the Wiener filtering algorithm can restore the image according to the blurring characteristic, so that the image becomes clearer in the blurring area. After Wiener filtering, the definition of the blurred regions is significantly improved. In a certain region in the image, the gaussian mixture model considers that noise exists, a median filtering algorithm can be applied to remove the noise of the region, and the median filtering algorithm can calculate a median according to values around the pixel points, so that the influence of the noise is removed. After median filtering, the image quality of the noise region is significantly improved. For areas where detail is lost or quality is below a threshold, a bicubic interpolation technique may be used for reconstruction. If in the reconstruction region, bicubic interpolation techniques may recalculate the missing details based on the values of surrounding pixels. Finally, wiener filtering algorithm and median filtering algorithm can be applied to eliminate noise possibly introduced, and natural appearance and consistency of the reconstruction region are maintained. By combining the methods of histogram equalization and color correction, the illumination, contrast and color balance of the entire image sequence can be unified and optimized. The number of pixels with contrast lower than the threshold in the image can be calculated when the contrast threshold is 5, or the number of regions in the Gaussian mixture model can be found out through statistical analysis.
Step S108, detecting pixel motion according to the enhanced medical image sequence, identifying and correcting abnormal motion and deformation conditions, performing image semantic analysis and label prediction, analyzing sequence continuity, and obtaining medical imaging data.
Preprocessing the enhanced image sequence, including denoising and contrast adjustment; detecting pixel position change in each frame by using a light flow method, and determining pixel motion between two continuous frames; extracting a motion mode from the pixel motion, and identifying abnormal motion in an image sequence, including burst movement or deformation; identifying new or residual deformation conditions according to the optical flow result, and determining a movement area which is inconsistent with the expected physiological movement mode; applying a histogram equalization technique and a fourier transform to the identified deformed portion to correct or minimize the deformed portion; if the image sequence interruption is detected, connecting faults by using a cubic spline interpolation method to generate a smooth motion track; selecting effective features and eliminating features irrelevant to lesions according to the image semantic tags; according to the preprocessed and enhanced medical image sequence, predicting the label of each image by using a convolutional neural network, and judging the label consistency of the whole sequence; analyzing the difference between the labels by a regression method, and determining the consistency of the sequences; generating space-time characteristics by an optical flow method, acquiring motion codes by using discrete cosine transform, and judging the continuity of the codes; counting the proportion of invalid frames in the sequence, measuring the proportion of available frames, and if the proportion of the invalid frames is higher than a proportion preset threshold value, performing a new round of data enhancement; setting a key performance index threshold, wherein the key performance index comprises an intersection ratio, a peak signal-to-noise ratio and a processing frame rate; and continuously iterating the optimization processing until all the set performance indexes are met.
For example, there is a set of sequences of medical images, each sequence consisting of 10 images. The images are first de-noised to reduce noise interference in the images. After denoising, the signal-to-noise ratio of the image is improved by 20dB. Next, contrast adjustment is performed on the image to enhance detail information in the image. After contrast adjustment, the contrast of the image was increased by 30%. The pixel position change in each frame is then detected using a light flow method to determine the pixel motion between two consecutive frames, knowing that 10% of the pixels have moved between the first frame and the second frame. From the pixel motion, a motion pattern is extracted, and abnormal motion including sudden movement or deformation is identified, and 5% of the abnormal motion is detected. And (3) applying an image correction technology to repair the identified deformed part, wherein the deformation of the deformed part is reduced by 30% after the image correction. If faults are detected in the image sequence, connecting the faults by using a cubic spline interpolation method, generating a smooth motion track, and obtaining a continuous motion track through interpolation. According to the image semantic tag, effective features are selected, features irrelevant to the pathological changes are eliminated, key feature details in the sequence are enhanced, and after feature selection and enhancement, the effective features relevant to the pathological changes are obtained. And then, performing label prediction on the preprocessed and enhanced medical image sequence by using a convolutional neural network, judging the label consistency of the whole sequence, and obtaining 90% of label consistency through the neural network. The consistency of the sequences was determined by regression analysis of the differences between the tags, and by regression analysis, 85% consistency of the sequences was obtained. The space-time characteristics are generated by an optical flow method, the motion codes are obtained by using discrete cosine transformation, the continuity of the codes is judged, and the continuity of 95% is obtained by the continuity judgment of the motion codes. The proportion of invalid frames in the sequence is counted and the proportion of available frames is measured, wherein the proportion of invalid frames in the sequence is 5%, and the proportion of available frames is 95%. If the invalid frame proportion is higher than the preset threshold, a new round of data enhancement is performed, and the preset threshold is 10%, so that the new round of data enhancement is required. Finally, key performance index thresholds are set, including the cross-over ratio, peak signal-to-noise ratio and processing frame rate. And (3) continuously iterating the optimization processing until all the set performance indexes are met. Finally, the performance index requirement that the cross ratio is 9, the peak signal-to-noise ratio is 30dB and the processing frame rate is 30fps is achieved.
Step S109, performing device difference recognition and calibration by adopting visual feature extraction according to the medical imaging data and the image metadata, and performing second modality conversion and standardization.
According to different types of medical imaging data and image metadata, a convolutional neural network is adopted to extract visual features in an image and the visual features are associated with equipment types, wherein the image metadata comprises manufacturer, model and scanning settings, and the visual features comprise textures, edges, contrast and shape of the image; comparing the device difference recognition results based on the metadata and the convolutional neural network model, and guiding the subsequent image calibration direction by using the comparison result; performing calibration conversion on the images judged to be from other devices; calculating the signal-to-noise ratio of the converted image, comparing the signal-to-noise ratio with the data of the target equipment, and evaluating the quality of parameter calibration; model training is carried out by adopting a generated countermeasure network according to images of different modes, key characteristics and distribution differences among different modes are learned, data distribution and characteristic layer conversion are carried out by using the generated countermeasure network model, and the image modes comprise CT and MRI; and on the standardized images of the equipment and the mode, the physiological motion model and the equipment difference model are repeatedly used for detection, so that self-optimized image processing circulation is realized, and standardized parameter images with signal to noise ratio larger than a preset threshold value of the signal to noise ratio and clear characteristics are obtained.
For example, there are two different types of CT scanners, type A and type B, respectively. And extracting visual features in the image by using a convolutional neural network, associating the visual features with equipment types, and carrying out equipment difference identification. For the CT image of device a, the extracted visual features include texture, edges, contrast, and shape. Using the image metadata of device a, such as manufacturer, model and scan settings, these features are associated with the resulting texture features of 8, edge features of 6, contrast features of 7 and shape features of 9. Similarly, the same visual features are extracted and correlated for the CT image of device B, resulting in texture features 7, edge features 5, contrast features 6, and shape features 8 for device B. By comparing these features, the difference between device a and device B can be found, the image of device a can be found to have higher texture and contrast features, while the image of device B has higher edge and shape features. Based on these difference recognition results, image calibration can be performed, and the image of the apparatus B is converted into a standardized parameter image of the apparatus a by calibration conversion. In the converted image, the signal to noise ratio can be calculated and compared with the data of device a. If the signal-to-noise ratio of the image of device B after conversion is 20, and the signal-to-noise ratio of the data of device a is 25. By comparing the results, the quality of parameter calibration can be evaluated and optimized to obtain a standardized parameter image with higher signal-to-noise ratio and clear characteristics. For images of different modalities, such as MRI, a modality transition generation network may be used for data distribution and feature level conversion. The MRI image may be converted into a feature distribution of the CT image to achieve standardization of devices and modalities. Finally, the physiological motion model and the equipment difference model can be used for detection, so that self-optimized image processing circulation is realized, and a standardized parameter image with signal to noise ratio larger than a preset threshold and clear characteristics is obtained.
It should be noted that the above list is only a few specific embodiments of the present invention. Obviously, the invention is not limited to the above embodiments, but many variations are possible. All modifications directly derived or suggested to one skilled in the art from the present disclosure should be considered as being within the scope of the present invention.
Claims (10)
1. A method for calibrating the quality of multi-modal medical data, the method comprising:
collecting a multi-device medical multi-mode medical image sequence containing metadata, judging parameter differences of the metadata, extracting characteristics of the image based on a large model Vision Transformer, and identifying and correcting the differences of multi-mode data among different devices;
extracting general features of different medical equipment images according to the multi-mode medical image sequence, comparing the similarity of the general features of the different medical equipment images, transferring the general features to the medical equipment images of a specified type when the similarity is higher than a preset threshold value, performing data cleaning and enhancement, performing parameter normalization before pathological analysis, and acquiring and uploading standardized images to a cloud platform, wherein the general features comprise textures, shapes and color distribution of the images;
According to the time sequence, identifying the pixel motion and structural change of the standardized image, constructing a physiological motion model for motion abnormality identification, and constructing an equipment difference model for structural deformation detection;
matching the detected deformation with the physiological motion model and the equipment difference model, and judging the coincidence degree of the deformation and the physiological motion model; quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image;
according to the characteristic of the image deformation incapable of judging the category, simultaneously using the physiological motion model and the equipment difference model to identify and correct the image deformation incapable of judging the category, and performing iterative optimization and effect evaluation on the processed image;
performing physiological periodic analysis on the processed image sequence, extracting and evaluating dynamic characteristics, performing data adjustment and first mode conversion, and obtaining an enhanced medical image sequence, wherein the dynamic characteristics comprise pixel brightness, pixel color intensity, local movement and local shape change; according to the enhanced medical image sequence, detecting pixel motion, identifying and correcting abnormal motion and deformation conditions, performing image semantic analysis and label prediction, analyzing sequence continuity, and obtaining medical imaging data;
And according to the medical imaging data and the image metadata, performing equipment difference identification and calibration by adopting visual feature extraction, and performing second mode conversion and standardization.
2. The method of claim 1, wherein collecting the sequence of multi-device medical multi-modality medical images containing metadata, determining parameter differences for the metadata, and extracting features of the images themselves based on the large model of Vision Transformer, identifying and correcting differences in multi-modality data between different devices, comprises:
acquiring a multi-mode image sequence through different medical imaging devices, wherein the multi-mode image sequence comprises metadata of each sequence, the metadata comprises a device type and imaging parameters, and the imaging parameters comprise resolution and contrast; comparing metadata parameters of different image sequences, and if the parameter difference in the metadata exceeds a parameter preset threshold, confirming that the difference between devices exists; according to medical images acquired by multiple devices, a Vision transducer-based model is applied, and quality features of the images are extracted, wherein the quality features comprise resolution, contrast and noise level; comparing the quality characteristic representations of the extracted images, and if the quality characteristic difference of the extracted images exceeds a quality preset threshold, determining that data difference exists between different devices; under the condition that the device difference cannot be clearly distinguished, combining the metadata and the image quality characteristics to form a comprehensive characteristic set; when the size of the image data set is lower than the preset number, the data diversity is increased through the technical means of image rotation, scaling and overturning; extracting structural features of the image, wherein the structural features comprise length, width proportion and edge features; if the structural features of the images have mutation in the time sequence, judging that potential deformation exists; and performing image processing on the image area with the deformation degree exceeding the deformation preset threshold, wherein the image processing comprises clipping the deformation area and interpolating the complement structure.
3. The method according to claim 1, wherein the steps of extracting common features of different medical device images according to the multi-modal medical image sequence, comparing similarities of the common features of the different medical device images, and when the similarities are higher than a preset threshold, migrating the common features into a medical device image of a specified type, performing data cleaning and enhancement, performing parameter normalization before pathology analysis, acquiring and uploading standardized images to a cloud platform, wherein the common features include textures, shapes and color distribution of the images, and the method comprises:
according to a multi-mode image sequence obtained from different medical imaging devices, model training is carried out by using a convolutional neural network algorithm, general features are extracted, and general feature similarity of images of different devices is compared, wherein the general features comprise textures, shapes and color distribution of the images; if the similarity is higher than a similarity preset threshold, directly migrating the general feature into the medical image of the appointed type; if the general features are not suitable for the medical images of the appointed type, acquiring multi-equipment images and labels thereof, extracting equipment invariant features by using a shared encoder, and predicting the labels by combining with an equipment exclusive decoder; continuously detecting and analyzing abnormal conditions predicted by the image labels, if the proportion of the abnormal samples is larger than a preset threshold value of the proportion of the abnormal samples, judging that interference difference parameters exist, performing data cleaning or enhancement processing, reducing differences among data distribution by adopting normalization processing, and generating new samples to supplement a data set; performing parameter normalization on images acquired by different devices, ensuring that all image data have consistent data quality and feature distribution before pathology analysis is performed, and using the normalized image data for a main task of pathology prediction; the median filtering algorithm is applied, so that the data precision is reduced, key characteristics are reserved, redundant parameter differences are eliminated, and standardized and good-quality image characterization is obtained; and uploading the standardized image to a cloud platform to realize sharing and cooperation among a plurality of devices.
4. A method according to claim 3, wherein said identifying pixel motion and structural changes of the normalized image from a time series, constructing a physiological motion model for motion anomaly identification, and constructing a device difference model for structural deformation detection, comprises:
extracting optical flow and key point descriptors of a time sequence on a standardized image; analyzing the motion of pixel points in an image sequence by an optical flow method, and estimating the motion mode of an image block; if the optical flow vector is discontinuous or suddenly changed in the sequence, judging that the motion is abnormal; according to the standardized medical image sequence, a physiological motion model is constructed by applying long-term motion constraint of a model learning time sequence based on a transducer; comparing the current image motion with the model constraint, judging whether the current image motion accords with the model constraint, and if not, judging that the current image motion is deformed; creating a human motion knowledge graph containing skeletal node and joint connection information; mapping a new sample to the human motion knowledge graph, comparing topological structure changes, and detecting structural changes; according to the human motion knowledge graph, building a device difference model by using characteristic representations of nodes and edges in the graph rolling network learning graph, analyzing the representation of a new sample on the graph, and comparing the differences between the characteristics of the nodes and edges and the known normal structure mode, wherein the characteristics of the nodes and edges comprise the positions, the sizes and the shapes of the nodes; if the characteristics of the new sample and the known distribution have obvious drift, the deformation is caused by the difference of the parameters of the equipment; if the new sample detects motion and structural abnormality at the same time, carrying out association analysis to judge an abnormality source; if the motion abnormality is associated with the time series model, determining a physiological change; if the structural abnormality is associated with the equipment difference model, judging that the parameter is deviated; according to the standardized image sequence data, long-term motion mode and structural change are identified by utilizing long-term memory network learning time-space sequence characteristics, association of two factors including time and topology is fused, accuracy of deformation detection and abnormal source analysis is improved, and the time-space sequence characteristics comprise motion mode and structural change.
5. The method of claim 1, wherein matching the detected deformation to the physiological motion model and the device difference model, and determining the degree of compliance of the deformation to both, comprises:
calculating the Jaccard distance between the deformation and the standard physiological motion mode, wherein the smaller the distance is, the more the deformation accords with the physiological motion mode; calculating Euclidean distance between the deformation and the topological structure difference of the designated equipment map, wherein the smaller the distance is, the more likely the deformation is from the equipment difference; the output of the physiological motion model and the device difference model is processed by using a normalization processing method, the dimension influence is eliminated, and the output of the physiological motion model and the device difference model is ensured to be in the same numerical range; model training is carried out by using a decision tree algorithm according to the matching measurement of the deformation characteristics, the physiological motion model and the equipment difference model, a potential source prediction model of the deformation is constructed, the potential source of the deformation is judged, and the matching measurement is Jaccard distance or Euclidean distance; analyzing the linear correlation of the deformation with the output of the physiological motion model and the device difference model by using the Pearson correlation coefficient; if the linear correlation between the deformation characteristic and the motion model is greater than the linear correlation between the deformation characteristic and the equipment difference model, judging that the deformation is caused by physiological motion; if the linear correlation between the deformation characteristic and the equipment difference model is greater than the linear correlation between the deformation characteristic and the motion model, judging that the deformation is caused by the difference of the equipment parameters; according to the characteristics extracted from the multi-mode medical image, including optical flow, key point descriptors and structural changes in the image sequence, model training is carried out by adopting a support vector machine algorithm, and the similarity between the detected deformation and the learned deformation mode is determined; judging the deformation mode category of the detected deformation according to the similarity between the detected deformation and the mode; the detection result with insufficient confidence is marked as a category which cannot be judged; taking the judgment of the physiological motion model and the equipment difference model about deformation as a new sample, training data, and improving the judgment capability of the model; and summarizing the detection results through repeated iterative learning, and judging the attribution of deformation.
6. The method as recited in claim 5, further comprising: quantifying nonlinear deformation evaluation criteria according to deformation characteristics in the multi-modal medical image;
the quantitative nonlinear deformation evaluation standard comprises the following specific steps of: acquiring medical image data of different modalities containing complex deformation features, wherein the complex deformation comprises deformation caused by tumor growth or organ movement; preliminary analysis is carried out on medical images of different modes containing complex deformation characteristics, and the general modes of the deformation characteristics are determined, wherein the deformation characteristics comprise intensity change and shape distortion; based on the result of the preliminary analysis, formula D is set nl (x, y) =α·Δi (x, y) +β·Δg (x, y), where Δi (x, y) represents a change in pixel intensity, Δg (x, y) represents a change in local geometry, and α and β are coefficients that regulate the importance of both; obtaining delta I (x, y) by calculating pixel intensity differences between adjacent frames, determining delta G (x, y) by evaluating shape changes of the local area; according to the historical data, determining optimal values of coefficients alpha and beta of images of different image modes, wherein the image modes comprise CT and MRI, so as to reflect the characteristics of different imaging technologies; applying a defined formula D on a sequence of medical images of different modalities nl (x, y) generating a non-linear deformation score for each pixel or region, identifying non-linear deformation regions in the image; analyzing the distribution of nonlinear deformation scores, and identifying the area with the highest deformation degree in the image; testing and evaluating the validity of the formula by using the image data of the actual clinical case, and ensuring the applicability of the formula in multi-modal medical image analysis; and adjusting parameters in the formula according to the test and evaluation results.
7. The method of claim 6, wherein the identifying and correcting the image distortion of the undetermined class based on the characteristic of the image distortion of the undetermined class using the physiological motion model and the device difference model, and performing iterative optimization and effect evaluation on the processed image, comprises:
aiming at complex image deformation which cannot be clearly distinguished from attribution, detecting by using the physiological motion model and the equipment difference model; acquiring the output of the physiological motion model and the device difference model, fusing the two outputs by using an attention mechanism, calculating the association weight of the output of each model, and evaluating the interpretation degree of the two models on the current deformation; if the weight of one model is higher than that of the other model, judging that the interpretation importance of the prediction result of the model with high weight on deformation is higher than that of the other model; if the weight difference value of the physiological motion model and the equipment difference model is smaller than a weight preset threshold value, the two model prediction results are equally important, and a detection result is obtained through joint decision; if the physiological motion model predicts that physiological motion abnormality exists, correcting deformation by using a data interpolation and motion regularization algorithm; if the equipment parameter deviation exists in the equipment difference model prediction, adopting an image enhancement and mode conversion method to adjust the image; for complex image deformation which cannot be clearly attributed, simultaneously applying a parameter adjustment method of physiological motion abnormality and equipment parameter deviation to perform joint optimization processing on the image; detecting again after treatment, extracting deformation characteristics, comparing with the deformation characteristics before optimization, and evaluating correction effects; after multiple iterations, selecting a model with processing precision and reasoning time being larger than preset threshold values, and applying the model to the deformation condition which cannot be clearly attributed to, so that the high efficiency of detection and optimization is ensured; further comprises: estimating and quantifying deformation characteristics between images according to the intensity and geometric variation of the time series medical images;
The method for estimating and quantifying deformation characteristics among images according to the intensity and geometric variation of the time series medical images specifically comprises the following steps: acquiring multi-modal medical images from different points in time showing potential deformation characteristics, determining changes in the imagesA pattern of variations including variations in pixel intensity and geometry; for image I 1 And I 2 Using the formula for each pixel position (x, y)Calculating absolute values of intensity differences of corresponding pixel points of the two images, and calculating average values of the intensity differences of all pixels to obtain an overall intensity difference metric delta I (I 1 ,I 2 ) N is the total number of pixels in the image; for each image, extracting boundaries of key structural elements by using a Canny algorithm; for each structure element, the formula Hausdorff (S 1 ,S 2 )＝max{sup s1∈S1 inf s2∈S2 d(s 1 ,s 2 ),sup s2∈S2 inf s1∈S1 d(s 1 ,s 2 ) Calculating Hausdorff distance between boundaries, wherein S 1 And S is 2 Is a set of boundary points of structural elements extracted from two images by the Canny algorithm, d (s 1 ,s 2 ) Is the point s 1 Sum s 2 Euclidean distance between them; using the formulaCalculating the degree of geometry change ΔG (I) 1 ,I 2 )，S 1k And S is 2k Respectively represent image I 1 And I 2 The boundary of corresponding structural elements in (a), wherein the structural elements comprise organs and tumors, and M is the number of the structural elements; definition formula F ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Delta t quantization of two time points t 1 And t 2 Medical image I in between 1 And I 2 Wherein Δt=t 2 -t 1 Wherein I 1 And I 2 Images at two different time points respectively, Δt being the time difference; applying a formula according to images in different modes and different time sequences, and adjusting values of gamma and delta according to the reaction condition of the formula to the change; in actual clinical casesThe image data of the example is applied with formula F ts (I 1 ,I 2 ,Δt)＝γ·ΔI(I 1 ,I 2 )+δ·ΔG(I 1 ,I 2 ) Δt, evaluate its behavior in actual situations, and adjust formulas and algorithms according to the evaluation result.
8. The method of claim 1, wherein the performing a physiological periodic analysis on the processed image sequence, performing extraction and evaluation of dynamic features, performing data adjustment and first modality conversion, obtaining an enhanced medical image sequence, the dynamic features including pixel brightness, pixel color intensity, local movement, local shape change, comprises:
performing physiological periodic analysis on the processed image sequence, acquiring time domain statistical characteristics under different scales by using wavelet transformation, and acquiring main frequency components of the image sequence by using fast Fourier transformation; if the main frequency component is missing, the image sequence is indicated to not reflect the complete physiological process; judging whether the dynamic characteristics of the image sequence accord with standard normal distribution or not by utilizing a Gaussian mixture model according to the dynamic characteristics extracted from the image sequence, wherein the dynamic characteristics extracted from the image sequence comprise time variation of pixel intensity and local motion characteristics; if the dynamic characteristics of the image sequence accord with single normal distribution, the image data are consistent, and no obvious abnormality exists; if the data accords with a plurality of normal distributions, different types of change modes are indicated; if the analysis result of the Gaussian mixture model shows that the dynamic characteristics do not accord with the standard distribution or have abnormality, the distribution shape is adjusted by using a data enhancement method, and the regularization characteristics are regulated by using the data enhancement method, wherein the data enhancement method comprises statistic attribute modification, affine transformation, elastic deformation, random noise addition and modal conversion; for samples with insufficient characteristics, generating diversified data by using a variable self-encoder according to the data with complete main frequency components; converting the image mode from CT to MRI, and increasing dynamic characterization of the physiological state under different modes; carrying out time domain and frequency domain feature extraction again on the processed image sequence, and judging whether periodic signal components are added; using a structural similarity index and a peak signal-to-noise ratio to evaluate the consistency of the image quality before and after processing, and ensuring that the generated data reach the standard in authenticity and effectiveness; further comprises: performing customized enhancement and quality improvement treatment according to the image region characteristics;
The customized enhancement and quality improvement processing is performed according to the image area characteristics, and specifically includes: analyzing the image by using a Canny edge detection algorithm, and identifying areas with contrast lower than a preset threshold; in the area with the contrast lower than the preset threshold value, sharpening enhancement and contrast enhancement are carried out by using a local histogram equalization technology; identifying a fuzzy or noise area in the image by using a Gaussian mixture model, improving the definition of the image by applying a Wiener filtering algorithm to the fuzzy area, removing noise in the image by applying a median filtering algorithm to the noise area, and enhancing and recovering details in the fuzzy area by using a super-resolution technology; for the region with the loss of details or the quality lower than a preset threshold value in the image, reconstructing by adopting a bicubic interpolation technology; applying a Wiener filtering algorithm and a median filtering algorithm to the reconstruction region, eliminating noise possibly introduced, and keeping the natural appearance and consistency of the image; the method of histogram equalization and color correction is combined to unify and optimize illumination, contrast and color balance of the whole image sequence; the image processing model is customized and optimized based on the characteristics of the CT and MRI images produced by the different devices.
9. The method of claim 8, wherein said performing pixel motion detection, identifying and correcting abnormal motion and deformation conditions, performing image semantic analysis and label prediction, analyzing sequence consistency, and acquiring medical imaging data based on said enhanced medical image sequence, comprises:
preprocessing the enhanced image sequence, including denoising and contrast adjustment; detecting pixel position change in each frame by using a light flow method, and determining pixel motion between two continuous frames; extracting a motion mode from the pixel motion, and identifying abnormal motion in an image sequence, including burst movement or deformation; identifying new or residual deformation conditions according to the optical flow result, and determining a movement area which is inconsistent with the expected physiological movement mode; applying a histogram equalization technique and a fourier transform to the identified deformed portion to correct or minimize the deformed portion; if the image sequence interruption is detected, connecting faults by using a cubic spline interpolation method to generate a smooth motion track; selecting effective features and eliminating features irrelevant to lesions according to the image semantic tags; according to the preprocessed and enhanced medical image sequence, predicting the label of each image by using a convolutional neural network, and judging the label consistency of the whole sequence; analyzing the difference between the labels by a regression method, and determining the consistency of the sequences; generating space-time characteristics by an optical flow method, acquiring motion codes by using discrete cosine transform, and judging the continuity of the codes; counting the proportion of invalid frames in the sequence, measuring the proportion of available frames, and if the proportion of the invalid frames is higher than a proportion preset threshold value, performing a new round of data enhancement; setting a key performance index threshold, wherein the key performance index comprises an intersection ratio, a peak signal-to-noise ratio and a processing frame rate; and continuously iterating the optimization processing until all the set performance indexes are met.
10. The method of any of claims 1-9, wherein performing device discrepancy identification and calibration based on the medical imaging data and image metadata using visual feature extraction, performing a second modality conversion and normalization, comprises:
according to different types of medical imaging data and image metadata, a convolutional neural network is adopted to extract visual features in an image and the visual features are associated with equipment types, wherein the image metadata comprises manufacturer, model and scanning settings, and the visual features comprise textures, edges, contrast and shape of the image; comparing the device difference recognition results based on the metadata and the convolutional neural network model, and guiding the subsequent image calibration direction by using the comparison result; performing calibration conversion on the images judged to be from other devices; calculating the signal-to-noise ratio of the converted image, comparing the signal-to-noise ratio with the data of the target equipment, and evaluating the quality of parameter calibration; model training is carried out by adopting a generated countermeasure network according to images of different modes, key characteristics and distribution differences among different modes are learned, data distribution and characteristic layer conversion are carried out by using the generated countermeasure network model, and the image modes comprise CT and MRI; and on the standardized images of the equipment and the mode, the physiological motion model and the equipment difference model are repeatedly used for detection, so that self-optimized image processing circulation is realized, and standardized parameter images with signal to noise ratio larger than a preset threshold value of the signal to noise ratio and clear characteristics are obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311679486.1A CN117671463B (en) | 2023-12-07 | 2023-12-07 | Multi-mode medical data quality calibration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311679486.1A CN117671463B (en) | 2023-12-07 | 2023-12-07 | Multi-mode medical data quality calibration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117671463A true CN117671463A (en) | 2024-03-08 |
| CN117671463B CN117671463B (en) | 2024-06-21 |
Family
ID=90069494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311679486.1A Active CN117671463B (en) | 2023-12-07 | 2023-12-07 | Multi-mode medical data quality calibration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117671463B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120155736A1 (en) * | 2005-07-01 | 2012-06-21 | Siemens Medical Solutions Usa, Inc. | Extension of Truncated CT Images For Use With Emission Tomography In Multimodality Medical Images |
| CN107403457A (en) * | 2017-07-28 | 2017-11-28 | 上海联影医疗科技有限公司 | Medical imaging procedure, equipment and multi-modal medical imaging procedure |
| US20180308245A1 (en) * | 2016-11-02 | 2018-10-25 | Shenzhen United Imaging Healthcare Co., Ltd. | System and method for registering multi-modality images |
| CN113962953A (en) * | 2021-10-20 | 2022-01-21 | 上海联影医疗科技股份有限公司 | Image processing method and system |
| CN114387364A (en) * | 2021-12-28 | 2022-04-22 | 江苏赛诺格兰医疗科技有限公司 | Linear attenuation coefficient acquisition method and reconstruction method for PET image reconstruction |
| US20230018833A1 (en) * | 2021-07-19 | 2023-01-19 | GE Precision Healthcare LLC | Generating multimodal training data cohorts tailored to specific clinical machine learning (ml) model inferencing tasks |
| CN116883281A (en) * | 2023-07-14 | 2023-10-13 | 首都医科大学附属北京潞河医院 | Image imaging image quality enhancement system and method for image guided radiotherapy |
| CN117115045A (en) * | 2023-10-24 | 2023-11-24 | 吉林大学 | Method for improving medical image data quality based on Internet generation type artificial intelligence |
| CN117115624A (en) * | 2023-10-25 | 2023-11-24 | 盛视科技股份有限公司 | Palm vein image quality assessment method |
-
2023
- 2023-12-07 CN CN202311679486.1A patent/CN117671463B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120155736A1 (en) * | 2005-07-01 | 2012-06-21 | Siemens Medical Solutions Usa, Inc. | Extension of Truncated CT Images For Use With Emission Tomography In Multimodality Medical Images |
| US20180308245A1 (en) * | 2016-11-02 | 2018-10-25 | Shenzhen United Imaging Healthcare Co., Ltd. | System and method for registering multi-modality images |
| CN107403457A (en) * | 2017-07-28 | 2017-11-28 | 上海联影医疗科技有限公司 | Medical imaging procedure, equipment and multi-modal medical imaging procedure |
| US20230018833A1 (en) * | 2021-07-19 | 2023-01-19 | GE Precision Healthcare LLC | Generating multimodal training data cohorts tailored to specific clinical machine learning (ml) model inferencing tasks |
| CN113962953A (en) * | 2021-10-20 | 2022-01-21 | 上海联影医疗科技股份有限公司 | Image processing method and system |
| CN114387364A (en) * | 2021-12-28 | 2022-04-22 | 江苏赛诺格兰医疗科技有限公司 | Linear attenuation coefficient acquisition method and reconstruction method for PET image reconstruction |
| CN116883281A (en) * | 2023-07-14 | 2023-10-13 | 首都医科大学附属北京潞河医院 | Image imaging image quality enhancement system and method for image guided radiotherapy |
| CN117115045A (en) * | 2023-10-24 | 2023-11-24 | 吉林大学 | Method for improving medical image data quality based on Internet generation type artificial intelligence |
| CN117115624A (en) * | 2023-10-25 | 2023-11-24 | 盛视科技股份有限公司 | Palm vein image quality assessment method |
Non-Patent Citations (2)
| Title |
|---|
| MANBER, R, ET AL.: "Practical PET Respiratory Motion Correction in Clinical PET/MR", 《JOURNAL OF NUCLEAR MEDICINE》, vol. 56, no. 6, 1 January 2015 (2015-01-01) * |
| 王振松;刘晓云;陈武凡;: "特征点对引导的医学图像弹性配准方法", 《电子科技大学学报》, no. 3, 30 May 2012 (2012-05-30) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117671463B (en) | 2024-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5695648B2 (en) | Medical image processing method and apparatus | |
| Stegmann et al. | Unsupervised motion-compensation of multi-slice cardiac perfusion MRI | |
| JP5893623B2 (en) | Anomaly detection method and system in data set | |
| NL1024314C2 (en) | Integrated image registration for cardiac magnetic resonance blood flow data. | |
| WO2017096407A1 (en) | Systems and methods for analyzing pathologies utilizing quantitative imaging | |
| US8849003B2 (en) | Methods, apparatus and articles of manufacture to process cardiac images to detect heart motion abnormalities | |
| US20100266188A1 (en) | Chest x-ray registration, subtraction and display | |
| US9111174B2 (en) | Machine learnng techniques for pectoral muscle equalization and segmentation in digital mammograms | |
| US20210068676A1 (en) | Method and system for automatically generating and analyzing fully quantitative pixel-wise myocardial blood flow and myocardial perfusion reserve maps to detect ischemic heart disease using cardiac perfusion magnetic resonance imaging | |
| Liu et al. | Cardiac magnetic resonance image segmentation based on convolutional neural network | |
| KR102178803B1 (en) | System and method for assisting chest medical images reading | |
| CN115861656A (en) | Method, apparatus and system for automatically processing medical images to output an alert | |
| Ramana | Alzheimer disease detection and classification on magnetic resonance imaging (MRI) brain images using improved expectation maximization (IEM) and convolutional neural network (CNN) | |
| CN113197558A (en) | Heart rate and respiratory rate detection method and system and computer storage medium | |
| CN117671463B (en) | Multi-mode medical data quality calibration method | |
| CN115810032A (en) | Image registration method and device, and medical image motion correction method and system | |
| Rougon et al. | Building and using a statistical 3D motion atlas for analyzing myocardial contraction in MRI | |
| Dufumier et al. | Exploring the potential of representation and transfer learning for anatomical neuroimaging: application to psychiatry | |
| Osadebey et al. | Brain MRI Intensity Inhomogeneity Correction Using Region of Interest, Anatomic Structural Map, and Outlier Detection | |
| CN109662689B (en) | Health early warning system based on electrocardiogram for hospital | |
| de Bruijne et al. | Automated segmentation of abdominal aortic aneurysms in multi-spectral MR images | |
| Wang et al. | Reinforcement learning for context aware segmentation | |
| CN111445447A (en) | CT image anomaly detection method and device | |
| US20240144469A1 (en) | Systems and methods for automatic cardiac image analysis | |
| CN113223104B (en) | Cardiac MR image interpolation method and system based on causal relationship |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |