CN117541493A - Three-dimensional feature medical image fusion method based on improved fractional order cumulant - Google Patents
Three-dimensional feature medical image fusion method based on improved fractional order cumulant Download PDFInfo
- Publication number
- CN117541493A CN117541493A CN202311641906.7A CN202311641906A CN117541493A CN 117541493 A CN117541493 A CN 117541493A CN 202311641906 A CN202311641906 A CN 202311641906A CN 117541493 A CN117541493 A CN 117541493A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- mri
- registration
- fractional order
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 10
- 230000004927 fusion Effects 0.000 claims abstract description 19
- 230000000694 effects Effects 0.000 claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 14
- 230000001186 cumulative effect Effects 0.000 claims description 14
- 230000009466 transformation Effects 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 10
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000009825 accumulation Methods 0.000 claims description 3
- 230000001174 ascending effect Effects 0.000 claims description 3
- 210000000887 face Anatomy 0.000 claims description 3
- 238000012417 linear regression Methods 0.000 claims description 3
- 210000004373 mandible Anatomy 0.000 claims description 3
- 210000000537 nasal bone Anatomy 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 210000004279 orbit Anatomy 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 claims description 3
- 210000004556 brain Anatomy 0.000 abstract description 8
- 238000012545 processing Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000002059 diagnostic imaging Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/094—Adversarial learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention relates to a three-dimensional feature medical image fusion method based on improved fractional order cumulant, belonging to the technical field of medical image registration in image processing, which comprises the steps of using super-resolution restoration to improve the resolution of an MR image so that the data is consistent with the resolution of a given CT image; three-dimensionally reconstructing CT and MRI images based on an MC algorithm, and performing coarse registration; calculating a feature image using the improved fractional order cumulant; and carrying out fine registration on the CT and MRI images based on a 3DSIFT algorithm to finish three-dimensional registration fusion of the brain medical images. According to the invention, the improved fractional order cumulant is combined with the traditional feature-based registration method, so that the effect and the precision of the registration fusion of the brain CT and MRI images are optimized, the problem of the registration fusion of the fine three-dimensional images in small sample medical images can be solved, and the error can reach the millimeter level.
Description
Technical Field
The invention belongs to the technical field of medical image registration in image processing, and particularly relates to a three-dimensional feature medical image fusion method based on improved fractional order cumulant.
Background
Medical image processing has become an area of intense research with the continued development of computer and medical imaging technology. Medical imaging techniques can assist doctors in making preliminary diagnoses of diseases, and computer technology makes it possible to store and process vast amounts of data. Medical image processing techniques are those that process medical images by means of a computer, combining the advantages of both techniques. The manual diagnosis is mostly carried out by combining with medical knowledge of the user, so that only significant information in the image can be focused, and some details are ignored. In addition, manual diagnosis requires a diagnostician to have very strong medical knowledge and certain practices, practical experiences, etc., which is extremely lengthy. Even if the manual diagnosis accuracy is high, the time cost is high, and a large number of images cannot be diagnosed in a short time, so that the computer automation is needed to assist, and the accuracy can be ensured. Thus, research in this area is of practical value, and its development is also immeasurable as a contribution to society.
In medical image processing, it is often necessary to perform contrast analysis in combination with images of different modalities to reach more accurate conclusions. Due to the influence of the acquisition environment, the posture of the patient and other factors, in practical operation, there is no way to ensure that two images are in one-to-one correspondence in space.
The image registration technology is initially applied to the aspects of aviation navigation and precise guidance, and then the technology is gradually applied to various fields, and relates to remote sensing, aerial photography and the like. The application of image registration techniques in the field of medical images is a research hotspot in recent years.
Image registration refers to the task of matching more than two images, such as two images taken at different times, instruments, locations. The medical image registration is to search a geometric transformation mode so that the same spatial points in two medical images can be in one-to-one correspondence. Image registration is, in colloquial terms, an operation of aligning two different images. Only after registration, the comparison of the two images is of practical significance. Image fusion is a subsequent application based on image registration.
Different medical imaging devices acquire different medical image characteristics, MRI can clearly reflect the shape of soft tissues, and CT images are mainly bone information. The image fusion reflecting different contents can better display the comprehensive information of the patient, and is beneficial to doctors to make more accurate diagnosis on the patient. The image may have rotational, displacement and scaling relationships due to the different devices of the patient's medical image acquisition, as well as the patient's movement. Therefore, registration of the different images must be done before fusion can take place.
Disclosure of Invention
In view of the problems that the existing feature-based method lacks image registration fusion precision of brain CT and MRI, the invention provides a three-dimensional feature medical image fusion method based on improved fractional order cumulant. The method is mainly suitable for the feature-based medical image registration fusion method with fewer medical image samples. The invention improves the fractional cumulative quantity based on the fractional cumulative quantity, provides an improved fractional cumulative quantity for processing medical images, and designs a feature extraction method based on a 3DSIFT algorithm.
A three-dimensional feature medical image fusion method based on improved fractional order cumulants, comprising the steps of:
1) Performing super-resolution restoration on MRI, comprising the following steps:
1.1, improving the image resolution of an MRI slice based on an SRGAN super-resolution network to obtain an MR slice and a CT slice after super-resolution restoration, so that the image resolution of the MRI slice is consistent with that of the CT slice;
2) Coarse registration of CT and MRI images, comprising the steps of:
2.1 respectively carrying out three-dimensional reconstruction on the MR slice and the CT slice which are obtained after super-resolution restoration and obtained in the step 1.1 based on an MC algorithm;
2.2 based on medical priori knowledge, by observing the positions of mandible, nasal bone and eye socket in the CT and MRI three-dimensional reconstruction effect images, taking the CT three-dimensional reconstruction effect image as a reference image, moving the MRI three-dimensional reconstruction effect image to the corresponding position of the T three-dimensional reconstruction effect image, and expressing the positions in the same coordinate system;
3) Calculating a complex feature map based on the improved fractional order cumulants, comprising the steps of:
3.1, based on a fractional calculus principle, fractional moment estimation is carried out on pixel points of CT and MRI images to obtain moment characteristic images, wherein the moment characteristic images are represented by the following formula:
wherein: τ x 、τ y And τ z Data point intervals in the x, y and z directions are respectively represented;
the moment signature in three-dimensional form is of the formula:
wherein: n is n x 、n y And n z Respectively representing accumulation points; k represents the number of points used in each iteration in the calculation; b represents a fraction order;
3.2 calculating fractional order cumulant based on fractional order moment estimation to obtain a cumulant characteristic image, wherein the cumulant characteristic image is represented by the following formula:
wherein: c (C) 4b Fractional order cumulants for the position sought; m is m kp Assigning fractional moment estimation values in a range for the pixel points; the fractional order cumulative amount can also be expressed as follows:
wherein: j is in complex numberω represents different angular frequencies in the cumulative amount generation function;
3.3 verifying the robustness of the fractional cumulative spectrum to noise as follows:
wherein: i represents image truth value data; n represents noise at the corresponding position;
4) The feature extraction is performed based on a 3DSIFT algorithm, and the method comprises the following steps:
4.1 constructing a scale space from the complex feature map result in the step 3), calculating a Gaussian difference function on a Gaussian pyramid, and searching a local extremum of the image on the scale and space, namely a possible key point;
4.2 tracking the general direction of the key points based on the correlation of gradient components, namely, the structure tensor, positioning the key points, and representing the local direction of the key points by using the structure tensor K, wherein the formula is as follows;
wherein:is the gradient of image I at location x; w (x) is a gaussian window centered on the keypoint;
4.3, arranging the eigenvalues of the structure tensors in ascending order, and removing unreliable key points when the following formula is satisfied;
wherein: lambda (lambda) i The i characteristic value of K, and beta is a constant parameter;
4.4, calculating the angle between the image gradient and the feature vector, and removing unreliable key points when the following formula is satisfied;
min i |cos(θ i )|<γ；
wherein: θ is the angle between the image gradient and the feature vector, and γ is a constant parameter;
4.5, for the detected key points, acquiring an icosahedron region of the key points, calculating the gradient size and the gradient direction in the icosahedron region, and interpolating vectors of 20 faces to 12 vertexes to obtain 12bin;
4.6 taking a key point as the center, constructing a spherical three-dimensional image with radius of 2σ, wherein σ is a scale constant in 4.1, dividing the spherical window into 4 3 For each subregion, a separate gradient histogram is calculated, each histogram having 12 vertices for a total of 4 3 X 12 = 768 components;
4.7 calculating a weighted value of each voxel by using a Gaussian function with a scale sigma based on the distance from the voxel to the key point by using a Gaussian window;
4.8 assigning the contribution of each voxel by tri-linear interpolation between the barycentric coordinates between its three vertices of the intersecting triangle and the centers of the eight sub-areas surrounding the voxel in the cube;
4.9 assuming that the location of the key point is k, the sub-region is centered on y, (λ1, λ2, λ3) is the centroid coordinate of the intersection of the gradient ray and the icosahedral plane, the increment value of the voxel x to the bin corresponding to λi is as follows:
wherein: the voxel coordinates are x; the exponential term is a gaussian window; the multiplier term is the tri-linear interpolation weight of y;
4.10 descriptor is l 2 Normalizing the result of the normalization after the constant threshold stage;
5) Registration fusion of CT and MRI based on 3DSIFT feature operator includes the following steps:
5.1 extracting key points and matching the key points in a pair of images, registering the images by utilizing affine transformation, wherein the affine transformation is as follows:
wherein:for a given coordinate; />Is a parameter;
5.2 fitting affine transformation by linear regression;
5.3 rejecting outliers based on RANSAC for false matches;
and 5.4, obtaining a registration fusion result through least square fitting, wherein the error unit is in millimeter level.
The invention includes using super-resolution restoration to increase the resolution of the MR image such that the data is consistent with a given CT image resolution; three-dimensionally reconstructing CT and MRI images based on an MC algorithm, and performing coarse registration; calculating a complex feature map through the proposed improved fractional order cumulant; the CT and MRI images are fine registered based on the 3DSIFT algorithm. The invention can optimize the registering and fusing effect and precision of the brain CT and MRI images.
Drawings
FIG. 1 is a flow chart of a method of three-dimensional feature medical image fusion based on improved fractional order cumulants;
FIG. 2 is a schematic diagram of a super resolution restored SRGAN network generation network;
FIG. 3 is a schematic diagram of a residual network of a super resolution restoration SRGAN network;
FIG. 4 is a schematic diagram of a CT three-dimensional reconstruction;
FIG. 5 is a schematic representation of an MRI three-dimensional reconstruction;
FIG. 6 is a schematic illustration of CT and MRI coarse registration;
FIG. 7 is an icosahedron diagram constructed from keypoints;
FIG. 8 is a schematic diagram of gradient vectors accumulating from triangle facets to vertices;
FIG. 9 is a view of dividing a spherical window into 4 3 A plan view of an array of cube subregions;
FIG. 10 is a graph of a 3D SIFT description of a set of CT and MRI;
fig. 11 is a graph of the results of registration fusion.
Detailed Description
The following description of the embodiments of the invention is presented in conjunction with the accompanying drawings to provide a better understanding of the invention to those skilled in the art.
As shown in fig. 1, the three-dimensional feature medical image fusion method based on the improved fractional cumulative quantity generally comprises super-resolution restoration, coarse registration under the same coordinate system after three-dimensional reconstruction, calculation of a complex feature map through the improved fractional cumulative quantity, and registration fusion of CT and MRI images of a brain based on a 3D SIFT algorithm.
A three-dimensional feature medical image fusion method based on improved fractional order cumulants comprises the following steps:
1) Performing super-resolution restoration on MRI, comprising the following steps:
1.1, improving the image resolution of an MRI slice based on an SRGAN super-resolution network to obtain an MR slice and a CT slice after super-resolution restoration, keeping the image resolution of the MRI slice consistent with that of the CT slice, wherein a generating network of the SRGAN network is shown in figure 2, and a residual network of the SRGAN network is shown in figure 3;
2) Coarse registration of CT and MRI images, comprising the steps of:
2.1 respectively carrying out three-dimensional reconstruction on the MR slice and the CT slice obtained after super-resolution restoration in the step 1.1 based on an MC algorithm, wherein a CT three-dimensional reconstruction result diagram is shown in figure 4, and a CT three-dimensional reconstruction result diagram is shown in figure 5;
2.2 based on medical priori knowledge, by observing the positions of mandible, nasal bone and eye socket in the CT and MRI three-dimensional reconstruction effect graphs, taking the CT three-dimensional reconstruction effect graph as a reference graph, moving the MRI three-dimensional reconstruction effect graph to the corresponding position of the T three-dimensional reconstruction effect graph, and expressing the positions in the same coordinate system, as shown in figure 6;
3) Calculating a complex feature map based on the improved fractional order cumulants, comprising the steps of:
3.1, based on a fractional calculus principle, fractional moment estimation is carried out on pixel points of CT and MRI images to obtain moment characteristic images, wherein the moment characteristic images are represented by the following formula:
wherein: τ x 、τ y And τ z Data point intervals in the x, y and z directions are respectively represented;
the moment signature in three-dimensional form is of the formula:
wherein: n is n x 、n y And n z Respectively representing accumulation points; k represents the number of points used in each iteration in the calculation; b represents a fraction order;
3.2 calculating fractional order cumulant based on fractional order moment estimation to obtain a cumulant characteristic image, wherein the cumulant characteristic image is represented by the following formula:
wherein: c (C) 4b Fractional order cumulating for the position soughtThe product quantity; m is m kp Assigning fractional moment estimation values in a range for the pixel points; the fractional order cumulative amount can also be expressed as follows:
wherein: j is in complex numberω represents different angular frequencies in the cumulative amount generation function;
3.3 verifying the robustness of the fractional cumulative spectrum to noise as follows:
wherein: i represents image truth value data; n represents noise at the corresponding position;
4) The feature extraction is performed based on a 3DSIFT algorithm, and the method comprises the following steps:
4.1 constructing a scale space from the complex feature map result in the step 3), calculating a Gaussian difference function on a Gaussian pyramid, and searching a local extremum of the image on the scale and space, namely a possible key point;
4.2 tracking the general direction of the key points based on the correlation of gradient components, namely, the structure tensor, positioning the key points, and representing the local direction of the key points by using the structure tensor K, wherein the formula is as follows;
wherein:is the gradient of image I at location x; w (x) is a gaussian window centered on the keypoint;
4.3, arranging the eigenvalues of the structure tensors in ascending order, and removing unreliable key points when the following formula is satisfied;
wherein: lambda (lambda) i The i characteristic value of K, and beta is a constant parameter;
4.4, calculating the angle between the image gradient and the feature vector, and removing unreliable key points when the following formula is satisfied;
min i |cos(θ i )|<γ；
wherein: θ is the angle between the image gradient and the feature vector, and γ is a constant parameter;
4.5, for the detected key points, acquiring an icosahedron region, wherein the icosahedron structure form is shown in fig. 7, calculating the gradient size and direction in the icosahedron region, and interpolating vectors of 20 faces to 12 vertexes to obtain 12bin, and the icosahedron structure form is shown in fig. 8;
4.6 taking a key point as the center, constructing a spherical three-dimensional image with radius of 2σ, wherein σ is a scale constant in 4.1, dividing the spherical window into 4 3 As shown in fig. 9, each sub-region calculates a separate gradient histogram with 12 vertices for each histogram for a total of 4 3 X 12 = 768 components;
4.7 calculating a weighted value of each voxel by using a Gaussian function with a scale sigma based on the distance from the voxel to the key point by using a Gaussian window;
4.8 assigning the contribution of each voxel by tri-linear interpolation between the barycentric coordinates between its three vertices of the intersecting triangle and the centers of the eight sub-areas surrounding the voxel in the cube;
4.9 assuming that the location of the key point is k, the sub-region is centered on y, (λ1, λ2, λ3) is the centroid coordinate of the intersection of the gradient ray and the icosahedral plane, the increment value of the voxel x to the bin corresponding to λi is as follows:
wherein: the voxel coordinates are x; the exponential term is a gaussian window; the multiplier term is the tri-linear interpolation weight of y;
4.10 descriptor is l 2 Normalizing the result of the normalization after the constant threshold stage;
5) Registration fusion of CT and MRI based on 3DSIFT feature operator includes the following steps:
5.1 extracting key points and matching the key points in a pair of images, registering the images by utilizing affine transformation, wherein the affine transformation is as follows:
wherein:for a given coordinate; />Is a parameter;
5.2 fitting affine transformation by linear regression;
5.3 rejecting outliers based on RANSAC for false matches;
and 5.4, obtaining a registration fusion result through least square fitting, wherein the error unit is in millimeter level.
The invention includes using super-resolution restoration to increase the resolution of the MR image such that the data is consistent with a given CT image resolution; three-dimensionally reconstructing CT and MRI images based on an MC algorithm, and performing coarse registration; calculating a complex feature map through the proposed improved fractional order cumulant; the CT and MRI images are fine registered based on the 3DSIFT algorithm. The invention can optimize the registering and fusing effect and precision of the brain CT and MRI images.
1. Working conditions
The experiment adopts an Intel (R) Core (TM) i5-9400F CPU@2.90GHz 2.90GHz processor, runs a PC of Windows10, and the display card is 1 block of GeForce GTX 1060. Using the pytorch deep learning framework and 3DSlicer software, the programming language uses python and matlab.
2. Experimental content and results analysis
The registration process for this experiment is shown in fig. 10. The solid line in the figure represents the matching relation pair of the 3DSIFT feature operator in the brain CT and MRI, so that the matching rate is high, and the corresponding relation is accurate. The registration fusion result is shown in fig. 11, and from the visual point of view, it can be found that the accurate registration fusion can be performed on the brain CT and MRI images.
The experimental result shows that the invention firstly uses super-resolution restoration to improve the resolution of the MR image, so that the data is consistent with the resolution of a given CT image, then three-dimensionally reconstructs the CT image and the MRI image based on the MC algorithm to perform coarse registration, and then improves the robustness of the image by improving the fractional order cumulant, and finally performs registration fusion on the CT image and the MRI image based on the 3DSIFT algorithm, so that the effect is clear and obvious.
Claims (1)
1. A three-dimensional feature medical image fusion method based on improved fractional order cumulants, comprising the steps of:
1) Performing super-resolution restoration on MRI, comprising the following steps:
1.1, improving the image resolution of an MRI slice based on an SRGAN super-resolution network to obtain an MR slice and a CT slice after super-resolution restoration, so that the image resolution of the MRI slice is consistent with that of the CT slice;
2) Coarse registration of CT and MRI images, comprising the steps of:
2.1 respectively carrying out three-dimensional reconstruction on the MR slice and the CT slice which are obtained after super-resolution restoration and obtained in the step 1.1 based on an MC algorithm;
2.2 based on medical priori knowledge, by observing the positions of mandible, nasal bone and eye socket in the CT and MRI three-dimensional reconstruction effect images, taking the CT three-dimensional reconstruction effect image as a reference image, moving the MRI three-dimensional reconstruction effect image to the corresponding position of the T three-dimensional reconstruction effect image, and expressing the positions in the same coordinate system;
3) Calculating a complex feature map based on the improved fractional order cumulants, comprising the steps of:
3.1, based on a fractional calculus principle, fractional moment estimation is carried out on pixel points of CT and MRI images to obtain moment characteristic images, wherein the moment characteristic images are represented by the following formula:
wherein: τ x 、τ y And τ z Data point intervals in the x, y and z directions are respectively represented;
the moment signature in three-dimensional form is of the formula:
wherein: n is n x 、n y And n z Respectively representing accumulation points; k represents the number of points used in each iteration in the calculation; b represents a fraction order;
3.2 calculating fractional order cumulant based on fractional order moment estimation to obtain a cumulant characteristic image, wherein the cumulant characteristic image is represented by the following formula:
wherein: c (C) 4b Fractional order cumulants for the position sought; m is m kp Assigning fractional moment estimation values in a range for the pixel points; the fractional order cumulative amount can also be expressed as follows:
wherein: j is in complex numberω represents different angular frequencies in the cumulative amount generation function;
3.3 verifying the robustness of the fractional cumulative spectrum to noise as follows:
wherein: i represents image truth value data; n represents noise at the corresponding position;
4) The feature extraction is performed based on a 3DSIFT algorithm, and the method comprises the following steps:
4.1 constructing a scale space from the complex feature map result in the step 3), calculating a Gaussian difference function on a Gaussian pyramid, and searching a local extremum of the image on the scale and space, namely a possible key point;
4.2 tracking the general direction of the key points based on the correlation of gradient components, namely, the structure tensor, positioning the key points, and representing the local direction of the key points by using the structure tensor K, wherein the formula is as follows;
wherein:is the gradient of image I at location x; w (x) is a gaussian window centered on the keypoint;
4.3, arranging the eigenvalues of the structure tensors in ascending order, and removing unreliable key points when the following formula is satisfied;
wherein: lambda (lambda) i The i characteristic value of K, and beta is a constant parameter;
4.4, calculating the angle between the image gradient and the feature vector, and removing unreliable key points when the following formula is satisfied;
min i |cos(θ i )|<γ；
wherein: θ is the angle between the image gradient and the feature vector, and γ is a constant parameter;
4.5, for the detected key points, acquiring an icosahedron region of the key points, calculating the gradient size and the gradient direction in the icosahedron region, and interpolating vectors of 20 faces to 12 vertexes to obtain 12bin;
4.6 taking a key point as the center, constructing a spherical three-dimensional image with radius of 2σ, wherein σ is a scale constant in 4.1, dividing the spherical window into 4 3 For each subregion, a separate gradient histogram is calculated, each histogram having 12 vertices for a total of 4 3 X 12 = 768 components;
4.7 calculating a weighted value of each voxel by using a Gaussian function with a scale sigma based on the distance from the voxel to the key point by using a Gaussian window;
4.8 assigning the contribution of each voxel by tri-linear interpolation between the barycentric coordinates between its three vertices of the intersecting triangle and the centers of the eight sub-areas surrounding the voxel in the cube;
4.9 assuming that the location of the key point is k, the sub-region is centered on y, (λ1, λ2, λ3) is the centroid coordinate of the intersection of the gradient ray and the icosahedral plane, the increment value of the voxel x to the bin corresponding to λi is as follows:
wherein: the voxel coordinates are x; the exponential term is a gaussian window; the multiplier term is the tri-linear interpolation weight of y;
4.10 descriptor is l 2 Normalizing the result of the normalization after the constant threshold stage;
5) Registration fusion of CT and MRI based on 3DSIFT feature operator includes the following steps:
5.1 extracting key points and matching the key points in a pair of images, registering the images by utilizing affine transformation, wherein the affine transformation is as follows:
wherein:for a given coordinate; />Is a parameter;
5.2 fitting affine transformation by linear regression;
5.3 rejecting outliers based on RANSAC for false matches;
and 5.4, obtaining a registration fusion result through least square fitting, wherein the error unit is in millimeter level.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311641906.7A CN117541493A (en) | 2023-12-04 | 2023-12-04 | Three-dimensional feature medical image fusion method based on improved fractional order cumulant |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311641906.7A CN117541493A (en) | 2023-12-04 | 2023-12-04 | Three-dimensional feature medical image fusion method based on improved fractional order cumulant |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117541493A true CN117541493A (en) | 2024-02-09 |
Family
ID=89784128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311641906.7A Pending CN117541493A (en) | 2023-12-04 | 2023-12-04 | Three-dimensional feature medical image fusion method based on improved fractional order cumulant |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117541493A (en) |
-
2023
- 2023-12-04 CN CN202311641906.7A patent/CN117541493A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Benameur et al. | A hierarchical statistical modeling approach for the unsupervised 3-D biplanar reconstruction of the scoliotic spine | |
| US20010036302A1 (en) | Method and apparatus for cross modality image registration | |
| EP3309749B1 (en) | Registration of a magnetic tracking system with an imaging device | |
| Mellado et al. | Relative scale estimation and 3D registration of multi-modal geometry using growing least squares | |
| WO2003046814A1 (en) | Automated registration of 3-d medical scans of similar anatomical structures | |
| SE528068C2 (en) | Three dimensional object recognizing method for e.g. aircraft, involves detecting image features in obtained two dimensional representation, and comparing recovered three dimensional shape with reference representation of object | |
| WO2009045471A1 (en) | System and method for organ segmentation using surface patch classification in 2d and 3d images | |
| Audette et al. | Level-set surface segmentation and fast cortical range image tracking for computing intrasurgical deformations | |
| EP4156096A1 (en) | Method, device and system for automated processing of medical images to output alerts for detected dissimilarities | |
| JP2019084349A (en) | Medical image processing apparatus and medical image processing program | |
| CN110634554A (en) | Spine image registration method | |
| Nikou et al. | A joint physics-based statistical deformable model for multimodal brain image analysis | |
| Lorenz et al. | Automated abdominal plane and circumference estimation in 3D US for fetal screening | |
| Hsu | A hybrid approach for brain image registration with local constraints | |
| EP1810216B1 (en) | 3d object recognition | |
| Collignon et al. | Surface-based registration of 3D medical images | |
| Al‐Kofahi et al. | Algorithms for accurate 3D registration of neuronal images acquired by confocal scanning laser microscopy | |
| Schmidt-Richberg et al. | Offset regression networks for view plane estimation in 3D fetal ultrasound | |
| Valizadeh et al. | Parametric-based feature selection via spherical harmonic coefficients for the left ventricle myocardial infarction screening | |
| WO2020058221A1 (en) | Analyzing symmetry in image data | |
| Li et al. | A Fast Subpixel Registration Algorithm Based on Single‐Step DFT Combined with Phase Correlation Constraint in Multimodality Brain Image | |
| CN112102327A (en) | Image processing method and device and computer readable storage medium | |
| Geng et al. | Diffusion MRI registration using orientation distribution functions | |
| CN117541493A (en) | Three-dimensional feature medical image fusion method based on improved fractional order cumulant | |
| WO2014106747A1 (en) | Methods and apparatus for image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |