CN112308765A - Method and device for determining projection parameters - Google Patents
Method and device for determining projection parameters Download PDFInfo
- Publication number
- CN112308765A CN112308765A CN202011091029.7A CN202011091029A CN112308765A CN 112308765 A CN112308765 A CN 112308765A CN 202011091029 A CN202011091029 A CN 202011091029A CN 112308765 A CN112308765 A CN 112308765A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- medical image
- determining
- projection parameters
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims description 32
- 238000012545 processing Methods 0.000 claims description 26
- 238000010606 normalization Methods 0.000 claims description 20
- 238000011176 pooling Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 abstract description 5
- 238000002591 computed tomography Methods 0.000 description 14
- 238000009826 distribution Methods 0.000 description 8
- 210000000988 bone and bone Anatomy 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 238000002595 magnetic resonance imaging Methods 0.000 description 2
- 238000002600 positron emission tomography Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000011524 similarity measure Methods 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G06T3/067—
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
- G06T2207/30012—Spine; Backbone
Abstract
The invention discloses a method and a device for determining projection parameters. The method comprises the following steps: determining attitude information of a target object in a three-dimensional medical image, and establishing a three-dimensional coordinate system corresponding to the attitude information; projecting the three-dimensional medical image in the three-dimensional coordinate system into a two-dimensional coordinate system of the two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image; calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and adjusting the projection parameters according to the similarity until the similarity between the two-dimensional digital reconstructed image obtained according to the adjusted projection parameters and the two-dimensional medical image meets a preset condition; and determining the corresponding projection parameters when the similarity meets the preset conditions as final projection parameters. The algorithm has high running speed and high registration precision.
Description
Technical Field
The invention relates to the technical field of medical imaging, in particular to a method and a device for determining projection parameters.
Background
During Image Guided Therapy (IGT), such as spinal surgery, Guided tracking during surgery is accomplished by registering a 2D (two-dimensional) X-ray Image of a patient with a 3D (three-dimensional) medical Image of the patient.
In the 2D/3D image registration process, a 3D medical image needs to be simulated into a two-dimensional digital Reconstructed image (DRR image) based on a DRR (Digitally Reconstructed radio) algorithm, and then the DRR image and the X-ray film image are registered. However, the currently adopted method for calculating the projection parameters of the DRR algorithm is complex, and the calculation amount is large, so that the real-time requirement is difficult to meet.
Disclosure of Invention
In view of the foregoing, the present invention provides a projection parameter determining apparatus.
Specifically, the invention is realized by the following technical scheme:
in a first aspect, a method for determining projection parameters is provided, including:
determining attitude information of a target object in a three-dimensional medical image, and establishing a three-dimensional coordinate system corresponding to the attitude information;
projecting the three-dimensional medical image in the three-dimensional coordinate system into a two-dimensional coordinate system of the two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image;
calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and adjusting the projection parameters according to the similarity until the similarity between the two-dimensional digital reconstructed image obtained according to the adjusted projection parameters and the two-dimensional medical image meets a preset condition;
and determining the corresponding projection parameters when the similarity meets the preset conditions as final projection parameters.
Optionally, determining pose information of the target object in the three-dimensional medical image comprises:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
and analyzing the pixel coordinates of each pixel point to determine the attitude information.
Optionally, analyzing the pixel coordinates of each pixel point to determine the posture information includes:
constructing a covariance matrix of each pixel point according to the pixel coordinates of each pixel point;
calculating an eigenvalue and an eigenvector of the covariance matrix;
and determining the attitude information according to the eigenvectors corresponding to the maximum three eigenvalues.
Optionally, projecting the three-dimensional medical image into a two-dimensional coordinate system of a two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstructed image, including:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
projecting the pixel coordinates of each pixel point into the two-dimensional coordinate system through a construction point projection matrix according to the projection parameters;
and carrying out weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the sub-region corresponding to the two-dimensional digital reconstructed image.
Optionally, calculating the similarity of the two-dimensional digitally reconstructed image and the two-dimensional medical image of the target object comprises:
respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the two-dimensional medical image;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the normalization processing and the two-dimensional medical image.
Optionally, calculating the similarity between the normalized two-dimensional digital reconstructed image and the two-dimensional medical image includes:
performing pooling operation on the two-dimensional digital reconstructed image and the two-dimensional medical image which are subjected to normalization processing respectively;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the pooling operation and the two-dimensional medical image.
In a second aspect, an apparatus for determining projection parameters is provided, including:
the system comprises an establishing module, a processing module and a display module, wherein the establishing module is used for determining the posture information of a target object in a three-dimensional medical image and establishing a three-dimensional coordinate system corresponding to the posture information;
the projection module is used for projecting the three-dimensional medical image in the three-dimensional coordinate system into a two-dimensional coordinate system of a two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image;
the calculation module is used for calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and adjusting the projection parameters according to the similarity until the similarity between the two-dimensional digital reconstructed image obtained according to the adjusted projection parameters and the two-dimensional medical image meets a preset condition;
and the determining module is used for determining the corresponding projection parameters as final projection parameters when the similarity meets the preset conditions.
Optionally, in determining pose information of the target object in the three-dimensional medical image, the establishing module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
and analyzing the pixel coordinates of each pixel point to determine the attitude information.
Optionally, when the pixel coordinates of each pixel point are analyzed to determine the posture information, the establishing module is configured to:
constructing a covariance matrix of each pixel point according to the pixel coordinates of each pixel point;
calculating an eigenvalue and an eigenvector of the covariance matrix;
and determining the attitude information according to the eigenvectors corresponding to the maximum three eigenvalues.
Optionally, the projection module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
projecting the pixel coordinates of each pixel point into the two-dimensional coordinate system through a construction point projection matrix according to the projection parameters;
and carrying out weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the sub-region corresponding to the two-dimensional digital reconstructed image.
Optionally, in calculating the similarity of the two-dimensional digitally reconstructed image and the two-dimensional medical image of the target object, the calculation module is configured to:
respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the two-dimensional medical image;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the normalization processing and the two-dimensional medical image.
Optionally, when calculating the similarity between the normalized two-dimensional digital reconstructed image and the two-dimensional medical image, the calculating module is configured to:
performing pooling operation on the two-dimensional digital reconstructed image and the two-dimensional medical image which are subjected to normalization processing respectively;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the pooling operation and the two-dimensional medical image.
In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method for determining projection parameters is implemented as any one of the above methods.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method for determining projection parameters of any of the above.
The technical scheme provided by the embodiment of the invention can have the following beneficial effects:
in the embodiment of the invention, the three-dimensional coordinate system corresponding to the posture information of the target object in the three-dimensional medical image is analyzed, and then the three-dimensional coordinate system information is utilized to reduce the projection parameter search space, so that compared with a mode of simultaneously searching six degrees of freedom in the related technology, the operation speed of the algorithm is greatly accelerated, and the registration precision is higher.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1a is a flow chart illustrating a method for determining projection parameters in accordance with an exemplary embodiment of the present invention;
FIG. 1b is a diagram illustrating a scatter distribution on a canvas in accordance with an exemplary embodiment of the present invention;
FIG. 2 is a flow chart illustrating another method of determining projection parameters in accordance with an exemplary embodiment of the present invention;
FIG. 3 illustrates an apparatus for determining projection parameters in accordance with an exemplary embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Fig. 1a is a flowchart illustrating a method for determining projection parameters according to an exemplary embodiment of the present invention, where the method may include:
The target object is a diseased region of a patient, such as a diseased spine. The three-dimensional medical image is an image obtained by scanning a target object, and may be, for example, a CT (computed tomography) image, an MRI (magnetic resonance imaging) image, a PET (positron emission tomography) image, or the like.
The 2D/3D image registration is achieved by placing an image in a fixed spatial coordinate system, e.g. the 2D medical image in an XY coordinate system, and performing a series of spatial transformations on the 3D medical image to spatially align points on the 3D medical image, or at least points of the region of interest, with the 2D medical image. Because the 2D/3D medical image registration is not the registration of a spatial free rigid body, the 2D medical image generally has obvious directionality, and therefore, before the registration, the posture information of the target object in the three-dimensional medical image, that is, the spatial vector of the posture, needs to be determined, and the "up, down, front, back, left and right" orientation of the bone can be represented.
The following describes a specific implementation of determining the posture information, which may include the following steps:
s1a, determining pixel coordinates of each pixel point of the region where the target object is located in the three-dimensional medical image.
Before determining the pose information of the target object, a region of interest in the three-dimensional medical image, that is, a region where the target object is located, may be determined first, taking the target object as a bone and the three-dimensional medical image as a CT image as an example, a gray value of the bone imaged in the CT image is greater than 150, which may be but is not limited to determining a pixel point with a gray value greater than 150 as a bone region, sampling the pixel point of the bone region as a sample, and a pixel coordinate of each sample may be but is not limited to expressed as (x1, y1, z1), (x2, y2, z2), … …, (xn, yn, zn). The pixel coordinates of the sample are represented by a pixel coordinate matrix S[3*n_samples]The pixel coordinates of the sample may be expressed as coordinates of a geometric center point in a three-dimensional space with respect to a region where the target object is located. Determining attitude information of a target object, i.e. from a pixel coordinate matrix S[3*n_samples]Determining pose information, the pixel coordinate matrix is a two-dimensional matrix of 3 × n, which may be expressed as, but is not limited to:
the determination of the geometric center point may be, but not limited to, performing weighted average of three dimensional directions on the pixel coordinate matrix, and determining the geometric center point according to a result of the weighted average.
And S2a, constructing a covariance matrix of the sample.
And constructing a covariance matrix of the sample, namely constructing covariance matrices of all pixel points in the region of interest. Wherein the covariance matrix is a 3 x 3 two-dimensional matrix.
And S3a, calculating the eigenvalue and the eigenvector of the covariance matrix.
And S4a, determining the posture information of the target object according to the eigenvectors corresponding to the maximum three eigenvalues.
Based on the eigenvalue and the eigenvector of the covariance matrix, the corresponding eigenvectors are orthogonal pairwise after being sorted according to the magnitude of the eigenvalue, and the larger the eigenvalue is, the larger the variance of the matrix on the corresponding eigenvector is, and the more the information content is. And applying the properties to the three-dimensional data to correspondingly obtain 3 characteristic values. The larger the eigenvalue is, the larger the variance of projection on the dimension is, the 3 eigenvalues are arranged in the order from large to small, the eigenvectors corresponding to the three sorted eigenvalues are orthogonal pairwise to form a group of orthogonal bases in the space, and the group of orthogonal bases can be determined as a three-dimensional coordinate system.
Specifically, the direction of the feature vector corresponding to the feature value having the maximum value may be determined as the X-axis direction of the three-dimensional coordinate system, the direction of the feature vector corresponding to the second largest feature value may be determined as the Y-axis direction of the three-dimensional coordinate system, and the direction of the feature vector corresponding to the third largest feature value may be determined as the Z-axis direction of the three-dimensional coordinate system. The X-axis direction, the Y-axis direction and the Z-axis direction can be defined as an Axial plane (Axial) direction, a Sagittal plane (Sagittal) direction and a Coronal plane (Coronal) direction according to actual requirements.
With respect to the origin of the three-dimensional coordinate system, the geometric center point of the region where the target object is located may be, but is not limited to, determined as the origin of the pixel coordinate matrix.
And 102, projecting the three-dimensional medical image in the three-dimensional coordinate system to a two-dimensional coordinate system of the two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image.
The two-dimensional medical image is a two-dimensional medical image of the target object, and may be, for example, an X-ray image obtained by imaging the target object.
The projection parameters may include three projection view angle parameters (α, β, γ) and a distance parameter d, which characterizes a distance between the photographing apparatus and the target object when the two-dimensional medical image is photographed).
In this embodiment, final projection parameters, initial projection parameters, that is, the three projection viewing angles may be selected from 0 ° to 360 °, and the initial values of the three projection viewing angles may be set to be the same or different; the distance parameter d may be set to be the same as the distance between the photographing apparatus and the target object when the two-dimensional medical image is photographed.
The three-dimensional medical image can be projected into the two-dimensional coordinate system according to the initially set projection parameters, that is, the pixel coordinates of each pixel point in the region where the target object is located are projected into the two-dimensional coordinate system according to the projection parameters. Specifically, the method is characterized in that the method is performed at a mathematical level, namely a 3 x n pixel coordinate matrix is converted into a 2 x n coordinate matrix; standing on the image layer, namely projecting each pixel point of the area where the target object is located in the three-dimensional image on a two-dimensional canvas, wherein the two-dimensional canvas presents some two-dimensional scattered points with gray information, the two-dimensional canvas and the two-dimensional medical image have the same coordinate system, and the two-dimensional scattered points are presented in each quadrant of the two-dimensional coordinate. And performing weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the corresponding sub-region to obtain the two-dimensional digital reconstructed image.
The size of the sub-region may be set according to actual conditions, and it can be understood that the smaller the size of the sub-region is, the more accurate the obtained result is, for example, each sub-region may be divided to be consistent with the size of pixels of the image.
The larger the number of the scattered points in the sub-region, the darker the color presented on the canvas, the larger the gray value should be, and the weighting coefficient may be set to 1, that is, the gray value corresponding to each scattered point in the sub-region is added as the gray value of the square. Taking the canvas shown in fig. 1b as an example, each square in the drawing represents a pixel, and each dot represents a scatter. For the pixel A, 3 scattered points are included, and the gray values corresponding to the 3 scattered points are added to be used as the gray value of the pixel A; if the pixel B comprises 1 scatter point, taking the gray value corresponding to the scatter point as the gray value of the pixel B; if the pixel C includes 2 scatters, the grayscale values corresponding to the 2 scatters are added to obtain the grayscale value of the pixel B. And processing scattered points in the canvas in sequence to obtain the two-dimensional digital reconstructed image.
And 103, calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and determining whether the similarity meets a preset condition.
Because the format of the two-dimensional digital reconstructed image is different from that of the two-dimensional medical image, the two-dimensional digital reconstructed image and the two-dimensional medical image need to be preprocessed before similarity calculation, so as to unify data structures, which may include:
s1b, an image area with the same size as the two-dimensional medical image is cut from the two-dimensional digital reconstruction image.
The center of the image region corresponds to the geometric center point of the region in which the target object is located in the three-dimensional medical image. The geometric center point of the region where the target object is located in the three-dimensional medical image is determined and projected into the two-dimensional canvas, namely the center of the geometric center point in the two-dimensional canvas can be determined.
If the size of the two-dimensional medical image is q × q, q may be, but is not limited to 1024, then an image region with a size of 1024 × 1024 is cut out from the two-dimensional digital reconstructed image.
And S2b, respectively carrying out normalization processing on the image area and the two-dimensional medical image.
And (3) carrying out normalization processing on the image area and the two-dimensional medical image, namely carrying out normal distribution processing on the image area and the two-dimensional medical image, and mapping the image area and the two-dimensional medical image to floating point numbers in a range of 0-1 according to normal distribution.
And S3b, calculating the similarity between the normalized image area and the two-dimensional medical image.
The nature of registration is to find a projective transformation that maximizes the similarity between the two data, so how to define and calculate the "similarity" is very important. There are many methods for estimating the difference between two data, which can be, but not limited to, using euclidean distance, mahalanobis distance, cosine similarity, mutual information, relative entropy, etc.
One possible implementation of calculating similarity is described below:
performing pooling operation on the normalized two-dimensional digital reconstructed image and the two-dimensional medical image by using pooling check to respectively obtain two pooled gray level matrixes; if "mutual information" is used as the similarity measure, since the mutual information is defined on two probability distributions, which is equivalent to regarding each element in the gray matrix as a sample in one probability distribution, the pooled gray matrices may be arranged in rows to form a queue, regarding the elements in the queue as a sample in one probability distribution, and calculating the mutual information of the two queues to be used as the similarity measure between the image region and the two-dimensional medical image. The value of k may be, but not limited to, a non-prime factor of n, and the step size may be the same as the value of k. The pooling kernel may be decremented from large to small to distinguish "levels" until all pixels participate in the similarity comparison. When the value of k is a non-prime factor of n, the distance between the layers cannot become larger, the final projection parameters can be searched more quickly, and meanwhile, the precision cannot be reduced. Pooling the images may reduce computational effort, memory usage, and prevent overfitting.
In step 103, if the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image of the target object does not meet the preset condition, which indicates that the projection parameters of the iteration do not meet the requirements, and the three-dimensional image and the two-dimensional image do not reach the optimal registration position, step 104 is executed.
The preset condition, that is, the condition for stopping the iteration, may be, for example, that the similarity is a maximum value, or that the similarity is greater than a similarity threshold (which may be set according to actual requirements).
In this embodiment, the projection parameter may be adjusted by an exhaustive search method, and at this time, the preset condition may be set to have the similarity as a maximum value; the projection parameters may also be adjusted by a gradient descent method, and at this time, the preset condition may be set to have the similarity as a maximum value or the similarity is greater than the similarity threshold.
And 104, adjusting the projection parameters. And then returning to the step 102, and projecting the three-dimensional medical image into the two-dimensional coordinate system according to the adjusted projection parameters in the step 102.
In step 103, if the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image of the target object meets a preset condition, which indicates that the projection parameters of the iteration meet the requirements, and the three-dimensional medical image and the two-dimensional medical image reach the optimal matching position, the iteration is stopped, and step 105 is executed.
And 105, determining the projection parameters of the iteration of the current round as final projection parameters.
In the embodiment of the invention, the three-dimensional coordinate system corresponding to the posture information of the target object in the three-dimensional medical image is analyzed, and then the three-dimensional coordinate system information is utilized to reduce the projection parameter search space, so that compared with a mode of simultaneously searching six degrees of freedom in the related technology, the operation speed of the algorithm is greatly accelerated, and the registration precision is higher.
Fig. 2 is a flowchart illustrating another method for determining projection parameters according to an exemplary embodiment of the present invention, in which, for example, the method for determining the projection parameters for projecting the CT image onto the coordinate system of the X-ray image in order to achieve the registration of the CT image of the spine with the X-ray image may include the following steps:
The coordinates of each pixel are present in the image, i.e. the 3D scatter set.
The pixel coordinates of each pixel point can be analyzed by but not limited to Principal Component Analysis (PCA) to determine the posture information of the spine in the CT image, the posture information can represent the upper, lower, left, right, front and back directions of the spine, the posture information of the spine can be used as three axes of a three-dimensional coordinate system, the geometric center point of the spine is used as the origin of the three-dimensional coordinate system, and the three-dimensional coordinate system of the CT image is established.
And 203, projecting the CT image in the three-dimensional coordinate system into a two-dimensional coordinate system of the X-ray picture according to the projection parameters to obtain a two-dimensional digital reconstructed image.
The projection parameters may include three projection view angle parameters (α, β, γ) and a distance parameter d, which characterizes a distance between the capturing device and the target object when the two-dimensional medical image is captured.
In this embodiment, final projection parameters, initial projection parameters, that is, the three projection viewing angles may be selected from 0 ° to 360 °, and the initial values of the three projection viewing angles may be set to be the same or different; the distance parameter d may be set to be the same as the distance between the photographing apparatus and the target object when the two-dimensional medical image is photographed. According to the projection parameters, the pixel coordinates of the region where the spine is located in the CT image in the three-dimensional coordinate system can be projected into the two-dimensional coordinate system of the X-ray picture through the construction point projection matrix, and a two-dimensional digital reconstruction image corresponding to the projection parameters adopted in the iteration of the current round is obtained.
And 204, respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the X-ray picture.
That is, the gray values of the two-dimensional digital reconstructed image and the X-ray picture are mapped to the floating point number within the range of 0-1 according to the normal distribution. Before normalization processing, an image area with the same size as that of the X-ray image can be cut from the two-dimensional digital reconstructed image, and normal distribution processing is performed on the image area and the two-dimensional medical image.
The value of the size k × k of the pooled kernel of the pooled operation may be a non-prime factor of n, and the step size may be the same as the value of k. The pooling kernel may be decremented from large to small until all pixels participate in the similarity rating. Pooling the images may reduce computational effort, memory usage, and prevent overfitting.
And step 206, calculating mutual information of the two feature matrixes.
The mutual information is used as similarity measurement of two characteristic matrixes, namely the similarity measurement of the two-dimensional digital reconstructed image and the X-ray picture, and can represent whether the registration position of the CT image and the X-ray picture meets the requirement or not.
And step 207, judging whether the mutual information meets the preset condition.
The mutual information is used as similarity measurement for judging whether to adjust the projection parameters, an exhaustive search method can be adopted for adjusting the projection parameters, and at the moment, the preset condition can be set as that the mutual information is the maximum value; the projection parameters may also be adjusted by a gradient descent method, and at this time, the preset condition may be set such that the mutual information is a maximum value or the mutual information is greater than a threshold value.
In step 207, if the mutual information does not meet the preset condition, which indicates that the CT image and the X-ray image do not reach the optimal matching position, step 208 is executed.
In step 207, if the mutual information meets the preset condition, which indicates that the projection parameters of the current iteration meet the requirements, the iteration is stopped, and step 209 is executed.
And step 209, determining the projection parameters of the iteration of the current round as final projection parameters.
Corresponding to the embodiment of the method for determining the projection parameters, the invention also provides an embodiment of a device for determining the projection parameters.
Fig. 3 is a device for determining projection parameters according to an exemplary embodiment of the present invention, which may include:
an establishing module 31, configured to determine pose information of a target object in a three-dimensional medical image, and establish a three-dimensional coordinate system corresponding to the pose information;
the projection module 32 is configured to project the three-dimensional medical image in the three-dimensional coordinate system to a two-dimensional coordinate system of a two-dimensional medical image to be registered according to the projection parameters, so as to obtain a two-dimensional digital reconstructed image;
a calculating module 33, configured to calculate similarity between the two-dimensional digitally reconstructed image and the two-dimensional medical image, and adjust the projection parameter according to the similarity until the similarity between the two-dimensional digitally reconstructed image obtained according to the adjusted projection parameter and the two-dimensional medical image meets a preset condition;
a determining module 34, configured to determine, as a final projection parameter, a corresponding projection parameter when the similarity satisfies a preset condition.
Optionally, in determining pose information of the target object in the three-dimensional medical image, the establishing module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
and analyzing the pixel coordinates of each pixel point to determine the attitude information.
Optionally, when the pixel coordinates of each pixel point are analyzed to determine the posture information, the establishing module is configured to:
constructing a covariance matrix of each pixel point according to the pixel coordinates of each pixel point;
calculating an eigenvalue and an eigenvector of the covariance matrix;
and determining the attitude information according to the eigenvectors corresponding to the maximum three eigenvalues.
Optionally, the projection module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
projecting the pixel coordinates of each pixel point into the two-dimensional coordinate system through a construction point projection matrix according to the projection parameters;
and carrying out weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the sub-region corresponding to the two-dimensional digital reconstructed image.
Optionally, in calculating the similarity of the two-dimensional digitally reconstructed image and the two-dimensional medical image of the target object, the calculation module is configured to:
respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the two-dimensional medical image;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the normalization processing and the two-dimensional medical image.
Optionally, when calculating the similarity between the normalized two-dimensional digital reconstructed image and the two-dimensional medical image, the calculating module is configured to:
performing pooling operation on the two-dimensional digital reconstructed image and the two-dimensional medical image which are subjected to normalization processing respectively;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the pooling operation and the two-dimensional medical image.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
Fig. 4 is a schematic diagram of an electronic device according to an exemplary embodiment of the present invention, and illustrates a block diagram of an exemplary electronic device 40 suitable for implementing embodiments of the present invention. The electronic device 40 shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in FIG. 4, electronic device 40 may take the form of a general purpose computing device, which may be a server device, for example. The components of electronic device 40 may include, but are not limited to: the at least one processor 41, the at least one memory 42, and a bus 43 connecting the various system components (including the memory 42 and the processor 41).
The bus 43 includes a data bus, an address bus, and a control bus.
The memory 42 may include volatile memory, such as Random Access Memory (RAM)421 and/or cache memory 422, and may further include Read Only Memory (ROM) 423.
The processor 41 executes various functional applications and data processing, such as the methods provided by any of the above embodiments, by running a computer program stored in the memory 42.
The electronic device 40 may also communicate with one or more external devices 44 (e.g., keyboard, pointing device, etc.). Such communication may be through an input/output (I/O) interface 45. Also, the model-generated electronic device 40 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via a network adapter 46. As shown, the network adapter 46 communicates with the other modules of the model-generated electronic device 40 over a bus 43. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the model-generating electronic device 40, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the method provided in any of the above embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A method for determining projection parameters, comprising:
determining attitude information of a target object in a three-dimensional medical image, and establishing a three-dimensional coordinate system corresponding to the attitude information;
projecting the three-dimensional medical image in the three-dimensional coordinate system into a two-dimensional coordinate system of the two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image;
calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and adjusting the projection parameters according to the similarity until the similarity between the two-dimensional digital reconstructed image obtained according to the adjusted projection parameters and the two-dimensional medical image meets a preset condition;
and determining the corresponding projection parameters when the similarity meets the preset conditions as final projection parameters.
2. The method for determining projection parameters according to claim 1, wherein determining pose information of a target object in a three-dimensional medical image comprises:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
and analyzing the pixel coordinates of each pixel point to determine the attitude information.
3. The method of claim 2, wherein analyzing pixel coordinates of the pixels to determine the pose information comprises:
constructing a covariance matrix of each pixel point according to the pixel coordinates of each pixel point;
calculating an eigenvalue and an eigenvector of the covariance matrix;
and determining the attitude information according to the eigenvectors corresponding to the maximum three eigenvalues.
4. The method for determining projection parameters according to claim 1, wherein projecting the three-dimensional medical image into a two-dimensional coordinate system of a two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstructed image comprises:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
projecting the pixel coordinates of each pixel point into the two-dimensional coordinate system through a construction point projection matrix according to the projection parameters;
and carrying out weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the sub-region corresponding to the two-dimensional digital reconstructed image.
5. The method for determining projection parameters according to claim 1, wherein calculating the similarity of the two-dimensional digitally reconstructed image and the two-dimensional medical image of the target object comprises:
respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the two-dimensional medical image;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the normalization processing and the two-dimensional medical image.
6. The method for determining projection parameters according to claim 5, wherein calculating the similarity between the normalized two-dimensional digitally reconstructed image and the two-dimensional medical image comprises:
performing pooling operation on the two-dimensional digital reconstructed image and the two-dimensional medical image which are subjected to normalization processing respectively;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the pooling operation and the two-dimensional medical image.
7. An apparatus for determining projection parameters, comprising:
the system comprises an establishing module, a processing module and a display module, wherein the establishing module is used for determining the posture information of a target object in a three-dimensional medical image and establishing a three-dimensional coordinate system corresponding to the posture information;
the projection module is used for projecting the three-dimensional medical image in the three-dimensional coordinate system into a two-dimensional coordinate system of a two-dimensional medical image to be registered according to the projection parameters to obtain a two-dimensional digital reconstruction image;
the calculation module is used for calculating the similarity between the two-dimensional digital reconstructed image and the two-dimensional medical image, and adjusting the projection parameters according to the similarity until the similarity between the two-dimensional digital reconstructed image obtained according to the adjusted projection parameters and the two-dimensional medical image meets a preset condition;
and the determining module is used for determining the corresponding projection parameters as final projection parameters when the similarity meets the preset conditions.
8. The apparatus for determining projection parameters according to claim 7, wherein in determining pose information of a target object in a three-dimensional medical image, the establishing module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
and analyzing the pixel coordinates of each pixel point to determine the attitude information.
9. The apparatus for determining projection parameters according to claim 8, wherein when analyzing the pixel coordinates of the pixels to determine the pose information, the establishing module is configured to:
constructing a covariance matrix of each pixel point according to the pixel coordinates of each pixel point;
calculating an eigenvalue and an eigenvector of the covariance matrix;
and determining the attitude information according to the eigenvectors corresponding to the maximum three eigenvalues.
10. The apparatus for determining projection parameters of claim 7, wherein the projection module is configured to:
determining pixel coordinates of each pixel point of the region of the target object in the three-dimensional medical image;
projecting the pixel coordinates of each pixel point into the two-dimensional coordinate system through a construction point projection matrix according to the projection parameters;
and carrying out weighted summation on the gray values of the sub-regions in the quadrants of the two-dimensional coordinate system, and determining the weighted summation result as the gray value of the sub-region corresponding to the two-dimensional digital reconstructed image.
11. The apparatus for determining projection parameters according to claim 7, wherein in calculating the similarity between the two-dimensional digitally reconstructed image and the two-dimensional medical image of the target object, the calculating module is configured to:
respectively carrying out normalization processing on the two-dimensional digital reconstructed image and the two-dimensional medical image;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the normalization processing and the two-dimensional medical image.
12. The apparatus for determining projection parameters according to claim 11, wherein in calculating the similarity between the normalized two-dimensional digitally reconstructed image and the two-dimensional medical image, the calculating module is configured to:
performing pooling operation on the two-dimensional digital reconstructed image and the two-dimensional medical image which are subjected to normalization processing respectively;
and calculating the similarity between the two-dimensional digital reconstructed image subjected to the pooling operation and the two-dimensional medical image.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method for determining projection parameters of any of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for determining projection parameters of one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011091029.7A CN112308765A (en) | 2020-10-13 | 2020-10-13 | Method and device for determining projection parameters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011091029.7A CN112308765A (en) | 2020-10-13 | 2020-10-13 | Method and device for determining projection parameters |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112308765A true CN112308765A (en) | 2021-02-02 |
Family
ID=74489659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011091029.7A Pending CN112308765A (en) | 2020-10-13 | 2020-10-13 | Method and device for determining projection parameters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112308765A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113421226A (en) * | 2021-06-03 | 2021-09-21 | 山东师范大学 | CT-DR multi-modal esophageal image registration method and system based on mutual information |
| CN114831732A (en) * | 2022-07-04 | 2022-08-02 | 真健康（北京）医疗科技有限公司 | Puncture position verification method and device based on X-ray image |
| CN114842004A (en) * | 2022-07-04 | 2022-08-02 | 真健康（北京）医疗科技有限公司 | Puncture position verification method and device based on neural network model |
| CN115082534A (en) * | 2022-07-21 | 2022-09-20 | 杭州三坛医疗科技有限公司 | Biplane image registration method and device and robot |
| WO2023006021A1 (en) * | 2021-07-30 | 2023-02-02 | 武汉联影智融医疗科技有限公司 | Registration method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104637061A (en) * | 2015-01-30 | 2015-05-20 | 中国科学院自动化研究所 | Two-dimensional and three-dimensional medical image registration method |
| CN110826499A (en) * | 2019-11-08 | 2020-02-21 | 上海眼控科技股份有限公司 | Object space parameter detection method and device, electronic equipment and storage medium |
-
2020
- 2020-10-13 CN CN202011091029.7A patent/CN112308765A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104637061A (en) * | 2015-01-30 | 2015-05-20 | 中国科学院自动化研究所 | Two-dimensional and three-dimensional medical image registration method |
| CN110826499A (en) * | 2019-11-08 | 2020-02-21 | 上海眼控科技股份有限公司 | Object space parameter detection method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 党建武: "基于GPU的2D-3D医学图像配准", 《计算机科学》 * |
| 刘达: "基于局部方差采样的2D/3D医学图像配准技术", 《高技术通讯》 * |
| 张薇: "基于灰度的二维/三维图像配准方法及其在骨科导航手术中的实现", 《中国医学影像技术》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113421226A (en) * | 2021-06-03 | 2021-09-21 | 山东师范大学 | CT-DR multi-modal esophageal image registration method and system based on mutual information |
| CN113421226B (en) * | 2021-06-03 | 2022-11-01 | 山东师范大学 | CT-DR multi-modal esophageal image registration method and system based on mutual information |
| WO2023006021A1 (en) * | 2021-07-30 | 2023-02-02 | 武汉联影智融医疗科技有限公司 | Registration method and system |
| CN114831732A (en) * | 2022-07-04 | 2022-08-02 | 真健康（北京）医疗科技有限公司 | Puncture position verification method and device based on X-ray image |
| CN114842004A (en) * | 2022-07-04 | 2022-08-02 | 真健康（北京）医疗科技有限公司 | Puncture position verification method and device based on neural network model |
| CN115082534A (en) * | 2022-07-21 | 2022-09-20 | 杭州三坛医疗科技有限公司 | Biplane image registration method and device and robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112308765A (en) | Method and device for determining projection parameters | |
| CN109949899B (en) | Image three-dimensional measurement method, electronic device, storage medium, and program product | |
| JP5111210B2 (en) | Image processing apparatus and image processing method | |
| US9119599B2 (en) | Medical image display apparatus, medical image display method and non-transitory computer-readable recording medium having stored therein medical image display program | |
| US7764817B2 (en) | Method for database guided simultaneous multi slice object detection in three dimensional volumetric data | |
| US9545217B2 (en) | Movement correction in MRI using a camera | |
| Mattes et al. | Nonrigid multimodality image registration | |
| JP5759161B2 (en) | Object recognition device, object recognition method, learning device, learning method, program, and information processing system | |
| CN107578376B (en) | Image splicing method based on feature point clustering four-way division and local transformation matrix | |
| CN107133946B (en) | Medical image processing method, device and equipment | |
| WO2001043070A2 (en) | Method and apparatus for cross modality image registration | |
| CN111275749A (en) | Image registration and neural network training method and device | |
| EP3828829B1 (en) | Method and apparatus for determining mid-sagittal plane in magnetic resonance images | |
| WO2019013162A1 (en) | Information processing device and information processing method | |
| Tang et al. | Retinal image registration based on robust non-rigid point matching method | |
| Suganya et al. | Intensity based image registration by maximization of mutual information | |
| Lötjönen et al. | Four-chamber 3-D statistical shape model from cardiac short-axis and long-axis MR images | |
| JP2000353244A (en) | Method for obtaining basic matrix, method for restoring euclidean three-dimensional information and device therefor | |
| CN113963057A (en) | Imaging geometric relation calibration method and device, electronic equipment and storage medium | |
| CN112308764A (en) | Image registration method and device | |
| CN100573594C (en) | Being used for the automatic optimal view that cardiac image obtains determines | |
| Savii | Camera calibration using compound genetic-simplex algorithm | |
| CN115861329B (en) | Registration method for skin area and related product | |
| Heo et al. | Form representions and means for landmarks: A survey and comparative study | |
| Bağci et al. | Orientation estimation of anatomical structures in medical images for object recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210202 |
|
| RJ01 | Rejection of invention patent application after publication |