WO2022247296A1

WO2022247296A1 - Mark point-based image registration method

Info

Publication number: WO2022247296A1
Application number: PCT/CN2022/070425
Authority: WO
Inventors: 魏军; 沈烁; 卢旭玲; 朱德明; 田孟秋
Original assignee: 广州柏视医疗科技有限公司
Priority date: 2021-05-27
Filing date: 2022-01-06
Publication date: 2022-12-01
Also published as: CN113298854A; CN113298854B

Abstract

A mark point-based image registration method, comprising: inputting two medical images in any mode; extracting pyramid features of the two input images by using a pre-trained neural network, wherein the training process of the network comprises a plurality of different tasks and relates to a plurality of different input modes; extracting the pyramid features by using the neural network, and obtaining a plurality of matching point pairs representing certain semantics between the two images by means of the processes of searching, screening, matching and the like; fitting a transformation matrix and a displacement vector of rigid registration by minimizing the sum of point distances between all the matching point pairs, to obtain a medical image Warped image after the rigid registration; and on the basis of the rigid registration, obtaining a displacement field three-dimensional matrix of non-rigid registration by means of an interpolation method based on a radial basis, to obtain a medical image Warped image after the non-rigid registration. Therefore, the method can effectively improve the image registration effect.

Description

Image registration method based on marker points

technical field

The invention relates to the field of image processing, deep learning, and medical treatment, in particular to an image registration method based on marker points.

Background technique

Image registration has many practical applications in medical image processing and analysis. With the advancement of medical imaging equipment, for the same patient, images of multiple different modalities containing accurate anatomical information can be collected, such as CT, CBCT, MRI, PET, etc. However, making a diagnosis by observing different images requires spatial imagination and the doctor's subjective experience. With the correct image registration method, a variety of information can be accurately fused into the same image, making it easier and more accurate for doctors to observe lesions and structures from various angles. At the same time, through the registration of dynamic images collected at different times, the changes in lesions and organs can be quantitatively analyzed, making medical diagnosis, surgical planning, and radiotherapy planning more accurate and reliable.

The traditional image registration method is based on the optimization of the similarity objective function to solve the problem, which is easy to converge to the local minimum, especially the registration effect on different modal images is poor, and the iterative solution process takes a long time. The image registration method based on markers can solve the above problems, but the acquisition of the gold standard of markers requires a lot of time for doctors and experts, and the cost is high. In recent years, there has been a lot of interest in exploring the use of artificial intelligence for diagnosis, and in some areas AI algorithms have been used to build mathematical models that outperform human medical experts. Therefore, there is reason to believe that using AI algorithms to improve traditional image registration methods can effectively improve the effect of image registration.

The information disclosed in this Background section is only for enhancing the understanding of the general background of the present invention and should not be taken as an acknowledgment or any form of suggestion that the information constitutes the prior art that is already known to those skilled in the art.

Contents of the invention

The object of the present invention is to provide an image registration method based on marker points, which can improve the traditional image registration method by using AI algorithm and effectively improve the effect of image registration.

In order to achieve the above object, the present invention provides a marker-based image registration method, which mainly includes the following steps: input two medical images of any modality (CT, CBCT, MRI, PET, etc.), one as a fixed image (reference image), and the other as a moving image (image to be registered); a pre-trained neural network is used to extract the pyramid features of the two input images. The training process of the network includes many different tasks and involves the above-mentioned many different input mode; use the above-mentioned neural network to extract pyramid features, and obtain multiple matching point pairs representing certain semantics between two images through searching, screening, matching and other processes; by minimizing the sum of the distances between all matching point pairs , to fit the transformation matrix and displacement vector of rigid registration, so as to obtain the warped image of the medical image after rigid registration. And on the basis of rigid registration, the three-dimensional matrix of the displacement field of non-rigid registration is obtained by interpolation method based on radial basis, so as to obtain the warped image of the medical image after non-rigid registration.

In a preferred embodiment, a pre-trained neural network is used to extract the pyramid features of two input images, including: the structure of the neural network is divided into a backbone network and a plurality of subsequent branch networks. The backbone network is shared among different tasks, and each branch network corresponds to a task. Finally, the backbone network is used to extract image features. The training process of the neural network involves a variety of different tasks and involves a variety of different input modalities, including but not limited to: CT-based NPC primary tumor (GTV) segmentation, MRI-based NPC primary tumor segmentation , CT-based cervical cancer primary tumor segmentation, PET-based lung primary tumor segmentation, CT-based organ-at-risk (OAR) segmentation, MRI-based organ-at-risk segmentation, CBCT-based organ-at-risk segmentation, CT-based lung Nodule object detection, etc. And first use one of the tasks to train the neural network, and then train each of the other input modal tasks at the same time, and then train each of the remaining tasks separately. The backbone network parameters are fixed during training, and finally all tasks are trained at the same time to fine-tune all parameter.

In a preferred embodiment, the image registration method based on markers further includes: using the above-mentioned neural network to extract pyramid features, and obtaining multiple matching points representing certain semantics between two images through processes such as searching, screening, and matching Yes, including the following steps: input _If (fixed image) and I _m (moving image) into the above-mentioned pre-trained neural network, and extract the pyramidal feature map (feature map) of the two input images

and

Among them, l∈{1,2,3,4,5} represents the l-level feature, and the larger the number, the deeper the layer, that is, the smaller the feature size is, the more high-level semantics it contains. The search for matching points needs to be generated within a specific search range, and the search range is set starting from l=5:

S ⁵ ＝{(P ⁵ ,Q ⁵ )}

in:

is the nth search range of the lth level of _If , correspondingly

is the nth search range of the lth level of _Im , ^N1 is the number of the lth level search range, and ^S1 is a set of multiple search range pairs of the lth level. When l=5, the search range is

and

The entire range of , that is, N ⁵ =1.

Search range by the following formula

and

feature map of

and

transform to get

and

in,

expressed in

Local feature maps in the range,

is the transformed feature map,

for

the mean value of

for

standard deviation of

in the search range

and

Search for matching point pairs within, when the following conditions are met, the two points p ^l and q ^l are matching point pairs:

That is to say, if

The point p ^l inside

When searching within the range, the point with the highest similarity is q ^l , and vice versa, then p ^l and q ^l are matching point pairs. The formula for calculating the similarity is:

Among them, ε(p ^l ) is the point set of the neighborhood within a specific range of point p ^l .

All the ^N1 search ranges of the lth level perform the above-mentioned steps of searching for matching points respectively, and then obtain the set ^Λl of all matching point pairs:

For the matching point pairs searched in the above steps, the following filter conditions must also be passed, that is, the value of the point in the feature map (feature map) must be large enough:

in,

is the final set of matching point pairs, and γ is the custom threshold.

get

After that, the upper-level search range set is obtained by the following formula,

in,

for

quantity,

is the receptive field of the neural network of level l-1 relative to level l, and the coordinates of point p are (p _x , p _y , p _z ). And after getting the upper-level search range set, repeat the above steps to get the final output result

That is, the set of matching point pairs of the two images.

In a preferred embodiment, the image registration method based on marker points further includes: by minimizing the sum of the distances between all matching point pairs, fitting the transformation matrix and displacement vector of rigid registration, thereby obtaining rigid registration The warped image of the final medical image includes the following steps: After obtaining all matching point pairs, the optimal solution of the transformation matrix and displacement vector of rigid registration is obtained by minimizing the following formula:

The optimal solution is:

R＝(P ^T P) ^-1 P ^T Q

A=R[0:3,0:3]

b=R[0:3,3]

Among them, N is the number of matching point pairs, p _n is the nth matching point of the fixed image, and q _n is the pixel point in the corresponding moving image. P is a matrix composed of all matching points of the fixed image, with a size of [N,4], that is, a matrix composed of N four-dimensional row vectors. The first three dimensions of the four dimensions are the physical coordinates of the pixels, and the fourth dimension is a fixed value of 1. Q is a matrix composed of all matching points of the moving image, with a size of [N, 4]. The size of matrix R is [4,4], R[0:3,0:3] refers to the matrix of size [3,3] composed of the first 3 rows and first 3 columns of matrix R, R[0:3 ,3] refers to the three-dimensional column vector of the first 3 rows and the 3rd column of the matrix R. A and b are the optimal solution of transformation matrix and displacement vector respectively. And finally, the warped image of the medical image after rigid registration is obtained through A and b.

In a preferred embodiment, the marker-based image registration method further includes: on the basis of rigid registration, obtain the three-dimensional displacement field matrix of non-rigid registration by interpolation method based on radial basis, so as to obtain the non-rigid registration The warped image of the medical image after registration includes the following steps: the size of the three-dimensional matrix of the displacement field is the same as that of the fixed image. After obtaining N matching point pairs, use the following interpolation method to obtain the value of the remaining pixels of the displacement field matrix:

A＝(a ₁ ,a ₂ ,a ₃ )

G(r)＝r ² lnr

Where p is the pixel whose coordinates are (x _p , y _p , z _p ) in the three-dimensional matrix of the displacement field, and p _n is the nth matching point in the fixed image. G() is the radial basis function. The values of A, b, w _n are solved in the following way:

Assume:

r _ij =‖ p _i -p _j ‖

V=(v ₁ ,v ₂ ,...,v _N ,0,0,0,0)

v _n ＝(q _n -p _n )[k]k∈(0,1,2)

Ω＝(w ₁ ,w ₂ ,...,w _N ,b,a ₁ ,a ₂ ,a ₃ )

Among them, P is a matrix composed of all matching points of the fixed image, with a size of [N, 4], that is, a matrix composed of n four-dimensional row vectors. The last three dimensions of the four dimensions are the physical coordinates of the pixels, and the first dimension is a fixed value of 1. q _n is the corresponding matching point of p _n in the moving image. k is the dimension, that is, because the displacement value is a three-dimensional vector (x, y, z direction), the above solution process is only for one of the dimensions, so when k=0, the x axis is taken, so when k=1, the y axis is taken, so k = 2, take the z axis.

From v _n =f(p _n ), we have:

V＝ ^LΩT

And then find the value of all parameters to be solved:

Ω＝(L ^- 1V) ^T

The displacement values of the pixel points in the displacement field except the matching points can be obtained by f() fitting. Since the displacement value is a three-dimensional vector, that is, the x, y, and z directions, the above interpolation process needs to be repeated three times, that is, each direction is performed once. And finally obtain the three-dimensional matrix of the displacement field, so as to obtain the warped image of the medical image after non-rigid registration.

Compared with the prior art, the marker-based image registration method of the present invention has the following beneficial effects: (1) The present invention can register images of any two modalities. (2) The present invention adopts a pre-trained neural network to extract image features. The training process of this network includes multiple different tasks and involves multiple different input modes, which can effectively improve the effectiveness and versatility of features. (3) The present invention utilizes a pre-trained neural network to extract image features, and obtains multiple matching point pairs representing certain semantics between two images through processes such as search, screening, and matching, which can effectively solve the problem of lack of gold standard for marking points question. (4) The present invention realizes rigid registration by minimizing the sum of the distances between all matching point pairs and solving the transformation matrix and displacement vector. (5) The present invention solves the three-dimensional matrix of the displacement field through the interpolation method based on the radial basis to realize non-rigid registration.

Description of drawings

Fig. 1 is a schematic flow chart of an image registration method based on markers according to an embodiment of the present invention.

Detailed ways

The specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

Unless expressly stated otherwise, throughout the specification and claims, the term "comprise" or variations thereof such as "includes" or "includes" and the like will be understood to include the stated elements or constituents, and not Other elements or other components are not excluded.

As shown in Figure 1, a kind of image registration method based on marker point according to the preferred embodiment of the present invention, comprises the following steps:

Input two medical images of any modality (CT, CBCT, MRI, PET, etc.), one as a fixed image and the other as a moving image;

Pyramid features of two input images are extracted using a pre-trained neural network, which is trained on a variety of tasks and involves the various input modalities mentioned above;

Use the above neural network to extract pyramid features, and obtain multiple matching point pairs representing certain semantics between two images through searching, screening, matching and other processes;

By minimizing the sum of the distances between all matching point pairs, the transformation matrix and displacement vector for rigid registration are fitted to obtain the warped image of the medical image after rigid registration.

On the basis of rigid registration, the three-dimensional matrix of the displacement field of non-rigid registration is obtained by interpolation method based on radial basis, so as to obtain the warped image of the medical image after non-rigid registration.

A specific implementation workflow of a marker-based image registration method of the present invention includes:

In some embodiments, step S1, constructing a pre-trained neural network to extract the pyramid features of two input images;

Step S1 specifically includes the following steps:

S11. The structure of the neural network is divided into a backbone network and a plurality of subsequent branch networks. The backbone network is shared among different tasks, and each branch network corresponds to a task. Finally, the backbone network is used to extract image features.

S12. The training process of the neural network involves a variety of different tasks and involves a variety of different input modalities, including but not limited to: CT-based nasopharyngeal carcinoma primary tumor (GTV) segmentation, MRI-based nasopharyngeal carcinoma primary Tumor segmentation, CT-based cervical cancer primary tumor segmentation, PET-based lung primary tumor segmentation, CT-based organ-at-risk (OAR) segmentation, MRI-based organ-at-risk segmentation, CBCT-based organ-at-risk segmentation, CT-based Pulmonary nodule target detection, etc.

S13. First use one of the tasks to train the neural network, and then train each of the other input modal tasks at the same time, and then train each of the remaining tasks separately. The parameters of the backbone network are fixed during training, and finally all tasks are trained and fine-tuned at the same time. All parameters.

In some embodiments, the image registration method based on markers also includes:

S2. Utilize the above-mentioned neural network to extract pyramid features, and obtain multiple matching point pairs representing certain semantics between the two images through processes such as searching, screening, and matching;

Step S2 specifically includes the following steps:

S21, input I _f (fixed image) and I _m (moving image) into the above-mentioned pre-trained neural network, and extract pyramidal feature maps (feature maps) of two input images

and

Among them, l∈{1,2,3,4,5} represents the l-level feature, and the larger the number, the deeper the layer, that is, the smaller the feature size is, the more high-level semantics it contains.

S22, the search for matching points needs to be generated within a specific search range, and the search range is set from l=5:

S ⁵ ＝{(P ⁵ ,Q ⁵ )}

in:

is the nth search range of the lth level of _If , correspondingly

and

The entire range of , that is, N ⁵ =1.

S23, through the following formula to search range

and

feature map of

and

transform to get

and

in,

expressed in

Local feature maps in the range,

is the transformed feature map,

for

the mean value of

for

standard deviation of

S24. In the search range

and

That is to say, if

The point p ^l inside

S25, all the ^N1 search ranges of the first level perform the above steps S23-S24 of searching for matching points respectively, and then obtain the set ^Λl of all matching point pairs:

S26. For the matching point pair obtained by searching in the above steps, the following screening conditions must also be passed, that is, the value of the point in the feature map (feature map) must be large enough:

in,

is the final set of matching point pairs, and γ is a custom threshold, which is 0.05 here.

S27, get

in,

for

quantity,

is the receptive field of the neural network of level l-1 relative to level l, and the coordinates of point p are (p _x , p _y , p _z ).

S28. After obtaining the upper-level search range set, for each search range, jump to step S23 and repeat steps S23-S28 to obtain the final output result

That is, the set of matching point pairs of the two images.

S3, by minimizing the sum of the distances between all matching point pairs, fitting the transformation matrix and displacement vector of the rigid registration, thereby obtaining the warped image of the medical image after the rigid registration;

Step S3 specifically includes the following steps:

S31. After all matching point pairs are obtained, the optimal solution of the transformation matrix and displacement vector for rigid registration is obtained by minimizing the following formula:

The optimal solution is:

R＝(P ^T P) ^-1 P ^T Q

A=R[0:3,0:3]

b=R[0:3,3]

Among them, N is the number of matching point pairs, p _n is the nth matching point of the fixed image, and q _n is the pixel point in the corresponding moving image. P is a matrix composed of all matching points of the fixed image, with a size of [N,4], that is, a matrix composed of N four-dimensional row vectors. The first three dimensions of the four dimensions are the physical coordinates of the pixels, and the fourth dimension is a fixed value of 1. Q is a matrix composed of all matching points of the moving image, with a size of [N, 4]. The size of matrix R is [4,4], R[0:3,0:3] refers to the matrix of size [3,3] composed of the first 3 rows and first 3 columns of matrix R, R[0:3 ,3] refers to the three-dimensional column vector of the first 3 rows and the 3rd column of the matrix R. A and b are the optimal solution of transformation matrix and displacement vector respectively.

S32. Finally, the warped image of the medical image after rigid registration is obtained through A and b.

S4. On the basis of rigid registration, the three-dimensional matrix of the displacement field of non-rigid registration is obtained by interpolation method based on radial basis, so as to obtain the warped image of the medical image after non-rigid registration;

Step S4 specifically includes the following steps:

S41. The size of the displacement field three-dimensional matrix is the same as that of the fixed image. After obtaining N matching point pairs, use the following interpolation method to obtain the value of the remaining pixels of the displacement field matrix:

A＝(a ₁ ,a ₂ ,a ₃ )

G(r)＝r ² lnr

Assume:

r _ij =‖ p _i -p _j ‖

V=(v ₁ ,v ₂ ,...,v _N ,0,0,0,0)

v _n ＝(q _n -p _n )[k]k∈(0,1,2)

Ω＝(w ₁ ,w ₂ ,...,w _N ,b,a ₁ ,a ₂ ,a ₃ )

From v _n =f(p _n ), we have:

V＝ ^LΩT

And then find the value of all parameters to be solved:

Ω＝(L ^- 1V) ^T

The displacement values of the pixel points in the displacement field except the matching points can be obtained by f() fitting.

S42. Since the displacement value is a three-dimensional vector, that is, the x, y, and z directions, the above interpolation process needs to be repeated three times, that is, step S41 is performed once for each direction.

S43. Finally, the three-dimensional matrix of the displacement field is obtained, so as to obtain the warped image of the medical image after non-rigid registration.

In summary, the marker-based image registration method of the present invention has the following advantages: (1) The present invention can register images of any two modalities. (2) The present invention adopts a pre-trained neural network to extract image features, and the training process of this network includes multiple different tasks and involves multiple different input modes, which can effectively improve the effectiveness and versatility of features. (3) The present invention utilizes a pre-trained neural network to extract image features, and obtains multiple matching point pairs representing certain semantics between two images through processes such as search, screening, and matching, which can effectively solve the problem of lack of gold standard for marking points question. (4) The present invention realizes rigid registration by minimizing the sum of the distances between all matching point pairs and solving the transformation matrix and displacement vector. (5) The present invention solves the three-dimensional matrix of the displacement field through the interpolation method based on the radial basis to realize non-rigid registration.

The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to make and use various exemplary embodiments of the invention, as well as various Choose and change. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims

A method for image registration based on markers, comprising the following steps:

Input two medical images of any modality, one as fixed image and the other as moving image;

Adopting a pre-trained neural network to extract the pyramidal features of the medical image of two input arbitrary modalities, the training process of the neural network includes multiple tasks and involves multiple input modalities;

Using the pyramid features extracted by the neural network to obtain a plurality of matching point pairs representing certain semantics between the two images through searching, screening and matching processes;

By minimizing the sum of the distances between all matching point pairs, the transformation matrix and displacement vector of rigid registration are fitted, so as to obtain the warped image of the medical image after rigid registration; and

On the basis of rigid registration, the three-dimensional matrix of displacement field of non-rigid registration is obtained by interpolation method based on radial basis, so as to obtain the medical image warped image after non-rigid registration;

Wherein said using a pre-trained neural network to extract the pyramidal features of the medical images of two input arbitrary modalities includes: the structure of the neural network is divided into a backbone network and a plurality of subsequent branch networks; The backbone network is shared between different tasks, and each of the branch networks corresponds to a task; the backbone network is finally used to extract image features; and

First use the neural network trained by one of the tasks, and then train each of the other input modal tasks at the same time, and then train the remaining tasks separately. During training, the backbone network parameters are fixed, and finally all tasks Simultaneously train and fine-tune all parameters.
The image registration method based on markers according to claim 1, wherein the pyramid feature of the medical image of the arbitrary modality of the two input is extracted by using a pre-trained neural network, Also includes:

The training process of the neural network includes a variety of different tasks and involves a variety of different input modalities, including: CT-based nasopharyngeal carcinoma primary tumor segmentation, MRI-based nasopharyngeal carcinoma primary tumor segmentation, CT-based Cervical cancer primary tumor segmentation, PET-based lung primary tumor segmentation, CT-based organ-at-risk segmentation, MRI-based organ-at-risk segmentation, CBCT-based organ-at-risk segmentation, and CT-based pulmonary nodule target detection.
The image registration method based on markers according to claim 1, wherein the pyramid feature extracted by the neural network is used to obtain the representation of certain semantics between the two images through searching, screening and matching processes. Multiple matching point pairs, including the following steps:

Input If and Im into the above-mentioned pre-trained neural network, and extract the pyramidal feature maps of the two input images
and
Among them, l∈{1,2,3,4,5} represents the l-level feature, and the larger the number, the deeper the layer, that is, the smaller the feature size but contains more high-level semantics; where I f is the fixed image, I m is the moving image;

The search for matching points needs to be generated within a specific search range, and the search range is set starting from l=5:

S 5 ＝{(P 5 ,Q 5 )}

in:

is the nth search range of the lth level of If , correspondingly
is the nth search range of the lth level of I m , N1 is the number of the search range of the lth level, and S1 is the collection of multiple search range pairs of the lth level; when l=5, the search range Yes
and
The entire range of , that is, N 5 =1;

Search range by the following formula
and
feature map of
and
transform to get
and

in,
expressed in
Local feature maps in the range,
is the transformed feature map,
for
the mean value of
for
standard deviation of

in the search range
and
Search for matching point pairs within, when the following conditions are met, the two points p l and q l are matching point pairs:

That is to say, if
The point p l inside
When searching within the range, the point with the highest similarity is q l , and vice versa, then p l and q l are matching point pairs; the calculation formula of similarity is:

Among them, ε(p l ) is the point set of the neighborhood within a specific range of point p l ;

All the N1 search ranges of the lth level perform the above-mentioned steps of searching for matching points respectively, and then obtain the set Λl of all matching point pairs:

For the matching point pairs searched in the above steps, the following filter conditions must also be passed, that is, the value of the point in the feature map must be large enough:

in,
is the final set of matching point pairs, and γ is the custom threshold;

get
After that, the upper-level search range set is obtained by the following formula:

in,
for
quantity,
is the neural network receptive field of level l-1 relative to level l, the coordinates of point p are (p x , p y , p z ), and ε(q l ) is a point in the neighborhood of point q l within a specific range set, ε(q l ) and ε(p l ) are the corresponding concepts, p l and q l are the matching point pairs of the two graphs of level l; and

After obtaining the search scope set of the upper level, repeat the above steps to obtain the final output result
That is, the set of matching point pairs of the two images.
The image registration method based on marker points according to claim 1, characterized in that, by minimizing the sum of the distances between all matching point pairs, the transformation matrix and displacement vector of rigid registration are fitted to obtain rigid registration The warped image of the medical image after the standard includes the following steps:

After all matching point pairs are obtained, the optimal solution of the transformation matrix and displacement vector for rigid registration is obtained by minimizing the following formula:

The optimal solution is:

R＝(P T P) -1 P T Q

A=R[0:3,0:3]

b=R[0:3,3]

Among them, N is the number of matching point pairs, p n is the nth matching point of the fixed image, q n is the pixel in the corresponding moving image; P is a matrix composed of all matching points of the fixed image, and the size is [N, 4], which is a matrix composed of N four-dimensional row vectors. The first three dimensions of the four dimensions are the physical coordinates of the pixels, and the fourth dimension is a fixed value of 1; Q is a matrix composed of all matching points of the moving image, with a size of [N,4] ;The size of the matrix R is [4,4], R[0:3,0:3] refers to the matrix whose size is [3,3] composed of the first 3 rows and the first 3 columns of the matrix R, R[0: 3,3] refers to the three-dimensional column vector of the first 3 rows and the 3rd column of the matrix R; A and b are the optimal solutions of the transformation matrix and the displacement vector respectively; and

Finally, the warped image of the medical image after rigid registration is obtained through A and b.
The image registration method based on marker points according to claim 1, characterized in that, on the basis of rigid registration, a three-dimensional displacement field matrix of non-rigid registration is obtained by interpolation method based on radial basis, thereby obtaining non-rigid registration. The medical image warped image after rigid registration includes the following steps:

The size of the three-dimensional matrix of the displacement field is the same as that of the fixed image; after obtaining N matching point pairs, the following interpolation method is used to obtain the value of the remaining pixels of the displacement field matrix:

A＝(a 1 ,a 2 ,a 3 )

G(r)＝r 2 ln r

Where p is the pixel whose coordinates are (x p , y p , z p ) in the three-dimensional matrix of the displacement field, p n is the nth matching point in the fixed image; G( ) is the radial basis function; A, b The values of , w n are solved in the following way:

Assume:

r ij =||p i -p j ||

V=(v 1 ,v 2 ,...,v N ,0,0,0,0)

v n ＝(q n -p n )[k]k∈(0,1,2)

Ω＝(w 1 ,w 2 ,...,w N ,b,a 1 ,a 2 ,a 3 )

Among them, P is a matrix composed of all matching points of the fixed image, the size is [N, 4], that is, a matrix composed of n four-dimensional row vectors, the last three dimensions of the four dimensions are the physical coordinates of the pixels, and the first dimension is a fixed value 1; q n is the matching point corresponding to p n in the moving image; k is the dimension, that is, since the displacement value is a three-dimensional vector (x, y, z direction), the above solution process is only for one of the dimensions, so when k=0, take x-axis, so take the y-axis when k=1, so take the z-axis when k=2;

From v n =f(p n ), we have:

V＝ LΩT

And then find the value of all parameters to be solved:

Ω＝(L - 1V) T

The displacement values of the pixel points except the matching point in the displacement field can be obtained by f( ) fitting;

Since the displacement value is a three-dimensional vector, i.e. in the x, y, and z directions, the above interpolation process needs to be repeated 3 times, that is, once in each direction; and

Finally, the three-dimensional matrix of the displacement field is obtained, so as to obtain the warped image of the medical image after non-rigid registration.