WO2022178997A1

WO2022178997A1 - Medical image registration method and apparatus, computer device, and storage medium

Info

Publication number: WO2022178997A1
Application number: PCT/CN2021/096702
Authority: WO
Inventors: 李雷来; 王健宗; 瞿晓阳
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-02-25
Filing date: 2021-05-28
Publication date: 2022-09-01
Also published as: CN112837357A; CN112837357B

Abstract

A medical image registration method and apparatus, a computer device, and a storage medium. The method comprises: obtaining a first medical image set and a second medical image set (S10); after inputting the first medical image set into a first registration transformation model, obtaining a first registered image outputted by the first registration transformation model for multi-view-angle fusion of the first medical image set (S20); after inputting the second medical image set into a second registration transformation model, obtaining a second registered image outputted by the second registration transformation model for multi-time-sequence fusion of the second medical image set (S30); and fusing the first registered image and the second registered image to obtain a third registered image (S40). The method replaces manual determination by means of an artificial intelligence model, thereby improving the determination rate and the determination accuracy.

Description

Medical image registration method, device, computer equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202110210897.0 and the invention titled "Medical Image Registration Method, Device, Computer Equipment and Storage Medium", which was filed with the China Patent Office on February 25, 2021, the entire content of which is approved by Reference is incorporated in this application.

technical field

The present application relates to the field of AI medical technology in digital medical treatment, and is applied to smart medical treatment in digital medical treatment, and particularly discloses a medical image registration method, device, computer equipment and storage medium.

Background technique

As one of the important research fields of artificial intelligence technology, medical technology has played a huge role in medical imaging, clinical decision support, drug research and development, and pathology. The integration of medical technology and technology can help doctors make diagnoses quickly and accurately , and truly achieve the medical goal of "early diagnosis, early treatment, and early recovery". Among them, medical image registration has great practical value in medical technology.

At present, with the continuous updating and development of medical imaging equipment, for the same patient, multiple imaging techniques such as CT and MRI can be used to collect the patient's pathological cell structure information from multiple angles and multiple aspects. The inventor found that the traditional diagnostic methods in the past often required some experienced doctors to make relevant prediction results through the observation of different images and comprehensive analysis of various aspects, as well as combining subjective experience and spatial imagination. Therefore, the inventor discovered the prior art. It still relies too much on the subjective experience and spatial imagination of doctors to make judgments, and the existing technology may have the problem of low judgment speed and judgment accuracy.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a medical image registration method, device, computer equipment and storage medium for the above-mentioned technical problems, which can be used to replace manual judgment with an artificial intelligence model and improve the judgment rate and judgment accuracy.

A medical image registration method, comprising:

Acquiring a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes multiple second medical images from the same viewing angle at different times Medical Imaging;

After the first medical image set is input into the first registration transformation model, a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set is obtained;

After the second medical image set is input into the second registration transformation model, a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set is obtained;

The first registration image and the second registration image are fused to obtain a third registration image; the third registration image reflects the first medical image set and the second medical image set growth tracking results.

A medical image registration device, comprising:

a first acquisition module, configured to acquire a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes different times Multiple second medical images from the same viewing angle;

The second acquisition module is configured to, after the first medical image set is input into the first registration transformation model, obtain the first registration transformation model for performing multi-view fusion on the first medical image set outputted by the first registration transformation model. a registered image;

The third acquiring module is configured to acquire the first multi-sequence fusion of the second medical image set output from the second registration transformation model after the second medical image set is input into the second registration transformation model. Two registered images;

a fusion module, configured to fuse the first registration image and the second registration image to obtain a third registration image; the third registration image reflects the first medical image set and the Growth tracking results for the second set of medical images.

A computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer-readable instructions:

One or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

The above-mentioned medical image registration method, apparatus, computer equipment and storage medium are used to obtain a first medical image in a first medical image set containing specific conditions (different viewing angles at the same time) and a second medical image set containing specific conditions (same viewing angle at different times) The second medical image contains features in the second medical image, and the corresponding first medical image and the second medical image are processed by two registration transformation models respectively to perform image registration, which can improve the model's processing ability for specific data and improve Data processing efficiency; then, a plurality of first medical images under specific conditions are fused into a first registration image through the first image interpolation module, and multiple second medical images under specific conditions are fused into a first registration image through the second image interpolation module. Second, register images, process images from different perspectives and at different times to obtain all image information under each image; then fuse the first and second registration images to obtain a more comprehensive The third registration image of the image information (growth tracking results); all in all, this method uses the artificial intelligence model for processing, and abandons the phenomenon that relies too much on the doctor's subjective experience and spatial imagination before (manual participation in the work is too much, and there is also too much Relying on manual experience), assisting the doctor to perform reference processing, improving the judgment rate and judgment accuracy.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. , for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic diagram of an application environment of a medical image registration method in an embodiment of the present application;

2 is a flowchart of a medical image registration method in an embodiment of the present application;

3 is a schematic structural diagram of a medical image registration device in an embodiment of the present application;

4 is a training flow chart of the first standard training image in an implementation of the present application;

Fig. 5 is the processing flow chart of the medical image set in one implementation of the present application;

FIG. 6 is a schematic diagram of a computer device in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The medical image registration method provided by the present application can be applied in the application environment as shown in FIG. 1 , wherein the client communicates with the server through the network. Wherein, the client may include but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2 , a medical image registration method is provided, which is described by taking the method applied to the server in FIG. 1 as an example, including the following steps:

S10: Acquire a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes multiple medical images from the same viewing angle at different times second medical image;

Understandably, there are multiple first medical images in the first medical image set, the first medical image may be a common 2D image, such as a B-ultrasound image, and the second medical image is consistent with the first medical image set, wherein the first medical image is The image and the second medical image are collected by medical imaging equipment (such as CT equipment, MRI equipment or B-ultrasound equipment, etc.), the first medical image reflects the imaging results of cell tissue results from different perspectives at the same time, and the second medical image reflects The imaging results of the cell tissue results from the same viewing angle at different times, and the first medical image and the second medical image are both for the same pathological growth site, such as the tumor growth site, when the growth site is the tumor growth site, the first medical image Both the image and the second medical image are associated with the tumor growth site.

S20, after the first medical image set is input into the first registration transformation model, obtain a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set;

Understandably, the first registration transformation model is composed of a convolutional neural network + an image interpolation module. The convolutional neural network is a deep feedforward artificial neural network, which is applied to image recognition. The first registration transformation model is mainly used for processing. The first medical image set, in which the most basic four layers can be set in the convolutional neural network model, namely convolutional layer, pooling layer, linear rectification layer and fully connected layer. The convolutional layer is used to extract the first Different features in a medical image, the pooling layer is used to optimize the features in the convolution layer to obtain new features with smaller dimensions, the linear rectification layer uses the activation function to perform linear rectification, and the fully connected layer is used to All the features are converted into global features, and the score of the category to which the first medical image belongs is calculated; A first deformation field image is generated from a plurality of first medical images, and after image interpolation is performed on the first deformation field image, a first registration image can be obtained; in this embodiment, the first medical images in the first medical image set can be fused into one. The first registered image.

S30, after inputting the second medical image set into the second registration transformation model, obtain a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set;

Understandably, the composition of the second registration transformation model is the same as that of the first registration transformation model, the difference is that the second registration transformation model processes the second medical image set. It can be seen that in the second registration transformation model, the The data trained by the convolutional neural network is different from the convolutional neural network in the first registration transformation model, and the network structure between the two can be exactly the same; in this embodiment, the second medical image in the second medical image set can be processed. fused into a second registered image.

S40, fuse the first registration image and the second registration image to obtain a third registration image; the third registration image reflects the first medical image set and the second medical image Growth tracking results for the image set.

Understandably, after obtaining the first registration image and the second registration image according to the above two registration transformation models, image fusion is performed on the first registration image and the second registration image through feature-level image fusion to obtain the first registration image and the second registration image. Three registration images, specifically, extract feature information (such as edge, shape, hub and local feature information) in the first registration image and the second registration image, and perform comprehensive processing on the extracted feature information. The comprehensive processing includes Either of target state feature fusion or target feature fusion, through target state feature fusion, target feature statistics are performed on the first registration image associated with different viewing angles at the same time and the second registration image at the same viewing angle at different times. The target features extracted by the user are strictly registered to obtain a third registration image containing more image features, and the feature vectors in the first registration image and the second registration image are recombined through target feature fusion, adding two The image feature dimension between the two is obtained to obtain a third registration image, wherein the third registration image can fuse multi-view and multi-sequence images, so the third registration image can complete the pathological growth tracking result; After this embodiment, the comprehensive transformation of the growth tracking result can be fed back to the preset data receiver.

In the embodiment in which steps S10 to S40 are located, a first medical image containing a specific condition (different views at the same time) in the first medical image set and a second medical image containing a specific condition (same view at different times) in the second medical image set are acquired For the second medical image containing features, two registration transformation models are used to process the corresponding first medical image and the second medical image respectively to perform image registration, which can improve the model's processing capability for specific data and improve data processing efficiency; then The first image interpolation module fuses multiple first medical images under specific conditions into one first registration image, and the second image interpolation module fuses multiple second medical images under specific conditions into one second registration image, The images at different viewing angles and at different times are processed separately to obtain all image information under each image; then the first registration image and the second registration image are fused to obtain more comprehensive image information (growth tracking). Results) of the third registration image; all in all, this application uses an artificial intelligence model for processing, assisting in the processing of doctor's documents, and improving the judgment rate and judgment accuracy. The present application can be applied in the field of smart medical care, thereby promoting the construction of smart cities.

Further, as shown in FIG. 5 , after obtaining a medical image set about a tumor patient through a medical imaging device, the medical image set is divided according to the relationship between the viewing angle and the time to obtain the first medical image set and the second medical image set, and then the first medical image set is obtained. After a medical image set and a second medical medical set are respectively input to the first registration network and the second registration network, the first registration image including the comprehensive situation and the second registration image including the change situation are obtained, and finally through fusion The two images resulted in a third registration image containing tumor growth tracking.

Further, the first registration transformation model includes a first convolutional neural network and a first image interpolation module, and after the first medical image set is input into the first registration transformation model, the first registration transformation model is obtained. The first registration image output by the registration transformation model for performing multi-view fusion on the first medical image set includes:

acquiring a plurality of the first medical images in the first medical image set;

After inputting all the first medical images into the first convolutional neural network, acquiring a plurality of first deformation field images generated after deforming each of the first medical images;

After performing image interpolation on all the first deformation field images through the first image interpolation module, a first registration image is obtained.

Understandably, the first registration transformation model in this embodiment includes an image interpolation module, where the purpose of image interpolation is to transform multiple deformation fields into the same image space, and the specific formula is as follows; A first medical image corresponding to a deformation field, and the deformation field includes image features obtained by processing the first medical image by a convolutional neural network.

Further, as shown in FIG. 4 , after the first medical image set is input into the first registration transformation model, the output of the first registration transformation model is obtained to perform multiple operations on the first medical image set. Before the first registration image for perspective fusion, it also includes:

Obtain a first standard training image and a first reference image, use a first standard training image and a first reference image as a pair of first registration After the first registration pair is deformed, a plurality of first training deformation fields are obtained; one of the first training deformation fields corresponds to a piece of the first standard training image;

After the first training deformation field is subjected to image interpolation by the first image interpolation module, a first interpolation result is obtained;

After the first interpolation result and the first reference image are input into the first preset loss function associated with the first convolutional neural network, the first loss of the first training deformation field and the first reference image is obtained value;

When the first loss value is greater than a first preset loss value, the network parameters of the first convolutional neural network are continuously updated through a back-propagation algorithm until the first loss value is less than or equal to the first preset loss value loss value;

When the first loss value is less than or equal to the first preset loss value, it is determined that the training of the first registration transformation model is completed, and the first registration transformation model is obtained.

Understandably, the first standard training images can be arranged in an image sequence, and the image sequence is Im ( _m =1, 2...n), each serial number represents the first standard training images from different perspectives at the same time, and the first reference The image (which can be understood as a standard image under one viewing angle) is represented by I ₀ , and a first reference image corresponds to a first standard training image of one viewing angle; the first training deformation field only has a corresponding relationship with the first standard training image; this implementation The example is the training method of the model, through which the loss function (f _m is the first training deformation field, assuming that the first interpolation result is I(f _m ), I(f _m ) contains the first training deformation field The mathematical relationship with the first reference image is the result obtained by linear interpolation, the first loss function is L(m)=∑ _m |I ₀ -I(f _m )|, and the first loss function is used to calculate the first loss function. The value of the difference between a standard training image and the first parameter image), when it is determined that the first loss value is greater than the first preset loss value, the network parameters in the hidden layer (full connection layer) in the convolutional neural network are continuously updated (weight) until the first loss value is less than or equal to the first preset loss value; compared with the previous traditional training method, this embodiment does not need to extract image features first and then select the model to perform the cumbersome process of iteratively optimizing the model. Unsupervised learning of convolutional neural network models directly implements training.

Further, the second registration transformation model includes a second convolutional neural network and a second image interpolation module, and after the second medical image set is input into the second registration transformation model, the second registration transformation model is obtained. The second registration image output by the registration transformation model for performing multi-sequence fusion on the second medical image set includes:

acquiring a plurality of second registration pairs in the second medical image set; a second registration pair includes a second medical image and a second reference image;

After inputting all the second registration pairs to the second convolutional neural network, acquiring a plurality of second deformation fields generated after deforming each of the second registration pairs;

After all the second deformation fields are subjected to image interpolation by the second image interpolation module, the second registration image is obtained.

Understandably, for the specific content of this embodiment, reference may be made to the above-mentioned embodiment where the first registered image is located, and details are not described here.

Further, before the inputting all the second registration pairs to the first registration transformation model constructed by the convolutional neural network, the method further includes:

Acquire a second standard training image and a second reference image, use one of the second standard training image and one of the second reference images as a pair of second registration pairs, and use the second convolutional neural network for all the After the second registration pair is deformed, a plurality of second training deformation fields are obtained; one of the second training deformation fields corresponds to a piece of the second standard training image;

After the second training deformation field is subjected to image interpolation by the second image interpolation module, a second interpolation result is obtained;

After the second interpolation result and the second reference image are input to a second preset loss function associated with the second convolutional neural network, a second loss of the second training deformation field and the second reference image is obtained value;

When the second loss value is greater than a second preset loss value, the network parameters of the second convolutional neural network are continuously updated through a back-propagation algorithm until the second loss value is less than or equal to the second preset loss value loss value;

When the second loss value is less than or equal to the second preset loss value, it is determined that the training of the second registration transformation model is completed, and the second registration transformation model is obtained.

Understandably, the second standard training images can be arranged in an image sequence, where the image sequence is Im ' ( _m '=1, 2...n), each serial number represents the second standard training images of the same viewing angle at different times, the first Two reference images (which can be understood as a standard image at a moment) are represented by I ₀ ′, and a second reference image corresponds to a second standard training image at a moment; for the specific content of this embodiment, refer to the above-mentioned first registration transformation The model is consistent and will not be repeated here.

Further, after performing image interpolation on the first training deformation field through the first image interpolation module, a first interpolation result is obtained, including:

Transform the first standard training image in the first training deformation field into the image space of the first reference image in the first training deformation field by using the first transformation formula, so as to obtain the first standard training image contained in the same image space. A first interpolation result of a target registration image; wherein, the first transformation formula is:

X'=ax'+by', Y'=cx'+dy'

in:

x' is the abscissa of the pixel in the first standard training image;

y' is the ordinate of the pixel in the first standard training image;

X' is the abscissa of the pixel in the first reference image or the first target registration image;

Y' is the ordinate of the pixel in the first reference image or the first target registration image;

a, b, c and d are all transformation parameter values, and the transformation matrix formed by a, b, c and d

Understandably, the transformation matrix can be obtained in advance through multiple sets of measured abscissa and ordinate data before and after transformation.

Further, after all the first registration pairs are deformed by the first convolutional neural network, a plurality of first training deformation fields are obtained, including:

After all the first registration pairs are input into the convolutional neural network, all the first registration pairs are deformed by the first preset expression corresponding to the convolutional neural network to obtain the same Multiple first training deformation fields corresponding to the first registration pair; the first preset expression includes:

in:

x is the intensity of the first standard training image;

y is the intensity of the first reference image;

X is the value range of x; X is x1, x2...xn, etc.;

Y is the value range of y; y such as y1, y2...yn, etc.;

D _m is the similarity between the first standard training image and the first reference image pixel;

x _i is the intensity of the ith pixel of the first standard training image;

y _i is the intensity of the ith pixel of the first reference image;

x _m is the average intensity of the first standard training image;

y _m is the average intensity of the first reference image;

p(x) is the probability density distribution function of the first standard training image;

p(y) is the probability density distribution function of the first reference image;

p(x,y) is the joint probability density function of the first standard training image and the first reference image.

Further, after performing image interpolation on the second training deformation field through the second image interpolation module, a second interpolation result is obtained, including:

Transform the second standard training image in the second training deformation field into the image space of the second reference image in the second training deformation field by using the second transformation formula, so as to obtain the first standard training image contained in the same image space. The second interpolation result of the two-target registration image; wherein, the second transformation formula is:

X"=a'x"+b'y", Y"=c'x"+d'y"

in:

x" is the abscissa of the pixel in the second standard training image;

y" is the ordinate of the pixel in the second standard training image;

X" is the abscissa of the pixel in the second reference image or the second target registration image;

Y" is the ordinate of the pixel in the second reference image or the second target registration image;

a', b', c' and d' are all transformation parameter values, and the transformation matrix composed of a', b', c' and d'

Further, after all the second registration pairs are deformed by the second convolutional neural network, a plurality of second training deformation fields are obtained, including:

After all the second registration pairs are input into the convolutional neural network, all the second registration pairs are deformed by a second preset expression corresponding to the convolutional neural network to obtain the same Multiple second training deformation fields corresponding to the second registration pair; the second preset expression includes:

in:

x"' is the intensity of the second standard training image;

y"' is the intensity of the second reference image;

X"' is the value range of x"';

Y"' is the value range of y"';

D _m ' is the similarity between the second standard training image and the second reference image pixel;

x _i ' is the intensity of the i-th pixel of the second standard training image;

y _i ' is the intensity of the ith pixel of the second reference image;

x _m ' is the average intensity of the second standard training image;

y _m ' is the average intensity of the second reference image;

p(x"') is the probability density distribution function of the second standard training image;

p(y"') is the probability density distribution function of the second reference image;

p(x"', y"') is the joint probability density function of the second standard training image and the second reference image.

To sum up, a medical image registration method is provided above, and a first medical image containing a specific condition (different viewing angles at the same time) in the first medical image set and a second medical image set containing specific conditions (the same viewing angle at different times) are obtained. ) in the second medical image of the second medical image containing features, and processing the corresponding first medical image and the second medical image through two registration transformation models respectively to perform image registration, which can improve the model's processing capability for specific data, Improve data processing efficiency; then fuse multiple first medical images under specific conditions into a first registration image through a first image interpolation module, and fuse multiple second medical images under specific conditions into a single image through a second image interpolation module For the second registration image, the images from different perspectives and at different times are processed separately to obtain all the image information under each image; then the first registration image and the second registration image are fused to obtain a more The third registration image of comprehensive image information (growth tracking results); in a word, this method uses artificial intelligence model for processing, abandons the phenomenon that relies too much on the doctor's subjective experience and spatial imagination, and assists the doctor's software in reference processing, improving Judgment rate and judgment accuracy. The method can be applied in the field of smart medical care, thereby promoting the construction of smart cities.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

In one embodiment, a medical image registration apparatus is provided, and the medical image registration apparatus is in one-to-one correspondence with the medical image registration method in the above embodiment. As shown in FIG. 3 , the medical image registration device includes a first acquisition module 11 , a second acquisition module 12 , a third acquisition module 13 and a fusion module 14 . The detailed description of each functional module is as follows:

The first acquisition module 11 is configured to acquire a first medical image set and a second medical image set; the first medical image set includes a plurality of first medical images from different viewing angles at the same time; the second medical image set includes different Multiple second medical images at the same viewing angle at the same time;

The second acquisition module 12 is configured to acquire, after the first medical image set is input into the first registration transformation model, the output of the first registration transformation model for performing multi-view fusion on the first medical image set the first registered image;

The third acquisition module 13 is configured to acquire, after the second medical image set is input into the second registration transformation model, the output of the second registration transformation model for performing multi-sequence fusion on the second medical image set the second registration image;

The fusion module 14 is configured to fuse the first registration image and the second registration image to obtain a third registration image; the third registration image reflects the first medical image set and the Growth tracking results for the second set of medical images described.

Further, the second acquisition module includes:

a first acquisition submodule, configured to acquire a plurality of the first medical images in the first medical image set;

a second acquisition sub-module, configured to acquire a plurality of first deformation field images generated by deforming each of the first medical images after inputting all the first medical images into the first convolutional neural network;

The first image interpolation module sub-module is configured to obtain a first registration image after performing image interpolation on all the first deformation field images through the first image interpolation module.

Further, the medical image registration device further includes:

The first deformation module is used to obtain a first standard training image and a first reference image, and use one of the first standard training image and one of the first reference images as a pair of first registration pairs, through the first After the convolutional neural network deforms all the first registration pairs, a plurality of first training deformation fields are obtained; one of the first training deformation fields corresponds to one of the first standard training images;

a first image interpolation module, configured to obtain a first interpolation result after performing image interpolation on the first training deformation field through the first image interpolation module;

The fourth obtaining module is configured to obtain the first training deformation field and the first training deformation field after inputting the first interpolation result and the first reference image into the first preset loss function associated with the first convolutional neural network. a first loss value of the reference image;

A first update module, configured to continuously update the network parameters of the first convolutional neural network through a back-propagation algorithm when the first loss value is greater than a first preset loss value until the first loss value is less than or is equal to the first preset loss value;

A first determination submodule, configured to determine that the training of the first registration transformation model is completed when the first loss value is less than or equal to a first preset loss value, and obtain the first registration transformation model.

Further, the third acquisition module includes:

The third acquisition sub-module is configured to acquire a plurality of second registration pairs in the second medical image set; a second registration pair includes a second medical image and a second reference image ;

a fourth acquisition sub-module, configured to acquire a plurality of second deformation fields generated after deforming each of the second registration pairs after inputting all the second registration pairs into the second convolutional neural network;

The second image interpolation module sub-module is configured to obtain the second registration image after performing image interpolation on all the second deformation fields through the second image interpolation module.

Further, the medical image registration device further includes:

The second deformation module is configured to obtain a second standard training image and a second reference image, and use a second standard training image and a second reference image as a pair of second registration pairs. After the convolutional neural network deforms all the second registration pairs, a plurality of second training deformation fields are obtained; one of the second training deformation fields corresponds to a second standard training image;

a second image interpolation module, configured to obtain a second interpolation result after performing image interpolation on the second training deformation field through the second image interpolation module;

A fifth acquisition module, configured to acquire the second training deformation field and the first the second loss value of the second reference image;

a second update module, configured to continuously update the network parameters of the second convolutional neural network through a back-propagation algorithm when the second loss value is greater than a second preset loss value until the second loss value is less than or is equal to the second preset loss value;

A second determination module, configured to determine that the training of the second registration transformation model is completed when the second loss value is less than or equal to a second preset loss value, and obtain the second registration transformation model.

Further, the first image interpolation module module includes:

The transformation sub-module is used to transform the first standard training image in the first training deformation field into the image space of the first reference image in the first training deformation field through the first transformation formula, and obtain the image space contained in the first training deformation field. The first interpolation result of the first target registration image in the same image space; wherein, the first transformation formula is:

X'=ax'+by', Y'=cx'+dy'

in:

x' is the abscissa of the pixel in the first standard training image;

y' is the ordinate of the pixel in the first standard training image;

X' is the abscissa of the pixel in the first reference image;

Y' is the ordinate of the pixel in the first reference image;

Further, the first deformation module includes:

The fifth acquisition sub-module includes, after inputting all the first registration pairs into the convolutional neural network, performing registration on all the first registrations through a first preset expression corresponding to the convolutional neural network Deformation is performed on the pair to obtain a plurality of first training deformation fields corresponding to the first registration pair; the first preset expression includes:

in:

x is the intensity of the first standard training image;

y is the intensity of the first standard training image;

X is the value range of x;

Y is the value range of y;

x _i is the intensity of the ith pixel of the first standard training image;

y _i is the intensity of the ith pixel of the first standard training image;

x _m is the average intensity of the first reference image;

y _m is the average intensity of the first reference image;

For the specific limitations of the medical image registration device, reference may be made to the above limitations on the medical image registration method, which will not be repeated here. Each module in the above-mentioned medical image registration apparatus may be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 4 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes computer readable instructions, internal memory. The computer readable instructions store an operating system, computer readable instructions and a database. The internal memory provides an environment for the operating system in the computer readable instructions and the execution of the computer readable instructions. The database of the computer equipment is used to store multiple pieces of historical test data, and each piece of historical test data corresponds to a test problem record. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a medical image registration method.

In one embodiment, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored on the memory and executable on the processor, the processor performing the following when executing the computer-readable instructions step:

In one embodiment, one or more readable storage media storing computer-readable instructions are provided, and the readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media medium; computer-readable instructions are stored on the readable storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium or a volatile readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims

A medical image registration method, comprising:

Acquiring a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes multiple second medical images from the same viewing angle at different times Medical Imaging;

After the first medical image set is input into the first registration transformation model, a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set is obtained;

After the second medical image set is input into the second registration transformation model, a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set is obtained;

The first registration image and the second registration image are fused to obtain a third registration image; the third registration image reflects the first medical image set and the second medical image set growth tracking results.
The medical image registration method according to claim 1, wherein the first registration transformation model comprises a first convolutional neural network and a first image interpolation module, and the first medical image set is input to the first After a registration transformation model, acquiring a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set, including:

acquiring a plurality of the first medical images in the first medical image set;

After inputting all the first medical images into the first convolutional neural network, acquiring a plurality of first deformation field images generated after deforming each of the first medical images;

After performing image interpolation on all the first deformation field images through the first image interpolation module, a first registration image is obtained.
The medical image registration method according to claim 1, wherein after the first medical image set is input into the first registration transformation model, a pair of the first registration transformation model outputted by the first registration transformation model is obtained. Before performing the first registration image for multi-view fusion, a medical image set further includes:

Obtain a first standard training image and a first reference image, use a first standard training image and a first reference image as a pair of first registration After the first registration pair is deformed, a plurality of first training deformation fields are obtained; one of the first training deformation fields corresponds to a piece of the first standard training image;

After the first training deformation field is subjected to image interpolation by the first image interpolation module, a first interpolation result is obtained;

After the first interpolation result and the first reference image are input into the first preset loss function associated with the first convolutional neural network, the first loss of the first training deformation field and the first reference image is obtained value;

When the first loss value is greater than a first preset loss value, the network parameters of the first convolutional neural network are continuously updated through a back-propagation algorithm until the first loss value is less than or equal to the first preset loss value loss value;

When the first loss value is less than or equal to the first preset loss value, it is determined that the training of the first registration transformation model is completed, and the first registration transformation model is obtained.
The medical image registration method according to claim 1, wherein the second registration transformation model comprises a second convolutional neural network and a second image interpolation module, and the second medical image set is input to the first After the second registration transformation model, acquiring the second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set, including:

acquiring a plurality of second registration pairs in the second medical image set; a second registration pair includes a second medical image and a second reference image;

After inputting all the second registration pairs to the second convolutional neural network, acquiring a plurality of second deformation fields generated after deforming each of the second registration pairs;

After all the second deformation fields are subjected to image interpolation by the second image interpolation module, the second registration image is obtained.
The medical image registration method according to claim 4, wherein before inputting all the second registration pairs to the first registration transformation model constructed by a convolutional neural network, the method further comprises:

Acquire a second standard training image and a second reference image, use one of the second standard training image and one of the second reference images as a pair of second registration pairs, and use the second convolutional neural network for all the After the second registration pair is deformed, a plurality of second training deformation fields are obtained; one of the second training deformation fields corresponds to a piece of the second standard training image;

After the second training deformation field is subjected to image interpolation by the second image interpolation module, a second interpolation result is obtained;

After the second interpolation result and the second reference image are input to a second preset loss function associated with the second convolutional neural network, a second loss of the second training deformation field and the second reference image is obtained value;

When the second loss value is greater than a second preset loss value, the network parameters of the second convolutional neural network are continuously updated through a back-propagation algorithm until the second loss value is less than or equal to the second preset loss value loss value;

When the second loss value is less than or equal to the second preset loss value, it is determined that the training of the second registration transformation model is completed, and the second registration transformation model is obtained.
The medical image registration method according to claim 3, wherein the obtaining a first interpolation result after performing image interpolation on the first training deformation field by a first image interpolation module, comprising:

Transform the first standard training image in the first training deformation field into the image space of the first reference image in the first training deformation field by using the first transformation formula, so as to obtain the first standard training image contained in the same image space. A first interpolation result of a target registration image; wherein, the first transformation formula is:

X'=ax'+by', Y'=cx'+dy'

in:

x' is the abscissa of the pixel in the first standard training image;

y' is the ordinate of the pixel in the first standard training image;

X' is the abscissa of the pixel in the first reference image;

Y' is the ordinate of the pixel in the first reference image;

a, b, c and d are all transformation parameter values, and the transformation matrix formed by a, b, c and d
The medical image registration method according to claim 3, wherein after deforming all the first registration pairs through the first convolutional neural network, a plurality of first training deformation fields are obtained, comprising:

After all the first registration pairs are input into the convolutional neural network, all the first registration pairs are deformed by the first preset expression corresponding to the convolutional neural network to obtain the same Multiple first training deformation fields corresponding to the first registration pair; the first preset expression includes:

in:

x is the intensity of the first standard training image;

y is the intensity of the first reference image;

X is the value range of x;

Y is the value range of y;

D m is the similarity between the first standard training image and the first reference image pixel;

x i is the intensity of the ith pixel of the first standard training image;

y i is the intensity of the ith pixel of the first reference image;

x m is the average intensity of the first standard training image;

y m is the average intensity of the first reference image;

p(x) is the probability density distribution function of the first standard training image;

p(y) is the probability density distribution function of the first reference image;

p(x,y) is the joint probability density function of the first standard training image and the first reference image.
A medical image registration device, comprising:

a first acquisition module, configured to acquire a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes different times Multiple second medical images from the same viewing angle;

The second acquisition module is configured to, after the first medical image set is input into the first registration transformation model, obtain the first registration transformation model for performing multi-view fusion on the first medical image set outputted by the first registration transformation model. a registered image;

The third acquiring module is configured to acquire the first multi-sequence fusion of the second medical image set output from the second registration transformation model after the second medical image set is input into the second registration transformation model. Two registered images;

a fusion module, configured to fuse the first registration image and the second registration image to obtain a third registration image; the third registration image reflects the first medical image set and the Growth tracking results for the second set of medical images.
A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer-readable instructions:

Acquiring a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes multiple second medical images from the same viewing angle at different times Medical Imaging;

After the first medical image set is input into the first registration transformation model, a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set is obtained;

After the second medical image set is input into the second registration transformation model, a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set is obtained;

The first registration image and the second registration image are fused to obtain a third registration image; the third registration image reflects the first medical image set and the second medical image set growth tracking results.
10. The computer device of claim 9, wherein the first registration transformation model includes a first convolutional neural network and a first image interpolation module, the first set of medical images being input to the first registration After transforming the model, acquiring the first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set, including:

acquiring a plurality of the first medical images in the first medical image set;

After inputting all the first medical images into the first convolutional neural network, acquiring a plurality of first deformation field images generated after deforming each of the first medical images;

After performing image interpolation on all the first deformation field images through the first image interpolation module, a first registration image is obtained.
The computer device according to claim 9, wherein, after the first medical image set is input into the first registration transformation model, a pair of the first medical images output by the first registration transformation model is acquired. Before collecting the first registration image for multi-view fusion, the processor implements the following steps when executing the computer-readable instructions:

Obtain a first standard training image and a first reference image, use a first standard training image and a first reference image as a pair of first registration After the first registration pair is deformed, a plurality of first training deformation fields are obtained; one of the first training deformation fields corresponds to a piece of the first standard training image;

After the first training deformation field is subjected to image interpolation by the first image interpolation module, a first interpolation result is obtained;

After the first interpolation result and the first reference image are input into the first preset loss function associated with the first convolutional neural network, the first loss of the first training deformation field and the first reference image is obtained value;

When the first loss value is greater than a first preset loss value, the network parameters of the first convolutional neural network are continuously updated through a back-propagation algorithm until the first loss value is less than or equal to the first preset loss value loss value;

When the first loss value is less than or equal to the first preset loss value, it is determined that the training of the first registration transformation model is completed, and the first registration transformation model is obtained.
9. The computer device of claim 9, wherein the second registration transformation model includes a second convolutional neural network and a second image interpolation module, the second set of medical images being input to the second registration After transforming the model, acquiring a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set, including:

acquiring a plurality of second registration pairs in the second medical image set; a second registration pair includes a second medical image and a second reference image;

After inputting all the second registration pairs to the second convolutional neural network, acquiring a plurality of second deformation fields generated after deforming each of the second registration pairs;

After all the second deformation fields are subjected to image interpolation by the second image interpolation module, the second registration image is obtained.
13. The computer device of claim 12, wherein before said inputting all said second registration pairs to a first registration transformation model constructed by a convolutional neural network, said processor executes said computer-readable The following steps are implemented when reading the command:

Acquire a second standard training image and a second reference image, use one of the second standard training image and one of the second reference images as a pair of second registration pairs, and use the second convolutional neural network for all the After the second registration pair is deformed, a plurality of second training deformation fields are obtained; one of the second training deformation fields corresponds to a piece of the second standard training image;

After the second training deformation field is subjected to image interpolation by the second image interpolation module, a second interpolation result is obtained;

After the second interpolation result and the second reference image are input to a second preset loss function associated with the second convolutional neural network, a second loss of the second training deformation field and the second reference image is obtained value;

When the second loss value is greater than a second preset loss value, the network parameters of the second convolutional neural network are continuously updated through a back-propagation algorithm until the second loss value is less than or equal to the second preset loss value loss value;

When the second loss value is less than or equal to the second preset loss value, it is determined that the training of the second registration transformation model is completed, and the second registration transformation model is obtained.
The computer device according to claim 11, wherein, after performing image interpolation on the first training deformation field through a first image interpolation module, obtaining a first interpolation result, comprising:

Transform the first standard training image in the first training deformation field into the image space of the first reference image in the first training deformation field by using the first transformation formula, so as to obtain the first standard training image contained in the same image space. A first interpolation result of a target registration image; wherein, the first transformation formula is:

X'=ax'+by', Y'=cx'+dy'

in:

x' is the abscissa of the pixel in the first standard training image;

y' is the ordinate of the pixel in the first standard training image;

X' is the abscissa of the pixel in the first reference image;

Y' is the ordinate of the pixel in the first reference image;

a, b, c and d are all transformation parameter values, and the transformation matrix formed by a, b, c and d
One or more readable storage media storing computer-readable instructions, wherein the computer-readable instructions, when executed by one or more processors, cause the one or more processors to perform the following steps:

Acquiring a first medical image set and a second medical image set; the first medical image set includes multiple first medical images from different viewing angles at the same time; the second medical image set includes multiple second medical images from the same viewing angle at different times Medical Imaging;

After the first medical image set is input into the first registration transformation model, a first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set is obtained;

After the second medical image set is input into the second registration transformation model, a second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set is obtained;

The first registration image and the second registration image are fused to obtain a third registration image; the third registration image reflects the first medical image set and the second medical image set growth tracking results.
16. The readable storage medium of claim 15, wherein the first registration transformation model comprises a first convolutional neural network and a first image interpolation module, and wherein the first set of medical images is input to the first After registering the transformation model, acquiring the first registration image output by the first registration transformation model for performing multi-view fusion on the first medical image set, including:

acquiring a plurality of the first medical images in the first medical image set;

After inputting all the first medical images into the first convolutional neural network, acquiring a plurality of first deformation field images generated by deforming each of the first medical images;

After performing image interpolation on all the first deformation field images through the first image interpolation module, a first registration image is obtained.
The readable storage medium according to claim 15, wherein after the first medical image set is input into the first registration transformation model, a pair of the first registration transformation model outputted by the first registration transformation model is obtained. Before the medical image set performs the first registration image for multi-view fusion, when the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

Obtain a first standard training image and a first reference image, use a first standard training image and a first reference image as a pair of first registration After the first registration pair is deformed, a plurality of first training deformation fields are obtained; one of the first training deformation fields corresponds to a piece of the first standard training image;

After the first training deformation field is subjected to image interpolation by the first image interpolation module, a first interpolation result is obtained;

After the first interpolation result and the first reference image are input into the first preset loss function associated with the first convolutional neural network, the first loss of the first training deformation field and the first reference image is obtained value;

When the first loss value is greater than a first preset loss value, the network parameters of the first convolutional neural network are continuously updated through a back-propagation algorithm until the first loss value is less than or equal to the first preset loss value loss value;

When the first loss value is less than or equal to the first preset loss value, it is determined that the training of the first registration transformation model is completed, and the first registration transformation model is obtained.
16. The readable storage medium of claim 15, wherein the second registration transformation model comprises a second convolutional neural network and a second image interpolation module, the second set of medical images being input to the second After registering the transformation model, acquiring the second registration image output by the second registration transformation model for performing multi-sequence fusion on the second medical image set, including:

acquiring a plurality of second registration pairs in the second medical image set; a second registration pair includes a second medical image and a second reference image;

After inputting all the second registration pairs to the second convolutional neural network, acquiring a plurality of second deformation fields generated after deforming each of the second registration pairs;

After all the second deformation fields are subjected to image interpolation by the second image interpolation module, the second registration image is obtained.
19. The readable storage medium of claim 18, wherein the computer readable instructions are executed by a prior to inputting all of the second registration pairs to a first registration transformation model constructed by a convolutional neural network. When executed by one or more processors, the one or more processors are caused to perform the following steps:

Obtain a second standard training image and a second reference image, use one of the second standard training image and one of the second reference images as a pair of second registration pairs, and use the second convolutional neural network to register all the After the second registration pair is deformed, a plurality of second training deformation fields are obtained; one of the second training deformation fields corresponds to a piece of the second standard training image;

After the second training deformation field is subjected to image interpolation by the second image interpolation module, a second interpolation result is obtained;

After the second interpolation result and the second reference image are input to a second preset loss function associated with the second convolutional neural network, a second loss of the second training deformation field and the second reference image is obtained value;

When the second loss value is greater than a second preset loss value, the network parameters of the second convolutional neural network are continuously updated through a back-propagation algorithm until the second loss value is less than or equal to the second preset loss value loss value;

When the second loss value is less than or equal to the second preset loss value, it is determined that the training of the second registration transformation model is completed, and the second registration transformation model is obtained.
The readable storage medium according to claim 17, wherein, after performing image interpolation on the first training deformation field by a first image interpolation module, obtaining a first interpolation result, comprising:

Transform the first standard training image in the first training deformation field into the image space of the first reference image in the first training deformation field by using the first transformation formula, so as to obtain the first standard training image contained in the same image space. A first interpolation result of a target registration image; wherein, the first transformation formula is:

X'=ax'+by', Y'=cx'+dy'

in:

x' is the abscissa of the pixel in the first standard training image;

y' is the ordinate of the pixel in the first standard training image;

X' is the abscissa of the pixel in the first reference image;

Y' is the ordinate of the pixel in the first reference image;

a, b, c and d are all transformation parameter values, and the transformation matrix formed by a, b, c and d