CN114549594A

CN114549594A - Image registration method and device and electronic equipment

Info

Publication number: CN114549594A
Application number: CN202210162850.6A
Authority: CN
Inventors: 沈逸; 莫展豪; 柳林; 廖术; 隋赫
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2022-02-22
Filing date: 2022-02-22
Publication date: 2022-05-27

Abstract

The application relates to an image registration method and device and an electronic device. The method comprises the following steps: a plurality of initial medical images of a region of interest are acquired. And obtaining a first target deformation matrix according to a first image in the plurality of initial medical images. And registering a second image corresponding to the first image based on the first target deformation matrix. The second image is an image with the similarity greater than or equal to a first preset threshold value with the first image in the initial medical images. The method can solve the problem that the registration efficiency is very low due to redundant calculation in the existing registration process of each time sequence image.

Description

Image registration method and device and electronic equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image registration method, an image registration device, and an electronic device

Background

Medical imaging techniques refer to the process of acquiring images of internal tissues in a non-invasive manner in a region of interest of a target object in a medical or medical study. When the region of interest needs to be observed continuously, a time axis is added, and a plurality of internal tissue images scanned in the time axis are acquired according to a certain time sequence arrangement, which is also called a time sequence image. However, during the process of scanning the region of interest, due to various uncontrollable factors, the region of interest may generate a large or small offset motion, so that the multiple time-series images obtained on the time axis cannot be aligned, and therefore, the time-series images need to be registered.

In the current mainstream image registration technology, redundant calculation exists in the process of registering each time sequence image, so that the registration efficiency is very low.

Disclosure of Invention

Therefore, it is necessary to provide an image registration method, an image registration device and an electronic device for solving the problem of low registration efficiency caused by redundant calculation in the process of registering each time sequence image in the prior art.

In a first aspect, the present application provides a method of image registration, the method comprising:

a plurality of initial medical images of a region of interest are acquired.

And obtaining a first target deformation matrix according to a first image in the plurality of initial medical images.

And registering a second image corresponding to the first image based on the first target deformation matrix. The second image is an image with the similarity greater than or equal to a first preset threshold value with the first image in the initial medical images.

In a possible implementation manner, the obtaining a first target deformation matrix according to a first image of the plurality of initial medical images includes:

a reference image is determined among the plurality of initial medical images.

And registering each first image in the plurality of initial medical images based on the reference image to obtain a first target deformation matrix of each first image.

In one possible implementation, the determining a reference image in the plurality of initial medical images includes:

and acquiring a mask image corresponding to each initial medical image in the plurality of initial medical images.

And determining the reference image according to the plurality of mask images.

In one possible implementation manner, the determining the reference image according to the plurality of mask images includes:

and carrying out average processing on the plurality of mask images to obtain an average image.

And determining the target mask image with the highest coincidence degree with the average image in the plurality of mask images.

And determining an initial image corresponding to the target mask image as the reference image.

In a possible implementation manner, the registering, based on the reference image, each first image in the plurality of initial medical images to obtain a first target deformation matrix of each first image includes:

and registering the mask image of the first image according to the mask image of the reference image to obtain a plurality of first deformation matrixes.

And deforming the mask image of the first image according to the first deformation matrix to obtain the mask image of the first image after registration.

Determining a target mask image having a minimum difference from a mask image of the reference image among the plurality of registered mask images of the first image.

And determining a first deformation matrix used by the target mask image as the first target deformation matrix.

In one possible implementation, the registering the second image based on the first target deformation matrix includes:

and deforming the mask image of the second image according to the first target deformation matrix to obtain an initial deformation mask image.

And registering the initial deformation mask image according to the mask image of the reference image to obtain a second target deformation matrix.

And deforming the second image by using the first target deformation matrix and the second target deformation matrix to obtain a registered second image.

In one possible implementation, the method further includes:

after the initial medical images are registered, determining a registration loss value of each registered image, wherein the registration loss value is used for representing the difference degree between the registered image and the reference image.

And determining an abnormal image in the plurality of registered images according to the plurality of registration loss values.

In one possible implementation, the determining an abnormal image in the plurality of registered images according to the plurality of registration loss values includes:

and determining the registered image with the registration loss value being greater than or equal to a second preset threshold value as a first abnormal image.

In a second aspect, the present application further provides an image registration method, including:

a plurality of initial medical images of a region of interest are acquired.

According to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image; the second image is an image with the similarity greater than or equal to a first preset threshold value with the first image in the initial medical images.

Determining an abnormal image in the plurality of registered images according to the plurality of registration loss values; and determining the registered image with the registration loss value being greater than or equal to a second preset threshold value as a first abnormal image.

In a third aspect, the present application further provides an image registration apparatus, the apparatus comprising:

the acquisition module is used for acquiring a plurality of initial medical images of the region of interest.

And the processing module is used for obtaining a first target deformation matrix according to a first image in the initial medical images acquired by the acquisition module.

The processing module is further configured to perform registration on a second image corresponding to the first image based on the first target deformation matrix; the second image is an image with the similarity greater than or equal to a first preset threshold value with the first image in the initial medical images.

In a fourth aspect, the present application further provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:

a plurality of initial medical images of a region of interest are acquired.

Or, the processor implements the following steps when executing the computer program:

a plurality of initial medical images of a region of interest are acquired.

Determining a target mask image which is the smallest difference with the mask image of the reference image in the mask images of the plurality of registered first images.

Deforming the mask image of the second image according to the first target deformation matrix to obtain an initial deformation mask image; the second image is an image, of the plurality of initial medical images, with the similarity to the first image being greater than or equal to a first preset threshold.

In a fifth aspect, the present application further provides a computer-readable storage medium. Having stored thereon a computer program which, when executed by a processor, performs the steps of:

a plurality of initial medical images of a region of interest are acquired.

Alternatively, the computer program when executed by a processor implements the steps of:

a plurality of initial medical images of a region of interest are acquired.

In a sixth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, comprises the steps of:

a plurality of initial medical images of a region of interest are acquired.

Alternatively, the computer program, when executed by a processor, comprises the steps of:

a plurality of initial medical images of a region of interest are acquired.

According to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image; the second image is an image, of the plurality of initial medical images, with the similarity to the first image being greater than or equal to a first preset threshold.

According to the image registration method, the image registration device and the electronic equipment, the first target deformation matrix is obtained through the first image in the multiple initial medical images of the region of interest, and the second image, which is similar to the first image and is larger than or equal to the first preset threshold value, in the multiple initial medical images is registered based on the first target deformation matrix. Therefore, the search range in the second image registration process is reduced, the difficulty and the calculation amount in registration of the second image are reduced, and the registration rate of a plurality of initial medical images of the region of interest is further improved.

Drawings

FIG. 1 is an architecture diagram of an image registration system in one embodiment;

FIG. 2 is a flow diagram illustrating an exemplary image registration method;

FIG. 3 is a second flowchart illustrating an image registration method according to an embodiment;

FIG. 4 is a diagram illustrating the effect of binarizing an image according to one embodiment;

FIG. 5 is a diagram illustrating the effect of averaging images in one embodiment;

FIG. 6 is a third flowchart illustrating an image registration method according to an embodiment;

FIG. 7 is a block diagram illustrating an exemplary image registration process;

FIG. 8 is a diagram illustrating comparison of image registration effects in one embodiment;

FIG. 9 is a fourth flowchart illustrating an image registration method according to an embodiment;

FIG. 10 is a schematic flow chart illustrating the determination of an anomalous image in one embodiment;

FIG. 11 is a fifth flowchart illustrating an image registration method according to an embodiment;

FIG. 12 is a schematic diagram showing the structure of an image registration apparatus according to an embodiment;

FIG. 13 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

The technical solutions in some embodiments of the present application will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the examples provided herein fall within the scope of the present application.

In the field of image processing, various inter-image registration technologies exist, and one or more images can be optimally mapped onto a target image based on certain evaluation criteria, so that the distance between the two images (i.e., the difference of the images) is reduced, and the complexity of contrast between the images is improved. For time series images, the time difference of scanning between images is relatively small, and the images are often relatively similar, and if the current mainstream image pair registration is used, the images in the time series are sequentially registered, so that extremely large redundant calculation is undoubtedly existed, and the registration efficiency is very low.

Based on the problems in the prior art, an embodiment of the present application provides an image registration method, which has the following principle: the method comprises the steps of determining a reference image, a first image and a second image in a plurality of medical images shot in a region of interest, wherein the similarity between the first image and the second image is larger than or equal to a first preset threshold value. And the deformation matrix determined after the first image is registered based on the reference image is used for deforming the second image, and the deformed second image is registered based on the reference image, so that the registration calculation amount of the second image is reduced, the registration process of the second image is simplified, and the registration rate of the multiple medical images in the region of interest is improved.

For the convenience of using the present embodiment, referring to the architecture of the image registration system 10 shown in fig. 1, the image registration system 10 includes an image registration apparatus 11 and an image device 12. The imaging device 12 is capable of imaging the region of interest and obtaining K-space data of the coil unit.

In an exemplary scheme, the image registration apparatus 11 may be a terminal device in a general case; the terminal device may have a multi-purpose or special purpose computing device environment or configuration. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like. The terminal equipment may be referred to by different names, such as User Equipment (UE), access equipment, terminal unit, terminal station, mobile station, remote terminal, mobile equipment, wireless communication equipment, terminal agent, or terminal device. In the embodiment of the present application, the apparatus for implementing the function of the image registration apparatus 11 may be a terminal device, or may be an apparatus capable of supporting the image registration apparatus 11 to implement the function, such as a chip system. In the present application, the chip system may have a chip configuration, and may also include a chip and other discrete devices.

With reference to fig. 1, a detailed description is made of an image registration method provided by an embodiment of the present application, and with reference to fig. 2, the method includes:

and S11, acquiring a plurality of initial medical images of the region of interest.

Illustratively, the plurality of initial medical images are a plurality of internal tissue images captured at the same position (i.e., the region of interest) of the target object in a time axis in a certain time sequence, and may also be referred to as time sequence images. Of course, the plurality of initial medical images in the embodiment of the present application are not limited to time-series images captured in one time axis, and may be sequence images captured of a region of interest to obtain certain similarities.

The initial medical image may be a tomographic image (tomosynthesis images), for example, a CT (computed tomography) image, an MRI (magnetic resonance imaging), a SPECT (single photon emission computed tomography), a PET (positron emission computed tomography), or other medical images. Embodiments of the present application are not so limited and may be applied to other types of medical images.

Further, the initial medical image may be a medical image sequence stored in a database of a medical image interpretation system; thus, the sequence of medical images may be retrieved from a database when performing the image registration. Alternatively, as another example, the sequence of medical images may also be acquired directly from the medical imaging device at the time of image registration.

And S12, obtaining a first target deformation matrix according to a first image in the plurality of initial medical images.

Illustratively, the edge recognition of the region of interest in each initial medical image is performed separately, resulting in an edge recognition result of each initial medical image, wherein the target may have a rigid feature.

The region of interest may also be a body part with rigid features, for example the region of interest may be a skull. The basic structure of the skull is considered to be a rigid body since it undergoes almost no overall skeletal deformation compared to other muscles and soft tissues. Therefore, it is very effective to perform image registration on the skull in this region by performing edge recognition and then performing registration. It should be understood that the region of interest may also be the chest of a human body or other body part with rigid characteristics.

In this embodiment, an edge detection operator may be used to perform edge detection or identification on each medical image in the medical image sequence, and the obtained three-dimensional edge identification result may include edge identification results (also referred to as edge detection results) on a plurality of two-dimensional medical images. The edge detection operator may be, for example, a Sobel, Canny, or Laplacian operator, and the edge detection operator is not limited in this embodiment of the present application.

In practical applications, it is first necessary to determine a reference image among a plurality of initial medical images, and to register other initial medical images based on the reference image. Based on the characteristic that the plurality of initial medical images are captured in time series, one medical image with the earliest capturing time among the plurality of initial medical images excluding the reference image may be determined as the first image. Of course, any one of the plurality of initial medical images excluding the reference image may be used as the first image.

In one possible implementation, the reference image is determined among a plurality of initial medical images. And registering each first image in the multiple initial medical images based on the reference image to obtain a first target deformation matrix of each first image.

In addition, a plurality of first images may exist in the plurality of initial medical images, or only one first image may exist, and the determination criterion is mainly determined based on the similarity between the first image and the initial medical image other than the reference image and the first image in the plurality of initial medical images.

For example, the manner of determining the first image and the number thereof for the plurality of initial medical images may be: firstly, a first image a is selected from other initial medical images except a reference image in a plurality of initial medical images, and when the similarity between the first image a and other initial medical images except the reference image and the first image a is larger than or equal to a first preset threshold, the first image is only the selected first image a. When the similarity between only one part of the initial medical images except the reference image and the first image a is greater than or equal to a first preset threshold value, determining other first images in the other rest of the initial medical images; at this time, the first image may exist as well as the first image b, the first image c, the first image d, and so on, excluding the above-mentioned selected first image a. Therefore, the embodiment of the present application does not set any limit to the specific number of the first images.

S13, registering a second image corresponding to the first image based on the first target deformation matrix; the second image is an image of which the similarity with the first image is greater than or equal to a first preset threshold value in the plurality of initial medical images.

Specifically, the similarity between the first image and the second image can be understood as the overlapping degree between the first image and the second image. For example, the degree of overlap between the first image and the second image may be determined by using a recognition model trained by a preset algorithm, and the preset algorithm may be an artificial intelligence algorithm such as a neural network and a deep learning network. For another example, the overlapping degree of the first image and the second image can be calculated by using a euclidean distance calculation method.

According to the image registration method, a first target deformation matrix is obtained through a first image in a plurality of initial medical images of an interested area, and a second image with the similarity larger than or equal to a first preset threshold value with the first image in the plurality of initial medical images is registered based on the first target deformation matrix. Therefore, the search range in the second image registration process is reduced, the calculation amount during registration of the second image is reduced, and the registration rate of a plurality of initial medical images of the region of interest is further improved.

In one possible implementation, referring to fig. 3, in order to highlight the region of interest more and facilitate the fast and accurate determination of the reference image, the determination of the reference image in the plurality of initial medical images includes:

and S21, acquiring a mask image corresponding to each initial medical image in the plurality of initial medical images.

Specifically, a mask image corresponding to each initial medical image is determined according to the prior knowledge of the region of interest.

It is understood that the image after binarization is a black and white image, which corresponds to a mask image. Therefore, one implementation of obtaining the mask image corresponding to each of the plurality of initial medical images may also be obtaining a binary image corresponding to each of the plurality of initial medical images.

Specifically, acquiring a binary image corresponding to each initial medical image in a plurality of initial medical images is to subdivide parameter values of pixel points in each initial medical image according to parameter values and binary segmentation threshold values of the pixel points in each initial medical image so as to present an obvious black-and-white effect. For example, a corresponding threshold T may be set according to the meaning represented by the parameter value of each pixel point after the initial medical image is imaged, and when the parameter value of the pixel point is smaller than the threshold T, the parameter value of the pixel point is set to 0, which indicates black; and when the parameter value of the pixel point is greater than or equal to the threshold value T, setting the parameter value of the pixel point to be 1 to represent white.

For example, taking the region of interest as a skull in a CT image of a brain, in general, the density of the skull in the CT image of the brain is greater than 400 Hounsfield Units (HU), the threshold may be set to 400HU based on the density of the skull, and the parameter value of a pixel point in the CT image of the brain less than 400HU may be set to 0, indicating black; and setting the parameter value of the pixel point which is larger than or equal to 400HU in the brain CT image as 1, and indicating white. Then, the binarization processing is performed on the skull in the brain CT image, and the binarized image containing the skull as shown in fig. 4 can be obtained.

Further, when a plurality of regions of interest exist in the same initial medical image, the plurality of regions of interest are generally located in different portions in the initial medical image and have different intensities. An adaptive threshold may be used, where the adaptive threshold is determined from each region of interest on the initial medical image to which it corresponds. In this way, different regions of interest in the same initial medical image may use different threshold values, thereby obtaining a binarized image containing multiple regions of interest.

And S22, determining a reference image according to the plurality of mask images.

It is understood that the reference image is determined among a plurality of initial medical images, and therefore, for a plurality of initial medical images captured in a time axis in a certain time sequence, one initial medical image with the earliest capture time may be used as the reference image or one initial medical image may be arbitrarily selected from the plurality of initial medical images.

In this embodiment, a plurality of mask images corresponding to a plurality of initial medical images are used to determine a reference image in the plurality of initial medical images, so that the registration rate and accuracy of the plurality of initial medical images are improved.

In one possible implementation, S22 includes: averaging the multiple mask images to obtain an average image; and determining the target mask image with the highest coincidence degree with the average image in the plurality of mask images. And then, determining an initial image corresponding to the target mask image as a reference image.

Specifically, the parameter values of the same pixel point in the multiple mask images are added and then divided by the number of the mask images to obtain the parameter value of the pixel point in the average image.

It can be understood that the average image obtained by averaging the plurality of mask images is a grayscale image. For example, in the case where the region of interest is a skull, the average processing is performed on a plurality of mask images, and the average image is obtained as shown in fig. 5.

Optionally, the weighted average processing is performed on the plurality of mask images to obtain an average image.

Illustratively, the coincidence degree of the mask image and the average image can be determined by means of Dice, mean square error, euler distance, or Cosin algorithm.

Therefore, the target mask image with the highest average image coincidence degree obtained by averaging the multiple mask images can be determined in the multiple mask images, so that the reference image is obtained, and the registration rate and accuracy of image registration are improved.

In a possible implementation manner, registering each first image of the plurality of initial medical images based on the reference image to obtain a first target deformation matrix of each first image specifically includes: and registering the mask image of the first image according to the mask image of the reference image to obtain a plurality of first deformation matrixes. And deforming the mask image of the first image according to each first deformation matrix to obtain the mask image of each registered first image. Then, a target mask image which is the smallest difference from the mask image of the reference image is determined in the mask images of the plurality of registered first images. Thus, the first deformation matrix used by the target mask image is determined as the first target deformation matrix.

Based on the embodiment in S12, in an implementation manner, considering that the initial medical image may be one-dimensional, two-dimensional, three-dimensional or more-dimensional, the mask image of the first image is registered according to the mask image of the reference image, the two-dimensional medical images may be respectively registered, or the three-dimensional medical image sequence may be registered. The process of registration may include: and determining the spatial transformation amount when the mask image of the reference image and the mask image of the first image have the maximum overlapping pixels by performing spatial transformation on the edge identification result, then iteratively performing spatial transformation on the residual pixels after the overlapping pixels are removed to determine the spatial transformation amount when the overlapping pixels are the maximum until an iteration stopping condition is met, and finally obtaining the spatial transformation amount (namely the first target deformation matrix). For example, the spatial transformation may include processes such as translation and/or rotation and/or scaling, and the order of the spatial transformation is not limited in the embodiments of the present application, and for example, the spatial transformation may be first translated and/or rotated and then scaled, or may be first scaled and then translated and/or rotated.

Further, the first image is registered based on the first target deformation matrix to obtain a first registered image (i.e., the registered first image).

In this embodiment, the mask image of the first image is registered according to the mask image of the reference image, so as to obtain a plurality of first deformation matrices. And deforming the mask image of the first image according to each first deformation matrix to obtain the mask image of each registered first image. Then, a target mask image which is the smallest difference from the mask image of the reference image is determined in the mask images of the plurality of registered first images. And determining the first deformation matrix used by the target mask image as the first target deformation matrix. The mask image is used for registration, so that the calculated amount in the process of determining the first target deformation matrix can be simplified, and the image registration rate is improved.

In a possible implementation manner, referring to fig. 6, in order to improve the registration rate of the second image, in an embodiment of the present application, after the mask image of the second image is deformed based on the first target deformation matrix, the mask image of the deformed second image may be registered according to the mask image of the reference image; thus, S13 includes:

s131, according to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image.

And S132, registering the initial deformation mask image according to the mask image of the reference image to obtain a second target deformation matrix.

In one implementation manner, considering that the similarity between the first image and the second image in the embodiment of the present application satisfies a condition that is greater than or equal to a first preset threshold, the second target deformation matrix may also be determined based on the mask image of the first image directly; therefore, the flow of determining the second target deformation matrix is not limited to S131 and S132, and may be, for example: and registering the mask image of the second image based on the mask image of the first image to obtain a second target deformation matrix.

S133, deforming the second image by using the first target deformation matrix and the second target deformation matrix to obtain a registered second image.

Optionally, consider a case where the plurality of initial medical images may further include a third image, where a similarity between the third image and the first image is smaller than a first preset threshold; accordingly, embodiments of the present application may further include a registration procedure for the third image: determining a third image in the plurality of initial medical images, wherein the similarity between the third image and the first image is smaller than a first preset threshold value; registering the mask image of the third image according to the mask image of the registered image to obtain a third target deformation matrix; and registering the third image by using the third target deformation matrix to obtain a registered third image.

For better understanding, taking a time-series image formed by shooting a plurality of initial medical images in a time axis as an example, the registration process of S131 to S133 and the third image in the embodiment of the present application is exemplarily described, where, among other initial medical images not including the reference image, an initial medical image with the earliest time may be selected as the first image according to the shooting time order, and the second image and the third image may be sequentially searched and determined according to the shooting time order. For example, referring to fig. 7, there are 5 initial medical images, which are defined as a time sequence image 1, a time sequence image 2, a time sequence image 3, a time sequence image 4 and a time sequence image 5 according to the sequence of shooting time, where a mask image of the time sequence image 1 is the mask image 1, a mask image of the time sequence image 2 is the mask image 2, a mask image of the time sequence image 3 is the mask image 3, a mask image of the time sequence image 4 is the mask image 4, and a mask image of the time sequence image 5 is the mask image 5; where time series picture 2 is the reference picture, then time series picture 1 is first taken as the first preferred picture, determining a second image with similarity greater than or equal to a first preset threshold value with the time-series image 1 from the time-series images 3, 4 and 5, assuming the time-series image 3, and then using a first target deformation matrix (i.e. the deformation matrix 1 in fig. 7) of the time-series image 1 to deform the mask image 3 to obtain an initial deformed mask image corresponding to the time-series image 3, and the initial deformation mask image corresponding to the time sequence image 3 is used for registering to the mask image 2 to obtain a second target deformation matrix of the time sequence image 3, and further deforming the time sequence graph 3 according to the first target deformation matrix of the time sequence image 1 and the second target deformation matrix of the time sequence image 3 to obtain the registered time sequence graph 3.

Further, the time-series image 3 may be regarded as the first image again, and the similarity with the time-series image 3 is determined to be greater than or equal to that of the second image in the time-series image 4 and the time-series image 5, assuming that the similarity between the time-series image 4 and the time-series image 5 and the time-series image 3 is greater than or equal to that of the second image; then, the mask image 4 may be deformed based on the first target deformation matrix of the time sequence image 1 and the second target deformation matrix of the time sequence image 3 to obtain an initial deformation mask image corresponding to the time sequence image 4, and the initial deformation mask image corresponding to the time sequence image 4 is used to perform registration on the mask image 2 to obtain a second target deformation matrix of the time sequence image 4, and then the time sequence image 4 is deformed according to the first target deformation matrix of the time sequence image 1, the second target deformation matrix of the time sequence image 3, and the second target deformation matrix of the time sequence image 4 to obtain the registered time sequence image 4. The registration process for the time-series image 5 is the same, and will not be described here. It can be understood that the initial medical images of the embodiment of the present application are not limited to the above 5 time-series images, and other numbers of initial medical images may also exist, and for other numbers of initial medical images, registration may be performed by referring to the above registration process of the 5 time-series images, and details are not described here again.

In one implementation manner, the time-series image 4 may also be used as a first image, and it is determined whether the similarity between the time-series image 5 and the time-series image 4 is greater than or equal to a first preset threshold; if so, refer to the above-mentioned registration process of the time-series image 1 and the time-series image 3, which is not described herein again. If not, the time-series image 4 and the time-series image 5 are both used as third images to be respectively registered with the mask image 2, and the registration process may refer to the registration process of the third images, which is not described herein again.

In this embodiment, the mask image of the second image is deformed through the first target deformation matrix to obtain an initial deformation mask image corresponding to the second image, and the initial deformation mask image is registered based on the mask image of the reference image to obtain a second target deformation matrix; and the second image is deformed according to the first target deformation matrix and the second target deformation matrix, so that the difficulty and the calculated amount of the second image registration process are reduced.

Exemplarily, a region of interest is taken as a skull for explanation, and referring to fig. 8, the embodiment of the present application further provides a comparison graph of registration effect after registration for skull images, where shown in region 1 are 8 initial medical images with the number of a1-a8 of the region of interest, and shown in region 2 are 8 registered images with the number of b1-b8 obtained by registering a1-a8 by the image registration method provided in the embodiment of the present application. Wherein b1 is the image after a1 registration, b2 is the image after a2 registration, … … and b8 are the images after a8 registration. The initial ends of all arrows in fig. 8 are located at the same position in the skull, the tail ends of all arrows are located at the same position in the skull, it can be seen that the deviation directions of the skull of a1-a8 are different through the arrow pointing directions in a1-a8, and the arrow pointing directions are the same through b1-b8 obtained after registration of a1-a8, and the effect diagram shown in fig. 8 can prove that the image registration method provided by the embodiment of the application improves the image registration rate, reduces the registration complexity, and simultaneously ensures the effectiveness of image registration.

In a possible implementation manner, in consideration of the existence of low-quality images such as severe artifacts in the registered images, in order to avoid that these low-quality images affect the subsequent image processing flow, the embodiment of the present application can further identify abnormal images in the registered images, and therefore, with reference to fig. 9, the method further includes the following steps:

and S41, after the registration of all the initial medical images is completed, determining a registration loss value of each registered image, wherein the registration loss value is used for representing the difference degree between the registered image and the reference image.

Specifically, a distance evaluation method may be employed, and the registration loss value of each registered image may be used.

And S42, determining abnormal images in the plurality of registered images according to the plurality of registration loss values.

In one possible implementation, determining an abnormal image in the plurality of registered images according to the plurality of registration loss values includes: and determining the registered image with the registration loss value larger than or equal to a second preset threshold value as a first abnormal image.

In another possible implementation manner, after determining the registered image with the registration loss value greater than or equal to the second preset threshold as the first abnormal image, the method further includes: calculating a mean value and a variance according to the registration loss value smaller than a second preset threshold value; and determining a second abnormal image according to the mean value and the variance.

Specifically, a third preset threshold is determined according to the mean and the variance, and the specific formula is as follows:

T3＝ε+k×δ

wherein T3 represents a third preset threshold; ε represents the mean value; δ represents the variance; k represents a preset coefficient of variance.

Based on the above example, the image corresponding to the registration loss value greater than the third preset threshold value in the registration loss values smaller than the second preset threshold value is determined as the second abnormal image.

For better understanding, referring to fig. 10, the execution steps of S41 and S42 are systematically explained. Wherein the registration loss values of the registered images may form a first registration loss sequence.

S51, traversing the first registration loss sequence; jump to S52.

S52, whether there is a registration loss value (i.e. a first type of registration loss value) in the first registration loss sequence that is greater than or equal to a second preset threshold; if yes, jumping to S53; if not, go to S55.

S53, marking the registered image corresponding to the first type of registration loss value as a first abnormal image; jump to S54.

S54, removing the first type of registration loss value from the first registration sequence to obtain a second registration loss sequence; jump to S55.

And S55, calculating the mean and the variance of the second registration loss sequence, and calculating a third preset threshold according to the mean and the variance.

S56, traversing the second registration loss sequence; jump to S57.

S57, whether there is a registration loss value greater than or equal to a third preset threshold in the second registration loss sequence (i.e. a second type of registration loss value); if yes, jumping to S58; if not, ending.

And S58, marking the registered image corresponding to the second type of registration loss value as a second abnormal image, and ending.

In this embodiment, after the registration of all of the plurality of initial medical images is completed, a registration loss value of each registered image is determined, and the registration loss value is used for representing the degree of difference between the registered image and the reference image; and determining an abnormal image in the plurality of registered images according to the plurality of registration loss values. The method can effectively prevent the data with low image quality such as serious artifacts from entering the subsequent image processing flow, and avoids the interference to the subsequent image processing flow.

Referring to fig. 11, an embodiment of the present application further provides an image registration method, including:

and S61, acquiring a plurality of initial medical images of the region of interest.

And S62, acquiring a mask image corresponding to each initial medical image in the plurality of initial medical images.

And S63, averaging the multiple mask images to obtain an average image.

And S64, determining the target mask image with the highest coincidence degree with the average image in the plurality of mask images.

And S65, determining the initial image corresponding to the target mask image as a reference image.

And S66, registering the mask image of the first image according to the mask image of the reference image to obtain a plurality of first deformation matrixes.

And S67, deforming the mask image of the first image according to the first deformation matrixes to obtain the mask image of each registered first image.

And S68, determining the target mask image with the minimum difference with the mask image of the reference image in the mask images of the plurality of registered first images.

S69, determining the first deformation matrix used by the target mask image as a first target deformation matrix.

S70, deforming the mask image of the second image according to the first target deformation matrix to obtain an initial deformation mask image; the second image is an image of which the similarity with the first image is greater than or equal to a first preset threshold value in the plurality of initial medical images.

And S71, registering the initial deformation mask image according to the mask image of the reference image to obtain a second target deformation matrix.

And S72, deforming the second image by using the first target deformation matrix and the second target deformation matrix to obtain a registered second image.

And S73, after the registration of all the initial medical images is completed, determining a registration loss value of each registered image, wherein the registration loss value is used for representing the difference degree between the registered image and the reference image.

S74, determining abnormal images in the plurality of registered images according to the plurality of registration loss values; and determining the registered image with the registration loss value larger than or equal to a second preset threshold as a first abnormal image.

It should be noted that, referring to the specific definition of the image registration method shown in fig. 10, reference may be made to the above definition of the image registration method, and details are not described here.

It should be understood that although the steps in the flowcharts of fig. 2, 3, 6, 9, 10, 11 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 3, 6, 9, 10, and 11 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least some of the other steps.

In one embodiment, referring to fig. 12, there is provided an image registration apparatus 11, the apparatus 11 comprising:

an acquisition module 111 for acquiring a plurality of initial medical images of the region of interest.

The processing module 112 is configured to obtain a first target deformation matrix according to a first image in the multiple initial medical images obtained by the obtaining module 111.

The processing module 112 is further configured to perform registration on a second image corresponding to the first image based on the first target deformation matrix; the second image is an image of which the similarity with the first image is greater than or equal to a first preset threshold value in the plurality of initial medical images.

In one possible implementation, the processing module 112 is specifically configured to,

a reference image is determined among a plurality of initial medical images.

And registering each first image in the multiple initial medical images based on the reference image to obtain a first target deformation matrix of each first image.

And determining a reference image according to the plurality of mask images.

and averaging the multiple mask images to obtain an average image.

And determining an initial image corresponding to the target mask image as a reference image.

And deforming the mask image of the first image according to each first deformation matrix to obtain the mask image of each registered first image.

And determining a target mask image which has the smallest difference with the mask image of the reference image in the mask images of the plurality of registered first images.

And determining a first deformation matrix used by the target mask image as a first target deformation matrix.

In one possible implementation, the processing module 112 is further configured to,

In a possible implementation manner, the processing module 112 is specifically configured to determine the registered image with the registration loss value greater than or equal to the second preset threshold as the first abnormal image.

For specific definition of the image registration apparatus, reference may be made to the above definition of the image registration method, which is not described herein again. The various modules in the image registration apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 13. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing initial data, and the network interface of the computer device is used for communicating with an external terminal through network connection. The computer program is executed by a processor to implement an image registration method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the configuration shown in fig. 13 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation on the computing device to which the present application may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

a plurality of initial medical images of a region of interest are acquired.

And registering a second image corresponding to the first image based on the first target deformation matrix. The second image is an image of which the similarity with the first image is greater than or equal to a first preset threshold value in the plurality of initial medical images.

Alternatively, the processor, when executing the computer program, implements the steps of:

a plurality of initial medical images of a region of interest are acquired.

And averaging the multiple mask images to obtain an average image.

And determining a target mask image which has the minimum difference with the mask image of the reference image in the mask images of the plurality of registered first images.

According to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image; the second image is an image of which the similarity with the first image is greater than or equal to a first preset threshold value in the plurality of initial medical images.

Determining an abnormal image in the plurality of registered images according to the plurality of registration loss values; and determining the registered image with the registration loss value larger than or equal to a second preset threshold as a first abnormal image.

In one embodiment, a computer-readable storage medium. Having stored thereon a computer program which, when executed by a processor, performs the steps of:

a plurality of initial medical images of a region of interest are acquired.

Alternatively, the computer program when executed by the processor implements the steps of:

a plurality of initial medical images of a region of interest are acquired.

And averaging the multiple mask images to obtain an average image.

In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:

a plurality of initial medical images of a region of interest are acquired.

And averaging the multiple mask images to obtain an average image.

According to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image; the second image is an image with similarity greater than or equal to a first preset threshold value among the plurality of initial medical images.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method of image registration, the method comprising:

acquiring a plurality of initial medical images of a region of interest;

obtaining a first target deformation matrix according to a first image in the plurality of initial medical images;

registering a second image corresponding to the first image based on the first target deformation matrix; the second image is an image with the similarity greater than or equal to a first preset threshold value with the first image in the initial medical images.

2. The image registration method according to claim 1, wherein the deriving a first target deformation matrix from a first image of the plurality of initial medical images comprises:

determining a reference image among the plurality of initial medical images;

3. The image registration method of claim 2, wherein the determining a reference image in the plurality of initial medical images comprises:

obtaining a mask image corresponding to each initial medical image in the plurality of initial medical images;

and determining the reference image according to the plurality of mask images.

4. The image registration method of claim 3, wherein the determining the reference image from the plurality of mask images comprises:

averaging the multiple mask images to obtain an average image;

determining a target mask image with the highest coincidence degree with the average image in the multiple mask images;

5. The image registration method according to claim 2, wherein the registering each first image of the plurality of initial medical images based on the reference image to obtain a first target deformation matrix of each first image comprises:

registering the mask image of the first image according to the mask image of the reference image to obtain a plurality of first deformation matrixes;

deforming the mask image of the first image according to each first deformation matrix to obtain the mask image of each registered first image;

determining a target mask image which has the smallest difference with the mask image of the reference image in the mask images of the plurality of registered first images;

6. The image registration method according to any one of claims 1-5, wherein the registering a second image based on the first target deformation matrix comprises:

according to the first target deformation matrix, deforming the mask image of the second image to obtain an initial deformation mask image;

registering the initial deformation mask image according to the mask image of the reference image to obtain a second target deformation matrix;

7. The image registration method of claim 1, further comprising:

after the initial medical images are registered, determining a registration loss value of each registered image, wherein the registration loss value is used for representing the difference degree between the registered image and the reference image;

8. The image registration method of claim 7, wherein determining an abnormal image in the plurality of registered images according to the plurality of registration loss values comprises:

9. An image registration apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring a plurality of initial medical images of the region of interest;

the processing module is used for obtaining a first target deformation matrix according to a first image in the initial medical images acquired by the acquisition module;

10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the computer program, when executed by the processor, causes the processor to perform the steps of the method according to any of claims 1 to 8.