CN115359011A - Image segmentation method, image segmentation model training device, electronic equipment and storage medium - Google Patents

Image segmentation method, image segmentation model training device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115359011A
CN115359011A CN202211026086.6A CN202211026086A CN115359011A CN 115359011 A CN115359011 A CN 115359011A CN 202211026086 A CN202211026086 A CN 202211026086A CN 115359011 A CN115359011 A CN 115359011A
Authority
CN
China
Prior art keywords
image
registration
model
sample
deformation field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211026086.6A
Other languages
Chinese (zh)
Inventor
刘泽庆
黄文豪
张欢
陈宽
王少康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Infervision Medical Technology Co Ltd
Original Assignee
Infervision Medical Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Infervision Medical Technology Co Ltd filed Critical Infervision Medical Technology Co Ltd
Priority to CN202211026086.6A priority Critical patent/CN115359011A/en
Publication of CN115359011A publication Critical patent/CN115359011A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30048—Heart; Cardiac
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The application provides an image segmentation and model training method, an image segmentation and model training device, an electronic device and a storage medium, a first image and a second image which are acquired from the same object are registered through a registration model to obtain a registration deformation field, and then the registration deformation field and a segmentation model of a pre-acquired second image are calculated to obtain a segmentation model of the first image to be segmented, so that segmentation identification of the first image with low tissue resolution can be realized.

Description

Image segmentation method, image segmentation model training device, electronic equipment and storage medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for training image segmentation and models, an electronic device, and a storage medium.
Background
Image segmentation techniques are of great use in imaging diagnostics, for example, segmenting the heart and coronary vessels may help physicians accurately diagnose heart disease. At present, most of the existing medical image segmentation technologies, whether based on a traditional algorithm or a deep learning method, are based on an enhanced CT (Computed Tomography) image, and perform better segmentation and identification on the image by using the effect of a contrast agent. However, in some medical scenes, image segmentation needs to be performed on a flat-scan CT image, which has low resolution on tissues and thus is difficult to perform.
Disclosure of Invention
In view of this, embodiments of the present application provide an image segmentation method, an image segmentation device, an image model training method, an electronic device, and a storage medium, which can quickly obtain a segmented image on a flat-scan CT image.
In a first aspect, an embodiment of the present application provides an image segmentation method, including:
acquiring a first image and a second image, wherein the first image and the second image are images acquired from the same object;
inputting the first image and the second image into a pre-trained registration model, and registering the first image and the second image by using the registration model to obtain a registration deformation field;
and obtaining a segmentation model of the first image through calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is acquired in advance.
In a second aspect, an embodiment of the present application provides a model training method, including:
acquiring a sample image, wherein the sample image comprises a first sample image and a second sample image, and the first sample image and the second sample image are images acquired for the same object;
training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image.
In a third aspect, an embodiment of the present application provides an image segmentation apparatus, including:
an acquisition module configured to acquire a first image and a second image, the first image and the second image being images acquired of a same object;
a registration module configured to input the first image and the second image into a pre-trained registration model, and register the first image and the second image by using the registration model to obtain a registration deformation field;
a segmentation module configured to obtain a segmentation model of the first image through calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is acquired in advance.
In a fourth aspect, an embodiment of the present application provides a model training apparatus, including:
a sample acquisition module configured to acquire a sample image, the sample image including a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired of the same object;
a model training module configured to train a neural network based on the first and second sample images to obtain a registration model capable of registering the first and second sample images.
In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, where the storage medium stores a computer program for executing the image segmentation method according to the first aspect or executing the model training method according to the second aspect.
In a sixth aspect, an embodiment of the present application provides an electronic device, including: a processor; a memory for storing processor-executable instructions, wherein the processor is configured to perform the image segmentation method of the first aspect or to perform the model training method of the second aspect.
The embodiment of the application provides an image segmentation and model training method, an image segmentation and model training device, an electronic device and a storage medium, a first image and a second image which are acquired from the same object are registered through a registration model to obtain a registration deformation field, then the registration deformation field and a segmentation model of a pre-acquired second image are calculated to obtain a segmentation model of the first image to be segmented, segmentation identification of the first image with low tissue resolution can be achieved, and compared with the situation that a large amount of manual labeling is needed in a supervision segmentation network mode, time and labor are wasted, the image segmentation method provided by the application can be used for rapidly obtaining a segmentation image with high accuracy for the first image.
Drawings
FIG. 1 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application.
Fig. 2 is a flowchart illustrating an image segmentation method according to an exemplary embodiment of the present application.
Fig. 3 is a flowchart illustrating an image segmentation method according to another exemplary embodiment of the present application.
FIG. 4a is a flat scan CT image of a coronary heart vessel as provided by an exemplary embodiment of the present application.
FIG. 4b is an enhanced CT image of a coronary heart disease provided by an exemplary embodiment of the present application.
Fig. 5 is a flowchart illustrating a method for registering a flat-scan CT image and an enhanced CT image according to an exemplary embodiment of the present application.
Fig. 6 is a flowchart illustrating a method for training a model according to an exemplary embodiment of the present application.
Fig. 7 is a schematic structural diagram of an image segmentation apparatus according to an exemplary embodiment of the present application.
Fig. 8 is a schematic structural diagram of a training apparatus for a model according to an exemplary embodiment of the present application.
Fig. 9 is a block diagram of an electronic device provided in an exemplary embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Summary of the application
Deep learning implements artificial intelligence in a computing system by building artificial neural networks with hierarchical structures. Because the artificial neural network of the hierarchical structure can extract and screen the input information layer by layer, the deep learning has the characteristic learning capability and can realize end-to-end supervised learning and unsupervised learning. The artificial neural network of the hierarchical structure used for deep learning has various forms, the complexity of the hierarchy is generally called 'depth', and the forms of deep learning comprise a multilayer perceptron, a convolutional neural network, a cyclic neural network, a deep belief network and other mixed structures according to the types of structures. The deep learning uses data to update parameters in the construction of the data to achieve a training target, the process is generally called 'learning', the deep learning provides a method for enabling a computer to automatically learn mode characteristics, and the characteristic learning is integrated into the process of establishing a model, so that the incompleteness caused by artificial design characteristics is reduced.
A neural network is an operational model, which is formed by a large number of nodes (or neurons) connected to each other, each node corresponding to a policy function, and the connection between each two nodes representing a weighted value, called weight, for a signal passing through the connection. The neural network generally comprises a plurality of neural network layers, the upper network layer and the lower network layer are mutually cascaded, the output of the ith neural network layer is connected with the input of the (i + 1) th neural network layer, the output of the (i + 1) th neural network layer is connected with the input of the (i + 2) th neural network layer, and the like. After the sample image is input into the cascaded neural network layers, an output result is output through each neural network layer and is used as the input of the next neural network layer, therefore, the output is obtained through calculation of a plurality of neural network layers, the prediction result of the output layer is compared with a real target value, then the weight matrix and the strategy function of each layer are adjusted according to the difference situation between the prediction result and the target value, the neural network continuously passes through the adjusting process by using the sample image, so that the parameters such as the weight of the neural network and the like are adjusted until the prediction result of the output of the neural network is consistent with the real target result, and the process is called the training process of the neural network. After the neural network is trained, a neural network model can be obtained.
The existing heart segmentation technology mainly adopts a traditional algorithm or a deep learning method, is based on an enhanced CT image, and utilizes the effect of a contrast agent to enable the heart and the artery to be obviously different from common tissues, so that better segmentation and identification are realized. In the case of a flat-scan CT image, HU values of high-density bone tissues, such as bones and calcifications, are usually 100 or more, whereas HU values of general tissues, such as muscles, organs, and blood, are usually 100 or less, and the degree of differentiation between tissues is small, and thus, it cannot be effectively distinguished. If a general supervised network segmentation mode is directly adopted, a large amount of manual data labeling is needed, and the labeling difficulty is high.
In view of the above problems, an embodiment of the present application provides an image segmentation method, where a registration model is used to register a first image and a second image acquired from a same object to obtain a registration deformation field, and then the registration deformation field and a segmentation model of a pre-acquired second image are calculated to obtain a segmentation model of the first image to be segmented, so as to achieve segmentation identification of the first image with a low tissue resolution.
Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary System
FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a CT scanner 130, a server 120, and a computer device 110. The computer device 110 may acquire a plurality of sets of medical images from a CT scanner 130 for X-ray scanning of human tissue, and the computer device 110 may be connected to the server 120 via a communication network. Optionally, the communication network is a wired network or a wireless network.
The computer device 110 may be a general-purpose computer or a computer device composed of an application-specific integrated circuit, and the like, which is not limited in this embodiment. For example, the Computer device 110 may be a mobile terminal device such as a tablet Computer, or may also be a Personal Computer (PC), such as a laptop portable Computer and a desktop Computer. One skilled in the art will appreciate that the number of computer devices 110 may be one or more, and the types may be the same or different. For example, the number of the computer devices 110 may be one, or the number of the computer devices 110 may be several tens or hundreds, or more. The number and the type of the computer devices 110 are not limited in the embodiments of the present application.
In some alternative embodiments, the computer device 110 obtains a plurality of sets of medical sample images from the CT scanner 130, the set of medical sample images including a CT enhanced sample image and a CT scout sample image, and the computer device 110 trains the neural network through the plurality of sets of medical sample images to obtain a network model for registering the CT enhanced sample image and the CT scout sample image within the set.
The server 120 is a server, or consists of several servers, or is a virtualization platform, or a cloud computing service center.
In some optional embodiments, the computer device 110 sends the sets of medical sample images acquired from the CT scanner 130 to the server 120, the sets of medical sample images include a CT enhanced sample image and a CT flat scan sample image, and the server 120 trains the neural network through the sets of medical sample images to obtain a network model for registering the CT enhanced sample image and the CT flat scan sample image.
Exemplary method
Fig. 2 is a flowchart illustrating an image segmentation method according to an exemplary embodiment of the present application. The method of fig. 2 is performed by a computing device, e.g., a server. As shown in fig. 2, the image segmentation method includes the following.
S210: a first image and a second image are obtained, wherein the first image and the second image are images acquired of the same object.
The first and second images are images acquired of the same part of the same object, which may be, for example, a coronary image of the heart or a medical image of another organ of the same patient. It should be noted that the first image and the second image are different types of medical images, for example, the first image may be a flat-scan CT image or other medical images with lower local resolution, and the second image may be an enhanced CT image or other medical images with higher resolution, which is not limited in this embodiment.
S220: and inputting the first image and the second image into a pre-trained registration model, and registering the first image and the second image by adopting the registration model to obtain a registration deformation field.
Specifically, medical image registration is a commonly used technique in medical image analysis, which is to convert the coordinates of one image (moving image) into the other image (fixed image) so that the corresponding positions of the two images are matched to obtain a registered image (Moved). One or a series of spatial transformations are sought for one medical image to bring it into spatial correspondence with corresponding points on the other medical image. The result of the registration should be such that all anatomical points, or at least all points of diagnostic significance and points of surgical interest, on both images are matched. In the task of image registration, the optimal spatial transformation relation and gray-scale transformation relation are mainly found, so that the two images are optimally aligned. Among them, spatial transformation is the key to achieve accurate registration. The transformation can be divided into rigid transformation and non-rigid transformation, the rigid transformation process is image translation and rotation transformation, images can be approximately matched, and the non-rigid transformation is mainly complex transformation such as expansion, affine transformation and the like to process and adjust the images. The registration model in this embodiment may be a rigid model, a non-rigid model, or both a rigid model and a non-rigid model, which is not limited in particular.
And registering the first image and the second image through the registration model to obtain a registration deformation field, wherein the registration deformation field is a matrix formed by vectors of pixel displacement in the images. In this embodiment, the first image is a fixed image, the second image is a moving image, and the second image is registered to the first image through the registration model, that is, coordinates of the second image are converted into the first image, and at this time, a registration deformation field is generated due to displacement of the second image.
S230: and obtaining a segmentation model of the first image through calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is acquired in advance.
Specifically, the segmentation model corresponding to the second image is obtained in advance, in this embodiment, the second image is an enhanced CT image, since the enhanced CT image has a high resolution, obvious data features, easy data annotation, and relatively simple segmentation network training, the segmentation model of the enhanced CT image is easily obtained by using the trained segmentation network, and in an exemplary embodiment, the segmentation model may be a segmentation mask (mask).
After the registration deformation field is obtained through the steps, the segmentation model of the first image can be obtained through calculation by combining with the segmentation model of the second image, the calculation method can realize pixel displacement transformation, such as dot product operation, the segmentation model of the first image with lower resolution is obtained through pixel displacement transformation by utilizing the segmentation model of the second image which is easy to obtain, and compared with a supervised segmentation network, the segmentation method provided by the embodiment is faster and more effective, the segmentation model of the first image with higher accuracy can be obtained, the medical disease diagnosis and treatment are assisted, and the dependence on artificial experience is reduced.
In another embodiment of the present application, the method shown in fig. 3 is an example of the method shown in fig. 2, and the method shown in fig. 3 further includes the following.
S310: acquiring a flat-scan CT image and an enhanced CT image, wherein the flat-scan CT image and the enhanced CT image are heart coronary images acquired for the same object.
The flat-scan CT image and the enhanced CT image are images acquired from the same part of the same object, for example, a cardiac coronary image, fig. 4a shows the flat-scan CT image of the cardiac coronary, and fig. 4b shows the enhanced CT image of the cardiac coronary, and as can be seen from fig. 4a and 4b, the flat-scan CT image has a high HU value of a bone-like high-density tissue, but has a low HU value of a general tissue such as muscle, organ, and blood, which cannot be distinguished effectively, and the enhanced CT image has a high resolution of each tissue.
It should be noted that the X-ray absorption coefficient and HU value of the objects with different densities, organs and tissues of the human body with different densities are correspondingly different. The HU value is a unit of measure for determining the size of the local tissue or organ density and is commonly referred to as hounsfield unit. The enhanced CT image acquires a CT image with higher resolution by means of a contrast medium, and is mainly used for identifying whether a lesion is vascular or non-vascular and determining the relationship between a mediastinal lesion and a great vessel of a heart.
S320: inputting the flat scan CT image and the enhanced CT image of the coronary artery of the heart into a pre-trained registration model, and registering the first image and the second image by adopting the registration model to obtain a registration deformation field.
The enhanced CT image is registered to the flat-scan CT image through the registration model after the displacement transformation of the pixels, namely, the one-to-one correspondence relationship is established between the flat-scan CT image and the pixel points in the enhanced CT image. In this embodiment, the type of the registration model is not limited, and may be a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or the like.
S330: and obtaining the heart coronary artery segmentation mask of the flat-scan CT image through calculation based on the registration deformation field and the segmentation mask of the heart coronary artery corresponding to the enhanced CT image, wherein the segmentation mask corresponding to the enhanced CT image is obtained in advance.
The heart coronary artery segmentation mask of the enhanced CT image is obtained through the preposed segmentation network, the heart coronary artery segmentation mask of the flat-scan CT image can be quickly obtained through dot product operation by combining the registration deformation field, and the problem that the flat-scan CT image segmentation model is difficult to obtain can be effectively solved.
In one embodiment, inputting the first image and the second image to a pre-trained registration model comprises: preprocessing the first image and the second image, and inputting the preprocessed first image and the preprocessed second image into a pre-trained registration model.
Specifically, the preprocessing in this embodiment is specifically image processing. In order to obtain the region of interest and reduce the consumption of computing resources, the second image may be cropped by using a minimum bounding box of a segmentation mask (mask) of the second image, and the first image and the second image may be resampled to a size of 128 × 128 × 128, which maintains isotropy, i.e., maintains the pixel pitches in the three scanning directions x, y, and z to be consistent, and the window width level is set to (300,800). For the first image without the segmentation mask, the invention uses a simple coarse registration mode to coarsely register the segmentation mask of the second image on the first image, and enlarges the minimum circumscribed frame by using a mode of inner distance padding =30, and only needs to acquire the approximate position of the region of interest. After the image preprocessing is performed on the first image and the second image, the registration accuracy of the registration model can be improved.
It should be noted that, in other embodiments, the foregoing preprocessing manner may be added or replaced, and the foregoing image processing is only an exemplary illustration of a preprocessing manner, and the present application does not limit this.
In one embodiment, the registration model comprises a first registration model and a second registration model,
the registering the first image and the second image by using the registration model to obtain a registration deformation field includes: registering the first image and the second image through a first registration model to obtain a first registration deformation field; registering the first image and the second image through a second registration model based on the first registration deformation field to obtain a second registration deformation field; and carrying out merging operation on the first registration deformation field and the second registration deformation field to obtain the registration deformation field.
In this embodiment, the first registration model and the second registration model are both neural network models, the first registration model and the second registration model are models of different registration types, the first registration model is a rigid registration model, and the second registration model is a non-rigid registration model. And carrying out rigid registration on the first image and the second image through a first registration model, and carrying out coarse matching on the first image and the second image through image translation and rotation transformation to obtain a first registration deformation field. And then the second registration model processes and adjusts the coarse registration, and performs non-rigid registration on the first image and the second image to obtain a second registration deformation field. And finally, vector addition is carried out on the first registration deformation field and the second registration deformation field, and a registration deformation field can be obtained. The first image and the second image can be optimally aligned through the first registration model and the second registration model, the alignment precision is improved, and a basis is provided for accurately obtaining a segmentation model of the first image.
It should be noted that, in this embodiment, the first registration model adopts a Deep residual network (ResNet), the second registration model adopts a ResUnet network, the ResUnet network is a combination of the ResNet network and a U-Net network, and the feature extraction can be effectively performed on the medical image through the ResNet network and the ResUnet network, so as to achieve accurate registration of the medical image. In other embodiments, the ResNet network and the ResUnet network may be replaced with other 3D semantic segmentation networks.
In one embodiment, registering the first image and the second image by a second registration model based on the first registration deformation field to obtain a second registration deformation field includes: performing dot product operation on the first registration deformation field and the second image to obtain a third image; and registering the first image and the third image through a second registration model to obtain a second registration deformation field.
Fig. 5 shows a method flow of registering a flat scan CT image 501 and an enhanced CT image 502 by a registration model, as shown in fig. 5, the registration model includes a first registration model 503 for rigid registration and a second registration model 505 for non-rigid registration, the flat scan CT image 501 and the enhanced CT image 502 are input into the first registration model to obtain a first registration deformation field 504, the first registration deformation field 504 and the enhanced CT image 502 are subjected to dot product operation to generate a third image, i.e., a new enhanced CT image, the new enhanced CT image and the flat scan CT image 501 are input into the second registration model 505 for registration to obtain a second registration deformation field 506, and the first registration deformation field 504 and the second registration deformation field 506 are subjected to vector addition to obtain a final registration deformation field 507. Through the combined action of the first registration model 503 and the second registration model 505, the accurate registration of the enhanced CT image 501 and the flat-scan CT image 502 is realized, and the overall registration accuracy is improved.
Fig. 6 is a flowchart illustrating a method for training a model according to an embodiment of the present application. The method illustrated in fig. 6 is performed by a computing device (e.g., a server), but the embodiments of the present application are not limited thereto. The server may be one server, or may be composed of a plurality of servers, or may be a virtualization platform, or a cloud computing service center, which is not limited in this embodiment of the present application. As shown in fig. 6, the method includes the following steps:
s610: acquiring a sample image, wherein the sample image comprises a first sample image and a second sample image, and the first sample image and the second sample image are images acquired of the same object.
The first sample image and the second sample image in the present embodiment are the same as the first image and the second image in the foregoing embodiment, and are both images acquired of the same part or organ of the same subject. The sample images may comprise images acquired for different parts or organs of the plurality of subjects, the first sample image being of a different type than the second sample image, e.g. the first sample image may be a scout CT image and the second sample image may be an enhanced CT image.
S620: training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image.
Training a neural network through the collected first sample image and the second sample image, updating parameters of the registration model through continuous iteration operation until the loss function is minimized and the images are matched, and obtaining the trained registration model.
In one embodiment, a loss function value is calculated from the registration deformation field, the first sample image and the second sample image output by the registration model, and parameters of the registration model are updated according to the loss function value.
In order to further improve the output accuracy of the registration model, the embodiment further performs constraint optimization on the overall result through a preset loss function, which is specifically, exemplarily, to perform constraint optimization on the overall result by using a global loss function and a gradient similarity loss function. Wherein, the global loss function is a Normalized Cross-Correlation loss function (NCC loss), and the similarity of the registration result is measured by calculating the Correlation coefficient, and the NCC loss calculation formula is as follows:
Figure BDA0003815784790000121
where X, Y represents the sample, cov represents the covariance, and Var represents the variance.
And constraining the image brightness difference matching degree through a gradient similarity loss function, wherein the calculation formula is as follows:
Figure BDA0003815784790000122
wherein x and y represent samples, x 'and y' represent gradients, μ represents mean, σ represents variance, and C 1 、C 2 Representing a constant.
In one embodiment, the neural network comprises a first neural network and a second neural network, the registration model comprises a first registration model and a second registration model;
training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image, comprising: training the first neural network based on the first sample image and the second sample image to obtain the first registration model capable of registering the first sample image and the second sample image; performing dot product operation on the first registration deformation field output by the first registration model and the second sample image to obtain a third sample image; training a second neural network based on the first and third sample images to derive the second registration model capable of registering the first and third sample images.
In this embodiment, the first neural network and the second neural network are a ResNet network and a ResUnet network respectively in the embodiment of the image segmentation method, the first sample image and the second sample image are a flat-scan CT image and an enhanced CT image, the ResNet network is trained through the flat-scan CT image and the enhanced CT image to obtain a first registration model, and then a dot product operation is performed on a first registration deformation field output by the first registration model and the second sample image to obtain a third sample image, that is, a new enhanced CT image. And training the ResUnet network through the flat-scan CT image and the third sample image to obtain a second registration model.
It should be noted that, before the registration model is trained through the first sample image and the second sample image, the first sample image and the second sample image need to be preprocessed, where the preprocessing includes image processing and data enhancement. The image processing is the same as that in the above embodiment, and is not described again here. In order to improve the generalization performance of the registration model, data enhancement, specifically random rotation, brightness adjustment, random small-angle dithering, and the like, needs to be performed on the first sample image and the second sample image, where the data enhancement operation is only used as an exemplary illustration, and a person skilled in the art may add or delete a preprocessing operation mode for the image according to actual needs, and is not limited herein.
In one embodiment, the training of the first neural network based on the first and second sample images to obtain the first registration model capable of registering the first and second sample images includes:
inputting the first and second sample images to the first neural network, outputting a first registered deformation field via the first neural network; obtaining a similarity value through mutual information calculation based on the first registration deformation field, the first sample image and the second sample image; and updating parameters of the first neural network according to the similarity numerical value, wherein the first neural network is a residual error network.
Specifically, the first sample image and the second sample image are input to the first neural network to obtain a first registration deformation field, a third sample image can be obtained through dot product operation according to the first registration deformation field and the second sample image, the third sample image is also a registered image, parameters of the first neural network are calculated and adjusted according to mutual information between the registered third sample image and the first sample image (fixed image), and after the similarity exceeds a certain threshold, the similarity is considered to be high, training of the first neural network is completed, and a first registration model is obtained.
The mutual information calculation formula is as follows:
Figure BDA0003815784790000131
wherein, H represents an entropy calculation function, p represents a probability distribution function, and a and b represent samples.
In one embodiment, the training a second neural network based on the first sample image and the third sample image to obtain the second registration model capable of registering the first sample image and the third sample image includes:
inputting the first and third sample images to the second neural network, outputting a second registered deformation field via the second neural network; calculating a loss function value based on the second registration deformation field, the third sample image and the first sample image, and updating the second neural network according to the loss function value, wherein the second neural network is a deep convolution neural network.
Inputting the first sample image and a third sample image obtained through the first registration into a second neural network, registering the first sample image and the third sample image, taking the first sample image as a fixed image, taking the third sample image as a moving image, performing dot product operation on the second registration deformation field and the third sample image to obtain a fourth sample image (namely, a registration image) after the second registration, and updating the second neural network by calculating a loss function value between the first sample image and the fourth sample image to obtain a second registration model.
Illustratively, the Loss function in this embodiment is a Smooth Loss function Smooth Loss, and the specific calculation formula is as follows:
Figure BDA0003815784790000141
wherein, y i And f xi Respectively representing the real value of the ith sample and the corresponding predicted value thereof, wherein n is the number of the samples.
Exemplary embodiments of the inventionDevice for measuring the position of a moving object
Fig. 7 is a schematic structural diagram of an image segmentation apparatus 700 according to an exemplary embodiment of the present application. As shown in fig. 7, the image segmentation apparatus 700 includes:
an obtaining module 710 configured to obtain a first image and a second image, where the first image and the second image are images acquired of a same object;
a registration module 720, configured to input the first image and the second image into a pre-trained registration model, and register the first image and the second image by using the registration model, so as to obtain a registration deformation field;
a segmentation module 730 configured to obtain a segmentation model of the first image by calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is obtained in advance.
It should be understood that, for the specific working processes and functions of the obtaining module 710 to the segmenting module 730 in the foregoing embodiment, reference may be made to the description in the image segmenting method provided in the foregoing embodiments of fig. 2 to fig. 5, and in order to avoid repetition, the description is not repeated herein.
Fig. 8 is a schematic structural diagram of a training apparatus 800 for a model according to an exemplary embodiment of the present application. As shown in fig. 8, the training apparatus 800 for the model includes:
a sample acquiring module 810 configured to acquire a sample image, the sample image including a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired of the same object;
a model training module 820 configured to train a neural network based on the first and second sample images to obtain a registration model capable of registering the first and second sample images.
It should be understood that, for the specific working processes and functions of the sample obtaining module 810 to the model training module 820 in the foregoing embodiment, reference may be made to the description in the model training method provided in the foregoing embodiment of fig. 6, and details are not described herein again to avoid repetition.
Exemplary electronic device and computer-readable storage Medium
Fig. 9 is a block diagram of an electronic device 900 provided by an exemplary embodiment of the present application.
Referring to fig. 9, electronic device 900 includes a processing component 910 that further includes one or more processors and memory resources, represented by memory 920, for storing instructions, such as applications, that are executable by processing component 910. The application program stored in memory 920 may include one or more modules that each correspond to a set of instructions. Further, the processing component 910 is configured to execute instructions to perform the methods described in the above embodiments.
The electronic device 900 may also include a power component configured to perform power management for the electronic device 900, a wired or wireless network interface configured to connect the electronic device 900 to a network, and an input-output (I/O) interface. The electronic device 900 may be operated based on an operating system, such as Windows Server, stored in the memory 920 TM ，Mac OS X TM ，Unix TM ，Linux TM ，FreeBSD TM Or the like.
A non-transitory computer readable storage medium, wherein instructions of the storage medium, when executed by a processor of the electronic device 900, enable the electronic device 900 to perform the method according to the above embodiments.
All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, and are not described herein again.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a portable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other media capable of storing a program check code.
It should be noted that, in the description of the present application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modifications, equivalents and the like that are within the spirit and principle of the present application should be included in the scope of the present application.

Claims (14)

1. An image segmentation method, comprising:
acquiring a first image and a second image, wherein the first image and the second image are images acquired for the same object;
inputting the first image and the second image into a pre-trained registration model, and registering the first image and the second image by using the registration model to obtain a registration deformation field;
and obtaining a segmentation model of the first image through calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is acquired in advance.
2. The method of claim 1, wherein inputting the first image and the second image to a pre-trained registration model comprises:
respectively preprocessing the first image and the second image, and inputting the preprocessed first image and the preprocessed second image into a pre-trained registration model.
3. The method according to claim 1, wherein the registration model comprises a first registration model and a second registration model,
the registering the first image and the second image by using the registration model to obtain a registration deformation field includes:
registering the first image and the second image through a first registration model to obtain a first registration deformation field;
registering the first image and the second image through a second registration model based on the first registration deformation field to obtain a second registration deformation field;
and carrying out a combination operation on the first registration deformation field and the second registration deformation field to obtain the registration deformation field.
4. The method of claim 3, wherein registering the first image and the second image with a second registration model based on the first registration deformation field to obtain a second registration deformation field comprises:
performing dot product operation on the first registration deformation field and the second image to obtain a third image;
and registering the first image and the third image through a second registration model to obtain a second registration deformation field.
5. The method of claim 1, wherein the obtaining the segmentation model of the first image by calculation based on the segmentation model corresponding to the registered deformation field and the second image comprises:
and performing dot product operation on the registration deformation field and the segmentation model corresponding to the second image to obtain the segmentation model of the first image.
6. A method of training a model, comprising:
acquiring a sample image, wherein the sample image comprises a first sample image and a second sample image, and the first sample image and the second sample image are images acquired for the same object;
training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image.
7. The method of claim 6, further comprising:
and calculating a loss function value according to the registration deformation field output by the registration model, the first sample image and the second sample image, and updating the parameters of the registration model according to the loss function value.
8. The method of claim 6, wherein the neural network comprises a first neural network and a second neural network, and the registration model comprises a first registration model and a second registration model;
training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image, including:
training the first neural network based on the first sample image and the second sample image to obtain the first registration model capable of registering the first sample image and the second sample image;
performing dot product operation on the first registration deformation field output by the first registration model and the second sample image to obtain a third sample image;
training a second neural network based on the first and third sample images to derive the second registration model capable of registering the first and third sample images.
9. The method of claim 8, wherein training the first neural network based on the first and second sample images to obtain the first registration model capable of registering the first and second sample images comprises:
inputting the first and second sample images to the first neural network, outputting a first registered deformation field via the first neural network;
obtaining a similarity value through mutual information calculation based on the first registration deformation field, the first sample image and the second sample image;
and updating parameters of the first neural network according to the similarity numerical value, wherein the first neural network is a residual error network.
10. The method of claim 8, wherein training a second neural network based on the first sample image and the third sample image to obtain the second registration model capable of registering the first sample image and the third sample image comprises:
inputting the first and third sample images to the second neural network, outputting a second registered deformation field via the second neural network;
calculating a loss function value based on the second registration deformation field, the third sample image and the first sample image, and updating the second neural network according to the loss function value, wherein the second neural network is a deep convolution neural network.
11. An image segmentation apparatus, comprising:
an acquisition module configured to acquire a first image and a second image, the first image and the second image being images acquired of a same object;
a registration module configured to input the first image and the second image into a pre-trained registration model, and register the first image and the second image by using the registration model to obtain a registration deformation field;
a segmentation module configured to obtain a segmentation model of the first image through calculation based on the registration deformation field and a segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is acquired in advance.
12. An apparatus for training a model, comprising:
a sample acquisition module configured to acquire a sample image, the sample image including a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired of the same object;
a model training module configured to train a neural network based on the first and second sample images to obtain a registration model capable of registering the first and second sample images.
13. A computer-readable storage medium, characterized in that the storage medium stores a computer program for performing the image segmentation method of any one of the preceding claims 1 to 5 and/or for performing the model training method of any one of the preceding claims 6 to 10.
14. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions,
wherein the processor is configured to perform the image segmentation method of any one of claims 1 to 5 and/or the model training method of any one of claims 6 to 10.
CN202211026086.6A 2022-08-25 2022-08-25 Image segmentation method, image segmentation model training device, electronic equipment and storage medium Pending CN115359011A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211026086.6A CN115359011A (en) 2022-08-25 2022-08-25 Image segmentation method, image segmentation model training device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211026086.6A CN115359011A (en) 2022-08-25 2022-08-25 Image segmentation method, image segmentation model training device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115359011A true CN115359011A (en) 2022-11-18

Family

ID=84003959

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211026086.6A Pending CN115359011A (en) 2022-08-25 2022-08-25 Image segmentation method, image segmentation model training device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115359011A (en)

Similar Documents

Publication Publication Date Title
WO2020238734A1 (en) Image segmentation model training method and apparatus, computer device, and storage medium
JP6947759B2 (en) Systems and methods for automatically detecting, locating, and semantic segmenting anatomical objects
Khan et al. Deep neural architectures for medical image semantic segmentation
US11593943B2 (en) RECIST assessment of tumour progression
JP7325954B2 (en) Medical image processing device, medical image processing program, learning device and learning program
EP3611699A1 (en) Image segmentation using deep learning techniques
EP0965104B1 (en) Autosegmentation/autocontouring methods for use with three-dimensional radiation therapy treatment planning
CN112508965B (en) Automatic outline sketching system for normal organs in medical image
Wolterink et al. Generative adversarial networks: a primer for radiologists
WO2021128825A1 (en) Three-dimensional target detection method, method and device for training three-dimensional target detection model, apparatus, and storage medium
JP2002529825A (en) System and method for 4D reconstruction and visualization
Pradhan et al. Transforming view of medical images using deep learning
CN111524109A (en) Head medical image scoring method and device, electronic equipment and storage medium
Banerjee et al. A CADe system for gliomas in brain MRI using convolutional neural networks
Wang et al. Deep transfer learning-based multi-modal digital twins for enhancement and diagnostic analysis of brain mri image
Chatterjee et al. A survey on techniques used in medical imaging processing
US11475568B2 (en) Method for controlling display of abnormality in chest x-ray image, storage medium, abnormality display control apparatus, and server apparatus
Wang et al. Convolutional neural network for automatically segmenting magnetic resonance images of the shoulder joint
CN111209946B (en) Three-dimensional image processing method, image processing model training method and medium
CN115359011A (en) Image segmentation method, image segmentation model training device, electronic equipment and storage medium
CN115482223A (en) Image processing method, image processing device, storage medium and electronic equipment
Rao et al. Deep learning-based medical image fusion using integrated joint slope analysis with probabilistic parametric steered image filter
CN116420165A (en) Detection of anatomical anomalies by segmentation results with and without shape priors
Pham et al. Shape collaborative representation with fuzzy energy based active contour model
Van Houtte et al. A deep learning approach to horse bone segmentation from digitally reconstructed radiographs

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination