WO2020134769A1 - 图像处理方法、装置、电子设备及计算机可读存储介质 - Google Patents

图像处理方法、装置、电子设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2020134769A1
WO2020134769A1 PCT/CN2019/120329 CN2019120329W WO2020134769A1 WO 2020134769 A1 WO2020134769 A1 WO 2020134769A1 CN 2019120329 W CN2019120329 W CN 2019120329W WO 2020134769 A1 WO2020134769 A1 WO 2020134769A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
preset
registered
reference image
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/120329
Other languages
English (en)
French (fr)
Inventor
宋涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Sensetime Intelligent Technology Co Ltd
Original Assignee
Shanghai Sensetime Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Sensetime Intelligent Technology Co Ltd filed Critical Shanghai Sensetime Intelligent Technology Co Ltd
Priority to SG11202102267XA priority Critical patent/SG11202102267XA/en
Priority to KR1020217001183A priority patent/KR20210021039A/ko
Priority to JP2021501292A priority patent/JP2021530061A/ja
Publication of WO2020134769A1 publication Critical patent/WO2020134769A1/zh
Priority to US17/194,790 priority patent/US20210192758A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung

Definitions

  • the present invention relates to the field of computer vision technology, and in particular to image processing methods, devices, electronic equipment, and computer-readable storage media.
  • Image registration is the process of registering two or more images of the same scene or the same target under different acquisition times, different sensors, and different conditions, and is widely used in medical image processing.
  • Medical image registration is an important technology in the field of medical image processing and plays an increasingly important role in clinical diagnosis and treatment.
  • Embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium.
  • a first aspect of an embodiment of the present application provides an image processing method, including:
  • the method before acquiring the image to be registered and the reference image used for registration, the method further includes:
  • the performing image normalization processing on the original image to be registered and the original reference image to obtain the image to be registered and the reference image satisfying a target parameter includes :
  • the training process of the preset neural network model includes:
  • the method further includes:
  • the preset to-be-registered image and the preset reference image satisfying the preset training parameters are input to the preset neural network model to generate a deformation field.
  • the method further includes:
  • the image normalization processing is performed on the preset to-be-registered image and the preset reference image, and obtaining the preset to-be-registered image and the preset reference image that meet preset training parameters includes:
  • the method before processing the converted preset image to be registered and the preset reference image according to the target window width, the method further includes:
  • the method further includes:
  • the preset neural network model is updated with preset learning rate and preset threshold times.
  • a second aspect of an embodiment of the present application provides an image processing apparatus, including: an acquisition module and a registration module, wherein:
  • the acquisition module is used to acquire an image to be registered and a reference image for registration
  • the registration module is used to input the image to be registered and the reference image into a preset neural network model, and the target function for measuring similarity in the training of the preset neural network model includes the preset image to be registered and Correlation coefficient loss of preset reference image;
  • the registration module is further configured to register the image to be registered with the reference image based on the preset neural network model to obtain a registration result.
  • the image processing device further includes:
  • the preprocessing module is used to obtain the original image to be registered and the original reference image, perform image normalization processing on the original image to be registered and the original reference image, and obtain the image to be registered that meets the target parameter and The reference image.
  • the pre-processing module is specifically used to:
  • the registration module includes a registration unit and an update unit, wherein:
  • the registration unit is configured to acquire the preset image to be registered and the preset reference image, and input the preset image to be registered and the preset reference image into the preset neural network model to generate Deformation field
  • the registration unit is further configured to register the preset image to be registered with the preset reference image based on the deformation field to obtain the registered image;
  • the updating unit is used to obtain the correlation coefficient loss of the registered image and the preset reference image; and to update the preset neural network model parameters based on the correlation coefficient loss to obtain the after training The default neural network model.
  • the pre-processing module is also used to:
  • the registration unit is specifically configured to input the preset to-be-registered image and the preset reference image that satisfy the preset training parameters into the preset neural network model to generate a deformation field.
  • the pre-processing module is specifically used to:
  • the pre-processing module is further specifically used for:
  • the update unit is further used to:
  • the preset neural network model is updated with preset learning rate and preset threshold times.
  • a third aspect of an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory is used to store one or more programs, the one or more programs are configured to be executed by the processor, the The program includes some or all of the steps described in any method of the first aspect of the embodiments of the present application.
  • a fourth aspect of embodiments of the present application provides a computer-readable storage medium for storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the first aspect of the embodiment of the present application Part or all of the steps described in any method.
  • a fifth aspect of an embodiment of the present application provides a computer program, including computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device executes the method for implementing the above .
  • the image to be registered and the reference image are input to a preset neural network model, and the target function for measuring similarity in the training of the preset neural network model Including the loss of the correlation coefficient of the preset image to be registered and the preset reference image.
  • the image to be registered is registered with the reference image to obtain a registration result, which can improve the accuracy of image registration and real-time.
  • FIG. 1 is a schematic flowchart of an image processing method disclosed in an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a preset neural network model training method disclosed in an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of an image processing device disclosed in an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.
  • the image processing apparatus involved in the embodiments of the present application may allow multiple other terminal devices to access.
  • the above image processing apparatus may be an electronic device, including a terminal device.
  • the above terminal device includes, but is not limited to, a mobile phone, a laptop computer, or a tablet such as a touch-sensitive surface (eg, touch screen display and/or touch pad) Other portable devices such as computers.
  • the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (eg, touch screen display and/or touch pad).
  • Deep learning originates from the research of artificial neural networks.
  • a multi-layer perceptron with multiple hidden layers is a deep learning structure. Deep learning combines the low-level features to form a more abstract high-level representation attribute category or feature to discover the distributed feature representation of the data.
  • Deep learning is a method of machine learning based on representational learning of data. Observed values (for example, an image) can be expressed in many ways, such as a vector of intensity values for each pixel, or more abstractly expressed as a series of edges, areas of a specific shape, etc. However, it is easier to learn tasks from examples (for example, face recognition or facial expression recognition) using certain specific representation methods.
  • the benefit of deep learning is to use unsupervised or semi-supervised feature learning and hierarchical feature extraction efficient algorithms to replace manual feature acquisition. Deep learning is a new field in machine learning research. Its motivation lies in the establishment and simulation of the human brain for neural network analysis and learning. It mimics the mechanism of the human brain to interpret data, such as images, sounds, and text.
  • FIG. 1 is a schematic flowchart of an image processing disclosed in an embodiment of the present application. As shown in FIG. 1, the image processing method may be executed by the above image processing apparatus, including the following steps:
  • Image registration is the process of registering two or more images of the same scene or the same target under different acquisition times, different sensors, and different conditions, and is widely used in medical image processing.
  • Medical image registration is an important technology in the field of medical image processing and plays an increasingly important role in clinical diagnosis and treatment. Modern medicine usually requires comprehensive analysis of medical images obtained from multiple modalities or multiple time points, then several images need to be registered before analysis.
  • the image to be registered (moving) and the reference image (fixed) used for registration mentioned in the embodiments of the present application may be medical images obtained by various medical imaging devices, and particularly may be images of deformable organs, For example, lung CT, where the image to be registered and the reference image used for registration are generally images acquired by the same organ at different time points or under different conditions, and the registration result image (moved) can be obtained after registration.
  • the medical images that need to be registered may have diversity, the image gray value, image size and other characteristics of the image can be reflected in the diversity.
  • the original to-be-registered image and the original reference image may be acquired, and the original to-be-registered image and the original reference image are subjected to image normalization processing to obtain the to-be-registered that meets the target parameter Images and reference images.
  • the above target parameter can be understood as a parameter describing the characteristics of the image, that is, a predetermined parameter used to make the original image data have a uniform style.
  • the above target parameters may include parameters for describing features such as image resolution, image grayscale, and image size.
  • the above-mentioned original image to be registered may be a medical image obtained by various medical imaging devices, especially an image of a deformable organ, which has a variety of features, which can be reflected in the image gray value, image size and other characteristics. Sex.
  • some basic preprocessing may be performed on the original image to be registered and the original reference image, or only the above original image to be registered may be preprocessed. This may include the above image normalization process.
  • the main purpose of image preprocessing is to eliminate irrelevant information in the image, restore useful real information, enhance the detectability of the relevant information and simplify the data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition.
  • the image normalization in the embodiments of the present application refers to a process of performing a series of standard processing transformations on the image to transform it into a fixed standard form, and the standard image is called a normalized image.
  • Image normalization can use the invariant moment of the image to find a set of parameters that can eliminate the impact of other transformation functions on the image transformation, and convert the original image to be processed into the corresponding unique standard form.
  • the standard form image is translated and rotated. , Scaling and other affine transformations have invariant characteristics. Therefore, a uniform style image can be obtained through the above-mentioned image normalization processing, and the stability and accuracy of subsequent processing are improved.
  • the image to be registered and the reference image may also be a mask or a feature point extracted by an algorithm.
  • the mask can be understood as a template of an image filter, and the image mask can be understood as using selected images, graphics or objects to block the processed image (all or part) to control the image processing area or processing process.
  • the mask is generally a two-dimensional matrix array, and sometimes multi-valued images are used, which can be used for structural feature extraction.
  • the interference in image processing can be reduced, and the registration result is more accurate.
  • the above original to-be-registered image may be converted into a to-be-registered image within a preset gray value range and a preset image size;
  • the image processing device in the embodiment of the present application may store the preset gray value range and the preset image size.
  • the position and resolution of the image to be registered and the reference image can be kept basically the same through simple ITK software resample operation.
  • ITK is an open source cross-platform system that provides developers with a complete set of software tools for image analysis.
  • the above preset image size can be length, width and height: 416 x 416 x 80, and the image size of the image to be registered and the reference image can be made to be 416 x 416 x 80 by cutting or filling (zero padding) .
  • mapping relationship P For registration of two medical images 1 and 2 acquired at different times or/and under different conditions, it is to find a mapping relationship P so that each point on image 1 has a unique point on image 2 corresponding to it . And these two points should correspond to the same anatomical position.
  • the mapping relationship P appears as a continuous set of spatial transformations.
  • Commonly used spatial geometric transformations include rigid transformation (Rigid body transformation), affine transformation (Affine transformation), projection transformation (Projective transformation) and nonlinear transformation (Nonlinear transformation).
  • rigid transformation means that the distance and parallel relationship between any two points within the object remain unchanged.
  • Affine transformation is the simplest non-rigid transformation. It is a transformation that maintains parallelism but does not conform to the angle and changes the distance.
  • deformable image registration methods For example, when studying image registration of the abdomen and chest organs, the position, size and internal organs and tissues due to physiological movements or patient movements When the shape changes, deformable transformation is needed to compensate for the image distortion.
  • the above-mentioned pre-processing may further include the above-mentioned rigid transformation, that is, the rigid transformation of the image is performed first, and the upper image registration is implemented according to the method in the embodiment of the present application.
  • the above-mentioned preset neural network model may be stored in the image processing device, and the preset neural network model may be obtained by training in advance.
  • the above-mentioned preset neural network model can be obtained by training based on the loss of correlation coefficients, and specifically can be obtained by training based on the loss of correlation coefficients of the preset image to be registered and the preset reference image as a target function for measuring similarity.
  • the correlation coefficient mentioned in the examples of the present application is the earliest statistical index designed by statistician Karl Pearson, and is the quantity of the linear correlation between the variables studied, generally indicated by the letter r. Due to the different research objects, there are many ways to define the correlation coefficient, the more commonly used is the Pearson correlation coefficient.
  • the general correlation coefficient is calculated according to the product difference method, which is also based on the dispersion of the two variables and their respective averages, and the correlation between the two variables is reflected by multiplying the two dispersions; the linear single correlation coefficient is emphasized.
  • the Pearson correlation coefficient is not the only correlation coefficient, but it is a common correlation coefficient.
  • the correlation coefficient in the embodiment of the present application may be a Pearson correlation coefficient.
  • the feature maps of the registered image and the preset reference image may be extracted through features, and the correlation coefficient loss between the feature maps may be used to obtain the aforementioned correlation coefficient loss.
  • F may represent the above-mentioned preset reference image
  • M( ⁇ ) may represent the above-mentioned registered image
  • ⁇ can represent the nonlinear relationship represented by the neural network.
  • Plus triangle Respectively represent the average value of the image after registration and the parameter average of the preset reference image. such as Represents the mean value of the preset reference image, then the above subtraction It can be understood that each pixel value of the foregoing preset reference image minus the average value of the parameter, and so on.
  • the training process of the aforementioned preset neural network model may include:
  • the loss function used in the deformation field generation may include an L2 loss function, so that the preset neural network model learns an appropriate deformation field to make the moved image and the fixed image more similar.
  • Image registration is generally to first extract feature points from two images to obtain feature points; then find the matching feature point pairs by performing similarity measurement; then obtain the image space coordinate transformation parameters from the matched feature point pairs; and finally perform the coordinate transformation parameters Image registration.
  • the convolutional layer of the preset neural network model in the embodiment of the present application may be a 3D convolution, a deformable field is generated through the above-mentioned preset neural network model, and then a deformation to be registered is required through a 3D spatial conversion layer
  • the image is deformably transformed to obtain the above registration result after registration, that is, including the generated registration result image (moved).
  • the L2 loss and the correlation coefficient are used as the loss function, which can achieve the advanced registration accuracy while smoothing the deformation field.
  • the existing method is to use supervised deep learning for registration. There is basically no gold standard.
  • the traditional registration method must be used to obtain the mark. The processing time is longer and the registration accuracy is limited.
  • the traditional method for registration needs to calculate the transformation relationship of each pixel, which is huge in calculation and consumes a lot of time.
  • unsupervised learning Solving various problems in pattern recognition based on training samples with unknown categories (not labeled) is called unsupervised learning.
  • the embodiments of the present application use a neural network based on unsupervised deep learning to perform image registration, and can be used for registration of any organs that may undergo deformation.
  • the embodiment of the present application can use the GPU to execute the above method to obtain a registration result within a few seconds, which is more efficient.
  • the image to be registered and the reference image are input to a preset neural network model, and the target function for measuring similarity in the training of the preset neural network model Including the loss of the correlation coefficient of the preset image to be registered and the preset reference image.
  • the image to be registered is registered with the reference image to obtain a registration result, which can improve the accuracy of image registration and real-time.
  • FIG. 2 is a schematic flowchart of another image processing method disclosed in an embodiment of the present application, specifically a schematic flowchart of a preset neural network training method.
  • FIG. 2 is further optimized on the basis of FIG. owned.
  • the subject performing the steps of the embodiments of the present application may be an image processing device, which may be the same or different image processing device as in the method of the embodiment shown in FIG. 1.
  • the image processing method includes the following steps:
  • the above-mentioned preset to-be-registered image (moving) and the above-mentioned preset reference image (fixed) can both be medical images obtained by various medical imaging devices, and in particular can be Images of deformable organs, such as lung CT, where the image to be registered and the reference image used for registration are generally images acquired by the same organ at different time points or under different conditions.
  • the term "preset" is to distinguish it from the image to be registered and the reference image in the embodiment shown in FIG. 1, where the preset image to be registered and the preset reference image are mainly used as the input of the preset neural network model For training the preset neural network model.
  • the method may also include:
  • inputting the preset image to be registered and the preset reference image into the preset neural network model to generate a deformation field includes:
  • the preset to-be-registered image and the preset reference image satisfying the preset training parameters are input into the preset neural network model to generate a deformation field.
  • the above-mentioned preset training parameters may include a preset gray value range and a preset image size (such as 416 x 416 x 80).
  • a preset gray value range such as 416 x 416 x 80.
  • the pre-processing first performed before registration may include rigid body transformation.
  • the simple ITK software can be used for resampling to make the positions and resolutions of the preset image to be registered and the preset reference image basically the same.
  • the image can be cropped or filled with a predetermined size.
  • the image size of the preset to-be-registered image and the preset reference image need to be the same by cutting or filling (zero padding) operation It is 416 x 416 x 80.
  • the converted preset image to be registered and the preset reference image may be processed according to the target window width to obtain the processed preset image to be registered and the preset reference image.
  • the corresponding gray levels may be different.
  • windowing refers to the process of calculating the image using the data obtained from the Hounsfield Unit (HU). Different radiation intensity (Raiodensity) corresponds to 256 different degrees. Gray scale value. These different gray scale values can be used to redefine the attenuation value according to the different range of CT value. Assuming that the central value of the CT range remains unchanged, once the defined range becomes narrow, we call it narrow window (Narrow Window) , Small changes in more detailed parts can be distinguished, which is called contrast compression in the concept of image processing.
  • the target window width can be set in advance, for example, the target window width is [-1200, 600] and the preset image to be registered and the preset reference image are normalized to [0, 1], That is, the original image is set to 1 greater than 600, and less than -1200 set to 0.
  • different organizations may set recognized window widths and window positions on the CT, so as to better extract important information.
  • the specific value of [-1200, 600] here -1200, 600 represents the window level, the range size is 1800, that is, the window width.
  • the above image normalization processing is to facilitate subsequent loss calculation without causing gradient explosion.
  • the embodiment of the present application proposes a normalization layer to improve the stability and convergence of training.
  • the size of the feature map is N ⁇ xC ⁇ D ⁇ xH ⁇ W, where N refers to batch size: the size of each batch of data, C is the number of channels, D is the depth, and H and W are the height and height of the feature map, respectively.
  • Width optionally, the above H, W, and D can also be parameters representing the length, width, and height of the feature map, respectively.
  • other image parameters can be used to describe the feature map.
  • the minimum and maximum values of C ⁇ D ⁇ H ⁇ W can be calculated to perform a normalized operation on each image data.
  • the method before processing the converted preset image to be registered and the preset reference image according to the preset window width, the method further includes:
  • the image processing apparatus may store at least one preset window width and at least one preset category label, and store the correspondence between the aforementioned preset category label and the preset window width, and the input preset image to be registered may be carried
  • the target category label or the user can select the target category label of the preset image to be registered by operating the image processing device.
  • the image processing device can find the target category label in the preset category label, according to the preset category label and Correspondence of the preset window width, determine the target window width corresponding to the target category label in the preset window width, and then process the converted preset image to be registered and the preset reference image according to the target window width .
  • the image processing device can quickly and flexibly select different preset window widths for image processing to be registered, which is convenient for subsequent registration processing.
  • the L2 loss function can be used for the gradient of the deformation field.
  • the above-mentioned registered image is an intermediate image after initial registration of the preset image to be registered to the preset reference image through the preset neural network model.
  • This process can be understood as multiple executions, that is, steps 202 and 203 can be repeated In order to continuously train and optimize the preset neural network model.
  • the correlation coefficient loss is used as the similarity evaluation standard of the registered image and the reference image, that is, steps 202 and 203 can be repeatedly executed to continuously update the parameters of the preset neural network model to guide Complete the network training.
  • the preset neural network model may be updated with a preset learning rate and a preset threshold value based on a preset optimizer.
  • the preset threshold times involved in the above update refer to the epoch in neural network training.
  • a period can be understood as a forward pass and a backward pass of all training samples.
  • the algorithm used in the optimizer generally has an adaptive gradient optimization algorithm (Adaptive Gradient, AdaGrad), which can adjust different learning rates for each different parameter, update the frequently changed parameters in smaller steps, and sparse The parameters are updated in larger steps; and the RMSProp algorithm, combined with the exponential moving average of the squared gradient to adjust the change in the learning rate, can converge well under the unstable (Non-Stationary) objective function.
  • AdaGrad adaptive Gradient, AdaGrad
  • the above-mentioned preset optimizer can use the ADAM optimizer, combining the advantages of the two optimization algorithms AdaGrad and RMSProp.
  • the first-order moment estimation (First Meanment Estimation of gradient) and the second-order moment estimation (SecondMoment Estimation, that is, the uncentralized variance of gradient) are considered comprehensively, and the update step size is calculated.
  • the image processing apparatus or the preset optimizer may store the preset threshold times and the preset learning rate to control update.
  • the learning rate is 0.001
  • the preset threshold is 300epoch.
  • the learning rate adjustment rule can be set, and the learning rate of the parameter update can be adjusted by the learning rate adjustment rule, for example, the learning rate can be halved at 40, 120, and 200 epoch, respectively.
  • the image processing apparatus may execute some or all of the methods in the embodiment shown in FIG. 1, that is, the image to be registered may be registered to the reference image based on the preset neural network model. To get the registration result.
  • Non-parametric methods for estimating mutual information are not only computationally intensive but also do not support backpropagation and cannot be applied to neural networks.
  • the embodiment of the present application uses the correlation coefficient of the local window as the similarity measurement loss.
  • the preset neural network model after training can be used for image registration, especially in the medical image registration of any deformable organs. The follow-up images at the time point are deformed for registration, which has high registration efficiency and more accurate results.
  • various scans of different quality and speed need to be performed before or during surgery to obtain medical images, but usually medical images are registered after various scans are completed, which is not satisfactory for surgery.
  • the real-time requirements are required, so it is generally necessary to determine the results of the operation through additional time. If the results of the operation are found to be unsatisfactory after registration, subsequent surgical treatment may be required, which will be a waste of time for doctors and patients. , Delay treatment.
  • the registration based on the preset neural network model of the embodiment of the present application can be applied to real-time medical image registration during surgery, such as real-time registration during tumor resection surgery to determine whether the tumor is completely removed, which improves timeliness .
  • the preset to-be-registered image and the preset reference image are obtained, the preset to-be-registered image and the preset reference image are input into the preset neural network model to generate a deformation field, and the above-mentioned
  • the image to be registered is registered with the preset reference image to obtain the registered image
  • the correlation coefficient loss between the registered image and the preset reference image is obtained, and the preset neural network model is performed based on the correlation coefficient loss.
  • the parameters are updated to obtain the preset neural network model after training, which can be applied to deformable registration to improve the accuracy and real-time performance of image registration.
  • the image processing device includes a hardware structure and/or a software module corresponding to each function.
  • the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driven hardware depends on the specific application of the technical solution and design constraints. A person skilled in the art may use different methods to implement the described functions for a specific application, but such implementation should not be considered beyond the scope of the present disclosure.
  • the image processing apparatus may be divided into function modules according to the above method examples.
  • each function module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules can be implemented in the form of hardware or software function modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a division of logical functions. In actual implementation, there may be another division manner.
  • FIG. 3 is a schematic structural diagram of an image processing apparatus disclosed in an embodiment of the present application.
  • the image processing apparatus 300 includes an acquisition module 310 and a registration module 320, where:
  • the above acquisition module 310 is used to acquire the image to be registered and the reference image used for registration;
  • the registration module 320 is configured to input the image to be registered and the reference image into a preset neural network model.
  • the target function for measuring similarity in the training of the preset neural network model includes a preset image to be registered and a preset reference Image correlation coefficient loss;
  • the registration module 320 is further configured to register the image to be registered with the reference image based on the preset neural network model to obtain a registration result.
  • the above image processing device 300 further includes: a preprocessing module 330, configured to obtain an original image to be registered and an original reference image, and perform image normalization processing on the original image to be registered and the original reference image to obtain The above-mentioned image to be registered and the above-mentioned reference image satisfying the target parameter.
  • a preprocessing module 330 configured to obtain an original image to be registered and an original reference image, and perform image normalization processing on the original image to be registered and the original reference image to obtain The above-mentioned image to be registered and the above-mentioned reference image satisfying the target parameter.
  • the above preprocessing module 330 is specifically used for:
  • the above registration module 320 includes a registration unit 321 and an update unit 322, where:
  • the registration unit 321 is configured to acquire the preset image to be registered and the preset reference image, and input the preset image to be registered and the preset reference image into the preset neural network model to generate a deformation field;
  • the registration unit 321 is further configured to register the preset image to be registered with the preset reference image based on the deformation field to obtain a registered image;
  • the updating unit 322 is used to obtain the correlation coefficient loss of the registered image and the preset reference image; and to update the preset neural network model parameters based on the correlation coefficient loss to obtain the trained preset nerve Network model.
  • the above preprocessing module 330 is also used to:
  • the registration unit 321 is specifically configured to input the preset to-be-registered image and the preset reference image that satisfy the preset training parameters into the preset neural network model to generate a deformation field.
  • the above preprocessing module 330 is specifically used for:
  • the above preprocessing module 330 is also specifically used for:
  • the update unit 322 is also used to:
  • the preset neural network model is preset to update the preset learning rate and preset threshold times.
  • the image processing device 300 in the embodiment shown in FIG. 3 may perform some or all of the methods in the embodiment shown in FIG. 1 and/or FIG. 2.
  • the image processing apparatus 300 may acquire the image to be registered and the reference image for registration, and input the image to be registered and the reference image into a preset neural network model, the preset
  • the target function for measuring the similarity in the training of the neural network model includes the loss of the correlation coefficient of the preset image to be registered and the preset reference image.
  • the image to be registered is registered with the reference image to obtain a registration
  • the accurate result can improve the accuracy and real-time of image registration.
  • FIG. 4 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.
  • the electronic device 400 includes a processor 401 and a memory 402, wherein the electronic device 400 may further include a bus 403, the processor 401 and the memory 402 may be connected to each other through the bus 403, and the bus 403 may be a peripheral component Peripheral Component Interconnect (PCI) bus or Extended Industry Standard Architecture (EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the bus 403 can be divided into an address bus, a data bus, and a control bus. For ease of representation, only a thick line is used in FIG. 4, but it does not mean that there is only one bus or one type of bus.
  • the electronic device 400 may further include an input and output device 404, and the input and output device 404 may include a display screen, such as a liquid crystal display screen.
  • the memory 402 is used to store one or more programs containing instructions; the processor 401 is used to call the instructions stored in the memory 402 to perform some or all of the method steps mentioned in the embodiments of FIGS. 1 and 2 above.
  • the above processor 401 may correspondingly implement the functions of each module in the electronic device 300 in FIG. 3.
  • the electronic device 400 can acquire the image to be registered and the reference image for registration, and input the image to be registered and the reference image into a preset neural network model, and the preset neural network
  • the objective function for measuring the similarity in model training includes the loss of the correlation coefficient of the preset image to be registered and the preset reference image. Based on the preset neural network model, the image to be registered is registered with the reference image to obtain a registration result , Can improve the accuracy and real-time nature of image registration.
  • An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute any of the images described in the foregoing method embodiments Some or all steps of the processing method.
  • Embodiments of the present application also provide a computer program, including computer readable code.
  • the processor in the electronic device executes any one of the methods described in the foregoing method embodiments. Or all steps of an image processing method.
  • the disclosed device may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the modules (or units) is only a division of logical functions.
  • there may be additional divisions, such as multiple modules or components. Can be combined or integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or modules, and may be in electrical or other forms.
  • modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules.
  • the integrated module is implemented in the form of a software function module and sold or used as an independent product, it may be stored in a computer-readable memory.
  • the technical solution of the present invention essentially or part of the contribution to the existing technology or all or part of the technical solution can be embodied in the form of a software product, the computer software product is stored in a memory, Several instructions are included to enable a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the aforementioned memory includes: U disk, Read-Only Memory (ROM), Random Access Memory (Random Access Memory, RAM), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
  • the program may be stored in a computer-readable memory, and the memory may include: a flash disk , Read-only memory, random access device, magnetic disk or optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例公开了一种图像处理方法、装置、电子设备及计算机可读存储介质，其中方法包括：获取待配准图像和用于配准的参考图像；将所述待配准图像和所述参考图像输入预设神经网络模型，所述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；基于所述预设神经网络模型将所述待配准图像向所述参考图像配准，获得配准结果，可以提高图像配准的精度和实时性。

Description

图像处理方法、装置、电子设备及计算机可读存储介质
本公开要求在2018年12月27日提交中国专利局、申请号为201811614468.4、申请名称为“图像处理方法、装置、电子设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本发明涉及计算机视觉技术领域，具体涉及图像处理方法、装置、电子设备及计算机可读存储介质。
背景技术
图像配准是将不同的获取时间、不同传感器、不同条件下的同一场景或者同一目标的两幅或者多幅图像进行配准的过程，被广泛应用于医学图像处理过程中。医学图像配准是医学图像处理领域中一项重要技术，对临床诊断和治疗起着越来越重要的作用。
发明内容
本申请实施例提供了图像处理方法、装置、电子设备及计算机可读存储介质。
本申请实施例第一方面提供一种图像处理方法，包括：
获取待配准图像和用于配准的参考图像；
将所述待配准图像和所述参考图像输入预设神经网络模型，所述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；
基于所述预设神经网络模型将所述待配准图像向所述参考图像配准，获得配准结果。
在一种可选的实施方式中，所述获取待配准图像和用于配准的参考图像之前，所述方法还包括：
获取原始待配准图像和原始参考图像，对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的待配准图像和参考图像。
在一种可选的实施方式中，所述对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的所述待配准图像和所述参考图像包括：
将所述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
将所述原始参考图像转换为所述预设灰度值范围内和所述预设图像尺寸的参考图像。
在一种可选的实施方式中，所述预设神经网络模型的训练过程包括：
获取所述预设待配准图像和所述预设参考图像，将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场；
基于所述形变场将所述预设待配准图像向所述预设参考图像配准，获得配准后图像；
获得所述配准后图像和所述预设参考图像的相关系数损失；
基于所述相关系数损失对所述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
在一种可选的实施方式中，所述获取所述预设待配准图像和所述预设参考图像之后，所述方法还包括：
对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
所述将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场包括：
将所述满足预设训练参数的预设待配准图像和预设参考图像输入所述预设神经网络模型生成形变场。
在一种可选的实施方式中，所述方法还包括：
将所述预设待配准图像的尺寸和所述预设参考图像的尺寸转换为预设图像尺寸；
所述对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像包括：
根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
在一种可选的实施方式中，所述根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理之前，所述方法还包括：
获取所述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定所述目标类别标签对应的所述目标窗宽。
在一种可选的实施方式中，所述方法还包括：
基于预设优化器对所述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
本申请实施例第二方面提供一种图像处理装置，包括：获取模块和配准模块，其中：
所述获取模块，用于获取待配准图像和用于配准的参考图像；
所述配准模块，用于将所述待配准图像和所述参考图像输入预设神经网络模型，所述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；
所述配准模块，还用于基于所述预设神经网络模型将所述待配准图像向所述参考图像配准，获得配准结果。
在一种可选的实施方式中，所述图像处理装置还包括：
预处理模块，用于获取原始待配准图像和原始参考图像，对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的所述待配准图像和所述参考图像。
在一种可选的实施方式中，所述预处理模块具体用于：
将所述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
将所述原始参考图像转换为所述预设灰度值范围内和所述预设图像尺寸的参考图像。
在一种可选的实施方式中，所述配准模块包括配准单元和更新单元，其中：
所述配准单元用于，获取所述预设待配准图像和所述预设参考图像，将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场；
所述配准单元还用于，基于所述形变场将所述预设待配准图像向所述预设参考图像配准，获得配准后图像；
所述更新单元用于，获得所述配准后图像和所述预设参考图像的相关系数损失；以及用于基于所述相关系数损失对所述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
在一种可选的实施方式中，所述预处理模块还用于：
对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
所述配准单元具体用于，将所述满足预设训练参数的预设待配准图像和预设参考图像输入所述预设神经网络模型生成形变场。
在一种可选的实施方式中，所述预处理模块具体用于：
将所述预设待配准图像的尺寸和所述预设参考图像的尺寸转换为预设图像尺寸；
根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
在一种可选的实施方式中，所述预处理模块还具体用于：
在所述根据预设窗宽对所述转换后的预设待配准图像和预设参考图像进行处理之前，获取所述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定所述目标类别标签对应的所述目标窗宽。
在一种可选的实施方式中，所述更新单元还用于：
基于预设优化器对所述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
本申请实施例第三方面提供一种电子设备，包括处理器以及存储器，所述存储器用于存储一个或多个程序，所述一个或多个程序被配置成由所述处理器执行，所述程序包括用于执行如本申请实施例第一方面任一方法中所描述的部分或全部步骤。
本申请实施例第四方面提供一种计算机可读存储介质，所述计算机可读存储介质用于存储电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如本申请实施例第一方面任一方法中所描述的部分或全部步骤。
本申请实施例第五方面提供一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现如上所述的方法。
本申请实施例通过获取待配准图像和用于配准的参考图像，将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失，基于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果，可以提高图像配准的精度和实时性。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是本申请实施例公开的一种图像处理方法的流程示意图；
图2是本申请实施例公开的一种预设神经网络模型训练方法的流程示意图；
图3是本申请实施例公开的一种图像处理装置的结构示意图；
图4是本申请实施例公开的一种电子设备的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有 作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本发明的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
本申请实施例所涉及到的图像处理装置可以允许多个其他终端设备进行访问。上述图像处理装置可以为电子设备，包括终端设备，具体实现中，上述终端设备包括但不限于诸如具有触摸敏感表面(例如，触摸屏显示器和/或触摸板)的移动电话、膝上型计算机或平板计算机之类的其它便携式设备。还应当理解的是，在某些实施例中，所述设备并非便携式通信设备，而是具有触摸敏感表面(例如，触摸屏显示器和/或触摸板)的台式计算机。
本申请实施例中的深度学习的概念源于人工神经网络的研究。含多隐层的多层感知器就是一种深度学习结构。深度学习通过组合低层特征形成更加抽象的高层表示属性类别或特征，以发现数据的分布式特征表示。
深度学习是机器学习中一种基于对数据进行表征学习的方法。观测值(例如一幅图像)可以使用多种方式来表示，如每个像素点强度值的向量，或者更抽象地表示成一系列边、特定形状的区域等。而使用某些特定的表示方法更容易从实例中学习任务(例如，人脸识别或面部表情识别)。深度学习的好处是用非监督式或半监督式的特征学习和分层特征提取高效算法来替代手工获取特征。深度学习是机器学习研究中的一个新的领域，其动机在于建立、模拟人脑进行分析学习的神经网络，它模仿人脑的机制来解释数据，例如图像，声音和文本。
下面对本申请实施例进行详细介绍。
请参阅图1，图1是本申请实施例公开的一种图像处理的流程示意图，如图1所示，该图像处理方法可以由上述图像处理装置执行，包括如下步骤：
101、获取待配准图像和用于配准的参考图像。
图像配准是将不同的获取时间、不同传感器、不同条件下的同一场景或者同一目标的两幅或者多幅图像进行配准的过程，被广泛应用于医学图像处理过程中。医学图像配准是医学 图像处理领域中一项重要技术，对临床诊断和治疗起着越来越重要的作用。现代医学通常需要将多个模态或者多个时间点获得的医学图像进行综合分析，那么在进行分析之前就需要将几副图像进行配准工作。
本申请实施例中提到的待配准图像(moving)和用于配准的参考图像(fixed)均可以为通过各种医学图像设备获得的医学图像，尤其可以是可形变的器官的图像，比如肺部CT，其中待配准图像和用于配准的参考图像一般为同一器官在不同时间点或不同条件下采集的图像，经过配准后可以获得配准结果图像(moved)。
由于需要进行配准的医学图像可能具有多样性，在图像中可以体现为图像灰度值、图像尺寸等特征的多样性。可选的，在步骤101之前，可以获取原始待配准图像和原始参考图像，对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的待配准图像和参考图像。
上述目标参数可以理解为描述图像特征的参数，即用于使上述原始图像数据呈统一风格的规定参数。例如，上述目标参数可以包括：用于描述图像分辨率、图像灰度、图像大小等特征的参数。
上述原始待配准图像可以为通过各种医学图像设备获得的医学图像，尤其可以是可形变的器官的图像，具有多样性，在图像中可以体现为图像灰度值、图像尺寸等特征的多样性。在进行配准前可以对原始待配准图像和原始参考图像做一些基本的预处理，也可以仅对上述原始待配准图像进行预处理。其中可以包括上述图像归一化处理。图像预处理的主要目的是消除图像中无关的信息，恢复有用的真实信息，增强有关信息的可检测性和最大限度地简化数据，从而改进特征抽取、图像分割、匹配和识别的可靠性。
本申请实施例中的图像归一化是指对图像进行一系列标准的处理变换，使之变换为一固定标准形式的过程，该标准图像称作归一化图像。图像归一化可以利用图像的不变矩寻找一组参数使其能够消除其他变换函数对图像变换的影响，将待处理的原始图像转换成相应的唯一标准形式，该标准形式图像对平移、旋转、缩放等仿射变换具有不变特性。因此，通过上述图像归一化处理可以获得统一风格的图像，提高后续处理的稳定性和准确度。
可选的，上述待配准图像和参考图像也可以是通过算法提取出的掩膜(mask)或者特征点。其中掩膜可以理解为一种图像滤镜的模板，图像掩膜可以理解为用选定的图像、图形或物体，对处理的图像(全部或局部)进行遮挡，来控制图像处理的区域或处理过程。数字图像处理中掩模一般为二维矩阵数组，有时也用多值图像，可以用于结构特征提取。
在提取特征或mask后，可以减少图像处理中的干扰，使得配准结果更准确。
具体的，可以将上述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
将上述原始参考图像转换为上述预设灰度值范围内和上述预设图像尺寸的参考图像。
本申请实施例中的图像处理装置可以存储有上述预设灰度值范围和上述预设图像尺寸。可以通过simple ITK软件做重采样(resample)的操作来使得上述待配准图像和上述参考图像的位置和分辨率基本保持一致。ITK是一个开源的跨平台系统，为开发人员提供了一整套用于图像分析的软件工具。
上述预设图像尺寸可以为长宽高：416 x 416 x 80，可以通过剪切或者填充(补零)的操作来使得上述待配准图像和上述参考图像的图像尺寸一致为416 x 416 x 80。
通过对原始图像数据进行预处理，可以降低其多样性，神经网络模型能够给出更稳定的判断。
对于在不同时间或/和不同条件下获取的两幅医学图像1和2配准，就是寻找一个映射关系P，使图像1上的每一个点在图像2上都有唯一的点与之相对应。并且这两点应对应同一解剖位置。映射关系P表现为一组连续的空间变换。常用的空间几何变换有刚体变换(Rigid body transformation)、仿射变换(Affine transformation)、投影变换(Projective transformation)和非线性变换(Nonlinear transformation)。
其中，刚性变换是指物体内部任意两点间的距离及平行关系保持不变。仿射变换是一种最为简单的非刚性变换，它一种保持平行性，但不保角的、距离发生变化的变换。而在许多重要的临床应用中，就经常需要应用可形变的图像配准方法，比如在研究腹部以及胸部脏器的图像配准时，由于生理运动或者患者移动造成内部器官和组织的位置、尺寸和形态发生改变，就需要可形变变换来补偿图像变形。
在本申请实施例中，上述预处理还可以包括上述刚性变换，即先进行图像的刚性变换，在根据本申请实施例中的方法实现上图像配准。
在图像处理领域，只有物体的位置(平移变换)和朝向(旋转变换)发生改变，而形状不变，得到的变换称为上述刚性变换。
102、将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失。
本申请实施例中，图像处理装置中可以存储有上述预设神经网络模型，该预设神经网络模型可以预先训练获得。
上述预设神经网络模型可以是基于相关系数损失进行训练获得，具体可以基于预设待配 准图像和预设参考图像的相关系数损失作为衡量相似度的目标函数进行训练获得。
本申请实施例中提到的相关系数是最早由统计学家卡尔·皮尔逊设计的统计指标，是研究变量之间线性相关程度的量，一般用字母r表示。由于研究对象的不同，相关系数有多种定义方式，较为常用的是皮尔逊相关系数。
一般相关系数是按积差方法计算，同样以两变量与各自平均值的离差为基础，通过两个离差相乘来反映两变量之间相关程度；着重研究线性的单相关系数。需要说明的是，皮尔逊相关系数并不是唯一的相关系数，但是为常见的相关系数，本申请实施例中的相关系数可以为皮尔逊相关系数。
具体的，在预设神经网络模型中可以通过特征提取配准后图像和预设参考图像的特征图，利用特征图之间的互相关系数，得到上述相关系数损失。
上述相关系数损失的可以基于以下公式(1)获得：
Figure PCTCN2019120329-appb-000001
其中，F可以表示上述预设参考图像，M(φ)可以表示上述配准后图像。φ可以表示神经网络代表的非线性关系。加上三角符号的
Figure PCTCN2019120329-appb-000002
分别表示配准后图像的均值和预设参考图像的参数均值。比如
Figure PCTCN2019120329-appb-000003
表示预设参考图像的参数均值，那么上述减法
Figure PCTCN2019120329-appb-000004
则可以理解为上述预设参考图像的每个像素值减掉参数均值，以此类推。
上述预设神经网络模型的训练过程可以包括：
获取上述预设待配准图像和上述预设参考图像，将上述预设待配准图像和上述预设参考图像输入上述预设神经网络模型生成形变场；
基于上述形变场将上述预设待配准图像向上述预设参考图像配准，获得配准后图像；
获得上述配准后图像和上述预设参考图像的相关系数损失；
基于上述相关系数损失对上述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
具体的，上述形变场生使用的损失函数可以包括L2损失函数，使预设神经网络模型学习到合适的形变场使moved图像和fixed图像更相似。
103、基于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果。
图像配准一般是首先对两幅图像进行特征提取得到特征点；再通过进行相似性度量找到匹配的特征点对；然后通过匹配的特征点对得到图像空间坐标变换参数；最后由坐标变换参数进行图像配准。
本申请实施例中的预设神经网络模型的卷积层可以为3D卷积，通过上述预设神经网络模型生成形变场(deformable field)，然后通过3D的空间转换层将需要形变的待配准图像进行可形变的变换，获得配准后的上述配准结果，即包括生成的配准结果图像(moved)。
其中，上述预设神经网络模型中，使用L2损失和相关系数作为损失函数，可以在使上述形变场平滑的同时达到先进的配准精度。
现有的方法是利用有监督深度学习来做配准，基本没有金标准，必须利用的、传统配准方法来获得标记，处理时间较长，且限制了配准精度。并且利用传统方法做配准需要计算每个像素点的变换关系，计算量巨大，消耗时间也很大。
根据类别未知(没有被标记)的训练样本解决模式识别中的各种问题，称之为无监督学习。本申请实施例使用基于无监督深度学习的神经网络来进行图像配准，可用于任何会发生形变的脏器的配准中。本申请实施例可以利用GPU执行上述方法在几秒内得到配准结果，更加高效。
本申请实施例通过获取待配准图像和用于配准的参考图像，将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失，基于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果，可以提高图像配准的精度和实时性。
请参阅图2，图2是本申请实施例公开的另一种图像处理方法的流程示意图，具体为一种预设神经网络的训练方法的流程示意图，图2是在图1的基础上进一步优化得到的。执行本申请实施例步骤的主体可以为一种图像处理装置，可以是与图1所示实施例的方法中相同或者不同的图像处理装置。如图2所示，该图像处理方法包括如下步骤：
201、获取预设待配准图像和预设参考图像，将上述预设待配准图像和上述预设参考图像输入上述预设神经网络模型生成形变场。
其中，与图1所示实施例中类似的，上述预设待配准图像(moving)和上述预设参考图像(fixed)，均可以为通过各种医学图像设备获得的医学图像，尤其可以是可形变的器官的图像，比如肺部CT，其中待配准图像和用于配准的参考图像一般为同一器官在不同时间点或不同条件下采集的图像。“预设”一词是为了区别于图1所示实施例中的待配准图像和参考图像区别，这里的预设待配准图像和预设参考图像主要作为该预设神经网络模型的输入，用 于进行该预设神经网络模型的训练。
由于需要进行配准的医学图像可能具有多样性，在图像中可以体现为图像灰度值、图像尺寸等特征的多样性。可选的，上述获取上述预设待配准图像和上述预设参考图像之后，上述方法也可以包括：
对上述预设待配准图像和上述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
其中，上述将上述预设待配准图像和上述预设参考图像输入上述预设神经网络模型生成形变场包括：
将上述满足预设训练参数的预设待配准图像和预设参考图像输入上述预设神经网络模型生成形变场。
上述预设训练参数可以包括预设灰度值范围和预设图像尺寸(如416 x 416 x 80)。上述图像归一化处理的过程可以参考图1所示实施例的步骤101中的具体描述。可选的，首先在配准前进行的预处理可以包括刚体变换。具体可以通过simple ITK软件做重采样的操作来使得预设待配准图像和预设参考图像的位置和分辨率基本保持一致。为了后续训练过程的方便操作，可以对图像进行预定大小的裁剪或者填充。假设预先设定的输入图像的图像尺寸长宽高为416 x 416 x 80，就需要通过剪切或者填充(补零)的操作来使得预设待配准图像和预设参考图像的图像尺寸一致为416 x 416 x 80。
可选的，可以根据目标窗宽对上述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
因为不同的器官组织在CT上的表现是不一样的，也就是对应的灰度级别可能不同。所谓的窗宽(windowing)就是指用韩森费尔德(发明者)单位(Hounsfield Unit，HU)所得的数据来计算出影像的过程，不同的放射强度(Raiodensity)对应到256种不同程度的灰阶值，这些不同的灰阶值可以依CT值的不同范围来重新定义衰减值，假设CT范围的中心值不变，定义的范围一变窄后，我们称为窄窗位(Narrow Window)，比较细部的小变化就可以分辨出来了，在影像处理的观念上称为对比压缩。
为了肺部CT中的重要信息，可以预先设置目标窗宽，比如通过目标窗宽为[-1200，600]对预设待配准图像和预设参考图像归一化到[0，1]，即对于原图像中大于600的设为1，小于-1200的设为0。
本申请实施例中不同组织在CT上可以设置公认的窗宽、窗位，是为了更好地提取重要的信息。这里的[-1200，600]的具体值-1200，600代表的是窗位，范围大小为1800，即窗宽。 上述图像归一化处理是为了方便后续的损失计算不造成梯度爆炸。
本申请实施例提出一种归一化层来提升训练的稳定性和收敛性。可以假设特征图大小为N x C x D x H x W，其中N指的是batch size：每批数据量的大小，C是通道数，D是深度，H和W分别为特征图的高和宽；可选的，上述H、W、D也可以分别为表示特征图的长、宽、高的参数，在不同的应用中可以是其他图像参数来描述特征图。本申请实施例可以通过计算C x D x H x W的最小值和最大值，来对每个图像数据做归一化处理操作。
可选的，上述根据预设窗宽对上述转换后的预设待配准图像和预设参考图像进行处理之前，上述方法还包括：
获取上述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定上述目标类别标签对应的上述目标窗宽。
具体的，图像处理装置可以存储有至少一个预设窗宽和至少一个预设类别标签，以及存储有上述预设类别标签与预设窗宽的对应关系，输入的预设待配准图像可以携带目标类别标签，或者用户可以通过操作图像处理装置选取该预设待配准图像的目标类别标签，图像处理装置可以在上述预设类别标签中查找到上述目标类别标签，根据上述预设类别标签与预设窗宽的对应关系，在上述预设窗宽中确定上述目标类别标签对应的目标窗宽，再根据该目标窗宽对上述转换后的预设待配准图像和预设参考图像进行处理。
通过上述步骤，图像处理装置可以快速灵活地选取不同的预设待配准图像处理使用的窗宽，便于进行后续的配准处理。
202、基于上述形变场将上述预设待配准图像向上述预设参考图像配准，获得配准后图像。
其中，由于L2具有光滑的性质，对于形变场的梯度可以使用L2损失函数。
将预处理过后的预设待配准图像和预设参考图像输入到待训练的神经网络中生成形变场(deformable field)，再基于上述形变场和上述预设待配准图像向上述预设参考图像配准，即利用该形变场和预设参考图像生成形变后的配准结果图像(moved)。
上述配准后图像即为预设待配准图像经过预设神经网络模型向预设参考图像初步配准后的中间图像，这个过程可以理解为多次执行，即可以重复执行步骤202和步骤203以不断训练和优化该预设神经网络模型。
203、获得上述配准后图像和上述预设参考图像的相关系数损失，基于上述相关系数损失对上述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
本申请实施例中，通过相关系数损失作为配准后的图像和参考图像的相似度评估标准， 即可以重复执行步骤202和步骤203，不断对上述预设神经网络模型的参数进行更新，来指导完成网络的训练。
可选的，可以基于预设优化器对所述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
上述更新时涉及的预设阈值次数，指的是神经网络训练中的时期(epoch)。一个时期可以理解为所有训练样本的一个正向传递和一个反向传递。
优化器中使用的算法一般有自适应梯度优化算法(Adaptive Gradient，AdaGrad)，它能够对每个不同的参数调整不同的学习率，对频繁变化的参数以更小的步长进行更新，而稀疏的参数以更大的步长进行更新；以及RMSProp算法，结合梯度平方的指数移动平均数来调节学习率的变化，能够在不稳定(Non-Stationary)的目标函数情况下进行很好地收敛。
具体的，上述预设优化器可以采用ADAM的优化器，结合AdaGrad和RMSProp两种优化算法的优点。对梯度的一阶矩估计(First Moment Estimation，即梯度的均值)和二阶矩估计(SecondMoment Estimation，即梯度的未中心化的方差)进行综合考虑，计算出更新步长。
图像处理装置或者上述预设优化器中可以存储上述预设阈值次数和预设学习率来控制更新。比如学习率0.001，预设阈值次数300epoch。以及可以设置学习率的调整规则，以该学习率的调整规则调整参数更新的学习率，比如可以设置分别在40、120和200epoch时学习率减半。
在获得上述训练后的预设神经网络模型之后，图像处理装置可以执行图1所示实施例中的部分或全部方法，即可以基于上述预设神经网络模型将待配准图像向参考图像配准，获得配准结果。
一般而言，大多数技术使用互信息的配准方法，需要估计联合分布密度。而非参数化方法估计互信息(比如使用直方图)，不仅计算量大并且不支持反向传播，无法应用到神经网络中。本申请实施例采用局部窗口的相关系数作为相似度度量损失，训练后的预设神经网络模型的可用于图像配准，尤其是任何会发生形变的脏器的医学图像配准中，可以对于不同时间点的随访图像进行形变配准，配准效率高、结果更加准确。
一般在某些手术中需要在术前或者手术期间进行不同质量和速度的各种扫描，获得医学图像，但通常需要做完各种扫描之后才可以进行医学图像配准，这是不满足手术中的实时需求的，所以一般需要通过额外的时间对手术的结果进行判定，如果配准后发现手术结果不够理想，可能需要进行后续的手术治疗，对于医生和病人来说都会带来时间上的浪费，耽误治疗。而基于本申请实施例的预设神经网络模型进行配准，可以应用于手术中实时的医学图像 配准，比如在做肿瘤切除手术中进行实时配准来判断肿瘤是否完全切除，提高了时效性。
本申请实施例通过获取预设待配准图像和预设参考图像，将上述预设待配准图像和上述预设参考图像输入上述预设神经网络模型生成形变场，基于上述形变场将上述预设待配准图像向上述预设参考图像配准，获得配准后图像，获得上述配准后图像和上述预设参考图像的相关系数损失，基于上述相关系数损失对上述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型，可以应用于可形变配准，提高图像配准的精度和实时性。
上述主要从方法侧执行过程的角度对本申请实施例的方案进行了介绍。可以理解的是，图像处理装置为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本公开能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对特定的应用使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
本申请实施例可以根据上述方法示例对图像处理装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
请参阅图3，图3是本申请实施例公开的一种图像处理装置的结构示意图。如图3所示，该图像处理装置300包括：获取模块310和配准模块320，其中：
上述获取模块310，用于获取待配准图像和用于配准的参考图像；
上述配准模块320，用于将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；
上述配准模块320，还用于基于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果。
可选的，上述图像处理装置300还包括：预处理模块330，用于获取原始待配准图像和原始参考图像，对上述原始待配准图像和上述原始参考图像进行图像归一化处理，获得满足目标参数的上述待配准图像和上述参考图像。
可选的，上述预处理模块330具体用于：
将上述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
将上述原始参考图像转换为上述预设灰度值范围内和上述预设图像尺寸的参考图像。
可选的，上述配准模块320包括配准单元321和更新单元322，其中：
上述配准单元321用于，获取上述预设待配准图像和上述预设参考图像，将上述预设待配准图像和上述预设参考图像输入上述预设神经网络模型生成形变场；
上述配准单元321还用于，基于上述形变场将上述预设待配准图像向上述预设参考图像配准，获得配准后图像；
上述更新单元322用于，获得上述配准后图像和上述预设参考图像的相关系数损失；以及用于基于上述相关系数损失对上述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
可选的，上述预处理模块330还用于：
对上述预设待配准图像和上述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
上述配准单元321具体用于，将上述满足预设训练参数的预设待配准图像和预设参考图像输入上述预设神经网络模型生成形变场。
可选的，上述预处理模块330具体用于：
将上述预设待配准图像的尺寸和上述预设参考图像的尺寸转换为预设图像尺寸；
根据目标窗宽对上述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
可选的，上述预处理模块330还具体用于：
在上述根据预设窗宽对上述转换后的预设待配准图像和预设参考图像进行处理之前，获取上述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定上述目标类别标签对应的上述目标窗宽。
可选的，上述更新单元322还用于：
基于预设优化器对上述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
图3所示的实施例中的图像处理装置300可以执行图1和/或图2所示实施例中的部分或全部方法。
实施图3所示的图像处理装置300，图像处理装置300可以获取待配准图像和用于配准的参考图像，将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失，基 于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果，可以提高图像配准的精度和实时性。
请参阅图4，图4是本申请实施例公开的一种电子设备的结构示意图。如图4所示，该电子设备400包括处理器401和存储器402，其中，电子设备400还可以包括总线403，处理器401和存储器402可以通过总线403相互连接，总线403可以是外设部件互连标准(Peripheral Component Interconnect，简称PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，简称EISA)总线等。总线403可以分为地址总线、数据总线、控制总线等。为便于表示，图4中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。其中，电子设备400还可以包括输入输出设备404，输入输出设备404可以包括显示屏，例如液晶显示屏。存储器402用于存储包含指令的一个或多个程序；处理器401用于调用存储在存储器402中的指令执行上述图1和图2实施例中提到的部分或全部方法步骤。上述处理器401可以对应实现图3中的电子设备300中的各模块的功能。
实施图4所示的电子设备400，电子设备400可以获取待配准图像和用于配准的参考图像，将上述待配准图像和上述参考图像输入预设神经网络模型，上述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失，基于上述预设神经网络模型将上述待配准图像向上述参考图像配准，获得配准结果，可以提高图像配准的精度和实时性。
本申请实施例还提供一种计算机可读存储介质，其中，该计算机可读存储介质存储用于电子数据交换的计算机程序，该计算机程序使得计算机执行如上述方法实施例中记载的任何一种图像处理方法的部分或全部步骤。
本申请实施例还提供一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行如上述方法实施例中记载的任何一种图像处理方法的部分或全部步骤。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本公开并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本公开所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置，可通过其它的方式实现。 例如，以上所描述的装置实施例仅仅是示意性的，例如所述模块(或单元)的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储器中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储器中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储器包括：U盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储器中，存储器可以包括：闪存盘、只读存储器、随机存取器、磁盘或光盘等。
以上对本申请实施例进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (19)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    获取待配准图像和用于配准的参考图像；
    将所述待配准图像和所述参考图像输入预设神经网络模型，所述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；
    基于所述预设神经网络模型将所述待配准图像向所述参考图像配准，获得配准结果。
  2. 根据权利要求1所述的图像处理方法，其特征在于，所述获取待配准图像和用于配准的参考图像之前，所述方法还包括：
    获取原始待配准图像和原始参考图像，对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的所述待配准图像和所述参考图像。
  3. 根据权利要求2所述的图像处理方法，其特征在于，所述对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的所述待配准图像和所述参考图像包括：
    将所述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
    将所述原始参考图像转换为所述预设灰度值范围内和所述预设图像尺寸的参考图像。
  4. 根据权利要求1-3任一项所述的图像处理方法，其特征在于，所述预设神经网络模型的训练过程包括：
    获取所述预设待配准图像和所述预设参考图像，将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场；
    基于所述形变场将所述预设待配准图像向所述预设参考图像配准，获得配准后图像；
    获得所述配准后图像和所述预设参考图像的相关系数损失；
    基于所述相关系数损失对所述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
  5. 根据权利要求4所述的图像处理方法，其特征在于，所述获取所述预设待配准图像和所述预设参考图像之后，所述方法还包括：
    对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
    所述将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场包括：
    将所述满足预设训练参数的预设待配准图像和预设参考图像输入所述预设神经网络模型生成形变场。
  6. 根据权利要求5所述的图像处理方法，其特征在于，所述方法还包括：
    将所述预设待配准图像的尺寸和所述预设参考图像的尺寸转换为预设图像尺寸；
    所述对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像包括：
    根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
  7. 根据权利要求6所述的图像处理方法，其特征在于，所述根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理之前，所述方法还包括：
    获取所述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定所述目标类别标签对应的所述目标窗宽。
  8. 根据权利要求5-7任一项所述的图像处理方法，其特征在于，所述方法还包括：
    基于预设优化器对所述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
  9. 一种图像处理装置，其特征在于，包括：获取模块和配准模块，其中：
    所述获取模块，用于获取待配准图像和用于配准的参考图像；
    所述配准模块，用于将所述待配准图像和所述参考图像输入预设神经网络模型，所述预设神经网络模型训练中衡量相似度的目标函数包括预设待配准图像和预设参考图像的相关系数损失；
    所述配准模块，还用于基于所述预设神经网络模型将所述待配准图像向所述参考图像配准，获得配准结果。
  10. 根据权利要求9所述的图像处理装置，其特征在于，还包括：预处理模块，用于获取原始待配准图像和原始参考图像，对所述原始待配准图像和所述原始参考图像进行图像归一化处理，获得满足目标参数的所述待配准图像和所述参考图像。
  11. 根据权利要求10所述的图像处理装置，其特征在于，所述预处理模块具体用于：
    将所述原始待配准图像转换为预设灰度值范围内和预设图像尺寸的待配准图像；
    将所述原始参考图像转换为所述预设灰度值范围内和所述预设图像尺寸的参考图像。
  12. 根据权利要求9-11任一项所述的图像处理装置，其特征在于，所述配准模块包括配准单元和更新单元，其中：
    所述配准单元用于，获取所述预设待配准图像和所述预设参考图像，将所述预设待配准图像和所述预设参考图像输入所述预设神经网络模型生成形变场；
    所述配准单元还用于，基于所述形变场将所述预设待配准图像向所述预设参考图像配准，获得配准后图像；
    所述更新单元用于，获得所述配准后图像和所述预设参考图像的相关系数损失；以及用于基于所述相关系数损失对所述预设神经网络模型进行参数更新，获得训练后的预设神经网络模型。
  13. 根据权利要求12所述的图像处理装置，其特征在于，所述预处理模块还用于：
    对所述预设待配准图像和所述预设参考图像进行图像归一化处理，获得满足预设训练参数的预设待配准图像和预设参考图像；
    所述配准单元具体用于，将所述满足预设训练参数的预设待配准图像和预设参考图像输入所述预设神经网络模型生成形变场。
  14. 根据权利要求13所述的图像处理装置，其特征在于，所述预处理模块具体用于：
    将所述预设待配准图像的尺寸和所述预设参考图像的尺寸转换为预设图像尺寸；
    根据目标窗宽对所述转换后的预设待配准图像和预设参考图像进行处理，获得处理后的预设待配准图像和预设参考图像。
  15. 根据权利要求14所述的图像处理装置，其特征在于，所述预处理模块还具体用于：
    在所述根据预设窗宽对所述转换后的预设待配准图像和预设参考图像进行处理之前，获取所述预设待配准图像的目标类别标签，根据预设类别标签与预设窗宽的对应关系，确定所述目标类别标签对应的所述目标窗宽。
  16. 根据权利要求13-15任一项所述的图像处理装置，其特征在于，所述更新单元还用于：
    基于预设优化器对所述预设神经网络模型进行预设学习率和预设阈值次数的参数更新。
  17. 一种电子设备，其特征在于，包括处理器以及存储器，所述存储器用于存储一个或多个程序，所述一个或多个程序被配置成由所述处理器执行，所述程序包括用于执行如权利要求1-8任一项所述的方法。
  18. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质用于存储电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如权利要求1-8任一项所述的方法。
  19. 一种计算机程序，包括计算机可读代码，其特征在于，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现如权利要求1-8中的任一项所述的方法。
PCT/CN2019/120329 2018-12-27 2019-11-22 图像处理方法、装置、电子设备及计算机可读存储介质 Ceased WO2020134769A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
SG11202102267XA SG11202102267XA (en) 2018-12-27 2019-11-22 Image processing method and apparatus, electronic device, and computer readable storage medium
KR1020217001183A KR20210021039A (ko) 2018-12-27 2019-11-22 이미지 처리 방법, 장치, 전자 기기 및 컴퓨터 판독 가능한 저장 매체
JP2021501292A JP2021530061A (ja) 2018-12-27 2019-11-22 画像処理方法及びその装置、電子機器並びにコンピュータ可読記憶媒体
US17/194,790 US20210192758A1 (en) 2018-12-27 2021-03-08 Image processing method and apparatus, electronic device, and computer readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811614468.4 2018-12-27
CN201811614468.4A CN109754414A (zh) 2018-12-27 2018-12-27 图像处理方法、装置、电子设备及计算机可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/194,790 Continuation US20210192758A1 (en) 2018-12-27 2021-03-08 Image processing method and apparatus, electronic device, and computer readable storage medium

Publications (1)

Publication Number Publication Date
WO2020134769A1 true WO2020134769A1 (zh) 2020-07-02

Family

ID=66404078

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/120329 Ceased WO2020134769A1 (zh) 2018-12-27 2019-11-22 图像处理方法、装置、电子设备及计算机可读存储介质

Country Status (7)

Country Link
US (1) US20210192758A1 (zh)
JP (1) JP2021530061A (zh)
KR (1) KR20210021039A (zh)
CN (2) CN109754414A (zh)
SG (1) SG11202102267XA (zh)
TW (1) TWI754195B (zh)
WO (1) WO2020134769A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070781A (zh) * 2020-08-13 2020-12-11 沈阳东软智能医疗科技研究院有限公司 颅脑断层扫描图像的处理方法、装置、存储介质及电子设备

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109754414A (zh) * 2018-12-27 2019-05-14 上海商汤智能科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质
US12573028B2 (en) * 2019-08-14 2026-03-10 Nvidia Corporation Neural network for image registration and image segmentation trained using a registration simulator
CN110782421B (zh) * 2019-09-19 2023-09-26 平安科技（深圳）有限公司 图像处理方法、装置、计算机设备及存储介质
CN110766729B (zh) * 2019-10-16 2023-05-16 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN111047629B (zh) * 2019-11-04 2022-04-26 中国科学院深圳先进技术研究院 多模态图像配准的方法、装置、电子设备及存储介质
CN111191675B (zh) * 2019-12-03 2023-10-24 深圳市华尊科技股份有限公司 行人属性识别模型实现方法及相关装置
CN110992411B (zh) * 2019-12-04 2023-05-02 图玛深维医疗科技(北京)有限公司 图像配准模型的训练方法和装置
US11348259B2 (en) * 2020-05-23 2022-05-31 Ping An Technology (Shenzhen) Co., Ltd. Device and method for alignment of multi-modal clinical images using joint synthesis, segmentation, and registration
CN111951311B (zh) * 2020-07-27 2024-05-28 上海联影智能医疗科技有限公司 图像配准方法、计算机设备和存储介质
CN114205642B (zh) * 2020-08-31 2024-04-26 北京金山云网络技术有限公司 一种视频图像的处理方法和装置
CN111932533B (zh) * 2020-09-22 2021-04-27 平安科技（深圳）有限公司 Ct图像椎骨定位方法、装置、设备及介质
CN116250012A (zh) * 2020-09-30 2023-06-09 斯纳普公司 用于图像动画的方法、系统和计算机可读存储介质
CN112307934B (zh) * 2020-10-27 2021-11-09 深圳市商汤科技有限公司 图像检测方法及相关模型的训练方法、装置、设备、介质
CN112348819A (zh) * 2020-10-30 2021-02-09 上海商汤智能科技有限公司 模型训练方法、图像处理及配准方法以及相关装置、设备
CN112560778B (zh) * 2020-12-25 2022-05-27 万里云医疗信息科技（北京）有限公司 Dr图像身体部位识别方法、装置、设备及可读存储介质
CN114820693B (zh) * 2021-01-19 2025-07-25 阿里巴巴集团控股有限公司 图像处理方法、装置、电子设备、介质及程序产品
US12190235B2 (en) * 2021-01-29 2025-01-07 Microsoft Technology Licensing, Llc System for training an artificial neural network
CN113570499B (zh) * 2021-07-21 2022-07-05 此刻启动(北京)智能科技有限公司 一种自适应图像调色方法、系统、存储介质及电子设备
US12033336B2 (en) * 2021-08-13 2024-07-09 Merative Us L.P. Deformable registration of medical images
CN113538539B (zh) * 2021-08-20 2023-09-22 浙江大学 基于布谷鸟搜索算法的肝脏ct图像配准方法及计算机可读存储介质
JP7566705B2 (ja) 2021-09-08 2024-10-15 株式会社東芝 学習方法、学習プログラム、および学習装置
CN113850852B (zh) * 2021-09-16 2024-10-18 北京航空航天大学 一种基于多尺度上下文的内窥镜图像配准方法及设备
CN114155376B (zh) * 2021-11-05 2026-05-08 苏州微创畅行机器人有限公司 目标特征点提取方法、装置、计算机设备和存储介质
CN114511599B (zh) * 2022-01-20 2022-09-20 推想医疗科技股份有限公司 模型训练方法及其装置、医学图像配准方法及其装置
KR102603177B1 (ko) * 2022-06-03 2023-11-17 주식회사 브라이토닉스이미징 영상 공간 정규화와 이를 이용한 정량화 시스템 및 그 방법
KR102681902B1 (ko) * 2022-06-29 2024-07-04 주식회사 에스아이에이 변화 탐지 방법
CN115393402B (zh) * 2022-08-24 2023-04-18 北京医智影科技有限公司 图像配准网络模型的训练方法、图像配准方法及设备
CN115690178B (zh) * 2022-10-21 2026-05-01 上海精劢医疗科技有限公司 基于深度学习的跨模态非刚体配准方法、系统及介质
KR20250102062A (ko) * 2022-11-01 2025-07-04 리제너론 파마슈티칼스 인코포레이티드 공간 전사체 슬라이드 정렬을 위한 방법, 장치 및 시스템
CN115908515B (zh) * 2022-11-11 2024-02-13 北京百度网讯科技有限公司 影像配准方法、影像配准模型的训练方法及装置
CN116342528B (zh) * 2023-03-21 2025-12-05 平安科技（深圳）有限公司 基于配准的图像比对训练方法、装置、计算机设备
CN117036471A (zh) * 2023-07-04 2023-11-10 北京宸普豪新科技有限公司 一种卫浴清洁方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103810699A (zh) * 2013-12-24 2014-05-21 西安电子科技大学 基于无监督深度神经网络的sar图像变化检测方法
CN108596961A (zh) * 2018-04-17 2018-09-28 浙江工业大学 基于三维卷积神经网络的点云配准方法
CN108776787A (zh) * 2018-06-04 2018-11-09 北京京东金融科技控股有限公司 图像处理方法及装置、电子设备、存储介质
CN108921100A (zh) * 2018-07-04 2018-11-30 武汉高德智感科技有限公司 一种基于可见光图像与红外图像融合的人脸识别方法及系统
CN109754414A (zh) * 2018-12-27 2019-05-14 上海商汤智能科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7967995B2 (en) * 2008-03-31 2011-06-28 Tokyo Electron Limited Multi-layer/multi-input/multi-output (MLMIMO) models and method for using
CN103714547B (zh) * 2013-12-30 2017-03-22 北京理工大学 一种结合边缘区域和互相关的图像配准方法
KR102294734B1 (ko) * 2014-09-30 2021-08-30 삼성전자주식회사 영상 정합 장치, 영상 정합 방법 및 영상 정합 장치가 마련된 초음파 진단 장치
US10235606B2 (en) * 2015-07-22 2019-03-19 Siemens Healthcare Gmbh Method and system for convolutional neural network regression based 2D/3D image registration
CN108960014B (zh) * 2017-05-23 2021-05-11 北京旷视科技有限公司 图像处理方法、装置和系统及存储介质
CN107578453B (zh) * 2017-10-18 2019-11-01 北京旷视科技有限公司 压缩图像处理方法、装置、电子设备及计算机可读介质
CN108345903B (zh) * 2018-01-25 2019-06-28 中南大学湘雅二医院 一种基于模态距离约束的多模态融合图像分类方法
CN108335322B (zh) * 2018-02-01 2021-02-12 深圳市商汤科技有限公司 深度估计方法和装置、电子设备、程序和介质
CN108416802B (zh) * 2018-03-05 2020-09-18 华中科技大学 一种基于深度学习的多模医学图像非刚性配准方法及系统
CN108629753A (zh) * 2018-05-22 2018-10-09 广州洪森科技有限公司 一种基于循环神经网络的人脸图像恢复方法及装置
CN108960300B (zh) * 2018-06-20 2021-03-02 北京工业大学 一种基于深度神经网络的城市土地利用信息分析方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103810699A (zh) * 2013-12-24 2014-05-21 西安电子科技大学 基于无监督深度神经网络的sar图像变化检测方法
CN108596961A (zh) * 2018-04-17 2018-09-28 浙江工业大学 基于三维卷积神经网络的点云配准方法
CN108776787A (zh) * 2018-06-04 2018-11-09 北京京东金融科技控股有限公司 图像处理方法及装置、电子设备、存储介质
CN108921100A (zh) * 2018-07-04 2018-11-30 武汉高德智感科技有限公司 一种基于可见光图像与红外图像融合的人脸识别方法及系统
CN109754414A (zh) * 2018-12-27 2019-05-14 上海商汤智能科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070781A (zh) * 2020-08-13 2020-12-11 沈阳东软智能医疗科技研究院有限公司 颅脑断层扫描图像的处理方法、装置、存储介质及电子设备
CN112070781B (zh) * 2020-08-13 2024-01-30 沈阳东软智能医疗科技研究院有限公司 颅脑断层扫描图像的处理方法、装置、存储介质及电子设备

Also Published As

Publication number Publication date
US20210192758A1 (en) 2021-06-24
CN111210467A (zh) 2020-05-29
JP2021530061A (ja) 2021-11-04
SG11202102267XA (en) 2021-04-29
KR20210021039A (ko) 2021-02-24
TWI754195B (zh) 2022-02-01
CN109754414A (zh) 2019-05-14
TW202025137A (zh) 2020-07-01

Similar Documents

Publication Publication Date Title
TWI754195B (zh) 圖像處理方法及其裝置、電子設備及電腦可讀儲存媒體
WO2020125221A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN110348515B (zh) 图像分类方法、图像分类模型训练方法及装置
CN104424629B (zh) 一种x光胸片肺部分割方法和装置
US12249030B2 (en) Generative nonlinear human shape models
US9561004B2 (en) Automated 3-D orthopedic assessments
WO2022213654A1 (zh) 一种超声图像的分割方法、装置、终端设备和存储介质
CN109767460A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN115115567B (zh) 一种图像处理方法、装置、计算机设备及介质
JP2022517769A (ja) 三次元ターゲット検出及びモデルの訓練方法、装置、機器、記憶媒体及びコンピュータプログラム
WO2021136368A1 (zh) 钼靶图像中胸大肌区域自动检测方法及装置
CN114332040B (zh) 一种基于多模态的甲状腺肿瘤图像分类方法及终端设备
CN110223289A (zh) 一种图像处理方法和系统
CN110310314A (zh) 图像配准方法、装置、计算机设备和存储介质
Nakao et al. Image-to-graph convolutional network for deformable shape reconstruction from a single projection image
CN107507189A (zh) 基于随机森林与统计模型的小鼠ct图像肾脏分割方法
CN112597847A (zh) 人脸姿态估计方法、装置、电子设备和存储介质
CN109087357B (zh) 扫描定位方法、装置、计算机设备及计算机可读存储介质
CN114184581A (zh) 基于oct系统的图像优化方法、装置、电子设备及存储介质
CN116012526B (zh) 一种基于二维影像的三维ct影像病灶重建方法
CN115439423B (zh) 一种基于ct图像的识别方法、装置、设备及存储介质
WO2023138273A1 (zh) 图像增强方法和系统
CN119445004B (zh) 基于人工智能技术的肝脏图像三维重建系统及方法
CN118657721A (zh) 图像处理方法、装置、电子设备及存储介质
CN116363332A (zh) 基于医学图像分割和特征匹配的前列腺超声矫正方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19904785

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20217001183

Country of ref document: KR

Kind code of ref document: A

Ref document number: 2021501292

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2101002165

Country of ref document: TH

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.10.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19904785

Country of ref document: EP

Kind code of ref document: A1