CN109767460A

CN109767460A - Image processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN109767460A
Application number: CN201811612706.8A
Authority: CN
Inventors: 宋涛
Original assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Priority date: 2018-12-27
Filing date: 2018-12-27
Publication date: 2019-05-17

Abstract

The embodiment of the present application discloses a kind of image processing method, device, electronic equipment and computer readable storage medium, and wherein method includes: to obtain image subject to registration and the reference picture for registration；The image subject to registration is inputted into affine transformation network, affine transformation is carried out to the image subject to registration by the affine transformation network, obtains the first image subject to registration；Described first image subject to registration and the reference picture are inputted into default neural network model, the described first image subject to registration is registrated to the reference picture by the default neural network model, registration result is obtained, registration operation step is can simplify, improves the precision and real-time of image registration.

Description

Image processing method, device, electronic equipment and computer readable storage medium

Technical field

The present invention relates to technical field of computer vision, and in particular to a kind of image processing method, device, electronic equipment and Computer readable storage medium.

Background technique

Image registration be by under different acquisition times, different sensors, different condition Same Scene or same mesh The process that two width of target or multiple image are registrated, is widely used in during Medical Image Processing.Medical image is matched Standard is an important technology in field of medical image processing, plays increasingly important role to clinical diagnosis and treatment.

Modern medicine usually requires the medical image that multiple mode or multiple time points obtain carrying out comprehensive analysis, that It just needs several sub-pictures carrying out registration work before being analyzed.It is traditional can deformable registration method be by constantly counting A corresponding relationship for calculating each pixel, the phase of image and reference picture after registration is calculated by similarity measurements flow function It is suitable as a result, this process usually requires several hours until reaching one like degree and a process of continuous iteration Even longer time completes, and the demand of patient's internal organs organ registration is larger in practical applications, and in many feelings As result of the operation consent to registration requires urgently under condition, it is seen that general method for registering lacks compared with the time of waste diagnostician Timeliness.

Summary of the invention

The embodiment of the present application provides a kind of image processing method, device and computer readable storage medium, can simplify Registration operation step improves the precision and real-time of image registration.

The embodiment of the present application first aspect provides a kind of image processing method, comprising:

Obtain image subject to registration and the reference picture for registration；

The image subject to registration is inputted into affine transformation network, by the affine transformation network to the image subject to registration Affine transformation is carried out, the first image subject to registration is obtained；

Described first image subject to registration and the reference picture are inputted into default neural network model, pass through the default mind The described first image subject to registration is registrated to the reference picture through network model, obtains registration result.

It is described that the image subject to registration is imitated by the affine transformation network in a kind of optional embodiment Transformation is penetrated, obtaining the first image subject to registration includes:

The processing parameter of the image subject to registration is obtained by the affine transformation network, is generated based on the processing parameter Transformation matrix；

Affine transformation is carried out to the image subject to registration using the transformation matrix, obtains first image subject to registration.

In a kind of optional embodiment, the processing parameter of the image subject to registration include rotation parameter, translation parameters, Zooming parameter and/or shear parameters.

In a kind of optional embodiment, the objective function of similarity is measured in the default neural network model training Related coefficient including presetting image subject to registration and preset reference image loses, or presets image subject to registration and institute including described State the mutual information loss of preset reference image.

In a kind of optional embodiment, it is described obtain image subject to registration and for the reference picture of registration before, institute State method further include:

Original image subject to registration and original reference image are obtained, to the original image subject to registration and the original reference figure As carrying out image normalization processing, the image and reference picture subject to registration for meeting target component are obtained.

It is described that the original image subject to registration and the original reference image are carried out in a kind of optional embodiment Image normalization processing, the image and reference picture subject to registration that acquisition meets target component include:

The original image subject to registration is converted in default intensity value ranges and the image subject to registration of preset image sizes； And

The original reference image is converted in the default intensity value ranges and the reference of the preset image sizes Image.

In a kind of optional embodiment, the objective function of similarity is measured in the default neural network model training When related coefficient including presetting image subject to registration and preset reference image loses, the default neural network model was trained Journey includes:

Obtain it is described preset image subject to registration and the preset reference image, preset described described in image subject to registration input Affine transformation network obtains the second image subject to registration；

Described second image subject to registration and the preset reference image input default neural network model are generated into shape Variable field；

The described second image subject to registration is schemed after being registrated to the preset reference image registration based on the Deformation Field Picture；

Obtain the related coefficient loss of the images after registration and the preset reference image；

Parameter update is carried out to the default neural network model based on related coefficient loss, it is pre- after being trained If neural network model.

In a kind of optional embodiment, it is described obtain it is described preset image subject to registration and the preset reference image it Afterwards, the method also includes:

Image subject to registration and preset reference image progress image normalization processing are preset to described, obtains to meet and preset Training parameter presets image subject to registration and preset reference image；

It is described to include: by second image subject to registration of the image input affine transformation network acquisition subject to registration of presetting

By it is described meet default training parameter preset image subject to registration input the affine transformation network obtain second to It is registrated image.

It is described to preset image subject to registration and preset reference image progress to described in a kind of optional embodiment Before image normalization processing, the method also includes:

It is preset image sizes by the size for presetting image subject to registration and the size conversion of the preset reference image；

It is described to preset image subject to registration and preset reference image progress image normalization processing to described, met Default training parameter preset image subject to registration and preset reference image includes:

According to target window width to after conversion preset image subject to registration and preset reference image is handled, after being handled Preset image subject to registration and preset reference image.

In a kind of optional embodiment, the objective function of similarity is measured in the default neural network model training When including the mutual information loss for presetting image subject to registration and the preset reference image, the default neural network model packet Registration model and Mutual Information Estimation network model are included, the training process of the default neural network model includes:

, to during the preset reference image registration, lead to based on the Deformation Field and second image subject to registration It crosses the Mutual Information Estimation network model to estimate the mutual information of images after registration and the preset reference image, obtain mutual Information loss；

Parameter update is carried out to the registration model and the Mutual Information Estimation network model based on mutual information loss, Default neural network model after being trained.

In a kind of optional embodiment, it is described by the Mutual Information Estimation network model to images after registration and institute The mutual information for stating preset reference image is estimated that obtaining mutual information loss includes:

By the Mutual Information Estimation network model, it is general that joint is obtained based on images after registration and the preset reference image Rate distribution and marginal probability distribution；

It is calculated according to the joint probability distribution parameter and the marginal probability distribution parameter and obtains the mutual information loss.

It is described to be lost based on the mutual information to the registration model and the mutual trust in a kind of optional embodiment Breath estimation network model carries out parameter update, and the default neural network model after being trained includes:

The parameter for carrying out first threshold number to the registration model based on mutual information loss updates, based on described mutual The parameter that information loss carries out second threshold number to the Mutual Information Estimation network model updates, pre- after obtaining the training If neural network model.

In a kind of optional embodiment, the training method of the affine transformation network includes:

With the objective function training affine transformation network that Mean square error loss function is the affine transformation network training.

In a kind of optional embodiment, it is described with Mean square error loss function be the affine transformation network training target The function training affine transformation network includes:

Obtain it is described preset image subject to registration based on the affine transformation network carry out affine transformation after third it is subject to registration Image；

Using the Mean square error loss function between third image subject to registration and the characteristic point of the preset reference image as The objective function of the affine transformation network training is trained the affine transformation network, described affine after being trained Converting network.

The embodiment of the present application second aspect provides a kind of image processing apparatus, comprising: obtain module, affine transformation module and Registration module, in which:

The acquisition module, for obtaining image subject to registration and for the reference picture of registration；

The affine transformation module passes through the affine change for the image subject to registration to be inputted affine transformation network Switching network carries out affine transformation to the image subject to registration, obtains the first image subject to registration；

The registration module, for the described first image subject to registration and the reference picture to be inputted default neural network mould Described first image subject to registration is registrated by type by the default neural network model to the reference picture, obtains registration knot Fruit.

In a kind of optional embodiment, the affine transformation module is specifically used for:

In a kind of optional embodiment, described image processing unit further include: preprocessing module is original for obtaining Image and original reference image subject to registration carry out image normalization to the original image subject to registration and the original reference image Processing obtains the image and reference picture subject to registration for meeting target component.

In a kind of optional embodiment, the preprocessing module is specifically used for:

In a kind of optional embodiment, the acquisition module is also used to, and acquisition is described to preset image subject to registration and institute State preset reference image；The affine transformation module is also used to, and presets the image input subject to registration affine transformation net for described Network obtains the second image subject to registration；

The registration module includes the first registration unit and the first updating unit, in which:

First registration unit is used for:

First updating unit is used for:

In a kind of optional embodiment, the preprocessing module is also used to:

The affine transformation module is specifically used for, and the image subject to registration of presetting for meeting default training parameter is inputted institute It states affine transformation network and obtains the second image subject to registration.

In a kind of optional embodiment, the preprocessing module also particularly useful for: preset image subject to registration for described Size and the preset reference image size conversion be preset image sizes；

In a kind of optional embodiment, the acquisition module is for obtaining the image subject to registration and described pre- preset If reference picture；The affine transformation module be used for by it is described preset image subject to registration and input the affine transformation network obtain the Two images subject to registration；

The default neural network model includes registration model and Mutual Information Estimation network model, and the registration module includes Second registration unit, Mutual Information Estimation unit and the second updating unit, in which:

Second registration unit is used for, and the described second image subject to registration and preset reference image input is described pre- If neural network model generates Deformation Field；

The Mutual Information Estimation unit is used for, and is being preset based on the Deformation Field and second image subject to registration to described During reference picture is registrated, by the Mutual Information Estimation network model to images after registration and the preset reference image Mutual information estimated, obtain mutual information loss；

Second updating unit is used for, and is lost based on the mutual information to the registration model and the Mutual Information Estimation Network model carries out parameter update, the default neural network model after being trained.

In a kind of optional embodiment, the Mutual Information Estimation unit is specifically used for:

In a kind of optional embodiment, second updating unit is specifically used for:

In a kind of optional embodiment, the affine transformation module further includes training unit, is used for:

In a kind of optional embodiment, the training unit is specifically used for:

The embodiment of the present application third aspect provides another image processing apparatus, including processor and memory, described For storing one or more programs, one or more of programs are configured to be executed by the processor memory, described Program includes for executing the step some or all of as described in the embodiment of the present application first aspect either method.

The embodiment of the present application fourth aspect provides a kind of computer readable storage medium, the computer readable storage medium For storing the computer program of electronic data interchange, wherein the computer program executes computer as the application is real Some or all of apply described in a first aspect either method step.

The embodiment of the present application is defeated by above-mentioned image subject to registration by obtaining image subject to registration and the reference picture for registration Enter affine transformation network, affine transformation is carried out to above-mentioned image subject to registration by above-mentioned affine transformation network, obtains first wait match Quasi- image, then the above-mentioned first image subject to registration and above-mentioned reference picture are inputted into default neural network model, by above-mentioned default Above-mentioned first image subject to registration is registrated by neural network model to above-mentioned reference picture, is obtained registration result, be can simplify registration Operating procedure improves the precision and real-time of image registration.

Detailed description of the invention

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described.

Fig. 1 is a kind of flow diagram of image processing method disclosed in the embodiment of the present application；

Fig. 2 is a kind of flow diagram of the training method of default neural network model disclosed in the embodiment of the present application；

Fig. 3 is a kind of structural schematic diagram of image processing apparatus disclosed in the embodiment of the present application；

Fig. 4 is the structural schematic diagram of a kind of electronic equipment disclosed in the embodiment of the present application.

Specific embodiment

In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present application Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.

Description and claims of this specification and term " first " in above-mentioned attached drawing, " second " etc. are for distinguishing Different objects, are not use to describe a particular order.In addition, term " includes " and " having " and their any deformations, it is intended that It is to cover and non-exclusive includes.Such as the process, method, system, product or equipment for containing a series of steps or units do not have It is defined in listed step or unit, but optionally further comprising the step of not listing or unit, or optionally also wrap Include other step or units intrinsic for these process, methods, product or equipment.

Referenced herein " embodiment " is it is meant that a particular feature, structure, or characteristic described can wrap in conjunction with the embodiments Containing at least one embodiment of the present invention.Each position in the description occur the phrase might not each mean it is identical Embodiment, nor the independent or alternative embodiment with other embodiments mutual exclusion.Those skilled in the art explicitly and Implicitly understand, embodiment described herein can be combined with other embodiments.

Image processing apparatus involved by the embodiment of the present application can permit other multiple terminal devices and access.On Stating image processing apparatus can be electronic equipment, including terminal device, in the specific implementation, above-mentioned terminal device includes but is not limited to Mobile phone, laptop computer or flat such as with touch sensitive surface (for example, touch-screen display and/or touch tablet) Other portable devices of plate computer etc.It is to be further understood that in certain embodiments, the equipment is simultaneously non-portable Communication equipment, but the desktop computer with touch sensitive surface (for example, touch-screen display and/or touch tablet).

The concept of deep learning in the embodiment of the present application is derived from the research of artificial neural network.Multilayer sense containing more hidden layers Know that device is exactly a kind of deep learning structure.Deep learning, which forms more abstract high level by combination low-level feature, indicates Attribute class Other or feature, to find that the distributed nature of data indicates.

Deep learning is a kind of based on the method for carrying out representative learning to data in machine learning.Observation (such as a width Image) various ways can be used to indicate, such as vector of each pixel intensity value, or be more abstractively expressed as a series of Side, region of specific shape etc..And use certain specific representation methods be easier from example learning tasks (for example, face Identification or human facial expression recognition).The benefit of deep learning is feature learning and the layered characteristic with non-supervisory formula or Semi-supervised It extracts highly effective algorithm and obtains feature by hand to substitute.Deep learning is a new field in machine learning research, motivation Be to establish, simulation human brain carries out the neural network of analytic learning, the mechanism that it imitates human brain explains data, such as image, Sound and text.

It describes in detail below to the embodiment of the present application.

Referring to Fig. 1, Fig. 1 is a kind of flow diagram of image processing method disclosed in the embodiment of the present application, such as Fig. 1 institute Show, which can be executed by above-mentioned image processing apparatus, be included the following steps:

101, image subject to registration and the reference picture for registration are obtained.

Image registration be by under different acquisition times, different sensors, different condition Same Scene or same mesh The process that two width of target or multiple image are registrated, is widely used in during Medical Image Processing.Medical image is matched Standard is an important technology in field of medical image processing, plays increasingly important role to clinical diagnosis and treatment.It is modern The medical image that medicine usually requires to say that multiple mode or multiple time points obtain carries out comprehensive analysis, then being analyzed It just needs several sub-pictures carrying out registration work before.

The image subject to registration (moving) mentioned in the embodiment of the present application and the reference picture (fixed) for registration Think the medical image obtained by various medical image equipments, it is particularly possible to be the image of deformable organ, such as lung CT is adopted in different time points or under different condition wherein image subject to registration and the reference picture for registration are generally same organs The image of collection.

Optionally, above-mentioned image subject to registration and reference picture be also possible to the exposure mask (mask) extracted by algorithm or Characteristic point.Wherein exposure mask can be understood as a kind of template of image filters, and image masks can be understood as with selected image, figure Shape or object block the image (all or part) of processing, to control region or the treatment process of image procossing.Number Mask is generally two-dimensional matrix array in image procossing, also uses multivalue image sometimes, can be used for structure feature extraction.

After extracting feature or mask, it is possible to reduce the interference in image procossing, so that registration result is more acurrate.

Since the medical image being registrated there may be diversity, it can be presented as image grayscale in the picture The diversity of the features such as value, picture size.Optionally, before step 101, available original image subject to registration and original ginseng Image is examined, image normalization processing is carried out to the original image subject to registration and the original reference image, acquisition meets target The image and reference picture subject to registration of parameter.

Image normalization in the embodiment of the present application refers to the processing transformation that series of standards is carried out to image, is allowed to convert For the process of a fixed standard form, which is referred to as normalized image.Image normalization can use the constant of image Square, which finds one group of parameter, can eliminate the influence that other transforming function transformation functions convert image, and original image to be processed is converted At corresponding sole criterion form, which has invariant feature to translation, rotation, scaling equiaffine transformation.Cause This, the image that can obtain unified style is handled by above-mentioned image normalization, improves the stability and accuracy of subsequent processing.

Above-mentioned target component can be understood as the parameter of description characteristics of image, i.e., for making above-mentioned raw image data in system The regulation parameter of one style.For example, above-mentioned target component may include: big for describing image resolution ratio, image grayscale, image The parameter of the features such as small.

Above-mentioned original image subject to registration can be the medical image obtained by various medical image equipments, it is particularly possible to be The image of deformable organ has diversity, can be presented as the more of the features such as gray value of image, picture size in the picture Sample.Some basic pretreatments can be done to original image subject to registration and original reference image before being registrated, it can also be with Only above-mentioned original image subject to registration is pre-processed.Above-mentioned image normalization processing can also be performed after pre-processing.Figure As pretreated main purpose is to eliminate unrelated information in image, restore useful real information, enhance for information about can Detection property and to the maximum extent simplified data, to improve the reliability of feature extraction, image segmentation, matching and identification.

Specifically, above-mentioned original image subject to registration can be converted in default intensity value ranges and preset image sizes Image subject to registration；And

Above-mentioned original reference image is converted in above-mentioned default intensity value ranges and the reference of above-mentioned preset image sizes Image.

Image processing apparatus in the embodiment of the present application can store above-mentioned default intensity value ranges and above-mentioned default figure As size.The operation of resampling (resample) can be done by simple ITK software to guarantee to need above-mentioned image subject to registration It is consistent substantially with the position of above-mentioned reference picture and resolution ratio.ITK is the cross platform system of an open source, is developer Provide a whole set of software tool for image analysis.

Above-mentioned preset image sizes can be length, width and height: 416x416x80, can be by shearing or filling (zero padding) Operation come guarantee above-mentioned image subject to registration it is consistent with the picture size of above-mentioned reference picture be 416x416x80.

By pre-processing to raw image data, its diversity can be reduced, neural network model can provide more Stable judgement.

102, above-mentioned image subject to registration is inputted into affine transformation network, by above-mentioned affine transformation network to above-mentioned subject to registration Image carries out affine transformation, obtains the first image subject to registration.

For two width medical images 1 and 2 registration obtained under different time or/and different condition, one is exactly found Mapping relations P makes each point on image 1 have unique point to correspond on image 2.And this two o'clock should correspond to Same anatomical position.Mapping relations P shows as one group of continuous spatial alternation.Common space geometry transformation has rigid body translation (Rigid body transformation), affine transformation (Affine transformation), projective transformation (Projective transformation) and nonlinear transformation (Nonlinear transformation).

Wherein, rigid body translation refers to that distance and parallel relation between interior of articles any two points remain unchanged.Affine transformation Be a kind of non-rigid transformation the simplest, a kind of its keeping parallelism, but not conformal, apart from changed transformation.And In many important clinical applications, just it is frequently necessary to using deformable method for registering images, and generally doing can deformable registration It needs first to do a simple rigid registration before or affine registration (affine registration includes above-mentioned rigid registration) will be subject to registration Image is simply aligned, then carry out again can deformable registration, with reduce can image subject to registration before deformable registration and reference In the case that otherness between image, i.e. such as two picture difference are too many, needing first substantially registration that could pass through can deformation change Bring acquisition registration result.Such as when studying the image registration of abdomen and chest internal organs, due to physiological movement or patient Movement cause position, size and the form of internal and tissue to change, it is necessary to can deformation transformation come compensate image become Shape, wherein change front and back organic image difference hour can directly carry out it is above-mentioned can deformable registration, and the Shi Zexian that differs greatly Carrying out that above-mentioned affine registration carries out again can deformable registration.

Step 102 then first preliminary before default neural network model is registrated can carry out affine transformation, complete affine registration, Step 103 can be executed, subsequent registration is more easily carried out, multiple step operations is avoided to influence efficiency.

Specifically, step 102 may include:

The processing parameter of above-mentioned image subject to registration is obtained by above-mentioned affine transformation network, is generated based on above-mentioned processing parameter Transformation matrix；

Affine transformation is carried out to above-mentioned image subject to registration using above-mentioned transformation matrix, obtains above-mentioned first image subject to registration.

Optionally, the processing parameter of above-mentioned image subject to registration may include rotation parameter, translation parameters, zooming parameter and cut Cut parameter.

Specifically, for example, 12 processing parameters can be set, wherein 3 rotation parameters, 3 translation parameters, 3 Zooming parameter and 3 shear parameters, such as 3 rotation parameters are referred to about x, y, the rotation angle of tri- axis of z, each rotation Angle is described by a matrix, is some sin cos functions, is restricted between [- 1,1].It is subject to registration after will be pretreated Image inputs in above-mentioned affine transformation network and returns above-mentioned 12 parameters, by giving this 12 parameters to constrain accordingly, can obtain It must be used for the 3D transformation matrix converted, carry out affine transformation for above-mentioned image subject to registration using above-mentioned transformation matrix, then it can be with Obtain affine registration as a result, i.e. above-mentioned first image subject to registration.

It optionally, can be the objective function of above-mentioned affine transformation network training with Mean square error loss function, training is above-mentioned imitative Penetrate converting network.

Mean square error (mean-square error, MSE) can be added in above-mentioned affine transformation network to constrain, The MSE loss mentioned in the embodiment of the present application is to reflect a kind of measurement of difference degree between estimator and the amount of being estimated, and is added MSE loses the stability that can promote network training to a certain degree.

Specifically, it is available it is above-mentioned preset image subject to registration based on above-mentioned affine transformation network carry out affine transformation after Third image subject to registration；

Using the Mean square error loss function between above-mentioned third image subject to registration and the characteristic point of above-mentioned preset reference image as The objective function of above-mentioned affine transformation network training is trained the affine transformation network, above-mentioned affine after being trained Converting network.

Wherein, image subject to registration is preset and preset reference image is also possible to the exposure mask extracted by algorithm due to above-mentioned (mask) or characteristic point, after extracting feature or mask, it is possible to reduce the interference in image procossing, so that registration result is more quasi- Really, correspondingly, above-mentioned MSE loss or affine transformation after moving image and fixed image characteristic point between MSE Loss.

The above-mentioned first image subject to registration can be obtained by above-mentioned affine transformation network processes, i.e., by the rigid body of conventional method Registration also includes, and integrated registration may be implemented, and avoids and first needs traditional method for registering to do rigid registration to do depth again The deformable registration work of degree study.Affine registration in the embodiment of the present application may include rigid registration, can specifically compare rigid body Registration increases the function of image cut and image scaling.

Affine transformation network in the embodiment of the present application can also be individually used at the medical image for only needing rigid registration In reason, such as the application of the image registrations such as brain or vertebra.

103, the above-mentioned first image subject to registration and above-mentioned reference picture are inputted into default neural network model, by above-mentioned pre- If the above-mentioned first image subject to registration is registrated by neural network model to above-mentioned reference picture, registration result is obtained.

Image registration is usually to carry out feature extraction to two images first to obtain characteristic point；Again by carrying out similarity measurements Amount finds matched characteristic point pair；Then by matched characteristic point to obtaining image space coordinate conversion parameter；Finally by sitting It marks transformation parameter and carries out image registration.

Above-mentioned default neural network model is can store in the embodiment of the present application, in image processing apparatus, the default mind Acquisition can be trained in advance through network model.

Above-mentioned default neural network model can be the mode based on neuron estimation mutual information and be trained acquisition, specifically It can be lost based on the mutual information for presetting image subject to registration and preset reference image and be trained acquisition.

Above-mentioned default neural network model may include registration model and Mutual Information Estimation network model, above-mentioned default nerve The training process of network model may include:

Obtain it is above-mentioned preset image subject to registration and above-mentioned preset reference image, by the above-mentioned image subject to registration and above-mentioned pre- preset If reference picture inputs above-mentioned registration model and generates Deformation Field；

Based on above-mentioned Deformation Field and it is above-mentioned preset image subject to registration to during above-mentioned preset reference image registration, lead to Above-mentioned Mutual Information Estimation network model is crossed to estimate the above-mentioned mutual information for presetting image subject to registration and above-mentioned preset reference image Meter obtains mutual information loss；

Parameter update is carried out to above-mentioned registration model and above-mentioned Mutual Information Estimation network model based on the loss of above-mentioned mutual information, Default neural network model after being trained.

The mutual information high-dimensional continuous random variable is carried out specifically, neural network gradient descent algorithm can be used Estimation.Such as MINE (mutualinformation neural estimaiton) algorithm, in dimension and it is on sample size It is linear measurable, back-propagation algorithm training can be used.MINE algorithm can be maximum or minimizes mutual information, is promoted and is generated The dual training of model breaks through the bottleneck of supervised learning classification task.

The convolutional layer of default neural network model in the embodiment of the present application can be 3D convolution, pass through above-mentioned default nerve Network model generates Deformation Field (deformable field), then will need the subject to registration of deformation by the space conversion layer of 3D Image carries out deformable transformation, the above-mentioned registration result after being registrated includes the registration result image generated (moved)。

Wherein, in above-mentioned default neural network model, in order to guarantee can Deformation Field flatness use L2 loss function Function constrains the gradient of Deformation Field.The objective function that similarity is measured in above-mentioned default neural network model training can be with Related coefficient including presetting image subject to registration and preset reference image loses, or including presetting image subject to registration and default ginseng Examine the mutual information loss of image.

The related coefficient mentioned in the embodiment of the present application is to be referred to earliest by the statistics that statistician's karr Pearson came designs Mark is the amount for studying linearly related degree between variable, is generally indicated with letter r.Due to the difference of research object, related coefficient There are many definition modes, and more the most commonly used is Pearson correlation coefficients.

General related coefficient is calculated by product moment method, equally based on the deviation of two variables and respective average value, is led to Two deviations are crossed to be multiplied to reflect degree of correlation between two variables；Linear simple correlation coefficient is studied emphatically.It should be noted that Pearson correlation coefficient is not unique related coefficient, but the most common related coefficient, the correlation in the embodiment of the present application Coefficient can be Pearson correlation coefficient.

Specifically, feature extraction images after registration and preset reference image can be passed through in default neural network model Characteristic pattern obtains above-mentioned related coefficient loss using the cross-correlation coefficient between characteristic pattern.

Existing method is registrated using there is supervision deep learning, substantially without goldstandard, it is necessary to be matched using tradition Quasi- method is marked, and the processing time is longer, and limits registration accuracy.And it does registration using conventional method to need to calculate The transformation relation of each pixel, calculation amount is huge, and elapsed time is also very big.

Solved the problems, such as according to the training sample of classification unknown (not being labeled) it is various in pattern-recognition, referred to as without prison Educational inspector practises.The embodiment of the present application carries out image registration using based on the neural network of unsupervised deep learning, can be used for any In the registration of the meeting internal organs that deformation occurs.The embodiment of the present application can use the GPU execution above method and is registrated in several seconds As a result, more efficiently.

Referring to Fig. 2, Fig. 2 is the flow diagram of another kind image processing method disclosed in the embodiment of the present application, specifically For a kind of flow diagram of the training method of default neural network, Fig. 2 is advanced optimized on the basis of Fig. 1. The main body for executing the embodiment of the present application step can be a kind of image processing apparatus, can be the method with embodiment illustrated in fig. 1 In same or different image processing apparatus.As shown in Fig. 2, the image processing method includes the following steps:

201, acquisition is above-mentioned presets image subject to registration and above-mentioned preset reference image, presets image input subject to registration for above-mentioned Above-mentioned affine transformation network obtains the second image subject to registration.

Wherein, similar with embodiment illustrated in fig. 1, it is above-mentioned to preset image subject to registration (moving) and above-mentioned preset reference Image (fixed), the medical image that all can be obtained by various medical image equipments, it is particularly possible to be deformable organ Image, such as lung CT, wherein image subject to registration and the reference picture for registration are generally same organs in different time The image acquired under point or different condition." default " word be for the image subject to registration that is different from embodiment illustrated in fig. 1 and Reference picture difference, here preset image subject to registration and preset reference image is mainly used for carrying out the default neural network model Training.

The above-mentioned processing parameter for presetting image subject to registration can be obtained by above-mentioned affine transformation network, be based on above-mentioned processing Parameter generates transformation matrix；

Image progress affine transformation subject to registration is preset to above-mentioned using above-mentioned transformation matrix, obtains above-mentioned second figure subject to registration Picture.

It wherein can be with reference to shown in Fig. 1 in fact by the process that above-mentioned affine transformation network obtains the second image image subject to registration Apply the specific descriptions in the step 101 of example.

202, the above-mentioned second image subject to registration and the above-mentioned default neural network model of above-mentioned preset reference image input is raw At Deformation Field.

Since the medical image being registrated there may be diversity, it can be presented as image grayscale in the picture The diversity of the features such as value, picture size.Optionally, above-mentioned acquisition is above-mentioned presets image subject to registration and above-mentioned preset reference image Later, the above method also may include:

Image subject to registration and the progress image normalization processing of above-mentioned preset reference image are preset to above-mentioned, obtains to meet and preset Training parameter presets image subject to registration and preset reference image；

Wherein, above-mentioned that the above-mentioned above-mentioned affine transformation network of image input subject to registration of presetting is obtained into the second image packet subject to registration It includes:

By above-mentioned satisfaction preset training parameter preset image subject to registration input above-mentioned affine transformation network obtain second to It is registrated image.

Above-mentioned default training parameter may include default intensity value ranges and preset image sizes (such as 416x 416x80). The process of above-mentioned image normalization processing can refer to the specific descriptions in the step 101 of embodiment illustrated in fig. 1.Optionally, first Rigid body translation and data normalization are first carried out before registration.Can specifically be done by simple ITK software the operation of resampling come Guarantee that the position for presetting image subject to registration and preset reference image and resolution ratio are consistent substantially.For subsequent training process Facilitate operation, the cutting or filling of predefined size can be carried out to image.Assuming that the image ruler of preset input picture The a height of 416x416x80 of modest ability width, it is necessary to guarantee to preset by shearing or filling the operation of (zero padding) image subject to registration and The picture size of preset reference image is unanimously 416x416x80.

It is above-mentioned to preset image subject to registration and the progress image normalization processing of above-mentioned preset reference image to above-mentioned, met Default training parameter preset image subject to registration and preset reference image may include:

For the important information in lung CT, target window width can be preset, for example be by target window width [- 1200, 600] to image subject to registration and preset reference image normalization is preset to [0,1], i.e., it is set as 1 for being greater than 600 in original image, 0 is set as less than -1200.

Generally acknowledged window width, window position can be set in different tissues on CT in the embodiment of the present application, is to preferably extract Important information.What the occurrence -1200,600 of [- 1200,600] here represented is window position, range size 1800, i.e. window It is wide.Above-mentioned image normalization processing is that subsequent costing bio disturbance does not cause gradient to explode for convenience.

The embodiment of the present application proposes a kind of normalization layer to guarantee trained stability and convergence.It assume that characteristic pattern Size is NxCxDxHxW, and wherein N refers to batchsize: the size of every batch of data volume.The embodiment of the present application can pass through meter The minimum value and maximum value of CxDxHxW is calculated, to do normalized operation to each image data.

203, above-mentioned Deformation Field is based on by the above-mentioned second image subject to registration to above-mentioned preset reference image registration, is registrated Image afterwards.

It, can be above-mentioned pre- by the above-mentioned second image subject to registration and the input of above-mentioned preset reference image after carrying out affine registration If neural network model generates Deformation Field (deformable field), then based on above-mentioned Deformation Field and above-mentioned presets figure subject to registration As to above-mentioned preset reference image registration, i.e., generating the registration result image after deformation using the Deformation Field and preset reference image (moved)。

Wherein it is possible to select L2 loss function, the characteristic of L2 loss function is smoother, here in order to cope with Deformation Field Changing greatly for gradient and cause to be mutated, generate the situation of fold and cavity, and gradient is the difference by adjacent pixels point It indicates, is to cause biggish deformation to guarantee that it is too big that neighbor pixel not change.

Above-mentioned images after registration is the second image subject to registration by the beginning of default neural network model to preset reference image Intermediate image after step registration, this process can be understood as being performed a plurality of times, it can repeat step 203 and step 204 With constantly training and optimize the default neural network model.

204, the related coefficient loss for obtaining above-mentioned images after registration and above-mentioned preset reference image, is based on above-mentioned phase relation Number loss carries out parameter update to above-mentioned default neural network model, the default neural network model after being trained.

Similarity assessment in the embodiment of the present application, by related coefficient loss as image and reference picture after registration Standard, it can repeat step 203 and step 204, constantly parameter is updated, to instruct to complete the training of network.

Optionally, default learning rate and default threshold can be carried out to above-mentioned default neural network model based on default optimizer The parameter for being worth number updates.

The preset threshold number being related to when above-mentioned update refers to the period (epoch) in neural metwork training.At one Phase can be understood as a positive transmitting and a back transfer for all training samples.

Algorithm used in optimizer generally have self-adaption gradient optimization algorithm (Adaptive Gradient, AdaGrad), it can adjust different learning rates to each different parameter, to the parameter frequently changed with smaller step-length It is updated, and sparse parameter is updated with bigger step-length；And RMSProp algorithm, in conjunction with the index of gradient square Moving average adjusts the variation of learning rate, can carry out in the objective function of unstable (Non-Stationary) It restrains well.

Specifically, above-mentioned default optimizer can use the optimizer of ADAM.

It can store above-mentioned preset threshold number and default learning rate in image processing apparatus or above-mentioned default optimizer It is updated to control.Such as learning rate 0.001, preset threshold number 300epoch.And the adjustment rule of learning rate can be set, With the learning rate that the adjustment rule adjustment parameter of the learning rate updates, for example can be set respectively in 40,120 and 200epoch Learning rate halves.

After obtaining the default neural network model after above-mentioned training, image processing apparatus can execute real shown in Fig. 1 Some or all of apply in example method, it can match image subject to registration to reference picture using above-mentioned default neural network model Standard obtains registration result.

In general, most of technologies use the method for registering of mutual information, need to estimate density of simultaneous distribution.And nonparametric Change method estimates mutual information (for example use histogram), not only computationally intensive and do not support backpropagation, can not be applied to mind Through in network.The embodiment of the present application is lost using the related coefficient of local window as measuring similarity, the default mind after training Can be used for image registration through network model, it is especially any can be in the medical figure registration of the internal organs that deformation occurs, can be right Deformable registration is carried out in the follow-up image of different time points, registration is high-efficient, result is more accurate.

Optionally, it includes above-mentioned default wait match for the objective function of similarity being measured in above-mentioned default neural network model training The mutual information of quasi- image and above-mentioned preset reference image lose when, above-mentioned default neural network model may include registration model and The training process of Mutual Information Estimation network model, above-mentioned default neural network model includes:

Obtain it is above-mentioned preset image subject to registration and above-mentioned preset reference image, to preset image subject to registration input above-mentioned by above-mentioned Affine transformation network obtains the second image subject to registration；

Above-mentioned second image subject to registration and the above-mentioned default neural network model of above-mentioned preset reference image input are generated into shape Variable field；

, to during above-mentioned preset reference image registration, lead to based on above-mentioned Deformation Field and above-mentioned second image subject to registration It crosses above-mentioned Mutual Information Estimation network model to estimate the mutual information of images after registration and above-mentioned preset reference image, obtain mutual Information loss；

Specifically, above-mentioned default neural network model may include Mutual Information Estimation network model and registration model.Registration Afterwards image be the second image subject to registration this by this with pseudo-crystalline lattice to the image after preset reference image registration.Specifically, It is general that joint can be obtained based on above-mentioned images after registration and above-mentioned preset reference image by above-mentioned Mutual Information Estimation network model Rate distribution and marginal probability distribution are calculated further according to above-mentioned joint probability distribution parameter and above-mentioned marginal probability distribution parameter and are obtained Mutual information loss.

Specifically, can be carried out based on neural network gradient descent algorithm between the mutual information high-dimensional continuous random variable Estimation.Such as MINE (mutualinformation neural estimaiton) algorithm, in dimension and it is on sample size It is linear measurable, back-propagation algorithm training can be used.MINE algorithm can be maximum or minimizes mutual information, is promoted and is generated The dual training of model, the bottleneck for breaking through supervised learning classification task can be above-mentioned mutually based on the calculating of following mutual information calculation formula Information loss:

Wherein, X, Z can be understood as two input pictures (images after registration and preset reference image), here X, and Z can be with It is interpreted as solution space, solution space refers to that the set of all solutions of system of homogeneous linear equations constitutes a vector space, that is, one The parameter of set, above-mentioned calculating mutual information loss belongs to the solution space of above-mentioned two input picture；P_XZFor joint probability distribution, P_X With P_ZFor marginal probability distribution；θ is the initiation parameter of above-mentioned Mutual Information Estimation network.

Wherein, mutual information is bigger in training, indicates that the result of registration is more accurate.Above-mentioned T can be understood as above-mentioned mutual information Estimate network model (including its parameter), mutual information can be estimated in conjunction with this formula, so T here is also there are parameter needs It updates.This formula and T collectively constitute mutual information loss.

Similarity assessment standard of the mutual information as image and reference picture after registration is estimated by neuron, it can not It is disconnected that the parameter of above-mentioned registration model and Mutual Information Estimation network model is updated, to instruct to complete the training of two networks.

Optionally, it can be lost based on above-mentioned mutual information and the parameter of first threshold number is carried out more to above-mentioned registration model Newly, the parameter for carrying out second threshold number to above-mentioned Mutual Information Estimation network model is lost based on above-mentioned mutual information to update, obtain Default neural network model after above-mentioned training.

Above-mentioned first threshold number and second threshold number are can store in image processing apparatus, wherein above-mentioned first Threshold number and second threshold number can be different, and above-mentioned first threshold number can be greater than above-mentioned second threshold number.

The first threshold number and second threshold number being related to when above-mentioned update, refer to the period in neural metwork training (epoch).One period can be understood as a positive transmitting and a back transfer for all training samples.

Specifically, above-mentioned registration model and Mutual Information Estimation network model can carry out independent parameter update, citing comes Say, first threshold number is 120, and second threshold number is 50, it can in preceding 50 epoch Mutual Information Estimation network models and Registration model updates together, and the network parameter of Mutual Information Estimation network model is freezed after 50 epoch, only updates and matches quasi-mode Type is completed until 120 epoch of registration model update.

Optionally, it is also based on default optimizer and default learning rate and third is carried out to above-mentioned default neural network model The parameter of threshold number updates, to obtain the default neural network model after last training.

Wherein, above-mentioned default optimizer can use the optimizer of ADAM.

Above-mentioned third threshold number refers to epoch as pre-determined first threshold number and second threshold number.Image Above-mentioned third threshold number and default learning rate be can store in processing unit or above-mentioned default optimizer to control and update.Than Such as learning rate 0.001, third threshold number 300epoch.And the adjustment rule of learning rate can be set, with the learning rate The learning rate that rule adjustment parameter updates is adjusted, for example can be set respectively that learning rate halves in 40,120 and 200epoch.

In general, most of technologies are not only counted using imparametrization method estimation mutual information (for example using histogram) Calculation amount is big and does not support backpropagation, can not be applied in neural network.The embodiment of the present application estimates mutual trust using neuron It ceases to measure the loss of the similitude of image, the default neural network model after training can be used for image registration, especially any Can in the medical figure registration of the internal organs that deformation occurs, can the follow-up image for different time points carry out deformable registration, match Standard is high-efficient, result is more accurate.

The various scannings for generally needing to carry out different quality and speed in certain operations in the preoperative or during operation, are obtained Medical image is obtained, but it is generally necessary to which medical figure registration can just be carried out later by finishing various scannings, this is unsatisfactory in operation Real-time requirement, determined by result of the additional time to operation so generally requiring, if finding hand after registration Art result is not ideal enough, it may be necessary to carry out subsequent operative treatment, can all bring for doctor and patient temporal Waste, delays treatment.And the default neural network model based on the embodiment of the present application is registrated, and can be applied to real in operation When medical figure registration, such as in doing tumor resection carry out in real time registration to judge whether tumour cuts off completely, mention High timeliness.

The embodiment of the present application by obtain it is above-mentioned preset image subject to registration and above-mentioned preset reference image, will it is above-mentioned preset to It is registrated image and inputs above-mentioned affine transformation network and obtain the second image subject to registration, then by the above-mentioned second image subject to registration and above-mentioned pre- If the above-mentioned default neural network model of reference picture input generates Deformation Field, above-mentioned Deformation Field is based on by the above-mentioned second figure subject to registration As obtaining images after registration, obtaining above-mentioned images after registration and above-mentioned preset reference image to above-mentioned preset reference image registration Related coefficient loss, and parameter update is carried out to above-mentioned default neural network model based on the loss of above-mentioned related coefficient, obtained Default neural network model after training, can be applied to can deformable registration, simplify registration operation step, improve image registration Precision and real-time.

It is above-mentioned that mainly the scheme of the embodiment of the present application is described from the angle of method side implementation procedure.It is understood that , in order to realize the above functions, it comprises execute the corresponding hardware configuration of each function and/or software for image processing apparatus Module.Those skilled in the art should be readily appreciated that, list described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps, the present invention can be realized with the combining form of hardware or hardware and computer software.Some function is actually It is executed in a manner of hardware or computer software driving hardware, the specific application and design constraint item depending on technical solution Part.Professional technician can be to specifically realizing described function using distinct methods, but this realization is not It is considered as beyond the scope of this invention.

The embodiment of the present application can carry out the division of functional module, example according to above method example to image processing apparatus Such as, each functional module of each function division can be corresponded to, two or more functions can also be integrated at one It manages in module.Above-mentioned integrated module both can take the form of hardware realization, can also use the form of software function module It realizes.It should be noted that being schematical, only a kind of logic function stroke to the division of module in the embodiment of the present application Point, there may be another division manner in actual implementation.

Referring to Fig. 3, Fig. 3 is a kind of structural schematic diagram of image processing apparatus disclosed in the embodiment of the present application.Such as Fig. 3 institute Show, which includes: to obtain module 310, affine transformation module 320 and registration module 330, in which:

Above-mentioned acquisition module 310, for obtaining image subject to registration and for the reference picture of registration；

Above-mentioned affine transformation module 320, for above-mentioned image subject to registration to be inputted affine transformation network, by above-mentioned affine Converting network carries out affine transformation to above-mentioned image subject to registration, obtains the first image subject to registration；

Above-mentioned registration module 330, for the above-mentioned first image subject to registration and above-mentioned reference picture to be inputted default nerve net Above-mentioned first image subject to registration is registrated to above-mentioned reference picture by above-mentioned default neural network model, is matched by network model Quasi- result.

Optionally, above-mentioned affine transformation module 320 is specifically used for:

Optionally, the processing parameter of above-mentioned image subject to registration includes rotation parameter, translation parameters, zooming parameter and/or cuts Cut parameter.

Optionally, measuring the objective function of similarity in above-mentioned default neural network model training includes presetting figure subject to registration The related coefficient of picture and preset reference image loss, or including above-mentioned image subject to registration and the above-mentioned preset reference image preset Mutual information loss.

Optionally, above-mentioned image processing apparatus 300 further include: preprocessing module 340, for obtaining original figure subject to registration Picture and original reference image carry out image normalization processing to above-mentioned original image subject to registration and above-mentioned original reference image, obtain The image and reference picture subject to registration of target component must be met.

Optionally, above-mentioned preprocessing module 340 is specifically used for:

Above-mentioned original image subject to registration is converted in default intensity value ranges and the image subject to registration of preset image sizes； And

Optionally, above-mentioned acquisition module 310 is also used to, and acquisition is above-mentioned to preset image subject to registration and above-mentioned preset reference figure Picture；Above-mentioned affine transformation module is also used to, by it is above-mentioned preset image subject to registration input above-mentioned affine transformation network obtain second to It is registrated image；

Above-mentioned registration module 330 includes the first registration unit 331 and the first updating unit 332, in which:

Above-mentioned first registration unit 331 is used for:

The above-mentioned second image subject to registration is schemed after being registrated to above-mentioned preset reference image registration based on above-mentioned Deformation Field Picture；

Above-mentioned first updating unit 332 is used for:

Obtain the related coefficient loss of above-mentioned images after registration and above-mentioned preset reference image；

Parameter update is carried out to above-mentioned default neural network model based on the loss of above-mentioned related coefficient, it is pre- after being trained If neural network model.

Optionally, above-mentioned preprocessing module 340 is also used to:

Above-mentioned affine transformation module is specifically used for, and the image subject to registration of presetting that above-mentioned satisfaction presets training parameter is inputted It states affine transformation network and obtains the second image subject to registration.

Optionally, above-mentioned preprocessing module 340 also particularly useful for: by the above-mentioned size for presetting image subject to registration and above-mentioned pre- If the size conversion of reference picture is preset image sizes；

Optionally, above-mentioned acquisition module 310 above-mentioned presets image subject to registration and above-mentioned preset reference image for obtaining；On Affine transformation module 320 is stated for the above-mentioned above-mentioned affine transformation network of image input subject to registration of presetting to be obtained the second figure subject to registration Picture；

Above-mentioned default neural network model includes registration model and Mutual Information Estimation network model, above-mentioned registration module 330 Including the second registration unit 333, Mutual Information Estimation unit 334 and the second updating unit 335, in which:

Above-mentioned second registration unit 333 is used for, will be on the above-mentioned second image subject to registration and the input of above-mentioned preset reference image It states default neural network model and generates Deformation Field；

Above-mentioned Mutual Information Estimation unit 334 is used for, based on above-mentioned Deformation Field and above-mentioned second image subject to registration to above-mentioned During preset reference image registration, by above-mentioned Mutual Information Estimation network model to images after registration and above-mentioned preset reference The mutual information of image is estimated, mutual information loss is obtained；

Above-mentioned second updating unit 335 is used for, and is lost based on above-mentioned mutual information to above-mentioned registration model and above-mentioned mutual information Estimate that network model carries out parameter update, the default neural network model after being trained.

Optionally, above-mentioned Mutual Information Estimation unit 334 is specifically used for:

By above-mentioned Mutual Information Estimation network model, it is general that joint is obtained based on images after registration and above-mentioned preset reference image Rate distribution and marginal probability distribution；

It is calculated according to above-mentioned joint probability distribution parameter and above-mentioned marginal probability distribution parameter and obtains above-mentioned mutual information loss.

Optionally, above-mentioned second updating unit 335 is specifically used for:

The parameter for carrying out first threshold number to above-mentioned registration model based on the loss of above-mentioned mutual information updates, based on above-mentioned mutual The parameter that information loss carries out second threshold number to above-mentioned Mutual Information Estimation network model updates, pre- after obtaining above-mentioned training If neural network model.

Optionally, above-mentioned affine transformation module 320 further includes training unit 321, is used for:

With the above-mentioned affine transformation network of objective function training that Mean square error loss function is above-mentioned affine transformation network training.

Optionally, above-mentioned training unit 321, is specifically used for:

Obtain it is above-mentioned preset image subject to registration based on above-mentioned affine transformation network carry out affine transformation after third it is subject to registration Image；

Using the Mean square error loss function between above-mentioned third image subject to registration and the characteristic point of above-mentioned preset reference image as The objective function of above-mentioned affine transformation network training is trained above-mentioned affine transformation network, above-mentioned affine after being trained Converting network.

Image processing apparatus 300 in embodiment shown in Fig. 3 can execute the portion in Fig. 1 and/or embodiment illustrated in fig. 2 Point or all methods.

Image processing apparatus 300 shown in implementing Fig. 3, the available image subject to registration of image processing apparatus 300 and are used for Above-mentioned image subject to registration is inputted affine transformation network by the reference picture of registration, by above-mentioned affine transformation network to it is above-mentioned to It is registrated image and carries out affine transformation, obtain the first image subject to registration, then by the above-mentioned first image subject to registration and above-mentioned reference picture Input default neural network model, by above-mentioned default neural network model by the above-mentioned first image subject to registration to above-mentioned with reference to figure As registration, registration result is obtained, registration operation step is can simplify, improves the precision and real-time of image registration.

Referring to Fig. 4, Fig. 4 is the structural schematic diagram of a kind of electronic equipment disclosed in the embodiment of the present application.As shown in figure 4, The electronic equipment 400 includes processor 401 and memory 402, wherein electronic equipment 400 can also include bus 403, processing Device 401 and memory 402 can be connected with each other by bus 403, and bus 403 can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, abbreviation PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, abbreviation EISA) bus etc..It is total that bus 403 can be divided into address bus, data Line, control bus etc..Only to be indicated with a thick line in Fig. 4, it is not intended that an only bus or a type convenient for indicating The bus of type.Wherein, electronic equipment 400 can also include input-output equipment 404, and input-output equipment 404 may include showing Display screen, such as liquid crystal display.Memory 402 is used to store one or more programs comprising instruction；Processor 401 is for adjusting With some or all of mentioning method and step in the above-mentioned Fig. 1 and Fig. 2 embodiment of the instruction execution being stored in memory 402.On The function of realizing each module in the electronic equipment 300 in Fig. 3 can be corresponded to by stating processor 401.

Implement electronic equipment 400 shown in Fig. 4, the available image subject to registration of electronic equipment 400 and the ginseng for registration Image is examined, above-mentioned image subject to registration is inputted into affine transformation network, by above-mentioned affine transformation network to above-mentioned image subject to registration Affine transformation is carried out, obtains the first image subject to registration, then the above-mentioned first image subject to registration and above-mentioned reference picture are inputted and preset Above-mentioned first image subject to registration is registrated by neural network model by above-mentioned default neural network model to above-mentioned reference picture, Registration result is obtained, registration operation step is can simplify, improves the precision and real-time of image registration.

The embodiment of the present application also provides a kind of computer readable storage medium, wherein the computer readable storage medium is deposited Storage is used for the computer program of electronic data interchange, which execute computer as remembered in above method embodiment Some or all of any image processing method of load step.

It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because According to the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules is not necessarily of the invention It is necessary.

In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, reference can be made to the related descriptions of other embodiments.

In several embodiments provided herein, it should be understood that disclosed device, it can be by another way It realizes.For example, the apparatus embodiments described above are merely exemplary, such as the division of the module (or unit), only Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple module or components can be tied Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or module Letter connection can be electrical or other forms.

The module as illustrated by the separation member may or may not be physically separated, aobvious as module The component shown may or may not be physical module, it can and it is in one place, or may be distributed over multiple On network module.Some or all of the modules therein can be selected to realize the mesh of this embodiment scheme according to the actual needs 's.

It, can also be in addition, each functional module in each embodiment of the present invention can integrate in a processing module It is that modules physically exist alone, can also be integrated in two or more modules in a module.Above-mentioned integrated mould Block both can take the form of hardware realization, can also be realized in the form of software function module.

If the integrated module is realized in the form of software function module and sells or use as independent product When, it can store in a computer-readable access to memory.Based on this understanding, technical solution of the present invention substantially or Person says that all or part of the part that contributes to existing technology or the technical solution can body in the form of software products Reveal and, which is stored in a memory, including some instructions are used so that a computer equipment (can be personal computer, server or network equipment etc.) executes all or part of each embodiment the method for the present invention Step.And memory above-mentioned includes: USB flash disk, read-only memory (Read-Only Memory, ROM), random access memory The various media that can store program code such as (Random Access Memory, RAM), mobile hard disk, magnetic or disk.

Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can It is completed with instructing relevant hardware by program, which can store in a computer-readable memory, memory It may include: flash disk, read-only memory, random access device, disk or CD etc..

The embodiment of the present application is described in detail above, specific case used herein to the principle of the present invention and Embodiment is expounded, and the above description of the embodiment is only used to help understand the method for the present invention and its core ideas； At the same time, for those skilled in the art can in specific embodiments and applications according to the thought of the present invention There is change place, in conclusion the contents of this specification are not to be construed as limiting the invention.

Claims

1. a kind of image processing method, which is characterized in that the described method includes:

The image subject to registration is inputted into affine transformation network, the image subject to registration is carried out by the affine transformation network Affine transformation obtains the first image subject to registration；

Described first image subject to registration and the reference picture are inputted into default neural network model, pass through the default nerve net Described first image subject to registration is registrated by network model to the reference picture, obtains registration result.

2. image processing method according to claim 1, which is characterized in that it is described by the affine transformation network to institute It states image subject to registration and carries out affine transformation, obtaining the first image subject to registration includes:

The processing parameter of the image subject to registration is obtained by the affine transformation network, and transformation is generated based on the processing parameter Matrix；

3. image processing method according to claim 2, which is characterized in that the processing parameter of the image subject to registration includes Rotation parameter, translation parameters, zooming parameter and/or shear parameters.

4. image processing method according to claim 3, which is characterized in that weigh in the default neural network model training The objective function of amount similarity includes the related coefficient loss for presetting image subject to registration and preset reference image, or including described Preset the mutual information loss of image subject to registration and the preset reference image.

5. image processing method according to claim 1-4, which is characterized in that the default neural network model The objective function that similarity is measured in training includes the mutual information damage for presetting image subject to registration and the preset reference image When mistake, the default neural network model includes registration model and Mutual Information Estimation network model, the default neural network mould The training process of type includes:

Obtain it is described preset image subject to registration and the preset reference image, to preset image subject to registration input described affine by described Converting network obtains the second image subject to registration；

Described second image subject to registration and the preset reference image input default neural network model are generated into Deformation Field；

, to during the preset reference image registration, passing through institute based on the Deformation Field and second image subject to registration It states Mutual Information Estimation network model to estimate the mutual information of images after registration and the preset reference image, obtains mutual information Loss；

Parameter update is carried out to the registration model and the Mutual Information Estimation network model based on mutual information loss, is obtained Default neural network model after training.

6. image processing method according to claim 5, which is characterized in that described to pass through the Mutual Information Estimation network mould Type estimates that the mutual information of images after registration and the preset reference image, obtaining mutual information loss includes:

By the Mutual Information Estimation network model, joint probability point is obtained based on images after registration and the preset reference image Cloth and marginal probability distribution；

7. image processing method according to claim 6, which is characterized in that the training method packet of the affine transformation network It includes:

8. a kind of image processing apparatus characterized by comprising obtain module, affine transformation module and registration module, in which:

The affine transformation module passes through the affine transformation net for the image subject to registration to be inputted affine transformation network Network carries out affine transformation to the image subject to registration, obtains the first image subject to registration；

The registration module, for the described first image subject to registration and the reference picture to be inputted default neural network model, The described first image subject to registration is registrated to the reference picture by the default neural network model, obtains registration result.

9. a kind of electronic equipment, which is characterized in that including processor and memory, the memory is for storing one or more A program, one or more of programs are configured to be executed by the processor, and described program includes for executing such as right It is required that the described in any item methods of 1-7.

10. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium is for storing electron number According to the computer program of exchange, wherein the computer program executes computer as claim 1-7 is described in any item Method.