CN110211193A - Three dimensional CT interlayer image interpolation reparation and super-resolution processing method and device - Google Patents
Three dimensional CT interlayer image interpolation reparation and super-resolution processing method and device Download PDFInfo
- Publication number
- CN110211193A CN110211193A CN201910415030.1A CN201910415030A CN110211193A CN 110211193 A CN110211193 A CN 110211193A CN 201910415030 A CN201910415030 A CN 201910415030A CN 110211193 A CN110211193 A CN 110211193A
- Authority
- CN
- China
- Prior art keywords
- image
- interpolation
- interlayer
- dimensional
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
Abstract
The present invention provides the reparation of three dimensional CT interlayer image interpolation and super-resolution processing method and device, interlayer interpolation is carried out to CT image using based on light stream estimation registration information, since the sports ground of light stream estimation is different, multiple pixels may be mapped to the same position in interpolation image in original image, so as to cause in interpolation image, there are the regions that no original image pixel-map arrives, such that there are said minuscule holes for interpolation image.So the present invention further repairs the interlayer slice that interpolation generates, interpolation image quality is improved.On the basis of light stream estimation progress interlayer interpolation, image repair is carried out using the non local self-similarity of image to the middle slice that interpolation generates, CT number of sections can be increased, improve CT image interlayer resolution ratio, to promote the quality of MFSR CT image reconstruction, patient is helped to obtain accurate diagnosing and treating in the case where not receiving unnecessary dose of radiation.
Description
Technical field
The present invention relates at CT technical field of image processing more particularly to three dimensional CT interlayer image interpolation reparation and super-resolution
Manage method and device.
Background technique
CT scan (computed tomography, CT) image is detector and X-ray beam, ultrasonic wave
Make what continuous profile scanning obtained Deng a certain position together around human body.CT as a kind of common medical imaging procedure,
Because it can show the details at a certain position in human body, controlled in recent years in the diagnosis of central nervous system disease, thoracopathy etc.
Play extremely important effect in treatment, but the resolution ratio of CT image and the number of x-ray dose have and closely contacts.Such as
Fruit reduces x-ray dose, serious artifact just probably occurs, lesions position is also difficult clearly to show on CT image
Come, reduces the reliability of diagnosis；If increasing x-ray dose, the damage of immune system may cause, have and induce the latent of cancer
In risk.
Wherein CT image super-resolution rebuilding method is quickly grown, and method in recent years is broadly divided into following two: 1) single
A Image Super-resolution (single image super resolution, SISR).It only with reference to current low-resolution image, and
Other associated pictures are not depended on.Such as Ledig et al. carries out single image super-resolution using the sense of reality for generating antagonism network.
The depth residual error network of enhancing is used for single image super-resolution by Lim et al..Medicine is realized using deconvolution deep neural network
Image super-resolution rebuilding, and single image is operated.2) multiple images super-resolution (multi-frame super
resolution,MFSR).It is exactly to replace single image with the adjacent image sequence of Same Scene, and refer to the mutual of these images
Information is mended, the image co-registration of this series of low resolution is generated into a high-resolution image.
In general, MFSR has compared to SISR more fully can refer to information, can obtain higher-quality high score
Resolution reconstruction image.But it to reduce X-ray to the radiation of patient and the limitation of existing apparatus, is carried out along upper and lower direction intensive
It samples often not practical, is typically only capable to obtain CT sections limited, causes the interlamellar spacing of CT image sequence larger.CT image along it is upper,
Lower direction lacks enough structural informations and may make adjacent two layers CT image there are the larger differences in partial structurtes, therefore
During MFSR is rebuild, these differences may interfere the high-definition picture of reconstruction, and preneoplasia is caused to diagnose
It is not accurate enough, so that the successive treatment to patient impacts.
Summary of the invention
The thought of interleave in video processing procedure is applied to the research of medical image super-resolution by the present invention.Original adjacent
It is inserted into a new slice in two CT slices, solves the problems, such as that adjacent two layers CT image has larger difference in partial structurtes.
The method of the present invention includes:
Step 1 estimates the corresponding relationship of pixel between model solution CT image contiguous slices by CLG-TV light stream；
Step 2 finds target position by the mapping function of corresponding relationship between pixel, and is inserted with image warpage mode
It is worth middle slice out；
Step 3, using the non local self-similarity of the intrinsic interframe of 4D-CT image, to the Small Holes generated in Interpolation Process
Hole and fuzzy progress image repair.
The present invention also provides a kind of device for realizing the three dimensional CT reparation of interlayer image interpolation and super-resolution processing method, packets
It includes:
Memory, for storing computer program and three dimensional CT interlayer image interpolation reparation and super-resolution processing method；
Processor, for executing the computer program and three dimensional CT interlayer image interpolation reparation and superresolution processing side
Method, the step of to realize three dimensional CT interlayer image interpolation reparation and super-resolution processing method.
As can be seen from the above technical solutions, the invention has the following advantages that
The present invention carries out interlayer interpolation to CT image using based on light stream estimation registration information, but in Interpolation Process,
Since the sports ground of light stream estimation is different, multiple pixels may be mapped to the same position in interpolation image in original image, thus
Cause in interpolation image there are the region that no original image pixel-map arrives, that is, pixel lacks, such that interpolation
There are said minuscule holes for image.So the present invention further repairs the interlayer slice that interpolation generates, interpolation graphs image quality is improved
Amount.
The present invention utilizes image on the basis of carrying out interlayer interpolation based on light stream estimation, to the middle slice that interpolation generates
Non local self-similarity carry out image repair, can increase CT number of sections, CT image interlayer resolution ratio be improved, to be promoted
The quality of MFSR CT image reconstruction, help patient obtained in the case where not receiving unnecessary dose of radiation accurately diagnose and
Treatment.
Detailed description of the invention
In order to illustrate more clearly of technical solution of the present invention, attached drawing needed in description will be made below simple
Ground introduction, it should be apparent that, drawings in the following description are only some embodiments of the invention, for ordinary skill
For personnel, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is sagittal plane interlayer interpolation schematic diagram；
Fig. 2 is cross section interlayer interpolation schematic diagram；
Fig. 3 is block flow diagram of the present invention；
Fig. 4 is the estimation schematic diagram based on optical flow field
Fig. 5 is to carry out light stream to three width coronal images using CLG-TV light stream algorithm for estimating to estimate schematic diagram；
Fig. 6, which generates new middle slice for the interpolation in two continuous CT images, may generate hole schematic diagram；
Fig. 7 is block-based non local self-similarity schematic diagram；
Fig. 8 is image repair effect picture of the present invention, and (a) is original image and partial enlarged view before repairing, (b) is former after repairing
Image and partial enlarged view；
Fig. 9 is interpolation result of the present invention displaying, and the interlayer slice generated in frame for interpolation, outer frame is that the continuous CT of input is cut
Piece, arrow are directed toward the obvious region of consecutive variations；
Figure 10 is that each algorithm realizes Contrast on effect and error image schematic diagram；
Figure 11 is the reparation of three dimensional CT interlayer image interpolation and super-resolution processing method flow chart.
Specific embodiment
The thought of interleave in video processing procedure is applied to the research of medical image super-resolution by the present invention.Original adjacent
It is inserted into a new slice in two CT slices, solves the problems, such as that adjacent two layers CT image has larger difference in partial structurtes.?
Effect in lung's sagittal plane is as shown in Figure 1, and solid horizontal line is the original CT slice that actual scanning obtains, and horizontal dotted line is desirable
The intermediate CT slice that interpolation obtains.Effect on cross section is as shown in Figure 2, and (a) indicates the image sequence of input, is (b) to need
The middle slice sequence for wanting interpolation to generate.
The present invention carries out interlayer interpolation to CT image using based on light stream estimation registration information, but in Interpolation Process,
Since the sports ground of light stream estimation is different, multiple pixels may be mapped to the same position in interpolation image in original image, thus
Cause in interpolation image there are the region that no original image pixel-map arrives, that is, pixel lacks, such that interpolation
There are said minuscule holes for image.So the present invention further repairs the interlayer slice that interpolation generates, interpolation graphs image quality is improved
Amount.It can be repaired while keeping image structure information, but this method is difficult to ensure finds in practical applications
Sample block be it is optimal, be easy to cause reparation error.It is synthesized using the weighted mean of multiple sample blocks to be repaired for filling
The filling block in region.
The intrinsic self-similarity of CT image sequence provides a precondition to repair, and therefore, the present invention is being based on light
On the basis of stream estimation carries out interlayer interpolation, the middle slice generated to interpolation carries out figure using the non local self-similarity of image
As reparation, CT number of sections can be increased, improve CT image interlayer resolution ratio, so that the quality of MFSR CT image reconstruction is promoted,
Patient is helped to obtain accurate diagnosing and treating in the case where not receiving unnecessary dose of radiation.
The present invention propose it is a kind of based on light stream estimation three dimensional CT interlayer image interpolation and repair algorithm.It is walked comprising three
It is rapid: as shown in figure 11,
S1 estimates the corresponding relationship of pixel between model solution CT image contiguous slices by CLG-TV light stream；
S2 finds target position by the mapping function of corresponding relationship between pixel, and is gone out with image warpage mode come interpolation
Middle slice；
S3, using the non local self-similarity of the intrinsic interframe of 4D-CT image, to the said minuscule hole generated in Interpolation Process and
It is fuzzy to carry out image repair.
The interpolation CT slice that high quality can be quickly obtained by operating above, improves the interlayer resolution ratio of CT image.It calculates
The entire frame of method is as shown in Figure 3, I1And I2It is the continuous CT image of two width of input, F is dense optical flow field, and f is correspondence between pixel
The mapping function of relationship, IipIt is the interlayer image of synthesis, IsIt is an original Slice Sequence set, passes through non local self similarity
Property obtain similar block matrix fill up pixel missing generate hole.
In S1, quickly due to the variation of human organ planform, if trace interval is longer, in different slices
On might have bigger difference, therefore the big displacement optical flow field for solving CT image is a relatively difficult problem.In light
In flow calculation methodologies, the data item in global (HS) algorithm is that the brightness conservation to single pixel point assumes that robustness is poor,
But available dense optical flow field；Data item in (LK) algorithm of part is to the picture in the small neighbourhood near single pixel point
The brightness conservation of vegetarian refreshments is assumed, has good robustness to noise, but can only obtain sparse optical flow field.
Full variation Optical flow estimation method based on CLG algorithm is can to solve the optical flow field of big displacement and have to noise
The optical flow computation model of good robustness can be in the hope of during CLG-TV optical flow estimation is applied to CT image layer interpolation
Solve the big displacement optical flow field between CT image contiguous slices.The present invention estimates model and its solution to CLG-TV light stream.Based on light
The estimation in flow field is as shown in Figure 4, wherein, (vx,vy) be pixel (x, y) at light stream.
CLG algorithm will be expanded in its neighborhood single pixel point using brightness conservation hypothesis in Global Algorithm, smoothly
Without modification, to obtain more accurate fine and close optical flow field.Model is estimated in CLG light stream are as follows:
Wherein, W (x, y) is the weight coefficient of each point in the neighborhood Ω centered on point (x, y), Ix、Iy、ItIt is pixel
Partial derivative of the gray scale of point along the direction x, y, tIt is flat
Sliding item.
For L in CLG light stream estimation model2Norm introduces the L to light stream to picture noise sensitive issue1Norm is about
Beam makes to estimate that model has more robust noiseproof feature
The smooth strategy of bilateral filtering technology and anisotropy parameter is introduced simultaneously, relative to pervious optical flow estimation, this
Item improves the influence that can weaken neighborhood territory pixel, preferably keeps image border, achievees the effect that protect side denoising.Add to data item
Add bilateral filtering constraint and anisotropy regularization to enhance specific filling process, obtains the more accurate CLG- of calculated result
Model is estimated in TV variation light stream:
In above formula,For data item, weight coefficient of the ω between data item and smooth item, bfw is indicated to data
Item carries out smothing filtering, and region represents the scope of two-sided filter.Smooth item EsmoothUse the anisotropy of image-driven
Invasin
Data itemIt is defined as follows:
Wherein, X=(x, y) is pixel coordinate, F=(vx,vy) be two-dimensional directional on light stream (direction x and the direction y),
F0For the initial estimation of optical flow field.
During solving optical flow estimation, using texture structure decomposition method and by slightly to essence gaussian pyramid algorithm,
Optical flow estimation can be avoided to be illuminated by the light the influence of variation to the full extent and realize the solution of big displacement optical flow field.To image
Structural texture decomposition is carried out, input picture I is decomposed into the part I comprising image structure informationsWith include image texture information
Part It,It=I-Is, due to ItIt there is no by shade, the influence blocked etc., by ItAs new input picture, just
Beginningization F0.Secondly, the pyramid number of plies is determined by image sizeEach layer of light stream has It is the light stream that k+1 layers are transmitted to from the k layer of low precision,It is to be transmitted to from k layers
K+1 layers of light stream increment.Alternating direction multipliers method (ADMM) iteratively faster is recycled to solve light stream, until k=n-1.Realization pair
Optical flow estimation rapid solving effectively increases the efficiency of algorithm.
Fig. 5 shows the example that using CLG-TV light stream algorithm for estimating three width coronal-plane CT images are carried out with light stream estimation.
Wherein reference picture and image a, image b are chosen from same CT image sequence, and light stream a is reference picture and image a
Between local optical flow field, light stream b is the local optical flow field between reference picture and image b.Image a and reference picture displacement compared with
Small (arrow indicating positions), image b and reference picture displacement are larger, this is consistent with the optical flow field of estimation.Illustrate that CLG-TV can
Accurately estimate the big displacement optical flow field between CT image contiguous slices.
In S2, it is based on three-dimensional CT image interlayer Interpolation Process, the warpage operation in conventional method is introduced into method,
Big offset issue can be effectively solved, reasonable interpolation result is obtained.Image warpage refers to each picture calculated in original image
Element moves the mapping function between vector, realizes the displacement of pixel, to obtain target image.To between two images
Middle layer to carry out the requirement of interpolation be that movement to generation carries out interpolation, the purpose is to make moving object along motion profile with one
The appropriate mode of kind nature generates interpolation image, moves simultaneously for non-linear translation, and warpage can compensate some geometric distortions
Problem.
The mapping function q of pixel is calculated by CLG-TV optical flow estimationi=f (pi), determine pixel p in former slicei's
Target position qi, then using bilinear interpolation algorithm in qiRealize warpage operation in place.Since target position will appear non-integer feelings
Condition then finds out target position qiWith it to angular neighborhood ND (qi) Euclidean distance ρ, selection make that the smallest pixel of Euclidean distance
Point is used as insertion point.
ρ=(| x1-x2|2+|y1-y2|2)1/2 (6)
In S3, according to the difference of sports ground, multiple pixels of original image may be mapped to the same position in interpolation image
It sets.On the other hand, there may be the regions that no original pixel is mapped in interpolation image, so as to cause occurring in interpolation image
Said minuscule hole, as shown in Figure 6.I1And I2Indicate two continuous CT images of input, IipIt is the interlayer image of output, based on box
Change the partial enlarged view in region.
In view of the above problems, the present invention uses the image restoration technology of block-based non local self-similarity.Due to medicine
The self-similarity of image can find the CONSTRUCTED SPECIFICATION of missing on other existing slices, then middle slice is divided
At the fritter of many overlappings, the image block repaired for each needs can be cut using the information of its surrounding neighbors at other
On piece finds matching similar block, then solves optimization problem and obtains optimal similar block matrix to fill up missing
Pixel finally reconstructs details middle slice abundant.As shown in fig. 7, the target image block on intermediate interpolated slice can lead to
It crosses and finds L most like similar block from the image block (in frame) on other n × N number of existing slice to estimate.
In Fig. 7, original image is sliced by methodN is of CT image sequence
Number, n is sequence IpIn former slice sum, and the middle slice I obtained based on light stream registration interpolation methodipAccording to step-length d points
It is not divided into M mutually overlapping image blocks Indicate sliceIn i-th of vectorization image block, often
The size of a block is m × m.ForIn image blockUsing block-matching technique and based on the similitude of Euclidean distance
Measurement criterion, fromMiddle searchSimilar block, for the ease of discuss, by xiIt is set as the interlayer slice that interpolation obtains
Image block,It is the image block on other slices.The judgment basis of similitude is:
T is threshold value.According to similarity measurement criterion choose withL most like image block is as column vector, construction pair
The group matrix H answeredj, that is:
HjIt can also be with representing matrix PjWith noise matrix NjThe sum of:
Hj=Pj+Nj. (9)
NjIt can be regarded as the additive noise that standard deviation is σ.H is replaced with H and PjAnd Pj.Singular value point is carried out to H and P
It solves (SVD):
Wherein, U is left singular vector, and V is right singular vector, and Σ is the diagonal matrix of a r × r, the member on diagonal line
Element is exactly the singular value of matrix.Due to highly relevant between the column vector of matrix P, therefore matrix P is low-rank matrix, order R (P) <
r.The value of P is obtained using minimum variance estimate (MVE) method
If known to P, then objective function above is set to the derivative of M as zero, available following result
M=(H*H)-1H*P. (13)
So minimum variance estimate of P is
BecauseDiagonal entry λ1,λ2,……,λkIt is non-zero positive integer, therefore
(15) are substituted into (14), so that it may obtain ideal matrix in the hope of solutionThen the matrix acquired is utilizedHandle interpolation
The said minuscule hole of generation, the image block for needing to repair can be indicated by its non local similar block weighting
xiIt is the image block to be repaired,It is and xiL most like similar block,It is weighting coefficient
τ is an adjusting parameter, the decaying of its control exponential function.
CT interpolation image (Fig. 8 first row, original image and part before repairing can be effectively repaired by similar block-matching technique
Enlarged drawing), obtain the good CT image of visual effect (Fig. 8 secondary series, original image and partial enlarged view after reparation).Due to image
Block be it is mutually overlapping, may result in this block region to be processed and obtain multiple pixel estimated values, herein to each pixel
Different estimated values takes average weighted method.
University of Texas Anderson Cancer center DIR- is based on by practical application using method of the present invention
The public data collection that the laboratory lab provides, the interlamellar spacing of data are 2.5mm.Therefrom having chosen interlamellar spacing is the continuous lung of 2.5mm
CT image, these CT images include coronal-plane, cross section.Wherein, the image size in lung cross section is 560 × 420, coronal-plane
Image size be 664 × 498.In experiment, due to being limited by factors such as medical device, sweep times, it can not be sliced
The smaller CT data of spacing.Therefore, the interlayer image that interpolation obtains carries out visual evaluation and root-mean-square error using error image
(RMSE) it carries out algorithm quantitatively evaluating and lacks true picture (Ground Truth) foundation.So CT slice is chosen to carry out in interval
Emulation experiment, for example three continuous CT slices are chosen, by first and third CT slice as input, second CT slice module
Intend true interlayer slice.Table 1 gives the running environment of algorithm.
1 algorithm running environment of table
In order to be quantitatively evaluated, interpolation error (interpolation error, IE), i.e. true picture are defined
(Ground Truth)IGTImage and estimation interpolation image IipRoot-mean-square error (RMSE),
Wherein, M × N is image size, and root-mean-square error is defined according to mean square error (MSE)
MSE is the range deviation that estimated value deviates true value.Therefore, root-mean-square error can be used to measure interpolation image with
Error size between true picture.The value of RMSE is smaller, and it is higher accurate to illustrate that the new images of experiment interpolation generation have
Degree, closer to true CT image.
Specific parameter is set as, and in light stream solution procedure, the weight coefficient ω between data item and smooth item is set
The bigger confidence level for indicating data item of value for being set to 50, ω is higher.The scope region of two-sided filter is set as 7 × 7,
Minwidth is set as 20, can be good at solving big displacement light stream estimation problem, the average optical flow computation time is 4.052s.
Image repair process is mainly influenced by four key parameters: the size m of image block, organizes the columns L of matrix H, mark
Quasi- difference σ, step-length d.Wherein the size m of image block influences the result of image repair most deep.The size of image block is smaller more advantageous
It in acquisition partial structurtes details, but will affect the selection of missing pixel surrounding neighbors size, lead to the estimation inaccuracy of similar block,
Reduce the performance and effect of algorithm；The big image block of size available more image informations carry out similitude matching, but
Higher calculating cost may be generated, based on the above analysis, in order to be weighed between algorithm performance and accuracy, takes m
=16, σ=40, L=60, d=4, available quality preferably repair interpolation image.
It, can be with by above-mentioned experiment method, and using the three dimensional CT interlayer image interpolation restorative procedure estimated based on light stream
Realize the middle slice that high quality is inserted between continuous two slices of CT image.Interpolation result of the present invention is as shown in Figure 9, chooses
Three groups of not homotactic continuous tomography example (a, b, c) CT slices, each group of outer frame is that the continuous CT of four width of input is sliced, in frame
Part is sliced for the three width middle layers generated by this method interpolation, and arrow, which is directed toward, changes obvious region.CT image outer profile becomes
It is coherent to change nature, arrow direction regional change is continuous, the effect that can be seamlessly transitted.
In terms of visual evaluation, randomly selects the CT that three width pixel sizes are 560 × 420 and has been sliced (Figure 10 the first row),
Middle layer slice is used as Ground Truth.Figure 10 is illustrated to be calculated using TV-L1 optical flow algorithm, the interpolation based on path respectively
Method, shape based interpolation algorithm, this method realize the interlayer interpolation result to upper layer slice a and lower layer slice b in Figure 10.
It is image Main change region in frame.
By comparison, it can be found that, the effect for the middle slice that inventive algorithm interpolation generates is better than TV-L1 algorithm and base
Similar with the algorithm effect based on path in the algorithm of shape, main portions change nature and link up, and it is preferable can to generate quality
Middle slice.Compared to the algorithm based on path, inventive algorithm time complexity is reduced, while in terms of handling larger displacement
More acurrate clearly effect can be obtained.In order to preferably carry out visual comparison, Figure 10 gives four kinds of algorithms in Figure 10
Upper layer slice a and lower layer slice b interpolation go out middle slice and true interlayer image error image comparison diagram (Figure 10 third
Row), it can be seen that inventive algorithm illustrates that inventive algorithm can accurately carry out CT image layer and interleave closest to true picture
Value.
True picture is more connect in order to verify the middle slice that inventive algorithm interpolation obtains, present invention introduces IE evaluation indexes
Quantitatively evaluating is carried out to the picture quality after reconstruction.For this 4 kinds of algorithms, three groups of lung CT images are had chosen, in each group
There is the continuous CT slice of three width, regard original middle layer slice as true picture, upper layer and lower layer slice is calculated separately as input
The RMSE value of middle slice and true picture that these four algorithm interpolation generate, as shown in table 2.
2 algorithms of different interlayer of table is sliced compared with the RMSE that original CT is sliced
From Figure 10 and table 2 as can be seen, no matter from visual effect or from the point of view of referring specifically to scale value, inventive algorithm is taken
The result better than other algorithms was obtained, is especially changing greatly region, inventive algorithm has obtained clearer structural information.
Therefore, the three dimensional CT interlayer image that inventive algorithm interpolation generates is more accurate, has been effectively maintained edge and local message, layer
Interpolation effect is more preferable.
In the present invention, the method for a kind of the three dimensional CT interlayer interpolation based on light stream estimation and reparation is proposed, is to utilize light
The registration information that stream obtains goes out a new middle layer slice in CT image upper layer and lower layer interpolation, and cuts to the centre that interpolation obtains
Piece is repaired using block-based non local self-similarity, to improve CT image interlayer resolution ratio, is promoted MFSR and is rebuild CT
The quality of image has great importance to diagnosis disease and patient undergoing treatment etc..No matter method proposed by the present invention
It visually or in terms of quantitatively evaluating will be better than based on shape scheduling algorithm.
The present invention also provides a kind of device for realizing the three dimensional CT reparation of interlayer image interpolation and super-resolution processing method, packets
It includes: memory, for storing computer program and three dimensional CT interlayer image interpolation reparation and super-resolution processing method；Processor,
For executing the computer program and realizing multi-memory pressure testing system, to realize three dimensional CT interlayer image interpolation reparation
The step of with super-resolution processing method.
Technology as described herein may be implemented in hardware, software, firmware or any combination of them.The various spies
Sign is module, and unit or assembly may be implemented together in integration logic device or separately as discrete but interoperable logic
Device or other hardware devices.In some cases, the various features of electronic circuit may be implemented as one or more integrated
Circuit devcie, such as IC chip or chipset.
If realized within hardware, the present invention relates to a kind of devices, such as can be used as processor or integrated circuit dress
It sets, such as IC chip or chipset.Alternatively or additionally, if realized in software or firmware, the technology can
Realize at least partly by computer-readable data storage medium, including instruction, when implemented, make processor execute one or
More above methods.For example, computer-readable data storage medium can store the instruction such as executed by processor.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, defined in the present invention
General Principle can realize in other embodiments without departing from the spirit or scope of the present invention.Therefore, this hair
It is bright to be not intended to be limited to these embodiments shown in the present invention, and be to fit to special with principles of this disclosure and novelty
The consistent widest scope of point.
Claims (7)
1. three dimensional CT interlayer image interpolation reparation and super-resolution processing method, which is characterized in that method includes:
Step 1 estimates the corresponding relationship of pixel between model solution CT image contiguous slices by CLG-TV light stream；
Step 2 finds target position by the mapping function of corresponding relationship between pixel, and is gone out with image warpage mode come interpolation
Middle slice；
Step 3, using the non local self-similarity of the intrinsic interframe of 4D-CT image, to the said minuscule hole generated in Interpolation Process and
It is fuzzy to carry out image repair.
2. three dimensional CT interlayer image interpolation reparation according to claim 1 and super-resolution processing method, which is characterized in that step
Rapid one further include:
Single pixel point will be expanded in its neighborhood using brightness conservation hypothesis in Global Algorithm by CLG algorithm, smoothly
Item without modification, obtains more accurate fine and close optical flow field；Model is estimated in CLG light stream are as follows:
ECLG=∫Ω(W2(x,y)·(Ixvx+Iyvy+It)2+S)dxdy. (1)
Wherein, W (x, y) is the weight coefficient of each point in the neighborhood Ω centered on point (x, y), Ix、Iy、ItIt is pixel
Partial derivative of the gray scale along the direction x, y, tIt is smooth
?；
For L in CLG light stream estimation model2Norm introduces the L to light stream to picture noise sensitive issue1Norm constraint makes
Estimate that model has more robust noiseproof feature
The smooth strategy of bilateral filtering technology and anisotropy parameter is introduced simultaneously；
Specific filling process is enhanced to data item addition bilateral filtering constraint and anisotropy regularization, obtains CLG-TV change
Light splitting stream estimation model:
In above formula,For data item, weight coefficient of the ω between data item and smooth item, bfw indicate to data item into
Row smothing filtering, region represent the scope of two-sided filter；Smooth item EsmoothUse the anisotropy parameter of image-driven
The factor
Data itemIt is defined as follows:
Wherein, X=(x, y) is pixel coordinate, F=(vx,vy) it is light stream on two-dimensional directional, x indicates that the direction x and y indicate y
Direction, F0For the initial estimation of optical flow field.
3. three dimensional CT interlayer image interpolation reparation according to claim 2 and super-resolution processing method, which is characterized in that step
Rapid one further include:
During solving optical flow estimation, using texture structure decomposition method and by slightly to essence gaussian pyramid algorithm, to figure
As carrying out structural texture decomposition, input picture I is decomposed into the part I comprising image structure informationsBelieve with comprising image texture
The part I of breatht,It=I-Is, due to ItIt there is no by shade, the influence blocked etc., by ItAs new input picture,
Initialize F0；
The pyramid number of plies is determined by image size
Each layer of light stream has It is the light stream that k+1 layers are transmitted to from the k layer of low precision,It is the light stream increment that k+1 layers are transmitted to from k layers；
Light stream is solved using alternating direction multipliers method (ADMM) iteratively faster, until k=n-1.
4. three dimensional CT interlayer image interpolation reparation according to claim 1 and super-resolution processing method, which is characterized in that step
Rapid two further include:
In three-dimensional CT image interlayer Interpolation Process, the warpage operation in conventional method is introduced into algorithm, interpolation knot is obtained
Fruit；
Each pixel in original image is calculated based on image warpage and moves the mapping function between vector, realizes pixel
Displacement, obtains target image；
Carrying out interpolation to the middle layer between two images is that the movement to generation carries out interpolation, makes moving object along motion profile
Interpolation image is generated in a kind of naturally appropriate mode, is moved simultaneously for non-linear translation, geometric distortion is compensated；
The mapping function q of pixel is calculated by CLG-TV optical flow estimationi=f (pi), determine pixel p in former sliceiTarget
Position qi；Using bilinear interpolation algorithm in qiRealize warpage operation in place；
Find out target position qiWith it to angular neighborhood ND (qi) Euclidean distance ρ, selection make that the smallest pixel of Euclidean distance
As insertion point；
ρ=(| x1-x2|2+|y1-y2|2)1/2 (6)。
5. three dimensional CT interlayer image interpolation reparation according to claim 1 and super-resolution processing method, which is characterized in that step
Rapid three further include:
Original image is slicedN is the number of CT image sequence, and n is sequence IpIn original
Slice sum, and the middle slice I obtained based on light stream registration interpolation methodipIt is mutually overlapping that M is respectively classified into according to step-length d
Image block Indicate sliceIn i-th of vectorization image block, each piece of size is m × m；
ForIn image blockUsing block-matching technique and the similarity measurement criterion based on Euclidean distance, fromMiddle searchSimilar block；
By xiIt is set as the image block for the interlayer slice that interpolation obtains,It is the image block on other slices；
The judgment basis of similitude is:
T is threshold value；According to similarity measurement criterion choose withL most like image block constructs corresponding as column vector
Group matrix Hj, that is:
HjIt is expressed as matrix PjWith noise matrix NjThe sum of:
Hj=Pj+Nj. (9)
NjIt can be regarded as the additive noise that standard deviation is σ；H is replaced with H and PjAnd Pj；Singular value decomposition is carried out to H and P
(SVD):
Wherein, U is left singular vector, and V is right singular vector, and Σ is the diagonal matrix of a r × r, and the element on diagonal line is just
It is the singular value of matrix；Due to highly relevant between the column vector of matrix P, matrix P is low-rank matrix, order R (P) < r；
Using minimum variance estimate (MVE) method, the value of P is obtained:
If P it is known that so set objective function above to the derivative of M as zero, obtains following result
M=(H*H)-1H*P. (13)
So minimum variance estimate of P is
It is based onDiagonal entry λ1,λ2,……,λkIt is non-zero positive integer,
(15) are substituted into (14), solution obtains ideal matrix
Utilize the matrix acquiredThe said minuscule hole that interpolation generates is handled, the image block for needing to repair passes through its non local similar block
Weighting indicates
xiIt is the image block to be repaired,It is and xiL most like similar block,It is weighting coefficient
τ is an adjusting parameter, the decaying of control exponential function；
CT interpolation image is repaired by similar Block- matching, obtains the good CT image of visual effect.
6. three dimensional CT interlayer image interpolation reparation according to claim 5 and super-resolution processing method, which is characterized in that step
Rapid three further include:
Average weighted method is taken to obtain described image block with multiple pixel estimated values and mutually overlapping image block
Pixel value.
7. a kind of device for realizing the three dimensional CT reparation of interlayer image interpolation and super-resolution processing method characterized by comprising
Memory, for storing computer program and three dimensional CT interlayer image interpolation reparation and super-resolution processing method；
Processor, for executing the computer program and three dimensional CT interlayer image interpolation reparation and super-resolution processing method, with
Realize as described in claim 1 to 6 any one the reparation of three dimensional CT interlayer image interpolation and the step of super-resolution processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910415030.1A CN110211193B (en) | 2019-05-17 | 2019-05-17 | Three-dimensional CT (computed tomography) interlayer image interpolation restoration and super-resolution processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910415030.1A CN110211193B (en) | 2019-05-17 | 2019-05-17 | Three-dimensional CT (computed tomography) interlayer image interpolation restoration and super-resolution processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110211193A true CN110211193A (en) | 2019-09-06 |
| CN110211193B CN110211193B (en) | 2020-08-04 |
Family
ID=67787565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910415030.1A Active CN110211193B (en) | 2019-05-17 | 2019-05-17 | Three-dimensional CT (computed tomography) interlayer image interpolation restoration and super-resolution processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110211193B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287973A (en) * | 2020-09-28 | 2021-01-29 | 北京航空航天大学 | Digital image countermeasure sample defense method based on truncated singular value and pixel interpolation |
| WO2022133806A1 (en) * | 2020-12-23 | 2022-06-30 | 深圳迈瑞生物医疗电子股份有限公司 | Fetal face volume image inpainting method and ultrasound imaging system |
| CN115082323A (en) * | 2022-08-19 | 2022-09-20 | 深流微智能科技(深圳)有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN116797457A (en) * | 2023-05-20 | 2023-09-22 | 北京大学 | Method and system for simultaneously realizing super-resolution and artifact removal of magnetic resonance image |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160110848A1 (en) * | 2013-08-30 | 2016-04-21 | A9.Com, Inc. | Self-portrait enhancement techniques |
| KR101711589B1 (en) * | 2015-12-08 | 2017-03-02 | 연세대학교 산학협력단 | Method and Apparatus of Dictionary Design on Super-Resolution, and Generating Super-Resolution Image based on the Dictionary |
| CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
| CN107067367A (en) * | 2016-09-08 | 2017-08-18 | 南京工程学院 | A kind of Image Super-resolution Reconstruction processing method |
| CN107221013A (en) * | 2017-05-16 | 2017-09-29 | 山东财经大学 | One kind is based on variation light stream estimation lung 4D CT Image Super Resolution Processing methods |
| CN107274347A (en) * | 2017-07-11 | 2017-10-20 | 福建帝视信息科技有限公司 | A kind of video super-resolution method for reconstructing based on depth residual error network |
| CN108154474A (en) * | 2017-12-22 | 2018-06-12 | 浙江大华技术股份有限公司 | A kind of super-resolution image reconstruction method, device, medium and equipment |
| CN109410177A (en) * | 2018-09-28 | 2019-03-01 | 深圳大学 | A kind of image quality analysis method and system of super-resolution image |
| CN109658361A (en) * | 2018-12-27 | 2019-04-19 | 辽宁工程技术大学 | A kind of moving scene super resolution ratio reconstruction method for taking motion estimation error into account |
-
2019
- 2019-05-17 CN CN201910415030.1A patent/CN110211193B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160110848A1 (en) * | 2013-08-30 | 2016-04-21 | A9.Com, Inc. | Self-portrait enhancement techniques |
| KR101711589B1 (en) * | 2015-12-08 | 2017-03-02 | 연세대학교 산학협력단 | Method and Apparatus of Dictionary Design on Super-Resolution, and Generating Super-Resolution Image based on the Dictionary |
| CN107067367A (en) * | 2016-09-08 | 2017-08-18 | 南京工程学院 | A kind of Image Super-resolution Reconstruction processing method |
| CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
| CN107221013A (en) * | 2017-05-16 | 2017-09-29 | 山东财经大学 | One kind is based on variation light stream estimation lung 4D CT Image Super Resolution Processing methods |
| CN107274347A (en) * | 2017-07-11 | 2017-10-20 | 福建帝视信息科技有限公司 | A kind of video super-resolution method for reconstructing based on depth residual error network |
| CN108154474A (en) * | 2017-12-22 | 2018-06-12 | 浙江大华技术股份有限公司 | A kind of super-resolution image reconstruction method, device, medium and equipment |
| CN109410177A (en) * | 2018-09-28 | 2019-03-01 | 深圳大学 | A kind of image quality analysis method and system of super-resolution image |
| CN109658361A (en) * | 2018-12-27 | 2019-04-19 | 辽宁工程技术大学 | A kind of moving scene super resolution ratio reconstruction method for taking motion estimation error into account |
Non-Patent Citations (4)
| Title |
|---|
| KAN CHANG，ET AL: "Single image super-resolution using collaborative representation and non-local self-similarity", 《ELSEVIER》 * |
| 王小江: "基于非局部相似性的图像超分辨率", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 肖珊: "基于运动估计的肺4D-CT图像冠矢状面超分辨率重建", 《中国优秀硕士学位论文全文数据库 医药卫生科技辑》 * |
| 谭君: "基于变分原理的光流估计方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287973A (en) * | 2020-09-28 | 2021-01-29 | 北京航空航天大学 | Digital image countermeasure sample defense method based on truncated singular value and pixel interpolation |
| WO2022133806A1 (en) * | 2020-12-23 | 2022-06-30 | 深圳迈瑞生物医疗电子股份有限公司 | Fetal face volume image inpainting method and ultrasound imaging system |
| CN115082323A (en) * | 2022-08-19 | 2022-09-20 | 深流微智能科技(深圳)有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN115082323B (en) * | 2022-08-19 | 2022-11-04 | 深流微智能科技(深圳)有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN116797457A (en) * | 2023-05-20 | 2023-09-22 | 北京大学 | Method and system for simultaneously realizing super-resolution and artifact removal of magnetic resonance image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110211193B (en) | 2020-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhou et al. | Handbook of medical image computing and computer assisted intervention | |
| CN110211193A (en) | Three dimensional CT interlayer image interpolation reparation and super-resolution processing method and device | |
| Mac Aodha et al. | Patch based synthesis for single depth image super-resolution | |
| Zheng et al. | Multi-contrast brain MRI image super-resolution with gradient-guided edge enhancement | |
| US20120302880A1 (en) | System and method for specificity-based multimodality three- dimensional optical tomography imaging | |
| Xia et al. | Super-resolution of cardiac MR cine imaging using conditional GANs and unsupervised transfer learning | |
| CN106981090B (en) | Three-dimensional reconstruction method for in-tube stepping unidirectional beam scanning tomographic image | |
| CN107464216A (en) | A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks | |
| US20150133784A1 (en) | Three-dimensional ultrasound reconstruction with confidence information | |
| CN102903103B (en) | Migratory active contour model based stomach CT (computerized tomography) sequence image segmentation method | |
| Moran et al. | Using super-resolution generative adversarial network models and transfer learning to obtain high resolution digital periapical radiographs | |
| WO2022105647A1 (en) | Method for establishing enhanced three-dimensional model of intracranial angiography | |
| CN110363802B (en) | Prostate image registration system and method based on automatic segmentation and pelvis alignment | |
| Wu et al. | An arbitrary scale super-resolution approach for 3d mr images via implicit neural representation | |
| CN103020898B (en) | Sequence iris image super resolution ratio reconstruction method | |
| Vu et al. | Perception-enhanced image super-resolution via relativistic generative adversarial networks | |
| CN107292819A (en) | A kind of infrared image super resolution ratio reconstruction method protected based on edge details | |
| CN114241077B (en) | CT image resolution optimization method and device | |
| CN107845065A (en) | Super-resolution image reconstruction method and device | |
| CN115578404A (en) | Liver tumor image enhancement and segmentation method based on deep learning | |
| US20170278244A1 (en) | Method and a system for non-rigid image registration | |
| CN116681894A (en) | Adjacent layer feature fusion Unet multi-organ segmentation method, system, equipment and medium combining large-kernel convolution | |
| CN103020936B (en) | A kind of face image super-resolution reconstructing method | |
| CN116758087B (en) | Lumbar vertebra CT bone window side recess gap detection method and device | |
| CN113870327A (en) | Medical image registration method based on multi-level deformation field prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |