CN105844586A - Features-based 2d/3d image registration - Google Patents

Features-based 2d/3d image registration Download PDF

Info

Publication number
CN105844586A
CN105844586A CN201610195981.9A CN201610195981A CN105844586A CN 105844586 A CN105844586 A CN 105844586A CN 201610195981 A CN201610195981 A CN 201610195981A CN 105844586 A CN105844586 A CN 105844586A
Authority
CN
China
Prior art keywords
described
dimensional
image
projection
registration
Prior art date
Application number
CN201610195981.9A
Other languages
Chinese (zh)
Inventor
L·G·扎戈尔谢夫
R·曼茨克
R·陈
Original Assignee
皇家飞利浦电子股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US1448807P priority Critical
Priority to US61/014,488 priority
Application filed by 皇家飞利浦电子股份有限公司 filed Critical 皇家飞利浦电子股份有限公司
Priority to CN2008801211036A priority patent/CN101903908A/en
Publication of CN105844586A publication Critical patent/CN105844586A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/02Devices for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03Computerised tomographs
    • A61B6/032Transmission computed tomography [CT]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/48Diagnostic techniques
    • A61B6/486Diagnostic techniques involving generating temporal series of image data
    • A61B6/487Diagnostic techniques involving generating temporal series of image data involving fluoroscopy
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/48Diagnostic techniques
    • A61B6/488Diagnostic techniques involving pre-scan acquisition
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/52Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5205Devices using data or image processing specially adapted for radiation diagnosis involving processing of raw data to produce diagnostic data
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/52Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5229Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
    • A61B6/5235Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from the same or different radiation imaging techniques, e.g. PET and CT
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/0068Geometric image transformation in the plane of the image for image registration, e.g. elastic snapping
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10081Computed x-ray tomography [CT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10088Magnetic resonance imaging [MRI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • G06T2207/10121Fluoroscopy
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • G06T2207/10124Digitally reconstructed radiograph [DRR]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30048Heart; Cardiac
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30101Blood vessel; Artery; Vein; Vascular

Abstract

An image registration apparatus comprises: a features detector (34) configured to extract a two-dimensional set of features (36) from a two-dimensional image (30) and to extract a three-dimensional set of features (38) from a three-dimensional image (32); a projection processor (40) configured to project three-dimensional data into two-dimensional projection data; and a registration processor (46, 52) configured to (i) adjust parameters to register the two-dimensional set of features and the three-dimensional set of features projected by the projection processor using a projection geometry (42), and to (ii) use the adjusted parameters to register the two-dimensional image and the three-dimensional image projected by the projection processor using the projection geometry.

Description

The 2D/3D image registration of feature based

The application is the Application No. 200880121103.6, entitled that December in 2008 is submitted on the 01st The divisional application of " the 2D/3D image registration of feature based ".

Technical field

The present invention relates to medical imaging technology.In certain embodiments, it relates to two dimension (2D) x Actinoscopy X image gathers with by computer tomography, nuclear magnetic resonance or other imaging patterns Three-dimensional (3D) image registrate.But, more generally, the present invention relates to by arbitrarily doctor Study the two dimensional image as type collection and the graphics gathered by identical or different medical imaging modalities As registrating.

Background technology

In medical imaging procedure, occasionally there are use both two and three dimensions imagings and gather dependent imaging The situation of data.In the case of some are such, generate the two-dimensional representation of 3-D view and by described three The two-dimensional representation of dimension image registrates with corresponding two dimensional image, thus compares and provided by two kinds of technology Information and be combined being useful by described information.

One example occurs sometimes in interventional cardiac electrophysiology.In this program, x-ray is saturating Depending on being sometimes used for conduit or the visualization of other intervention tools.Advantageously, " C-arm " class is used Device can gather x-ray fluoroscopic image, x-ray tube and x-ray detection in C-arm class device Device is arranged on the opposite end of C-arm, and patient is placed in gap.C-arm class device is mutually split Put, so that the easily accessible medical personnel of patient.But, can not be to some by x-ray perspective Soft tissue anatomical structures imaging effectively.It addition, generally gather fluoroscopy images under low x-ray dosage, This can reduce resolution.

It is thus known that before carrying out cardiac electrophysiology program, use such as many section computers to break The 3 Dimension Image Technique of layer photography (CT) or nuclear magnetic resonance (MRI) gathers the operation consent figure of patient Picture, any one of described many slice computed tomography (CT) or nuclear magnetic resonance (MRI) carries For having an X-rayed more preferable soft tissue contrast than x-ray.Afterwards, CT or MRI operation consent gathered Image merges with the x-ray fluoroscopy images gathered in cardiac electrophysiology program, so that CT or MRI image provide the soft tissue contrast lost.

CT or MRI image are generally directed to three-D volumes and generate;But, x-ray fluoroscopic image is two dimension 's.Known use ray casting technique by 3-D view mathematical projection to two dimensional image.To CT or MRI The projection of image application ray generates two dimensional image.But, the CT mathematically projected or MRI image lead to Normal and x-ray fluoroscopy images is not spatial registration, and this is owing to the x-ray about patient is had an X-rayed Perspective geometry is generally different from the perspective geometry used in the mathematical generation that CT or MRI projects.? Under certain situation, due in x-ray fluoroscopy images and/or the distortion in three dimensional CT or MRI image Or other defect or artifact can cause other error.

Summary of the invention

Provide below the improvement overcoming the problems referred to above and other problems.

Disclose a kind of process of image registration, including: extract two dimensional character collection from two dimensional image;From three Dimension image zooming-out three-dimensional feature collection;Use perspective geometry that three-dimensional feature collection mathematical projection is become two-dimensional projection Feature set;Carry out two dimensional character collection and two-dimensional projection feature set registrating for the first time;And use from The mathematical projection of two dimensional image with 3-D view is carried out registrating for the second time by the parameter once registrating derivation.

Digital storage media or medium can store and can be run, by digital display circuit, the method described in leading portion that performs Instruction.

Disclose a kind of image registration device, including: property detector, it is configured to from X-Y scheme As extracting two dimensional character collection and extracting three-dimensional feature collection from 3-D view;Projection processor, its configuration For three-dimensional data is projected into two-dimensional projection data;And registration processor, it is configured to (i) Regulation parameter is with by two dimensional character collection and the three-dimensional feature collection being used perspective geometry projection by projection processor Registrate, and (ii) uses the parameter being adjusted to be thrown with being used by projection processor by two dimensional image The 3-D view of shadow geometric projection registrates.

Also disclose that a kind of device, including: two dimensional imager, it is configured to gather two dimensional image; Three-dimensional imager, it is configured to gather 3-D view;Property detector, it is configured to from two dimension Image zooming-out two dimensional character collection and from 3-D view extract three-dimensional feature collection;Projection processor, it is joined Put for three-dimensional data is projected as two-dimensional projection data;And registration processor, it is configured to make Carry out two dimensional image and the 3-D view that projected by projection processor registrating by such parameter, institute State parameter to be adjusted for joining two dimensional character collection with the three-dimensional feature collection projected by projection processor Accurate.

One advantage is 2D/3D image registration faster.

Another advantage is that more accurate 2D/3D image registration.

Another advantage is that and get involved imaging faster.

Those of ordinary skill in the art will can appreciate the fact that this after reading and understanding following detailed description Bright other advantage.

Accompanying drawing explanation

Accompanying drawing is only for explanation preferred embodiment, and should not be considered to limit the present invention.

Fig. 1 schematically shows 2D/3D multi-modality imaging device;

Fig. 2 schematically shows a kind of 3D/3D multi-modality imaging device, is wherein adopted by different modalities The two-dimensional projection of the 3-D view of collection is registration.

Detailed description of the invention

Include that two dimensional imager 10, such as x-ray are saturating with reference to Fig. 1,2D/3D multi-mode imaging system View apparatus, and three-dimensional imager 12, such as nuclear magnetic resonance (MRI) system, computerized tomography Photography (CT) imaging system, PET (positron emission tomography) (PET) scanning device, Gamma camera Deng.Two dimensional imager 10 is optionally able to carry out three-dimensional imaging, but in current 2D/3D imaging system In be used as two dimensional imager.For example, it is contemplated that two dimensional imager 10 is the feelings not rotated at stand The CT scanner worked under condition.

Two dimensional imager 10 is the projection type imager including source 20, and this source 20 is for example, in x-ray X-ray tube in the case of arrangement for perspective, it sends radiation 22 by comprising the one-tenth of object (not shown) As region 24, so that two-dimensional detector array 26 the most staggered relatively detects along with change in location The radiation sent is to form the two dimensional image 30 of projection type.Therefore, two dimensional image 30 has with projection The perspective geometry of parameter characterization, described projective parameter such as angulation, source position, detector location or its His geometry parameter, and the most also to be characterized as and the relevant projective parameter that distorts, such as, characterize For the so-called " pillow sometimes observed in x-ray arrangement for perspective or other projection types 2D imaging device Shape " one or more distortion parameters of distorting.Perspective geometry at least approximately it is known that such as, based on x The nominal of radiographic source and detector location is arranged.In certain embodiments, two dimensional imager 10 is accurate Calibration provides the high accuracy projective parameter for perspective geometry, including accurate geometry parameter and right Quantitative values in distortion parameter.

Three-dimensional imager 12 gathers 3-D view 32.Such as, if three-dimensional imager 12 is MRI, It by carrying out three-dimensional sample and described k-spatial sampling is redeveloped into 3-D view 32 adopting to k-space Collect this 3-D view 32.If three-dimensional imager 12 is CT scanner, its acquired projections data, with This simultaneously, x-ray tube rotates around object, and the third dimension (is cut into slices by having the detector of multirow more CT) and/or by with discrete increment or continuously move patient's (spiral CT) and provide, it is afterwards Filtered back projection or by reconstructing projection data be 3-D view 32 another reconstruct.Can be used it His method, this type depending on three-dimensional imager 12 and radiologist or other medical professions The type of desired collection.

How to position in two different imagers 10,12 based on object, the projection of two dimensional image 30 Relation between the spatial reference frames of geometry and 3-D view 32 is the most known.In certain embodiments, This relation the most known, such as, if two different imagers 10,12 are by integrally common real It is now hybrid imaging system, or uses and intersect imager mechanical registeration mechanism.But, in any feelings Under condition, the two dimensional image 30 on the one hand gathered and another aspect will be usually present by two dimensional imager 10 Some mismatches between the 3-D view 32 gathered by three-dimensional imager 12 are accurate.This mismatch will definitely be with body It is now the combination of various mode and mode, such as rigid translation mismatch is accurate, rigid rotating mismatch is accurate, Non-rigid translation and/or rotational misalignment be accurate, due to the pincushion in one or two of image 30,32 abnormal The mismatch that change or other kinds of distortion cause is accurate, etc..Therefore, it is desirable to by 3-D view 32 mathematics Project to form the projection picture of two dimension, and by the projection picture of this two dimension and by two dimensional imager 10 The two dimensional image 30 gathered registrates.

Property detector 34 processes two dimensional image 30 to extract two dimensional character collection from described two dimensional image 30 36.Property detector 34 also processes 3-D view 32 to extract three-dimensional feature from described 3-D view 32 Collection 38.In the embodiment in figure 1, identical property detector 34 is applied to two dimensional image 30 He Both 3-D views 32;However, it is also contemplated that use two different property detectors, such as, wherein, One property detector carries out optimization for two dimensional image, and another property detector is for three-dimensional Image carries out optimization.If using two different property detectors by this way, detected Feature should be comparable, such as, for same type.

Property detector 34 can detect, and such as, is suitably denoted as the corner characteristics of angle point.In order to Detection corner characteristics, operates this feature detector 34 via angle detection algorithm, such as, by knowing Do not generally correspond to the high brightness gradient region at angle, and by identifying the discrete set of line intersection point, wherein Identify that the high brightness gradient region corresponding to angle is the inertia by identifying the image gradient along each direction The local maxima characteristic value of matrix is carried out.Advantageously, it practice, be generally directed to two and three dimensions figure As identical corner characteristics is detected, even if the contrast mechanism of two kinds of images 10,12 is actually by both Different (such as, x-ray is to magnetic resonance).The corner structure of angle based on derivative character detection and object The probability of high-contrast combines, it is ensured that angle detection process is substantially independent from contrast type, right Than degree level and other picture characteristics.Another advantage being used angle to detect by property detector 34 is Angle point is all discrete in both two and three dimensions.

Property detector 34 alternatively, or in addition can also detect other kinds of feature.Such as, Property detector 34 alternatively, or in addition can detect limit feature.In certain embodiments, feature Detector 34 can be via frontier inspection method of determining and calculating detection limit feature, and this realizes as follows.Thrown Line in the two dimensional image of shadow is corresponding to the throwing at the interface in the 3-D view along x-ray beam 22 orientation Shadow.By using voxel intensity gradient amplitude and direction, interface, penetrate together with from x known to perspective geometry Wire harness direction, can suitably detect these interfaces.The projection matrix using equation (2) can be by boundary Position, face maps from 3D to 2D, to form the mapping graph on limit and Angle Position.

Generally, image 30,32 is reduced to respective two-dimensional or three-dimensional feature by property detector 34 respectively Collection 36,38, it is the subset of less respective data, and is thus susceptible to space correlation and with complete Whole image 30,32 is compared and can be more effectively carried out processing from the point of view of the angle calculated.Corresponding feature The geometry of collection 36,38 holding source images 30,32.Therefore, for detecting the feature of corner characteristics Detector 34, two dimensional character collection 36 includes the set of the point in plane, and three-dimensional feature collection 38 includes Three-dimensional point " cloud ".Similarly, for limit detector, two dimensional character collection 36 includes the most coplanar The set of line, and three-dimensional feature collection 38 includes the three dimensional arrangement of line.

Projection processor 40 carries out mathematical projection according to perspective geometry 42 to three-dimensional feature collection 38, described Perspective geometry 42 is at least initially set and is made when gathering two dimensional image 12 by two dimensional imager 10 Perspective geometry.(can be from the Philips of Eindhoven, Holland for such as Allura XPer FD10 Medical Systems obtain) exemplary Interventional C-arm x-ray arrangement for perspective, this projection Geometry is suitably limited by following.Vector s from etc. center (iso-center) extend to x-ray source 20, Meanwhile, vector d from etc. center extend to the center of detector 26.Article two, normal n1And n2Limit inspection Survey device plane, and known for each projection.Therefore the angulation giving any specific C-arm is permissible Arbitrary 3 D point P is mapped (that is, projection) two-dimensional points p to detector 26.By these vectors Expand to cartesian coordinate obtain:

s = [ s x , s y , s z ] T d = [ d x , d y , d z ] T n 1 = [ n 1 x , n 1 y , n 1 z ] T n 2 = [ n 2 x , n 2 y , n 2 z ] T P = [ X , Y , Z ] T p = [ u , v , μ ] T - - - ( 1 )

The matrix-vector equation limiting perspective geometry 42 can be written as:

u v μ = n 1 x n 2 x s x - X n 1 y n 2 y s y - Y n 1 z n 2 z s z - Z - 1 s x - d x s y - d y s z - d z - - - ( 2 )

Used selected perspective geometry 42 that equation (2) is applied to three-dimensional feature collection by projection processor 40 The each three-dimensional angle point P (in the case of angle detector) of 38 is to generate the two-dimensional projection of corresponding point p Feature set 44.

Registration processor 46 is by two-dimensional projection's feature set 44 and the two dimensional character extracted from two dimensional image 30 Collection 36 registrates.If registration needs to regulate projective parameter, then, this projection process is optionally Iteration, use the projective parameter iteratively regulated by this registration to three-dimensional according to iterative cycles 48 Feature set 38 carries out re-projection.Registration processor 46 is output as one or more registration parameter 50 Set.Registration may need to regulate each parameter, such as projective parameter (such as, angulation, amplification, Source/detector location parameter, the parameter of pincushion distortion of quantifying, etc.), rigid translation or rotation, non-rigid Translation or rotation, etc..Registration may need the projection to the perspective geometry operated for mathematical projection to join Number selects or refines.But, calculating registration parameter 50 based on complete image 30,32 is to calculate Intensive, especially for iteration registration technique.

Effectively adjusted about less 36,38 pairs of registration parameters 50 of feature set by regulation processor 46 Joint (includes alternatively via circulation 48 and the iteration re-projection of projection processor 40).Show as one Example, using the two dimensional character collection 36 of extraction from two dimensional image 30 as reference, and to perspective geometry 42 Projective parameter and/or the spatial parameter of two-dimensional projection's feature set 44 be adjusted.

If accurately or exactly known projection geometry 42 (such as based on two dimensional imager 10 school Accurate), and only perform Rigid Registration, then, optimization space only includes six parameters, such as, phase For three of spatial parameter rotations and three translations of two-dimensional projection's feature set, and registration processor 46 can use simplex method (downhill simplex method) for the number of these six parameters Value regulation and optimization.It is suitably regulated about similarity measurement or optimization, described similarity Measure and calculated (such as) for each angle point in two dimensional character collection 36 with in two-dimensional projection's feature set The quadratic sum of the distance between corresponding projection angle point in 44.

If not enough precisely and accurately knowing perspective geometry 42, then, registration processor 46 Alternatively the projective parameter of perspective geometry 42 is adjusted as a part for registration.Such as, projection Processor 40 is applied to the three-dimensional feature collection 38 with multiple different projection angulation, described projection angulation From the nominal angulation deviation selected amount for gathering two dimensional image 30.This registration application is thrown to by mathematics Two-dimensional projection's feature set 44 that shadow generates in each selected angulation, selects " coupling " to registrate, and with Described mate most perspective geometry 42 that corresponding angulation is selected as being adjusted be adjusted angulation. This brute force method is feasible, this is because the non-registered by only registration features (such as, angle point) The dimension simplification that whole image is provided provides quickly process.Alternatively or additionally, angulation or Other projective parameters can be included as being used method of least square or other optimization skills by registration processor 46 The parameter that art is optimized.Optional changing is schematically shown in FIG by iterative cycles arrow 48 Generation or registrate completely, in registration, registration processor 46 is applied to by projection processor 40 Different two generated with different mathematical projection angulations (or with its dependent variable in perspective geometry 42) Dimension projection properties collection 44.

In most instances, it is desirable to the perspective geometry of the relatively high known two dimensional image of degree of accuracy 30, Such as, calibrated projection based on the two dimensional imager 10 used in gathering two dimensional image 30 is several What.In such embodiments, it generally is suitable for supposing each feature in two dimensional character collection 36 and two dimension The identical angle point corresponding to object closest to both features in projection properties collection 44.In such situation Under, registration processor 46 similarity measurement optimized is suitable for being calculated as the quadratic sum of distance, its In, each distance is the closest spy of the feature at two dimensional character collection 36 and two-dimensional projection's feature set 44 Distance between levying.

However, it is contemplated that, in some cases, by with the most limited degree of accuracy and/or accurately Property known two dimensional image 30 perspective geometry, it is assumed that each feature in two dimensional character collection 36 and In two-dimensional projection's feature set 44 to both correspond to the identical angle point of object closest to feature be irrational. In this case, it is contemplated that registration processor 46 application combination algorithm is with by two-dimensional projection's feature set 44 Feature is associated with the character pair of the two dimensional character collection 36 extracted from two dimensional image 30.

As further illustrated in Figure 1, in second time step of registration, by image projector and tune Joint device 52 uses the registration parameter 50 being adjusted corresponding 2D image 30 and 3D rendering 32 to be carried out Registration.About two dimensional character collection 36 and two-dimensional projection's feature set 44, registration parameter 50 is adjusted; But, these features 36,44 represent corresponding two dimensional image 30 and the spatial character of 3-D view 32, And thus registration parameter 50 can be applicable to the second registration performed by image projector and actuator 52 During, image projector and actuator 52 projection three-dimensional image 32 according to perspective geometry 42 with join The image that the regulation of quasi-parameter 50 is projected is two-dimentional through projecting and the image 54 of registration to generate, itself and two dimension Image 30 is registration.

The projection performed by image projector and actuator 52 can substantially use any kind of 3D To 2D projecting method, such as digital reconstruction radiography (DRR) method, it is by projection plane The line that this (virtual) that each point is set as along connection (virtual) source and projection plane puts is mathematically The line integral calculated.It is also contemplated that other projecting method, such as each by projection plane Point is set as along the maximum being somebody's turn to do the line that (virtual) puts connected in (virtual) source and projection plane Maximum intensity projection (MIP).

Suitably the projection of two dimension is gathered with by two dimensional imager 10 as 54 by image processor 56 Two dimensional image 30 compare or combine, described image processor 56 such as image combiner or melt Close processor, Image Comparator, image display (such as there is the user interface of pictorial displays) Deng.Such as, 2D image 30 and 2D can pass through image co-registration skill through the image 54 of projection and registration Art carries out merging and showing merged image, or two width images 30,54 can be displayed side by side or Person shows with vertically arranged form.In the later case, it is contemplated that have at the figure shown by two width Mouse that same spatial location in 30,54 shows or the locking pointer of other instruction equipment, thus Characteristic of correspondence can be easily positioned in two width images 30,54 by radiologist.

Although in many cases it is desirable to described registration process provides accurately and result accurately, But in some cases, obtained image registration may be the most of great satisfaction.In some feelings In shape, the two dimensional image 30,54 being registered is compared, if alignment is the most previously selected Threshold value or can not meet radiologist, then experience another image registration program, such as by image procossing The image registration program based on intensity that device 56 or another parts perform.

An intended application for the multi-mode imaging system of Fig. 1 is raw for applying at interventional cardiac electricity In field of science.Interventional cardiac electrophysiology program generally x-ray have an X-rayed under perform with relative to Conduit or other intervening equipments are carried out visually by the highly attenuating structure of the such as patient of thoracic cavity spinal column and rib Change.But, these projections do not comprise the information about soft tissue anatomical structures.Hence it is advantageous to will Volume cardiac spiral that these two dimension x-ray fluoroscopy images and operation consent gather or many slice CT or MRI Image merges.The system of Fig. 1 provides by Image Comparator or combiner 56 (in this enforcement In example, it is image co-registration processor) by two dimension x-ray fluoroscopy images (corresponding to two dimensional image 30) With three dimensional CT or the rapid fusion of MRI image (corresponding to 3-D view 32), thus provide existence In intrathoracic conduit about the Real time visible of cardiovascular anatomical structure visible in volumetric data sets 32 Change.

With reference to Fig. 2, in another intended application, two imagers of multi-mode imaging system can generate 3-D view also is used for generating 3-D view.In other words, in the embodiment of fig. 2, two dimensional imager 10 by the second three-dimensional imager 60 replacement, compared with three-dimensional imager 12 its can be identical molds state or Different modalities.Projection processor 40 is applied to the 3-D view generated by the second three-dimensional imager 60 To generate the two dimensional image 30 with perspective geometry 42, perspective geometry 42 is in this embodiment for being used for The selected geometry of the mathematical projection of the 3-D view generated by the second three-dimensional imager 60.After this, The parts of the multi-mode imaging system of Fig. 2 and process be similar to Fig. 1 multi-modality imaging device parts and Process.The method of Fig. 2 can provide by different three-dimensional imagers (such as by CT and MRI imager Or by two kinds of different CT imagers) the digital reconstruction radiography (DRR) that generates or other projections Rapid registering.Owing to performing registration with 2D, and in addition only about little characteristic data set 36, 44 perform registration, so very fast process is possible.

The person skilled in the art will easily understand that process of image registration disclosed herein can be by by storage Digital display circuit performs digital storage media or the medium realization of the instruction to realize disclosed method.Such as, Digital storage media or medium can be disk, CD, tape, FLASH memory or other electrostatic Memorizer, random access memory (RAM), read only memory (ROM), Internet server etc. Or be the combination of these media, and the instruction stored can such as computer, digital network, Perform in the digital display circuit of Internet server etc..

Have been described with preferred embodiment.Other people read and understand described in detail above after can think To modifications and changes.It is contemplated that be read as including all such modifications and changes, as long as they Fall in the range of claims or its equivalent substitute.

Claims (14)

1. a process of image registration, including:
Two dimensional character collection (36) is extracted from two dimensional image (30);
Three-dimensional feature collection (38) is extracted from 3-D view (32);
Wherein, the operation of described extraction includes applying angle detection algorithm to extract the feature including angle point, And wherein, described angle detection algorithm is by identifying the inertial matrix of the image gradient along each direction Local maxima characteristic value is to identify high brightness gradient region, and identifies the discrete set of line intersection point, carries Take described angle point;
Use perspective geometry that described three-dimensional feature collection is mathematically projected into two-dimensional projection's feature set (44);And
By about similarity measurement regulate or optimization registration parameter and by described two-dimensional projection feature set Registrating with the described two dimensional character collection extracted from described two dimensional image, described similarity measurement is counted Calculate each angle point and described two-dimensional projection that the described two dimensional character for extracting is concentrated from described two dimensional image The quadratic sum of the distance between corresponding projection angle point in feature set.
Process of image registration the most according to claim 1, wherein, the operation of described extraction is also wrapped Include:
Application edge inspection algorithms extracts the feature including line segment.
3. according to the process of image registration according to any one of claim 1-2, wherein, described registration Including:
Regulation selects from the group including at least three rotation parameter and at least three translation parameters, institute State the space of at least one ginseng in two dimensional character collection (36) and described two-dimensional projection feature set (44) Number, the spatial parameter being adjusted is used for second time and registrates.
4. according to the process of image registration according to any one of claim 1-2, wherein, described registration Including:
Regulate from include into angular dimensions, magnification parameters, source location parameter, detector positional parameter with And one or more parameters of the described perspective geometry (42) selected in the group of distortion parameter, described throwing The parameter being adjusted of shadow geometry is used for the second time of the mathematical projection for described 3-D view and joins Accurate.
5. according to the process of image registration according to any one of claim 1-4, wherein, described registration Including:
Optimization characterizes the feature of described two dimensional character collection (36) and described two-dimensional projection statistically The distance figure of merit of the distance between the immediate character pair of feature set (44).
6. according to the process of image registration according to any one of claim 1-5, wherein, described projection Geometry (42) is the throwing of two dimensional imager (10) during gathering described two dimensional image (30) Shadow geometry.
7. according to the process of image registration according to any one of claim 1-6, wherein, described registration Application combination algorithm is with by the feature of described two dimensional character collection (36) and described two-dimensional projection feature set (44) Character pair be associated.
8., according to the process of image registration according to any one of claim 1-7, also include:
Use and include that the two dimensional imager (10) of x-ray arrangement for perspective gathers described two dimensional image (30); And
Use includes MR imager (MRI) or the three of computer tomography (CT) imager Dimension imager (12) gathers described 3-D view (32).
9., according to the process of image registration according to any one of claim 1-7, also include:
Gather the second 3-D view;And
Formed by using described perspective geometry (42) mathematically to project described second 3-D view Described two dimensional image (30).
10., according to the process of image registration according to any one of claim 1-9, also include:
After second time registrates using described registration parameter to carry out, show described two dimensional image and described The combination of the mathematical projection of 3-D view, merge or compare.
11. 1 kinds of image registration devices, including:
For extracting the module of two dimensional character collection from two dimensional image;
For extracting the module of three-dimensional feature collection from 3-D view;
Wherein, the described module for extracting includes including angle point for applying angle detection algorithm to extract The module of feature, and wherein, described angle detection algorithm is by identifying the image ladder along each direction The local maxima characteristic value of the inertial matrix of degree is to identify high brightness gradient region, and identifies line intersection point Discrete set, extract described angle point;
For using perspective geometry that described three-dimensional feature collection is mathematically projected into two-dimensional projection's feature set Module;And
For by described two-dimensional projection is special about similarity measurement regulation or optimization registration parameter Collection and the described two dimensional character collection extracted from described two dimensional image carry out the module registrated, described similar Property measure each angle point and the institute that the described two dimensional character being calculated as extracting from described two dimensional image is concentrated State the quadratic sum of distance between the corresponding projection angle point in two-dimensional projection's feature set.
12. 1 kinds of image registration devices, including:
Including the property detector (34) of angle detector, it is configured to extract from two dimensional image (30) Two dimensional character collection (36) and from 3-D view (32) extract three-dimensional feature collection (38), wherein, institute State angle detector to be configured to apply angle detection algorithm and extract the feature including angle point, and wherein, Described angle detection algorithm is by identifying the local maxima of the inertial matrix of the image gradient along each direction originally Value indicative is to identify high brightness gradient region, and identifies the discrete set of line intersection point, extracts described angle point;
Projection processor (40), it is configured to described three-dimensional feature collection is projected into two-dimensional projection's feature Collection;And
Registration processor (46,52), it is configured to
(i) about similarity measurement regulation parameter with by described two-dimensional projection feature set with from described The described two dimensional character collection that two dimensional image extracts registrates, and described similarity measurement is calculated as Each angle point that the described two dimensional character extracted from described two dimensional image is concentrated and described two-dimensional projection The quadratic sum of the distance between corresponding projection angle point in feature set, and
(ii) use the parameter that is adjusted to make by described two dimensional image with by described projection processor Registrate with the described 3-D view of perspective geometry projection.
13. image registration devices according to claim 12, wherein, described property detector (34) Also include edge detector.
14. according to the image registration device according to any one of claim 12-13, wherein, described in join Quasi-processor (46,52) is configured to regulate described perspective geometry (42) with by described two dimensional character Collection (36) and described two-dimensional projection feature set carry out registrating or contribute to described registration.
CN201610195981.9A 2007-12-18 2008-12-01 Features-based 2d/3d image registration CN105844586A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US1448807P true 2007-12-18 2007-12-18
US61/014,488 2007-12-18
CN2008801211036A CN101903908A (en) 2007-12-18 2008-12-01 Features-based 2D/3D image registration

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN2008801211036A Division CN101903908A (en) 2007-12-18 2008-12-01 Features-based 2D/3D image registration

Publications (1)

Publication Number Publication Date
CN105844586A true CN105844586A (en) 2016-08-10

Family

ID=40548797

Family Applications (2)

Application Number Title Priority Date Filing Date
CN2008801211036A CN101903908A (en) 2007-12-18 2008-12-01 Features-based 2D/3D image registration
CN201610195981.9A CN105844586A (en) 2007-12-18 2008-12-01 Features-based 2d/3d image registration

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN2008801211036A CN101903908A (en) 2007-12-18 2008-12-01 Features-based 2D/3D image registration

Country Status (6)

Country Link
US (1) US9135706B2 (en)
EP (1) EP2225723B1 (en)
JP (1) JP5906015B2 (en)
CN (2) CN101903908A (en)
RU (1) RU2568635C2 (en)
WO (1) WO2009081297A2 (en)

Families Citing this family (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5304172B2 (en) * 2007-12-04 2013-10-02 株式会社リコー File management apparatus, file management method, and file management program
WO2010108146A2 (en) 2009-03-20 2010-09-23 Orthoscan Incorporated Moveable imaging apparatus
WO2010129363A2 (en) * 2009-04-28 2010-11-11 The Regents Of The University Of California Markerless geometric registration of multiple projectors on extruded surfaces using an uncalibrated camera
DE102009056721A1 (en) * 2009-12-02 2011-06-09 Siemens Aktiengesellschaft Method for supporting the selection of a recording configuration, computed tomography device and data carrier
US8942457B2 (en) * 2010-01-12 2015-01-27 Koninklijke Philips N.V. Navigating an interventional device
RU2569532C2 (en) 2010-03-31 2015-11-27 Конинклейке Филипс Электроникс Н.В. Automatic identification of anatomical part
RU2012148549A (en) * 2010-04-15 2014-05-20 Конинклейке Филипс Электроникс Н.В. Combining images using the instrument for combining images with tubular structures
DE102010020350B4 (en) * 2010-05-12 2017-02-23 Siemens Healthcare Gmbh Method for positioning the focus of a gradient field and treatment device
US8723886B2 (en) * 2010-09-08 2014-05-13 Navteq B.V. Generating a multi-layered geographic image and the use thereof
WO2012082799A1 (en) 2010-12-13 2012-06-21 Orthoscan, Inc. Mobile fluoroscopic imaging system
RU2586448C2 (en) * 2011-03-04 2016-06-10 Конинклейке Филипс Н.В. Combination of two-dimensional/three-dimensional images
JP5950619B2 (en) * 2011-04-06 2016-07-13 キヤノン株式会社 Information processing device
CN102222331B (en) * 2011-05-16 2013-09-25 付东山 Dual-flat panel-based two-dimensional to three-dimensional medical image registering method and system
CN102222330B (en) * 2011-05-16 2013-04-10 付东山 Two-dimensional and three-dimensional medical image registration method and system
GB201203883D0 (en) * 2012-03-05 2012-04-18 King S College London Method and system to assist 2D-3D image registration
US9972120B2 (en) 2012-03-22 2018-05-15 University Of Notre Dame Du Lac Systems and methods for geometrically mapping two-dimensional images to three-dimensional surfaces
WO2013142819A1 (en) * 2012-03-22 2013-09-26 University Of Notre Dame Du Lac Systems and methods for geometrically mapping two-dimensional images to three-dimensional surfaces
DE102012111385B4 (en) * 2012-11-23 2018-05-03 Diers Engineering Gmbh Determining the spatial position and orientation of the vertebral bodies of the spine
CN103871036B (en) * 2012-12-12 2017-11-28 上海联影医疗科技有限公司 Rapid registering and joining method for three dimensional digital subtraction angiography image
JP6145870B2 (en) * 2013-05-24 2017-06-14 富士フイルム株式会社 Image display apparatus and method, and program
JP6182045B2 (en) * 2013-10-11 2017-08-16 キヤノン株式会社 Image processing apparatus and method
EP3175791A1 (en) * 2013-11-04 2017-06-07 Surgivisio Method for reconstructing a 3d image from 2d x-ray images
CN103948361B (en) * 2014-04-14 2016-10-05 中国人民解放军总医院 Endoscope's positioning and tracing method of no marks point and system
CN103942824B (en) * 2014-05-15 2017-01-11 厦门大学 Linear feature extracting method for three-dimensional point cloud
EP3161785B1 (en) * 2014-09-22 2019-08-28 Shanghai United Imaging Healthcare Co., Ltd. System and method for image composition
CN104268846B (en) * 2014-09-22 2017-08-22 上海联影医疗科技有限公司 Image split-joint method and device
US9974525B2 (en) 2014-10-31 2018-05-22 Covidien Lp Computed tomography enhanced fluoroscopic system, device, and method of utilizing the same
DE102015208929B3 (en) * 2015-05-13 2016-06-09 Friedrich-Alexander-Universität Erlangen-Nürnberg Method for 2D-3D registration, computing device and computer program
US10235606B2 (en) * 2015-07-22 2019-03-19 Siemens Healthcare Gmbh Method and system for convolutional neural network regression based 2D/3D image registration
CA2937825C (en) * 2015-08-06 2020-05-12 Covidien Lp System and method for local three dimensional volume reconstruction using a standard fluoroscope
US20190005737A1 (en) * 2015-12-30 2019-01-03 Koninklijke Philips N.V. Synthetic representation of a vascular structure
CN105913461A (en) * 2016-03-31 2016-08-31 中国人民解放军第四军医大学 Rat brain function magnetic resonance imaging standard space map template construction method
CN106204511B (en) * 2016-07-15 2018-10-26 西安交通大学第一附属医院 A kind of three-dimensional fusion method of two dimensional image and CT, MR image
JP6436141B2 (en) * 2016-09-20 2018-12-12 オムロン株式会社 X-ray inspection apparatus and control method thereof
JP2019013724A (en) 2017-07-03 2019-01-31 株式会社リコー Diagnosis support system, diagnosis support method, and diagnosis support program
EP3429206A1 (en) * 2017-07-13 2019-01-16 Thomson Licensing Method and device for encoding a point cloud
JP6461257B2 (en) * 2017-07-24 2019-01-30 キヤノン株式会社 Image processing apparatus and method
US10460458B1 (en) * 2017-09-14 2019-10-29 United States Of America As Represented By The Secretary Of The Air Force Method for registration of partially-overlapped aerial imagery using a reduced search space methodology with hybrid similarity measures
CN111050647A (en) * 2017-10-03 2020-04-21 株式会社岛津制作所 Radiographic apparatus
CN108182699B (en) * 2017-12-28 2019-11-08 北京天睿空间科技股份有限公司 Three-dimensional registration method based on two dimensional image local deformation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7010080B2 (en) * 2003-05-20 2006-03-07 Siemens Aktiengesellschaft Method for marker-free automatic fusion of 2-D fluoroscopic C-arm images with preoperative 3D images using an intraoperatively obtained 3D data record
CN101084840A (en) * 2006-06-08 2007-12-12 西门子公司 Method for registering functional mr image data using radioscopy

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6488694A (en) 1987-09-29 1989-04-03 Toshiba Corp Three-dimensional reprojector
JPS6488792A (en) 1987-09-30 1989-04-03 Toshiba Corp Three-dimensional reprojecting device
RU2071725C1 (en) * 1993-06-22 1997-01-20 Научно-производственная коммерческая фирма "Ренси Лтд." Computer-based tomograph
JP4104054B2 (en) * 2001-08-27 2008-06-18 富士フイルム株式会社 Image alignment apparatus and image processing apparatus
DE10210646A1 (en) * 2002-03-11 2003-10-09 Siemens Ag Method for displaying a medical instrument brought into an examination area of a patient
US7409108B2 (en) * 2003-09-22 2008-08-05 Siemens Medical Solutions Usa, Inc. Method and system for hybrid rigid registration of 2D/3D medical images
JP4512824B2 (en) 2005-02-04 2010-07-28 国立大学法人 千葉大学 Image processing apparatus and program used therefor
WO2006084385A1 (en) * 2005-02-11 2006-08-17 Macdonald Dettwiler & Associates Inc. 3d imaging system
JP4480083B2 (en) * 2005-02-23 2010-06-16 アイシン精機株式会社 Object recognition device
CN101138010B (en) * 2005-03-10 2011-05-18 皇家飞利浦电子股份有限公司 Image processing system and method for registration of two-dimensional with three-dimensional volume data during interventional procedures
DE102005023167B4 (en) * 2005-05-19 2008-01-03 Siemens Ag Method and device for registering 2D projection images relative to a 3D image data set
DE102005023194A1 (en) * 2005-05-19 2006-11-23 Siemens Ag Method for expanding the display area of 2D image recordings of an object area
JP2007014435A (en) * 2005-07-06 2007-01-25 Fujifilm Holdings Corp Image processing device, method and program
DE102006024000A1 (en) * 2006-05-22 2007-11-29 Siemens Ag Non-rigid object e.g. heart, visualizing method, involves generating series of two-dimensional image data sets or three-dimensional image data sets and individually registering each image data set of series
US7925049B2 (en) * 2006-08-15 2011-04-12 Sri International Stereo-based visual odometry method and system
FR2907569B1 (en) * 2006-10-24 2009-05-29 Jean Marc Robin Method and device for virtual simulation of a video image sequence
US8401276B1 (en) * 2008-05-20 2013-03-19 University Of Southern California 3-D reconstruction and registration

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7010080B2 (en) * 2003-05-20 2006-03-07 Siemens Aktiengesellschaft Method for marker-free automatic fusion of 2-D fluoroscopic C-arm images with preoperative 3D images using an intraoperatively obtained 3D data record
CN101084840A (en) * 2006-06-08 2007-12-12 西门子公司 Method for registering functional mr image data using radioscopy

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HAREL LIVYATAN等: "Gradient-Based 2-D/3-D Rigid Registration of Fluoroscopic X-Ray to CT", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 *

Also Published As

Publication number Publication date
CN101903908A (en) 2010-12-01
RU2010129960A (en) 2012-01-27
EP2225723A2 (en) 2010-09-08
JP2011508620A (en) 2011-03-17
JP5906015B2 (en) 2016-04-20
US20100266220A1 (en) 2010-10-21
US9135706B2 (en) 2015-09-15
RU2568635C2 (en) 2015-11-20
EP2225723B1 (en) 2019-02-20
WO2009081297A3 (en) 2009-08-13
WO2009081297A2 (en) 2009-07-02

Similar Documents

Publication Publication Date Title
Aide et al. EANM/EARL harmonization strategies in PET quantification: from daily practice to multicentre oncological studies
US8625869B2 (en) Visualization of medical image data with localized enhancement
Benameur et al. 3D/2D registration and segmentation of scoliotic vertebrae using statistical models
Demirkaya et al. Image processing with MATLAB: applications in medicine and biology
Maurer et al. A review of medical image registration
Kim et al. Mutual information for automated unwarping of rat brain autoradiographs
Gilhuijs et al. Automatic on‐line inspection of patient setup in radiation therapy using digital portal images
Vandemeulebroucke et al. The POPI-model, a point-validated pixel-based breathing thorax model
Schnabel et al. Validation of nonrigid image registration using finite-element methods: application to breast MR images
EP2174294B1 (en) Motion correction in nuclear imaging
US6990368B2 (en) Method and apparatus for virtual digital subtraction angiography
Van de Kraats et al. Standardized evaluation methodology for 2-D-3-D registration
EP0389333B1 (en) Process for the acquisition of radiological data and the reconstruction of structures of an irradiated body
US7142633B2 (en) Enhanced X-ray imaging system and method
US6359960B1 (en) Method for identifying and locating markers in a 3D volume data set
US6487432B2 (en) Method and system for selecting and displaying medical image data
JP4965433B2 (en) Cone beam CT apparatus using truncated projection and pre-acquired 3D CT image
Van den Elsen et al. Medical image matching-a review with classification
JP3022773B2 (en) How to align nuclear medicine images
Mattes et al. PET-CT image registration in the chest using free-form deformations
JP4576228B2 (en) Non-rigid image registration based on physiological model
Bai et al. Motion correction and attenuation correction for respiratory gated PET images
US7106891B2 (en) System and method for determining convergence of image set registration
CN102727258B (en) Image processing apparatus, ultrasonic photographing system, and image processing method
JP5734664B2 (en) Image Restoration Method Using Dilution Constraint Correction

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20160810

WD01 Invention patent application deemed withdrawn after publication