CN109564693A - Three-dimensional data integrating apparatus and method - Google Patents

Three-dimensional data integrating apparatus and method Download PDF

Info

Publication number
CN109564693A
CN109564693A CN201780049378.2A CN201780049378A CN109564693A CN 109564693 A CN109564693 A CN 109564693A CN 201780049378 A CN201780049378 A CN 201780049378A CN 109564693 A CN109564693 A CN 109564693A
Authority
CN
China
Prior art keywords
data
dimensional
curvature
point
initial position
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201780049378.2A
Other languages
Chinese (zh)
Other versions
CN109564693B (en
Inventor
权基勋
李承炫
金珉永
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koh Young Technology Inc
Industry Academic Cooperation Foundation of KNU
Original Assignee
Koh Young Technology Inc
Industry Academic Cooperation Foundation of KNU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koh Young Technology Inc, Industry Academic Cooperation Foundation of KNU filed Critical Koh Young Technology Inc
Publication of CN109564693A publication Critical patent/CN109564693A/en
Application granted granted Critical
Publication of CN109564693B publication Critical patent/CN109564693B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G06T7/0014—Biomedical image inspection using an image reference approach
    • G06T3/14—
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/0035—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G06T3/06—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Abstract

A kind of method and device of integrating three-dimensional data is disclosed.First three-dimensional data on the threedimensional model surface for representing target is carried out two-dimensional transform and generates the first 2-D data by three-dimensional data integration method, at least part of second three-dimensional data for the three-dimensional surface for representing target is subjected to two-dimensional transform and generates the second 2-D data, second 2-D data is matched with the first 2-D data, the second matching area on the first matching area and the second 2-D data on the first 2-D data is determined respectively, initial position is set by and multiple points of corresponding first three-dimensional data of the first matching area and multiple points of the second three-dimensional data corresponding with the second matching area, utilize initial position, integrate the first three-dimensional data and the second three-dimensional data.

Description

Three-dimensional data integrating apparatus and method
Technical field
The present invention relates to three-dimensional data integrating apparatus and methods.
Background technique
In medical field, for the purpose of checking and treat, it can be directed to the object of such as patient affected part, by means of mutually not phase With device and obtain multiple three-dimensional datas.These three-dimensional datas, which respectively can have, to be determined by the device of generation corresponding data Intrinsic coordinate system.For example, modeled using affected part of the MRI device to patient and generate MRI data, filled using three-dimensional measurement Setting the 3 d measurement data for shooting corresponding affected part and generating has coordinate system different each other.
Multiple three-dimensional datas with coordinate system different each other, can in the case where together for inspection or therapeutic purposes To combine these three-dimensional datas in a suitable approach.For example, when doctor is to patients surgery, in order to accurately grasp patient affected part and External structure and shape around the affected part can utilize together the MRI data generated in advance to affected part modeling, shooting should The 3 d measurement data generated outside affected part.In order to combine have the data of different coordinate system and use each other, need by with The relevant coordinate system transformation of these data at so that be arranged in a coordinate system.For example, can be by the coordinate system transformation of MRI data At the coordinate system of 3 d measurement data, or by the coordinate system transformation of 3 d measurement data at the coordinate system of MRI data, or by MRI The coordinate system transformation of data and 3 d measurement data is at third coordinate system.
On the other hand, in order to have the integration between the data of different coordinate system each other, the integration algorithm of multiplicity is utilized.According to One example of integration algorithm, the relative position between data can determine have to the raised direction change of similar degree between data Relative position between the data of maximum similar degree.This integration algorithm can specify initial position to integrate in data. At this point, the similar degree between the data of integration is only partly first when mistakenly specifying initial position in the data in quasi- integration Beginning position periphery reaches maximum, and possibly can not reach maximum in the whole region of data.In other words, in the data that will be integrated In, when the specified mistake of initial position, it will do it the integration of inaccuracy between data.In addition, in the data of so quasi- integration, When the specified mistake of initial position, there are problems that realizing that optimum integration needs the plenty of time.
Summary of the invention
The technical issues of solution
The present invention provides a kind of three-dimensional data integration method and device, using the 2-D data converted from three-dimensional data as base Plinth, initial position of the setting for integration in corresponding three-dimensional data, so as to improve the accuracy and efficiency of Data Integration Property.
Technical solution
One aspect of the present invention provides a kind of method of integrating three-dimensional data.The method of exemplary embodiment include: by The step of representing the first three-dimensional data progress two-dimensional transform on the threedimensional model surface of target and generating the first 2-D data；It will generation At least part of second three-dimensional data of entry target three-dimensional surface carries out two-dimensional transform and generates the step of the second 2-D data Suddenly；Second 2-D data is matched with the first 2-D data and determines the first matching area on the first 2-D data and respectively The step of the second matching area on two 2-D datas；By multiple points of the first three-dimensional data corresponding with the first matching area and The step of multiple points of the second three-dimensional data corresponding with the second matching area are set as initial position；And initial position is utilized, The step of integrating the first three-dimensional data and the second three-dimensional data.
In addition, accoding to exemplary embodiment, the step of determining the first matching area and the second matching area respectively packet It includes: partial differential being carried out to the first 2-D data and the second 2-D data respectively and generates the first two-dimensional curvature data and the second two dimension The step of curvature data；And the first two-dimensional curvature data and the second two-dimensional curvature data are matched, the first two-dimensional curvature is determined respectively The step of the second matching area on the first matching area and the second two-dimensional curvature data in data.
Invention effect
At least one embodiment according to the present invention replaces real to specify the initial position for three-dimensional data integration It applies to integrate the direct matching between the three-dimensional data of object, but implements between the 2-D data that three-dimensional data converts Matching.The data in such matched region are re-started into three-dimension varying, initial position of the setting for integration.As a result, can be with It realizes appropriate initial position setting, the effect for improving the accuracy and efficiency of the integration between three-dimensional data can be obtained.
In addition, at least one embodiment according to the present invention, real in terms of the 2-D data that matching is converted from three-dimensional data It applies and respective two-dimensional data is subjected to partial differential respectively and the matching between the curvature data that generates.As a result, initial position obtains more Accurately it is arranged, it is hereby achieved that the effect of the accuracy and efficiency of the integration between further increasing three-dimensional data.
Detailed description of the invention
Fig. 1 is the figure for being shown schematically as the surface integration system of one embodiment of the invention.
Fig. 2 is the block diagram constituted in detail for illustrating the three-dimensional data integrating apparatus of one embodiment of the invention.
Fig. 3 is to show the model surface two-dimemsional number from model surface data through two-dimensional transform according to an embodiment of the present invention According to figure.
Fig. 4 and Fig. 5 is shown according to an embodiment of the present invention from measurement measurement surface of the surface data through two-dimensional transform The figure of 2-D data.
Fig. 6 and Fig. 7 be show according to an embodiment of the present invention with the matched measurement surface two of model surface 2-D data The figure of dimension data.
Fig. 8 to Figure 10 be show according to an embodiment of the present invention to model surface two-dimensional curvature data carry out partial differential and The figure of the measurement surface two-dimensional curvature data of generation.
Figure 11 and Figure 12 is to show the model surface data and survey that are provided with initial position according to an embodiment of the present invention The figure of scale face data.
Figure 13 is to show according to embodiments of the present invention and the precedence diagram of the method for integrating three-dimensional data.
Specific embodiment
The embodiment of the present invention is to illustrate for the purpose of illustrating the invention.The embodiment of the present invention can be with multiplicity Form implement, shall not be construed as the present invention be defined in the embodiment of following prompt or these embodiments illustrated.
Term used in this specification " portion ", it is meant that software, such as FPGA (field-programmable gate Array, field programmable gate array), ASIC (application specific integrated circuit, it is dedicated integrated Circuit) hardware constituent element.But " portion " is not limited to hardware and software." portion " both may be constructed to be located at addressable Storage medium in, also may be constructed to make one or more than it processor operation.Therefore, as an example, " portion " is wrapped Include software sharing element, the software sharing element of object-oriented, such as constituent element of class constituent element and task constituent element, Processor, function, attribute, program, subprogram, the segment of program code, driver, firmware, microcode, circuit, data, data Library, data structure, workbench, array and parameter.The function of providing in constituent element and " portion ", can constituting by more smallest number Element and " portion " combine, or are further separated into additional constituent element and " portion ".
As long as not defining differently, all technical terms and scientific term used in this specification have belonging to the present invention The meaning that those skilled in the art are commonly understood by.All terms used in this specification are for more clearly illustrating this The purpose of invention and select, be not intended to limit the scope of the invention and select.
As long as not referring to differently, the expression of the singular type described in present specification also includes complex number type table together It reaches, this is applied equally to the expression for the singular type recorded in claim.
The expression of " first ", " second " etc. used in various embodiment of the invention, is intended merely to mutually distinguish multiple Constituent element and use, and the sequence or different degree of non-limiting corresponding constituent element.
Such as " including " and " having " expression used in this specification, if in the sentence comprising accordingly expressing or It is not referred to differently especially in article, is then interpreted as the opening term (open- for a possibility that including including other embodiments ended terms)。
In the present specification, the expression of " based on~", for describing to determining described in the sentence comprising accordingly expressing The more than one factor that is fixed or judge behavior or movement is had an impact, the expression be not excluded for decision or judge behavior or Act the addition sex factor having an impact.
In the present specification, when referring to certain constituent element " being connected to " or " being connected in " another constituent element, Ying Li Solution is either certain described constituent element is directly connected in or is connected in another constituent element, it will be also be appreciated that in institute It states and there are new other constituent elements between certain constituent element and another constituent element.
With reference to the accompanying drawings, the embodiment that the present invention will be described in detail.For the identical constituent element on attached drawing, use is identical Reference marks, omit repeated explanation to identical constituent element.
Fig. 1 is the figure for being shown schematically as the surface integration system 100 of one embodiment of the invention.As shown in Figure 1, surface is whole Collaboration system 100 includes three-dimensional data integrating apparatus 120, spatial digitizer 140 and optical tracker 160.
In the present embodiment, three-dimensional data integrating apparatus 120 can store and show the mesh to such as affected part of patient 20 Mark the model data for being modeled and being generated.Wherein, " model data " may mean that the 3D shape of target or structure into Row modeling, three dimensions of the target of generation by means of the model building device (being not shown on figure) of such as CT device or MRI device According to.Model data can have the intrinsic coordinate system (for example, x1y1z1 coordinate system) about model building device.Shown in Fig. 1 shows In example, model data can be such as CT device or the model building device of MRI device, with patient's 20 organism at least part for mesh Mark the CT/MRI data 10 for being modeled and being generated.At this point, CT/MRI data 10 have the intrinsic coordinate about CT/MRI device System.
Spatial digitizer 140 can use such as photogrammetry (photogrammetry), structured pattern light measurement Method (structured light 3D scanning), is stood modulation light mensuration (Modulated light3D scanning) Body Photographic technique (stereo photography), laser radar (LIDAR), ultrasonic wave TOF (Ultrasonic Time-of- Flight) or various method of laser TOF etc., the surface of target is measured in three dimensions and generates measurement surface data.Such as Fig. 1 Shown, spatial digitizer 140 can measure at least part of 20 organism surface of patient as target, generate measurement Surface data.Spatial digitizer 140 measure target, can be front with model data modeling target outer surface or its A part.For example, when model data is the data modeled to 20 head 3D shape of patient, measurement surface data can be with It is the data for measuring the outer shape of the eye in 20 head surface of patient, nose, mouth, ear etc..Measurement surface data can have pass In spatial digitizer 140 intrinsic coordinate system (for example, x2y2z2Coordinate system).Measure surface data coordinate system can both with mould The coordinate system of type data is (for example, x1y1z1Coordinate system) it is different, also with the coordinate system of optical tracker 160 (for example, x0y0z0Coordinate System) it is different.
The surface integration system 100 of one embodiment of the invention can be by the coordinate system of model data (for example, x1y1z1It sits Mark system) and the coordinate system of surface data is measured (for example, x2y2z2Coordinate system) transformation or even it is arranged in the seat of optical tracker 160 Mark system is (for example, x0y0z0Coordinate system).Three-dimensional data integrating apparatus 120 can be implemented coordinate system each other different model data with Measure the integration of surface data.For the integration of implementation model data and measurement surface data, three-dimensional data integrating apparatus 120 can To extract surface data (hereinafter referred to as " model surface data ") from model data, implementation model surface data and measurement surface number Integration between.Wherein, the coordinate system of model surface data can be with the coordinate system of model data (for example, x1y1z1Coordinate System) it is identical.In addition, optical tracker 160 can label 142 to be installed in spatial digitizer 140 for medium, surface will be measured The coordinate system of data is (for example, x2y2z2Coordinate system) coordinate system of optical tracker 160 is transformed into (for example, x0y0z0Coordinate system). As a result, model data and model surface data by means of three-dimensional data integrating apparatus 120 and with measurement surface data integration Coordinate system is (for example, x1y1z1Coordinate system) coordinate system of optical tracker 160 is also transformed into (for example, x0y0z0Coordinate system).
Three-dimensional data integrating apparatus 120 can use the integration algorithm of multiplicity, implementation model surface data and measurement surface Integration between data.For example, three-dimensional data integrating apparatus 120 can use ICP (Iterative Closest Point, repeatedly For closest approach) algorithm implementation integration.Initial bit can be arranged in order to integrate in three-dimensional data integrating apparatus 120 in three-dimensional data It sets, integration algorithm is executed based on the initial position of setting.For example, three-dimensional data integrating apparatus 120 is used as initial position, it can By by model surface data and measurement surface data on given area or both pinpoint based on, computation model surface data and survey Similar degree between scale face data makes model surface data relative to the relative position of measurement surface data, to similar degree liter High direction change.For example, three-dimensional data integrating apparatus 120 can will be in most in model surface data and measurement surface data It is short-range, it is defined as corresponding point set, calculates similar degree using the sum of the distance between corresponding point set.
Three-dimensional data integrating apparatus 120 can use in order to which the initial position utilized in integration is arranged in by model surface Data carry out the model surface 2-D data of two-dimensional transform and will measure the measurement surface two dimension of surface data progress two-dimensional transform Data.Three-dimensional data integrating apparatus 120 can be by the three-dimensional coordinate of each point of model surface data (alternatively, measurement surface data) It is mapped to the two-dimensional coordinate of each point of model surface 2-D data (alternatively, measurement surface 2-D data) and the value (example of respective point Such as, brightness value).According to one embodiment, (spherical can be unfolded by spherical shape in three-dimensional data integrating apparatus 120 Unwrapping 2-D data transformation) is executed.It is unfolded by means of spherical shape, the two-dimensional coordinate of each point of 2-D data can be from three Two angular components for tieing up corresponding point in spherical coordinate system determine.It, can be according to from three-dimensional in addition, the value of each point of 2-D data The distance of corresponding point to origin determines in spherical coordinate system.According to another embodiment, three-dimensional data integrating apparatus 120 can use (equirectangular is unfolded in such as cylindrical expansion (cylindrical unwrapping) transformation, equidistant column Unwrapping the 2-D data of various form) converts algorithm.
According to one embodiment, three-dimensional data integrating apparatus 120 can calculate model surface 2-D data and measurement surface The respective curvature of 2-D data generates model surface two-dimensional curvature data and measurement surface two-dimensional curvature data.According to a reality Example is applied, three-dimensional data integrating apparatus 120 can be corresponding from model surface 2-D data (alternatively, measurement surface 2-D data) The difference of the value of point and the value of adjacent multiple points, decision model surface two-dimensional curvature data are (alternatively, measurement surface two-dimensional curvature number According to) each point value.According to one embodiment, three-dimensional data integrating apparatus 120 can carry out partial differential to the value of 2-D data And generate two-dimensional curvature data.
Three-dimensional data integrating apparatus 120 can be with Matching Model surface 2-D data and measurement surface 2-D data, decision With region.In addition, three-dimensional data integrating apparatus 120 is in the model surface data as three-dimensional data and measures in surface data, The surface region of model surface data corresponding with the matching area of model surface 2-D data can be provided for integrating (registration) initial position, can be by measurement surface data corresponding with the measurement matching area of surface 2-D data Surface region, be provided for integration initial position.According to one embodiment, three-dimensional data integrating apparatus 120 is in order to certainly Determine matching area, Matching Model surface 2-D data can also be replaced and measures surface 2-D data, but Matching Model surface Two-dimensional curvature data and measurement surface two-dimensional curvature data.About the initial position of model surface data and measurement surface data Setting, will be illustrated in more details referring to Fig. 2 to Fig. 8.
On the other hand, optical tracker 160 can shoot the label 142 for being installed in spatial digitizer 140, generate label figure Picture.Optical tracker 160 can analyze the size and location of the label 144 occurred in tag image, judge spatial digitizer Therefore 140 position and posture may establish that 140 relative coordinate system of spatial digitizer and 160 relative coordinate system of optical tracker Between transformation relation.
The measurement surface data that spatial digitizer 140 dependent coordinate in front is fastened with the coordinate system for model data Model surface Data Integration, thus optical tracker 160 may establish that the coordinate system and 140 phase of spatial digitizer of model data Close the transformation relation between coordinate system.Optical tracker 160 can use 140 relative coordinate system of spatial digitizer and optical tracking Between 140 relative coordinate system of coordinate system and spatial digitizer of transformation relation and model data between 160 relative coordinate system of device Transformation relation, obtain the transformation relation between 160 relative coordinate system of coordinate system and optical tracker of model data, Ke Yi Model data is disposed in 160 relative coordinate system of optical tracker.
In Fig. 1, as model data and the target of measurement surface data, patient 20 is illustrated, but the present invention not limits Due to this.Target of the invention can be for example the arbitrary object with 3D shape, human or animal.
Fig. 2 is the block diagram constituted in detail for illustrating the three-dimensional data integrating apparatus 200 of one embodiment of the invention.According to more A implementation form, the three-dimensional data integrating apparatus 200 of Fig. 2 may include all technologies of the three-dimensional data integrating apparatus 120 of Fig. 1 Feature.As shown in Fig. 2, three-dimensional data integrating apparatus 200 includes communication unit 210, processing unit 220 and database 240.
Communication unit 210 can be with external device (ED), for example, (can not show on figure with the model building device for generating model data Out), the spatial digitizer 140 of Fig. 1 and optical tracker 160 are communicated.In communication unit 210, for being filled with these outsides Junior's constituent element of communication is set, can be made of a hardware device integration.
Processing unit 220 can handle information relevant to three-dimensional data integration.Processing unit 220 include surface treating part 222, First transformation component 224, the second transformation component 226, initial position setting unit 228, curvature transformation component 230 and data interfacing block 232.Separately Outside, database 240 can store data relevant to three-dimensional data integration.Database 240 may include model data DB242, Measure surface data DB244, model surface data DB246, model surface 2-D data DB248, measurement surface 2-D data DB250, model surface two-dimensional curvature data DB252 and measurement surface two-dimensional curvature data DB254.In Fig. 2, database is constituted 240 junior's constituent element is respectively illustrated as independent composition, but the present invention is not limited thereto.For example, the junior of database 240 At least part of constituent element can be constituted with integration.
In model data DB242, it is stored with the model data for representing the threedimensional model of target.According to one embodiment, Model data can be the CT/MRI data 10 for modeling and generating using head shape of the CT/MRI device to patient 20.This When, model data can be received by communication unit 210 from CT/MRI device, be stored in model data DB242.
In measurement surface data DB244, at least part of measurement table for the three-dimensional surface for representing target can store Face data.According to one embodiment, measure surface data can be using the spatial digitizer 140 of Fig. 1 measure patient 20 eye, The data of the outer shape of ear, nose etc..At this point, measurement surface data can be deposited from spatial digitizer 140 by communication unit 210 It is stored in measurement surface data DB244.
The model data that surface treating part 222 can be stored from model data DB242 extracts model surface data.Mould Data of the type surface data as the threedimensional model surface for representing target, can have coordinate system (example identical with model data Such as, x1y1z1Coordinate system).For example, when model data is to represent the CT/MRI data 10 of 20 head 3D shape of patient, model Surface data is the data for representing 20 head three-dimensional surface of patient.Model surface data can be stored in by surface treating part 222 Model surface data DB246.
First transformation component 224 can carry out two-dimensional transform to the model surface data stored in model surface data DB246. First transformation component 224 can carry out two-dimensional transform to model surface data by expansion.Wherein, so-called expansion, such as can be Paper " Unwrapping highly-detailed 3D meshes of rotationally symmetric man-made objects(Rieck,Bastian,Hubert Mara,and Susanne Kromker；(2013): 259-264 definition in) " 2-D data convert algorithm.As expansion, spherical expansion (spherical unwrapping), cylindrical expansion can use (equirectangular unwrapping) etc. is unfolded in (cylindrical unwrapping), equidistant column.First transformation Portion 224 can select the type of expansion based on the 3D shape that model surface data represent.As an example, work as mould When type surface data is the spherical shape as head, the first transformation component 224 can choose spherical expansion.As another example, when When model surface data are such as cylinders of arm, leg, the first transformation component 224 can choose cylindrical expansion.
When the spherical expansion of selection, the first transformation component 224 can be with the centre coordinate of decision model surface data.According to one The average value of all the points of model surface data can be determined as centre coordinate by embodiment, the first transformation component 224.In addition, the One transformation component 224 can move in parallel model surface data, so that the centre coordinate of model surface data becomes coordinate system (example Such as, x1y1z1Coordinate system) origin.In addition, the first transformation component 224 can will be from centre coordinate to each point of model surface data Distance average value calculate on the basis of distance.Then, the first transformation component 224 can be based on the reference distance of calculating, benefit With mathematical expression 1 the following, by the three-dimensional system of coordinate being made of x-axis, y-axis and z-axis (for example, x1y1z1Coordinate system) it is transformed by φ axis And the two-dimensional coordinate system that θ axis is constituted is (for example, φ1θ1Coordinate system).As a result, the first transformation component 224 can be obtained from model surface number According to the model surface 2-D data through two-dimensional transform.
[mathematical expression 1]
Mathematical expression 1 assumes that the situation that on the basis of y-axis model surface data are carried out with spherical expansion.Wherein, riMeaning From centre coordinate (for example, origin of three-dimensional system of coordinate) to the distance of each point of model surface data.According to one embodiment, The value (for example, brightness value) of each point of model surface 2-D data can be represented from model surface data corresponding with respective point It puts to the distance of centre coordinate.For example, distance from predetermined point to the centre coordinate of model surface data it is longer (that is, further away from Centre coordinate), then the brightness value of the point of corresponding model surface 2-D data can more increase.First transformation component 224 can will be from The model surface two-dimensional data storage of model surface data transformation is in model surface 2-D data DB248.For to model surface The transformation of 2-D data, behind described referring to Fig. 3.
Second transformation component 226 can carry out two-dimensional transform to the measurement surface data stored in measurement surface data DB244. Second transformation component 226 can carry out two-dimensional transform to measurement surface data by expansion.As expansion, spherical exhibition can use It opens, cylindrical expansion, equidistant column expansion etc..The expansion that second transformation component 226 can choose and be selected by the first transformation component 224 The transformation of identical type.For example, when the spherical expansion by means of the first transformation component 224 of model surface data, the second transformation component 226 can also carry out measurement surface data spherical expansion.
Even if the model surface data for representing the surface of target have the spherical form on such as head, the surface of target is scanned Measurement surface data that is a part of and obtaining can also not have spherical form as eyes or nose.Second transformation component 226 In order to which measurement surface data as described above and aspherical is carried out spherical expansion, paper " Unwrapping can be used for example highly-detailed 3D meshes of rotationally symmetric man-made objects(Rieck、 Bastian,Hubert Mara,and Susanne Kromker；(2013): 259-264 deployment algorithm used in) ".Example Such as, the second transformation component 226, which can be generated, to make when carrying out the two-dimensional transform of model surface data by means of the first transformation component 224 Reference distance (for example, from centre coordinate to the average value of the distance of each point of model surface data) is as the virtual of radius Ball.Second transformation component 226 can move in parallel measurement surface data or moving in rotation, so that measurement surface data is located at virtually Spherical surface on.Then, the second transformation component 226 can be based on reference distance, will be by x-axis, y using the mathematical expression 1 The three-dimensional system of coordinate that axis and z-axis are constituted is (for example, x2y2z2Coordinate system) it is transformed into the two-dimensional coordinate system (example being made of φ axis and θ axis Such as, φ2θ2Coordinate system).As a result, the second transformation component 226 can be obtained from measurement measurement surface two of the surface data through two-dimensional transform Dimension data.
According to one embodiment, measure each point of surface 2-D data value (for example, brightness value) can represent from phase The distance of centre coordinate of the point of corresponding measurement surface data to virtual ball should be put.For example, from the predetermined of measurement surface data Point to virtual ball centre coordinate distance it is longer (that is, further away from centre coordinate), it is corresponding measurement surface 2-D data point Brightness value more increase.Second transformation component 226 can by from measurement surface data convert measurement surface two-dimensional data storage in Measure surface 2-D data DB250.For to measurement surface 2-D data transformation, behind described referring to Fig. 4 and Fig. 5.
Initial position setting unit 228 can model surface two-dimemsional number to store in the 2-D data DB248 of Matching Model surface According to the measurement surface 2-D data that stores in measurement surface 2-D data DB250.According to one embodiment, initial position setting Portion 228 uses NCC (Normalized Cross Correlation, normalized crosscorrelation) algorithm, Matching Model surface two dimension Data and measurement surface 2-D data.Initial position setting unit 228 determines and measurement surface two in model surface 2-D data The presumptive area of dimension data has the region of maximum similar degree, so as to Matching Model surface 2-D data and measurement surface two Dimension data utilizes the region of such matched model surface 2-D data and measurement surface 2-D data, in that case it can be decided that Matching band Domain.For example, initial position setting unit 228 in model surface 2-D data, makes the presumptive area for measuring surface 2-D data Position is constantly changed by preset distance, and computation model surface 2-D data is similar with measurement surface 2-D data simultaneously Degree.In addition, initial position setting unit 228 utilizes the model surface 2-D data and measurement surface two-dimemsional number with maximum similar degree According to region, determine matching area respectively.
According to embodiment, initial position setting unit 228 can be shown in model surface 2-D data with multiple points and be determined Matching area.For example, initial position setting unit 228 can use corresponding Matching band when the matching area of decision is rectangle Two opposite vertex in domain, or using four vertex of corresponding region, show this rectangular area.In addition, initial bit installs Setting portion 228 can use the matching area of multiple point performance measurement surface 2-D datas.It is specific by initial position setting unit 228 It is multiple, it can have φ θ coordinate system respectively (for example, φ1θ1Coordinate system and φ2θ2Coordinate system).In addition, for model surface Matching between 2-D data and measurement surface 2-D data, is described later with reference to Fig. 6 and Fig. 7.
Initial position setting unit 228 can be by model corresponding with the matching area determined in model surface 2-D data The region of surface data and the region of measurement surface data corresponding with the matching area determined in measurement surface 2-D data, It is respectively set to initial position.For example, initial position setting unit 228 can be in model surface data, decision and model surface The corresponding multiple coordinates of the coordinate of the matching area of 2-D data, set the region defined according to multiple coordinates of decision to Initial position.In addition, initial position setting unit 228 can determine with measurement surface 2-D data in measurement surface data The corresponding multiple coordinates of coordinate with region, set initial position for the region defined according to multiple coordinates of decision.
Initial position setting unit 228 can use in order to which the initial positions of model surface data is arranged by means of first Transformation component 224 carries out the transformation by reciprocal direction of used expansion when two-dimensional transform.For example, when the first transformation component 224 passes through spherical exhibition When opening model surface data progress two-dimensional transform, the transformation by reciprocal direction that initial position setting unit 228 can be unfolded by spherical shape, if Set the initial position of model surface data.At this point, initial position setting unit 228 can carry out spherical exhibition with the first transformation component 224 Based on the reference distance once utilized when opening, three-dimension varying is executed.The initial position of model surface data has xyz coordinate system (for example, x1y1z1Coordinate system).
Initial position setting unit 228 measures the initial position of surface data in order to be arranged, and can use and becomes by means of second Change the transformation by reciprocal direction that portion 226 carries out the expansion utilized when two-dimensional transform.For example, when the second transformation component 226 passes through spherical expansion And when carrying out two-dimensional transform to measurement surface data, the transformation by reciprocal direction that initial position setting unit 228 can be unfolded by spherical shape, if Set the initial position of measurement surface data.At this point, initial position setting unit 228 can carry out spherical exhibition with the second transformation component 226 Based on the reference distance once utilized when opening, three-dimension varying is executed.The initial position for measuring surface data has xyz coordinate system (for example, x2y2z2Coordinate system).Setting for model surface data and the initial position for measuring surface data, later with reference to Figure 11 and Figure 12 are described.
According to multiple embodiments, model surface 2-D data and measurement surface 2-D data can be carried out with respectively partially micro- Based on the data divided, model surface data are set and measure the initial position of surface data.For this purpose, curvature transformation component 230 can To carry out partial differential to model surface 2-D data and measurement surface 2-D data respectively, model surface two-dimensional curvature data are generated And measurement surface two-dimensional curvature data.At this point, initial position setting unit 228 can be with Matching Model surface two-dimensional curvature data and survey Scale face two-dimensional curvature data determine matching area.
Curvature transformation component 230 can value and adjacent multiple points based on the respective point of model surface 2-D data value, Generate model surface two-dimensional curvature data.According to one embodiment, the value of each point of model surface two-dimensional curvature data (for example, Brightness value) it can be with the value of the respective point of representative model surface 2-D data and adjacent multiple points (for example, 4,8,14 Deng) average value difference.Model surface two-dimensional curvature data can be the flat of the average curvature of representative model surface 2-D data The Gaussian curvature data of the Gaussian curvature of equal curvature data or representative model surface 2-D data.
Curvature transformation component 230 can value based on the respective point of measurement surface two-dimensional curvature data and adjacent multiple points Value generates measurement surface two-dimensional curvature data.According to one embodiment, the value (example of each point of surface two-dimensional curvature data is measured Such as, brightness value) can represent measurement surface 2-D data respective point value and adjacent multiple points (for example, 4,8,14 It is a etc.) average value difference.Measurement surface two-dimensional curvature data can be the average curvature for representing measurement surface 2-D data Average curvature data, or represent the Gaussian curvature data of the Gaussian curvature of measurement surface 2-D data.
According to one embodiment, curvature transformation component 230 can generate the average song about model surface 2-D data respectively Rate data and Gaussian curvature data generate average curvature data and Gaussian curvature number about measurement surface 2-D data respectively According to.At this point, initial position setting unit 228 can be with the average curvature data of Matching Model surface 2-D data and measurement surface two dimension The average curvature data of data, the identification measurement matched first area of surface 2-D data in model surface 2-D data.Separately Outside, initial position setting unit 228 can be matched about the Gaussian curvature data of model surface 2-D data and about measurement surface The Gaussian curvature data of 2-D data, identification measurement matched secondth area of surface 2-D data in model surface 2-D data Domain.Then, the median of first area and second area can be judged as matching area by initial position setting unit 228, setting Initial position.
According to one embodiment, curvature transformation component 230 executes partial differential using mask operation.For example, when utilizing 3*3 size Mask when, curvature transformation component 230 calculate predetermined point value and 8 adjacent points value difference, to calculate the song of respective point Rate.For example, curvature is bigger, the brightness value of respective point can be shown smaller by curvature transformation component 230.By means of curvature transformation component 230 and generate model surface two-dimensional curvature data and measurement surface two-dimensional curvature data, model surface can be stored respectively in Two-dimensional curvature data DB252 and measurement surface two-dimensional curvature data DB254.For to model surface two-dimensional curvature data and measurement The transformation of surface two-dimensional curvature data, is described later with reference to Fig. 8 to Figure 10.
According at least one embodiment, the model surface two-dimensional curvature data that are generated by means of curvature transformation component 230 and Surface two-dimensional curvature data are measured, it, can more clearly generation compared with model surface 2-D data and measurement surface 2-D data The variation of table data value.Therefore, when Matching Model surface two-dimensional curvature data and measurement surface two-dimensional curvature data, setting are used for When the initial position of integration, the effect for integrating accuracy and efficiency improved between data can be obtained.
Data interfacing block 232 can be based on the initial position being arranged by means of initial position setting unit 228, integration Between model surface data and measurement surface data.According to one embodiment, data interfacing block 232 can use ICP (Iterative Closest Point) algorithm executes the integration between data.The model surface data of integration and measurement surface Data have mutually the same coordinate system.For example, measurement surface data is by the coordinate system of model surface data (for example, x1y1z1It sits Mark system) arrangement.
Fig. 3 is to show the model surface two from model surface data 300 through two-dimensional transform according to an embodiment of the present invention The figure of dimension data 310.It can be by means of Fig. 2's from model surface data 300 to the shift process of model surface 2-D data 310 First transformation component 224 and implement.
As shown in figure 3, the surface on the commissarial head of model surface data 300.According to one embodiment, model surface number It can be the model data (for example, CT/MRI data 10) for modeling and generating from the head to people according to 300 and extract surface number According to and generate data.The each point P of model surface data 300iWith x-axis, y-axis and z-axis coordinate value (xi、yi、zi)。
First transformation component 224 calculates the average value of all the points of model surface data 300, determines centre coordinate Co.First Transformation component 224 moves in parallel model surface data 300, so that centre coordinate CoOrigin (0,0,0) as coordinate system.First becomes It changes portion 224 and utilizes mathematical expression 1 illustrated in fig. 2, by xyz coordinate system (for example, x1y1z1Coordinate system) it is transformed into φ θ coordinate system (example Such as, φ1θ1Coordinate system), from the model surface data 300 as three-dimensional data, generate the model surface two dimension as 2-D data Data 310.The width coordinate of model surface 2-D data 310 is φ, height coordinate θ.As shown in figure 3, model surface data 300 predetermined point Pi(xi、yi、zi) correspond to model surface 2-D data 310 predetermined point Pi (φi、θi)。
In model surface data 300, riIt represents from centre coordinate to predetermined point PiDistance.In model surface two-dimemsional number In 310, both fixed point P of model surface data 300iRiWith respective point PiBrightness value represent.For example, in model surface number In 300, from centre coordinate, if the point near distant nose, lower jaw, ear, forehead and the back side of head carries out Two-dimensional transform then has the biggish data of brightness value.In addition, in model surface data 300, from centre coordinate, distance than If the point near closer eye, cheek carries out two-dimensional transform, there are the lesser data of brightness value.
Fig. 4 is shown according to an embodiment of the present invention from measurement measurement surface two of the surface data 400 through two-dimensional transform The figure of dimension data 410.In addition, Fig. 5 is shown according to an embodiment of the present invention from measurement surface data 500 through two-dimensional transform Measure the figure of surface 2-D data 510.Transformation from from measurement surface data 400,500 to measurement surface 2-D data 410,510 Process can be implemented by means of the second transformation component 226 of Fig. 2.
As shown in figure 4, the surface of measurement surface data 400 commissarial eye and nose, measurement surface data 500 are commissarial The surface of ear.According to one embodiment, measuring surface data 400,500 can be the scanning of spatial digitizer 140 of Fig. 1 and gives birth to At data.The each point for measuring surface data 400,500 has x-axis, y-axis and z-axis coordinate value.
Measurement surface data 400,500 is carried out spherical expansion by the second transformation component 226 respectively, generates measurement surface two-dimemsional number According to 410,510.Three-dimensional integration between the measurement surface data 400,500 for the model surface data 300 and Fig. 4 for executing Fig. 3 When, the second transformation component 226 can use to be become with the two dimension of the model surface data 300 carried out by means of the first transformation component 224 The congener transformation of commutation (for example, spherical expansion).
Second transformation component 226 is generated reference distance used in the two-dimensional transform of model surface data 300 as radius Virtual ball.Second transformation component 226 makes measurement surface data 400,500 parallel or moving in rotation respectively, so that measurement surface number It is located at according to 400,500 on the surface of virtual ball.Then, second transformation component 226 is with reference distance (for example, virtual ball Radius) based on, using mathematical expression 1 illustrated in fig. 2, by xyz coordinate system (for example, x2y2z2Coordinate system) it is transformed into φ θ coordinate System is (for example, φ1θ1Coordinate system).As a result, generating from the measurement surface data 400,500 as three-dimensional data and being used as 2-D data Measurement surface 2-D data 410,510.The width coordinate for measuring surface 2-D data 410,510 is φ, height coordinate θ.
Fig. 6 be show according to an embodiment of the present invention with the matched measurement surface of model surface 2-D data 600 two dimension The figure of data 610.In addition, Fig. 7 be show one embodiment of the invention with the matched measurement table of model surface 2-D data 700 The figure of face 2-D data 710.Model surface 2-D data 600,700 can be the model surface 2-D data 310 of Fig. 3.In addition, Measuring surface 2-D data 610 can be at least part of the measurement surface 2-D data 410 of Fig. 4, measure surface two-dimemsional number It can be at least part of the measurement surface 2-D data 510 of Fig. 5 according to 710.Matching process in Fig. 6 and Fig. 7 can be by In Fig. 2 initial position setting unit 228 and implement.
If initial position setting unit 228 will measure surface two-dimemsional number in model surface 2-D data 600 referring to Fig. 6 Preset distance, or rotation predetermined angular are moved in parallel according to 610 position, and is directed to model surface 2-D data 600 simultaneously, is calculated Measure the similar degree of surface 2-D data 610.Initial position setting unit 228 determines the model surface two dimension with maximum similar degree The position of data 600, so as to Matching Model surface 2-D data 600 and measurement surface 2-D data 610.In addition, initial Position setting unit 228 can determine the matching area between Definition Model surface 2-D data 600 and measurement surface 2-D data 610 The point 620,630,640,650 of four required vertex correspondences.Similar, initial position setting unit 228 can be such as Fig. 7 institute Show, Matching Model surface 2-D data 700 and measurement surface 2-D data 710 determine and define needed for matching area four The point 720,730,740,750 of vertex correspondence.
As described above, at least one embodiment according to the present invention, will be used for the initial of three-dimensional data integration in order to specified Position replaces and implements as the direct matching between the three-dimensional data for integrating object, but can be implemented and convert from three-dimensional data 2-D data between matching.According to embodiment, the present invention can be processed the 2-D data converted from three-dimensional data, Implement matching between the new each 2-D data generated respectively.
In addition, three-dimension varying is re-started to matching area, and in three-dimensional data, initial position of the setting for integration. As a result, it is possible to which initial position appropriate is arranged, the effect for improving the accuracy and efficiency of the integration between three-dimensional data can be obtained Fruit.
Fig. 8 shows and carries out partial differential according to an embodiment of the present invention to the model surface 2-D data 310 of Fig. 3 and generate Model surface two-dimensional curvature data 800.In addition, Fig. 9 show the measurement surface 2-D data 410 to Fig. 4 carry out partial differential and The measurement surface two-dimensional curvature data 900 of generation.In addition, Figure 10 shows that the progress of the measurement surface 2-D data 510 to Fig. 5 is partially micro- The measurement surface two-dimensional curvature data 1000 divided and generated.Curvature shift process of the Fig. 8 into Figure 10 can be by means of the song of Fig. 2 Rate transformation component 230 and implement.
If the biggish point of difference referring to Fig. 8, in model surface two-dimensional curvature data 800, with data adjacent to each other (for example, biggish ear, eye, nose and mouth contour line near point) there is greater curvature value.On the contrary, with number adjacent to each other According to the lesser point (for example, point near cheek or forehead) of difference have compared with curvature value.If surveyed referring to Fig. 9 and Figure 10 It is bigger with the difference of point adjacent to each other in scale face two-dimensional curvature data 900,1000, then there is bigger curvature value.
Model surface two-dimensional curvature data 800 shown in Fig. 8 are compared with the model surface 2-D data 310 of Fig. 3, data value Variation show more clear.In addition, Fig. 9 and measurement surface shown in Fig. 10 two-dimensional curvature data 900,1000 and Fig. 4 and Fig. 5 Measurement surface 2-D data 410,510 compare, the variation of data value shows more clear.As described above, according to the present invention At least one embodiment, in terms of the 2-D data that matching is converted from three-dimensional data, implementation carries out respective two-dimensional data respectively Matching between the curvature data of partial differential.As a result, initial position is more accurately arranged, it is hereby achieved that further increasing three The effect of the accuracy and efficiency of integration between dimension data.
Figure 11 is to show the model surface data 1100 provided with initial position according to an embodiment of the present invention and measurement The figure of surface data 1150.In addition, Figure 12 is to show the model surface for being provided with initial position according to an embodiment of the present invention Data 1200 and the figure for measuring surface data 1250.Initial position setting procedure in Figure 11 and Figure 12 can be by means of Fig. 2's Initial position setting unit 228 and implement.
If referring to Fig.1 1, initial position setting unit 228 by with the point that determines to show matching area in Fig. 6 620, the point 1110,1120,1130,1140 of 630,640,650 corresponding model surface data 1100 and with point 620,630, 640, the point 1160,1170,1180,1190 of 650 corresponding measurement surface datas 1150, is determined as initial position respectively.Point 1110,1120,1130,1140 and point 1160,1170,1180,1190 respectively by with two-dimensional coordinate system (for example, φ1θ1Coordinate System and φ2θ2Coordinate system) point 620,630,640,650 by spherical shape expansion transformation by reciprocal direction be transformed into three-dimensional coordinate System is (for example, x1y1z1Coordinate system and x2y2z2Coordinate system).
Point 1110,1120,1130,1140 and point 1160,1170,1180,1190 are used separately as model surface data 1100 And required initial position is integrated between measurement surface data 1150.In other words, with point 1110,1120,1130,1140 and point 1160, based on 1170,1180,1190, model surface data 1100 and measurement surface data 1150 carry out three-dimensional integration.Example Such as, by put 1110,1120,1130,1140 and point 1160,1170,1180,1190 based on execute ICP integration algorithm as a result, The coordinate system of model surface data 1100 is (for example, x1y1z1Coordinate system) it integrates that there is the coordinate system for measuring surface data 1150 (for example, x2y2z2Coordinate system).
If referring to Fig.1 2, initial position setting unit 228 determines and determines to have to show in Fig. 7 respectively to match The point 1210,1220,1230,1240 of the corresponding model surface data 1200 of point 720,730,740,750 in region and with point 720, the point 1260,1270,1280,1290 of 730,740,750 corresponding measurement surface datas 1250.Point 1210,1220, 1230,1240 and point 1260,1270,1280,1290 respectively by with two-dimensional coordinate system (for example, φ1θ1Coordinate system and φ2θ2It sits Mark system) point 720,730,740,750 by the transformation by reciprocal direction of spherical expansion convert with three-dimensional system of coordinate (for example, x1y1z1Coordinate system and x2y2z2Coordinate system).
Point 1210,1220,1230,1240 and point 1260,1270,1280,1290 are used separately as model surface data 1200 And initial position needed for the integration between measurement surface data 1250.In other words, with point 1210,1220,1230,1240 and point 1260, based on 1270,1280,1290, model surface data 1200 and measurement surface data 1250 carry out three-dimensional integration.Example Such as, by put 1210,1220,1230,1240 and point 1260,1270,1280,1290 based on execute ICP integration algorithm as a result, The coordinate system of model surface data 1200 is (for example, x1y1z1Coordinate system) it integrates that there is the coordinate system for measuring surface data 1250 (for example, x2y2z2Coordinate system).
According to the present invention, by means of initial position is accurately arranged, the standard for improving the integration between three-dimensional data can be obtained The effect of true property and efficiency.
Figure 13 is to show according to embodiments of the present invention and the precedence diagram of the method for integrating three-dimensional data.Step shown in Figure 13 At least part can be executed by means of Fig. 1 and composition shown in Fig. 2.
Firstly, model surface data are carried out two-dimensional transform and generate model surface 2-D data in step S1300. For example, each point of the model surface data with three-dimensional system of coordinate is carried out average and determines center by the first transformation component 224 of Fig. 2 After coordinate, respective center coordinate can use, move in parallel model surface data.First transformation component 224 can be to sit from center Model surface data spherical shape is launched into two by mark to distance on the basis of the average value of the distance of each point of model surface data Tie up the model surface 2-D data of coordinate system.The value (for example, brightness value) of each point of model surface 2-D data can represent from With respective point at a distance from point to the centre coordinate of corresponding model surface data.
Then, in step S1310, measurement surface data is subjected to two-dimensional transform and generates measurement surface 2-D data. For example, the reference distance that will calculate in step S1300 can be generated as the virtual ball of radius in the second transformation component 226.Second Transformation component 226 can make to measure surface data movement, so that measurement surface data is located at the surface of virtual ball.Then, second becomes Measurement surface data spherical shape can be launched into measurement surface 2-D data by changing portion 226.Measure each point of surface 2-D data Value (for example, brightness value) can represent from it is corresponding with respective point measurement surface data point to virtual ball centre coordinate away from From.
Then, it in step S1320, calculates separately model surface 2-D data and measures the curvature of surface 2-D data. For example, curvature transformation component 230 can be by the model surface 2-D data generated in step S1300 and the life in step S1310 At measurement surface 2-D data distinguish partial differential and generate model surface two-dimensional curvature data and measurement surface two-dimensional curvature number According to.The value (for example, brightness value) of each point of model surface two-dimensional curvature data can be with the corresponding of representative model surface 2-D data The difference of the average value of the value and adjacent multiple points of point.In addition, the value of each point of measurement surface two-dimensional curvature data is (for example, brightness Value) can represent surface measurement 2-D data respective point value and adjacent multiple points average value difference.
Then, in step S1330, Matching Model surface two-dimensional curvature data and measurement surface two-dimensional curvature data, certainly Determine matching area.For example, initial position setting unit 228 can model surface two-dimensional curvature number to generate in step S1320 According to and measurement surface two-dimensional curvature data based on, in model surface two-dimensional curvature data, determine bent with measurement surface two dimension The presumptive area of rate data has the region of maximum similar degree.Initial position setting unit 228 can be in model surface two-dimensional curvature In the region determined in data, matching area is determined.In addition, initial position setting unit 228 can be used for matched measurement table In the presumptive area of face two-dimensional curvature data, matching area is determined.
Then, in step S1340, model surface data is respectively set and measure the initial position of surface data.For example, Initial position setting unit 228 can use the model table determined in step S1330 with the transformation by reciprocal direction by spherical shape expansion The point of the corresponding model surface data of the matching area of face two-dimensional curvature data and with the measurement table that is determined in step S1330 The point of the corresponding measurement surface data of the matching area of face two-dimensional curvature data, is arranged initial position.
Then, in step S1350, Integrated Models surface data and measurement surface data.For example, data interfacing block 232 It can use the initial position for the model surface data being arranged in step S1340 and measure the initial position of surface data, it is whole Molding type surface data and measurement surface data.
In addition, a part of step can be omitted in step shown in Figure 13, or more than two steps can be real simultaneously Apply or step between implementation sequence can change.According to one embodiment, step S1320 be can be omitted, in step S1330, With Matching Model surface 2-D data and surface 2-D data can be measured, respectively decision model surface 2-D data and measurement surface Matching area on 2-D data.It according to another embodiment, can be with before step S1330 further include: reception represents target The step of CT data or MRI data of threedimensional model；And from corresponding CT data or MRI data extract CT data surface data or The step of surface data of MRI data is as model surface data.
The method of integrating three-dimensional data is illustrated by specific embodiment, but the method can also can in computer In the recording medium of reading, with the embodiment of computer-readable code.Computer-readable recording medium includes that be stored with can be by means of Computer system and the recording device of all kinds of data read.As the example of computer-readable recording medium, have ROM, RAM, CD-ROM, tape, floppy disk, optical data storage devices etc., in addition, also including with carrier wave (for example, by internet Transmission) form embody situation.In addition, the computer system that computer readable recording medium is scattered in network connection, with Dispersing mode is stored with computer-readable code and runs.Moreover, the functionality for embodying the embodiment (functional) program, code and code snippet can be by programmer's easily inferences of the technical field of the invention.
The present invention is illustrated by preferred embodiment, is illustrated, but as long as be person of ordinary skill in the field just It is found that can realize various deformation and change in the case where the item and scope without departing from appended claims.

Claims (19)

1. a kind of three-dimensional data integration method, wherein include:
The first three-dimensional data progress two-dimensional transform on the threedimensional model surface for representing target is generated to the step of the first 2-D data Suddenly；
At least part of second three-dimensional data for the three-dimensional surface for representing the target is subjected to two-dimensional transform and generates second The step of 2-D data；Second 2-D data is matched with first 2-D data and determines first two dimension respectively The step of the second matching area on the first matching area and second 2-D data in data；
By multiple points of first three-dimensional data corresponding with first matching area and with second matching area pair The step of multiple points for second three-dimensional data answered are set as initial position；And
Using the initial position, the step of integrating first three-dimensional data and second three-dimensional data.
2. three-dimensional data integration method according to claim 1, wherein
Described the step of determining the first matching area and the second matching area respectively includes:
Partial differential is carried out to first 2-D data and second 2-D data respectively and generates the first two-dimensional curvature data And the step of the second two-dimensional curvature data；And
The first two-dimensional curvature data and the second two-dimensional curvature data are matched, determine the first two-dimensional curvature number respectively The step of according to the second matching area in the first upper matching area and the second two-dimensional curvature data.
3. three-dimensional data integration method according to claim 2, wherein
The step of the first two-dimensional curvature data of the generation and the second two-dimensional curvature data includes:
The value of value and adjacent multiple points based on each point on first 2-D data and generate first two-dimensional curvature The step of data；And
The value of value and adjacent multiple points based on each point on second 2-D data and generate second two-dimensional curvature The step of data.
4. three-dimensional data integration method according to claim 2, wherein
The step of the first two-dimensional curvature data of the generation and the second two-dimensional curvature data includes:
As the first two-dimensional curvature data, the average curvature data for representing the average curvature of first 2-D data are generated Or the step of representing the Gaussian curvature data of Gaussian curvature of first 2-D data；And
As the second two-dimensional curvature data, the average curvature data for representing the average curvature of second 2-D data are generated Or the step of representing the Gaussian curvature data of Gaussian curvature of second 2-D data.
5. three-dimensional data integration method according to claim 1, wherein
It is described first three-dimensional data to be subjected to two-dimensional transform and the step of generating the first 2-D data includes:
The step of determining the centre coordinate of first three-dimensional data；
The step of calculating from the centre coordinate to the average value of the distance of the first three-dimensional data each point；And
(spherical is unfolded in the first three-dimensional data spherical shape by the distance on the basis of the average value of the calculating Unwrapping) the step of being first 2-D data, the value of each point of first 2-D data are represented from described the Corresponding first three-dimensional data points of one 2-D data each point to the centre coordinate distance.
6. three-dimensional data integration method according to claim 5, wherein
It is described second three-dimensional data to be subjected to two-dimensional transform and the step of generating the second 2-D data includes:
It moves second three-dimensional data and second three-dimensional data is located at using the reference distance as the spherical shape of radius Step on surface；And
Based on the reference distance, the step of the second three-dimensional data spherical shape is expanded into second 2-D data, The value of each point of second 2-D data is represented from second three-dimensional data corresponding with the second 2-D data each point Point to the ball centre coordinate distance.
7. three-dimensional data integration method according to claim 1, wherein
Described the step of determining the first matching area and the second matching area respectively includes:
In first 2-D data, the region with the presumptive area of second 2-D data with maximum similar degree is determined The step of；
In the region determined in first 2-D data, determine first matching area the step of；And
In the presumptive area of second 2-D data, determine second matching area the step of.
8. three-dimensional data integration method according to claim 1, wherein
The step of setting initial position includes:
In first three-dimensional data, the initial position is set by multiple points corresponding with first matching area, And in second three-dimensional data, it sets multiple points corresponding with second matching area to the step of the initial position Suddenly.
9. three-dimensional data integration method according to claim 1, wherein further include:
The step of receiving the CT data or MRI data for the threedimensional model for representing the target；And
From the CT data or the MRI data, the surface data of the surface data of the CT data or the MRI data is mentioned The step of being taken as first three-dimensional data.
10. a kind of three-dimensional data integrating apparatus, wherein include:
First three-dimensional data on the threedimensional model surface for representing target is carried out two-dimensional transform and generates first by the first transformation component 2-D data；
Second transformation component, at least part of second three-dimensional data that will represent the three-dimensional surface of the target carry out two-dimentional change It changes and generates the second 2-D data；
Second 2-D data is matched with first 2-D data by initial position setting unit, determines described respectively The second matching area on the first matching area and second 2-D data on one 2-D data, will match with described first Multiple points of corresponding first three-dimensional data in region and second three-dimensional data corresponding with second matching area Multiple points be set as initial position；And
Data interfacing block utilizes the initial position, integrates first three-dimensional data and second three-dimensional data.
11. three-dimensional data integrating apparatus according to claim 10, wherein
Further include: curvature transformation component, first 2-D data and second 2-D data are subjected to partial differential respectively and Generate the first two-dimensional curvature data and the second two-dimensional curvature data；
The initial position setting unit matches the first two-dimensional curvature data and the second two-dimensional curvature data, determines respectively The second matching area on the first matching area and the second two-dimensional curvature data in the first two-dimensional curvature data.
12. three-dimensional data integrating apparatus according to claim 11, wherein
The value of value and adjacent multiple points of the curvature transformation component based on each point on first 2-D data, described in generation First two-dimensional curvature data, the value of value and adjacent multiple points based on each point on second 2-D data, described in generation Second two-dimensional curvature data.
13. three-dimensional data integrating apparatus according to claim 11, wherein
The curvature transformation component generates the average curvature for representing first 2-D data as the first two-dimensional curvature data Average curvature data or represent first 2-D data Gaussian curvature Gaussian curvature data；
As the second two-dimensional curvature data, the average curvature data for representing the average curvature of second 2-D data are generated Or represent the Gaussian curvature data of the Gaussian curvature of second 2-D data.
14. three-dimensional data integrating apparatus according to claim 10, wherein
First transformation component determines the centre coordinate of first three-dimensional data, calculates from the centre coordinate to described first The average value of the distance of each point of three-dimensional data, the distance on the basis of the average value of the calculating, by first three-dimensional data Spherical shape expands into first 2-D data, the value of each point of first 2-D data represent from first 2-D data Corresponding first three-dimensional data points of each point to the centre coordinate distance.
15. three-dimensional data integrating apparatus according to claim 14, wherein
Second transformation component moves second three-dimensional data and second three-dimensional data is located at the stand-off From on the spherical surface for radius；Based on the reference distance, the second three-dimensional data spherical shape is expanded into described Two 2-D datas, the value of each point of second 2-D data are represented from corresponding with the second 2-D data each point described the The point of two three-dimensional datas to the ball centre coordinate distance.
16. three-dimensional data integrating apparatus according to claim 10, wherein
The initial position setting unit determines have with the presumptive area of second 2-D data in first 2-D data There is the region of maximum similar degree；In the region determined in first 2-D data, first matching area is determined；Institute In the presumptive area for stating the second 2-D data, second matching area is determined.
17. three-dimensional data integrating apparatus according to claim 10, wherein
The initial position setting unit sets multiple points corresponding with first matching area in first three-dimensional data It is set to the initial position, and in second three-dimensional data, it will multiple point settings corresponding with second matching area For the initial position.
18. three-dimensional data integrating apparatus according to claim 10, wherein further include:
Communication unit receives the CT data or MRI data for representing the threedimensional model of the target；And
Surface extraction portion, from the CT data or the MRI data, by the surface data of the CT data or the MRI number According to surface data be extracted as first three-dimensional data.
19. a kind of computer readable storage medium, storage is each to any one of 9 methods according to claim 1 comprising executing The program of the order of step.
CN201780049378.2A 2016-08-10 2017-08-09 Three-dimensional data integration device and method Active CN109564693B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
KR1020160102070A KR101812001B1 (en) 2016-08-10 2016-08-10 Apparatus and method for 3d data registration
KR10-2016-0102070 2016-08-10
PCT/KR2017/008609 WO2018030781A1 (en) 2016-08-10 2017-08-09 Three-dimensional data matching device and method

Publications (2)

Publication Number Publication Date
CN109564693A true CN109564693A (en) 2019-04-02
CN109564693B CN109564693B (en) 2024-04-09

Family

ID=60938539

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201780049378.2A Active CN109564693B (en) 2016-08-10 2017-08-09 Three-dimensional data integration device and method

Country Status (6)

Country Link
US (1) US11481867B2 (en)
EP (1) EP3499464B1 (en)
JP (1) JP6785362B2 (en)
KR (1) KR101812001B1 (en)
CN (1) CN109564693B (en)
WO (1) WO2018030781A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10534962B2 (en) * 2017-06-17 2020-01-14 Matterport, Inc. Automated classification based on photo-realistic image/model mappings
WO2020005204A1 (en) * 2018-06-25 2020-01-02 Hewlett-Packard Development Company, L.P. Three-dimensional registration procedures
KR102433473B1 (en) * 2019-10-30 2022-08-19 주식회사 스키아 Method, apparatus and computer program for providing augmented reality based medical information of patient
US11536843B2 (en) * 2020-02-08 2022-12-27 The Boeing Company De-jitter of point cloud data for target recognition

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5999840A (en) * 1994-09-01 1999-12-07 Massachusetts Institute Of Technology System and method of registration of three-dimensional data sets
CN104318552A (en) * 2014-10-15 2015-01-28 北京理工大学 Convex hull projection graph matching based model registration method
CN105225219A (en) * 2014-06-25 2016-01-06 联想(北京)有限公司 Information processing method and electronic equipment
CN105261052A (en) * 2015-11-03 2016-01-20 沈阳东软医疗系统有限公司 Lumen image unfolding and drawing method and device
CN105654029A (en) * 2015-11-30 2016-06-08 辽宁师范大学 Three-dimensional point cloud auricle identification method for increasing identification precision and efficiency

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5531520A (en) 1994-09-01 1996-07-02 Massachusetts Institute Of Technology System and method of registration of three-dimensional data sets including anatomical body data
JP3404675B2 (en) 1999-03-19 2003-05-12 日本電信電話株式会社 Three-dimensional tomographic image interpretation method, automatic collation method, device thereof, and recording medium recording the program
US6904163B1 (en) 1999-03-19 2005-06-07 Nippon Telegraph And Telephone Corporation Tomographic image reading method, automatic alignment method, apparatus and computer readable medium
KR100819960B1 (en) * 2002-11-06 2008-04-07 지오메트릭 인포매틱스 인코퍼레이티드 Analysis of geometric surfaces by conformal structure
WO2007011306A2 (en) 2005-07-20 2007-01-25 Bracco Imaging S.P.A. A method of and apparatus for mapping a virtual model of an object to the object
US20060293557A1 (en) 2005-03-11 2006-12-28 Bracco Imaging, S.P.A. Methods and apparati for surgical navigation and visualization with microscope ("Micro Dex-Ray")
KR100884066B1 (en) 2007-03-30 2009-02-19 한국전자통신연구원 System and method for comparing image based on singular value decomposition
JP5595070B2 (en) * 2010-02-26 2014-09-24 三菱重工業株式会社 CAD data transmission method, CAD data reception method, CAD data transmission device, CAD data reception device, CAD data transmission program, CAD data reception program, and data structure
JP6161004B2 (en) * 2011-03-09 2017-07-12 国立大学法人大阪大学 Image data processing apparatus and transcranial magnetic stimulation apparatus
KR101638477B1 (en) 2014-09-19 2016-07-11 주식회사 고영테크놀러지 Optical tracking system and registration method for coordinate system in optical tracking system
DE102015210453B3 (en) * 2015-06-08 2016-10-13 Bitmanagement Software GmbH METHOD AND APPARATUS FOR GENERATING DATA FOR A TWO OR THREE-DIMENSIONAL PRESENTATION OF AT LEAST ONE PART OF AN OBJECT AND FOR PRODUCING THE TWO OR THREE-DIMENSIONAL PRESENTATION AT LEAST THE PART OF THE OBJECT
JP6503906B2 (en) * 2015-06-10 2019-04-24 富士通株式会社 Image processing apparatus, image processing method and image processing program
WO2016206942A1 (en) * 2015-06-25 2016-12-29 Koninklijke Philips N.V. Image registration

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5999840A (en) * 1994-09-01 1999-12-07 Massachusetts Institute Of Technology System and method of registration of three-dimensional data sets
CN105225219A (en) * 2014-06-25 2016-01-06 联想(北京)有限公司 Information processing method and electronic equipment
CN104318552A (en) * 2014-10-15 2015-01-28 北京理工大学 Convex hull projection graph matching based model registration method
CN105261052A (en) * 2015-11-03 2016-01-20 沈阳东软医疗系统有限公司 Lumen image unfolding and drawing method and device
CN105654029A (en) * 2015-11-30 2016-06-08 辽宁师范大学 Three-dimensional point cloud auricle identification method for increasing identification precision and efficiency

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BASTIAN RIECK ET AL: "Unwrapping highly-detailed 3D meshes of rotationally symmetric man-made objects", 《XXIV INTERNATIONAL CIPA SYMPOSIUM》 *
PARK,TAEYONG ET AL: "Rapid Rigid registration method between intra-operative 2D XA and pre-operative 3D CTA images，journal of Korea multimedia society", 《JOURNAL OF KOREA MULTIMEDIA SOCIETY》 *
乔学军等: "基于参数化的三维颅面配准", 《计算机应用》 *
尹吉林等: "一种改进的曲面配准方法的研究", 《微型电脑应用》 *

Also Published As

Publication number Publication date
CN109564693B (en) 2024-04-09
US20210287331A1 (en) 2021-09-16
EP3499464A4 (en) 2019-08-21
EP3499464B1 (en) 2021-06-23
EP3499464A1 (en) 2019-06-19
KR101812001B1 (en) 2017-12-27
JP6785362B2 (en) 2020-11-18
JP2019533218A (en) 2019-11-14
WO2018030781A1 (en) 2018-02-15
US11481867B2 (en) 2022-10-25

Similar Documents

Publication Publication Date Title
Bartol et al. A review of body measurement using 3D scanning
KR101519775B1 (en) Method and apparatus for generating animation based on object motion
CN108875524B (en) Sight estimation method, device, system and storage medium
Breuer et al. Hand gesture recognition with a novel IR time-of-flight range camera–a pilot study
CA3051795A1 (en) Apparatus, method, and system for alignment of 3d datasets
ES2693028T3 (en) System and method for deriving accurate body size measurements from a sequence of 2D images
US6664956B1 (en) Method for generating a personalized 3-D face model
CN109564693A (en) Three-dimensional data integrating apparatus and method
JP2019534510A5 (en)
Canessa et al. Calibrated depth and color cameras for accurate 3D interaction in a stereoscopic augmented reality environment
JP5924862B2 (en) Information processing apparatus, information processing method, and program
US20100328308A1 (en) Three Dimensional Mesh Modeling
KR20150113751A (en) Method and apparatus for acquiring three-dimensional face model using portable camera
US10776978B2 (en) Method for the automated identification of real world objects
KR20150024899A (en) Avatar construction using depth camera
WO2006049147A1 (en) 3d shape estimation system and image generation system
JP2019096113A (en) Processing device, method and program relating to keypoint data
KR20150107063A (en) 3d scanning system using facial plastic surgery simulation
Jia et al. Scene depth perception based on omnidirectional structured light
Wang et al. Accuracy of monocular gaze tracking on 3d geometry
JP6770208B2 (en) Information processing device
Seo et al. A breast modeler based on analysis of breast scans
Liu et al. A portable projection mapping device for medical augmented reality in single-stage cranioplasty
Liem et al. Coupled person orientation estimation and appearance modeling using spherical harmonics
US20180274918A1 (en) Three-dimensional measurement apparatus, non-transitory computer readable medium, and three-dimensional measurement system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 15 / F, 14 / F, No.53, Jiashan digital 2 Road, Jinchuan District, Seoul, Korea

Applicant after: Gaoying Technology Co.,Ltd.

Applicant after: Qingbei University School Industry University Cooperative Force

Address before: 15 / F, 14 / F, No.53, Jiashan digital 2 Road, Jinchuan District, Seoul, Korea

Applicant before: KOH YOUNG TECHNOLOGY Inc.

Applicant before: Qingbei University School Industry University Cooperative Force

CB02 Change of applicant information
CB02 Change of applicant information

Address after: Han Guoshouershi

Applicant after: Gaoying Technology Co.,Ltd.

Applicant after: Qingbei University School Industry University Cooperative Force

Address before: 15 / F, 14 / F, No.53, Jiashan digital 2 Road, Jinchuan District, Seoul, Korea

Applicant before: Gaoying Technology Co.,Ltd.

Applicant before: Qingbei University School Industry University Cooperative Force

CB02 Change of applicant information
GR01 Patent grant