CN110462681B

CN110462681B - Multiple surfaces for physical-to-image/image-to-physical registration and image verification

Info

Publication number: CN110462681B
Application number: CN201780087268.5A
Authority: CN
Inventors: 林宇恒; 熊体超; 陆伟权; 鲍浩能; 颜旭东
Original assignee: Versitech Ltd; Hong Kong Baptist University HKBU
Current assignee: Versitech Ltd; Hong Kong Baptist University HKBU
Priority date: 2017-02-23
Filing date: 2017-02-23
Publication date: 2023-06-30
Anticipated expiration: 2037-02-23
Also published as: CN110462681A; WO2018152742A1

Abstract

The present invention relates to novel image registration. The rigid registration is based on a plurality of surfaces arranged in a 3D relationship in a cartesian coordinate system by fiducial markers. This registration process provides more accurate registration or requires smaller fiducial markers to achieve the same level of registration accuracy as compared to surface-based registration. Furthermore, the fiducial markers enable measurement of coordinates in the physical domain; thus, registration errors can be measured by comparing the coordinates of specific points in both the physical and virtual domains. This has potential application in comparing imaging physical to image (PI) or printing images to physical (IP) modalities, for example, for comparing the accuracy of 3D models created by computed tomography and optical scanners. Other potential applications also include assisting surgical navigation in dental implant placement. Thus, the proposed fiducial markers have potential developments in virtual reality applications.

Description

Multiple surfaces for physical-to-image/image-to-physical registration and image verification

Technical Field

The invention relates to a method based onThree-dimensional(3D) The plurality of surfaces of the relational arrangement are registered as images of the fiducial markers.

Background

Virtual images are an extension of the real physical world, and precise connections between physical and virtual domains have many potential applications. A 3D virtual image of a physical object may be generated by a variety of imaging techniques. Non-contact optical scanners are commonly used to scan the surface topography of an object. Multiple 2D images are taken and reconstructed into 3D images (physical to image registration, PI). The accuracy of the surface scan depends on the surface characteristics of the scanner's detector and the object, such as its optical reflection. Before scanning a particular object, it is recommended to calibrate the optical scanner with objects having similar surface characteristics and known surface coordinates. On the other hand, non-reflective powders may be sprayed onto the surface of the object to normalize their surface characteristics. However, the thickness thereof is difficult to control.

In the medical field, surgeons can acquire 3D of their patientsComputed tomography(CT) images to study the interior parts of the human body and plan any required surgical treatment. Will generate the selected from a series of 2D imagesVisual fieldA 3D gray scale image and orthogonal cross-sectional (sagittal, coronal, and horizontal) views of the patient's jaw in (FOV). The surgeon may use surgical planning software to plan and place virtual dental implants at locations that will facilitate biological and aesthetic tooth replacement in CT images. Real-time computer-aided navigation (dynamic) or template-based (static) surgical guidance has been developed to facilitate accurate placement of implants by passing this pre-operative virtual plan to an operating table (image-to-physical registration, IP).

Multiple imaging techniques may be combined for clinical use. While CT provides a 3D image of the internal structure of the skull, optical surface scanning provides a high resolution 3D tooth surface model that can be used as an IP medium. The 3D images can thus be superimposed to complement each other (image-to-image registration, II).

Currently, there are several algorithms for image registration that are common. For example, best fit superposition (intensity-based), point-based, and surface-based registration algorithms. By imaging techniques that match the intensities of the image elements between 2 images, a best fit algorithm is used for II registration. However, it does not allow any PI/IP registration conversion nor II registration between multiple imaging techniques. The best fit algorithm has an average error of 0.16 mm (linear) and 1.07 mm (3D). Lagravre, manuel o et al, "Three-dimensional accuracy of measurements made with software on cone-beam computed tomography images (Three-dimensional accuracy of cone beam computed tomography image measurement with software)", journal of the united states Orthodontics and Dentofacial Orthopedics (orthodontic and dental face orthopaedics) 134.1 (2008): 112-116. Point-based registration involves physically placing some markers on the object. More points will reduce registration errors. See Fitzpatrick et al, "Predicting error in rigid-body point-based registration (predicting errors in rigid body point based registration)",IEEE Transactions on Medical Imaging（IEEE (institute of Electrical and electronics Engineers) journal relating to medical imaging) 17.5 (1998): 694-702, which is incorporated herein by reference in its entirety, hereinafter referred to as "Fitzpatrick et al, 1998". However, this will increase the computational power/time required for registration and may make it practically very difficult to place enough points in a limited space such as the oral cavity. The method has an error of 0.3 mm of the center point in the detected physical domain and an error of 0.4 mm in the CT scan. See Maurer, calvin r et al, "Registration of head volume images using implantable fiducial markers (registration of head volume images using implantable fiducial markers)",IEEE Transactions on Medical Imaging16.4 Pages 447-462, which are incorporated by reference in their entirety (1997), hereinafterAbbreviated as "Maurer 1997". For surface-based registration, it is not typically used in PI and IP transformations because it requires a large surface to obtain acceptable registration accuracy and the use of elastic soft tissue in registration. The use of an optical surface scanner in capturing the surface of an object may be problematic due to the reflective nature of the surface of the object and this error may result in capturing the real surface of the object, which is difficult to verify by existing registration methods. See Ireland, A.J. et al, "3D surface imaging in dentistry-what we are looking at (3D surface imaging in dentistry-what we are looking at)",British Dental Journal（british dental journal) 205.7 (2008), pages 387-392, which is incorporated herein by reference in its entirety, hereinafter abbreviated as "Ireland et al, 2008".

Ideally, the image registration error should be minimal and a large portion of the safety margin should be reserved for the surgical treatment itself, such as vibration of the surgical instrument. When any dental implant is placed, the usual clinical safety margin will be 2mmAnd any registration errors in computer-assisted surgery may already account for half of this safety margin. Furthermore, all of these image registrations should be verified physically, however the current point-based/surface-based fiducial mark arrangement does not allow for easy physical coordinate measurements.

The present inventors have found that by using a plurality of surfaces (1cm×1cm×1cmManual and semi-automatic registration of one cube corner/corner cube ("CC")) achieves promising preliminary results. See Lam, walter YH et al, "Validation of a Novel Geometric Coordination Registration using Manual and Semi-automatic Registration in Cone-beam Computed Tomogram (verification of new geometrically coordinated registration using manual and semi-automatic registration in cone beam computed tomography)",IS&T Electronic Imaging（IS&t-electricity Sub-imaging) 2016.14 (2016): 1-6, which are incorporated herein by reference in their entirety. The disclosed manual and semi-automatic registration is used to detect cube corners.

By usingSimPlant ProSoftware-implemented Manual Registration (MR), i.e. manual registration is used to registerx，y，z) The axes are registered to three orthogonal windows in SimPlant Pro. Semi-automatic registration (SR) is used forMeshLabAnd3D slicer(both free and open source software) to make sure that by first using the merlab and then fitting these surfaces to thex，y，z) An axis defines each surface.

For manual and semi-automatic registration, registration errors of 0.56 mm and 0.39 mm were found, respectively. The work predicts that registration errors can be further reduced if the registration can be performed automatically. Furthermore, the registration allows for easy physical measurement of the coordinates of a specific point. This facilitates comparison with the virtual domain and errors of the registration process can be calculated. The registration process also allows for selection of better image modalities (e.g., different types of optical scanners) by comparing their PI registration errors. It is assumed that this technique is likely to be converted to ISO standards for measuring the accuracy of the image modality/system. This may also be potentially useful in measuring the print (IP) quality of future 3D printers. Clinically, a clinician can evaluate the accuracy of various surgical navigation or guidance methods by comparing preoperative and postoperative series of images.

Disclosure of Invention

The present invention relates to the use of multiple surfaces (e.g., three orthogonal surfaces forming a corner of a cube) for series image registration and calibration. These surfaces have an inherent relationship that defines the origin of the Cartesian coordinate system and the x-y-axis and z-axis. Physically, these surfaces are fitted to a subject such as a patient wearing a surgical instrument/template. This allows physical coordinate measurements to be made of a particular point of an object by a Coordinate Measuring Machine (CMM). In a computer environment, these surfaces are used to register a 3D image of a scanned surface to cartesian coordinates. Thus, object locations in physical and virtual computer environments may be matched to the same coordinate system. By comparing the coordinates of the selected points, the registration error (target registration error TRE) of the method can be identified. By using the invention, the automatic device is usedCube corner(CC) defining Cartesian coordinates with the corner point as the origin O and three line angles asx，y，z) A shaft. The automatic registration reduces human error.

With the present invention, automatic registration is achieved with stereotactic/fiducial markers that are +/-color by their surface smoothness in both CT and optical surface scanningRed-green-blue(RGB) +/-radiopacity is identifiable. The identification can be carried out byMATLABSoftware-bound computers. The accuracy of registration is increased by adding two flat surfaces away from the CC, which effectively increases the area of registration while allowing for the use of smaller sized fiducial markers.

With knowledge of registration errors of multiple surfaces, the method can be extended to measure the accuracy of imaging techniques (i.e., PI), such as computed tomography and optical scanners. Thus, the method allows a direct comparison of the virtual domain with the physical domain and allows calibrating the imaging device. Furthermore, the method defines the reference coordinates of the object and allows comparing the changes in the series of images. Thus, it can be used to compare the progression of a disease and evaluate the outcome of a treatment, such as surgical navigation or guidance.

The structure may be modified by Lego ^® Building blocks ("flat blocks") or "wall corners" are implemented as fiducial marks. CC facilitates defining cartesian coordinates in both the physical domain (real patient/object) and in multiple imaging domains such as CT and optical surface scanners. This process is called "registration". Thus, both the physical and image will share the same coordinates in perfect registration.

The present invention proposes a new algorithm for automatically registering fiducial markers (surfaces) on an instrument (including a template) and in a series of images and reducing errors in IP, II and PI registration. The dental surgical instrument may be a tooth support device that fits over the teeth and guides the oral surgeon to properly place the implant in the patient's mouth. They are typically made of acrylic, polypropylene or similar materials. The proposed surface on the device/scaffold defines cartesian coordinates in the physical and virtual domains and allows verification of IP, II and PI registration (TRE). Individual imaging and/or printing techniques may be verified and compared to other techniques.

A plurality of main surfaces (1)cm×1cm) Are arranged in a 3D orthogonal relationship, either physically or virtually. Physically, this may be a machine milled cube corner or a commercially available cube block, such as Lego ^® "Flat block" of building blocks. Alternatively, the cube corner spaces (inverted cube corners) can be milled or reused with commercially available Lego ^® "wall angle" of building blocks. Inverted square angles may be useful for locating the tip of a surgical instrument, and may be useful in IP. Virtually, this can be easily designed by Computer Aided Design (CAD) software.

Two complementary geometries, such as flat surfaces, may be formed with a circle/ellipse (centroid) having a diameter of 0.5 cm. Physically, they are tailored by milling the plate, e.g. from acrylic acid or commercially available Lego ^® The flat block of the building block is a flat plate. Alternatively, two "wall angles" may be used. Virtually, this can again be easily designed by software.

The golden ratio/triangle/geometry algorithm may be included between i) the origin (corner point) of the major surface and two complementary centroids or ii) the three surfaces forming the primary fiducial mark. Physically, there are manual instruments (e.g., golden ratio calipers) to facilitate their positioning. Virtually, this can also be easily designed by software.

For PI registration, these physical surfaces may be smoothed (for optical scanners and computed tomography), painted (for optical scanners), or coated with a film of radiopaque material (for computed tomography), depending on the nature of the imaging modality. The algorithm for detecting the surface may be based on surface smoothness, color (e.g., RGB color), and radiopacity (gray scale).

This registration process provides a more accurate registration compared to a surface-based registration of similar regions. In other words, smaller fiducial markers are required for the same level of registration accuracy. Furthermore, the fiducial markers enable measurement of coordinates in the physical domain; thus, registration errors (TREs) can be measured by comparing the coordinates of specific points in both the physical and virtual domains. This has potential application in comparing imaging (PI)/printing (IP) modalities. For example for comparing the accuracy of 3D images created by computed tomography and by an optical scanner.

Currently, there are several algorithms commonly used for image registration. For example, best fit superposition (intensity-based), point-based, and surface-based registration algorithms. By matching the intensities of the image elements between 2 images, a best fit algorithm is used for II registration. However, it does not allow any PI or IP registration transformations. The best fit algorithm has an average error of 0.16 mm (linear) and 1.07 mm (3D). See Lagravere et al, 2008. Point-based registration involves physically placing some markers on the object. More will reduce registration errors. See Fitzpatrick et al, 1998. However, this will increase the computational power/time required for registration and may in practice be difficult to place enough points in a limited space such as the oral cavity. The method has an error of 0.3 mm of the center point in the detected physical domain and an error of 0.4 mm in the CT scan. See Maurer 1997. For surface-based registration, it is not typically used in PI and IP transformations, and it requires a large surface to obtain acceptable registration accuracy and the problem of using elastic soft tissue in registration. The use of an optical surface scanner in capturing the surface of an object may be problematic due to the surface reflectivity properties of the object, and errors may result in capturing the real surface of the object, which is difficult to verify by existing registration methods. See Ireland 2008.

Ideally, the image registration error should be minimal and a large portion of the safety margin should be reserved for the surgical treatment itself, such as vibration of the surgical instrument. When any dental implant is placed, the usual clinical safety margin will be 2mmAnd any registration errorsMay already account for half of the safety margin. Furthermore, all of these image registrations should be verified physically, however the current point-based/surface-based fiducial mark arrangement does not allow for easy physical coordinate measurements.

Drawings

The foregoing and other objects and advantages of the invention will become more apparent when taken in conjunction with the following detailed description and drawings wherein like reference characters designate the same elements throughout the several views, and wherein:

FIG. 1 illustrates a cube corner fiducial marker according to the invention in which three surfaces are arranged in an orthogonal relationship matching the x-y-z axis of Cartesian coordinates;

fig. 2A shows automatic surface detection by surface roughness, and fig. 2B shows automatic surface detection by color;

FIG. 3A shows CT images of milled multiple surface fiducial marks (one cube corner and two flat surfaces) on an oral appliance/bracket on the maxillary (upper) dental arch of a patient, and FIG. 3B shows automated software-determined images of the fiducial mark surfaces, their smoothness for image registration;

FIG. 4A shows a commercially available Lego used as a fiducial marker on an oral appliance ^® Building blocks "flat blocks", FIG. 4B is a CT image of a top view of a "wall angle" mark and implement, and FIG. 4C is a schematic view of a computer generated mark and its positioning;

FIG. 5A is a software-generated image of a milling principal cube corner and two centroid-supplementing surfaces arranged in a triangular algorithm for image registration in accordance with the invention, FIG. 5B is an example of a triangle/golden triangle ratio algorithm, and FIG. 5C is an example of a principal fiducial marker in which three centroid surfaces are arranged in both orthogonal and triangular algorithm relationships; and

fig. 6 shows the use of a surface plate to position a milled flat surface (i.e., a "flat block") on an oral appliance/stand.

Detailed Description

The present invention is to use multiple geometric surfaces arranged in certain 3D relationships as fiducial markersIs described. The primary arrangement uses three surfaces, such as cube corners, arranged in an orthogonal relationship. The proposed registration process is a modification to the surface-based registration and the 3D relationship between these surfaces allows for a more accurate registration than the surfaces alone. Thus, smaller fiducial markers may be used that have the same level of registration error as the current fiducial markers. The strict registration process is applicable toImage-to-physical（IP）、Image-to-image(II) A method for producing a polypeptidePhysical to image(PI) registration.

Multiple (primary) surface fiducial markers may be physically or virtually attached to an object and will define the Cartesian @ of the object in both physical and virtual domainsx，y，z) Coordinates. Points in the objectx，y，z) The coordinates may be physically determined byCoordinate measuring machineBoth (CMM) and virtual software. The proposed registration process bridges the virtual and physical domains in that they can mutually verify through a comparison of the coordinates of the virtual and/or physical measurements.

The algorithm detects the surface roughness, color or radiopacity of the selected surface and automatically fits them to the Cartesian coordinate systemx，y，z) A shaft. In particular, the computer program will automatically detect these surfaces and minimize manual errors in the registration process.

In deriving the algorithm, the following assumptions are made:

A1. the registration in the physical domain may be compared to the registration in the virtual domain.

A2. Registration at the marker A, B, C causes registration errors.

A3. Registration error is less than or equal to clinical error

Definition 1:rectangular markers in Real Domain (RD)Is defined as

And is also provided withRectangular marks in Virtual Domain (VD)

Is defined as +.>

Wherein->

And->

Is thatm×nMatrix, which representsAdditive clinical errorAnd->

，/>

，1≤x≤m，1≤y≤n。

Definition 2: given any twom×nThe marking is carried out by a marking device,

and->

Between two marksk Norm metricIs defined as

Wherein the method comprises the steps ofN=m·n。

Remarks: for having radius r and centroid in RD

Can be defined as a circular mark of (2)

Wherein, if for diameter2r，/>

Then->

。Circular marks in VD/>

Is defined as +.>

Wherein->

And->

. This can be seen as a disc embedded in a square, with any pixel outside the disc equal to zero.

Definition of the definition3: given any ofmxnMarking

。MIs an ordered sequence of all elements of

。

Matrix of matrixShift ofOperation ofHProduction ofm×nNew marks are made such that

Is defined as

，

，

…

。

Given a given

And->

Wherein->

Is one of the shift matrices, and the 1-norm metric therebetween is defined as

. For any two markers->

、/>

，/>

Wherein->

。

Definition 4: given any twom×nMarking

And->

For->

、/>

There is a set of shift marks +.>

Andm·n1 norm metric->

And->

And->

Between (a) and (b)Of displacement difference (Energy)Is defined as +.>

Wherein->

。

Definition 5: three markers in a given RDM ^A 、M ^B 、M ^C And three marks in VD

、/>

、/>

There are several types of errors in the registration from RD to VD:

(a) In x, y, z axesTranslational error：

Wherein the method comprises the steps ofe _A 、e _B 、e _C Respectively from the marksM ^A 、M ^B 、M ^C 。

(b) In x, y, z axesAngle error：

Theorem 6 (qualified registration error): marking of three offers in a given RDM ^A 、M ^B 、M ^C Three marks in VD

、

、/>

So that->

，/>

，/>

Wherein->

，/>

，/>

For translational and angular errors, respectivelye _A （x，y）、e _B （x，y）、e _C （x，y）。

Meanwhile, the 1-norm metric between two markers is as follows:

whereinM ^R And->

The labels in RD and VD, respectively. It can be defined as

（1）1-norm metric between RD and VD markers

A kind of electronic deviceRangeIs->

Wherein the method comprises the steps of

Is a 1-norm metric between two markers.

（2）For e+.0, for 1-norm metric, there isTolerance sigma of registration error _ij . For the followingσ _ij 0, the difference in 1-norm metrics between RD and the markers in RD is at

Within (1), wherein>

，/>

。

Remarks: in this case, acceptable registration errorsσ _ij =0.02mmAnd acceptable clinical error e=2mm. Thus, the first and second substrates are bonded together,σ _ij =0.02mm≤measurement(measurement result). Ltoreq.e=2mm。

Proof of evidence：For (i)There are considerations ofM ^R And

a case of a range of differences in 1-norm metrics between the marks of (1) wherein +.>

Is shifted.

...（a0）。

Case 1: for the following

(a 0) becomes

According to the rule of triangles, |a+b|a|+|b|, we have

。

Case 2: for the following

(a 0) becomes

Since for all real numbers x, y, the value of the X-y is not less than the value of the X-y, therefore we have

。

Case 3: for the following

This is trivial (trivia),>

。

in combination with the results from all of these cases,

...(a1)

wherein the method comprises the steps of

X is 1.ltoreq.x, i.ltoreq.m and 1.ltoreq.y, j.ltoreq.n.

Tolerance of registration

Since for each e +.0,

equivalent to/>

Thus, there is a tolerance for registration errors of the 1-norm metricσ _ij 0 or more, makeM ^R And->

The difference in 1-norm metrics between

Within the range of (2),M ^R and->

The labels in RD and VD, respectively.

Automatic registration algorithms for these orthogonal surfaces reduce any manual errors in image registration. At the position ofCone beam meter Computed tomographyIn (CBCT), a PI registration error of less than 1 was foundmmAnd this accuracy is comparable to the registration based on the current point.

Definition 7 (error limit of golden triangle ratio):

three markers in a given RDM ^A 、M ^B 、M ^C Three marks in VD

、/>

、/>

The presence of markers in RDM ^A 、M ^B 、M ^C Three corresponding centroid points in (a)f _A （x _c ，y _c ）、f _B （x _c ，y _c ）、f _C （x _c ，y _c ) And the markers ∈D in VD>

、/>

、/>

Centroid point +.>

、/>

、/>

. Then, there are three corresponding sets of energies +.>

、

、/>

. In RD, if the Euclidean distance between each pair of centroids isD _A 、D _B 、D _C And the corresponding angles are alpha, beta, gamma, the two golden triangle ratios are derived by sine law as

And cosine law is

Sine and cosine laws are also valid in VD. The two triangular golden ratios are patient-oriented so that they can be used to measure registration errors.

Modification of the main arrangement

For surface-based registration, the use of fiducial markers of a larger surface area will increase registration accuracy. In case of limited space for locating the primary fiducial markers, such as in the oral cavity, two smaller (complementary) geometries, such as circular/oval surfaces (centroids)/wall angles (corner tips), may be added and they should be placed at the same plane/level or at least parallel to one main surface. It is suggested to use golden ratio/triangle/geometry algorithms in the positioning of the two centroid/angular tip surfaces and the major surface to further improve registration accuracy. Alternatively, the primary fiducial markers may be modified by arranging the three centroid surfaces in an orthogonal relationship and using a golden ratio/triangle algorithm to determine the space between their centroid positions fig. 5C. The golden ratio/triangle algorithm is shown in fig. 5B.

A plurality of primary fiducial markers (i.e., cube corners) may be used in rigid registration. However, according to the invention, only one marker is selected to define the cartesian coordinates. Multiple cube corners may be useful for surgical navigation, where a virtual surgical plan is delivered to a real physical patient, which requires accurate calibration of the position of a surgical device (e.g., a drill dental bur) between virtual and physical domains. The dihedral/wall angle provides a solid point during this calibration process.

For the proposed registration procedure, two perpendicular surfaces are sufficient. The third surface may be extrapolated from both surfaces by cartesian points instead (e.g., to define an origin) or by other means.

The cube corner may be used in the following: i) Image registration and ii) linking the virtual and physical domains together. Although image registration is based on multiple surfaces arranged in a 3D relationship (such as a sphere-like geometry), linking virtual and physical domains is a unique function of cube corners that defines the cartesian coordinates of both physical and virtual domains.

The invention may be used in surgical navigation or in guiding surgical procedures. To do this, conventional optical scanners/computed tomography are used, which typically involve such a procedure. In addition, these scanners are connected or image files are transferred to a computer running a program that allows for preoperative surgical planning. To utilize the present invention, a computer runs an additional software module that processes the auto-registration based on the scanner's image captured by the computer. The process comprises the following steps:

(1) In the physical domain, the fiducial marks may be customized by milling, or commercially available Lego may be used ^® Building blocks. In the virtual domain, these markers can be easily designed using appropriate software. One of the marks is a cube corner 10, as shown in fig. 1 and 2B.

(2) These indicia are attached to the oral appliance/support 12, which oral appliance/support 12 is repeatedly placed over the patient's teeth 14. The positioning of the markers on the instrument/stent is critical for image registration and linking the virtual and physical domains together. As shown in fig. 6, the cube corners and two planes are positioned in one plane by the machine milled surface plate. Furthermore, they may be arranged in a relationship such as golden ratio, triangular and other geometric algorithms, as shown in fig. 5A and 5B, by using golden ratio calipers or rectangular Lego blocks, or the like. The marking surface is polished as shown in fig. 2A or painted as shown in fig. 1 and 2B to aid in automatic identification by MATLAB software. For detection in Computed Tomography (CT), the coating has a different radiopacity than the marker, whereas for optical surface scanning, colors such as red, green and blue may be applied to the marker surface. Auto-alignment may be accomplished in a separate computer using data acquired from the scanning device(s) employed. Alternatively, the indicia and oral appliance/support may be virtually designed and then printed out (stereolithography). These markers can be easily located virtually by software.

(3) For the imaging method, the oral appliance with the fiducial markers is digitized and automatically identified by registration software. The accuracy of registration is increased by adding two flat surfaces or blocks 16 at a distance from the cube corner. The CT scan and the optical surface scan of FIGS. 3A and 3BBoth images are shown. In FIG. 4A, shown in Lego ^® Prototype in the form of flat block 16, and Lego is shown in FIG. 4B ^® The optical surface of the wall angle 15 is scanned. The positions of these "corners" 15 may be guided by rectangular Lego blocks (examples of geometric algorithms) shown in fig. 4C. The cube corners facilitate defining Cartesian coordinates in both the physical domain (real patient) and in multiple imaging domains, such as CT and optical surface scanners. The coordination process is referred to as "registration". Thus, when there is perfect registration, both the physical structure and the image will share the same coordinates.

Fig. 5A shows a CT image of the improved stent of the present invention, i.e. a model image with two centroid-supplementing discs and one milling principal cube corner, the arrangement of which is modified by the triangle algorithm of the present invention to solve the problem of automatic registration (detection and reconstruction) of the series of images and to reduce errors in the 3D virtual image. Fig. 5B is an example of a triangle/golden triangle ratio algorithm. The corner tips of the cube corners may act as centroids and the distance between the corner tips/centroids A, B and C in the present invention is determined, for example, by causingABEqual toACAnd is also provided withAB(or AC):BCis 1.6181 to 1 to incorporate the triangle algorithm. By employing this relationship between centroids, the size of the fiducial markers can be reduced to a minimum while maintaining a similar degree of registration accuracy.AB、AC、BCIs flexible and may be in golden triangle ratio in both physical and virtual domains. Fig. 5C is an example of multiple surfaces arranged in an orthogonal relationship and their centroids arranged in a triangular ratio.

(4) The clinician then virtually designs a plan for the surgical procedure.

(5) For the printing method, the virtually designed oral appliance can be 3D printed.

(6) During a surgical procedure, the physical surgical instrument may be calibrated to link both physical and virtual domains by fiducial markers (such as cube or wall angles). The wall angle in fig. 4B and 4C allows for calibration of the surgical instrument to the virtual planning domain. During calibration, the tip of the surgical instrument may be placed in a corner and its position captured by the tracking system, and this may correlate the physical instrument with virtual imaging. Thus, virtual surgical planning can be performed in the real physical world (image-to-physical registration).

(7) After surgery, the patient may wear the oral appliance/support. They may then be digitized again. Image registration allows image-to-image (II) comparisons to reveal surgical results. Because the physical and virtual domains are linked together, the patient can avoid any post-operative CT, use only optical surface scanning to determine the surgical outcome (e.g., implant location) and compare the preoperative planning in CT with respect to CC.

Verification of registration accuracy and instrument calibration

The proposed fiducial markers may be attached to a test object. Verification of registration (PI, II and IP) accuracy and instrument calibration may be performed by comparing cartesian coordinates of specific points obtained by the physical CMM and by virtual software.

This procedure can be used to calibrate imaging/printing instruments and potentially serve as an ISO standard for both imaging and printing modalities. By comparing the physical-virtual errors (target registration errors TRE) of the imaging/printing instrument, it is possible to select a more accurate imaging/printing instrument.

While the invention has been particularly shown and described with reference to preferred embodiments thereof; it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A method of registering or aligning a physical object in a cartesian coordinate system with a virtual three-dimensional image, comprising the steps of:

positioning a cube corner on a physical object as a fiducial marker, the cube corner having one of a smooth surface and a surface coated with a different color/radiopacity, at least two of the cube corner surfaces being orthogonal to each other;

scanning the physical object to capture an image of the physical object and a surface of the cube corner;

automatically identifying surfaces of cube corners in the image based on one of their smoothness and color/radiopacity; and

the virtual cartesian coordinates are aligned with the capture surface of the cube corner in the image.

2. The method of claim 1, wherein one of the surfaces orthogonal to each other is a major surface, and further comprising the steps of

Before the step of scanning, two flat surfaces or blocks are added to the physical object at a distance from the cube corner, the blocks lying in the same plane as the major surfaces of the cube corner,

locating two planar surfaces or blocks and a major surface by using a golden ratio/triangle/geometry algorithm;

automatically identifying flat surfaces and cube corners of the block; and

the positions of the block and cube corners are used to help align the virtual cartesian coordinates.

3. The method of claim 2, wherein the flat surface or block has a surface with a centroid such as a circle/oval, or another wall/cube corner.

4. The method of claim 1, wherein the cube corners have smooth surfaces and the step of scanning is performed with a Computed Tomography (CT) scanner or an optical surface scanner.

5. The method of claim 1 wherein the cube corners have a colored/radio-opaque surface and the step of scanning is performed with an optical surface scanner detecting red-green-blue colors/computed tomography detecting radio-opacity.

6. The method of claim 1, wherein the step of aligning is performed using a golden ratio/triangle/geometry algorithm between centroid/angular tip positions.

7. A method according to claim 1, wherein the tip of the neutral square angle acts as a centroid during registration, the centroid also coming from one of the major surfaces.

8. The method of claim 1, wherein the cube corner is located on a physical object.

9. The method of claim 1, wherein the physical object is an oral appliance or a stand.

10. The method of claim 1, wherein the cube corner has only two orthogonal surfaces and the third orthogonal surface is extrapolated from the two surfaces.

11. The method of claim 1 wherein the cube corners are made by mechanical milling of flat acrylic and polypropylene sheets.

12. The method of claim 1, wherein the surface of the cube corner comprises a building block wall angle.

13. A method of image-to-image registration or alignment, wherein images are selected from a series of images of an object at different points in time in a cartesian coordinate system, the method comprising the steps of:

positioning virtual cube corners in each image as fiducial markers, the cube corners having one of a smooth surface and a surface coated with a different color/radiopacity, at least two of the cube corner surfaces being orthogonal to each other;

automatically identifying the surface of the cube corners in each image based on one of their smoothness and color/radiopacity; and

the series of images are aligned according to the cube corners in each image.

14. The method of claim 13, wherein the virtual cube corners have a smooth surface and computer software to detect surface roughness.

15. The method of claim 13, wherein the virtual cube corners have a colored/radiopaque surface and the step of scanning is performed with computer software that detects color or radiopacity in the image.

16. The method of claim 13, wherein the step of aligning is performed using a golden ratio/triangle/geometry algorithm between centroid/angular tip positions.

17. A method according to claim 13, wherein the tip of the neutral square angle acts as a centroid during registration, the centroid also coming from one of the major surfaces.

18. The method of claim 13, wherein the surfaces of the cube corners and flat surface blocks are building block flat blocks.

19. A method of forming a three-dimensional object based on a three-dimensional image in a cartesian coordinate system, comprising the steps of:

positioning virtual cube corners in the image as fiducial markers, the virtual cube corners having one of smooth surfaces and if the image originates from a physical object, the surfaces are coated with different colors/radiopacity, at least two of the surfaces of the virtual cube corners being orthogonal to each other;

automatically identifying the surfaces of virtual cube corners in the image based on one of their smoothness and color/radiopacity; and

milling or 3D printing the physical object based on the image, such that the cube corners in the physical object are aligned with virtual cartesian coordinates in the image,

whereby the cube corners in the physical object can be aligned with the physical cartesian coordinates in a Coordinate Measuring Machine (CMM).

20. The method of claim 19, wherein the step of aligning is performed using a golden ratio/triangle/geometry algorithm between centroid/angular tip positions.

21. A method according to claim 19, wherein the tip of the neutral square angle acts as a centroid during registration, which centroid may also be from one of the major surfaces.

22. The method of claim 19, wherein printing is an additive manufacturing method.

23. The method of claim 19, wherein milling is subtracted from the block.

24. The method of claim 23, further comprising the step of adding two auxiliary wall angles to the model at a distance from the main wall angle, the auxiliary wall angles lying in the same plane as the surface of the main wall angle,

locating the two auxiliary wall corner cubes and the main wall corner by using a golden ratio/triangle/geometry algorithm;

automatically identifying a planar surface of a cube corner; and

the position of the primary cube corner is used to help align the virtual cartesian coordinates.

25. The method of claim 23, wherein the surgical procedure is a dental procedure and the model is an oral appliance positioned on a patient's teeth.

26. A method for planning a guided navigation surgical procedure, comprising the steps of:

scanning the body part by computed tomography to form a virtual model of the body part;

a plan for the surgical procedure is virtually designed on the image,

positioning one or more wall angle fiducial markers on the virtual model, the wall angle having an origin that provides a positive stop for the physical surgical instrument;

milling or printing the fiducial markers if they originate from the virtual model; and

during a surgical guidance operation, the physical surgical instrument is calibrated to both the physical and virtual domains by fiducial markers.

27. A method for evaluating the outcome of a surgical procedure, comprising the steps of:

forming a virtual model of the body part and one or more cube/wall corner fiducial markers by computed tomography or surface scanning after surgery, the cube/wall corners having one of smooth surfaces, at least two of the cube/wall corner surfaces being orthogonal to each other;

automatically identifying the surface of the cube/wall corner in the image based on one of their smoothness and color/radiopacity;

aligning the virtual cartesian coordinates with a capture surface of a cube/wall corner in the image; and

the surgical outcome of interest is compared to the plan in preoperative computed tomography.

28. A method for assessing the accuracy of an imaging/printing modality, comprising the steps of:

locating cube/wall angle fiducial markers on the reference model;

forming a test model by scanning/printing a physical/virtual reference model and fiducial marks, the cube/wall corners having one of smooth surfaces, at least two of the cube/wall corner surfaces being orthogonal to each other, the surfaces being coated with different colors/radiopacity if the physical object is scanned;

aligning the physical/virtual Cartesian coordinates with the surface of the cube/wall corner; and

the coordinates of the selected points of the physical/virtual domain are compared.