WO2019227215A1

WO2019227215A1 - Method, system and kit for vision based tooltip calibration

Info

Publication number: WO2019227215A1
Application number: PCT/CA2019/050735
Authority: WO
Inventors: Doron Dekel
Original assignee: Claronav Inc.
Priority date: 2018-05-30
Filing date: 2019-05-30
Publication date: 2019-12-05

Abstract

Surgical systems, kits and methods for calibrating a tooltip relative to a surgical device are provided. The surgical device can interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips. The systems and kits include a camera securable to the surgical device and an image processor for receiving an image from the camera and processing the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image. The method involves securing a camera to the surgical device, operating the camera to provide an image of the tooltip, providing the image to an image processor, and operating the image processor to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip.

Description

METHOD, SYSTEM AND KIT FOR VISION BASED TOOLTIP CALIBRATION

PRIORITY

[0001] This application claims the benefit of U.S. provisional Patent Application No. 62/678,249, filed May 30, 2018, which is incorporated herein by reference in its entirety.

FIELD

[0002] The described embodiments relate to the field of dentistry, in particular, the field of dental navigation systems.

BACKGROUND

[0003] Dental navigation systems are well known, available commercially, and described in US patents 9, 125,624 and 9,402,691 , and many other publications.

SUMMARY OF THE INVENTION

[0004] In accordance with aspects of embodiments of the present invention, a digital video camera can be rigidly attached, or mounted within, an optically tracked dental handpiece such that the tips of tools inserted in the handpiece’s rotating head will be visible in the camera’s image. The camera can be connected to a digital processor. Using a mapping of the handpiece head rotation axis to image coordinates and an edge detection algorithm, the processor can detect the presence of a tooltip inserted into the head of the handpiece, and, when the handpiece is present, can measure the length of the tooltip and, optionally, its diameter and/or shape. These parameters can then be reported to the processor of a surgical navigation system, which can use them as input in performing its guidance functions.

[0005] An example method for calibrating a tooltip relative to a surgical device, the surgical device being operable to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips, involves:

securing a camera to the surgical device, and positioning and orienting the camera to provide an image containing a tooltip projection of at least part of each of the tooltips in the set when that tooltip is held by the surgical device; operating the camera secured to the surgical device to provide the image containing the tooltip projection of the at least part of the tooltip held by the surgical device;

providing the image to an image processor; and

operating the image processor to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip.

[0006] In at least one embodiment, the surgical device can have a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip can be rotatable about an axis of rotation to perform a surgical procedure when held in the chuck.

[0007] In at least one embodiment, operating the image processor to process the image can also involve determining a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension can be at least partly defined relative to the axis of rotation.

[0008] In at least one embodiment, determining the location on the surface of the tooltip projection can also involve determining an image location on an edge of the tooltip projection on the camera image, and operating the image processor to process the image can also involve:

determining and storing in memory mapping parameters enabling mapping between image locations along a projection of the axis of rotation in the image and locations along the axis of rotation in the coordinate system associated with the surgical device;

retrieving the mapping parameters from memory; and

using the retrieved mapping parameters to map the image location to a location in the coordinate system associated with the surgical device.

[0009] In at least one embodiment, the method can also involve:

storing, for each tooltip in the set, an associated descriptor comprising tooltip projection recognition parameters and a position in a coordinate system associated with the surgical device;

and operating the image processor can also involve: selecting the descriptor associated with the tooltip held by the surgical device based on the tooltip recognition parameters; and

retrieving the position stored in that descriptor.

[0010] In at least one embodiment, operating the image processor to process the image can also involve detecting at least one edge in the camera image.

[0011] In at least one embodiment, the mapping parameters can also include distance calibration parameters for calibrating different point locations on the axis projection line to different positions in the coordinate system associated with the surgical device. Using the retrieved mapping parameters to map the image location to the location in the coordinate system associated with the surgical device can comprise determining a position of the tip of the one tooltip held in the chuck based at least partly on the distance calibration parameters.

[0012] In at least one embodiment, the pose of the chuck in the coordinate system associated with the surgical device can be adjustable; and

the mapping parameters can include parameters that enable adjusting the mapping according to the adjusted pose of the chuck

[0013] In at least one embodiment, the method can also involve:

operating a pose sensor to measure a pose mapping between a coordinate system associated with the surgical device and a coordinate system associated with the pose sensor; and

determining the location on the surface of the tooltip in the coordinate system associated with the pose sensor.

[0014] In at least one embodiment, the method can also involve:

selecting a region in a coordinate system associated with the pose sensor in which the detection of the tooltip edges in the camera image may be unreliable; and

stopping determining the location on the surface of the tooltip when the tooltip is within the selected region. [0015] In at least one embodiment, the method can also involve detecting when no tooltip is held by the surgical device by operating the image processor to process the image to determine when the image does not contain the tooltip projection.

[0016] In accordance with an embodiment present invention, an example tooltip calibration kit for use with a surgical device designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips includes:

a camera securable to the surgical device to be positioned and oriented to provide an image of at least part of each of the tooltips in the set when that tooltip is held by the surgical device; and

instructions stored on a non-transitory computer-readable media for configuring an image processor receiving the image from the camera to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image.

[0017] In at least one embodiment, the surgical device can include a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip can be rotatable about an axis of rotation to perform a surgical procedure when held in the chuck; and, when the image processor is configured by the instructions stored on the non- transitory computer-readable media, the image processor receiving the image from the camera to process the image can involve configuring the image processor to determine a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension can be at least partly defined relative to the axis of rotation.

[0018] In at least one embodiment, when the image processor is configured by the instructions stored on the non-transitory computer-readable media, determining the location on the surface of the tooltip projection can involve determining an image location on an edge of the tooltip projection on the camera image, and

the image processor can also be configured to:

store in a memory in electronic communication with the image processor, mapping parameters enabling mapping between image locations along a projection of the axis of rotation in the image and locations along the axis of rotation in the coordinate system associated with the surgical device;

retrieve the mapping parameters from memory; and map the image location to a location in the coordinate system associated with the surgical device using the retrieved mapping parameters.

[0019] In at least one embodiment, when the image processor is configured by the instructions stored on the non-transitory computer-readable media, the image processor can also be configured to:

for each tooltip in the set, store in a memory in electronic communication with the image processor an associated descriptor comprising tooltip projection recognition parameters and a position in a coordinate system associated with the surgical device;

select the descriptor associated with the tooltip held by the surgical device based on the tooltip recognition parameters; and

retrieve the position stored in that descriptor.

[0020] In at least one embodiment, the image processor can also be configured to select edge locations in the image based at least partly on the local gradient magnitude, and the camera components can be selected to blur edges of projections of objects further than 5cm behind the tooltip sufficiently to ensure they fail the edge detection test.

[0021] In accordance with an embodiment of the present invention, an example tooltip calibration system for a surgical device designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips includes:

an image processor for, in operation, receiving the image from the camera and processing the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image.

[0022] In at least one embodiment, the tooltip calibration system can also include a pose sensor for measuring a pose mapping between a coordinate system associated with the surgical device and a coordinate system associated with the pose sensor. The pose sensor can be in electronic communication with the image processor. In operation and using the pose mapping between the coordinate system associated with the surgical device and the coordinate system associated with the pose sensor, the image processor can determine the location on the surface of the tooltip in the coordinate system associated with the pose sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Several embodiments will now be described in detail with reference to the drawings, in which:

FIG. 1 is an example illustration of a tooltip calibration system.

FIG. 2 is an example illustration of an image taken by a camera from the tooltip calibration system.

FIG. 3 is a flowchart of an example tooltip calibration method.

FIG. 4 is a flowchart of an example tooltip calibration method.

FIG. 5 is a flowchart of an example tooltip calibration method.

[0024] The drawings, described below, are provided for purposes of illustration, and not of limitation, of the aspects and features of various examples of embodiments described herein. For simplicity and clarity of illustration, elements shown in the drawings have not necessarily been drawn to scale. The dimensions of some of the elements may be exaggerated relative to other elements for clarity. It will be appreciated that for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the drawings to indicate corresponding or analogous elements or steps.

DETAILED DESCRIPTION

[0025] It will be appreciated that numerous specific details are set forth in order to provide a thorough understanding of the example embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the embodiments described herein. Furthermore, this description and the drawings are not to be considered as limiting the scope of the embodiments described herein in any way, but rather as merely describing the implementation of the various embodiments described herein. [0026] It should be noted that terms of degree such as "substantially", "about" and "approximately" when used herein mean a reasonable amount of deviation of the modified term such that the end result is not significantly changed. These terms of degree should be construed as including a deviation of the modified term if this deviation would not negate the meaning of the term it modifies.

[0027] In addition, as used herein, the wording“and/or” is intended to represent an inclusive-or. That is,“X and/or Y” is intended to mean X or Y or both, for example. As a further example, “X, Y, and/or Z” is intended to mean X or Y or Z or any combination thereof.

[0028] It should be noted that the term“coupled” used herein indicates that two elements can be directly coupled to one another or coupled to one another through one or more intermediate elements.

[0029] In some embodiments, aspects of methods described herein, such as method 200 described with reference to FIG. 3 below, may be implemented in hardware or software, or a combination of both. These embodiments may be implemented in computer programs executing on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication component. For example and without limitation, the programmable computer (referred to below as data processor) may be a server, network appliance, embedded device, computer expansion module, a personal computer, laptop, personal data assistant, cellular telephone, smart-phone device, tablet computer, a wireless device or any other computing device capable of being configured to carry out the methods described herein.

[0030] In some embodiments, the communication component may be a network communication interface. In embodiments in which elements are combined, the communication component may be a software communication interface, such as those for inter-process communication (IPC). In still other embodiments, there may be a combination of communication components implemented as hardware, software, and combination thereof. [0031] Program code may be applied to input data to perform the functions described herein and to generate output information. The output information is applied to one or more output devices, in known fashion.

[0032] Each program may be implemented in a high level procedural or object oriented programming and/or scripting language, or both, to communicate with a computer system. However, the programs may be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Each such computer program may be stored on a storage media or a device (e.g. ROM, magnetic disk, optical disc) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein. Embodiments of the system may also be considered to be implemented as a non-transitory computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner to perform the functions described herein. Alternatively, embodiments of the system and method may be limited by replacing some or all of the specific computer programs and software with a gate array approach employing specific hardware, such as processors or chips comprising logical components specifically configured to provide the required processing.

[0033] In some embodiments, a drilling or cutting plan is prepared by a user in reference to a CT (X-ray Computerized Tomography) volumetric image of a patient’s jaw. The plan comprises desired drilling and/or cutting paths for the edge of a drill or saw bit, herein generically named“tooltip”, spun by the rotating head of a dental handpiece. To provide geometrical user guidance, the tooltip can be characterized by its rotation axis direction and a cutting profile, rotationally symmetrical around the axis. When a cylindrical drill is used, the exact cutting profile can be unimportant, and only the location of the tip of the drill, and, optionally, its diameter, may be incorporated into the computation of the guidance display by the navigation system.

[0034] Dental handpieces can drive exchangeable tooltips of different shapes, lengths and diameter, such as burs, drill bits and disc saws. In some cases, the tooltip is inserted in a chuck. When the handpiece is activated, the chuck, and with it the tooltip, can be rotated around the axis of the chuck. In other tools, the tooltip, which can be designed for cutting or sawing, can be attached rigidly to a rapidly vibrating tip of a vibration motor embedded in the handpiece using, for example, a screw.

[0035] For various practical reasons, it can be extremely challenging to track the position of the tooltip directly. Instead, the navigation system can track the pose of the tool’s handle, or body, usually by tracking optical marking on a part that is rigidly coupled to the body. When a chuck is used, its rotation axis is independent of the specific tooltip it holds and its parameters in the tools’ coordinate system can thus be measured infrequently (once per surgical procedure or less). However, the specific location of the tooltip along its rotation axis (tooltip length), and the tooltip’s shape or diameter, can vary with each new tooltip, and thus may need to be calibrated following each tooltip change, or manually selected from a list of stored tooltip calibrations. These per-tooltip steps can be time consuming.

[0036] When a vibrating handpiece is used, the overall shape of each cutting/sawing tooltip can be different, and calibrating the shape, dimensions and location of its cutting edge following each tooltip change can be challenging. Furthermore, failure by a user to indicate to the navigation system that a new tooltip has been inserted may lead to unpredictable guidance errors, leading to a usage hazard.

[0037] According to aspects of embodiments of the present invention, the surgical navigation system can automatically recognize when a change of tooltip has occurred, and automatically obtain the coordinates of one or more locations on the cutting edge of the currently attached tooltip in the coordinate system being dynamically tracked by the pose tracking system, which is associated with the tool’s body.

[0038] In accordance with an aspect of an embodiment of the present invention, FIG. 1 shows a tooltip calibration system 1 for a surgical device 10. As exemplified, the surgical device 10 is a dental handpiece 10. It will be appreciated that the surgical device 10 can be any instrument capable of exchanging tooltips. FIG. 1 shows a side view of the dental handpiece 10. The dental handpiece 10 has a handle 12. The handpiece 10 has a cutting portion 16 with chuck head 18 and a rotating tooltip (or bit) 30. The rotating tooltip 30 can be provided with drive force by a drive motor 14. The drive motor 14 can be detachable, and can be coupled to the handle 12. In some embodiment, the drive motor 14 can be rotatably coupled to the handle 12, such that the drive motor 14 can rotate freely about the handle 12. The chuck head 18 drives the tooltip 30. As exemplified, the tooltip 30 is a dental drill.

[0039] In some embodiments, the chuck head 18 can be movable between a plurality of different operating positions. Within each operating position, axis of rotation 32 can remain in a fixed spatial relationship with other parts of the dental handpiece 10, but the fixed spatial relationship of axis of rotation 32 with other parts of the dental handpiece 10 can change between different operating positions. The fixed spatial relationship can be maintained in any particular operating position. In some embodiments, the chuck head 18 can remain in a fixed spatial relationship with the handle 12. As shown, the handpiece 10 has an optically marked tag 20 with optical markers 22. The optically marked tag 20 has an associated coordinate system 24. The optically marked tag 20 can be rigidly attached to a part of the handpiece 10, for example handle 12, to establish an association of coordinate system 24 with the handpiece handle 12. The chuck 18 being movable in a plurality of operating positions means that the axis of rotation 32 can include a plurality of axes of rotation at different positions and orientations in the coordinate system 24 associated with the surgical device 10.

[0040] The tooltip 30 has a bit drive rotation axis 32. The bit drive rotation axis 32 has a fixed pose in coordinate system 24 when the chuck head 18 is fixed in an operating position. In some embodiments, the surgical device 10 can be designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips 30. In other words, the tooltip 30 can be exchangeable with another exchangeable tooltip 30. The length, shape and diameter of each exchangeable tooltip 30 can vary from one tooltip 30 to another. The chuck or head 18 can interchangeably hold each tooltip 30 in the set of tooltips 30. Each tooltip 30 can be rotatable about the axis of rotation 32 to perform a surgical procedure when held in the chuck 18.

[0041] In the embodiment shown, the tooltip calibration system 1 includes a camera 50. The camera 50 is securable to the surgical device 10. The camera 50 can be positioned and orientated to provide an image of at least part of each of the tooltips in the set when that tooltip 30 is held by the surgical device 10. As exemplified, the camera 50 is mounted on the handpiece 10. It will be appreciated that the camera 50 can be any camera capable of taking images or video of the tooltip 30. As exemplified, the camera 50 can be a miniature (endoscopic) digital video camera. The camera 50 is mounted such that a viewing angle 56 of the camera includes the bottom of the head 18 and the tip 34 of tooltip 30. The viewing angle 56 can include the entire range of lengths of possible tooltips to be tracked. The depth of focus of camera 50 can be adjusted to a distance range 58, such that tooltips 30 will appear sharp while objects in the background behind the tools will be out of focus, and thus show blurred edges in the images captured by the camera 50. In other words, the depth of focus of camera 50 can be adjusted so that camera 50 can capture, within the camera image, a sharp- lined tooltip sub-image corresponding to the tooltip 30. In some embodiments, the distance range 58 can be defined relative to the shortest distance from the camera lens 52 to the rotation axis 32. For example, the distance range 58 may cover the rotation axis 32, and extend slightly beyond the rotation axis 32, by 2 or 3 inches, or by 20% or 50% of the shortest distance from the camera lens 52 to the rotation axis 32, such that objects beyond this range are out of focus.

[0042] In some embodiments, a glass 54 in front of the camera lens 52 can be used to seal the camera against moisture. To prevent fogging that can blur sharp edges in the image, the glass can have a water-repellent coating and can be heated, by, for example, a glass-heating element, when the camera is activated.

[0043] In the embodiment shown, the tooltip calibration system 1 includes an image processor 70. As exemplified, the camera 50 is connected to the processor 70 by a wire 60. It will be appreciated that the processor 70 can be connected to the camera 50 wirelessly. In some embodiments, the processor 70 can be built into the camera 50. In other embodiments, the processor 70 can be built into the surgical device 10. In still other embodiments, the processor 70 can be a part of a surgical navigation system.

[0044] The processor 70 may be any suitable processor, controller or digital signal processor that provides sufficient processing power depending on the configuration, purposes and requirements of the tooltip calibration system 1 . In some embodiments, the processor 70 can include more than one processor with each processor being configured to perform different dedicated tasks. In some embodiments, the processor 70 can be a general purpose processor such as a CPU or a specific purpose processor such as a field programmable gate array (FPGA). For example, the tooltip calibration system 1 can use a gate array approach employing specific hardware, such as processors or chips comprising logical components specifically configured to provide the required processing.

[0045] The processor 70 is configured to process the images captured by the camera. For example, the image processor 70, in operation, can receive the image from the camera 50 and process the image to determine a position in the coordinate system 24 associated with the surgical device 10 of a location on the surface of the tooltip 30 visible in the image. Processing the images can include detecting the presence of tooltips 30 inserted in head 18, and, when present, calculating their length and cutting profile around rotation axis 32.

[0046] FIG. 2 shows an example of an image 90 that can be obtained by the processorfrom camera 50. In handpiece configurations where the camera is in a fixed spatial relationship with tooltip rotation axis 32 the axis 32 is projected to a fixed line 96 in the image. Locations of points (“point locations”) along line 96 represent different distances from the bottom of head 18 shown as head projection 97 in image 90. Being fixed, the line projection 96 on the image 90 and the mapping between point locations and distances can be measured once at the factory and stored in the processor’s memory. The scale (pixels per mm) of the tooltip appearance at each location along line 96 can be directly derived from the length calibration.

[0047] In some embodiments, the handpiece 10 can enable moving, tilting or rolling the head 18 relative to the camera 50. For example, the surgical device 10 can also include at least one actuator (not shown) to move the cutting portion 16 relative to the handle 12 and the camera 50. In particular, the at least one actuator can drive the cutting portion 16 in at least one of a rotating or translating motion to adjust a pose of the cutting portion 16 relative to the handle 12, that is adjust at least one of the tilt, roll, and translation of the cutting portion 16 relative to the handle 12 and the camera 50. Referring to FIG 1 , shown therein is the axis of rotation 32 for the tooltip 30. FIG. 1 also shows the roll axis 33 of the head 18. The roll axis 33 is orthogonal to both the axis of rotation 32, and the tilt axis of the head 18 (not shown). Therefore, the head 18 can roll about the roll axis 33 and tilt about the tilt axis (not shown). Additionally, translation of head 18 can occur in an x axis (parallel to the roll axis 33), and a y axis (parallel to the tilt axis. [0048] The cutting portion 16 includes the tooltip 30, the end of which can be used for cutting into a subject. For example, the tooltip 30 can be a drill, or saw, or other tissue manipulation tool. For example, if a bone needs to be cut, the tooltip 30 can be a saw.

[0049] In such embodiments, the line 96 and the scaling of distances to pixel positions along the line can vary depending on the position and orientation of the head 18 relative to the handle 12 of the handpiece 10, to which the camera 50 is connected. However, these scaling variations are predictable and can be factory calibrated in advance.

[0050] In some embodiments, the head 18 can be attached to the handle 12 of the dental handpiece by an adjustable coupling, and can be adjusted between a plurality of different operating positions. Each operating position can define a different fixed spatial relationship between the head 18 and the camera 50. In some embodiments, the adjustable coupling can automatically communicate the operating position of the head 18 to the processor 70. In other embodiments, the processor 70 can determine the present operating position from elements of the camera image 90, such as, for example, the location in the image 90 of the object corresponding to the head 18.

[0051] It will be appreciated that the operating positions of the head 18 and the tooltip 30 can be represented in a Cartesian coordinate system, such as along the orthogonal tilt, roll, and translation axes described above, or other coordinate systems such as a polar coordinate system. It will also be appreciated that the operating positions of the head 18 and the tooltip 30 can be represented in one coordinate system and converted to another. For example, the position of the tooltip 30 and the head 18 can be represented in a polar coordinate system and converted to a Cartesian coordinate system.

[0052] In some embodiments, the processor 70 can perform edge detection on the image 90. Edge detection can be used to isolate edges that are sharp enough, that have a high local gradient magnitude and extend over a short distance, to be located at a projection silhouette 92 of the tooltip 30. Edges of background objects may be out of focus, and, therefore, blurred and thus not typically detected. For example, the processor 70 can be configured to select edge locations in the image based at least partly on the local gradient magnitude, while the camera components for camera 50 can be selected to blur edges of projections of objects further than a distance behind the tooltip sufficiently to ensure they fail the edge detection test. For example, in some embodiments, the distance behind the tooltip can be greater than 5 cm. In some embodiments, the depth range of the camera can cause edges of background objects more than 3 cm beyond the farthest tooltip held by the surgical device to be sufficiently blurred in the image such that the background objects will fail the edge detection test.

[0053] In some embodiments, a depth of field formula can be used to determine the desired parameters of the camera. The depth of field formula is:

where N is the aperture’s F number, C is the diameter of circle of confusion, d is the distance from the lens focal point to the center of the field, and f is the focal length of the lens.

[0054] The edge detector can be constructed such that edges that spread wider than a certain number of pixels are rejected. For example, if the desired number of pixels to be rejected is 5 pixels, and if a sensor with 1 micron pixels is selected, the C value would be 5 microns, or 0.005mm. If a desired DOF is about 20 mm and a 5 mm lens is used, located 80 mm from the camera, the f number of the lens would then need to be smaller than:

10

3.9

80²

2 0.005

[0055] It will be appreciated that this formula can be used to select another set of design parameters that will meet the desired edge detection rejection threshold.

[0056] Edges that are not part of the silhouette 92, such as projections of features on the surface of the tooltip 30, can be eliminated. For example, projections of features can be eliminated by determining that they are between edges that are further away from axis projection line 96. Any gaps in the edge detected contour can be bridged, or interpolated, using image processing algorithms. Ambiguities in the detection of silhouette 92 due to lighting, image noise, or background edges behind the tooltip can affect the accuracy of the edge detection. These ambiguities can be resolved by combining the edge detection results from sequences of video images, for example by averaging or removing transient detections. In some embodiments, a small light source (LED) can be positioned near the camera 50 to illuminate the tooltip 30 in a sufficiently strong light to promote a clearly visible silhouette line.

[0057] The tip projection 94 of the tip 34 of the tooltip 30 can be detected as the intersection of silhouette 92 with axis projection line 96. Since the tooltip 30 rotates, its cutting volume can be fully described as cylinderformed by rotating a cutting profile. The cutting profile can be obtained by converting to mm, using the pre-measured mm/pixel scale, at multiple locations along axis projection line 96, the larger of two horizontal distances 98, 100 between line 96 and the silhouette 92. As exemplified in FIG. 2, this projection algorithm can be used for drill bits. However, this algorithm can also be used for practically any tooltip shape. For example, this algorithm can obtain an accurate shape of bur tips that are cylindrical, ellipsoid, conic, beveled, tapered with round tip, or any other shape. The shape information can then be made available to the dental navigation system, or any other system that can utilize the shape of the tooltip for its beneficial operation.

[0058] Referring now to FIG. 3, shown therein is a flowchart illustrating an example method 200 for calibrating a tooltip relative to a surgical device. To illustrate the method 200, reference will be made to the surgical device 10 of FIG. 1 and the image 90 of FIG. 2.

[0059] At 210, a surgical device is provided, for example, the dental handpiece 10 as shown in FIG. 1. The surgical device is operable to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips. In some embodiments, as described previously, the surgical device can include a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip being rotatable about an axis of rotation to perform a surgical procedure when held in the chuck. For example, the chuck 18 on the handpiece 10 can receive the interchangeable tooltip 30.

[0060] At 220, a camera is secured to the surgical device. As shown in FIG. 1 , the camera 50 can be secured to the dental handpiece 10. The camera can be positioned and oriented to provide an image containing a tooltip projection of at least part of each of the tooltips in the set when that tooltip is held by the surgical device. As shown in FIG. 2, the image 90 can contain the tooltip projection 92.

[0061] At 230, the camera is operated to provide the image containing the tooltip projection of the at least part of the tooltip held by the surgical device. As in FIG. 2, the camera 50 can take image 90.

[0062] At 240, the image is provided to the image processor and the image processor is operated to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip. As exemplified, the image processor 70 is operated to process the image 90 to determine a position in the coordinate system 24 associated with the surgical device 10 of a location on the surface of tooltip 30. In some embodiments, operating the image processor can involve processing the image to detect at least one edge in the camera image. For example, the edge detection described above can be used to process the image.

[0063] In some embodiments, act 240 of operating the processor can involve determining a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension are at least partly defined relative to the axis of rotation. A location on the surface of the tooltip projection can be determined based on an image location on an edge of the tooltip projection on the camera image 90. The image processor 70 can determine, and store in memory, mapping parameters enabling mapping between image locations along a line projection 96 of the axis of rotation 32 in the image 90 and locations along the axis of rotation in the coordinate system associated with the surgical device 10. In some embodiments, the mapping parameters can include the radii of the tooltip along the rotation axis. Once the processor receives the image, the radii of the tooltip along the rotation axis can be calculated. The radii data can be converted to a cutting profile descriptor in the coordinate system associated with the surgical device 10.

[0064] For example, referring now to FIG. 4, shown therein is a flowchart illustrating example calibration steps for act 240, wherein mapping parameters can be determined and used to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip. [0065] At 242, the processor determines at least one image location on an edge of the tooltip projection on the camera image. For example, the image locations can be taken along the line projection 96 of the axis of rotation in the image 90.

[0066] At 244, the processor determines, and stores in memory, mapping parameters enabling mapping between image locations along the line projection 96 of the axis of rotation in the image, and locations along the axis of rotation in the coordinate system associated with the surgical device. The mapping parameters can be used for calibrating different point locations on the axis projection line 96 to different positions in the coordinate system 24 associated with the surgical device. In some embodiments, the mapping parameters can include distance calibration parameters for calibrating the different point locations on the axis projection line to different positions in the coordinate system associated with the surgical device.

[0067] In some embodiments, the mapping parameters can include a plurality of position-specific mapping parameters, which include, for each operating position in the plurality of operating positions, position-specific mapping parameters for that operating position. For each operating position in the plurality of operating positions, the position-specific mapping parameters for that operating position can include distance calibration parameters for calibrating different point locations on the axis projection line to different positions in the coordinate system associated with the surgical device.

[0068] In some embodiments, as described above, the pose of the chuck 18 in the coordinate system 24 associated with the surgical device 10 can be adjustable. In such embodiments, the mapping parameters can include parameters that enable adjusting the mapping according to the adjusted pose of the chuck.

[0069] At 246, the processor 70 can retrieve the mapping parameters from memory.

[0070] At 248, the processor can use the retrieved mapping parameters to map the image location to a location in the coordinate system associated with the surgical device. In some embodiments, act 248 can involve determining a position of the tip of the one tooltip held in the chuck based at least partly on the distance calibration parameters from act 244. For example, the processor 70 can use the mapping parameters, including the distance calibration parameters, to map the image location of the tip 34 of the one tooltip 30 held in the chuck 18 to the coordinate system 24 associated with the surgical device 10.

[0071] In some embodiments, where the mapping parameters include a plurality of position-specific mapping parameters, the retrieved mapping parameters can be used to map the image location to the location in the coordinate system associated with the surgical device. For each operating position in the plurality of operating positions, this mapping can involve determining a position of the tip of the one tooltip held in the chuck in that operating position based at least partly on the distance calibration parameters for that operating position.

[0072] As described above, act 240 involves using mapping parameters to process the image to determine a position in the coordinate system associated with the surgical device of a location on the surface of the tooltip. As exemplified in act 240, the mapping parameters can be calibrated during use. In some embodiments, the mapping parameters can be pre-calibrated before use, or pre-surgery. For example, act 240 can involve operating the image processor to identify one of several pre- calibrated tooltips and return a pre-calibrated tooltip location, for example its tip, in the coordinate system 24 associated with the surgical device. For example, in a pre- surgical step, the tip of each tooltip can be calibrated to finds its location in coordinate system 24 using known manual methods, and image locations along its silhouette in the image can be stored as a silhouette descriptor associated with the tip location data. During surgery, the silhouette descriptor of each tooltip can be retrieved in turn and compared to edge locations in the image to obtain a descriptor fitness score. For example, the descriptor may be a list of pixel locations where the silhouette projection 92 may be found, and the fitness score could be the average of the edge strengths in the image at the descriptor’s pixel locations. If at least one of the fitness scores passes some recognition threshold (which can be determined empirically), the tooltip with the highest fitness score can be selected and its associated calibrated information can be retrieved and provide processor 70 with a tool location in the coordinate system 24 associated with the surgical device.

[0073] Referring now to FIG. 5, shown therein is a flowchart illustrating an example method 300 for pre-calibrating mapping parameters and mapping the location of the tooltip to the coordinate system associated with the surgical device. The method 300 can be used for pre-calibrating the mapping parameters prior to use of the surgical device. For example, in some embodiments, the handpiece 10 may not have a rotating chuck for tooltips, but may have a vibrating tooltip. When the handpiece 10 vibrates tooltip 30 designed, for example, for cutting or sawing, both the shape and the distance of the tooltip 30 from the camera 50 may vary from one tooltip 30 to the next. In method 300, the pre-surgical preparation step, which can also be done at the factory, involves the pre-calibration of the set of tooltips that may be used during surgery. That pre- calibration can involve computing and persistently storing a visual appearance descriptor of the tooltip’s silhouette projection on the camera’s image, together with locations of interest on the tooltip, usually on its cutting edge, in the tool’s coordinate system. That measurement may be done using known methods, such as using a tracked probe to touch such points, or a specially designed calibration tool.

[0074] At 310, a memory stores an associated descriptor for each tooltip in the set. The associated descriptor can include tooltip projection recognition parameters and a position in a coordinate system associated with the surgical device. For example, the tooltip projection recognition parameters can include, but not be limited to, edge locations, length of the tooltip, width of the tooltip, and distance from the surgical instrument. In some embodiments, the tooltip projection recognition parameters can include shape and dimension parameters and a tooltip identifier for each tooltip. The shape and dimension parameters of that tooltip can be retrievable by the tooltip identifier.

[0075] At 320, the camera captures an image of the tooltip. The image can then be provided to the image processor. The image processor can perform an edge detection algorithm to identify a set of edge locations likely to be on the tooltip’s silhouette projection. These edge locations can be referred to as measured edge locations.

[0076] At 330, the image processor selects the descriptor associated with the tooltip held by the surgical device based on the tooltip recognition parameters. The set of measured edge locations can be compared to the stored tooltip recognition parameters. Scores representing the degree of match between the edge locations and each of the stored tooltip recognition parameters can be determined to identify which, if any, of the tooltips is currently installed. [0077] At 340, the processor retrieves the position for that identified descriptor. For example, if the scores representing the degree of match between the edge location and the tooltip recognition parameters indicate a successful identification, the tool coordinate locations stored with the descriptor for that particular tool can be retrieved, and the position of the tooltip determined. The processor can then determine the position of the tooltip in the coordinate system associated with the surgical device. This act can also be referred to as the matching algorithm.

[0078] In embodiments where the tooltip recognition parameters include the tooltip identifier, when the tooltip identifier is identified for the particular tooltip in use, the shape and dimension parameters are retrieved for that tooltip. Once the shape and dimension parameters are retrieved, the shape and dimension parameters, as well as the tooltip projection in the image of the tooltip, can be used to represent the tooltip in the coordinate system associated with the surgical device.

[0079] In some embodiments, as described above, the matching algorithm can operate to determine the match between the descriptor and the tooltip’s appearance in the image. The matching algorithm can also be designed to accommodate small shifts and rotations in the tooltip’s appearance and to apply a corresponding small correction to the retrieved position in the coordinate system associated with the surgical device.

[0080] It should be noted that method 300 can also be applied to rotating tooltips, although, unlike method 200, each tooltip to be navigated must be separately calibrated in advance.

[0081] In some embodiments, as exemplified in FIG. 1 , the tooltip calibration system 1 can include a pose sensor (not shown). The pose sensor can be used to determine a pose mapping between a coordinate system associated with the surgical device, and a coordinate system associated with the pose sensor. For example, the pose sensor can be an optical pose sensor that can track the optically marked tag 20. Thus, the pose sensor can map the coordinate system 24 associated with the surgical device 10 to a coordinate system associated with the pose sensor.

[0082] The pose sensor can be in electronic communication with the image processor. During operation, the image processor can use the pose mapping between the coordinate system associated with the surgical device and the coordinate system associated with the pose sensor to determine the location on the surface of the tooltip in the coordinate system associated with the pose sensor.

[0083] In some embodiments, methods 200 and 300 can involve operating the pose sensor. The pose sensor can operate to determine the pose mapping between the coordinate system associated with the surgical device and the coordinate system associated with the pose sensor. Once the position in the coordinate system associated with the surgical device of a location on the surface of the tooltip is determined (e.g. in act 240 or act 340), the processor can determine the location on the surface of the tooltip in the coordinate system associated with the pose sensor. For example, once the tooltip location is determined in the coordinate system 24, the pose mapping can be used to map the tooltip location from the coordinate system 24 to the coordinate system associated with the pose sensor. In some embodiments, the pose sensor can concurrently measure the pose of a coordinate system associated with the anatomy being treated, thereby providing a concatenated mapping between the tooltip location and the anatomy being treated to provide positioning guidance to the surgeon operating handpiece 10.

[0084] In some circumstances, detection of the tooltip edges in the camera image may become unreliable. For example, when inside the patient’s mouth, part or all of tooltip 30 may be obscured from view of camera 50, for example by the material the tooltip is drilling or cutting into, making the tooltip length and shape detection unreliable. In some embodiments, the tooltip calibration system 1 can detect when the tooltip calibration becomes unreliable, and the detection can be stopped. Similarly, methods 200 and 300 can involve selecting a region in the coordinate system associated with the pose sensor in which the detection of the tooltip edges in the camera image may be unreliable and stopping determining the location on the surface of the tooltip when the tooltip is within the selected region. For example, updating of tooltip length and shape can be turned off when the processor 70 senses that the handpiece head 18 is in the vicinity of the mouth, for example, at a distance of less than 50mm from the arch line of the jaw being treated. In some embodiments, methods 200 and 300 can involve selecting a region in the coordinate system associated with the anatomy being treated in which the detection of the tooltip edges in the camera image may be unreliable and stopping determining the location on the surface of the tooltip when the tooltip is within the selected region. [0085] In some embodiments, the tooltip calibration system 1 can determine when no tooltip is held by the surgical device. Similarly, methods 200 and 300 can involve detecting when no tooltip is held by the surgical device by operating the image processor to process the image to determine when the image does not contain the tooltip projection. For example, when the head 18 of the handpiece 10 is outside the patient’s mouth, tooltip 30 may be removed from the head 18, for example in order to be exchanged for a different tooltip. In such a case, processor 70, upon failing to detect a proper tooltip silhouette 92, may signal to the navigation system that the tooltip has been removed and its associated data is no longer valid. Following the insertion of a new tooltip, the silhouette can be detected by the processor 70, and the length and the set of calibrated tooltip locations provided to the navigation system can be updated to reflect characteristics of the new tooltip.

[0086] A tooltip calibration system may originate as a kit of parts to be assembled by a builder at a later date. The tooltip calibration kit can be used with a surgical device designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips. For example, the tooltip calibration kit can be used with the surgical device 10. The tooltip calibration kit includes a camera. As described herein, the camera is securable to the surgical device to be positioned and oriented to provide an image of at least part of each of the tooltips in the set when that tooltip is held by the surgical device. The tooltip calibration kit also includes instructions stored on a non-transitory computer-readable media for configuring an image processor receiving the image from the camera to process the image to determine apposition in the coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image.

[0087] In some embodiments, the surgical device to be used with the tooltip calibration kit can have a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip being rotatable about an axis of rotation to perform a surgical procedure when held in the chuck. When the image processor is configured by the instructions stored on the non-transitory computer-readable media, receiving the image from the camera to process the image can further include operating the image processor to determine a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension are at least partly defined relative to the axis of rotation. [0088] In some embodiments, the image processor is configured by the instructions stored on the non-transitory computer-readable media to perform the acts of method 200. In some embodiments, the image processor is configured by the instructions stored on the non-transitory computer-readable media to perform the acts of method 300.

[0089] Various embodiments have been described herein by way of example only. Various modification and variations may be made to these example embodiments without departing from the spirit and scope of the invention, which is limited only by the appended claims.

Claims

1 . A method for calibrating a tooltip relative to a surgical device, the surgical device being operable to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips, the method comprising:

securing a camera to the surgical device, and positioning and orienting the camera to provide an image containing a tooltip projection of at least part of each of the tooltips in the set when that tooltip is held by the surgical device; operating the camera secured to the surgical device to provide the image containing the tooltip projection of the at least part of the tooltip held by the surgical device; providing the image to an image processor; and operating the image processor to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip.

2. The method as defined in claim 1 , wherein the surgical device comprises a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip being rotatable about an axis of rotation to perform a surgical procedure when held in the chuck.

3. The method as defined in claim 2, wherein operating the image processor to process the image further comprises determining a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension are at least partly defined relative to the axis of rotation.

4. The method as defined in claim 3, wherein: determining the location on the surface of the tooltip projection comprises determining an image location on an edge of the tooltip projection on the camera image, and operating the image processor to process the image further comprises: determining and storing in memory mapping parameters enabling mapping between image locations along a projection of the axis of rotation in the image and locations along the axis of rotation in the coordinate system associated with the surgical device; retrieving the mapping parameters from memory; and using the retrieved mapping parameters to map the image location to a location in the coordinate system associated with the surgical device.

5. The method as defined in claim 1 further comprising: storing, for each tooltip in the set, an associated descriptor comprising tooltip projection recognition parameters and a position in a coordinate system associated with the surgical device, and wherein operating the image processor comprises: selecting the descriptor associated with the tooltip held by the surgical device based on the tooltip recognition parameters; and retrieving the position stored in that descriptor.

6. The method as defined in claims 3 or 5 wherein operating the image processor to process the image comprises detecting at least one edge in the camera image.

7. The method as defined in claim 4 wherein: the mapping parameters comprise distance calibration parameters for calibrating different point locations on the axis projection line to different positions in the coordinate system associated with the surgical device; and using the retrieved mapping parameters to map the image location to the location in the coordinate system associated with the surgical device comprises determining a position of the tip of the one tooltip held in the chuck based at least partly on the distance calibration parameters.

8. The method as defined in claim 4 wherein: the pose of the chuck in the coordinate system associated with the surgical device is adjustable; and the mapping parameters comprise parameters that enable adjusting the mapping according to the adjusted pose of the chuck.

9. The method as defined in any one of claims 1 -8 further comprising: operating a pose sensor to measure a pose mapping between a coordinate system associated with the surgical device and a coordinate system associated with the pose sensor; and determining the location on the surface of the tooltip in the coordinate system associated with the pose sensor.

10. The method as defined in claim 9, further comprising: selecting a region in a coordinate system associated with the pose sensor in which the detection of the tooltip edges in the camera image may be unreliable; and stopping determining the location on the surface of the tooltip when the tooltip is within the selected region.

1 1. The method as defined in claims 1 -10 further comprising detecting when no tooltip is held by the surgical device by operating the image processor to process the image to determine when the image does not contain the tooltip projection.

12. A tooltip calibration kit for use with a surgical device designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips, the tooltip calibration kit comprising: a camera securable to the surgical device to be positioned and oriented to provide an image of at least part of each of the tooltips in the set when that tooltip is held by the surgical device; and instructions stored on a non-transitory computer-readable media for configuring an image processor receiving the image from the camera to process the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image.

13. The tooltip calibration kit as defined in claim 12, wherein: the surgical device comprises a chuck for interchangeably holding each tooltip in the set of tooltips, each tooltip being rotatable about an axis of rotation to perform a surgical procedure when held in the chuck; and, when the image processor is configured by the instructions stored on the non- transitory computer-readable media, the image processor receiving the image from the camera to process the image further comprises configuring the image processor to determine a shape and dimension of the one tooltip held in the chuck, wherein the shape and dimension are at least partly defined relative to the axis of rotation.

14. The tooltip calibration kit as defined in claim 13, wherein, when the image processor is configured by the instructions stored on the non-transitory computer- readable media; determining the location on the surface of the tooltip projection comprises determining an image location on an edge of the tooltip projection on the camera image; and the image processor is further configured to: store in a memory in electronic communication with the image processor, mapping parameters enabling mapping between image locations along a projection of the axis of rotation in the image and locations along the axis of rotation in the coordinate system associated with the surgical device; retrieve the mapping parameters from memory; and map the image location to a location in the coordinate system associated with the surgical device using the retrieved mapping parameters.

15. The tooltip calibration kit as defined in claim 12, wherein, when the image processor is configured by the instructions stored on the non-transitory computer- readable media, the image processor is further configured to: for each tooltip in the set, store in a memory in electronic communication with the image processor an associated descriptor comprising tooltip projection recognition parameters and a position in a coordinate system associated with the surgical device; select the descriptor associated with the tooltip held by the surgical device based on the tooltip recognition parameters; and retrieve the position stored in that descriptor.

16. The tooltip calibration kit as defined in any one of claims 12-15, wherein the image processor is further configured to select edge locations in the image based at least partly on the local gradient magnitude, and the camera components are selected to blur edges of projections of objects further than 5cm behind the tooltip sufficiently to ensure they fail the edge detection test.

17. A tooltip calibration system for a surgical device designed to interchangeably hold and drive one of a set of differently shaped or dimensioned tooltips, the tooltip calibration system comprising: a camera securable to the surgical device to be positioned and oriented to provide an image of at least part of each of the tooltips in the set when that tooltip is held by the surgical device; and an image processor for, in operation, receiving the image from the camera and processing the image to determine a position in a coordinate system associated with the surgical device of a location on the surface of the tooltip visible in the image.

18. The tooltip calibration system as defined in claim 17 further comprising a pose sensor for measuring a pose mapping between a coordinate system associated with the surgical device and a coordinate system associated with the pose sensor, the pose sensor being in electronic communication with the image processor; wherein, in operation and using the pose mapping between the coordinate system associated with the surgical device and the coordinate system associated with the pose sensor, the image processor determines the location on the surface of the tooltip in the coordinate system associated with the pose sensor.