JP2013034764A - Surgical guide device and method for positioning drill - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow viewing of the tip of a drill, to make an error difficult to occur in a drilling direction of the drill, to sufficiently supply cooling water to sufficiently cool an alveolar bone, and to cope with several kinds of drills having different diameters.SOLUTION: This surgical guide device includes: a photographing display device having a camera photographing a video inside the oral cavity of a patient including a first AR-recognizing marker attached to a mouthpiece mounted on a residual tooth part of the patient, and a display; and an AR image processor connected to the photographing display device, superposing three-dimensional position information indicating an implanting position and an implanting direction of a fixture simulated based on a position and an attitude of the first AR-recognizing marker in the video inside the oral cavity of the patient during the photographing by the camera on the video inside the oral cavity of the patient during the photographing by the camera, and displaying the superposed video on the display.

Description

  The present invention relates to a surgical guide device that guides the drilling position and direction of an embedding hole in a patient's oral cavity, which is used for accurately implanting an implant fixture into a predetermined position and direction of a patient's alveolar bone. The present invention relates to a drill positioning method using the same.

  Dental implants used for implant treatment (hereinafter simply referred to as “implants”) include a fixture (implant body, for example, made of titanium) that is inserted into an alveolar bone that has lost teeth, and a fixture connected to the fixture. It is comprised by the abutment used as a stand | base and the upper structure (artificial dental crown) with which an abutment is mounted | worn. In implant treatment using this implant, it is important to accurately drill a hole for embedding a fixture into a predetermined position and direction of a patient's alveolar bone (for example, patents). Reference 1).

There are two types of implant treatment: prosthesis-driven (top-down treatment) and surgery-driven.
In prosthetic-driven implant treatment, mock-ups (full-scale gypsum model) of the patient's teeth and jaws are prepared, and judged from the occlusal relationship with the opposing teeth and the state of the dentition, the prosthetic (functional)・ Determine the shape and position of the superstructure that seems to be optimal.
Then, based on the determined shape and position of the superstructure, the position and direction of the fixture are determined.
In surgically-driven implant treatment, the fixture should be placed where it appears to be surgically optimal, judging from the anatomical shape of the patient's alveolar bone (width, thickness and density of the alveolar bone, nerve travel, etc.) Determine the embedding position and embedding direction.

At present, the use of CT image data obtained by CT (Computed Tomography) imaging and simulation software to examine the implantation position and orientation of the fixture from both the prosthetic and surgical aspects. In addition, it is possible to determine an appropriate position and direction of the fixture both prosthetically and surgically.
In addition, by using a template containing a contrast medium during CT imaging, it is possible to indicate the implantation position and direction of the simulated fixture with the planting position and direction of the wire planted on the mockup. Is possible.

However, since drilling a hole into the patient's alveolar bone to insert the fixture requires three-dimensional positioning of the drill in the patient's mouth, simulation software screens and mockups It is extremely difficult to drill the insertion hole accurately with a free hand in a predetermined position and direction of the patient's alveolar bone while taking into account the wire implanted above.
For this reason, various jigs, that is, surgical guides have been devised.

  A typical example is a surgical guide using a metal guide ring having a guide hole for guiding a drill to a predetermined position and direction of a patient's alveolar bone. By installing this surgical guide in the patient's oral cavity, passing the drill through the guide hole of the guide ring and entering the drill following the guide hole, the insertion hole is accurately formed in a predetermined position and direction of the patient's alveolar bone. Drilling is possible.

Japanese Patent Laid-Open No. 2001-170080

However, the above-described surgical guide using a guide ring has the following problems.
(1) When the surgical guide is mounted in the patient's mouth, the surgical field is covered with the surgical guide and cannot be seen, so the surgeon cannot see the part where the tip of the drill is scraping the alveolar bone, It will be performed in an uneasy state,
(2) An appropriate gap (play) is required between the drill and the guide hole of the guide ring. However, since the thickness of the guide ring is smaller than the depth of the embedding hole, this gap (play) ) May cause an error in the drilling direction of the drill, and the embedding hole may be displaced from the predetermined position and direction.
(3) The alveolar bone being cut is covered with the guide ring and the peripheral edge holding the guide ring, and the guide hole of the guide ring is closed with a drill, and a closed space is formed around the alveolar bone. Therefore, it becomes difficult to supply sufficient cooling water to the alveolar bone during cutting, and there is a possibility that the alveolar bone cannot be sufficiently cooled.
(4) In order to drill holes for embedding fixtures, it is necessary to start with a pilot drill with a small diameter and use several types of drills with larger diameters in stages. Several types of surgical guides with a guide ring matched to the diameter of the are required, which increases the cost.

  Therefore, an object of the present invention is to allow the tip of the drill to be visually observed, to prevent an error in the drilling (advancing) direction of the drill, and to sufficiently cool the alveolar bone by sufficiently supplying cooling water. Another object of the present invention is to provide a surgical guide device that can handle several types of drills having different diameters and a drill positioning method using the surgical guide device.

An invention (surgical guide device) according to claim 1 is a photographing display device having a camera and a display for photographing an image in a patient's oral cavity including a first AR recognition marker provided in the patient's oral cavity, Based on the position and orientation of the first AR recognition marker in the image in the oral cavity of the patient being photographed by the camera connected to the photographing display device, the simulated placement position and orientation of the fixture And an AR image processing device that displays the three-dimensional position information indicating the image on the display in a manner superimposed on the intraoral image of the patient being imaged by the camera.
Here, AR is Augmented Reality, that is, an abbreviation of augmented reality, and means that virtual information such as three-dimensional position information is added to real information such as video to expand the quality and quantity of information.

  According to a second aspect of the present invention, there is provided a surgical guide device according to the first aspect, wherein a mouthpiece detachably attached to a remaining tooth portion of a patient and a remaining tooth portion of a mock-up imitating a patient's oral cavity, and the mouthpiece And a marker member having the first AR recognition marker attached thereto.

  The invention according to claim 3 is the surgical guide device according to claim 2, wherein the mouthpiece of the marker member is provided with a plurality of planar or three-dimensional calibration markers using an X-ray contrast medium. It is characterized by that.

  According to a fourth aspect of the present invention, in the surgical guide apparatus according to any one of the first to third aspects, the three-dimensional position information indicating the embedding position and embedding direction of the simulated fixture is a simulated fixture. It is displayed as a three-dimensional image of the char.

  According to a fifth aspect of the present invention, in the surgical guide device according to the fourth aspect, the three-dimensional position information indicating the position and direction in which the simulated fixture is embedded is included in the three-dimensional image image of the fixture. The image of the central axis of the fixture is displayed as an added image, and the image of the central axis is displayed in a color different from the stereoscopic image of the fixture.

  According to a sixth aspect of the present invention, in the surgical guide device according to any one of the first to fifth aspects, the mouthpiece is provided with the first AR recognition markers at a plurality of locations. Any one of the first AR recognition markers attached to the camera is arranged so as to be included in a captured image frame of the camera during the operation.

According to a seventh aspect of the present invention, in the surgical guide apparatus according to any one of the first to sixth aspects, the AR image processing apparatus includes computer hardware in which AR software is installed. A stereoscopic image including three-dimensional position information indicating an embedding position and an embedding direction is captured as a 3D object, and the display is performed based on the position and orientation of the first AR recognition marker. .
Here, the AR software refers to a coordinate conversion of a stereoscopic image captured as a 3D object based on the position and orientation of the first AR recognition marker in the video being captured by the camera, and the first software Software that superimposes in real time on an image being captured by the camera and displays it on the display while maintaining a three-dimensional positional relationship with the AR recognition marker.

  The invention according to claim 8 is the surgical guide device according to claim 7, wherein a second AR recognition marker is attached to a contra head to which a drill for drilling a hole for embedding the fixture is attached, The 3D image image that matches the shape of the drill is captured as a 3D object in the AR software, and the 3D image image that matches the shape of the drill is superimposed and displayed on the image of the drill that the camera is capturing. It is characterized by that.

  According to a ninth aspect of the present invention, in the surgical guide device according to any one of the third to eighth aspects, the AR image processing device uses the marker member provided with a plurality of the calibration markers at the remaining tooth of the patient. A CT image data of a patient that has been CT-taken while being mounted on a part is read into simulation software, and a three-dimensional image image of the simulated fixture obtained by executing a fixture embedding simulation on the simulation software; and A composite image with a stereoscopic CT image of a patient's alveolar bone volume-rendered by simulation software is captured as a 3D object.

According to a tenth aspect of the present invention, in the surgical guide device according to any one of the first to eighth aspects, the AR image processing device is configured such that a wire indicating a embedment position and a embedment direction of a simulated fixture is planted. 3D paint software installed in addition to the AR software using a mock-up that imitates the patient's oral cavity, and captures a 3D image that completely matches the wire as a 3D object as a 3D object It is characterized by that.
Here, the 3D paint software adds a function to the AR software. A 3D drawing locus drawn using a paint marker in a three-dimensional space is taken in as a 3D object, and the three-dimensional drawing locus is Software in which the camera superimposes in real time on the image being captured and displays it on the display while maintaining a three-dimensional positional relationship with the first AR recognition marker in the image being captured by the camera.

  According to an eleventh aspect of the present invention, in the surgical guide device according to any one of the first to tenth aspects, the display is a head-mounted display configured integrally with the camera. .

  According to a twelfth aspect of the present invention, in the surgical guide device according to the eleventh aspect, an optical transmission type display is used as the head mount type display, and the simulated fixture is embedded in a transmission image in the oral cavity of a patient. Three-dimensional position information indicating the position and the embedding direction is superimposed and displayed.

  The invention according to claim 13 is the surgical guide device according to any one of claims 1 to 10, characterized in that the display is a display independent of the camera.

  The invention according to claim 14 is the surgical guide device according to any one of claims 1 to 13, wherein a planar or stereoscopic image registered in advance is used as the first AR recognition marker. Features.

  According to a fifteenth aspect of the present invention, in the surgical guide device according to any one of the first to fourteenth aspects, the AR software recognizes and stores a three-dimensional spatial structure from an image captured by the camera. It is possible that it is possible.

  According to a sixteenth aspect of the present invention, the surgical guide device according to any one of the second to eleventh aspects is used to drill an insertion hole for implanting an implant fixture into the alveolar bone of a patient. In the drill positioning method, a step of taking a three-dimensional image including three-dimensional position information indicating an embedding position and an embedding direction of the simulated fixture into the AR image processing apparatus as a 3D object, An image of the patient's oral cavity including a first AR recognition marker attached to a mouthpiece attached to the patient's remaining teeth is photographed, and the camera includes the first image in the image of the patient's oral cavity being photographed. Three-dimensional position information indicating the position and orientation of the simulated fixture based on the position and orientation of one AR recognition marker , And a step of the camera is displayed on the display superimposed on the image of the oral cavity of the patient during imaging, characterized in that.

The invention according to claim 17 uses a surgical guide device according to claim 9 in a drill positioning method when drilling an embedding hole for embedding an implant fixture in a patient's alveolar bone. , said data conversion to the installed a R software capable format captures data of the composite image on the AR image processing apparatus, incorporating the AR software the composite image as a 3D object, a composite image capture process, the The marker member is attached to the remaining tooth portion of the mock-up, and the AR software displays the composite image superimposed on the mock-up image being photographed by the camera, and the calibration marker in the image The marker member is arranged so that the X-ray image of the calibration marker in the composite image completely matches three-dimensionally. An image position calibration step for adjusting and fixing the position and posture of the first AR recognition marker attached to the mouthpiece, and mounting the marker member on the remaining tooth portion of the patient. A drill positioning step of superimposing and displaying the composite image on the intraoral image of the patient being photographed by the camera, and positioning the drill using a three-dimensional image of the simulated fixture in the composite image as a guide And comprising.

  The invention according to claim 18 uses a surgical guide device according to claim 10 in a drill positioning method when drilling an embedding hole for embedding an implant fixture in a patient's alveolar bone. The marker member is attached to the remaining tooth portion of the mock-up on which the wire is planted, and the wire is traced three-dimensionally with the wire using the paint marker of the 3D paint software. A 3D image image that is completely matched to the 3D paint software, and the 3D paint software captures the wire image, and attaches the marker member to the patient's remaining teeth, and the 3D paint software captures the camera. A stereoscopic image corresponding to the wire is superimposed on the intraoral video of the patient And to position the drill stereoscopic images that match the wire as a guide, comprising a drill positioning step, and wherein the.

  According to the first aspect of the present invention, the three-dimensional position information indicating the position and direction of the simulated fixture is superimposed on the image of the intraoral area of the patient being photographed by the camera. Therefore, the tip of the drill can be visually observed, it is difficult to cause an error in the drilling (advancing) direction of the drill, and cooling water can be sufficiently supplied to sufficiently cool the alveolar bone. It is possible to provide a surgical guide device that can cope with a plurality of fixtures with a single diameter and that can accommodate several types of drills having different diameters.

  According to invention of Claim 2, a patient's remaining tooth | gear part can be made into the fixed source of the marker for 1st AR recognition.

  According to invention of Claim 3, the surgical guide apparatus shown in Claim 9 and the drill positioning method shown in Claim 17 are attained.

  According to the invention of claim 4, it is possible to use a three-dimensional image of the same fixture as that used when simulating on a computer as a guide.

  According to the invention of claim 5, it becomes easy to make the axis of the drill coincide with the center axis of the stereoscopic image of the simulated fixture.

  According to the invention of claim 6, by attaching the first AR recognition marker to a plurality of positions of the mouthpiece in advance, any one of the first AR recognition markers is left in the captured image frame of the camera. It is possible to prevent the 3D object from being displayed.

  According to the invention of claim 7, the drill positioning method shown in claim 16 is possible.

  According to the eighth aspect of the present invention, it becomes easier to make the axis of the drill coincide with the central axis of the stereoscopic image of the simulated fixture.

  According to the ninth aspect of the present invention, a composite image of the stereoscopic image of the simulated fixture and the stereoscopic CT image of the alveolar bone of the patient volume-rendered by simulation software is used. A drill positioning method is possible.

Furthermore, it is also possible to perform prosthetic verification on the simulated fixture placement position on the mockup.
In addition, surgically important indices such as the mandibular canal and maxillary sinus can be displayed superimposed on the image in the patient's mouth.

  According to the invention of claim 10, the mock-up imitating the oral cavity of the patient in which the wire indicating the implantation position and direction of the simulated fixture has been implanted is used. A positioning method is possible.

  According to the invention of claim 11, the surgeon can perform an operation without moving the viewpoint from the operation field.

  According to the twelfth aspect of the present invention, the surgeon can see the surgical field not by an image through the camera but by a transmission image of the display.

  According to the thirteenth aspect of the present invention, it becomes possible to confirm the state of the operation also to the operation staff other than the operator.

  According to the invention of claim 14, the mouthpiece or the remaining tooth itself can be used as the first AR recognition marker.

  According to the fifteenth aspect of the present invention, it is possible to continue displaying the 3D object even if the first AR recognition marker deviates from the captured image frame of the camera.

  According to the invention of claim 16, the camera guides the three-dimensional position information indicating the implantation position and direction of the simulated fixture displayed on the display superimposed on the image in the mouth of the patient being imaged. As a drill can be positioned.

  According to the invention of claim 17, the drill can be positioned using the simulated three-dimensional image of the fixture as a guide.

  According to the eighteenth aspect of the present invention, the drill can be positioned by using, as a guide, a three-dimensional image image corresponding to the wire indicating the simulated position and direction of the fixture.

It is explanatory drawing of an implant. It is explanatory drawing which shows the state which mounted | wore with the marker member to the residual tooth | gear part of a mockup (gypsum model). It is a general | schematic block diagram of the surgical guide apparatus of 1st Embodiment. (A), (b) is explanatory drawing at the time of the drilling in 1st Embodiment. It is a general | schematic block diagram of the surgical guide apparatus of 2nd Embodiment. (A), (b) is explanatory drawing at the time of the drilling in 2nd Embodiment. It is a three-dimensional CT image of alveolar bone.

  Preferred embodiments of the present invention will be described below in detail with reference to the drawings.

[Overview]
First, prior to specific description, an implant will be described.
FIG. 1 is an explanatory view of an implant.
After the teeth (not shown) are removed, an alveolar bone 62 composed of soft cancellous bone 60 and hard cortical bone 61 and a mucosa 63 covering the alveolar bone 62 remain. The implant 70 includes a fixture 71 embedded in the alveolar bone 62 with an insertion hole H, an abutment 72 connected to the fixture 71 and serving as an abutment, and an upper structure 73 attached to the abutment 72. It is constituted by.

FIG. 2 is an explanatory view showing a state in which the marker member 11A is attached to the remaining tooth portion of the mockup 30 (gypsum model).
As shown in FIG. 2, the mock-up 30 is a full-scale gypsum model of the upper jaw that is produced by taking (impression) from the patient's mouth.
In the example shown in FIG. 2, the upper right back tooth (molar) is shown. In the part where the back teeth are missing, the CT image data of the patient acquired by CT imaging is read into the simulation software, the fixture placement simulation is executed while considering both the prosthesis and the surgery, and the determined fixture 71 A wire W indicating the embedding position and the embedding direction is planted. The planting position and planting direction of this wire W are determined as follows, for example.

The patient's CT image data obtained by CT imaging is read into simulation software and simulated, and the three-dimensional relationship between the implantation position and orientation of the fixture 71 (see FIG. 1) and the CT imaging template containing the contrast medium The physical positional relationship is measured on the screen of the simulation software, and a mark indicating the embedding position and the embedding direction of the fixture 71 is attached to the CT imaging template containing the contrast medium based on the three-dimensional positional relationship. This is mounted on the mock-up 30 and a hole (wire-planting hole) having a diameter of about 1 mm and a depth of about 10 mm is formed on the mock-up 30 using this mark as a guide, and the wire W is planted in the hole.
Thereby, the embedment position and the embedment direction of the simulated fixture 71, in other words, the piercing position and the piercing direction of the embedding hole H are set as the planting position and the planting direction of the wire W on the mockup 30. Can show.
[1] First Embodiment First, a first embodiment using a composite image of a stereoscopic image of a simulated fixture and a stereoscopic CT image of a patient's alveolar bone volume-rendered by simulation software will be described.

[1.1] Structure FIG. 3 is an explanatory diagram of a schematic configuration of the surgical guide apparatus 10 according to the first embodiment.
As shown in FIG. 3, the surgical guide device 10 according to the first embodiment is roughly provided with a marker member 11, an imaging display device 14, and an AR image processing device 15.
The marker member 11 is a resin or polycarbonate mouthpiece M that is detachably attached to the remaining tooth portion of the patient and the remaining tooth portion of the mock-up 30 imitating the inside of the patient's mouth, and X provided at a plurality of positions on the mouthpiece M. A calibration marker K using a line contrast agent and a first AR recognition marker MK1 attached to the mouthpiece M are provided.
The photographic display device 14 includes a camera 12 and a head-mounted display 13 configured integrally with the camera 12.
The AR image processing device 15 is connected to the photographing display device 14 and is configured as computer hardware in which AR software is installed.

  Here, the AR software means that the first image obtained as a 3D object is coordinate-converted based on the position and orientation of the first AR recognition marker MK1 in the image being captured by the camera 12, and the first software is used. This software is displayed on the display 13 by superimposing it in real time on the image being photographed by the camera 12 while maintaining the three-dimensional positional relationship with the AR recognition marker MK1.

[1.2] Operation Procedure Next, an operation procedure of the surgical guide apparatus 10 according to the first embodiment will be described.
(1) The marker member 11 is mounted on the remaining tooth portion of the patient, CT imaging is performed, and CT image data is acquired.
(2) The obtained CT image data is read into the simulation software, and the fixture embedding simulation is executed. The three-dimensional image image F of the simulated fixture and the three-dimensional of the alveolar bone of the patient volume-rendered by the simulation software Data of the composite image G with the CT image B is obtained (see FIG. 7).
(3) Data of the composite image G is converted into a format G ′ (not shown) that can be imported by the AR software installed in the AR image processing device 15.
(4) The synthesized image G ′ is captured as a 3D object into the AR software installed in the AR image processing apparatus 15 (the synthesized image G ′ is captured in the coordinate system of the first AR recognition marker MK1).
(5) The marker member 11 is attached to the remaining tooth portion of the mock-up 30 and the camera 12 photographs the mock-up 30 with the marker member 11 attached, and the AR software displays the image of the mock-up 30 being photographed by the camera 12 The three-dimensional CT image B of the patient's alveolar bone is superimposed on the display 13 and displayed.
(6) The calibration marker K in the image of the mockup 30 and the X-ray image KB (not shown) of the calibration marker K in the stereoscopic CT image B of the patient's alveolar bone completely match three-dimensionally. The position and posture of the first AR recognition marker MK1 attached to the mouthpiece M are adjusted and fixed to the mouthpiece M so that the three-dimensional deviation is within a predetermined error range.
The pre-preparation is now complete. Next, a procedure for actually perforating a patient's alveolar bone will be described.
FIG. 4 is an explanatory diagram at the time of drilling.
(7) As shown in FIG. 4 (a), the marker member 11 is attached to the patient's remaining teeth, and the inside of the patient's mouth where the marker member 11 is attached is photographed by the camera 12, and FIG. As shown, a display image 13 </ b> V is displayed on the display 13 by superimposing the simulated three-dimensional image image F of the simulated image on the intraoral image of the patient being photographed by the camera 12.

Here, the processing contents of the AR software are specifically shown as follows.
(Step 1) Capture of the pattern of the first AR recognition marker MK1.
(Step 2) Capture of a 3D object (in this embodiment, a composite image G ′) into the coordinate system of the first AR recognition marker MK1.
(Step 3) Acquisition of video by the camera 12 and display on the display 13.
(Step 4) Comparison with the recognition (extraction) of the pattern of the first AR recognition marker MK1 in the video by the camera 12.
(Step 5) Calculation of the three-dimensional position and orientation of the first AR recognition marker MK1 in the video by the camera 12.
(Step 6) Calculation of a transformation determinant for coordinate transformation to the coordinate system of the first AR recognition marker MK1 in the video by the camera 12.
(Step 7) The 3D object is drawn on the coordinate system of the first AR recognition marker MK1 in the video by the camera 12 and superimposed on the video being displayed on the display 13.
(8) The surgeon positions the drill 21 by using, as a guide, the three-dimensional image F of the simulated fixture displayed superimposed on the intraoral image of the patient.
(9) The surgeon starts drilling according to the positioning of the drill 21 and drills the embedding hole H.
By the above procedure, the operator can accurately drill the embedding hole H with the drill 21 in a predetermined position (of the simulated fixture) and direction of the alveolar bone of the patient.

  In the above description, for simplification of description, the case where one marker for AR recognition MK1 for AR recognition is attached to the marker member 11 is described. However, the first marker for AR recognition MK1 is the camera 12. 3D objects (in this embodiment, the composite image G ′) will not be displayed. Therefore, by attaching the first AR recognition marker MK1 to a plurality of locations of the mouthpiece M in advance, any of the first AR recognition markers MK1 can be left in the captured image frame of the camera 12. Can prevent this.

In the above description, only the stereoscopic image F of the fixture has been described. However, the image image Z of the central axis may be added in a different color to the simulated stereoscopic image F of the fixture. Good.
Thereby, it becomes easy to make the axis of the drill 21 coincide with the central axis of the stereoscopic image F of the simulated fixture.

Further, the second AR recognition marker MK2 is attached to the contra head 20 on which the drill 21 is mounted, and the stereoscopic image image D that matches the shape of the drill 21 is taken into the AR software as a 3D object, and the camera 12 takes a picture. A stereoscopic image image D that matches the shape of the drill 21 may be superimposed and displayed on the image of the drill 21.
This makes it easier to match the axis of the drill 21 with the central axis of the stereoscopic image F of the simulated fixture.

In the above description, the calibration marker K has a planar shape. However, the calibration marker K may be configured with a three-dimensional structure with unevenness.
This makes image position calibration accurate and easy.

Further, an optical transmission type display may be used as the head mounted type display 13 so that the simulated stereoscopic image F of the fixture is superimposed on the transmission image in the oral cavity of the patient. In this case, a calibration mechanism that completely matches the image in the patient's mouth taken by the camera 12 with the transmitted image viewed by the operator is required. Specifically, the field of view corresponding to the captured image of the camera 12 is the closest to the field of view that can be seen with the naked eye when a human is not gazing (for example, an image of a single focus 50 mm lens in a 35 mm single-lens reflex camera). Corners), recognize objects (eg, mouthpiece M or remaining teeth) in the captured image, detect the viewing direction from their positions and orientations, and the camera 12 captures images in real time. What is necessary is just to make it match | combine completely the image | video in the oral cavity of a patient and the transmitted image which the operator is seeing.
As a result, the surgeon can see the surgical field not by the video through the camera 12 but by the transmitted image of the display.

In the above, an example in which the head-mounted display 13 integrated with the camera 12 is used has been described. Alternatively, the display may be performed on a display independent of the camera 12.
Thereby, it becomes possible to confirm the state of the operation to the operation staff other than the operator.

Instead of the conventional first AR recognition marker MK1, a previously registered planar or stereoscopic image may be used as the first AR recognition marker.
Thereby, the mouthpiece M or the remaining tooth itself can be used as the first AR recognition marker.

As the AR software, software that can recognize and store a three-dimensional spatial structure from an image captured by the camera 12 may be used.
As a result, even when the first AR recognition marker MK1 deviates from the captured image frame of the camera 12, the 3D object (in this embodiment, the composite image G ′) can be continuously displayed.

[2] Second Embodiment Next, a second embodiment using a mock-up in which a wire indicating the embedding position and the embedding direction of the simulated fixture is raised will be described.

[2.1] Structure FIG. 5 is an explanatory diagram of a schematic configuration of a surgical guide apparatus 10A according to the second embodiment.
As shown in FIG. 5, the surgical guide apparatus 10A according to the second embodiment is roughly provided with a marker member 11A, an imaging display apparatus 14, and an AR image processing apparatus 15A.
The marker member 11A includes a resin or polycarbonate mouthpiece M that is detachable from the remaining tooth portion of the patient and the remaining tooth portion of the mockup 30 simulating the patient's mouth, and a first mouthpiece M attached to the mouthpiece M. And an AR recognition marker MK1.
The photographic display device 14 includes a camera 12 and a head-mounted display 13 configured integrally with the camera 12.
The AR image processing device 15A is connected to the photographing display device 14 and is configured as computer hardware in which 3D paint software is installed in addition to the AR software.

  Here, the 3D paint software adds a function to the AR software. A 3D drawing trajectory S drawn using a paint marker P (not shown) in a three-dimensional space is taken in as a 3D object, and the three-dimensional The drawing trajectory S of the camera 12 is superimposed on the image being captured by the camera 12 in real time while maintaining the three-dimensional positional relationship with the first AR recognition marker MK1 in the image being captured by the camera 12. It is software to display.

[2.2] Operation Procedure Next, an operation procedure of the surgical guide apparatus 10A according to the second embodiment will be described.
(1) The marker member 11A is attached to the remaining tooth portion of the mockup 30 where the wire W indicating the embedding position and the embedding direction of the simulated fixture is erected.
(2) By tracing from the base end to the front end of the wire W using the paint marker P of the 3D paint software installed in the AR image processing apparatus 15A, a three-dimensional image image L that completely matches the wire W three-dimensionally Is taken into the 3D paint software as a 3D object (a three-dimensional image L matching the wire W is taken into the coordinate system of the first AR recognition marker MK1).

FIG. 6 is an explanatory diagram at the time of drilling.
(3) As shown in FIG. 6 (a), the marker member 11A is attached to the patient's remaining teeth, and the inside of the patient's oral cavity where the marker member 11A is attached is photographed by the camera 12, and FIG. As shown, the display image 13V in which the stereoscopic image image L matching the wire W is superimposed on the image in the mouth of the patient being photographed by the camera 12 is displayed on the display 13 by the 3D paint software.
Here, the processing contents of the 3D paint software are specifically shown as follows. (Step 1) Capture of patterns of the first AR recognition marker MK1 and the paint marker P.
(Step 2) Acquisition of video by the camera 12 and display on the display 13.
(Step 3) Comparison with the recognition (extraction) of the pattern of the first AR recognition marker MK1 and the paint marker P in the video by the camera 12.
(Step 4) Calculation of the three-dimensional position and orientation of the first AR recognition marker MK1 in the video by the camera 12

(Step 5) The camera 12 calculates the three-dimensional position and orientation of the locus S of the paint marker P in the video.
(Step 6) Calculation and storage of the three-dimensional position and orientation of the trajectory S with respect to the first AR recognition marker MK1 (capturing the trajectory S as a 3D object into the coordinate system of the first AR recognition marker MK1).
(Step 7) Calculation of a conversion determinant for coordinate conversion to the coordinate system of the first AR recognition marker MK1 in the video by the camera 12.
(Step 8) The camera 12 draws the locus S on the coordinate system of the first AR recognition marker MK1 in the video and superimposes it on the video currently displayed on the display 13.

(4) The surgeon positions the drill 21 using the three-dimensional image L corresponding to the wire W displayed superimposed on the image in the oral cavity of the patient as a guide.
(5) The surgeon starts drilling according to the positioning of the drill 21 and drills the embedding hole H.
According to the above procedure, also in the second embodiment, the operator can accurately drill the embedding hole H with the drill 21 in a predetermined position (of the simulated fixture) and direction of the patient's alveolar bone. .

[3] Effects of Embodiments According to the first and second embodiments, the following effects can be obtained.
[3.1] Since only the marker member 11 (or 11A) is attached to the remaining tooth in the patient's oral cavity, there is nothing to cover the surgical field, so that it does not interfere with the operation. Further, the opening amount of the patient is the same as when the surgical guide device is not used.
[3.2] Since the tip of the drill 21 can be visually observed, the drilling direction can be accurately controlled so that no error occurs in the drilling position and the drilling direction of the drill 21. Moreover, it is easy to inject cooling water.

[3.3] Since the surgical guide device 10 (or 10A) and the drill 21 do not contact each other, the surgical guide device 10 (or 10A) does not become unstable even in a free end case, and both the flap method and the flapless method are used. It can correspond to.
[3.4] A single surgical device 10 (or 10A) can support a plurality of fixtures, and can also support several types of drills having different diameters. It can also be used with non-drilling instruments such as piezos.
[3.5] It is possible to perform prosthetic verification on the mockup 30 with respect to the position and direction of insertion of the simulated fixture.
[3.6] Surgically important indices such as the mandibular canal and maxillary sinus can be superimposed on the intraoral image during operation.
[3.7] In the above description, the simulation software and the AR software are separated from each other. However, the simulation software and the AR software are integrated to perform simulation on a transparent mockup. Is also possible.

10, 10A Surgical guide device 11, 11A Marker member 12 Camera 13 Display 13V Display screen 14 Shooting display device 15, 15A AR image processing device 20 Contra head 21 Drill 30 Mock-up D Three-dimensional image image corresponding to drill 21 F Simulated Fixture stereoscopic image H Filling hole K Calibration marker KB X-ray image of calibration marker K L Stereoscopic image matching the wire M Mouthpiece MK1 First AR recognition marker MK2 Second AR recognition Marker Z F center image image

Claims (18)

An imaging display device having a camera and a display for imaging an image in the oral cavity of the patient including a first AR recognition marker provided in the oral cavity of the patient;
Based on the position and orientation of the first AR recognition marker in the image in the oral cavity of the patient being photographed by the camera connected to the photographing display device, the simulated placement position and orientation of the fixture An AR image processing device that superimposes the three-dimensional position information indicating the image on the intraoral image of the patient being imaged by the camera, and displays the image on the display.
Surgical guide device characterized by that. A marker member having a mouthpiece detachably attached to the remaining tooth portion of the patient and the remaining tooth portion of the mockup simulating the patient's mouth, and the first AR recognition marker attached to the mouthpiece. Prepare
The surgical guide apparatus according to claim 1. The mouthpiece of the marker member is provided with a plurality of planar or three-dimensional calibration markers using an X-ray contrast medium.
The surgical guide apparatus according to claim 2, wherein: The three-dimensional position information indicating the embedding position and the embedding direction of the simulated fixture is displayed as a three-dimensional image of the simulated fixture.
The surgical guide apparatus according to any one of claims 1 to 3, wherein The simulated three-dimensional position information indicating the embedded position and direction of the fixture is displayed as an image obtained by adding an image image of the central axis of the fixture to the stereoscopic image image of the fixture. The image of the axis is displayed in a different color from the stereoscopic image of the fixture.
The surgical guide apparatus according to claim 4, wherein The mouthpiece has the first AR recognition markers attached to a plurality of locations, and any one of the first AR recognition markers attached to the locations can be used during the operation. Arranged to be included in the captured image frame of
The surgical guide apparatus according to any one of claims 1 to 5, wherein The AR image processing apparatus includes computer hardware in which AR software is installed, and takes in a stereoscopic image including three-dimensional position information indicating an embedding position and an embedding direction of the simulated fixture as a 3D object, Performing the display based on the position and orientation of the first AR recognition marker;
The surgical guide device according to any one of claims 1 to 6, wherein A second AR recognition marker is attached to the contra head to which a drill for drilling the hole of the fixture is mounted;
The 3D image image that matches the shape of the drill is taken into the AR software as a 3D object, and the 3D image image that matches the shape of the drill is superimposed and displayed on the video of the drill that the camera is shooting. To
The surgical guide apparatus according to claim 7. The AR image processing apparatus reads the CT image data of a patient obtained by CT imaging with the marker member provided with a plurality of calibration markers provided on the remaining tooth portion of the patient, on the simulation software. A composite image of a stereoscopic image of the simulated fixture obtained by executing the fixture embedding simulation and a stereoscopic CT image of the patient's alveolar bone volume-rendered by the simulation software is captured as a 3D object. ,
The surgical guide device according to any one of claims 3 to 8, wherein The AR image processing apparatus was installed in addition to the AR software using a mockup that imitates the oral cavity of a patient in which a wire indicating the implantation position and direction of the simulated fixture was implanted. 3D paint software captures a 3D image that is three-dimensionally identical to the wire as a 3D object.
The surgical guide apparatus according to any one of claims 1 to 8, wherein The display is a head-mounted display configured integrally with the camera.
The surgical guide apparatus according to any one of claims 1 to 10, wherein Claim 11 and Claim 12 are exchanged. An optical transmission type display is used for the head mount type display, and the simulated fixture insertion position and direction are shown in a transmission image in the oral cavity of the patient. 3D position information is superimposed and displayed.
The surgical guide apparatus according to claim 11. The display is a display independent of the camera;
The surgical guide apparatus according to any one of claims 1 to 10, wherein As the first AR recognition marker, a pre-registered planar or stereoscopic image is used.
The surgical guide device according to any one of claims 1 to 13, wherein The AR software is capable of recognizing and storing a three-dimensional spatial structure from an image captured by the camera.
The surgical guide device according to any one of claims 1 to 14, wherein In the positioning method of a drill using the surgical guide apparatus of any one of Claim 2 thru | or 11, when drilling the embedding hole for embedding the fixture of an implant in a patient's alveolar bone with a drill ,
Capturing a stereoscopic image including three-dimensional position information indicating an embedding position and an embedding direction of the simulated fixture into the AR image processing apparatus as a 3D object;
An image of the patient's oral cavity including the first AR recognition marker attached to a mouthpiece attached to the patient's remaining teeth is captured by the camera, and the intraoral image of the patient being captured by the camera Based on the position and orientation of the first AR recognizing marker, the three-dimensional position information indicating the position and direction of the simulated fixture in the oral cavity of the patient being imaged by the camera A step of superimposing the image on the display and displaying on the display.
A drill positioning method characterized by the above. In the positioning method of a drill using the surgical guide device according to claim 9, when drilling an embedding hole for embedding a fixture of an implant in a patient's alveolar bone,
A composite image capturing step of converting the composite image data into a format that can be captured by AR software installed in the AR image processing apparatus, and capturing the composite image as a 3D object in the AR software;
The marker member is mounted on the remaining tooth portion of the mockup, and the AR software displays the composite image superimposed on the mockup image being captured by the camera, and the calibration marker in the image is displayed. The position and orientation of the first AR recognition marker attached to the mouthpiece of the marker member are adjusted so that the X-ray image of the calibration marker in the composite image completely matches three-dimensionally And fixing the image position,
The marker member is attached to the patient's remaining teeth, and the AR image is used to display the composite image superimposed on the image of the patient's mouth being photographed by the camera, and the simulated fixture in the composite image A drill positioning step of positioning the drill using a three-dimensional image of the char as a guide, and
A drill positioning method characterized by the above. In the positioning method of a drill using the surgical guide device according to claim 10, when drilling an embedding hole for embedding an implant fixture in a patient's alveolar bone,
The marker member is attached to the remaining tooth portion of the mockup in which the wire is planted, and the wire is traced from the proximal end to the distal end of the wire three-dimensionally using the paint marker of the 3D paint software. A wire image capturing step of capturing a perfectly matched stereoscopic image as a 3D object into the 3D paint software;
The marker member is attached to the patient's remaining teeth, and the 3D paint software displays a stereoscopic image corresponding to the wire superimposed on the intraoral image of the patient being photographed by the camera. A drill positioning step for positioning the drill using a three-dimensional image image matching the wire as a guide,
A drill positioning method characterized by the above.

Images