KR20170111707A - The augmented reality system reflected estimation of movement of maxillary - Google Patents

Abstract

As an embodiment of the present invention, an Augmented Reality (AR) system and an augmented reality providing method in which motion estimation of a jaw of a patient is reflected can be disclosed. The AR system reflecting movement estimation of a jaw of a patient according to an embodiment of the present invention includes an image acquisition unit for acquiring a CT image of a head and a neck of a patient, A coordinate system setting unit for defining a coordinate system for the patient using a plurality of points included in the converted image, a plurality of markers attached to the patient for motion estimation of the jaw of the patient, A marker recognizing unit for recognizing a plurality of markers, a coordinate estimating unit for estimating coordinates of each of the plurality of markers recognized based on the defined coordinate system, augmenting unit for generating an augmented reality image of the jaw relative to the estimated coordinates And a display unit for displaying the generated augmented reality image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an augmented reality system and a method of providing augmented reality,

The present invention relates to an augmented reality system in which motion estimation of a jaw of a patient is reflected, and a method of providing the augmented reality.

Orthognathic surgery refers to surgery performed to correct for variations in shape and size when the growth of the maxilla or mandible deviates from normal and has excess, deficiency, or asymmetry. Orthognathic surgery usually involves a change in the position of the jawbone (eg, the jawbone or lower jawbone), so orthodontic treatment is usually performed before and after surgery. It is different from general surgery because of the long preparation period and the complexity of the treatment. Orthognathic Orthodontic treatment can be preceded by an orthodontic treatment to align the teeth. If an extraction is required, an extraction may be performed. In addition, impression taking, photographing, and X-ray imaging can be performed. A surgical wafer to be used at the time of surgery can be preliminarily manufactured to suit the patient's oral structure and the like. Orthognathic surgery can take as long as several hours depending on the type of operation, and orthodontic treatment can be performed after checking whether the dentition has been selected based on the repositioned upper and lower jawbone and whether there is a problem in function.

It is necessary to prepare the jawbone quickly and precisely at the target position for orthodontic surgery in order to prepare and perform the orthognathic surgery and to shorten the time required for the surgery.

The present invention relates to an augmented reality system that reflects the motion estimation of a patient's jaw and a method of providing the augmented reality. We propose an augmented reality system and a method of providing the augmented reality to intuitively grasp the target location.

As an embodiment of the present invention, an Augmented Reality (AR) system and an augmented reality providing method in which motion estimation of a jaw of a patient is reflected can be disclosed.

An Augmented Reality (AR) system that reflects motion estimation of a patient's jaw according to an embodiment of the present invention includes an image acquisition unit for acquiring a CT image of the head and neck of a patient, A coordinate system setting unit for defining a coordinate system for the patient using a plurality of points included in the converted image, a plurality of markers attached to the patient for motion estimation of the patient's jaw, A coordinate estimating unit for estimating coordinates of each of the plurality of markers recognized based on the defined coordinate system, an augmented reality image of the jaws with respect to the estimated coordinates, An augmented reality image generation unit for generating augmented reality image, and a display unit for displaying the generated augmented reality image.

In the image conversion unit according to an embodiment of the present invention, CT images of the head and the neck of a patient can be converted into 3D images using DICOM (Digital Imaging and Communications in Medicine) data.

In addition, in the coordinate system setting unit according to an embodiment of the present invention, the Frankfort Horizontal (FH) plane is created for the patient using three or more points in the 3D image, (MSR) plane using the bases, and a coordinate system for the patient can be defined at the center point where the generated two sides intersect.

A plurality of markers according to an embodiment of the present invention is in the form of a polygon with different patterns displayed on each face, and includes a first marker (M _P ) that can be placed adjacent to the patient's forehead, A second marker M _M attachable to the wafer, and a third marker M _R for pointing the reference point of the movement of the patient's jaws.

The marker recognizing unit according to an embodiment of the present invention may be a dual camera for acquiring an image for each marker having a different pattern.

The third marker M _R according to an embodiment of the present invention can be used for pointing a point that is a reference point for tracking movement of the patient's jaws. At a designated point, a point between the central incisor of the patient, , A left canine point, a right first molar point, and a left first molar point.

In the coordinate estimating unit according to an embodiment of the present invention, a change in coordinates of a point corresponding to a designated point on the wafer, which can move corresponding to movement of the patient's jaw, is estimated based on the second marker M _M , The change of the coordinates can be estimated as the value of the movement distance from the designated point.

Information on coordinates estimated by the coordinate estimating unit according to an embodiment of the present invention is displayed on the display unit in a different color according to the range of the moving distance value, and if the moving distance value is 0 or more and 1 millimeter or less, Information about the coordinates can be displayed in red if the range is from 1.5 to 2 millimeters, orange if the range is from 1.5 to 2 millimeters, and red if the range is greater than 2 to the limit value of the travel distance value.

According to an embodiment of the present invention, there is provided a method of providing an augmented reality (AR) reflecting movement estimation of a jaw of a patient, comprising: obtaining a CT image of a head and a neck of a patient; Defining a coordinate system for the patient using a plurality of points included in the converted image, recognizing a plurality of markers attached to the patient for motion estimation of the jaw of the patient, recognizing a plurality of recognized points based on the defined coordinate system Estimating coordinates of each of the plurality of markers, generating an augmented reality image of the jaws with respect to the estimated coordinates, and displaying the generated augmented reality image.

Meanwhile, as an embodiment of the present invention, a computer program stored in a medium may be provided in combination with hardware such as a computer to execute the above-described method.

According to the augmented reality system and method of the present invention, motion estimation of a patient's jaw is reflected, motion of the jaw of the patient estimated in real time at the time of surgery can be displayed as an augmented reality, Position and target position can be intuitively grasped.

Further, according to the augmented reality system and method thereof according to an embodiment of the present invention, since the movement information of the jaws is displayed in color as well as numerical values, the user can monitor the movement and position of the jaws through monitoring of the augmented reality images of the jaws And the exact position of the jawbone (e.g., the target position) can be quickly and accurately searched.

1 is a block diagram illustrating an augmented reality system in which motion estimation of a jaw of a patient is reflected according to an embodiment of the present invention.
2 is a schematic diagram of an augmented reality system that reflects motion estimation of a jaw of a patient according to an embodiment of the present invention.
3 is an example of a coordinate system definition according to an embodiment of the present invention.
4A and 4B show an example of a marker according to an embodiment of the present invention.
5 is an example of coordinate information acquisition of a marker according to an embodiment of the present invention.
6 is an example of a marker recognition unit according to an embodiment of the present invention.
7 is a view illustrating a reference point designation process according to an exemplary embodiment of the present invention.
Figure 8 shows a designated reference point in accordance with an embodiment of the present invention.
9A is an example of providing an augmented reality using a marker according to an embodiment of the present invention.
FIG. 9B is an example of providing estimated motion information of a jaw according to an embodiment of the present invention.
FIG. 10 is a flowchart illustrating a method for providing an augmented reality (AR) in which a motion estimation of a jaw of a patient is reflected according to an embodiment of the present invention.
FIG. 11 is an example of an augmented reality (AR) screen reflecting the estimated movement of the jaws of a patient provided according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software . In addition, when a part is referred to as being "connected" to another part throughout the specification, it includes not only "directly connected" but also "connected with other part in between".

Throughout the specification, the term "image" can refer to multi-dimensional data composed of discrete image elements (e.g., pixels in a two-dimensional image and voxels in a three-dimensional image). For example, the image may include a medical image or the like of the object obtained by the CT photographing apparatus.

In the present specification, a "CT (Computed Tomography) image" may mean a composite image of a plurality of x-ray images obtained by photographing an object while rotating around at least one axis of the object.

As used herein, an " object "may include a person or an animal, or a portion of a person or an animal. For example, the subject may comprise a bone, liver, heart, uterus, organs such as the brain, breast, abdomen, or blood vessels.

As used herein, the term "user" may be a doctor, a nurse, a clinical pathologist, a medical imaging specialist, or the like, and may be a technician repairing a medical device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an augmented reality system in which motion estimation of a jaw of a patient is reflected according to an embodiment of the present invention. Referring to FIG.

An augmented reality (AR) system 1000 incorporating motion estimation of a jaw of a patient according to an embodiment of the present invention includes an image acquisition unit 100 for acquiring a CT image of the head and the neck of a patient, A coordinate system setting unit 300 for defining a coordinate system for a patient using a plurality of points included in the converted image, a motion estimation unit 300 for calculating a motion estimation A marker recognition unit 400 for recognizing a plurality of markers attached to the patient, a plurality of markers 10, 20, and 30 attached to the patient for each of the plurality of markers recognized based on the defined coordinate system, An augmented reality image generation unit 600 for generating an augmented reality image of a jaw relative to the estimated coordinates, and a display unit 700 for displaying the generated augmented reality image, Include Can.

In the image conversion unit 200 according to an embodiment of the present invention, the CT image of the head and the neck of a patient can be converted into a three-dimensional image using DICOM (Digital Imaging and Communications in Medicine) data. In other words, in the image converting unit 200, the CT image of the head and the neck of the patient can be converted into a three-dimensional image. The user can perform a surgical simulation on the patient using the converted three-dimensional image through the image conversion unit 200. [ The user can estimate the target position of the jawbone of the patient (for example, a point, a direction, and the like to be positioned through the operation) through simulation of a surgical operation on the patient.

2 is a schematic diagram of an augmented reality system that reflects motion estimation of a jaw of a patient according to an embodiment of the present invention.

The first marker 10 may be placed on the support of the forehead region of the patient 1 on the table for surgery. In addition, the real-time movement path of the jawbone can be recognized (estimated) through the coordinate estimating unit 500 based on the second marker 20 attached to the wafer. The augmented reality image generator 600 may generate an augmented reality image along the estimated movement path. The third marker 30 may set a reference point for tracking the movement path and track the movement of the markers 10, 20 and 30 through the marker recognition unit 400 in real time. The image converting unit 200, the coordinate system setting unit 300, the coordinate estimating unit 500, and the augmented reality image generating unit 600 may be included in the user device provided with the display unit 700. The user device may be, but is not limited to, a personal computer (PC), a tablet PC, a notebook, or a smart phone. In other words, the user device may be an electronic device capable of wired / wireless communication with another device. Also, the user device may be connected to a server provided at a medical institution such as a hospital by wire or wirelessly. In other words, the user device may be connected to a server 2000 such as a Picture Archiving and Communication System (PACS), an Electronic Medical Record (EMR), a Personal Health Record (PHR), or a Radiology Information System (RIS) Read, store, update, and so on.

3 is an example of a coordinate system definition according to an embodiment of the present invention.

The coordinate system setting unit 300 according to an exemplary embodiment of the present invention generates a Frankfort horizontal plane (FHP) for a patient using three or more points in a three-dimensional image, We can use the bases to create a Midsagittal Reference Plane (MSRP), and define a patient coordinate for the patient at the intersection of the two generated faces. In addition, the coordinate system according to an embodiment of the present invention can be newly defined using various points other than the above-described method. For example, points for creating Frankfort horizontal plane may include Porion, Obitale, and the like. In addition, Crista galli, anterior nasal spine, posterior nasal spine, Basion, Opisthion, and Nasion may be included in the creation of the median sagittal plane, and Crista galli, Anterior nasal spine, Basion or Crista galli, Anterior nasal spine, Opisthion May be used to generate a median-sagittal reference plane.

4A and 4B show an example of a marker according to an embodiment of the present invention.

A plurality of markers 10, 20, 30 according to an embodiment of the present invention is in the form of a polygon having a different pattern on each face, and a first marker M _P 10 ), A second marker (M _M ) 20 attachable to the wafer made according to the shape of the patient's tooth, a third marker M _R for pointing the reference point of movement of the patient's jaws 30).

Different patterns may be displayed on each side of the markers as in FIG. 4A, and this pattern may perform an identifier function for each side of the marker. In other words, each marker can be distinguished through a pattern included in the image obtained through the marker recognition unit 400, and the position (e.g., coordinate information) of each marker can be recognized.

The first marker (MP) 10 may be disposed adjacent to the forehead of the patient 1. For example, as shown in Fig. 2 of the patient 1, it may be coupled to one end of an unvinvention intubation tube fixing support attached to the forehead of the patient 1. [ For convenience of explanation and understanding of the present invention, a first marker (MP) 10 may be positioned on the upper part of the skull modeled as shown in FIG. 4A. In other words, the first marker (MP) 10 of Fig. 4A shows an example of a state of being attached to the forehead of the patient 1. Fig.

In addition, the second marker 20 can be attached to a wafer 21 manufactured according to the shape of the teeth of the patient 1. As described above, a surgical wafer to be used before or at the time of orthognathic surgery can be manufactured in advance using a synthetic material or the like suitable for a patient's oral structure or the like. The second marker 20 attached to the wafer 21 may become a reference body for producing an augmented reality image for the upper part of the patient 1. [ In other words, the maximal motion of the patient 1 can be estimated based on the motion information of the second marker 20 attached to the wafer 21, and an augmented reality image can be generated according to the estimated motion.

The third marker 30 may be used for pointing a reference point (for example, a reference point) for tracking the movement of the jaws of the patient 1. [ One end of the third marker 30 may be used to designate a predetermined point (for example, a point between the central teeth, etc.) in the oral cavity of the patient in a pointed shape.

5 is an example of coordinate information acquisition of a marker according to an embodiment of the present invention.

The relationship between the patient 1 and the first marker 10 can be determined as shown in Fig. 5 based on the coordinate system defined by the coordinate system setting unit 300. [ In addition, the relationship between the patient 1 and the second marker 20 can also be determined based on the defined coordinate system. In other words, the coordinate value of the first marker 10 can be determined and the coordinate value of the second marker 20 can also be determined based on the center point of the coordinate system (e.g., {p}) defined for the patient. Determination of the initial coordinate values of the markers 10, 20, 30 may be referred to as registration. The markers 10,20 and 30 may be registered on the system 1000 in the order of the first marker 10, the third marker 30 and the second marker 20 in this order. In other words, the first marker 10 disposed adjacent to the forehead of the patient 1 is recognized through the marker recognition unit 400 and registered (for example, the marker recognition unit 400) through the coordinate estimation unit 500 A predetermined point in the oral cavity of the patient 1 is registered based on the third marker 30 and then the second marker 20 is registered on the wafer to which the second marker 20 is attached The second marker 20 can be registered in a state where the first marker 21 is in contact with the oral cavity of the patient 1. [

6 is an example of a marker recognition unit according to an embodiment of the present invention.

The marker recognition unit 400 according to an exemplary embodiment of the present invention may be a dual camera for acquiring images of respective markers having different patterns. The marker recognition unit 400 may be a single camera. The system 1000 according to an embodiment of the present invention can grasp the position of the head and neck of the patient in real time using the marker and can acquire the image obtained through the marker recognition unit 400 Can be utilized. In other words, the image photographed through the marker recognition unit 400 can be utilized as a reference image for generating an augmented reality image.

According to an embodiment of the present invention, the marker recognition unit 400 may be a camera including a plurality of photographing modules arranged side by side, preferably a dual camera. In other words, the marker recognition unit 400 may be a combination of a plurality of photographing modules arranged horizontally or vertically. The operation of the dual camera may be the same as the operation principle of a general dual camera.

FIG. 7 illustrates an example of a reference point designation process according to an exemplary embodiment of the present invention, and FIG. 8 illustrates reference points designated according to an exemplary embodiment of the present invention. FIG. 9A is an example of providing an augmented reality using a marker according to an embodiment of the present invention, and FIG. 9B is an example of providing estimated motion information of a jaw according to an embodiment of the present invention.

The third marker (MR) 30 according to an embodiment of the present invention can be used for pointing a point that is a reference point for tracking movement of the patient's jaw, A midpoint value P1, a right canine point P2, a left canine point P3, a right first molar point P4, and a left first molar point P5. However, the points in Fig. 8 are exemplary for explanation, and the designation of the reference point may be determined differently depending on the operation site, the osteotomized site, and the like. As shown in FIG. 7, the user can designate the reference point directly using the third marker 30. Such a reference point may be previously determined on the basis of a surgical site, a bone cutting site, or the like. Information (for example, coordinate values) for the designated point can be displayed on the screen as shown in FIG.

In the coordinate estimating unit 500 according to the embodiment of the present invention, a change in coordinates of a point corresponding to a designated point on the wafer 21 that can move in accordance with the movement of the patient's jaw, And the change of the coordinates can be estimated as the movement distance value d from the designated point. The travel distance of each point may be different.

Information on coordinates estimated by the coordinate estimating unit 500 according to an embodiment of the present invention is displayed through the display unit 700 in a different color according to the range of the moving distance value, Information about the coordinates may be displayed in red if the range is from 2 to the limit value of the travel distance value. If the range is less than 1 millimeter, green is displayed. As shown in FIG. 9B, the motion information may be expressed in color according to the movement distance value. The user can intuitively grasp the amount of movement through the represented color. In FIG. 9B, the distance indicates the movement distance of each point from the reference point (e.g., initial position) in accordance with the movement of the wafer, and the offset indicates the position of each point in the three-dimensional space in each direction of the x axis, y axis, Represents the distance traveled.

Also, as in 9a, the augmented reality image VI_2 can be generated according to the movement estimated by the reference point and the second marker 20. [ The generated augmented reality image VI_2 can be displayed through the display unit 700 together with the augmented reality image VI_1 of the patient's skull as described later with reference to Fig.

FIG. 10 is a flowchart illustrating a method for providing an augmented reality (AR) in which a motion estimation of a jaw of a patient is reflected according to an embodiment of the present invention.

A method of providing an AR with a motion estimation of a jaw of a patient according to an embodiment of the present invention includes acquiring a CT image of a head and a neck of a patient (S100), converting the obtained CT image into a 3D image (S200), defining a coordinate system for the patient using a plurality of points included in the converted image (S300), recognizing a plurality of markers attached to the patient for motion estimation of the patient's jaw S400), estimating coordinates of each of the plurality of recognized markers based on the defined coordinate system (S500), generating a jugular augmented reality image for the estimated coordinates (S600), and generating the generated augmented reality image (Step S700).

FIG. 11 is an example of an augmented reality (AR) screen reflecting the estimated movement of the jaws of a patient provided according to an embodiment of the present invention.

The augmented reality image VI_1 of the skull of the patient and the augmented reality image VI_2 based on the second marker 20 may be output together through the display unit 700. [ According to the embodiment of the present invention, the motion of the jaw of the patient estimated in real time can be represented as an augmented reality image as VI_2, and since VI_1 and VI_2 are simultaneously provided, the user can intuitively It is fast and accurate.

Also, as the user moves the wafer, the augmented reality image VI_2 may be output through the display unit 700 by moving in accordance with the movement. The user can follow the real time tracking of how the position of the jaw changes as the wafer moves. In other words, since movement information of the jaws is displayed in color as well as numerical values, the user can predict the movement and position of the jaw through monitoring the movement of the augmented reality image VI_2 on the wafer corresponding to the jaw to be operated on, The exact position of the jawbone (e.g., the final position of the operation) can be quickly and accurately searched.

With respect to the method according to an embodiment of the present invention, the contents of the above-described system can be applied. Therefore, the description of the same contents as the above-mentioned system is omitted in connection with the method.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

1: patient
10: first marker (M _P ) 20: second marker (M _M )
21: wafers 30: a third marker (M _R)
100: image acquisition unit 200: image conversion unit
300: coordinate system setting unit 400: marker recognition unit
410: first photographing module 420: second photographing module
500: Coordinate estimation unit 600: Augmented reality image generation unit
700:
1000: augmented reality system reflecting the motion estimation of the patient's jaw
2000: Server

Claims (10)

An Augmented Reality (AR) system that reflects the motion estimation of a patient's jaw,
An image obtaining unit for obtaining a CT image of the head and the neck of the patient;
An image converter for converting the obtained CT image into a three-dimensional image;
A coordinate system setting unit for defining a coordinate system for the patient using a plurality of points included in the converted image;
A plurality of markers attached to the patient for motion estimation of the jaw of the patient;
A marker recognition unit for recognizing a plurality of markers attached to the patient;
A coordinate estimator for estimating coordinates of each of the recognized plurality of markers based on the defined coordinate system;
An augmented reality image generation unit for generating an augmented reality image of a jaw relative to the estimated coordinates; And
And a display unit for displaying the generated augmented reality image.
The method according to claim 1,
Wherein the image conversion unit converts the CT image of the head and the neck of the patient into a three-dimensional image using DICOM (Digital Imaging and Communications in Medicine) data.
3. The method of claim 2,
Wherein the coordinate system setting unit generates a Frankfort Horizontal (FH) plane for the patient using three or more points in the 3D image, and generates a Frankfort Horizontal (FH) plane using the Nasion and Basion Wherein a midsagittal reference (MSR) plane is defined and a coordinate system for the patient is defined at a center point where the two generated faces cross each other.
The method according to claim 1,
Wherein the plurality of markers are in the form of polygons each having a different pattern on their respective sides, the first marker (M _P ) being arranged adjacent to the patient's forehead, the second markers 2 markers M _M and a third marker M _R for pointing the reference point of the movement of the patient's jaws in the augmented reality system.
5. The method of claim 4,
Wherein the marker recognition unit is a dual camera for acquiring images of the respective markers displayed with the different patterns.
5. The method of claim 4,
The third marker M _R may be used for pointing the reference point of the movement of the jaw of the patient. The designated point may include a point between the central incisor of the patient, a right canine point, A right first molar location, and a left first molar location.
The method according to claim 6,
Wherein the coordinate estimating unit estimates, based on the second marker (M _M ), a change in coordinates of a point corresponding to the designated point on the wafer that can move in accordance with the movement of the patient's jaw, Wherein the change is estimated as a movement distance value from the designated point.
8. The method of claim 7,
Information on coordinates estimated by the coordinate estimating unit is displayed through the display unit in a different color according to the range of the movement distance value,
If the moving distance value is equal to or greater than 0 and less than or equal to 1 millimeter, then green; if less than 1.5 millimeters, yellow; if less than 1.5 and greater than 2 millimeters, orange; Wherein the motion estimation of the jaw of the patient is reflected.
A method for providing an augmented reality (AR) in which a motion estimation of a jaw of a patient is reflected,
Obtaining a CT image of the head and the neck of the patient;
Converting the obtained CT image into a three-dimensional image;
Defining a coordinate system for the patient using a plurality of points included in the transformed image;
Recognizing a plurality of markers attached to the patient for motion estimation of the jaw of the patient;
Estimating coordinates for each of the recognized plurality of markers based on the defined coordinate system;
Generating an augmented reality image of a jaw corresponding to the estimated coordinates; And
And displaying the generated augmented reality image, wherein the augmented reality image is displayed on the display unit.
A computer program stored in a medium for execution in accordance with the method of claim 9 in combination with hardware.

Images