KR102383544B1 - Method for detecting marker in oral image and apparatus and method for oral image registration using the same - Google Patents

Abstract

The present invention provides a method for detecting a marker that is a reference in registering an oral scan image and a CT image, comprising the steps of aligning the oral scan image in a horizontal plane direction, estimating a marker candidate region from the aligned oral scan image, and providing a marker Detecting an undivided marker in the candidate region, dividing the undivided marker to estimate the marker from the undivided marker, and estimating an area up to a certain depth from the top surface of the undivided marker to the bottom as the marker To provide a method for detecting markers in an oral image.

Description

Method for detecting a marker in an oral image, and an apparatus and method for matching an oral image using the same

The present invention relates to a method for detecting a marker in an oral image, and an apparatus and method for matching an oral image using the same.

In computer vision, when a scene or object is photographed from a different time or point of view, images according to different coordinate systems are obtained. Image registration refers to processing for transforming such different images and displaying them in one coordinate system.

Through such image registration, it is possible to check the correspondence between images obtained through different measurement methods.

In a dental surgical guide software, an image registration process between a computed tomography (CT) image and an oral scan image is generally performed before entering the dental implant procedure planning stage.

The image registered through this image registration process becomes the basis for implant planning work that determines the safe and optimal implant placement position by identifying the bone tissue and neural tube position, so the accuracy of image registration is very important in the subsequent procedure. have

In the image registration method provided by the conventional medical software, an operator manually inputs a point that is a reference point for image registration, and image registration is performed based on this.

According to such a conventional image registration method, since the operator selects the reference point by roughly judging by the eye, the result is very inaccurate, so that the operator's manual operation process is inevitably followed after image registration. That is, the operator modifies the registration result by changing the position of the point or reselecting the point. As described above, according to the prior art, the operator spends a lot of time in image registration due to the continuous iterative process of registration and correction, and there is a problem in that satisfactory results cannot be obtained as much as the time spent.

As another conventional method, there is a method of acquiring an image including a marker for use as an intraoral registration standard, and matching an image obtained from a heterogeneous imaging device based on the marker in the image.

1 is a view showing a marker attached to the inside of the oral cavity for image registration.

Referring to FIG. 1 , a plurality of markers 1 is attached to a predetermined position in the oral cavity of the recipient using an adhesive (eg, resin). After that, when a certain time elapses, the adhesive layer 2 is formed between the marker 1 and the gum. At this time, since the amount and shape of the adhesive are different for each attachment position of the marker 1 , the shape of the adhesive layer 2 becomes non-uniform.

On the other hand, in detecting the marker 1 in the oral image for image registration, since the marker 1 and the adhesive layer 2 are integrally formed, the marker 1 and the adhesive layer 2 can be detected together. However, the non-uniform adhesive layer 2 becomes an obstacle to registration during image registration, and causes a problem of lowering the registration accuracy during image registration.

An object of the present invention is to provide an oral image registration apparatus and method for detecting a marker in an oral image, but removing a factor that interferes with registration from the detected marker, and automatically performing image registration with high accuracy using the detected marker do it with

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. There will be.

In order to solve the above problems, the present invention provides a method for detecting a marker as a reference in matching an oral scan image and a CT image, the steps of aligning the oral scan image in a horizontal plane direction; estimating a marker candidate region and detecting an undivided marker in the marker candidate region; dividing the undivided marker to estimate the marker from the undivided marker, but an area from the upper surface of the undivided marker to a predetermined depth in the lower direction It provides a method for detecting a marker of an oral image comprising the step of estimating as a marker.

In addition, the method for detecting a marker in an oral image of the present invention further includes the step of detecting the marker in the CT image by using the volume information of the marker.

In addition, the detecting of the undivided marker includes: generating a learning model by learning a plurality of learning images to which the markers are attached; inputting an oral scan image to the learning model; and an oral scan image based on the learning image. Including the step of detecting the undivided marker.

In addition, the estimating of the marker includes estimating an area exceeding a predetermined depth in the undivided marker as an adhesive layer interposed between the marker and the gum.

In addition, the step of detecting the undivided marker is a step of acquiring information on the number, size, and coordinates of the undivided marker.

Also, detecting the marker in the CT image includes generating a mask in a shape corresponding to the marker using volume information of the marker, and searching for the marker in the CT image using the mask.

In addition, the step of detecting the marker in the CT image is a step of obtaining information about the number, size, and coordinates of the markers.

In addition, the present invention provides a method of registering an oral scan image and a CT image using a marker, comprising the steps of aligning the oral scan image in a horizontal plane direction, estimating a marker candidate region from the aligned oral scan image, and performing a marker candidate region Detecting an undivided marker in Provided is an oral image registration method, comprising: detecting a marker in a CT image using volume information; and registering an oral scan image and a CT image using the marker.

In addition, the present invention provides an apparatus for registering an oral scan image and a CT image using a marker, comprising: an image aligning unit that aligns the oral scan image in a horizontal plane direction; An undivided marker detecting unit for detecting an undivided marker in the candidate region, dividing the undivided marker and estimating the marker from the undivided marker, and estimating an area up to a certain depth from the upper surface of the undivided marker to the lower side as the marker Provided is an oral image matching device comprising a marker estimator, a marker detector that detects a marker from a CT image by using the volume information of the marker, and an image matcher that matches an oral scan image and a CT image using the marker.

According to the present invention, the user's convenience is improved by detecting a marker in the oral image, but removing the element that interferes with the registration from the detected marker, and automatically performing image registration with high accuracy using the detected marker, and accordingly It has the effect of reducing the time required for implant planning and improving the accuracy of implant planning.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

1 is a view showing a marker attached to the inside of the oral cavity for image registration.
2 is a schematic block diagram of an oral image registration apparatus according to an embodiment of the present invention.
FIG. 3 is a detailed block diagram of the control unit of FIG. 2 .
4 is a flowchart of an oral image registration method according to an embodiment of the present invention.
5 is a diagram illustrating a marker candidate region estimated from an oral scan image.
6 is a diagram illustrating an undivided marker detected in the marker candidate region of FIG. 5 .
FIG. 7 is a diagram illustrating a marker included in the undivided marker of FIG. 6 and an adhesive layer separately.
FIG. 8 is an enlarged view of the undivided marker of FIG. 7 .
9 is a diagram illustrating markers estimated from an oral scan image.
10 is a perspective view illustrating a marker according to an embodiment of the present invention.
11 is a view illustrating a CT image generated by photographing the oral cavity to which the marker of FIG. 10 is attached.
12 is a diagram illustrating a marker detected in the CT image of FIG. 11 .
13 is a view illustrating registration of an oral scan image and a CT image using the oral image registration method according to an embodiment of the present invention.

The above object and means of the present invention and its effects will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily understand the technical idea of the present invention. will be able to carry out In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In this specification, terms such as “or” and “at least one” may indicate one of the words listed together, or a combination of two or more. For example, “or B” and “at least one of B” may include only one of A or B, or both A and B.

In this specification, terms such as 'first' and 'second' may be used to describe various components, but the components should not be limited by the above terms. In addition, the above terms should not be construed as limiting the order of each component, and may be used for the purpose of distinguishing one component from another. For example, a 'first component' may be referred to as a 'second component', and similarly, a 'second component' may also be referred to as a 'first component'.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a schematic block diagram of an oral image registration apparatus according to an embodiment of the present invention, and FIG. 3 is a detailed block diagram of the controller of FIG. 2 .

The oral image registration apparatus 100 according to an embodiment of the present invention is an electronic device executable by a computer program, and is an apparatus for matching an oral scan image and a CT image of a subject by using a marker as a registration reference point. Here, the number of markers may be at least three, but is not limited thereto.

The oral scan image may provide information on the shape of the crown portion of the tooth exposed to the outside as a 3D image and the shape of the gum around the tooth. At this time, the oral scan image is obtained by directly scanning the inside of the recipient's oral cavity through an oral scanner, etc. The scan image of the impression model can be inverted and used as an oral scan image.

The CT image, as a three-dimensional image, may provide information on not only the crown, but also the shape of the root and alveolar bone, and the bone density of the crown, root, and alveolar bone. In this case, the CT image may be obtained by performing a CT scan of the inside of the recipient's oral cavity.

The registered image, in which the oral scan image and the CT image are matched, becomes the basic image for implant planning work that determines the safe and optimal implant placement position by identifying the bone tissue and neural tube position of the recipient.

Referring to FIG. 2 , the oral image matching apparatus 100 according to an embodiment of the present invention may include a communication unit 110 , an input unit 120 , a display unit 130 , a storage unit 140 , and a control unit 150 . can

The communication unit 110 is a component that communicates with an external device such as an image acquisition device (not shown) and a server (not shown), and may receive image data. For example, the communication unit 110 is 5 ^th generation communication (5G), long term evolution-advanced (LTE-A), long term evolution (LTE), Bluetooth, bluetooth low energe (BLE), near field communication (NFC). It is possible to perform wireless communication such as cable communication and the like, and can perform wired communication such as cable communication. In this case, the image data may include oral scan image data and CT image data.

The input unit 120 generates input data in response to a user's input, and may include at least one input means. For example, the input unit 120 includes a keyboard, a keypad, a dome switch, a touch panel, a touch key, a mouse, and a menu button ( menu button) and the like.

The display unit 130 displays display data according to the operation of the oral image matching apparatus 100 . The display data may include image data. For example, the display unit 130 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, and a micro electromechanical system (MEMS). mechanical systems) displays and electronic paper displays. Also, the display unit 130 may be combined with the input unit 120 to be implemented as a touch screen.

The storage unit 140 stores various information and programs necessary for the operation of the oral image matching apparatus 100 . For example, the storage unit 140 may store image data received from an image acquisition device, etc., and an algorithm related to image alignment, marker estimation and detection, and image registration method according to an embodiment of the present invention, which will be described later. . Also, the storage unit 140 may store a learning model obtained by learning a plurality of learning images by a deep learning method in order to estimate the marker.

The controller 150 detects a marker from oral image data received from an image acquisition device or server or pre-stored in the storage 140, and performs an operation of matching the oral image data using the detected marker as a reference point. To this end, the control unit 150 may receive the oral image data from the image acquisition device or the server and store it in the storage unit 140 . Also, the controller 150 may control the operations of the communication unit 110 , the input unit 120 , the display unit 130 , and the storage unit 140 .

Referring to FIG. 3 , the controller 150 may include an image alignment unit 151 , an undivided marker detection unit 152 , a marker estimator 153 , a marker detection unit 154 , and an image matching unit 155 . there is.

First, the image acquisition device acquires an oral scan image and a CT image, respectively, by photographing the inside of the oral cavity to which the marker is attached. In this case, the acquired oral scan image and CT image may be directly transmitted to the image alignment unit 151 or may be stored in the storage unit 140 and then transmitted to the image alignment unit 151 .

The image alignment unit 151 aligns the oral scan image in the horizontal direction so that the marker faces upward in the oral scan image.

The undivided marker detection unit 152 estimates a marker candidate region in the oral scan image aligned by the image alignment unit 151 and detects an undivided marker in the marker candidate region.

The marker estimating unit 153 divides the undivided marker detected by the marker candidate detection unit 152 up and down to estimate the marker from the undivided marker, An area is estimated as a marker, and the area other than that is estimated as an adhesive layer.

The marker detection unit 154 detects the marker by using the volume information of the marker in the CT image.

The image matching unit 155 matches the oral scan image and the CT image using the markers estimated and detected by the marker estimator 153 and the marker detector 154 , respectively.

As such, according to the oral image registration apparatus 100 of the present invention, a marker is detected in an oral image, an adhesive layer that interferes with registration is removed from the detected marker, and the detected marker is used as a registration reference point to image with high accuracy. By automatically performing registration, the user's convenience is improved, and accordingly, there is an effect of reducing the time required for implant planning and improving the accuracy of implant planning.

Hereinafter, a method for detecting a marker of an oral image and a method for matching an oral image according to an embodiment of the present invention that are controlled by the controller 150 will be described.

The method for detecting a marker of an oral image and a method for registering an oral image according to an embodiment of the present invention is characterized in that the oral image of the recipient is registered, in particular, an edentulous oral image. Here, since there is no tooth that can be used as a reference point for registration in the edentulous jaw, an oral image is taken while a marker is attached to the recipient's gums, and the oral image is matched using this marker as a reference point.

4 is a flowchart of an oral image registration method according to an embodiment of the present invention.

First, a marker is attached to a plurality of positions (eg, gums) in the oral cavity of the recipient using an adhesive such as resin. At this time, when a predetermined time elapses, an adhesive layer is formed between the marker and the gum. Thereafter, a heterogeneous image acquisition device captures the inside of the oral cavity to which the marker is attached to acquire an oral scan image and a CT image, respectively.

Referring to FIG. 4 , the oral image registration method according to an embodiment of the present invention is a method of registering an oral scan image and a CT image using a marker, and includes the steps of aligning the oral scan image (S10) and the marker candidate region Detecting the undivided marker in (S20), estimating the marker from the undivided marker (S30), detecting the marker from the CT image (S40), and using the marker to obtain an oral scan image and CT image It may include a matching step (S50).

5 is a diagram illustrating a marker candidate region estimated from an oral scan image, and FIG. 6 is a diagram illustrating an undivided marker detected in the marker candidate region of FIG. 5 .

Hereinafter, the marker 11 and the adhesive layer 12 will be collectively referred to as an undivided marker 11b.

4 to 6 , the oral scan image 10 is aligned in the horizontal direction so that the marker 11 faces upward in the oral scan image 10 ( S10 ). Then, the candidate marker region 11a is estimated from the aligned oral scan image 10 , and the undivided marker 11b is detected from the candidate marker region 11a ( S20 ).

The undivided marker 11b may be detected using a learning model. That is, by using a deep learning method (eg, YOLO (You Only Look Once) algorithm), a number of learning images to which the marker 11 is attached are learned to generate a learning model, and to the generated learning model The unsegmented marker 11b may be detected from the oral scan image 10 by inputting the recipient's oral scan image 10 . Here, information on the number, size, and coordinates of the undivided markers 11b may be obtained.

Since the undivided marker 11b includes not only the marker 11 but also the non-uniform adhesive layer 12, that is, since the marker 11 and the adhesive layer 12 are integrally formed, the shape is non-uniform. Accordingly, in order to improve the matching accuracy, it is necessary to divide the marker 11 from the undivided marker 11b.

FIG. 7 is a view showing the marker included in the undivided marker of FIG. 6 and the adhesive layer separately, FIG. 8 is an enlarged view of the undivided marker of FIG. 7 , and FIG. 9 shows the marker estimated from the oral scan image it is one drawing

Referring to FIGS. 7 to 9 , an area from the top surface of the undivided marker 11b to a predetermined depth d1 in the downward direction is estimated as the marker 11 ( S30 ). In addition, an area exceeding the predetermined depth d1 (eg, 5 mm) in the undivided marker 11b is estimated as the adhesive layer 12 interposed between the marker 11 and the gum. That is, an area from the lower surface of the undivided marker 11b to a predetermined height d2 in the upper direction is estimated as the adhesive layer 12 . Accordingly, the marker 11 can be divided from the undivided marker 11b.

On the other hand, unlike the drawing, by aligning the oral scan image 10 in the horizontal plane direction so that the marker 11 faces downward in the oral scan image 10, the undivided marker 11b is detected and the undivided marker 11b) It is also possible to divide the marker 11 in . At this time, an area from the lower surface of the undivided marker 11b to a predetermined height d1 in the upward direction is estimated as the marker 11, and the predetermined height d1 (eg, 5 mm) from the undivided marker 11b. ) is estimated as the adhesive layer 12 interposed between the marker 11 and the gum.

10 is a perspective view exemplarily showing a marker according to an embodiment of the present invention, FIG. 11 is a view showing a CT image generated by photographing an oral cavity to which the marker of FIG. 10 is attached, and FIG. It is a diagram showing a marker detected in a CT image.

10 to 12 , the marker 11 may have a three-dimensional shape (eg, a cuboid) having a predetermined volume. Here, the volume of the rectangular parallelepiped may be defined as the product of width, length, and height (eg, 4 mm×5 mm×5 mm). In addition, at least three or more markers 11 may be attached to a predetermined position in the oral cavity (eg, gums), but the present invention is not limited thereto.

Meanwhile, referring to FIG. 11 , the marker 11 may be made of a radiopaque material (eg, alumina), and the adhesive layer 12 may be made of a radioactive material. Accordingly, in the CT image 20 , the marker 11 including the crown, root, and alveolar bone is displayed, but the adhesive layer 12 is not displayed. Accordingly, in the CT image 20 , it is not necessary to divide the marker 11 and the adhesive layer 12 like the oral scan image 10 . However, it is difficult to accurately detect the marker 11 because it is not easy to distinguish between the crown, tooth root, and alveolar bone and the marker 11 .

In order to accurately detect the marker 11 in the CT image 20 as described above, the marker 11 is detected in the CT image 20 by using the volume information of the marker 11 ( S40 ). Specifically, a mask is created in a shape corresponding to the marker 11 by using the volume information of the marker 11 . Then, the marker 11 is searched for in the CT image 20 using the mask, and the marker 11 having a shape corresponding to the mask is detected. Here, information on the number, size, and coordinates of the markers 11 may be obtained.

13 is a diagram illustrating registration of an oral scan image and a CT image by an oral image registration method according to an embodiment of the present invention.

Referring to FIG. 13 , the oral scan image 10 and the CT image 20 are automatically matched using the marker 11 estimated and detected from the oral scan image 10 and the CT image 20 as a matching reference point. (S50).

As such, according to the oral image registration method of the present invention, the marker 11 is detected in the oral image, but the adhesive layer 12 that interferes with the registration is removed, and the detected marker 11 is used as a registration reference point with high accuracy. By automatically performing image registration, user convenience is improved, and accordingly, there is an effect of reducing the time required for implant planning and improving the accuracy of implant planning.

In the detailed description of the present invention, although specific embodiments have been described, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention is not limited to the described embodiments, and should be defined by the following claims and their equivalents.

100: oral image registration device
150: control unit
151: image alignment unit
152: undivided marker detection unit
153: marker estimation unit
154: marker detection unit
155: image matching unit

Claims (9)

A method for detecting a marker that is a reference in matching an oral scan image and a CT image, the method comprising:
aligning the oral scan image in a horizontal plane direction by an image aligning unit;
estimating, by an undivided marker detector, a marker candidate region from the aligned oral scan image, and detecting an undivided marker in the marker candidate region; and
A marker estimating unit dividing the undivided marker and estimating the marker from the undivided marker, and estimating an area from the top surface of the undivided marker to a predetermined depth in the downward direction as the marker,
The step of the marker estimator estimating the marker
estimating an area exceeding the predetermined depth in the undivided marker as an adhesive layer interposed between the marker and the gum
A method for detecting markers in oral images.
The method of claim 1,
The method further comprising the step of detecting the marker in the CT image by a marker detection unit using the volume information of the marker
A method for detecting markers in oral images.
The method of claim 1,
The step of detecting the undivided marker by the undivided marker detection unit comprises:
generating a learning model by learning the plurality of learning images to which the markers are attached;
inputting the oral scan image to the learning model; and
detecting the unsegmented marker in the oral scan image based on the learning image
A method for detecting a marker in an oral image comprising a.
delete The method of claim 1,
The step of detecting the undivided marker by the undivided marker detection unit comprises:
acquiring the number, size, and coordinate information of the undivided marker
A method for detecting markers in oral images.
3. The method of claim 2,
The step of the marker detecting unit detecting the marker in the CT image
generating a mask in a shape corresponding to the marker by using the volume information of the marker; and
Searching for the marker in the CT image using the mask
A method for detecting a marker in an oral image comprising a.
3. The method of claim 2,
The step of the marker detecting unit detecting the marker in the CT image
The step of obtaining the number, size, and coordinate information of the marker
A method for detecting markers in oral images.
A method of registering an oral scan image and a CT image using a marker, comprising:
aligning the oral scan image in a horizontal plane direction by an image aligning unit;
estimating, by an undivided marker detector, a marker candidate region from the aligned oral scan image, and detecting an undivided marker in the marker candidate region;
estimating the marker from the undivided marker by dividing the undivided marker by a marker estimator, and estimating an area from the upper surface of the undivided marker to a predetermined depth downward as the marker;
detecting, by a marker detector, the marker in the CT image by using the volume information of the marker; and
Comprising the step of registering the oral scan image and the CT image by an image matching unit using the marker,
The step of the marker estimator estimating the marker
estimating an area exceeding the predetermined depth in the undivided marker as an adhesive layer interposed between the marker and the gum
Oral image registration method.
An apparatus for registering an oral scan image and a CT image using a marker, comprising:
an image alignment unit for aligning the oral scan images in a horizontal direction;
an undivided marker detection unit estimating a marker candidate region from the aligned oral scan image and detecting an undivided marker from the marker candidate region;
a marker estimating unit estimating the marker from the undivided marker by dividing the undivided marker, and estimating, as the marker, an area up to a predetermined depth from the top surface of the undivided marker to the bottom;
a marker detector configured to detect the marker in the CT image by using the volume information of the marker; and
An image matching unit for matching the oral scan image and the CT image using the marker,
The marker estimating unit
Estimating the area exceeding the predetermined depth in the undivided marker as an adhesive layer interposed between the marker and the gum
Oral image registration device.

Images