TWI576084B - Intraoral scanning device and scanning method thereof - Google Patents

Description

Intraoral scanning device and scanning method

BACKGROUND OF THE INVENTION 1. Field of the Invention This invention relates to an intraoral scanning device, and more particularly to a line laser intraoral scanning device for establishing a three-dimensional tooth mold.

Intraoral scanners are a device that is increasingly valued by the dental community and are considered to be an important key to digital dentistry. This device is primarily operated by a dentist and takes a digital model of the tooth directly into the patient's mouth. Intraoral scanners are classified into several types according to their light source and imaging method, such as structured light, laser light, or the use of images for identification and modeling. However, because of the need to multi-angle the teeth in the oral cavity, the image capture module (eg, image sensor) typically needs to be rotated or moved, which also increases the size or weight of the device. Therefore, how to solve this problem is a topic of concern to those skilled in the field.

Embodiments of the present invention provide an intraoral scanning device including a projection source, a stereo splitting module, a reflective module, an image sensor, and a processor. The projection source is used to project a line of lasers. The vertical splitting module is set on the projection path of the projection source On the path. The reflection module is disposed on the other side of the vertical splitting module relative to the projection source, and is disposed on the projection path of the projection source to reflect the line laser to the object. The object is located on the image sensing path of the image sensor, and the image sensor is used to capture an image of the corresponding object. The processor is configured to establish a stereo tooth model of the corresponding object according to the image.

In some embodiments, the stereo splitting module includes a Mitsubishi mirror. The Mitsubishi mirror has a first side, a second side and a pair of sides, the first side and the opposite side are disposed on the projection path of the projection source, and the second side and the opposite side are disposed on the image sensing path. In some embodiments, the projection source is disposed opposite the first side of the Mitsubishi mirror, and the image sensor is disposed opposite the second side of the Mitsubishi mirror.

In some embodiments, the intraoral scanning device described above includes a body having an extension and a grip. The reflective module is disposed on a side of the extending portion relative to the grip portion; the projection source, the vertical splitting module, and the image sensor are disposed in the grip portion.

Embodiments of the present invention provide an intraoral scanning device including a projection source, a rotation module, an image sensor, and a processor. The projection source is used to project a line of lasers. The first reflection module is disposed on the projection path of the projection source for reflecting the line laser to the object. The rotating module is coupled to the first reflective module, and the object is located on the image sensing path of the image sensor. When the rotating module rotates the first reflective module, the image sensor is used to obtain multiple images of the corresponding object, and the processor is configured to establish a stereo tooth model of the corresponding object according to the image.

In some embodiments, the intra-oral scanning device further includes a body having an extension and a grip. The first reflection module and the rotation module are disposed at one end of the extension portion, and the projection source and the image sensor are disposed at the grip portion.

In some embodiments, the intraoral scanning device further includes a second reflective module disposed between the first reflective module and the projection source, and the second reflective module is on the image sensing path and not on the projection path.

Embodiments of the present invention provide an intraoral scanning method suitable for use in an intraoral scanning device. The intraoral scanning device includes a projection source to project a line of laser light onto the object. This method includes the following steps. First, an image of the corresponding object is obtained. Then, a plurality of feature points corresponding to the line laser are obtained on the image, and the step includes the following three sub-steps: obtaining a plurality of pixels of a line width corresponding to the line laser; and calculating a plurality of virtual pixels between the pixels according to the filter And obtaining the virtual pixel having the largest grayscale value from the virtual pixel as one of the feature points. Finally, the total least squares feature point sampling method is performed according to the feature points to calculate the line segment.

In the above-described intraoral scanning device, the volume and weight of the intraoral scanning device can be reduced due to the arrangement of the rotation module and the standing beam module.

The above described features and advantages of the invention will be apparent from the following description.

100‧‧‧ intraoral scanning device

110‧‧‧ Subject

112‧‧‧ grips

114‧‧‧Extension

120‧‧‧Projection source

122‧‧‧ line laser

130‧‧‧Reflective Module

132‧‧‧ objects

140‧‧‧Rotary Module

150‧‧‧Image sensor

152‧‧‧Reflective Module

160‧‧‧ processor

200‧‧‧ intraoral scanning device

210‧‧‧ Subject

212‧‧‧ grip

214‧‧‧Extension

220‧‧‧Projection source

230‧‧‧立分光模块

231‧‧‧Mitsubishi mirror

231a, 231b‧‧‧ side

231c‧‧‧ opposite

240‧‧‧Reflective Module

242‧‧‧ objects

250‧‧‧Image Sensor

260‧‧‧ processor

310‧‧‧Projection source

311‧‧‧ line laser

320‧‧‧ objects

321, 331‧‧ ‧ line segments

322, 332‧‧‧ endpoints

330‧‧‧ images

340‧‧‧Image sensor

410‧‧‧Standard objects

511~514‧‧‧Endpoint

601~607‧‧‧ pixels

611‧‧‧virtual pixels

701~704, 721~724‧‧‧ feature points

710, 730‧‧ ‧ line segments

S801~S810, S901~S907, S1010, S1020~1023, S1030‧‧‧ steps

1 is a schematic view showing the configuration of an intra-oral scanning device according to a first embodiment.

FIG. 2 is a schematic configuration diagram of an intra-oral scanning device according to a second embodiment.

FIG. 3 is a schematic view showing calculation of a three-dimensional coordinate according to a line laser according to a third embodiment.

FIG. 4 is a schematic diagram showing a case where a correction program is executed according to the third embodiment.

Fig. 5A is a plan view showing a standard article according to a third embodiment.

Fig. 5B is a side view showing a standard article according to a third embodiment.

Fig. 6 is a schematic view showing the acquisition of feature points according to the third embodiment.

FIG. 7 is a schematic diagram showing calculation of a line segment according to different methods according to a third embodiment.

FIG. 8 is a flow chart showing a coarse sweep phase according to a third embodiment.

Fig. 9 is a flow chart showing a fine sweeping stage according to a third embodiment.

FIG. 10 is a flow chart showing a method of scanning in the mouth according to the third embodiment.

The terms "first", "second", "etc." used in this document are not intended to mean the order or the order, and are merely to distinguish between elements or operations described in the same technical terms. In addition, as used herein, "coupled" may mean that two elements are electrically connected, either directly or indirectly. That is, when the following description "the first object is coupled to the second object", other items may be disposed between the first object and the second object.

[First Embodiment]

1 is a schematic view showing the configuration of an intra-oral scanning device according to a first embodiment. Referring to FIG. 1 , the intraoral scanning device 100 includes a main body 110 . The main body 110 is provided with a projection source 120 and a reflection module 130 (also referred to as a reflection module 130 ). The first reflection module), the image sensor 150 and the processor 160. In this embodiment, the body 110 includes a grip portion 112 and an extension portion 114, and the user can hold the grip portion 112 and extend the extension portion 114 into the oral cavity of the patient. However, the present invention does not limit the specific shape and material of the grip portion 112 and the extension portion 114. In some embodiments, the body 110 can also be assembled from a plurality of different parts.

Projection source 120 is used to project line laser 122. In some embodiments, the projection source 120 projects a single wavelength source of short wavelength (eg, less than 460 nm) that effectively reduces the reflection of saliva, and is generally a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensors have strong energy sensitivity to this band, helping to achieve finer images.

The reflection module 130 is disposed on a projection path of the projection source 120 for reflecting the line laser 122 to the object 132 (eg, a tooth). The reflective module 130 can be an active component or a passive component, such as a laser light source projector, for converting a light source projected onto the surface of the reflective module 130 into a specific linear pattern, and These line patterns are reflected to the surface of the object 132 so that no mechanical transmission elements are required.

If the reflection module 130 is implemented as a passive component (for example, a mirror), it can be used together with the rotation module 140. The rotating module 140 is coupled to the reflective module 130 for rotating the reflective module 130. The rotary module 140 can be a piezoelectric motor or other suitable form of motor. According to actual needs, the rotation module 140 can have any The rotation angle and the direction of rotation are not limited thereto.

The image sensor 150 is configured to generate a digital image according to the sensed light, wherein the object 132 is on the image sensing path of the image sensor 150. For example, the intra-oral scanning device 100 may further include a reflective module 152 (also referred to as a second reflective module). The reflective module 152 is disposed between the reflective module 130 and the projection source 120. The reflective module 152 is located. The image sensing path described above is not on the projection path. The reflection of the line laser on the object 132 passes through the reflection module 152 and enters the image sensor 150. However, those skilled in the art can design other projection paths and image sensing paths, for example, more reflective modules or lenses, and the present invention is not limited thereto.

The processor 160 is electrically coupled to the projection source 120 and the image sensor 150, such as a microprocessor or any type of programmable circuit. When the reflection module 130 reflects the line laser to the object 132, that is, when the rotation module 140 rotates the reflection module 130, the image sensor 150 acquires multiple images of the corresponding object 132. The processor 160 will take these images and create a stereo model corresponding to the object 132 based on the images. However, it is understood by those skilled in the art that when the plane equation to which the line laser belongs and the internal parameters of the image sensor 150 are both known, the correspondence on the object 132 can be calculated from the pixels in the two-dimensional image. The three-dimensional coordinates of the point. The present invention does not limit the processor 160 to establish a stereo tooth profile based on what algorithm.

In this embodiment, the projection source 120, the image sensor 150 and the processor 160 are disposed in the grip portion 112, and the reflection modules 130, 152 and the rotation module 140 are disposed in the extension portion 114. However, in other embodiments these elements may also be arranged differently depending on the actual needs. The location of the invention is not limited thereto.

[Second embodiment]

FIG. 2 is a schematic diagram showing the configuration of an intra-oral scanning device according to a second embodiment. In the second embodiment, the intra-oral scanning device 200 includes a body 210 having a grip portion 212 and an extension portion 214. The intraoral scanning device 200 includes a projection source 220, a cube beam splitting module 230, a reflection module 240, an image sensor 250, and a processor 260.

The projection source 220 is used to project a line laser, and the vertical beam splitting module 230 is disposed on a projection path of the projection source 220 for splitting the line laser. For example, the stereo splitting module 230 includes a Mitsubishi mirror 231 that generates a plurality of line lasers after passing through the Mitsubishi mirror 231. In some embodiments, the vertical splitting module 230 further includes one or more lenses, but the invention is not limited thereto. The vertical splitting module 230 has two sides, the projection source 220 is located on one side thereof, and the reflection module 240 is disposed on the other side with respect to the projection source 220. The reflection module 240 is located on the projection path of the projection source 220 for reflecting the split line laser to the object 242. In particular, the light reflected from the object 242 is also reflected by the reflection module 240 and then enters the image sensor 250 through the vertical light module 230. Therefore, the object 242 will be in the image sensing path of the image sensor 250, and the image sensor 250 will capture the image of the corresponding object 242, and the processor 260 will establish a stereo tooth model corresponding to the object 242 according to the images. In the arrangement of FIG. 2, the projection path and the image sensing path are partially overlapped, which forms a coaxial optical path architecture, which can reduce the volume of the intra-oral scanning device 200. More specifically, the Mitsubishi mirror 231 has a first side 231a, a second side 231b, and a opposite side 231c. The projection source 220 and the first side 231a are Oppositely, the image sensor 250 is disposed opposite to the second side 231b, that is, the first side 231a and the opposite side 231c are disposed on the projection path of the projection source 220, and the second side 231b and the opposite side The 231c is placed on the image sensing path.

In this embodiment, the reflective module 240 is disposed on the side of the extending portion 214 relative to the grip portion 212; and the remaining components, such as the projection source 220, the vertical splitting module 230, the image sensor 250, and the processing The device 260 is disposed in the grip portion 212. However, the positions of the components in FIG. 2 are merely examples, and the present invention should not be limited thereto.

[Third embodiment]

The present disclosure proposes a scanning method and a correction method in the third embodiment, which can be used in the intraoral scanning device in the first embodiment or the second embodiment. 3 is a schematic diagram of calculating a three-dimensional coordinate from a line laser according to a third embodiment. In the example of FIG. 3, projection source 310 projects line laser 311 to spherical object 320 and forms line segment 321 on object 320. The image sensor 340 captures the image 330 of the object 320, and the line segment 321 corresponds to the line segment 331 on the image 330, wherein the endpoint 322 corresponds to the endpoint 332. Depending on the model of the image sensor 340 and the different algorithms, there are many ways to calculate the three-dimensional coordinates, which are described below in the simplest manner. It is assumed that the model of the image sensor 340 is a pinhole, with the image sensor 340 as the origin, and the coordinates of the end point 322 in the real world coordinates are (X, Y, Z). In addition, the coordinates of the endpoint 332 in the image coordinates are (x, y). The distance between the image 330 and the image sensor 340 is the focal length of the image sensor 340 (hereinafter referred to as f). According to the above settings, the following equations (1) and (2) are satisfied.

On the other hand, the equation of the plane formed by the line laser 311 can be expressed as AX+BY+CZ+D=0, where A, B, C, and D are real numbers. Substituting equations (1) and (2) into the above plane equation, the following equation (3) can be obtained.

In other words, according to the coordinates (x, y), the focal length f and the plane equation, the depth of field Z can be calculated, and the equations (1) and (2) can be substituted to obtain the coordinates X and Y. In the above calculation, the focal length f is a necessary parameter and must be obtained by means of correction. When the model of the image sensor 340 is more complicated, more parameters are required. In the general case, if represented by a homogeneous coordinate, the coordinates of a point on the image 330 will have three dimensions, and the coordinates of a point on the real world coordinate will have four dimensions. The camera matrix converts four-dimensional coordinates into three-dimensional coordinates. Therefore, the size of this camera matrix is 3x4, with a total of 12 variables. If the coordinates and correspondence of endpoints 322 and endpoints 332 are known, then three solutions can be provided, so that four sets of known correspondences are needed to obtain the 12 variables in the camera matrix.

FIG. 4 is a schematic diagram showing the execution of a correction procedure according to the third embodiment. Fig. 5A is a plan view showing a standard article according to a third embodiment. Fig. 5B is a side view showing a standard article according to a third embodiment. Please refer to the figure 4. Figures 5A and 5B, in this embodiment, the standard object 410 is used to perform the correction of the parameters. When the line laser is projected on the standard object 410, a polygonal line is generated at the end points 511 to 514. Since the size and size of the standard object 410 are known, the end points 511 to 514 can be obtained at the real world coordinates by image processing. The location on the top. As described above, the variables in the camera matrix can be calculated after four sets of known correspondences are obtained. In some embodiments, more than one line of laser light can also be projected onto the standard object 410, resulting in more than four correspondences. After obtaining these correspondences, an optimization algorithm can be performed to calculate the variables in the camera matrix. However, the shape of the above-described standard object 410 is merely an example, and those skilled in the art have a standard object that can design other shapes, and the present invention is not limited thereto. It is to be noted that, in FIG. 4, the intraoral scanning device in the second embodiment is taken as an example, and the standard object 410 can also be used in conjunction with the intraoral scanning device in the first embodiment.

As can be seen from the above discussion, it is necessary to obtain a specific point in the image whether in the scanning program or the calibration program. However, referring back to FIG. 3, when the line laser 311 is projected onto the object 320, the line segment 331 presented in the image 330 will have a certain width, and how to take a point from the line segment having the width will be explained below ( Also known as feature points).

FIG. 6 is a schematic diagram showing the acquisition of feature points according to the third embodiment. Referring to FIG. 6, the horizontal axis on the coordinate axis represents the position of the pixel, and the pixels 601 to 607 are from left to right. These pixels are, for example, across the line segment 331 of FIG. 3; the vertical axis represents the gray of the pixels 601-607. Order value. In this embodiment, the goal is to obtain the point with the largest grayscale value (ie, the brightest) as the feature point, and it can be seen from the figure that the grayscale values of the pixels 603, 604 are the largest. But because of the resolution The relationship between the pixels 603, 604 having the largest gray scale value does not necessarily have corresponding pixels, so a filter can be used to interpolate the virtual pixels between the pixels 603, 604. For example, the Remez finite impulse response filtering method may be used in some embodiments to calculate virtual pixels, although other types of filters may be used in other embodiments, although the invention does not limit the type and coefficients of the filters. Next, one of the virtual pixels having the largest grayscale value (ie, the virtual pixel 611) can be obtained as a feature point.

The above method can solve the problem of the width of the line segment 331. However, after obtaining a plurality of feature points on the line segment 331, it is found that these feature points are not necessarily connected in a line, and the following will be based on the total least square (TLS) feature. Point sampling is used to solve this problem. Please refer to FIG. 7. The left side of FIG. 7 is a curve drawn according to the total least squares feature point sampling method, and the right side of FIG. 7 is a curve drawn according to the algebraic least square (ALS) feature point sampling method. It can be seen from Fig. 7 that the total least squares feature point sampling method calculates the vertical distance between the feature points 701~706 and the like and the line segment 710, and the algebra least squares feature point sampling method calculates the feature points 721~724. The distance on the Y-axis between line segments 730 (for simplicity, not all feature points are indicated). It can be known from the experimental results that the results of the total least squares feature point sampling method will be better. Specifically, the coefficient of the equation of the line segment 710 can be set as a variable, and the vertical distance between the feature points 701-704 and the line segment 710 can be set as the objective function, and then the coefficient of the line segment 710 can be obtained by performing the optimization algorithm. .

The steps of the scanning procedure will be described in detail below, in which the scanning procedure is divided into a coarse sweep phase and a fine sweep phase. Figure 8 is based on the third real The embodiment shows a flow chart of the rough sweep phase, please refer to FIG. 8 first. In step S801, an initial procedure is first performed to adjust the angle of the rotation module 140 of FIG. 1 to an initial angle to ensure that the reference points of each scan are consistent. In step S802, it is determined whether or not the correction procedure is performed. If the intraoral scanning device does not rotate accurately or is subjected to an external force, the original calibration position will be shifted. Therefore, if this happens, a calibration procedure must be performed.

If the result of step S802 is YES, a correction procedure is performed (steps S803 to S805). In step S803, two-dimensional coordinates of four feature points on the standard object are extracted from the image. In step S804, three-dimensional coordinates of these feature points on real world coordinates are obtained. In step S805, a deviation value is calculated according to the two-dimensional coordinates and the three-dimensional coordinates described above, and the deviation value can be used to correct the variable in the camera matrix. In some embodiments, the average longitude information, the standard calibration point information, and the mode orientation difference information are also calculated in step S805. The average accuracy information is used to reconstruct the average error rate of the Y-axis and the Z-axis in the three-dimensional coordinate system. The standard calibration point information is used to reconstruct the standard deviation of the standard calibration points of the Y-axis and the Z-axis in the three-dimensional coordinate system. The error, the mode orientation difference information is a pixel for measuring the difference in orientation between the calibration mode and the moving direction of the Y-axis and the Z-axis in the three-dimensional coordinate system.

In step S806, the scan range is confirmed, and the scan path is planned. Specifically, the system will import a stereoscopic range of initial model volume files to plan the scan path. This initial model volume file limits the boundaries of the scan range by 100 cubic centimeters and performs various path planning for different directions, including The direction of rotation such as yaw, pitch, and roll, and the direction of movement on the X-axis.

In step S807, for different faces to start scanning, the scanning of this step will be divided into three faces, namely, occlusion, lingual and buccal side, and each face will be scanned every 30 degrees (four in total) Times). Therefore, a total of 12 scan files are generated. In step S808, the 12 oriented scan files are generated. For example, this file is an OBJ file, but the invention is not limited thereto.

In step S809, these OBJ scan files are meshed and an automatic fit trimming process is performed. In step S810, the above 12 OBJ scan files are integrated into one OBJ file, and then converted into STL files for subsequent browsing.

Please refer to FIG. 9. FIG. 9 is a flow chart showing a fine sweeping stage according to a third embodiment. In step S901, the position of the tooth is confirmed, and the dentist can select the position of the tooth that is pre-finished. In step S902, a section of the tooth is sampled and a two-dimensional vector is calculated. In detail, since the step S902 is for one section, the coordinate representation of each point on the tooth body is reduced from three dimensions to two dimensions. In step S903, the two-dimensional vector is compared with the deviation value, and if necessary, the correction is performed. In step S904, three-dimensional coordinates are calculated and cloud of points are generated. In step S905, the cloud point is pasted and the model is reformed. In step S906, the cloud point is converted into an STL file and previewed. The above steps S901 to S906 complete the scanning of a tooth, and if the scanning of the next tooth is continued, the process returns to step S901. Finally, the STL file can be exported and edited in step S907.

FIG. 10 is a flow chart showing a method of scanning in the mouth according to the third embodiment, and the intraoral scanning method is applicable to the above-described intraoral scanning device. At least a projection source is included to project a line of laser light onto an object. Referring to FIG. 10, in step S1010, an image of the corresponding object is acquired. In step S1020, a plurality of feature points corresponding to the line laser are acquired on the image, and the step further includes steps S1021 to 1023. In step S1021, a plurality of pixels corresponding to the line width of the line laser are acquired. In step S1022, virtual pixels between the pixels are calculated based on the filter. In step S1023, the virtual pixel having the largest grayscale value is obtained as one of the feature points from the virtual pixels. In step S1030, the total least squares feature point sampling method is performed based on the feature points to calculate the line segment. However, the steps in FIG. 10 have been described in detail above, and will not be described again here. It should be noted that the steps in FIG. 10 can be implemented as multiple codes or circuits, and the present invention is not limited thereto. In addition, the method of FIG. 10 can be used in conjunction with the above embodiments, or can be used alone. In other words, other steps can be added between the steps of FIG.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

100‧‧‧ intraoral scanning device

110‧‧‧ Subject

112‧‧‧ grips

114‧‧‧Extension

120‧‧‧Projection source

122‧‧‧ line laser

130‧‧‧Reflective Module

132‧‧‧ objects

140‧‧‧Rotary Module

150‧‧‧Image sensor

152‧‧‧Reflective Module

160‧‧‧ processor

Claims (4)

An intraoral scanning device includes: a projection source for projecting a line of laser; a first reflection module disposed on a projection path of the projection source for reflecting the line of the laser to an object; a rotating module coupled to the first reflective module for rotating the first reflective module to change an incident angle of the line laser relative to the first reflective module; an image sensor, wherein the object An image sensing path of the image sensor; and a processor, wherein the image sensor is configured to obtain a plurality of images corresponding to the object when the rotating module rotates the first reflective module, the processing The device is configured to establish a stereo tooth model corresponding to the object according to the images. The in-oral scanning device of claim 1, further comprising: a main body having an extension portion and a grip portion, wherein the first reflection module and the rotation module are disposed at one end of the extension portion, The projection source and the image sensor are disposed on the grip portion. The intraoral scanning device of claim 1, further comprising a second reflective module disposed between the first reflective module and the projection source, wherein the second reflective module senses the image On the path, and not On the projection path. An intraoral scanning method is suitable for an intraoral scanning device, the intraoral scanning device comprising a projection source for projecting a line of laser light to an object, the intraoral scanning method comprising: obtaining an image corresponding to the object; Obtaining a plurality of feature points corresponding to the line laser, the step comprising: obtaining a plurality of pixels corresponding to a line width of the line laser; calculating a plurality of virtual pixels between the pixels according to a filter; The virtual pixels having the largest grayscale value are obtained as one of the feature points; and a total least squares feature point sampling method is performed according to the feature points to calculate a line segment.